Target post-processing method and system based on intra-box point cloud, and product

Abstract

Provided in the embodiments of the present disclosure are a target post-processing method and system based on an intra-box point cloud, and a product. The method comprises: processing point cloud data to determine target boxes and target drivable area points therein; on the basis of the target drivable area points, determining convex hull point sets within the boxes that encompass all the target drivable area points; determining convex hull points at the maximum longitudinal position, convex hull points at the maximum lateral position and convex hull points at the minimum lateral position from the sets; if a target box is on the left side of an ego vehicle, constructing a right-side convex hull point set using convex hull points, that satisfy a first condition in terms of longitudinal distance to the convex hull point at the minimum lateral position, from among convex hull points sequentially located between the convex hull point at the maximum longitudinal position and the convex hull point at the minimum lateral position within the box; if a target box is on the right side of the ego vehicle, constructing a left-side convex hull point set using convex hull points, that satisfy a second condition in terms of longitudinal distance to the convex hull point at the maximum lateral position, from among convex hull points sequentially located between the convex hull point at the maximum longitudinal position and the convex hull point at the maximum lateral position within the box; if a box is on the left side of the ego vehicle and is turning left, correcting a right-side boundary line using right-side convex hull points; and if a box is on the right side of the ego vehicle and is turning right, correcting a left-side boundary line using left-side convex hull points.

Description

A target post-processing method, system, and product based on in-frame point cloud

[0001] Cross-references

[0002] This application claims priority to Chinese Patent Application No. 202510094749.5, filed on January 21, 2025, entitled "A Post-processing Method, System and Product for Targets Based on In-Frame Point Clouds", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to, but is not limited to, the field of point cloud processing technology, such as a target post-processing method, system, and product based on in-frame point clouds. Background Technology

[0004] Autonomous vehicles rely on onboard sensors to acquire information about their surroundings. The environmental perception module processes this sensor information and transmits it to subsequent modules for planning vehicle behavior. Therefore, the reliability and accuracy of sensor information are crucial to ensuring the safety of autonomous vehicles. The main types of sensors are cameras, millimeter-wave radar, and LiDAR.

[0005] Public content

[0006] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0007] In view of this, embodiments of this disclosure provide a target post-processing method, system, and product based on in-frame point clouds. The aim is to improve the accuracy of radar output results.

[0008] A first aspect of this disclosure provides a target post-processing method based on in-frame point clouds, the method comprising:

[0009] The point cloud data is used to identify targets using a target recognition model, resulting in multiple bounding boxes.

[0010] By analyzing and processing the point cloud data, the target drivable area points located within the target frame are determined;

[0011] Based on the target drivable area points in the target box, determine the convex hull of the target box and the set of convex hull points corresponding to the convex hull, wherein the convex hull points in the set of convex hull points are all the target drivable area points in the target box;

[0012] Determine the vertical maximum convex hull point, the horizontal maximum convex hull point, and the horizontal minimum convex hull point in the set of convex hull points of the target bounding box;

[0013] When the target box is located on the left side of the vehicle, the right convex hull point set of the target box is constructed by taking the convex hull points in the target box as being located in the order of the longitudinal maximum convex hull point and the transverse minimum convex hull point, and the right convex hull point whose distance from the transverse minimum convex hull point in the longitudinal direction satisfies the first condition.

[0014] When the target box is located on the right side of the vehicle, the left convex hull point set of the target box is constructed by taking the convex hull points in the target box as being located in the order of the longitudinal maximum convex hull point and the transverse maximum convex hull point, and the left convex hull point whose longitudinal distance from the transverse maximum convex hull point satisfies the second condition.

[0015] When the target frame is located on the left side of the vehicle and the vehicle is turning left, the right boundary line of the target frame is corrected using the right convex hull points from the set of right convex hull points of the target frame; and

[0016] When the target frame is located on the right side of the vehicle and the vehicle is turning right, the left boundary line of the target frame is corrected using the left convex hull points in the set of left convex hull points of the target frame.

[0017] Optionally, by analyzing and processing the point cloud data, the target drivable area points located within the target frame are determined, including:

[0018] By analyzing and processing the point cloud data, drivable areas within the point cloud data are determined; and

[0019] Based on the location of the target frame and the location of the drivable area point, determine the target drivable area point located within the target frame.

[0020] Optionally, before analyzing and processing the point cloud data to determine the target drivable area points located within the target bounding box, the method further includes:

[0021] Based on the attribute information of the target bounding boxes, determine the virtual target bounding boxes among the plurality of target bounding boxes; and

[0022] Filter the dummy target boxes among the multiple target boxes to obtain the remaining target boxes;

[0023] The step of analyzing and processing the point cloud data to determine the target drivable area points located within the target frame includes:

[0024] By analyzing and processing the point cloud data, the drivable area points of the target located in the remaining target boxes after filtering are determined.

[0025] Optionally, before determining the convex hull of the target bounding box and the set of convex hull points corresponding to the convex hull based on the target drivable area points in the target bounding box, the method further includes:

[0026] Based on the attribute information of the target bounding boxes, determine the target bounding boxes that need orientation correction from all the target bounding boxes;

[0027] The step of determining the convex hull of the target frame and the set of convex hull points corresponding to the convex hull based on the target drivable area points in the target frame includes: determining the convex hull of the target frame that needs to be oriented and the set of convex hull points corresponding to the convex hull based on the target drivable area points in the target frame that need to be oriented and the convex hull points corresponding to the convex hull

[0028] Optionally, when the target frame is located on the left side of the vehicle and the vehicle is turning left, the right boundary line of the target frame is corrected using the right convex hull points from the set of right convex hull points of the target frame, including:

[0029] When the target bounding box is located to the left of the vehicle and the vehicle is turning left, traverse the right convex hull points in the set of right convex hull points of the target bounding box, and connect the currently traversed right convex hull point with the horizontal minimum convex hull point to form the right predicted boundary line; and

[0030] If the angle deviation between the obtained right predicted boundary line and the right boundary line of the target box meets the set conditions, the traversal ends and the right predicted boundary line of the target box is corrected with the right predicted boundary line whose deviation meets the set conditions.

[0031] When the target frame is located on the right side of the vehicle and the vehicle is turning right, the left boundary line of the target frame is corrected using the left convex hull points from the set of left convex hull points of the target frame, including:

[0032] When the target bounding box is located on the right side of the vehicle and the vehicle is turning right, traverse the left convex hull points in the set of left convex hull points of the target bounding box, and connect the currently traversed left convex hull point with the largest lateral convex hull point to form the left predicted boundary line; and

[0033] If the angle deviation between the obtained left predicted boundary line and the left boundary line of the target box meets the set conditions, the traversal ends and the left boundary line of the target box is corrected using the left predicted boundary line whose deviation meets the set conditions.

[0034] Optionally, the method further includes:

[0035] The point cloud data is clustered using a clustering algorithm to obtain the target bounding boxes for each cluster.

[0036] Determine the proportion of point clouds for different semantic categories of the clustering target box, determine the size of the clustering target box, determine the reflectance information of the point cloud within the clustering target box, and determine the first point cloud density of the point cloud within a set range of the centroid of the clustering target box.

[0037] The clustering target box is divided into multiple segments vertically, and the cloud density of the second point around the center point of the first segment in the direction of movement of the clustering target box is calculated.

[0038] Based on the point cloud proportion, the size, the reflectivity information, the first point cloud density, and the second point cloud density, the virtual clustering target boxes in each clustering target box are determined; and

[0039] The dummy clustering boxes in each clustering target box are filtered to obtain the clustering result of the clustering algorithm.

[0040] Optionally, determine the reflectance information of the point cloud within the clustering target bounding box, including:

[0041] Based on the attribute information of the point cloud within the clustering target box, determine the number of low reflectivity point clouds within the clustering target box, as well as the maximum reflectivity and total reflectivity of the point cloud within the clustering target box.

[0042] Based on the total number of point clouds within the clustering target box and the number of low-reflectivity point clouds, determine the proportion of low-reflectivity point clouds within the clustering target box; and

[0043] The average reflectance of the point cloud within the clustering target box is determined based on the total number of point clouds and the total reflectance value within the clustering target box.

[0044] Optionally, based on the point cloud proportion, the size, the reflectivity information, the first point cloud density, and the second point cloud density, the virtual clustering target boxes in each clustering target box are determined, including:

[0045] When the proportion of low reflectivity point cloud corresponding to the clustered target box is a set value, the clustered target box is determined to be a virtual clustered target box;

[0046] If the proportion of low reflectivity point cloud corresponding to the clustered target box exceeds the first reflectivity threshold and the corresponding average reflectivity does not exceed the second reflectivity threshold, the clustered target box is determined to be a virtual clustered target box.

[0047] If the proportion of low reflectivity point cloud corresponding to the clustered target box exceeds the third reflectivity threshold, and the corresponding average reflectivity does not exceed the fourth reflectivity threshold and the corresponding maximum reflectivity does not exceed the fifth reflectivity threshold, the clustered target box is determined to be a virtual clustered target box.

[0048] When the proportion of point clouds of noisy semantic categories within the clustering target box exceeds the category threshold, the size relationship between the size and the first size threshold is determined, the density relationship between the first point cloud density and the first density threshold is determined, and the quantity relationship between the total number of point clouds and the total number threshold is determined.

[0049] Based on the size relationship, the density relationship, and the quantity relationship, determine whether the clustering target box is a dummy clustering target box; and

[0050] If the proportion of point clouds representing noise semantic categories within the clustering target box does not exceed the category threshold, the clustering target box is determined to be a virtual clustering target box based on the proportion of point clouds representing background semantic categories within the clustering target box, the size, and the total number of point clouds. Furthermore, the clustering target box is also determined to be a virtual clustering target box based on the proportion of point clouds representing general obstacle semantic categories within the clustering target box, the location of the clustering target box, and the size.

[0051] Optionally, the method further includes:

[0052] Determine whether there are small, single clustered bounding boxes within a preset range around the vehicle category clustering bounding box; and

[0053] If there are small single clustering target boxes within a preset range around the vehicle category clustering target box, the vehicle category clustering target box is determined to be a dummy clustering target box.

[0054] A second aspect of this disclosure provides a target post-processing system based on in-frame point clouds, the system comprising:

[0055] The target recognition unit is configured to perform target recognition on point cloud data using a target recognition model to obtain multiple target bounding boxes.

[0056] The analysis and processing unit is configured to determine the target drivable area points located in the target frame by analyzing and processing the point cloud data;

[0057] The convex hull point set determination unit is configured to determine the convex hull of the target frame and the convex hull point set corresponding to the convex hull based on the target drivable area points in the target frame, wherein the convex hull points in the convex hull point set are all the target drivable area points in the target frame;

[0058] The extreme convex hull point determination unit is configured to determine the vertical maximum convex hull point, the horizontal maximum convex hull point, and the horizontal minimum convex hull point in the set of convex hull points of the target box;

[0059] The right convex hull point set determination unit is configured to construct the right convex hull point set of the target frame when the target frame is located on the left side of the vehicle, based on the convex hull points in the target frame being located in the order of the longitudinal maximum convex hull point and the lateral minimum convex hull point, and the right convex hull points whose longitudinal distance from the lateral minimum convex hull point satisfies a first condition.

[0060] The left convex hull point set determination unit is configured to construct the left convex hull point set of the target frame when the target frame is located on the right side of the vehicle, based on the left convex hull points in the target frame being located in the order of the longitudinal maximum convex hull point and the transverse maximum convex hull point, and the left convex hull points whose longitudinal distance from the transverse maximum convex hull point satisfies the second condition.

[0061] The first correction unit is configured to correct the right boundary line of the target frame using the right convex hull points from the set of right convex hull points of the target frame when the target frame is located on the left side of the vehicle and the vehicle is turning left; and

[0062] The second correction unit is configured to correct the left boundary line of the target frame using the left convex hull points in the set of left convex hull points of the target frame when the target frame is located on the right side of the vehicle and the vehicle is turning right.

[0063] A third aspect of this disclosure provides an electronic device, including: a processor, a memory, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, implement the steps of a target post-processing method based on in-frame point clouds as described in the first aspect of this disclosure.

[0064] A fourth aspect of this disclosure provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the steps of a target post-processing method based on in-frame point clouds as described in the first aspect of this disclosure.

[0065] The target post-processing method based on in-frame point cloud provided in this disclosure has the following advantages:

[0066] This disclosure provides a target post-processing method based on in-frame point clouds. The method uses a target recognition model to identify targets in the point cloud data, obtaining multiple target bounding boxes. By analyzing the point cloud data, it determines the drivable area points of the targets within the target bounding boxes. Based on these drivable area points, it determines the convex hull of the target bounding box and a set of convex hull points corresponding to the convex hull, where the convex hull points in the set represent all drivable area points within the target bounding box. It also determines the maximum vertical convex hull point, the maximum horizontal convex hull point, and the minimum horizontal convex hull point in the set of convex hull points. When the target bounding box is located on the left side of the vehicle, the convex hull points are ranked according to their position relative to the maximum vertical convex hull point and the minimum horizontal convex hull point. The right convex hull point set of the target box is constructed by selecting the right convex hull point whose longitudinal distance from the minimum convex hull point satisfies the first condition. When the target box is located on the right side of the vehicle, the left convex hull point set is constructed by selecting the left convex hull point whose longitudinal distance from the longitudinal maximum convex hull point and the transverse maximum convex hull point satisfies the second condition. When the target box is located on the left side of the vehicle and is turning left, the right boundary line of the target box is corrected using the right convex hull points from the right convex hull point set. When the target box is located on the right side of the vehicle and is turning right, the left boundary line of the target box is corrected using the left convex hull points from the left convex hull point set. Therefore, the present invention determines the convex hull of the target box based on the drivable points within the target box, which can reduce the amount of computation and improve the efficiency of convex hull calculation, while supplementing the target box with more complete information. In addition, the boundary line correction of the target box is performed by using the convex hull points in the convex hull within the target box, which can reduce the information deviation of some target boxes in the model output or traditional clustering fitting, and provide more accurate target information for the backend.

[0067] After reading and understanding the accompanying diagrams and detailed descriptions, other aspects can be understood. Attached Figure Description

[0068] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments of this disclosure will be briefly introduced below. Obviously, the accompanying drawings described below are only embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0069] Figure 1 is a flowchart illustrating a target post-processing method based on point clouds within a frame, according to an embodiment of this disclosure.

[0070] Figure 2 is a schematic diagram of the target recognition result in a target post-processing method based on in-frame point cloud according to an embodiment of this disclosure;

[0071] Figure 3 is a schematic diagram of the convex hull of the target bounding box in a target post-processing method based on in-box point cloud according to an embodiment of the present disclosure;

[0072] Figure 4 is a schematic diagram of target bounding box optimization in a target post-processing method based on in-box point cloud according to an embodiment of this disclosure;

[0073] Figure 5 is another flowchart illustrating a target post-processing method based on in-frame point cloud according to an embodiment of this disclosure;

[0074] Figure 6 is a schematic diagram of a target post-processing system based on in-frame point cloud according to an embodiment of this disclosure. Detailed Implementation

[0075] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments, not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0076] In implementing the embodiments of this disclosure, the applicant discovered that because LiDAR has high detection accuracy, dust, water vapor, and small particulate matter in vehicle exhaust can also be detected. In these scenarios and in adverse weather conditions such as rain, snow, and fog, the performance of LiDAR may be reduced to some extent. At the same time, the output of the LiDAR deep learning model and the fitting of the traditional clustering target box will inevitably have some deviation. For these two situations, there is a need for countermeasures to minimize the decline in perception performance caused by adverse weather or environmental factors and to improve the accuracy of LiDAR output information.

[0077] Referring to Figure 1, Figure 1 is a schematic diagram of a target post-processing method based on in-frame point clouds according to an embodiment of this disclosure. As shown in Figure 1, the method includes:

[0078] Step S1: Use the target recognition model to identify targets in the point cloud data and obtain multiple target boxes.

[0079] In this embodiment, point cloud data of the current driving scene is collected by the vehicle-mounted radar. The collected point cloud data of the current frame is then input into a target recognition model for target identification, obtaining a bounding box for each target in the current driving scene. The bounding box corresponds to the target, which refers to various objects in the current driving scene, including cars, bicycles, motorcycles, pedestrians, road obstacles, etc. The radar used to collect point cloud data includes, but is not limited to, lidar, millimeter-wave radar, continuous-wave radar, and pulse radar. If there are two vehicles in front of the radar, the collected point cloud data is input into the target recognition model for target identification, obtaining the bounding boxes for each of the two vehicles. This is a simplified explanation for understanding the meaning of the bounding box; however, in practice, the bounding boxes obtained through the target recognition model may contain misidentifications, such as a non-existent target being identified as a target. The target recognition model is a radar model used to identify targets in point cloud data and obtain target boxes corresponding to the targets, as shown in Figure 2. Figure 2 shows the target box recognition results obtained by the target recognition model through target recognition of point cloud data (the figure shows the thick frame part and the thin frame part), where the thick frame part (e.g., thick frame hexahedron) is the target box recognition result obtained by the target recognition model.

[0080] Step S2: By analyzing and processing the point cloud data, determine the target drivable area points located in the target box.

[0081] In this embodiment, the target box recognition results output by the target recognition model all have a certain degree of deviation. Therefore, in order to avoid the impact of the target box corresponding to the vehicle target with the deviation on the vehicle's driving when the surrounding vehicles (i.e., vehicle targets) turn, this embodiment of the disclosure will optimize the target boxes in the target box recognition results output by the target recognition model.

[0082] Optionally, the point cloud data of the current frame is analyzed and processed to determine the freespace points, i.e., drivable area points, within a certain height range, which, without considering traffic rules, provide unobstructed, flat road surface areas for vehicles to drive freely. There are various and relatively mature methods for determining freespace points in point cloud data, which will not be detailed here. Then, based on the positional relationship between the determined drivable area points and each target bounding box, each drivable area point within the target bounding box is determined, and these drivable area points within the target bounding box are identified as target drivable area points.

[0083] Step S3: Based on the target drivable area points in the target box, determine the convex hull of the target box and the set of convex hull points corresponding to the convex hull. The convex hull points in the set of convex hull points are all the target drivable area points in the target box.

[0084] In this embodiment, after determining the target drivable area points located within the target frame, for each target frame, using the target drivable area points within that target frame as boundaries, the convex hull of the target frame is calculated using a corresponding algorithm. This convex hull will encompass all target drivable area points within the target frame. The set of all target drivable area points within this convex hull is defined as the convex hull point set of the convex hull corresponding to the target frame. A single convex hull point in this set corresponds to a single target drivable area point within the convex hull. The preferred algorithm for calculating the convex hull of the target frame is the Graham scan algorithm, which is a preferred algorithm. However, other algorithms can also be used to determine the convex hull of the target frame, and no specific limitation is made here.

[0085] Step S4: Determine the vertical maximum convex hull point, the horizontal maximum convex hull point, and the horizontal minimum convex hull point in the set of convex hull points of the target bounding box.

[0086] In this embodiment, for ease of description and understanding, the coordinate system of each target box is defined with the vertical direction of travel along its own target box as the x-axis and the horizontal direction as the y-axis. After obtaining the set of convex hull points corresponding to each target box in step S3, for each target box, the convex hull point with the largest vertical coordinate value (i.e., the vertical maximum convex hull point) and the convex hull point with the largest horizontal coordinate value (i.e., the horizontal maximum convex hull point) and the convex hull point with the smallest horizontal coordinate value (i.e., the horizontal minimum convex hull point) can be determined. Thus, for each target box, a vertical maximum convex hull point, a horizontal maximum convex hull point, and a horizontal minimum convex hull point can be determined. As shown in Figure 3, Figure 3 schematically illustrates the convex hull corresponding to a target bounding box, the set of convex hull points corresponding to the convex hull, and the maximum vertical convex hull point, the maximum horizontal convex hull point, and the minimum horizontal convex hull point in the determined set of convex hull points. The rectangle in the figure is the target bounding box, and the area enclosed by the dashed line is the convex hull corresponding to the target bounding box. The dots in the convex hull are the convex hull points. All the convex hull points in the convex hull constitute the set of convex hull points corresponding to the target bounding box. In the figure, X_max is the maximum vertical convex hull point in the set of convex hull points, Y_max is the maximum horizontal convex hull point in the set of convex hull points, and Y_min is the minimum horizontal convex hull point in the set of convex hull points.

[0087] Step S5: When the target box is located on the left side of the vehicle, construct the set of right convex hull points of the target box, which are located in the order of the longitudinal maximum convex hull point and the transverse minimum convex hull point, and whose longitudinal distance from the transverse minimum convex hull point satisfies the first condition.

[0088] In this embodiment, to avoid the impact of misidentified target boxes on the vehicle's driving when surrounding vehicles (i.e., vehicle targets) turn, and considering that the target box is located on the left side of the vehicle, the main factor affecting the vehicle's driving is its right boundary, this embodiment filters convex hull points whose scanning order is between the vertical maximum convex hull point and the horizontal minimum convex hull point of the target box, and whose vertical distance from the horizontal minimum convex hull point of the target box satisfies a first condition. These filtered convex hull points are then identified as the right convex hull points of the target box and stored in the right convex hull point set corresponding to that target box. The method for determining the right convex hull point set for each target box located on the left side of the vehicle is the same and will not be repeated here. The right convex hull point set of the target box is used to optimize the target box when optimization is required. The first condition is preferably that the distance between the convex hull point and the minimum convex hull point of the target box in the longitudinal direction is greater than half of the longitudinal dimension of the target box (that is, half the length of the target box in the longitudinal direction).

[0089] In this embodiment, the LiDAR records and stores point cloud data in the scanning order. In this implementation, when the target bounding box to be optimized is located on the left side of the vehicle, the boundary between the vertical maximum convex hull point and the horizontal minimum convex hull point refers to the convex hull points in the set of convex hull points of the target bounding box that are recorded and stored after the vertical maximum convex hull point and before the horizontal minimum convex hull point. That is, when subsequently determining convex hull points that satisfy the first condition, the focus is on the convex hull points in the set of convex hull points of the target bounding box that are recorded and stored after the vertical maximum convex hull point and before the horizontal minimum convex hull point.

[0090] Step S6: When the target box is located on the right side of the vehicle, construct the set of left convex hull points of the target box, which are located in the order of the longitudinal maximum convex hull point and the transverse maximum convex hull point, and whose longitudinal distance from the transverse maximum convex hull point satisfies the second condition.

[0091] In this embodiment, when the target box is located on the right side of the vehicle, the main area affected by the vehicle's movement is its left boundary. Therefore, this embodiment selects convex hull points whose scanning order is between the maximum longitudinal and lateral convex hull points of the target box, and whose longitudinal distance from the maximum lateral convex hull point of the target box satisfies the second condition. These selected convex hull points are then identified as the left convex hull points of the target box and stored in the corresponding left convex hull point set. The method for determining the left convex hull point set for each target box located on the right side of the vehicle is the same and will not be repeated here. The left convex hull point set of the target box is used to optimize the target box when optimization is required. Preferably, the second condition is that the longitudinal distance between the convex hull point and the maximum lateral convex hull point of the target box is greater than half the longitudinal dimension of the target box (i.e., half the longitudinal length of the target box).

[0092] In this embodiment, the LiDAR records and stores point cloud data in the scanning order. In this implementation, when the target bounding box to be optimized is located on the right side of the vehicle, the boundary defined by the vertical and horizontal maximum convex hull points refers to the convex hull points in the set of convex hull points of the target bounding box that are recorded and stored after and before the vertical and horizontal maximum convex hull points. That is, when subsequently determining convex hull points that satisfy the second condition, the determination is made for the convex hull points in the set of convex hull points of the target bounding box that are recorded and stored after and before the vertical and horizontal maximum convex hull points.

[0093] Step S7: When the target frame is located on the left side of the vehicle and the vehicle is turning left, the right boundary line of the target frame is corrected using the right convex hull points in the set of right convex hull points of the target frame.

[0094] In this embodiment, the positional relationship between the target frame and the vehicle is determined. If the target frame is located to the left of the vehicle, it is determined whether the target frame is turning left. If the target frame to the left of the vehicle is turning left, and its original right boundary line encroaches on the lane where the vehicle is located, the right boundary line of the target frame is corrected towards the inside of the target frame using the right convex hull points in the set of right convex hull points, resulting in a more accurate target frame. The corrected target frame may no longer encroach on the lane where the vehicle is located, thus ensuring that the originally misidentified target frame will no longer affect the vehicle's driving, allowing the vehicle to better execute control actions. In real-world scenarios, a vehicle target may not actually intrude into the lane where the vehicle is located (i.e., it will not affect the normal driving of the vehicle). However, due to a deviation in the target bounding box obtained by the target recognition model, the obtained target bounding box intrudes into the lane where the vehicle is located. As a result, a vehicle target that would not normally affect the normal driving of the vehicle is now considered to intrude into the lane and thus affect the normal driving of the vehicle. Consequently, the subsequent control actions of the vehicle will also be deviated (e.g., the vehicle could normally drive along its lane to overtake the vehicle in front, but due to the recognition result, the vehicle cannot drive normally to overtake). The method of this embodiment is used to correct the target bounding box in the recognition result, so that the target bounding box is closer to the real target, thereby enabling the vehicle to better perform control actions (e.g., normally drive along its lane to overtake the vehicle target in front).

[0095] In this embodiment, one method for determining whether a target frame located on the left side of the vehicle is turning left involves determining the angle between the orientation of the target frame on the left side of the vehicle and the vehicle's orientation. If this angle is greater than zero, the target frame on the left side of the vehicle is determined to be turning left. The vehicle's coordinate system also uses the longitudinal direction of travel of the vehicle as the x-axis (as shown in Figure 3). Therefore, if the angle between the orientation of the target frame on the left side of the vehicle and the vehicle's orientation is greater than zero, the target frame on the left side of the vehicle is determined to be turning left.

[0096] Step S8: When the target frame is located on the right side of the vehicle and the vehicle is turning right, the left boundary line of the target frame is corrected using the left convex hull points in the set of left convex hull points of the target frame.

[0097] In this embodiment, the positional relationship between the target frame and the vehicle is determined. If the target frame is located to the right of the vehicle, it is determined whether the target frame is turning right. If the target frame to the right of the vehicle is turning right, and its original left boundary line encroaches on the lane where the vehicle is located, the left boundary line of the target frame is corrected towards the inside of the target frame using the left convex hull points in the set of left convex hull points, to obtain a more accurate target frame. As shown in Figure 4, two close solid-line quadrilateral frames (including a thick solid-line quadrilateral frame and a thin solid-line quadrilateral frame) are shown. The thin solid-line quadrilateral frame corresponds to one target frame, and the thick solid-line quadrilateral frame is the target frame obtained after correction.

[0098] This disclosure provides a target post-processing method based on in-frame point clouds. The method uses a target recognition model to identify targets in the point cloud data, obtaining multiple target bounding boxes. By analyzing the point cloud data, it determines the drivable area points of the targets within the target bounding boxes. Based on these drivable area points, it determines the convex hull of the target bounding box and a set of convex hull points corresponding to the convex hull, where the convex hull points in the set represent all drivable area points within the target bounding box. It also determines the maximum vertical convex hull point, the maximum horizontal convex hull point, and the minimum horizontal convex hull point in the set of convex hull points. When the target bounding box is located on the left side of the vehicle, the convex hull points are ranked according to their position relative to the maximum vertical convex hull point and the minimum horizontal convex hull point. The right convex hull point set of the target box is constructed by selecting the right convex hull point whose longitudinal distance from the minimum convex hull point satisfies the first condition. When the target box is located on the right side of the vehicle, the left convex hull point set is constructed by selecting the left convex hull point whose longitudinal distance from the longitudinal maximum convex hull point and the transverse maximum convex hull point satisfies the second condition. When the target box is located on the left side of the vehicle and is turning left, the right boundary line of the target box is corrected using the right convex hull points from the right convex hull point set. When the target box is located on the right side of the vehicle and is turning right, the left boundary line of the target box is corrected using the left convex hull points from the left convex hull point set. Therefore, the present invention determines the convex hull of the target box based on the drivable points within the target box, which can reduce the amount of computation and improve the efficiency of convex hull calculation, while supplementing the target box with more complete information. In addition, the boundary line correction of the target box is performed by using the convex hull points in the convex hull within the target box, which can reduce the information deviation of some target boxes in the model output or traditional clustering fitting, and provide more accurate target information for the backend.

[0099] In conjunction with the above embodiments, in one implementation, this disclosure also provides a target post-processing method based on in-frame point clouds. In this target post-processing method based on in-frame point clouds, step S2 may include: analyzing and processing the point cloud data to determine drivable area points in the point cloud data; and determining target drivable area points located within the target frame based on the location of the target frame and the locations of the drivable area points.

[0100] In this embodiment, the point cloud data of the current frame is analyzed and processed to determine the drivable area points in the point cloud data. Then, based on the center point and size information of each target box, coordinate transformation is used to calculate the coordinates of the four corner points of each target box in the xy plane. Then, based on the coordinates of its four corner points, the maximum and minimum ranges in the horizontal and vertical directions of each target box in the xy plane are obtained. Then, based on the coordinates of the drivable area points, it is determined whether they fall within any target box. If the coordinates of the drivable area point fall within any target box, then the drivable area point is determined as a target drivable area point located within a target box and stored for use in subsequent steps.

[0101] In conjunction with the above embodiments, in one implementation, this disclosure also provides a target post-processing method based on in-frame point clouds. In this target post-processing method based on in-frame point clouds, before step S2, the method further includes steps S01 to S02:

[0102] Step S01: Determine the virtual target box among the multiple target boxes based on the attribute information of the target box.

[0103] In this embodiment, as mentioned in step S1 above, there may be misidentifications in the target boxes obtained through the target recognition model, such as a non-existent target being identified as a target. Therefore, to avoid the impact of misidentified target boxes on the vehicle's driving and to reduce the resource consumption associated with optimizing misidentified target boxes, this embodiment filters the target boxes obtained by the target recognition model from the point cloud data, identifies the false target boxes that are misidentified, and then filters them out. Subsequent optimization only targets the target boxes remaining after filtering out the false target boxes. This embodiment mainly determines whether each target box is a false target based on the point cloud reflectivity information, the confidence level of the target box, the positional relationship between target boxes, and the point cloud distribution within the target box.

[0104] Optionally, in scenarios with dense dust or dense water vapor near water trucks, if the proportion of low-reflectivity point clouds within the target bounding box reaches a certain threshold, and the average reflectivity information is below a certain set threshold and the maximum reflectivity information is below a certain set threshold, then the target bounding box is determined to be a false target. To address situations where incomplete point clouds lead to low accuracy in target recognition models, low-confidence target bounding boxes are identified as false target bounding boxes based on the target bounding box's position, orientation, and point cloud distribution within it.

[0105] Step S02: Filter the dummy target boxes among the multiple target boxes to obtain the remaining target boxes.

[0106] In this embodiment, the dummy target boxes in all target boxes determined by step S01 are filtered to obtain the remaining target boxes.

[0107] In this embodiment of the disclosure, if the method further includes steps S01 to S02, step S2 may include: determining the target drivable area points located in the remaining target boxes after filtering by analyzing and processing the point cloud data.

[0108] In this embodiment, before step S2, steps S01 to S02 are included to filter the virtual target boxes in all the obtained target boxes. The implementation of step S2 is as follows: by analyzing and processing the point cloud data, the drivable area points in the point cloud data are determined, and then it is determined which drivable area points are located in the remaining target boxes after filtering. Then, the drivable area points located in the remaining target boxes after filtering are determined as target drivable area points. The subsequent steps of determining the convex hull point set, determining the right convex hull point set, determining the left convex hull point set, and correcting and optimizing the boundary line are all based on the target drivable area points located in the remaining target boxes after filtering and the remaining target boxes after filtering.

[0109] In conjunction with the above embodiments, in one implementation, this disclosure also provides a target post-processing method based on in-box point clouds. In this target post-processing method based on in-box point clouds, before step S3, the method further includes: determining the target boxes requiring orientation correction from all target boxes based on the attribute information of the target boxes.

[0110] In this embodiment, the target recognition model outputs target bounding box recognition results with a certain degree of deviation. This deviation is only considered significant when a vehicle in the adjacent lane intrudes into the lane where the vehicle is located; such intrusion typically occurs only with large vehicles. Therefore, to improve processing efficiency and reduce computational resource consumption, this embodiment can determine whether correction is needed for relatively large target bounding boxes and / or those in a turning state, and then implement subsequent corrections.

[0111] Optionally, before step S3 in this embodiment, if the scheme does not perform virtual target box filtering, the target box requiring orientation correction is determined from all target boxes based on the category, size, and / or orientation in the target box's attribute information. If the scheme performs virtual target box filtering, the target box requiring orientation correction is determined from all remaining target boxes after filtering based on the category, size, and / or orientation in the target box's attribute information. One possible implementation for determining the target box requiring orientation correction is: determining whether the target box is of the BIG_VEHICLE or HUGE_VEHICLE category, whether the length of the target box exceeds a certain threshold (e.g., 6 meters), whether the width of the target box exceeds a certain threshold (e.g., 2 meters), and whether the orientation of the target box is within a range of ±25°. If any one of these conditions is met, the target box is determined to be a target box requiring orientation correction.

[0112] In this embodiment of the present disclosure, before step S3 of the target post-processing method based on in-frame point cloud provided in this embodiment of the present disclosure, if the target frame that needs to be oriented corrected is determined from all the target frames according to the attribute information of the target frame, step S3 may include: determining the convex hull of the target frame that needs to be oriented corrected and the set of convex hull points corresponding to the convex hull, based on the target drivable area points in the target frame that needs to be oriented corrected.

[0113] In this embodiment, when the target box requiring orientation correction is determined, subsequent correction operations will all target boxes requiring orientation correction. Therefore, in this case, this embodiment will only distinguish the convex hull and the set of convex hull points for the determined target box requiring orientation correction. The determination method is the same as the implementation method of step S3 above, except that at this time, the convex hull of the target box requiring orientation correction and the set of convex hull points corresponding to the convex hull are determined based on the target drivable area point of the target box requiring orientation correction.

[0114] In conjunction with the above embodiments, in one implementation, this disclosure also provides a target post-processing method based on in-frame point clouds. In this target post-processing method based on in-frame point clouds, step S7 may include:

[0115] Step S71: When the target box is located on the left side of the vehicle and the vehicle is turning left, traverse the right convex hull points in the set of right convex hull points of the target box, and connect the currently traversed right convex hull point with the horizontal minimum convex hull point to form the right prediction boundary line.

[0116] In this embodiment, when the target box is located on the left side of the vehicle and the target box is turning left, if there is an error in the model recognition causing its right boundary line to intrude into the lane where the vehicle is located, it will affect the driving of the vehicle (if it does not actually intrude, but the recognition result is due to an error in the recognition). In this case, this embodiment will traverse the right convex hull points in the set of right convex hull points of the target box, and take out one convex hull point in the set of right convex hull points in turn. Then, connect the convex hull point with the horizontal minimum convex hull point of the target box to form a new right predicted boundary line.

[0117] Step S72: If the angle deviation between the obtained right predicted boundary line and the right boundary line of the target box meets the set conditions, end the traversal and correct the right boundary line of the target box with the right predicted boundary line whose deviation meets the set conditions.

[0118] In this embodiment, after forming a new right predicted boundary line of the target box in step S71, the new right predicted boundary line is compared with the right boundary line of the target box (i.e., the right boundary point of the target box obtained by the model recognition) to determine whether the angle deviation between the two meets the set conditions. If the set conditions are met, the new right predicted boundary line replaces the right boundary line of the target box, thereby achieving correction, and the traversal ends. If the set conditions are not met, a new convex hull point is taken from the set of right convex hull points of the target box, and the new convex hull point is connected to the horizontal minimum convex hull point of the target box to form a new right predicted boundary line. The angle deviation between the new right predicted boundary line and the right boundary line of the target box is then determined to meet the set conditions. If it still does not meet the conditions, the process continues until the angle deviation between the new convex hull point taken from the set of right convex hull points of the target box and the right boundary line of the target box meets the set conditions. Preferably, the preset condition is that the angle deviation does not exceed 10°, which is to prevent over-correction of the target box and more serious errors.

[0119] In conjunction with the above embodiments, in one implementation, this disclosure also provides a target post-processing method based on in-frame point clouds. In this target post-processing method based on in-frame point clouds, step S8 may include:

[0120] Step S81: When the target box is located on the right side of the vehicle and the vehicle is turning right, traverse the left convex hull points in the set of left convex hull points of the target box, and connect the currently traversed left convex hull point with the largest lateral convex hull point to form the left prediction boundary line.

[0121] In this embodiment, when the target box is located on the right side of the vehicle and the target box is turning right, if there is an error in the model recognition causing its left boundary line to intrude into the lane where the vehicle is located, it will affect the driving of the vehicle (if it does not actually intrude, but the recognition result is due to an error in the recognition). In this case, this embodiment of the present disclosure will traverse the left convex hull points in the set of left convex hull points of the target box, and take out one convex hull point in the set of left convex hull points in turn. Then, connect the convex hull point with the horizontal maximum convex hull point of the target box to form a new left predicted boundary line.

[0122] Step S82: If the angle deviation between the obtained left predicted boundary line and the left boundary line of the target box meets the set condition, end the traversal and correct the left boundary line of the target box with the left predicted boundary line whose deviation meets the set condition.

[0123] In this embodiment, after forming a new left predicted boundary line of the target box in step S81, the new predicted boundary line is compared with the left boundary line of the target box (i.e., the left boundary point of the target box obtained by the model recognition) to determine whether the angle deviation between the two meets the set conditions. If the set conditions are met, the new predicted boundary line replaces the left boundary line of the target box, thereby achieving correction, and the traversal ends. If the set conditions are not met, a new convex hull point is taken from the set of left convex hull points of the target box, and the new convex hull point is connected to the horizontal maximum convex hull point of the target box to form a new left predicted boundary line. The angle deviation between the new predicted boundary line and the left boundary line of the target box is then determined to meet the set conditions. If the conditions are still not met, the process continues until the angle deviation between the new convex hull point taken from the set of left convex hull points of the target box and the left boundary line of the target box meets the set conditions. Preferably, the preset condition is that the angle deviation does not exceed 10°, which is to prevent over-correction of the target box and more serious errors.

[0124] In conjunction with the above embodiments, in one implementation, this disclosure also provides a target post-processing method based on in-frame point clouds. This target post-processing method based on in-frame point clouds further includes:

[0125] Step S101: Cluster the point cloud data using a clustering algorithm to obtain the target boxes for each cluster.

[0126] In this embodiment, in the implementation of processing point cloud data to obtain target boxes, the LiDAR model can only identify the target categories involved in the training model. Therefore, in actual target box recognition, traditional clustering algorithms are usually used to identify target boxes. However, the recognition method of traditional clustering algorithms may also involve the identification of some virtual targets. Based on this, the target post-processing algorithm based on in-box point cloud provided in this embodiment is also used to filter virtual clustered target boxes obtained by traditional clustering algorithms based on the attribute information of the in-box point cloud.

[0127] Optionally, point cloud data can be clustered using traditional clustering algorithms to obtain cluster target boxes. Figure 2 exemplarily shows the cluster target box recognition results obtained by clustering point cloud data using traditional clustering algorithms (the figure shows the thick and thin line boxes), where the thin line box (e.g., the thin line box hexahedron) is the cluster target box recognition result obtained by traditional clustering algorithms.

[0128] Step S102: Determine the proportion of point clouds for different semantic categories of the clustering target box, determine the size of the clustering target box, determine the reflectance information of the point cloud within the clustering target box, and determine the first point cloud density of the point cloud located within a set range of the centroid of the clustering target box.

[0129] In this embodiment, after clustering the point cloud data using a traditional clustering algorithm to obtain various clustering target boxes, for each clustering target box, the proportion of point clouds of different semantic categories within each box is determined, as are the dimensions (length, width, and height) of each box, the reflectance information of the point clouds within each box, and the point cloud density within a set range of the centroid of the clustering target box. This point cloud density is the first point cloud density. The set range can be dynamically adjusted according to the size of the clustering target box, thereby improving the filtering accuracy of the virtual clustering target boxes. For example, if the clustering target box is relatively large, the corresponding set range is set to a larger value; if the clustering target box is relatively small, the corresponding set range is set to a smaller value.

[0130] The semantic categories include at least: invalid points (including noise and ground categories), background, vehicles, cyclists, and general obstacles. The point cloud reflectance information includes at least: the number of low-reflectance point clouds within the clustering bounding boxes, the maximum reflectance and total reflectance of point clouds within the clustering bounding boxes, the proportion of low-reflectance point clouds within the clustering bounding boxes, and the average reflectance of point clouds within the clustering bounding boxes.

[0131] Step S103: Divide the clustering target box into multiple segments in the vertical direction, and calculate the cloud density of the second point around the center point of the first segment in the direction of movement of the clustering target box.

[0132] In this embodiment, for each cluster target box obtained by clustering point cloud data through traditional clustering algorithms, each cluster target box is divided into multiple segments (preferably 4 segments) in its own vertical direction, and the point cloud density around the center point of the first segment (such as the center point of the first 1 / 4 segment) facing the movement direction of the cluster target box itself is calculated. This point cloud density is called the second density.

[0133] Step S104: Based on the point cloud proportion, the size, the reflectivity information, the first point cloud density, and the second point cloud density, determine the virtual clustering target boxes in each clustering target box.

[0134] In this embodiment, based on the point cloud proportion, size, reflectivity information, first point cloud density and second point cloud density of each cluster target box determined in steps S101 to S13, it is determined which of these cluster target boxes are virtual cluster target boxes.

[0135] Step S105: Filter the dummy clustering target boxes in each clustering target box to obtain the clustering result of the clustering algorithm.

[0136] In this embodiment, the clustering target boxes determined in step S104 are filtered, and the remaining clustering target boxes are the final clustering results obtained by the clustering algorithm.

[0137] In conjunction with the above embodiments, in one implementation, this disclosure also provides a target post-processing method based on in-frame point clouds. In this target post-processing method based on in-frame point clouds, determining the reflectance information of point clouds within a clustered target frame includes: determining the number of low-reflectance point clouds within the clustered target frame based on attribute information, and determining the maximum reflectance and total reflectance value of the point clouds within the clustered target frame; determining the proportion of low-reflectance point clouds within the clustered target frame based on the total number of point clouds and the number of low-reflectance point clouds; and determining the average reflectance of the point clouds within the clustered target frame based on the total number of point clouds and the total reflectance value.

[0138] In this embodiment, the collected point cloud data records the reflectance of the point clouds. Based on the reflectance of the point cloud data, point clouds with reflectance lower than a set value are identified as low-reflectance point clouds. Then, the number of low-reflectance point clouds within each cluster target box is determined, where the set value is preferably 10. Simultaneously, based on the reflectance of the point cloud data, the total reflectance value and the maximum reflectance of the point clouds within each cluster target box are determined. The number of low-reflectance point clouds within a single cluster target box is divided by the total number of point clouds within that single cluster target box to obtain the proportion of low-reflectance point clouds within that single cluster target box. Based on the same implementation method, each cluster target box will obtain its own corresponding proportion of low-reflectance point clouds. The total reflectance value within a single cluster target box is divided by the total number of point clouds within that single cluster target box to obtain the average reflectance of the point clouds within that single cluster target box.

[0139] In conjunction with the above embodiments, in one implementation, this disclosure also provides a target post-processing method based on in-frame point clouds. In this target post-processing method based on in-frame point clouds, step S104 may include: determining the clustered target frame as a dummy clustered target frame when the proportion of low-reflectivity point clouds corresponding to the clustered target frame is a set value; determining the clustered target frame as a dummy clustered target frame when the proportion of low-reflectivity point clouds corresponding to the clustered target frame exceeds a first reflectivity threshold and the corresponding average reflectivity does not exceed a second reflectivity threshold; determining the clustered target frame as a dummy clustered target frame when the proportion of low-reflectivity point clouds corresponding to the clustered target frame exceeds a third reflectivity threshold and the corresponding average reflectivity does not exceed a fourth reflectivity threshold and the corresponding maximum reflectivity does not exceed a fifth reflectivity threshold; and determining the dummy clustered target frame as a dummy clustered target frame when the proportion of point clouds representing noise semantic categories within the clustered target frame exceeds a category threshold. The system determines the size relationship between the stated size and a first size threshold, the density relationship between the first point cloud density and a first density threshold, and the quantity relationship between the total number of point clouds and a total number threshold. Based on the size relationship, the density relationship, and the quantity relationship, it determines whether the clustering target box is a dummy clustering target box. If the proportion of point clouds of noise semantic categories within the clustering target box does not exceed the category threshold, it determines whether the clustering target box is a dummy clustering target box based on the proportion of point clouds of background semantic categories within the clustering target box, the stated size, and the total number of point clouds. It also determines whether the clustering target box is a dummy clustering target box based on the proportion of point clouds of general obstacle semantic categories within the clustering target box, the location of the clustering target box, and the stated size.

[0140] In this embodiment, if the proportion of low-reflectivity point cloud corresponding to a clustering target box obtained by the traditional clustering algorithm is a set value, the clustering target box is determined to be a dummy clustering target box and will be filtered out. The set value is preferably 1. If the proportion of low-reflectivity point cloud corresponding to a clustering target box obtained by the traditional clustering algorithm exceeds a first reflectivity threshold (preferably 0.9) and the corresponding average reflectivity does not exceed a second reflectivity threshold (preferably 3), the clustering target box is determined to be a dummy clustering target box and will be filtered out. If the proportion of low-reflectivity point cloud corresponding to a clustering target box obtained by the traditional clustering algorithm exceeds a third reflectivity threshold (preferably 1), the corresponding average reflectivity does not exceed a fourth reflectivity threshold (preferably 0.5), and the corresponding maximum reflectivity does not exceed a fifth reflectivity threshold (preferably 3), the clustering target box is determined to be a dummy clustering target box and will be filtered out. The judgment conditions in this part are mainly aimed at the situation where noise such as water vapor and dust is mistakenly identified as target boxes. The reflectivity of noise such as water vapor and dust is generally low. For virtual targets such as water vapor near sprinkler trucks, virtual targets of dust at the rear of vehicles, virtual targets of water vapor in rainy weather, and virtual targets of vehicle exhaust, this embodiment of the disclosure filters virtual targets of noise such as dust and water vapor based on the reflectivity information of the point cloud within the box.

[0141] In this embodiment, since semantic segmentation may misdetect general obstacles as noise or vice versa, this embodiment of the present disclosure, in response to such situations, combines information such as the position and size of the target box, the number of point clouds within the box, the point cloud density around the center point and centroid of the target box, and the proportion of point clouds of each semantic category within the box to determine the virtual clustered target box. Optionally, if the proportion of point clouds of noise semantic categories within a clustered target box obtained by a traditional clustering algorithm exceeds a category threshold (preferably 0.6), the size relationship between the size of the clustered target box and a first size threshold, the density relationship between the first point cloud density of the clustered target box and a first density threshold, and the quantity relationship between the total number of point clouds of the clustered target box and a total number threshold are determined. Then, based on the size relationship, density relationship, and quantity relationship corresponding to the clustered target box, it is determined whether the clustered target box is a virtual clustered target box. The first size threshold includes a length threshold and a width threshold for the target box. These thresholds can be set according to the actual application scenario and are not specifically limited here. The first density threshold can also be set according to the actual application scenario and is not specifically limited here. The total number threshold can also be set according to the actual application scenario and is not specifically limited here. If the size of the cluster target box is not within the first size threshold range, and / or the density is greater than the first density threshold, and / or the total number of points in the cluster target box is greater than the total number threshold, then the cluster target box is determined to be a dummy cluster target box and will be filtered out. In other words, if at least one of these three relationships is satisfied, the cluster target box will be determined to be a dummy cluster target box. If the proportion of point cloud representing noise semantic category within a clustering target box obtained by traditional clustering algorithm does not exceed the category threshold (preferably 0.6), then the clustering target box is determined to be a dummy clustering target box based on the proportion of point cloud representing background semantic category (or noise and background semantic category) within the clustering target box, the size of the clustering target box, and the total number of point clouds within the clustering target box. Furthermore, the clustering target box is also determined to be a dummy clustering target box based on the proportion of point cloud representing general obstacle semantic category within the clustering target box, the location of the clustering target box, and the size of the clustering target box.Optionally, if the proportion of background semantic category (or background and noise semantic category) of the point cloud within the clustering target box exceeds a certain threshold (preferably 0.9), and the size of the clustering target box exceeds a certain set threshold (i.e., the size of the clustering target box is large), and the number of point clouds within the target clustering box is lower than a certain threshold (or the height of the clustering target box above the ground exceeds a certain threshold), then the clustering target box is determined to be a virtual clustering target box and will be filtered out. If the proportion of general obstacle semantic category of the point cloud within the clustering target box exceeds a certain threshold (preferably 0.9), and the clustering target box is not located in this lane and the size of the clustering target box exceeds a certain threshold (i.e., the size is large), then the clustering target box is determined to be a virtual clustering target box and will be filtered out.

[0142] In conjunction with the above embodiments, in one implementation, this disclosure also provides a target post-processing method based on in-frame point clouds. In this target post-processing method based on in-frame point clouds, the method further includes: determining whether there are small-sized single-cluster target boxes within a preset range around the vehicle category cluster target box; and if small-sized single-cluster target boxes exist within the preset range around the vehicle category cluster target box, determining that the vehicle category cluster target box is a dummy cluster target box.

[0143] In this embodiment, for each vehicle category clustering target box obtained by the traditional clustering algorithm, it is determined whether there are small-sized single clustering target boxes within a preset range around the single vehicle category clustering target box. If small-sized single clustering target boxes exist within the preset range of the single vehicle category clustering target box, then the single vehicle category clustering target box is determined to be a dummy clustering target box and will be filtered out. The preset range can be set according to the actual application scenario and is not specifically limited here. Similarly, the small size is set according to the actual application scenario and is not specifically limited here.

[0144] In this embodiment, as shown in Figure 5, an optional implementation of a target post-processing method based on in-box point clouds provided by this disclosure is as follows: Point cloud data is processed using both a target recognition model and a traditional clustering algorithm. The target recognition model identifies multiple target boxes, and the traditional clustering algorithm also clusters multiple target boxes. For the multiple target boxes identified by the target recognition model, dummy target boxes are filtered based on the reflectance information of the point clouds within the target boxes and the distribution information of the point clouds within the target boxes. For the multiple target boxes clustered by the traditional clustering algorithm, dummy target boxes are filtered based on the quantity and proportion of point cloud data obtained from each semantic category within the target boxes, as well as the reflectance information of the point clouds within the target boxes and the point cloud density information around the centroid and center point of the target boxes. After filtering multiple target boxes identified by the target recognition model, the boundary lines of the remaining target boxes are corrected. The point cloud data is analyzed and processed to determine the drivable area points of the target boxes located in the remaining target boxes after filtering. Then, based on the drivable area points of the target boxes, the convex hull of the individual target boxes is determined. When the boundary lines of the individual target boxes need to be corrected, the boundary lines of the individual target boxes are corrected based on the convex hull.

[0145] This disclosure provides a target post-processing system based on in-frame point clouds, as shown in Figure 6. The system 600 includes:

[0146] The target recognition unit 601 is configured to perform target recognition on point cloud data through a target recognition model to obtain multiple target boxes;

[0147] The analysis and processing unit 602 is configured to determine the target drivable area points located in the target frame by analyzing and processing the point cloud data;

[0148] The convex hull point set determination unit 603 is configured to determine the convex hull of the target frame and the convex hull point set corresponding to the convex hull based on the target drivable area points in the target frame, wherein the convex hull points in the convex hull point set are all the target drivable area points in the target frame;

[0149] The extreme convex hull point determination unit 604 is configured to determine the longitudinal maximum convex hull point, the transverse maximum convex hull point, and the transverse minimum convex hull point in the set of convex hull points of the target box;

[0150] The right convex hull point set determination unit 605 is configured to construct the right convex hull point set of the target frame when the target frame is located on the left side of the vehicle, based on the convex hull points in the target frame being located in the order of the longitudinal maximum convex hull point and the lateral minimum convex hull point, and the right convex hull points whose longitudinal distance from the lateral minimum convex hull point satisfies a first condition.

[0151] The left convex hull point set determination unit 606 is configured to construct the left convex hull point set of the target frame when the target frame is located on the right side of the vehicle, based on the left convex hull points in the target frame being located in the order of the longitudinal maximum convex hull point and the transverse maximum convex hull point, and the left convex hull points whose longitudinal distance from the transverse maximum convex hull point satisfies the second condition.

[0152] The first correction unit 607 is configured to correct the right boundary line of the target frame using the right convex hull point in the set of right convex hull points of the target frame when the target frame is located on the left side of the vehicle and the vehicle is turning to the left.

[0153] The second correction unit 608 is configured to correct the left boundary line of the target frame using the left convex hull point in the set of left convex hull points of the target frame when the target frame is located on the right side of the vehicle and the vehicle is turning right.

[0154] Optionally, the analysis and processing unit 602 includes:

[0155] The drivable area point determination unit is configured to determine drivable area points in the point cloud data by analyzing and processing the point cloud data.

[0156] The target drivable area point determination unit is configured to determine the target drivable area point located within the target frame based on the location of the target frame and the location of the drivable area point.

[0157] Optionally, the system 600 further includes:

[0158] The virtual target bounding box determination unit is configured to determine the virtual target bounding box among the plurality of target bounding boxes based on the attribute information of the target bounding boxes;

[0159] The filtering unit is configured to filter the dummy target boxes in the plurality of target boxes to obtain the remaining target boxes;

[0160] The analysis and processing unit 602 is configured to analyze and process the point cloud data to determine the target drivable area points located in the remaining target boxes after filtering.

[0161] Optionally, the system 600 further includes:

[0162] The target bounding box determination unit is configured to determine the target bounding box that needs orientation correction from all target bounding boxes based on the attribute information of the target bounding box;

[0163] The convex hull point set determination unit 603 is configured to determine the convex hull of the target frame that needs to be oriented and the convex hull point set corresponding to the convex hull, based on the target drivable area points in the target frame that need to be oriented.

[0164] Optionally, the first amendment 607 includes:

[0165] The traversal unit is configured to, when the target box is located on the left side of the vehicle and the vehicle is turning left, traverse the right convex hull points in the set of right convex hull points of the target box, and connect the currently traversed right convex hull point with the horizontal minimum convex hull point to form the right prediction boundary line.

[0166] The first correction sub-unit is configured to end the traversal and correct the right boundary line of the target box with the right predicted boundary line whose deviation meets the set condition if the angle deviation between the obtained right predicted boundary line and the right boundary line of the target box meets the set condition.

[0167] The second correction unit 608 includes:

[0168] The traversal unit is configured to, when the target box is located on the right side of the vehicle and the vehicle is turning right, traverse the left convex hull points in the set of left convex hull points of the target box, and connect the currently traversed left convex hull point with the largest lateral convex hull point to form the left prediction boundary line.

[0169] The second correction sub-unit is configured to end the traversal and correct the left boundary line of the target box with the left predicted boundary line whose deviation meets the set condition if the angle deviation between the obtained left predicted boundary line and the left boundary line of the target box meets the set condition.

[0170] Optionally, the system 600 further includes:

[0171] The clustering department is configured to perform clustering processing on point cloud data using clustering algorithms to obtain various cluster target boxes;

[0172] The first attribute determination unit is configured to determine the proportion of point clouds of different semantic categories of the clustering target box, determine the size of the clustering target box, determine the reflectance information of the point cloud within the clustering target box, and determine the first point cloud density of the point cloud located within a set range of the centroid of the clustering target box.

[0173] The second attribute determination unit is configured to divide the clustering target box into multiple segments in the vertical direction and calculate the cloud density of the second point around the center point of the first segment in the direction of movement of the clustering target box.

[0174] The virtual clustering target box determination unit is configured to determine the virtual clustering target boxes in each clustering target box based on the point cloud proportion, the size, the reflectivity information, the first point cloud density, and the second point cloud density;

[0175] The filtering unit is configured to filter the dummy clustering target boxes in each of the clustering target boxes to obtain the clustering results of the clustering algorithm.

[0176] Optionally, the reflectance information determining unit is configured to determine the reflectance information of the point cloud within the clustering target bounding box; the reflectance information determining unit includes:

[0177] The first reflectivity information determination unit is configured to determine the number of low reflectivity point clouds within the clustering target box, and to determine the maximum reflectivity and total reflectivity of the point clouds within the clustering target box, based on the attribute information of the point clouds within the clustering target box.

[0178] The second reflectivity information determination unit is configured to determine the proportion of low reflectivity point clouds within the clustering target box based on the total number of point clouds within the clustering target box and the number of low reflectivity point clouds.

[0179] The third reflectance information determination unit is configured to determine the average reflectance of the point cloud within the clustering target box based on the total number of point clouds and the total reflectance value within the clustering target box.

[0180] Optionally, the virtual clustering target bounding box determination part includes:

[0181] The first virtual clustering target box determination unit is configured to determine the clustering target box as a virtual clustering target box when the proportion of low reflectivity point cloud corresponding to the clustering target box is a set value;

[0182] The second virtual clustering target box determination unit is configured to determine the clustering target box as a virtual clustering target box when the proportion of low reflectivity point cloud corresponding to the clustering target box exceeds the first reflectivity threshold and the corresponding average reflectivity does not exceed the second reflectivity threshold.

[0183] The third virtual clustering target box determination unit is configured to determine the clustering target box as a virtual clustering target box when the proportion of low reflectivity point cloud corresponding to the clustering target box exceeds the third reflectivity threshold and the corresponding average reflectivity does not exceed the fourth reflectivity threshold and the corresponding maximum reflectivity does not exceed the fifth reflectivity threshold.

[0184] The fourth virtual clustering target box determination unit is configured to determine the size relationship between the size and the first size threshold, the density relationship between the first point cloud density and the first density threshold, and the quantity relationship between the total number of point clouds and the total number threshold when the proportion of point clouds of the noise semantic category in the clustering target box exceeds the category threshold.

[0185] The fifth virtual clustering target box determination unit is configured to determine whether the clustering target box is a virtual clustering target box based on the size relationship, the density relationship, and the quantity relationship;

[0186] The sixth virtual clustering target box determination unit is configured to determine whether the clustering target box is a virtual clustering target box based on the point cloud proportion of the background semantic category within the clustering target box, the size, and the total number of point clouds, when the point cloud proportion of the noise semantic category within the clustering target box does not exceed the category threshold; and to determine whether the clustering target box is a virtual clustering target box based on the point cloud proportion of the general obstacle semantic category within the clustering target box, the location of the clustering target box, and the size.

[0187] Optionally, the system 600 further includes:

[0188] The single clustering target box determination unit is configured to determine whether there are small-sized single clustering target boxes within a preset range around the vehicle category clustering target box;

[0189] The seventh virtual clustering target box determination unit is configured to determine the vehicle category clustering target box as a virtual clustering target box when there is a small-sized single clustering target box within a preset range around the vehicle category clustering target box.

[0190] This disclosure provides an electronic device, including: a processor, a memory, and computer instructions stored in the memory and running on the processor, wherein when executed by the processor, the computer instructions implement the steps of a target post-processing method based on in-frame point clouds as described in the first aspect of this disclosure.

[0191] This disclosure provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the steps of a target post-processing method based on in-frame point clouds as described in the first aspect of this disclosure.

[0192] As the system implementation is basically similar to the method implementation, it is described in a relatively simple way. For relevant details, please refer to the description of the method implementation.

[0193] For the sake of simplicity, the method embodiments are all described as a series of actions. The embodiments of this disclosure are not limited to the described order of actions, as some steps may be performed in a different order or simultaneously according to the embodiments of this disclosure. The actions involved in the embodiments described in the specification are not necessarily essential to the embodiments of this disclosure.

[0194] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0195] This disclosure can be provided as a method, system, or computer program product. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable instructions.

[0196] This disclosure describes embodiments of methods, terminal devices (systems), and computer program products according to embodiments of this disclosure with reference to flowchart illustrations and / or block diagrams. Each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer instructions. These computer instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0197] These computer instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0198] These computer instructions may also be loaded onto a computer or other programmable data processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable terminal equipment, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0199] Although embodiments of this disclosure have been described, other changes and modifications may be made to these embodiments. Therefore, the appended claims are intended to be interpreted as including the embodiments as well as all changes and modifications falling within the scope of this disclosure.

[0200] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0201] The above provides a detailed description of a target post-processing method, system, and product based on in-frame point cloud provided by the embodiments of this disclosure. The embodiments used in this document illustrate the implementation of this application. The description of the above embodiments is only for the purpose of helping to understand this application. At the same time, for those skilled in the art, there will be changes in the implementation and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A target post-processing algorithm based on in-frame point clouds, comprising: The point cloud data is used to identify targets using a target recognition model, resulting in multiple bounding boxes. By analyzing and processing the point cloud data, the target drivable area points located within the target frame are determined; Based on the target drivable area points in the target box, determine the convex hull of the target box and the set of convex hull points corresponding to the convex hull, wherein the convex hull points in the set of convex hull points are all the target drivable area points in the target box; Determine the vertical maximum convex hull point, the horizontal maximum convex hull point, and the horizontal minimum convex hull point in the set of convex hull points of the target bounding box; When the target box is located on the left side of the vehicle, the right convex hull point set of the target box is constructed by taking the convex hull points in the target box as being located in the order of the longitudinal maximum convex hull point and the transverse minimum convex hull point, and the right convex hull point whose distance from the transverse minimum convex hull point in the longitudinal direction satisfies the first condition. When the target box is located on the right side of the vehicle, the left convex hull point set of the target box is constructed by taking the convex hull points in the target box as being located in the order of the longitudinal maximum convex hull point and the transverse maximum convex hull point, and the left convex hull point whose longitudinal distance from the transverse maximum convex hull point satisfies the second condition. When the target frame is located on the left side of the vehicle and the vehicle is turning left, the right boundary line of the target frame is corrected using the right convex hull points in the set of right convex hull points of the target frame. and When the target frame is located on the right side of the vehicle and the vehicle is turning right, the left boundary line of the target frame is corrected using the left convex hull points in the set of left convex hull points of the target frame.

2. The target post-processing algorithm based on in-frame point cloud according to claim 1, wherein, By analyzing and processing the point cloud data, target drivable area points located within the target frame are determined, including: By analyzing and processing the point cloud data, drivable areas within the point cloud data are determined; and Based on the location of the target frame and the location of the drivable area point, determine the target drivable area point located within the target frame.

3. The target post-processing algorithm based on in-frame point cloud according to claim 1, wherein, Before determining the target drivable area points located within the target frame by analyzing and processing the point cloud data, the method further includes: Based on the attribute information of the target bounding boxes, determine the virtual target bounding boxes among the plurality of target bounding boxes; and Filter the dummy target boxes among the multiple target boxes to obtain the remaining target boxes; The step of analyzing and processing the point cloud data to determine the target drivable area points located within the target frame includes: By analyzing and processing the point cloud data, the drivable area points of the target located in the remaining target boxes after filtering are determined.

4. A target post-processing algorithm based on in-frame point clouds according to claim 1 or 3, wherein, Before determining the convex hull of the target bounding box and the set of convex hull points corresponding to the convex hull based on the target drivable area points in the target bounding box, the method further includes: Based on the attribute information of the target bounding boxes, determine the target bounding boxes that need orientation correction from all the target bounding boxes; The step of determining the convex hull of the target frame and the set of convex hull points corresponding to the convex hull based on the target drivable area points in the target frame includes: determining the convex hull of the target frame that needs to be oriented corrected and the set of convex hull points corresponding to the convex hull based on the target drivable area points in the target frame that need to be oriented corrected.

5. The target post-processing algorithm based on in-frame point cloud according to claim 1, wherein, When the target frame is located on the left side of the vehicle and the vehicle is turning left, the right boundary line of the target frame is corrected using the right convex hull points from the set of right convex hull points of the target frame, including: When the target bounding box is located to the left of the vehicle and the vehicle is turning left, traverse the right convex hull points in the set of right convex hull points of the target bounding box, and connect the currently traversed right convex hull point with the horizontal minimum convex hull point to form the right predicted boundary line; and If the angle deviation between the obtained right predicted boundary line and the right boundary line of the target box meets the set conditions, the traversal ends and the right predicted boundary line of the target box is corrected with the right predicted boundary line whose deviation meets the set conditions. When the target frame is located on the right side of the vehicle and the vehicle is turning right, the left boundary line of the target frame is corrected using the left convex hull points from the set of left convex hull points of the target frame, including: When the target bounding box is located on the right side of the vehicle and the vehicle is turning right, traverse the left convex hull points in the set of left convex hull points of the target bounding box, and connect the currently traversed left convex hull point with the largest lateral convex hull point to form the left predicted boundary line; and If the angle deviation between the obtained left predicted boundary line and the left boundary line of the target box meets the set conditions, the traversal ends and the left boundary line of the target box is corrected using the left predicted boundary line whose deviation meets the set conditions.

6. The target post-processing algorithm based on in-frame point cloud according to claim 1 further includes: The point cloud data is clustered using a clustering algorithm to obtain the target bounding boxes for each cluster. Determine the proportion of point clouds for different semantic categories of the clustering target box, determine the size of the clustering target box, determine the reflectance information of the point cloud within the clustering target box, and determine the first point cloud density of the point cloud within a set range of the centroid of the clustering target box. The clustering target box is divided into multiple segments vertically, and the cloud density of the second point around the center point of the first segment in the direction of movement of the clustering target box is calculated. Based on the point cloud proportion, the size, the reflectivity information, the first point cloud density, and the second point cloud density, the virtual clustering target boxes in each clustering target box are determined; and The dummy clustering boxes in each clustering target box are filtered to obtain the clustering result of the clustering algorithm.

7. A target post-processing algorithm based on in-frame point clouds according to claim 6, wherein, Determine the reflectance information of the point cloud within the cluster target bounding box, including: Based on the attribute information of the point cloud within the clustering target box, determine the number of low reflectivity point clouds within the clustering target box, as well as the maximum reflectivity and total reflectivity of the point clouds within the clustering target box. Based on the total number of point clouds within the clustering target box and the number of low-reflectivity point clouds, determine the proportion of low-reflectivity point clouds within the clustering target box; and The average reflectance of the point cloud within the clustering target box is determined based on the total number of point clouds and the total reflectance value within the clustering target box.

8. The target post-processing algorithm based on in-frame point cloud according to claim 7, wherein, Based on the point cloud proportion, the size, the reflectivity information, the first point cloud density, and the second point cloud density, the virtual clustering target boxes in each clustering target box are determined, including: When the proportion of low reflectivity point cloud corresponding to the clustered target box is a set value, the clustered target box is determined to be a virtual clustered target box; If the proportion of low reflectivity point cloud corresponding to the clustered target box exceeds the first reflectivity threshold and the corresponding average reflectivity does not exceed the second reflectivity threshold, the clustered target box is determined to be a virtual clustered target box. If the proportion of low reflectivity point cloud corresponding to the clustered target box exceeds the third reflectivity threshold, and the corresponding average reflectivity does not exceed the fourth reflectivity threshold and the corresponding maximum reflectivity does not exceed the fifth reflectivity threshold, the clustered target box is determined to be a virtual clustered target box. When the proportion of point clouds of noisy semantic categories within the clustering target box exceeds the category threshold, the size relationship between the size and the first size threshold is determined, the density relationship between the first point cloud density and the first density threshold is determined, and the quantity relationship between the total number of point clouds and the total number threshold is determined. Based on the size relationship, the density relationship, and the quantity relationship, determine whether the clustering target box is a dummy clustering target box; and If the proportion of point clouds representing noise semantic categories within the clustering target box does not exceed the category threshold, the clustering target box is determined to be a virtual clustering target box based on the proportion of point clouds representing background semantic categories within the clustering target box, the size, and the total number of point clouds. Furthermore, the clustering target box is also determined to be a virtual clustering target box based on the proportion of point clouds representing general obstacle semantic categories within the clustering target box, the location of the clustering target box, and the size.

9. The target post-processing algorithm based on in-frame point cloud according to claim 6 further includes: Determine whether there are small single clustering boxes within a preset range around the vehicle category clustering target box; and If there are small-sized single clustering target boxes within a preset range around the vehicle category clustering target box, the vehicle category clustering target box is determined to be a dummy clustering target box.

10. A target post-processing system based on in-frame point clouds, comprising: The target recognition unit is configured to perform target recognition on point cloud data using a target recognition model to obtain multiple target bounding boxes. The analysis and processing unit is configured to determine the target drivable area points located in the target frame by analyzing and processing the point cloud data; The convex hull point set determination unit is configured to determine the convex hull of the target frame and the convex hull point set corresponding to the convex hull based on the target drivable area points in the target frame, wherein the convex hull points in the convex hull point set are all the target drivable area points in the target frame; The extreme convex hull point determination unit is configured to determine the vertical maximum convex hull point, the horizontal maximum convex hull point, and the horizontal minimum convex hull point in the set of convex hull points of the target box; The right convex hull point set determination unit is configured to construct the right convex hull point set of the target frame when the target frame is located on the left side of the vehicle, based on the convex hull points in the target frame being located in the order of the longitudinal maximum convex hull point and the lateral minimum convex hull point, and the right convex hull points whose longitudinal distance from the lateral minimum convex hull point satisfies a first condition. The left convex hull point set determination unit is configured to construct the left convex hull point set of the target frame when the target frame is located on the right side of the vehicle, based on the left convex hull points in the target frame being located in the order of the longitudinal maximum convex hull point and the transverse maximum convex hull point, and the left convex hull points whose longitudinal distance from the transverse maximum convex hull point satisfies the second condition. The first correction unit is configured to correct the right boundary line of the target frame using the right convex hull points in the set of right convex hull points of the target frame when the target frame is located on the left side of the vehicle and the vehicle is turning to the left. and The second correction unit is configured to correct the left boundary line of the target frame using the left convex hull points in the set of left convex hull points of the target frame when the target frame is located on the right side of the vehicle and the vehicle is turning right.

11. An electronic device, comprising: A processor, a memory, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, implement the steps in a target post-processing method based on in-frame point clouds as described in claims 1 to 9.

12. A computer-readable storage medium storing computer instructions that, when executed by a processor, implement the steps of a target post-processing method based on in-frame point clouds as described in claims 1 to 9.

Images