Dynamic loading target point positioning method and system based on laser radar point cloud analysis

Abstract

The present application relates to the technical field of target point positioning based on point cloud data, in particular to a dynamic loading target point positioning method and system based on laser radar point cloud analysis; the method of the present application firstly extracts the core geometric features of the carriage based on the multi-frame point cloud data of the laser radar, constructs the hash key to quickly search the coarse pose of the carriage and the candidate area; then, the coarse pose is taken as the initial value to obtain the fine pose of the carriage and the region of interest through point cloud fine registration, and the target instance point cloud cluster is obtained through dynamic and static point division and adaptive clustering; then, the virtual self-reference point is screened to generate the candidate target point based on the repeatability score, and the target function is constructed by combining the observation uncertainty covariance to solve the optimal loading target point; finally, the target point confidence is calculated by fusing multi-dimensional information and the positioning result is output, and the reliability is quantified, thereby comprehensively improving the robustness and safety of the intelligent loading system.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, specifically to a method and system for dynamic vehicle loading target point localization based on lidar point cloud analysis. Background Technology

[0002] In machine vision-based logistics loading operations, to achieve automated grasping, alignment, handling, and placement of goods, pallets, or vehicle-related loading and unloading objects, it is usually necessary to first quickly locate the vehicle compartment area and then perform stable and accurate 3D positioning of the dynamic loading target points (such as goods grasping points, pallet fork hole alignment points, and placement alignment points) required for the loading operation in dynamic loading scenarios. Because LiDAR has advantages such as being unaffected by lighting conditions and being able to directly acquire 3D spatial information, existing loading systems often use LiDAR or multi-sensor fusion methods to acquire point cloud data of the vehicle compartment and loaded objects, and then use point cloud processing to achieve region recognition, target segmentation, and target point calculation.

[0003] However, existing technologies still have several shortcomings in dynamic loading applications. Regarding the localization of the loading compartment area, current solutions typically employ point cloud-based global feature matching or iterative registration (e.g., iterative optimization based on ICP / GICP) to estimate the compartment's pose and position. When the compartment structure is obstructed, the point cloud is sparse, noise is high, or initial values ​​are poor, the registration algorithm often requires numerous iterations to converge, resulting in lengthy localization times for the loading compartment area, which is difficult to meet the real-time cycle requirements of loading robots. Simultaneously, existing solutions often obtain target points by clustering point clouds and calculating geometric centers, boundary centers, or fitting regular geometry (e.g., cuboid / plane fitting); or by relying on deep learning networks to output key points. These methods are prone to drift or misjudgment, leading to significant target point localization errors in dynamic loading scenarios, thus affecting the success rate and safety of grasping, alignment, and placement operations.

[0004] Therefore, there is an urgent need for a method that can achieve rapid and stable positioning of the carriage area in dynamic loading scenarios and perform high-precision and sustainable tracking of dynamic loading target points. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for dynamic vehicle loading target point localization based on lidar point cloud analysis.

[0006] The technical solution of this invention is as follows: A method for dynamic vehicle loading target point localization based on lidar point cloud analysis includes: S1: Acquire multi-frame point cloud data collected by the lidar at continuous intervals. After distortion correction and intensity normalization, construct a sliding window fusion point cloud of length K and extract the geometric feature set of the carriage structure. The geometric feature set includes at least three of the following: the carriage floor plane, the left and right side wall planes, and the front wall plane. Based on the geometric feature set, construct the carriage structure signature, generate a hash key through quantization encoding, and retrieve the carriage coarse pose and candidate range of the carriage region from the carriage template hash library. S2: Starting with the coarse pose of the carriage, register the fused point cloud and the carriage template point cloud within the candidate area of ​​the carriage region to obtain the fine pose of the carriage and the region of interest of the carriage; within the region of interest of the carriage, divide the point cloud into static background points and dynamic candidate points, and perform adaptive clustering on the dynamic candidate points to obtain the target instance point cloud cluster; S3: Obtain all candidate key points for each target instance point cloud cluster, filter candidate key points based on repeatability score to obtain a virtual self-calibration point set; generate a target point candidate set based on the virtual self-calibration point set, estimate the observation uncertainty covariance for each candidate target point, substitute it into the objective function, and solve to obtain the dynamic loading target point; S4: Based on the set of virtual self-calibrated points and the dynamic loading target points, the confidence level of the target points is calculated, and the dynamic loading target points represented by the car body coordinate system are used to form the dynamic loading target point positioning result.

[0007] In S1, the method for obtaining the coarse pose of the carriage and the candidate range of the carriage region is as follows: Based on the geometric feature set, a carriage structure signature is constructed. The carriage structure signature is a feature vector composed of the set of angles between plane normal vectors, the relative distance between key planes, and the effective length of the plane intersection line. After coarse-grained quantization and fine-grained quantization, it is concatenated in a preset order to form a hash key. The hash key is input into the carriage template hash library. First, a preliminary matching is performed using the coarse-grained key to obtain a set of candidate carriage templates. The candidate template set is then filtered using the fine-grained key. During the filtering process, the carriage coarse pose and the candidate range of the carriage region are obtained based on the plane residual and the intersection line alignment error.

[0008] In S2, the method for obtaining the fine pose of the carriage and the region of interest (ROI) of the carriage is as follows: Within the candidate range of the carriage region, the coarse pose of the carriage is initialized to obtain the initial pose parameters for registration; simultaneously, point clouds outside the candidate range of the carriage region are removed from the fused point cloud to obtain the target subset of the fused point cloud; the target subset of the fused point cloud is processed by point weight calculation to obtain the weighted target point cloud; the weighted target point cloud and the carriage template point cloud are registered using the weighted least squares method of point-to-plane error based on the initial pose parameters to obtain the optimal registration transformation matrix; the optimal registration transformation matrix is ​​processed by pose analysis to obtain the fine pose of the carriage; the weighted point cloud and the initial ROI are processed by region filtering to obtain the corrected candidate ROI; the corrected candidate ROI is then processed by boundary fitting to obtain the carriage ROI.

[0009] The point weights are derived from temporal stability weights, intensity consistency weights, and geometric consistency weights. Temporal stability weights are derived from the number of times the voxel in the point cloud is reproduced within the sliding window. Intensity consistency weights are derived from the point intensity and local neighborhood intensity of the point cloud. Geometric consistency weights are derived from the distance residual and the angle residual between the point cloud and the plane of the carriage structure.

[0010] In S2, the method for dividing static background points and dynamic candidate points is as follows: within the region of interest of the carriage, a voxel grid is established, and for each voxel, the residual variance from the point within the voxel to the carriage structural plane and the number of times the voxel recurs within the sliding window are calculated; if the number of voxel recursors is greater than the recursor threshold and the residual variance is less than the residual variance threshold, it is determined to be a static background point, and the remaining point cloud is a dynamic candidate point.

[0011] In S3, the repeatability score is obtained based on the number of repeated observations of candidate keypoints within the sliding window, the variance of repeated observation locations, the variance of intensity, and the variance of the normal vector.

[0012] In S3, the objective function is constructed based on the observation uncertainty covariance, the visibility of candidate target points, and the safety of candidate target points.

[0013] A dynamic loading target point localization system based on lidar point cloud analysis, used to implement the aforementioned dynamic loading target point localization method based on lidar point cloud analysis, includes: The module for generating the coarse pose and candidate region of the carriage is used to acquire multi-frame point cloud data collected by the lidar at continuous time. After distortion correction and intensity normalization, a sliding window fusion point cloud of length K is constructed to extract the geometric feature set of the carriage structure. The geometric feature set includes at least three of the following: the carriage floor plane, the left and right side wall planes, and the front wall plane. Based on the geometric feature set, a carriage structure signature is constructed, and a hash key is generated through quantization encoding. The coarse pose and candidate region of the carriage are retrieved from the carriage template hash library. The target instance point cloud cluster generation module is used to initialize the coarse pose of the carriage, register the fused point cloud and the carriage template point cloud within the candidate range of the carriage region to obtain the fine pose of the carriage and the region of interest of the carriage; within the region of interest of the carriage, the point cloud is divided into static background points and dynamic candidate points, and adaptive clustering is performed on the dynamic candidate points to obtain the target instance point cloud cluster. The dynamic loading target point generation module is used to obtain all candidate key points of each target instance point cloud cluster, filter candidate key points based on repeatability score to obtain a set of virtual self-calibrated points; based on the set of virtual self-calibrated points, generate a set of target point candidates, estimate the observation uncertainty covariance for each candidate target point, substitute it into the objective function, and solve to obtain the dynamic loading target point; The dynamic loading target point positioning result generation module is used to calculate the target point confidence based on the set of virtual self-calibrated points and the dynamic loading target point, and form the dynamic loading target point positioning result by combining it with the dynamic loading target point represented by the car body coordinate system.

[0014] A dynamic vehicle loading target point localization device based on lidar point cloud analysis includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the above-mentioned dynamic vehicle loading target point localization method based on lidar point cloud analysis.

[0015] A computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-described method for dynamic loading target point localization based on lidar point cloud analysis.

[0016] The beneficial effects of this invention are as follows: This invention provides a dynamic loading target point localization method based on LiDAR point cloud analysis. First, distortion correction, intensity normalization, and sliding window fusion processing are performed on multiple consecutive frames of LiDAR point clouds to extract the core geometric features of the vehicle compartment. A hash key is then constructed to retrieve the coarse pose and candidate regions of the vehicle compartment, quickly completing coarse localization of the compartment area and effectively solving the time-consuming problem of traditional localization, thus improving the system's real-time performance. Next, fine registration of the point cloud is performed using the coarse pose as the initial value to obtain the fine pose of the vehicle compartment and the region of interest. Then, target instance point cloud clusters are obtained through dynamic and static point segmentation and adaptive clustering, ensuring the stability of target extraction. Next, virtual self-calibrated points are selected based on repeatability scores to generate candidate target points. An objective function is constructed by combining observation uncertainty covariance to solve for the optimal loading target point, significantly reducing target localization errors and achieving high-precision sustainable tracking. Finally, multi-dimensional information is fused to calculate the target point confidence and output the localization result, achieving reliability quantification and comprehensively improving the robustness and security of the intelligent loading system. This invention provides a dynamic loading target point localization method based on lidar point cloud analysis, which can be applied in practical engineering such as dynamic loading, intelligent loading, and automated loading and unloading in mining / logistics. By accurately extracting the edge area and geometrically degraded area of ​​the truck body, it can effectively identify and avoid areas that are prone to positioning errors and operational risks, such as the edge of the truck body, sparse point cloud, and weak planar structure. It reduces the problem of target point misselection and positioning deviation from the source, significantly improves the positioning safety, operational stability and environmental adaptability of the intelligent loading system in complex real-world scenarios, and reduces the probability of failures such as equipment collisions and loading deviations. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the exemplary embodiments of this application clearer, the technical solutions in the exemplary embodiments of this application are described clearly and completely below. Obviously, the described exemplary embodiments are only some embodiments of this application, and not all embodiments.

[0018] This embodiment addresses the problems of long positioning time in the cargo compartment area and large positioning errors caused by the complex and variable position, posture and shape of target points (such as goods, pallets, vehicles, etc.) and occlusion. It proposes a dynamic loading target point positioning method based on lidar point cloud analysis to achieve rapid and stable positioning of the cargo compartment area, high-precision determination of dynamic loading target points and confidence output, thereby supporting the stable tracking and decision-making of target points in the intelligent loading robot system.

[0019] In this embodiment, "carriage area" refers to the three-dimensional spatial range of the carriage that needs to be identified and located during the loading process, which can be determined by the carriage structure (floor, side walls, front wall, etc.); "target point" refers to the key positioning point related to the loading operation in the dynamic loading scenario, including but not limited to cargo grabbing / placement points, pallet fork hole alignment points, vehicle-related loading and unloading positioning points, etc.; "self-marked point" refers to the set of key points that are automatically generated from the target point instance point cloud without the need to paste physical markers and can be repeatedly observed in time sequence.

[0020] This embodiment provides a method for dynamic vehicle loading target point localization based on lidar point cloud analysis, including: S1: Acquire multi-frame point cloud data collected by the lidar at continuous intervals. After distortion correction and intensity normalization, construct a sliding window fusion point cloud of length K and extract the geometric feature set of the carriage structure. The geometric feature set includes at least three of the following: the carriage floor plane, the left and right side wall planes, and the front wall plane. Based on the geometric feature set, construct the carriage structure signature, generate a hash key through quantization encoding, and retrieve the carriage coarse pose and candidate range of the carriage region from the carriage template hash library. S2: Starting with the coarse pose of the carriage, register the fused point cloud and the carriage template point cloud within the candidate area of ​​the carriage region to obtain the fine pose of the carriage and the region of interest of the carriage; within the region of interest of the carriage, divide the point cloud into static background points and dynamic candidate points, and perform adaptive clustering on the dynamic candidate points to obtain the target instance point cloud cluster; S3: Obtain all candidate key points for each target instance point cloud cluster, filter candidate key points based on repeatability score to obtain a virtual self-calibration point set; generate a target point candidate set based on the virtual self-calibration point set, estimate the observation uncertainty covariance for each candidate target point, substitute it into the objective function, and solve to obtain the dynamic loading target point; S4: Based on the set of virtual self-calibrated points and dynamic loading target points, the confidence level of the target points is calculated, and the dynamic loading target points represented by the car body coordinate system are compared to form the dynamic loading target point positioning result; The specific steps are detailed below.

[0021] S1: Acquire multi-frame point cloud data collected by the lidar at continuous time intervals. After distortion correction and intensity normalization, construct a sliding window fusion point cloud of length K and extract the geometric feature set of the carriage structure. Based on the geometric feature set, construct the carriage structure signature, generate a hash key through quantization encoding, and retrieve the coarse pose of the carriage and the candidate range of the carriage region from the carriage template hash library.

[0022] First, multiple frames of point cloud data collected continuously by the LiDAR on the robot used for grasping the target are acquired to obtain a point cloud dataset. Each point in the dataset contains three-dimensional coordinates, point intensity (signal strength), and sampling timestamp information.

[0023] Then, in order to solve the problem that points with different timestamps within the same frame will shift due to motion, causing distortions such as "stretching, twisting, and edge misalignment" in the carriage point cloud, this embodiment performs distortion removal processing on the point cloud dataset to obtain a distortion-removed point cloud dataset. The distortion removal processing operation is as follows.

[0024] When IMU / odometry information is available, based on the pose increment data within a single frame scan cycle of the lidar, lidar point cloud data with different timestamps in the same frame are uniformly spatially transformed to the reference time of that frame (corresponding frame), thus completing the distortion correction of the point cloud within the frame. The IMU data is inertial data collected by the inertial measurement unit mounted on the same mobile platform as the lidar.

[0025] When IMU / odometry information is unavailable, the nearest-point algorithm is used to perform small-scale registration of the point clouds in the overlapping area between the current frame and adjacent frames, and to solve for the rigid body transformation matrix that minimizes the point cloud registration error. Based on the rigid body transformation matrix and the LiDAR scanning cycle, the short-time rotational angular velocity and linear velocity of the LiDAR within the scanning cycle of the current frame are calculated as the short-time motion state of the LiDAR. Based on this short-time motion state and the acquisition timestamp of each point cloud in the current frame, a distortion compensation transformation matrix (homogeneous coordinate form) is constructed for each point cloud. Each point cloud in the current frame is converted into homogeneous coordinates, and after inverse transformation by the distortion compensation transformation matrix, the pose offset caused by motion is eliminated to obtain the compensated homogeneous coordinates. Then, the coordinates are restored to three-dimensional coordinates to achieve distortion removal of the point clouds in adjacent frames.

[0026] Next, the point cloud distortion-reduced dataset is subjected to distance-based point intensity normalization to reduce the impact of distance attenuation and material reflection differences on subsequent weight calculations, resulting in preprocessed point cloud data.

[0027] Distance-based point intensity normalization can be achieved using the following formula: , For the normalized point intensity, For the initial point intensity, The straight-line distance between the point cloud detection point and the lidar sensor. This is the distance attenuation compensation coefficient.

[0028] Subsequently, based on the preprocessed point cloud data, a sliding window fusion point cloud of length K is constructed. This can be achieved by transforming the most recent K frame point clouds to a unified reference coordinate and then overlaying them, as shown in the following formula: , For the present t The point cloud is merged using a sliding window at any given time, where the length of the sliding window is the value. For the first i Frame point cloud to the first t The spatial transformation matrix of the frame reference coordinate system is calculated by accumulating the results of point cloud distortion correction and adjacent frame registration. For the first i Point cloud data of frames.

[0029] Furthermore, a set of geometric features of the carriage structure is extracted from the fused point cloud. Specifically, at least three approximately orthogonal carriage structure planes are extracted from the fused point cloud using the RANSAC method, including but not limited to: the carriage floor plane, the left and right side wall planes, and the front wall plane. Each plane can be represented by a parametric normal vector and a plane bias.

[0030] Finally, a carriage structure signature is constructed based on the geometric feature set, a hash key is generated through quantization encoding, and the coarse pose of the carriage and the candidate range of the carriage region are retrieved from the preset carriage template hash library. The specific steps are as follows.

[0031] Step 1: Construct a carriage structure signature based on the geometric feature set. The carriage structure signature is a feature vector composed of a set of plane normal vector angles, relative distances between key planes, and the effective length of plane intersection lines. The set of plane normal vector angles is obtained from the angles formed between multiple plane normal vectors and is used to describe the spatial relationships between different planes inside the carriage. The relative distances between key planes include the distance between the left and right side walls of the carriage and the distance between the front wall and the direction of the carriage opening. The effective length of plane intersection lines (e.g., the intersection line between the floor and the side walls) reflects the shape and size of the carriage structure. The carriage structure signature constructed from these geometric feature sets provides the foundation for subsequent hash key generation. This signature can fully describe the geometric structural characteristics of the carriage, providing reliable information for carriage template matching.

[0032] Step 2: To achieve efficient carriage template matching and fast retrieval, the geometric features of the carriage structure signature are quantized. Specifically, coarse-grained quantization and fine-grained quantization are performed sequentially. In coarse-grained quantization, the plane normal vector angle is binned at 5°~10°, the distance between key planes is binned at 5~10 cm, and the intersection length is binned at 10~20 cm. In fine-grained quantization, the plane normal vector angle is binned at 1°~2°, the distance is binned at 1~2 cm, and the intersection length is binned at 2~5 cm. Then, each quantized geometric feature is concatenated in a preset order to form a hash key. The hash key contains the quantized values ​​of multiple geometric features and can be used for efficient carriage template matching.

[0033] Step 3: In the preset carriage template hash library, a search is performed based on the generated hash key. Specifically, by inputting the hash key into the carriage template hash library, a preliminary matching is first performed using a coarse-grained key to obtain a set of candidate carriage templates. Then, the candidate template set is filtered using a fine-grained key. During the filtering process, the templates are selected based on the planar residual and the intersection line alignment error. If the planar matching error is small (less than the planar matching error threshold) and the intersection line alignment error is small (less than the intersection line alignment error threshold), then the candidate template is considered to be similar to the current carriage structure, thereby determining the coarse pose of the carriage and its corresponding region range, and obtaining the coarse pose of the carriage and the candidate region range of the carriage.

[0034] If the hash key retrieval fails, meaning a matching template cannot be found in the hash database using the carriage structure signature, then the principal component analysis results of the carriage's ground plane and the direction of the carriage opening are used as the carriage's coarse pose to ensure the process is usable.

[0035] In this embodiment, in S1 above, multi-frame point cloud data collected by the lidar at continuous intervals is acquired, and after distortion removal and intensity normalization processing, a sliding window fusion point cloud of length K is constructed. This can enhance the temporal stability and usable density of the carriage structure points while suppressing the effects of single-frame noise, sparsity, and short-term occlusion. Furthermore, the geometric feature set of the carriage structure is extracted from the fusion point cloud and a carriage structure signature is constructed. A hash key is generated through quantization encoding, and the coarse pose of the carriage and the candidate range of the carriage region are quickly retrieved in the preset carriage template hash library. This can replace the long initialization process of global iterative registration with a "hash retrieval + a small amount of consistency verification" method, thereby significantly reducing the carriage region localization time and improving the robustness to occlusion, point cloud missing and external interference in dynamic loading scenarios. This provides a stable and reliable initial region and pose prior for subsequent fine registration, target segmentation and target point localization.

[0036] S2: Initialize the coarse pose of the carriage, register the fused point cloud and the carriage template point cloud within the candidate area of ​​the carriage region to obtain the fine pose of the carriage and the region of interest of the carriage; within the region of interest of the carriage, divide the point cloud into static background points and dynamic candidate points, and perform adaptive clustering on the dynamic candidate points to obtain the target instance point cloud cluster.

[0037] First, using the coarse pose of the carriage as initialization, the fused point cloud and the carriage template point cloud are registered within the candidate range of the carriage area to obtain the fine pose of the carriage and the region of interest (cargo area) of the carriage. The specific steps are detailed below.

[0038] Step 1: Within the candidate area of ​​the carriage region, the coarse pose of the carriage is initialized to obtain initial rotation / translation parameters, thus obtaining the initial pose parameters for registration. This avoids the registration algorithm getting trapped in local optima and significantly improves the convergence efficiency of registration. Simultaneously, points outside the candidate area of ​​the carriage region are removed from the fused point cloud, limiting the spatial range of registration, reducing the computational load, and obtaining the target subset of the fused point cloud.

[0039] Step 2: The target subset of the fused point cloud is processed by point weight calculation to obtain a weighted target point cloud, which enhances the robustness to occlusion, dynamic objects and outliers in dynamic loading scenarios, and makes the registration process more focused on the "temporally stable, intensity consistent and geometrically fit" carriage structure points.

[0040] Point weights are derived from temporal stability weights, intensity consistency weights, and geometric consistency weights, and are calculated using the following formula: , For point weights, , , These are time-series stability weight, intensity consistency weight, and geometric consistency weight, respectively.

[0041] Among them, time-stable weights The weight is determined by the number of times the voxel of the point cloud is reproduced within the sliding window. The more times it is reproduced, the greater the weight. The calculation formula is as follows: , For the voxel containing the point cloud, This represents the number of frames (replay count) occupied by the voxel within the sliding window. This is the normalization constant.

[0042] Strength Consistency Weight Determined by the point intensity and local neighborhood intensity of the point cloud, anomalies in intensity can be suppressed using an exponential decay function, calculated as follows: , For point p Intensity value after distance normalization For point p The local neighborhood set of points, used to represent the points p The local spatial environment in which it is located For point p local neighborhood The average intensity is used to reflect the representative point. p Typical intensity level of the local area For point p local neighborhood The standard deviation of intensity reflects the range of intensity fluctuations within a local area. This is the weighted smoothing term.

[0043] Geometric Consistency Weight The distance residual and the included angle residual between the point cloud and the plane of the carriage structure are used to enhance the contribution of the stable points of the carriage structure to the registration. The calculation formula is as follows: , For point p The distance residual to the nearest structural plane of the carriage, please provide guidance. p The smaller the residual between the three-dimensional coordinates of a point and the vertical distance between the point and the corresponding carriage structure plane, the better. p The closer it conforms to the actual geometry of the carriage, the better. For point p The variance hyperparameter of the point-to-plane residual is an empirical value set based on the geometric accuracy requirements of the carriage structure plane. It is used to control the degree of attenuation of the weights by the point-to-plane residual. For point p The angular residual between the normal vector and the normal vector of the nearest carriage structural plane, indicating... p The angle between its local normal vector and the global normal vector of its corresponding carriage structural plane; the smaller the angle, the more significant the difference. p The more consistent the geometric orientation is with the plane of the carriage, the better. For point p The variance of the residuals of the normal vector angle is an empirical value set based on the normal consistency requirements of the carriage structure. It is used to control the degree to which the residuals of the normal vector angle attenuate the weights. This is the balance coefficient.

[0044] Step 3: The weighted target point cloud and the carriage template point cloud, based on the initial pose parameters, are registered using the least squares method of weighted point-to-plane error to obtain the optimal registration transformation matrix. This optimal transformation matrix is ​​then processed through pose analysis to obtain the rotation matrix and translation vector, which are then orthogonalized and output as preset pose parameters to obtain the carriage's precise pose. Simultaneously, the weighted point cloud and the initial region of interest (ROI) undergo region filtering to remove point clouds outside the current observation coordinate system (this can be achieved by updating the initial ROI to the current observation coordinate system based on the carriage's precise pose). This results in a corrected candidate ROI. The corrected candidate ROI is then fitted to the boundary to obtain the carriage's ROI, clarifying the core range for subsequent cargo area processing. The initial ROI is obtained by transforming the original coordinates of the template ROI with the optimal registration transformation matrix.

[0045] Then, within the region of interest in the carriage, the point cloud is divided into static background points and dynamic candidate points. Specifically, a voxel mesh is established (the voxel side length can be 3-8 cm), and for each voxel, the residual variance from the points within the voxel to the carriage structural plane and the number of times the voxel reappears within the sliding window are calculated.

[0046] If the number of voxel recurrences is greater than the recurrence threshold and the residual variance is less than the residual variance threshold, it is determined to be a static background point (carriage structure or fixed component), and the remaining point cloud is a dynamic candidate point (the point cloud to which the target points such as cargo, pallet, and vehicle parts belong).

[0047] Finally, adaptive clustering is performed on the dynamic candidate points to avoid interference from the background of the carriage structure on the target segmentation, reduce false detections and target point offset, and obtain the target instance point cloud cluster. The operation steps of adaptive clustering are as follows.

[0048] Step 1: Based on the distance of each dynamic candidate point to the origin of the lidar, generate their respective adaptive neighborhood radii in a segmented manner. The farther the distance, the larger the adaptive neighborhood radius, which adapts to the "near dense and far sparse" characteristics of the point cloud and avoids the breakage or misconnection of the sparse point cloud at a long distance.

[0049] Step 2: Adaptive neighborhood radius based on each dynamic candidate point Each obtains its own set of candidate neighborhood points; where the adjacency constraint criterion is: if point satisfy and (Adjacent scan lines / ring numbers), then the point For point p Neighborhood candidate points, For point p The adaptive neighborhood radius, , For point p、 Scan line / ring number information.

[0050] Step 3: Use each dynamic candidate point as its initial clustering seed point. After clustering, obtain the initial target instance point cloud cluster. During the clustering process, iteratively merge nodes based on the seed point, satisfying the condition "neighborhood radius ≤ Adjacent density ≥ density threshold The neighboring points of “” form an independent initial point cloud cluster.

[0051] Step 4: Based on the inter-cluster overlap rate and centroid motion deviation between the initial target instance point cloud cluster and the target cluster of the previous frame, perform merging or splitting correction processing to obtain the target instance point cloud cluster.

[0052] Center of mass motion deviation The calculation formula is as follows: , , The current point cloud cluster's centroid, For the current point cloud cluster, For the point cloud in the current point cloud cluster, The centroid of the point cloud cluster in the previous frame. The velocity of the point cloud cluster in the previous frame. This is the frame interval time.

[0053] If, in the current frame, two or more initial point cloud clusters establish valid matching relationships with the same target point cloud cluster from the previous frame, then a merging correction is performed. A valid matching relationship is defined as follows: the overlap rate between clusters is not less than the overlap rate threshold, and the centroid motion deviation is not greater than the motion deviation threshold. If, in the current frame, one initial point cloud cluster establishes a valid matching relationship with two or more different target point cloud clusters from the previous frame, then a split correction is performed.

[0054] In S2 of this embodiment, the fused point cloud and the point cloud of the carriage template are registered within the candidate range of the carriage region using the coarse pose of the carriage as initialization. This can quickly converge and obtain the accurate fine pose of the carriage and the region of interest of the carriage while reducing the search space, thereby reducing the registration iteration overhead and improving robustness to initial value deviation and local occlusion. Furthermore, the point cloud is divided into static background points and dynamic candidate points within the region of interest of the carriage, and adaptive clustering is performed only on the dynamic candidate points. This can effectively eliminate the interference of the carriage structure background on target segmentation, avoid false detection and target point offset caused by the adhesion of the carriage wall / floor to the loaded target, and obtain a stable target instance point cloud cluster through distance adaptive neighborhood and cross-frame consistency constraints, providing a clean and consistent instance-level input for subsequent key point / self-label point extraction, target point calculation and stable tracking.

[0055] S3: Obtain candidate key points for each target instance point cloud cluster, and obtain a virtual self-calibrated point set based on the repeatability score of the candidate key points; generate a target point candidate set based on the virtual self-calibrated point set, estimate the observation uncertainty covariance for each candidate target point, substitute it into the objective function, and solve to obtain the dynamic loading target point.

[0056] First, obtain candidate key points for each target instance point cloud cluster. The set of candidate key points can contain at least the following two types of key points: points with maximum curvature (e.g., edges, corners), fitted plane boundary points (e.g., top / side boundary of the box), and concave structure boundary points (e.g., pallet fork hole edge).

[0057] Then, the repeatability score of each candidate key point is calculated within the sliding window, and the m key points with the highest scores are selected to form a set of self-labeled points, thus obtaining a set of virtual self-labeled points. A stable "reference point set" can be formed without pasting physical markers, providing anchor points for subsequent target point candidate generation, data association and stable tracking.

[0058] The repeatability score is determined by the number of repeated observations of the candidate keypoint within the sliding window, the variance of the repeated observation location, the variance of the intensity, and the variance of the normal vector. The calculation formula is as follows: , For the first The repeatability score of each candidate key point is used to measure the reliability of the point as a self-targeting point. The higher the score, the more suitable it is as a self-targeting point for stable tracking in dynamic scenarios. For the first The number of times a candidate keypoint is repeatedly observed within the sliding window (or the number of frames occupied by the voxel) reflects the frequency at which the point is observed by the LiDAR in time sequence. The more times it is observed, the more stable the point is in the scene. For the first The variance of the position of each candidate keypoint within the sliding window is calculated from the dispersion of the three-dimensional coordinates of the point in different frames, reflecting the spatial stability of the point. The smaller the variance, the more stable the position. For the first The intensity variance of each candidate keypoint within the sliding window is calculated from the dispersion of the intensity value of the point in different frames, reflecting the stability of the point's reflection intensity. The smaller the variance, the more stable the intensity. For the first The variance of the normal vector of each candidate keypoint within the sliding window is calculated from the dispersion of the normal vector of that point in different frames. The smaller the variance, the more stable the local geometry. This is the normalization coefficient (prior value) for the location variance, used to scale the location variance term and balance the influence of different variance magnitudes. This is the normalization coefficient (prior value) for the strength variance, used to scale the strength variance term; This is the normalization coefficient (prior value) of the normal vector variance, used to scale the normal vector variance term; The weighting coefficients for the repeated observations, location variance, intensity variance, and normal vector variance terms are hyperparameters obtained through experiments or training. They are used to adjust the contribution of each factor to the overall stability score. ， The more times it is realized, the more stable it becomes; the larger the variance, the less stable it becomes.

[0059] Next, based on the set of virtual self-target points, a candidate set of target points is generated. The generation methods include, but are not limited to, […]. For regular goods (boxes or panels): the top plane is fitted based on the set of virtual self-calibrated points, and the center of the top surface and the edge offset points are taken as candidate target points; for pallets: the center line of the fork hole is fitted based on the self-calibrated points of the concave structure, and the intersection of the center line and the preset height section is taken as candidate target points; for irregular goods: the center of the grabbable area and its surrounding disturbance points are generated based on the set of virtual self-calibrated points and the main axis direction.

[0060] Subsequently, for each candidate target point, the observation uncertainty covariance is estimated, and the covariance is used to explicitly characterize the observation reliability of the candidate target point, making the subsequent target point selection interpretable and automatically penalizing occlusion, sparsity and degenerate geometry.

[0061] The observation uncertainty covariance is obtained based on the distance noise variance of the candidate target point, the degree of local geometric degradation, and the local normal vector. The calculation formula is as follows: , , The observation uncertainty covariance matrix is ​​used to quantify the location uncertainty of a point during the observation process. The distance noise variance is a function of distance. The varying measurement noise term reflects the characteristic that the ranging error of the lidar increases with increasing distance. , is a candidate target point g To the origin of the lidar o The three-dimensional Euclidean distance is the input variable for distance noise. These are the weighting coefficients. The identity matrix is ​​used to expand the distance noise variance into an isotropic covariance matrix, representing the uniform distribution of ranging error in all directions of space. Candidate target points g The degree of local geometric degradation in the neighborhood is calculated from the ratio of the eigenvalues ​​of the neighborhood covariance matrix. A larger value indicates a more degraded geometry in the region where the point is located (e.g., a thin plane or blurred edges), resulting in lower observation reliability. Candidate target points g The local normal vector is used to characterize the main direction of geometric degradation, along which the degradation covariance term is amplified. This is the outer product matrix, used to project the geometric degradation terms onto the normal vector direction, thus giving the covariance matrix greater uncertainty in the degradation direction. Candidate target points g neighborhood point set The first, second, and third eigenvalues ​​of the covariance matrix are given by: The largest eigenvalue, The minimum eigenvalue is used to describe the geometric distribution characteristics of the neighborhood point cloud. This is the smoothing term for observation uncertainty.

[0062] Finally, the candidate target points are substituted into the objective function to obtain the dynamic loading target points. The objective function is based on the observation uncertainty covariance, the visibility of candidate target points, and the safety of candidate target points. This makes target point selection no longer dependent on a single geometric center or a single frame network score, and can output more robust and safer target points under occlusion, missing targets, and multi-target interference in dynamic loading scenarios. The calculation formula is as follows: , , , , Candidate target points g The observation uncertainty covariance matrix The trace value represents the uncertainty of the location at that point. A larger trace value indicates that the observation is less reliable. Candidate target points g The visibility value ranges from [0,1]. A larger value indicates less obstruction and clearer observation. Candidate target points g The safety level represents the minimum distance from that point to the danger zone. , These are visibility weight and security weight, respectively. Candidate target points g The occlusion ratio is calculated using methods such as ray sampling; a larger value indicates a higher degree of occlusion at that point. Candidate target points g To the candidate range boundary of the carriage area distance, Candidate target points g To other target instance clusters in the current frame The minimum distance, For the first j The set of candidate target points for a frame. For the set of candidate target points In, make the objective function The point that yields the minimum value is the final selected dynamic loading target point.

[0063] In this embodiment, S3 generates candidate key points for each target instance point cloud cluster and filters them based on repeatability scores within a sliding window to obtain a set of virtual self-calibrated points, enabling the system to obtain stable reference points without additional physical markings. Furthermore, based on the set of virtual self-calibrated points, a candidate set of target points is generated, and the observation uncertainty covariance of the candidate target points is estimated. The candidate target points are then substituted into an objective function that includes uncertainty, visibility, and safety to solve for the optimal dynamic loading target point. This significantly reduces the risk of target point offset and misselection under conditions of occlusion, sparsity, and geometric degradation, and improves the stability and reliability of dynamic loading target point positioning.

[0064] S4: Based on the set of virtual self-calibrated points and the dynamic loading target points, the confidence level of the target points is calculated, and the dynamic loading target points represented by the car body coordinate system are used to form the dynamic loading target point positioning result.

[0065] By calculating the confidence level of the target point based on the set of virtual self-calibrated points and the solved dynamic loading target point, and outputting the target point in the car body coordinate system to form the dynamic loading target point positioning result, the "target point position" and "reliability" can be expressed simultaneously in a quantitative way.

[0066] The confidence level is specifically based on the observation uncertainty covariance and visibility of the dynamic loading target points, the average repeatability score of the virtual self-calibration point set, and the marginal degradation error. The calculation formula is as follows: , , For the first Frame-based dynamic loading target point Confidence level, For loading target point The trace of the observation uncertainty covariance matrix represents the location uncertainty at that point. The larger the trace value, the less reliable the observation is and the lower the confidence level. For the first The average repeatability score of the set of virtual self-labels in a frame reflects the overall stability of candidate points in that frame as self-labels. Edge degradation error measures how close a target point is to the edge of the carriage or a geometrically degraded area. A larger value indicates that it is closer to a danger zone and a lower confidence level. For loading target point The straight-line distance to the nearest edge of the carriage. For loading target point The straight-line distance to the nearest geometrically degraded region can be obtained by weighting the degradation factors of neighboring points. The larger the degradation factor, the higher the degree of degradation of the corresponding region, and the greater the weight of the distance. , These are the preset maximum edge distance and maximum degradation area distance for dynamic loading scenarios, respectively, to avoid calculation errors caused by excessively large distances. For loading target point The visibility value indicates less obstruction, clearer observation, and higher confidence. For the Sigmoid function, These are, respectively, the observation uncertainty weight, the average repeatability score weight, the marginal degradation error weight, and the visibility weight of the loading target point. This is a smoothing term.

[0067] The method for obtaining the aforementioned geometrically degraded regions is as follows: Based on all dynamic loading point cloud data, key edge regions such as the edges of the carriage walls and the edges of the carriage openings are extracted using an edge detection algorithm; within these key edge regions, a local geometric degradation detection method is used to detect dynamic loading target points. neighborhood point set Construct the covariance matrix and calculate its largest eigenvalue. and minimum eigenvalue When the ratio of eigenvalues When the degradation threshold is greater than the preset degradation threshold ( (The eigenvalue ratio smoothing term) determines that the region is a geometrically degenerate region, such as a sparse point cloud region near the edge of the carriage or a region with an excessively thin plane.

[0068] This embodiment also provides a dynamic vehicle loading target point localization system based on lidar point cloud analysis, used to implement the above-mentioned dynamic vehicle loading target point localization method based on lidar point cloud analysis, including: The module for generating the coarse pose and candidate region of the carriage is used to acquire multi-frame point cloud data collected by the lidar at continuous time. After distortion correction and intensity normalization, a sliding window fusion point cloud of length K is constructed to extract the geometric feature set of the carriage structure. The geometric feature set includes at least three of the following: the carriage floor plane, the left and right side wall planes, and the front wall plane. Based on the geometric feature set, a carriage structure signature is constructed, and a hash key is generated through quantization encoding. The coarse pose and candidate region of the carriage are retrieved from the carriage template hash library. The target instance point cloud cluster generation module is used to initialize the coarse pose of the carriage, register the fused point cloud and the carriage template point cloud within the candidate range of the carriage region to obtain the fine pose of the carriage and the region of interest of the carriage; within the region of interest of the carriage, the point cloud is divided into static background points and dynamic candidate points, and adaptive clustering is performed on the dynamic candidate points to obtain the target instance point cloud cluster. The dynamic loading target point generation module is used to obtain all candidate key points of each target instance point cloud cluster, filter candidate key points based on repeatability score to obtain a set of virtual self-calibrated points; based on the set of virtual self-calibrated points, generate a set of target point candidates, estimate the observation uncertainty covariance for each candidate target point, substitute it into the objective function, and solve to obtain the dynamic loading target point; The dynamic loading target point positioning result generation module is used to calculate the target point confidence based on the set of virtual self-calibrated points and the dynamic loading target point, and form the dynamic loading target point positioning result by combining it with the dynamic loading target point represented by the car body coordinate system.

[0069] This embodiment also provides a dynamic vehicle loading target point positioning device based on lidar point cloud analysis, including a processor and a memory, wherein the processor executes the computer program stored in the memory to implement the above-mentioned dynamic vehicle loading target point positioning method based on lidar point cloud analysis.

[0070] This embodiment also provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-described method for dynamic vehicle loading target point localization based on lidar point cloud analysis.

[0071] This embodiment provides a dynamic loading target point localization method based on LiDAR point cloud analysis. First, distortion correction, intensity normalization, and sliding window fusion processing are performed on multiple consecutive frames of LiDAR point cloud data to extract the core geometric features of the vehicle compartment. A hash key is then constructed to retrieve the coarse pose and candidate regions of the vehicle compartment, quickly completing the coarse localization of the vehicle compartment area and effectively solving the problem of time-consuming traditional localization, thus improving the real-time performance of the system. Next, fine registration of the point cloud is performed using the coarse pose as the initial value to obtain the fine pose of the vehicle compartment and the region of interest. Then, target instance point cloud clusters are obtained through dynamic and static point division and adaptive clustering to ensure the stability of target extraction. Next, virtual self-calibrated points are selected based on repeatability scores to generate candidate target points. An objective function is constructed by combining the observation uncertainty covariance to solve for the optimal loading target point, significantly reducing the target localization error and achieving high-precision sustainable tracking. Finally, multi-dimensional information is fused to calculate the confidence of the target point and output the localization result, realizing reliability quantification, thereby comprehensively improving the robustness and security of the intelligent loading system.

[0072] This embodiment provides a dynamic loading target point localization method based on lidar point cloud analysis, which can be applied in practical engineering such as dynamic loading, intelligent loading, and automated loading and unloading in mining / logistics. By accurately extracting the edge area and geometrically degraded area of ​​the truck body, it can effectively identify and avoid areas that are prone to positioning errors and operational risks, such as the edge of the truck body, sparse point cloud, and weak planar structure. It reduces the problem of target point misselection and positioning offset from the source, significantly improves the positioning safety, operational stability and environmental adaptability of the intelligent loading system in complex real-world scenarios, and reduces the probability of failures such as equipment collisions and loading deviations.

[0073] While exemplary embodiments of the invention have been described herein, many other variations or modifications conforming to the principles of the invention can be directly determined or derived from the disclosure of this invention without departing from its spirit and scope. Therefore, the scope of the invention should be understood and recognized to cover all such other variations or modifications.

Claims

1. A method for dynamic vehicle loading target point localization based on lidar point cloud analysis, characterized in that, include: S1: Acquire multi-frame point cloud data collected by the lidar at continuous times, and after distortion removal and intensity normalization processing, construct a sliding window fusion point cloud of length K to extract the geometric feature set of the carriage structure. The set of geometric features includes at least three of the following: the floor plane of the carriage, the left and right side wall planes, and the front wall plane; Based on the set of geometric features, a carriage structure signature is constructed, and a hash key is generated through quantization encoding. The coarse pose of the carriage and the candidate range of the carriage region are retrieved from the carriage template hash library. S2: Initialize the coarse pose of the carriage, and within the candidate range of the carriage region, register the fused point cloud and the carriage template point cloud to obtain the fine pose of the carriage and the region of interest of the carriage. Within the region of interest in the carriage, the point cloud is divided into static background points and dynamic candidate points, and adaptive clustering is performed on the dynamic candidate points to obtain the target instance point cloud clusters; S3: Obtain all candidate key points for each target instance point cloud cluster, filter candidate key points based on repeatability score to obtain a virtual self-calibration point set; generate a target point candidate set based on the virtual self-calibration point set, estimate the observation uncertainty covariance for each candidate target point, substitute it into the objective function, and solve to obtain the dynamic loading target point; S4: Based on the set of virtual self-calibrated points and the dynamic loading target points, the confidence level of the target points is calculated, and the dynamic loading target points represented by the car body coordinate system are used to form the dynamic loading target point positioning result.

2. The method for dynamic vehicle loading target point localization based on lidar point cloud analysis according to claim 1, characterized in that, In S1, the methods for obtaining the coarse pose of the carriage and the candidate range of the carriage area are as follows: Based on the set of geometric features, a carriage structure signature is constructed. The carriage structure signature is a feature vector composed of the set of angles between plane normal vectors, the relative distance between key planes, and the effective length of the plane intersection line. After coarse-grained metric and fine-grained metric, the feature vector is concatenated in a preset order to form a hash key. Input the hash key into the carriage template hash library, and first use coarse-grained keys to perform preliminary matching to obtain a set of candidate carriage templates; The candidate template set is filtered using fine-grained keys. During the filtering process, the templates are selected based on planar residuals and intersection alignment errors to obtain the coarse pose of the carriage and the candidate range of the carriage area.

3. The method for dynamic vehicle loading target point localization based on lidar point cloud analysis according to claim 1, characterized in that, In S2, the method for obtaining the precise pose of the carriage and the region of interest of the carriage is as follows: Within the candidate area of ​​the carriage region, the coarse pose of the carriage is initialized to obtain the initial pose parameters for registration; at the same time, point clouds that are not within the candidate area of ​​the carriage region are removed from the fused point cloud to obtain the target subset of the fused point cloud. The target subset of the fused point cloud is processed by point weight calculation to obtain a weighted target point cloud; the weighted target point cloud and the carriage template point cloud are then processed by the weighted point-to-plane error least squares method for registration based on the initial pose parameters to obtain the optimal registration transformation matrix. The optimal registration transformation matrix is ​​processed by pose analysis to obtain the precise pose of the carriage; the weighted point cloud and the initial region of interest are processed by region filtering to obtain the corrected candidate region of interest; the corrected candidate region of interest is then processed by boundary fitting to obtain the region of interest of the carriage.

4. The dynamic vehicle loading target point localization method based on lidar point cloud analysis according to claim 3, characterized in that, Point weights are derived from temporal stability weights, intensity consistency weights, and geometric consistency weights. The temporal stability weight is obtained based on the number of times the voxel of the point cloud recurs within the sliding window; the intensity consistency weight is obtained based on the point intensity and the intensity of the local neighborhood; and the geometric consistency weight is obtained based on the distance residual and the included angle residual between the point cloud and the plane of the carriage structure.

5. The method for dynamic vehicle loading target point localization based on lidar point cloud analysis according to claim 1, characterized in that, In S2, the method for dividing static background points and dynamic candidate points is as follows: Within the region of interest of the carriage, a voxel mesh is established, and for each voxel, the residual variance from the points within the voxel to the carriage structural plane and the number of times the voxel recurs within the sliding window are calculated. If the number of voxel recursors is greater than the recursor threshold and the residual variance is less than the residual variance threshold, it is determined to be a static background point, and the remaining point cloud is a dynamic candidate point.

6. The method for dynamic vehicle loading target point localization based on lidar point cloud analysis according to claim 1, characterized in that, In S3, the repeatability score is obtained based on the number of repeated observations of candidate keypoints within the sliding window, the variance of repeated observation locations, the variance of intensity, and the variance of the normal vector.

7. In the dynamic loading target point localization method based on lidar point cloud analysis according to claim 1, in S3, the objective function is constructed based on the observation uncertainty covariance, the visibility of candidate target points, and the safety of candidate target points.

8. A dynamic loading target point localization system based on lidar point cloud analysis, used to implement the dynamic loading target point localization method based on lidar point cloud analysis as described in claim 1, characterized in that, include: The module for generating the coarse pose of the carriage and the candidate range of the region is used to acquire multi-frame point cloud data collected by the lidar at continuous time. After distortion removal and intensity normalization, a sliding window fusion point cloud of length K is constructed to extract the geometric feature set of the carriage structure. The geometric feature set includes at least three of the following: the floor plane, the left and right side wall planes, and the front wall plane. Based on the geometric feature set, a car structure signature is constructed, and a hash key is generated through quantization encoding. The coarse pose of the car and the candidate range of the car region are retrieved from the car template hash library. The target instance point cloud cluster generation module is used to initialize the coarse pose of the carriage, register the fused point cloud and the carriage template point cloud within the candidate range of the carriage region to obtain the fine pose of the carriage and the region of interest of the carriage; within the region of interest of the carriage, the point cloud is divided into static background points and dynamic candidate points, and adaptive clustering is performed on the dynamic candidate points to obtain the target instance point cloud cluster. The dynamic loading target point generation module is used to obtain all candidate key points of each target instance point cloud cluster, filter candidate key points based on repeatability score to obtain a set of virtual self-calibrated points; based on the set of virtual self-calibrated points, generate a set of target point candidates, estimate the observation uncertainty covariance for each candidate target point, substitute it into the objective function, and solve to obtain the dynamic loading target point; The dynamic loading target point positioning result generation module is used to calculate the target point confidence based on the set of virtual self-calibrated points and the dynamic loading target point, and form the dynamic loading target point positioning result by combining it with the dynamic loading target point represented by the car body coordinate system.

9. A dynamic vehicle loading target point positioning device based on lidar point cloud analysis, characterized in that, It includes a processor and a memory, wherein when the processor executes a computer program stored in the memory, it implements the dynamic vehicle loading target point localization method based on lidar point cloud analysis as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the dynamic loading target point localization method based on lidar point cloud analysis as described in any one of claims 1-7.