Container truck guiding and auxiliary box grabbing and placing method and system based on laser radar

Abstract

The invention relates to the field of port container loading and unloading operation auxiliary control, and discloses a container truck guiding and auxiliary container grabbing and releasing method and system based on a laser radar. The method comprises the following steps: carrying out real-time three-dimensional perception on a container truck operation area below a rail-mounted gantry crane to obtain space position and attitude information of a container truck and a container carried by the container truck; dynamic calculation is carried out according to transverse, longitudinal and directional deviations between the current position of the vehicle and the target box grabbing and releasing position of the lifting appliance, alignment adjustment guide information is generated and fed back to a driver or an automatic driving system in real time through field display equipment; container posture changes are continuously tracked in the lifting appliance descending process, deviation compensation judgment is carried out by fusing historical information under the shielding condition, and container grabbing and releasing operation condition prompts are provided for a control end. According to the invention, rapid and accurate vehicle parking positioning and box grabbing and releasing auxiliary control can be realized, repeated adjustment is reduced, the loading and unloading efficiency is improved, the collision risk is reduced, and the method is suitable for intelligent and automatic port loading and unloading operation application.

Description

Technical Field

[0001] This invention belongs to the field of intelligent positioning and collaborative control technology for port container loading and unloading operations, and particularly relates to a method and system for guiding and assisting in the handling of containers based on lidar. Background Technology

[0002] As a core infrastructure in the global logistics and transportation system, container ports directly impact ship turnaround times, port throughput capacity, and the stability of cross-regional supply chains through their loading and unloading efficiency and operational safety. During the collaborative operation of rail-mounted gantry cranes and container trucks, the trucks need to be precisely positioned under the spreader to ensure the smooth handling or placement of containers.

[0003] In traditional operations, truck drivers typically position their vehicles based on hand gestures, verbal commands, or single-view camera footage from on-site supervisors. Due to factors such as obstructed visibility, difficulty in judging distances, and the fact that the position of the trailer lock is not visible to the naked eye, positional deviations between the vehicle, container, and spreader are often difficult to eliminate in one go. Repeated forward and reverse maneuvers and fine-tuning of direction are required to achieve alignment, resulting in excessively long handling times per container and a significant cumulative loss of loading and unloading efficiency. Furthermore, the port operating environment is complex and variable. Insufficient lighting at night, low visibility in rain or fog, and interference from dust and moisture at the dock can easily reduce human judgment capabilities. Narrow spaces alongside ships and obstructions caused by spreader descent make it even more difficult to judge the relative position of containers and vehicles in real time. Some drivers struggle to accurately detect changes in container posture, often leading to accumulated deviations and misjudgments. This results in spreader collisions, limit alarms, and failed container handling, impacting efficiency and posing a risk of equipment damage.

[0004] To ensure operational safety, many ports still require on-site supervisors to assist in assessing positional deviations and guiding truck movement. However, this method not only suffers from inconsistent operational standards and is susceptible to differences in personnel experience, but also exposes personnel to areas with densely packed large equipment for extended periods, posing potential safety hazards. Furthermore, with the accelerated development of port automation and unmanned operations, the existing manual guidance methods are clearly insufficient to meet the technological demands of intelligent equipment collaboration. Even more critically, when handling external truck operations, the extremely small tolerance for the alignment of the pallet locks, and the fact that the locks are hidden at the bottom of the containers, often prevents drivers from directly observing and assessing the alignment, making it heavily reliant on supervisors for confirmation. This results in external truck alignment becoming a bottleneck for port efficiency. With continuously increasing throughput, repeated adjustments to the existing technological model will directly impact terminal throughput capacity and overall operating costs. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a method and system for guiding and assisting in the placement and retrieval of containers using lidar. This system can operate stably in complex environments, provide accurate information on the positional relationship between vehicles and containers, and reliably assist and guide the alignment process. It significantly reduces reliance on manual labor, improves overall operational efficiency, and lowers safety risks.

[0006] Specifically, the technical solution provided by this invention is as follows: A method for guiding and assisting in the placement and retrieval of containers using a lidar-based truck includes the following steps: S1. Real-time data collection of the working area below the spreader to obtain corresponding laser point cloud data; S2. Process and analyze the point cloud data to identify and obtain the pose information of the truck and container; S3. Calculate the truck alignment deviation information based on the deviation between the truck's position information and the preset parking space. S4. Generate a position adjustment instruction based on the truck alignment deviation information to guide the driver to complete the truck alignment and parking; S5. During the process of lowering the spreader to grab the container, the position of the truck or spreader is finely adjusted based on the real-time position of the container.

[0007] Further, step S2 includes ground plane fitting to obtain a set of foreground target point clouds within the width range of the truck lanes above the ground; the ground plane fitting is based on a random sample consensus algorithm and includes the following steps: First, in the acquired point cloud data, the data located below the LiDAR and adjacent to the LiDAR installation location will be analyzed. All point clouds whose vertical distances meet the set threshold range are considered as candidate ground point clouds. S :

[0008] in, The installation height of the lidar. For high fluctuation deviation, z i For the point cloud data i The Z-axis coordinates of each point in the spatial coordinate system; Then, each time, a candidate set of ground point clouds is randomly selected. S Choose three points p 1. p 2. p 3. Construct the plane normal vector: , in Let be the plane normal vector obtained by the cross product of two vectors, used to represent the orientation information of the fitting plane in three-dimensional space. Then, the equation of the fitting plane is derived as follows: ; Next, the candidate set of ground point clouds was analyzed. S any point in Calculate its distance to the fitted plane. ,like If the number of points is less than a set threshold, the point belongs to the set of ground inliers. Through iteration, the fitted plane with the largest number of points in the set of ground inliers is selected as the candidate plane. Finally, the set of ground inliers corresponding to the candidate plane is fitted with a plane based on the least squares method to obtain the final ground fitted plane.

[0009] Furthermore, step S2 also includes clustering and segmenting the foreground target point cloud set to obtain several target point cloud clusters, identifying the actual category of each target point cloud cluster based on the size information of the truck and container, and then obtaining the corresponding pose information, including position and orientation, based on the three-dimensional bounding box of the target point cloud cluster corresponding to the truck and container. The clustering segmentation employs a dynamic threshold-based clustering algorithm, including the following steps: S201. Selection of initial seed points for clustering For each point in the foreground target point cloud set Obtain its radius as r neighborhood point set ; for All points in, with Construct a covariance matrix around the center :

[0010] in, The outer product of the covariance terms. for Center of gravity: , express The number of points, express Points in j ; For covariance matrix Eigenvalue decomposition yields three eigenvalues. and calculate the points Local curvature ;like If the curvature is less than the set curvature threshold, it will be used as the initial seed point for clustering; S202, Neighborhood Search and Dynamic Threshold Setting Calculate seed point Local density , , Seed point The set of neighborhood points, express The number of points, Let be the volume of the neighborhood point set; Set dynamic distance threshold and dynamic normal angle threshold : ,

[0011] in, d 0 is the base distance threshold. α This is the adjustment coefficient used to control sensitivity. Based on the threshold of the included normal angle, This is an adjustment coefficient used to control the degree of diagonal constraint relaxation; S203, Cluster Growth and Termination for Candidate neighborhood points ,like and If a point is not found in the current cluster, it is added to the current cluster and used as a seed point for the next round of region growing. This process continues iteratively until no further growth is possible. The current cluster is then recorded, and the next unclustered point is processed, ultimately resulting in several target point cloud clusters. and Points and The normal vector of each point is equal to the direction of the eigenvector corresponding to the smallest eigenvalue of the covariance matrix of their respective neighborhood point sets.

[0012] Furthermore, the truck alignment deviation information mentioned in step S3 includes lateral deviation, longitudinal deviation, and directional deviation: Let the current pose of the container truck in the ground coordinate system be... The preset parking space position is ,in, and These are the coordinates of the truck's center along the X and Y axes, respectively. Let X be the truck's orientation angle, i.e., the angle between the vehicle's main shaft and the X-axis. The preset parking position represents the ideal position the truck should be in before the spreader grabs the container; then: lateral deviation This indicates the projected distance between the truck's current position and the preset parking space in the direction perpendicular to the vehicle's orientation, reflecting whether the truck has deviated from the centerline; Longitudinal deviation This indicates the positional offset of the truck's current position from the preset parking space in the directional direction, determining whether forward or backward compensation is needed. Directional deviation This indicates the angle between the truck's current orientation and the preset parking space orientation, reflecting whether the direction needs to be rotated for correction.

[0013] Furthermore, in step S4: like , If a lateral deviation tolerance is set, a pose adjustment instruction is generated to the left or right. like , If a set longitudinal deviation tolerance is specified, a pose adjustment instruction is generated to adjust forward or backward. like , If the set orientation deviation tolerance is specified, a pose adjustment instruction to turn left or right is generated. The posture adjustment instruction or alignment deviation information is displayed to the truck driver through the lane-side large screen or directly pushed to the truck's on-board display module; the on-board display module includes: displaying the preset parking space as a rectangular parking space and displaying the truck as a rectangular area relative to the road surface, and displaying the relative position of the truck and the preset parking space to the driver in real time through video.

[0014] Furthermore, in step S5, to address the partial occlusion of the container point cloud caused by the lowering of the spreader, the current pose information of the container is obtained based on the historical complete point cloud data and the current partial point cloud data, including the following steps: Before the spreader is lowered, the historical complete point cloud data of the container is stored. As the spreader is lowered and the point cloud is obscured, the current local point cloud data of the container is acquired in real time. The current local point cloud data is registered with the historical complete point cloud data using the least mean square error method, and the rigid transformation matrix between the current local point cloud data and the historical complete point cloud data is calculated. Using the rigid transformation matrix, the container pose corresponding to the current local point cloud data is obtained by mapping the container pose corresponding to the historical complete point cloud data.

[0015] Preferably, the rigid transformation matrix is ​​calculated using the ICP algorithm, including the rotation matrix. R Translation vector t :

[0016] in, For historical complete point cloud data and current local point cloud data The points are registered, where m is the number of point pairs registered; for the container center coordinates corresponding to the complete historical point cloud data. The container center coordinates obtained by mapping the current local point cloud data are represented as follows: ; Then, by calculating the real-time relative positional deviation between the spreader and the container or the container lifting hole, an adjustment command is generated to adjust the position of the spreader or the container-mounted truck, thereby achieving alignment and locking of the spreader lock head and the container lifting hole.

[0017] A container truck guidance and auxiliary container placement system based on the above method includes the following modules: The data acquisition and preprocessing module is used to scan the working area below the spreader in real time using a lidar sensor to obtain corresponding point cloud data, and to perform noise reduction processing and ground plane fitting on the point cloud data to obtain a set of foreground target point clouds within the width range of the truck lane above the ground. The point cloud segmentation and target recognition module is used to cluster and segment the foreground target point cloud set to obtain several target point cloud clusters, and to identify the actual category of each target point cloud cluster according to the size information of the truck and container. Then, it obtains the corresponding pose information of the target point cloud clusters corresponding to the truck and container based on the three-dimensional bounding box of the target point cloud cluster. The truck alignment guidance control module is used to calculate the alignment deviation between the truck and the preset parking space based on the real-time position and posture information of the truck, and to feed back the alignment deviation information or the position and posture adjustment instructions generated based on the alignment deviation information to the truck driver in real time, so as to guide the driver to complete the alignment and parking of the truck. The container grabbing and positioning auxiliary module is used to calculate the relative posture deviation between the spreader and the container based on their real-time posture during the process of lowering the spreader to grab the container. This deviation is then fed back to the spreader control system or the truck in real time to guide the spreader or the container to adjust its posture on the truck, thereby achieving alignment and locking of the spreader lock head with the container lifting hole.

[0018] Preferably, the truck alignment guidance control module also provides a standard communication protocol interface with the port automation system or the truck automatic driving system, which is used to send the calculated alignment deviation information or direct control commands to the control module of the port automation system or the truck automatic driving system to realize the automatic alignment and docking operation of the truck.

[0019] A computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the truck guidance and assisted loading / unloading method as described above.

[0020] Compared with the prior art, the present invention has at least the following beneficial effects: This invention can acquire accurate vehicle position and container attitude information in real time during truck parking and rail-mounted gantry crane container handling, and continuously evaluate alignment deviations. It assists drivers or automatic control systems in completing parking and handling operations through an intuitive and reliable guidance method, significantly reducing the time consumed by repeated vehicle adjustments. Compared to the traditional manual command-based model, it greatly improves single-container handling efficiency and the overall port throughput capacity. Furthermore, this invention maintains stable positioning and recognition performance even in complex environments, effectively overcoming alignment interference caused by low light conditions at night, rain, fog, and spreader obstruction, giving loading and unloading operations stronger environmental adaptability and all-weather working characteristics.

[0021] By accurately grasping the relative spatial relationship between vehicles and containers, this invention can reduce the risk of collisions during container handling and minimize the occurrence of spreader limit alarms and container handling failures due to misjudgments, thus providing a reliable guarantee for improving port loading and unloading safety. Simultaneously, this invention shifts from manual experience-based judgment to data-driven decision-making, enabling drivers of varying skill levels to receive consistent operational guidance, helping to reduce labor intensity and shorten training cycles. With the accelerating trend towards port automation and unmanned operations, this invention can also serve as a core perception and guidance technology, efficiently collaborating with autonomous driving truck systems and intelligent spreader control systems, laying the foundation for achieving fully intelligent loading and unloading processes. Attached Figure Description

[0022] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0023] Figure 1 This is a schematic diagram of a truck guidance and assisted container placement method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a truck guidance and auxiliary grabbing and placing system module provided in an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, other embodiments obtained by those skilled in the art without creative effort are all within the scope of protection of the present invention.

[0025] Example 1 This embodiment provides a method for guiding and assisting in the placement and berthing of container trucks based on lidar. It achieves container truck docking guidance and auxiliary control for container placement and berthing through real-time sensing and deviation calculation. Figure 1As shown, this method mainly includes the following steps: I. Data Acquisition and Preprocessing To obtain effective spatial information for guiding the container truck and assisting in the placement and removal of the grabbing box, a high-performance lidar sensor is first used to scan the working area below the rail-mounted gantry crane. The system actively emits laser beams and receives return signals, generating raw 3D point cloud data covering the entire working lane in real time. Point cloud acquisition is performed at a preset scanning frequency (e.g., 10Hz), ensuring that each acquisition moment corresponds to a complete scan result, thus achieving dynamic perception of the target object.

[0026] After point cloud data acquisition, filtering is performed to address outliers, environmental clutter, and noise interference caused by rain, fog, floating objects, and light reflection that often appear in the original data. Specifically, this includes determining whether isolated noise points deviate from the normal structure range based on spatial distribution density and removing them accordingly. Simultaneously, statistical filtering is used to smooth local point cloud regions, improving the stability and accuracy of subsequent analysis. The processed point cloud effectively reduces interference from non-real object signals on feature identification, ensuring a more robust and reliable recognition process.

[0027] To facilitate the extraction of effective information from container trucks and containers and improve computational efficiency, a ground extraction operation is further performed. By selecting height distribution features from the point cloud for planar fitting, the location of the port area's road surface is estimated. Using the fitted plane as a boundary, point clouds within this plane or below a set height threshold are classified as ground point clouds, while point clouds above the threshold and located within the lanes are considered foreground target point clouds. The foreground target point cloud contains operationally relevant physical objects; its structure is clear and its features are prominent, making it suitable for subsequent target object identification and pose calculation.

[0028] In this embodiment, the lidar is fixedly mounted on the support column of the rail-mounted gantry. Its spatial position is known, and after calibration, its three-dimensional installation coordinates in the global coordinate system can be obtained. It also includes the scanning attitude parameters (such as pitch, yaw, and roll angles). This installation information allows the acquired point cloud data to be directly converted to a unified reference coordinate system for ground identification and structure segmentation.

[0029] To quickly identify the plane on which the ground lies, the Random Sample Consensus (RANSAC) algorithm is used for plane fitting.

[0030] First, in the collected point cloud data, the data located below the lidar and at a vertical distance from the lidar's installation position is... H All point clouds near 0 are considered as candidate ground point clouds. S ,Right now:

[0031] in, The installation height of the lidar. For high fluctuation deviation, z i Indicates the first i Vertical distance coordinates of point cloud data ( z (axis coordinates).

[0032] A plane is determined based on three non-collinear points, and each iteration starts from the candidate set of ground point clouds. S Three points are randomly selected from the middle. p 1. p 2. p 3. Construct the plane normal vector ,in That is, the plane normal vector obtained by the cross product of two vectors represents the directional information of the fitted plane in three-dimensional space, and thus the plane equation is derived as follows: Candidate set of ground point clouds S any point in Calculate its distance to the fitted plane. ,like If the number of points is less than a set threshold, the point belongs to the set of ground inliers. Through iteration, the fitted plane with the largest number of points in the set of ground inliers is selected as the candidate plane, and the set of ground inliers corresponding to the candidate plane is fitted using the least squares method to obtain the final fitted ground plane.

[0033] Assume the plane equation of the fitted plane of the ground is: In order to determine the parameters a , b , c For all points in the set of points on the ground Solve the following minimization problem:

[0034] in N This represents the number of point clouds in the set of points on the ground. The optimal fitting plane parameters can be obtained by solving a linear least squares problem using either a system of normal equations or singular value decomposition. a , b , c .

[0035] Further considering lane boundary information, the foreground target point should be located below the LiDAR, above the ground, and within the width of the left and right lanes. Let the lane centerline be the projection of the LiDAR onto the x-axis, and the total lane width be... W Then, non-ground points that meet the following conditions will be included in the foreground target point cloud set. P fg :

[0036] in, It is the minimum effective foreground height, usually set to 0.1~0.2m, to exclude ground-level clutter.

[0037] II. Truck and Container Identification After extracting the foreground target point cloud, the next stage is the identification and pose calculation of the trucks and containers. The core objective of this stage is to accurately extract the spatial distribution of the trucks and their containers from the 3D point cloud, and obtain their position and orientation information, providing a precise geometric basis for subsequent guidance of truck alignment and assisted container placement.

[0038] In real-world laser point cloud analysis, due to differences in material reflectivity between trucks and containers, as well as near-far occlusion and angular distortion, there are significant differences in point cloud density and boundary sharpness between different targets. Traditional fixed-threshold clustering methods struggle to guarantee boundary connectivity when density distribution is uneven. Therefore, this embodiment introduces a dynamic threshold function during region growth, allowing the connectivity conditions for cluster growth to adaptively adjust with point cloud position and density, thereby improving the robustness and integrity of clustering.

[0039] 1. Selection of initial seed points for clustering For each point Within a certain range (radius) r Estimating normal vectors and local geometry within a spherical neighborhood (the point's natural vector and local geometry). Using a KD-Tree to construct an efficient spatial index, [the index is used for point...]. The search radius is r The set of neighborhood points:

[0040] for N i All points in, with Construct a covariance matrix around the center :

[0041] in, The outer product of the covariance terms. The centroid of the neighboring points: .

[0042] For covariance matrix Eigenvalue decomposition yields three eigenvalues. And the corresponding eigenvectors. Normal vector Take the smallest eigenvalue The corresponding eigenvector direction.

[0043] Based on the three sets of extracted feature values, calculate the point. Local curvature This index measures the flatness of a point; a local curvature closer to 0 indicates a flatter surface, while a curvature closer to 1 indicates more drastic changes (such as edges or sharp corners). In this embodiment, an empirical curvature threshold is set. (e.g., 0.08), when When the point is in a relatively flat region, it can be used as the initial seed point for subsequent clustering.

[0044] 2. Neighborhood search and dynamic threshold determination To achieve dynamic thresholding, seed points need to be estimated. Local density The larger the value, the denser the point cloud in that region. A common method is to use the number of points per unit sphere:

[0045] in, , This represents the number of points within the neighborhood. .

[0046] For candidate neighborhood points Determine whether it satisfies the two constraints: Distance constraints: The dynamic distance threshold is:

[0047] d 0 is the base distance threshold (e.g., 0.2m). α This is an adjustment coefficient used to control sensitivity (e.g., 0.5~1.5). Continuity of the included normal angle: The dynamic normal angle threshold is:

[0048] The basic normal angle threshold (e.g., 10~20°). This is an adjustment coefficient used to control the degree of diagonal constraint relaxation. and Points and The normal vector. The smaller the point density, the larger the allowed angle, avoiding the accidental exclusion of points with large angle errors due to sparse sampling.

[0049] 3. Cluster growth and termination conditions Whenever a neighboring point satisfies the above connectivity condition, it is added to the current cluster and used as the seed for the next round of expansion. That is: for seed points... any neighborhood point If the following conditions are met simultaneously: and

[0050] If a point is identified as a target point of the same type, it is added to the current cluster and continues to serve as a seed point for the next round of region growth. This process iteratively expands until further growth is impossible. Record the current cluster. Then proceed with the processing of the next unclustered points.

[0051] The final target point cloud cluster set is formed: Each point cloud cluster corresponds to an independent object candidate region (such as a truck, container, etc.).

[0052] 4. Truck and container identification After completing the point cloud clustering, the structural features of each target point cloud cluster are further extracted in order to accurately determine the type of the corresponding object, such as a truck, a container, or a lowered lifting device.

[0053] Within each point cloud cluster, dimensions are estimated based on its three-dimensional bounding box, including the span in the X, Y, and Z directions, to obtain its length, width, and height. These dimensional parameters serve as the basis for identification. In typical port operation environments, container trucks typically exhibit a long, narrow structure with a flat chassis and a low center of gravity, and their length, width, and height ratios are relatively stable. Standard containers, on the other hand, have a regular cuboid shape (e.g., approximately 6.1 × 2.44 × 2.6 m for 20 feet and approximately 12.2 × 2.44 × 2.6 m for 40 feet), with flat top edges, structural protrusions at the corners, and length, width, and height conforming to international standards. The spreader attachments also have corresponding dimensions.

[0054] III. Truck docking guidance and alignment correction Based on the spatial difference between the current position of the truck and the preset parking space, the lateral, longitudinal and directional angular deviations are calculated, and the deviation information is fed back to the truck driver in real time to guide the truck to complete the alignment and parking.

[0055] Assume that the current pose of the container truck in the ground coordinate system is: , and These are the coordinates of the truck's center along the X and Y axes, respectively. The truck's orientation angle is the angle between the vehicle's main shaft and the X-axis; the preset parking space's pose is represented as... This refers to the ideal target state that the container truck must be in before the spreader grabs the container.

[0056] The alignment deviation includes: lateral deviation This indicates the projected distance between the truck's current position and the preset parking space in the direction perpendicular to the vehicle's orientation, reflecting whether the truck has deviated from the centerline; Longitudinal deviation This indicates the positional offset of the truck's current position from the preset parking space in the directional direction, determining whether forward or backward compensation is needed. Directional deviation This indicates the angle between the truck's orientation and the preset parking space's orientation, reflecting whether the direction needs to be rotated for correction.

[0057] Then, shunting instructions are generated based on the magnitude and direction of the current deviation. For example, if (If the lateral deviation exceeds the set tolerance), a guide command to adjust to the left or right is generated; if Then guide a slight turn; if This will prompt you to move forward or backward.

[0058] The system acquires the real-time position and orientation status of the container truck and its alignment deviation to guide the driver in adjusting the truck's position until the alignment deviation is within acceptable limits, at which point the truck is considered properly positioned and parked. In some embodiments, the preset optimal parking position is represented as a rectangular parking space, and the container truck is also represented as a rectangular area relative to the road surface. The relative position of the container truck and the optimal parking space is displayed to the driver in real-time via video, facilitating the driver's adjustment of the truck's position.

[0059] IV. Grab and Drop Box-Assisted Identification and Grab Alignment After the truck has docked, the spreader is slowly lowered. When the spreader is lowered close to the container, the relative positional deviation between the spreader lock head and the container lifting hole is obtained in real time through the point cloud clustering and target recognition methods mentioned above. Based on this, control instructions are given to adjust the position of the spreader or container-mounted truck to achieve alignment and locking of the lock head and the lifting hole.

[0060] To ensure high-precision alignment between the spreader and the container, this embodiment continuously monitors the container's attitude changes during the spreader's descent. However, in actual operation, as the spreader descends closer to the container's top surface, the point cloud data acquired by the lidar often becomes incomplete due to spreader obstruction, making conventional attitude calculations inaccurate. To address this issue, this embodiment proposes an attitude compensation method that integrates historical point clouds and locally visible point clouds.

[0061] Before the spreader begins to lower the container, record the completed container point cloud data, and denote it as follows. This serves as a reference benchmark. Each point in the point cloud... This represents a surface feature in space. As the lifting device descends and causes point cloud occlusion, the current point cloud data acquired in real time is recorded as follows. ,in This reflects the incomplete data due to partial obstruction.

[0062] To achieve compensation in attitude calculation, the following strategy is adopted: First, view the current point cloud. Using the least mean square error method and historical point clouds Registration is performed on the corresponding regions to obtain the pose estimate of the current container. The rigid transformation matrix is ​​calculated using a typical ICP (Iterative Closest Point) iterative algorithm. , so that:

[0063] For rotation matrix, It is a translation vector; Pointing out the history Center and current point The reference point for the best match.

[0064] Then, based on historical point clouds The corresponding container pose mapping yields the partially occluded current point cloud. The corresponding container pose, for example, historical point cloud. The corresponding container center coordinates are Then the coordinates of the container corresponding to the current point cloud are .

[0065] Then, by calculating the real-time relative positional deviation between the spreader and the container or the container lifting hole, an adjustment command is generated to adjust the position of the spreader or the container-mounted truck, thereby achieving alignment and locking of the spreader lock head and the lifting hole.

[0066] Example 2 Based on the above method, this embodiment provides a lidar-based truck guidance and assisted container placement system, such as... Figure 2 As shown, the system mainly consists of a lidar sensor, a data acquisition and preprocessing module, a point cloud segmentation and target recognition module, a truck alignment and guidance control module, and a grabbing and releasing box auxiliary positioning module.

[0067] The lidar sensor is fixedly installed at a specific height and angle on the rail-mounted gantry crane support, and its three-dimensional installation pose has been obtained through calibration. In this embodiment, two high-performance lidar sensors are installed in the middle of the trolley frame crossbeam of the rail-mounted gantry crane. The two sensors are deployed at a certain angle to form a complete coverage of the entire truck operating lane below without blind spots.

[0068] The data acquisition and preprocessing module first scans the work area in real time using deployed LiDAR sensors to generate high-density point cloud data. For multiple LiDAR sensors, a point cloud stitching and fusion algorithm is used to obtain complete point cloud data of the work area. Then, the point cloud data is preprocessed, including noise filtering and ground fitting. By setting a threshold, foreground target point clouds that are above the ground and within the lane width are identified for subsequent target recognition.

[0069] The point cloud segmentation and target recognition module utilizes a region-growing clustering method based on dynamic thresholds to segment the foreground target point cloud. By analyzing the geometric features of the point cloud, such as curvature, density, and normal angle, the module adaptively controls the clustering connectivity conditions to achieve stable segmentation of targets such as trucks, containers, and spreaders. For each segmented target point cloud cluster, the size is estimated based on its 3D bounding box. Then, combined with the actual dimensions of the targets such as trucks, containers, and spreaders, the module determines the type of point cloud target and estimates its pose.

[0070] The truck positioning guidance control module calculates the spatial deviation between the truck and the preset parking position in real time based on the truck's pose obtained by the point cloud segmentation and recognition module. This deviation includes lateral, longitudinal, and angular deviations. The module then feeds this deviation information back to the truck driver in real time to guide them in positioning the truck. In this embodiment, a large display screen (e.g., 1.5 × 1.0 m) is prominently installed on the side facing the truck lane to issue driving guidance instructions to the truck driver in real time. These instructions are generated based on the deviation information, such as "Please proceed 0.25 meters" or "Please turn right 8 degrees." Alternatively, the driving guidance instructions or positioning deviation information can be directly pushed to the truck's onboard display module for the driver to observe and obtain relevant information. For example, the preset parking position can be represented as a rectangular parking space, and the truck can be represented as a rectangular area relative to the road surface. The relative position of the truck and the optimal parking space can be displayed to the driver in real time via video, facilitating the driver's adjustment of the truck's position. In some embodiments, the module also provides a standard communication protocol interface with a port automation system or a truck autopilot system, which can send the calculated precise position and attitude data or direct control commands to the PLC to achieve automatic alignment operation.

[0071] The container grabbing and positioning auxiliary module provides real-time feedback on the relative positional deviation between the spreader and the container during the container grabbing and lowering process, assisting in the alignment and locking of the spreader's locking head with the container's lifting hole. During the spreader lowering phase, when the spreader approaches the top of the container, the lack of a point cloud due to obstruction makes attitude calculation difficult. To address this issue, the module saves a complete container point cloud as a reference historical point cloud before the spreader descends. As the spreader gradually obstructs the view, it uses the ICP registration results between the currently visible point cloud and the historical point cloud to calculate the rigid transformation matrix and estimate the container's pose under the current obstruction. This pose information is then compared with the spreader's current pose to generate a grabbing deviation, which is fed back to the spreader control system or truck in real time, guiding the control equipment to make precise adjustments and achieve automatic alignment of the locking head with the lifting hole.

[0072] The above system can execute the lidar-based truck guidance and assisted box-grabbing method described in Embodiment 1, and has the corresponding functional modules and beneficial effects of the method. For technical details not described in detail in this embodiment, please refer to the lidar-based truck guidance and assisted box-grabbing method provided in Embodiment 1 of this invention.

[0073] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; under the concept of the present invention, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the present invention as described above, which are not provided in detail for the sake of brevity; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for truck guiding and assisting picking and placing a box based on a laser radar, characterized by, The method comprises the steps of: S1, real-time data acquisition is performed on the working area below the spreader to obtain corresponding laser point cloud data; S2, the point cloud data is processed and analyzed to identify and obtain the pose information of the container truck and the container; S3, according to the deviation between the pose information of the container truck and the preset parking space, the deviation information of the container truck is calculated and obtained; S4, the pose adjustment instruction is generated according to the deviation information of the container truck to guide the driver to complete the parking of the container truck; S5, during the process of lowering the spreader to grab the container, the pose of the container truck or the spreader is fine-tuned based on the real-time pose of the container.

2. The method of claim 1, wherein, In step S2, ground plane fitting is included to obtain a foreground target point cloud set within the width of the container truck lane above the ground; the ground plane fitting is based on a random sample consensus algorithm, and comprises the steps of: First, in the acquired point cloud data, the data located below the LiDAR and adjacent to the LiDAR installation location will be analyzed. All point clouds whose vertical distances meet the set threshold range are considered as candidate ground point clouds. S : in, The installation height of the lidar. For high fluctuation deviation, z i For the point cloud data i The Z-axis coordinates of each point in the spatial coordinate system; Then, three points are randomly selected from the ground point cloud candidate set S at each time p 1、 p 2、 p 3, a plane normal vector is constructed: ， wherein is the plane normal vector obtained by the cross product of two vectors, which is used to represent the direction information of the fitting plane in three-dimensional space, and the equation of the fitting plane is derived as: ; Next, the candidate set of ground point clouds was analyzed. S any point in Calculate its distance to the fitted plane. ,like If the number of points is less than a set threshold, the point belongs to the set of ground inliers. Through iteration, the fitted plane with the largest number of points in the set of ground inliers is selected as the candidate plane. Finally, the set of ground inliers corresponding to the candidate plane is fitted with a plane based on the least squares method to obtain the final ground fitted plane.

3. The truck guidance and assisted container placement method as described in claim 2, characterized in that, In step S2, the foreground target point cloud set is also subjected to clustering segmentation to obtain a plurality of target point cloud clusters, and the actual categories of the target point cloud clusters are identified according to the size information of the container truck and the container, and then the corresponding pose information, including the position and the orientation, of the target point cloud clusters of the container truck and the container is obtained according to the three-dimensional bounding box thereof; The clustering segmentation adopts a clustering algorithm based on a dynamic threshold, comprising the steps of: S201, initial seed point selection for clustering For each point in the foreground target point cloud set , a neighborhood point set with a radius of r is obtained ; For all points in a covariance matrix is constructed centered at :​ wherein is the outer product of the covariance terms, is the barycenter of , denotes the number of points of denotes a point in j ; Eigenvalue decomposition is performed on the covariance matrix to obtain three eigenvalues , and the local curvature of the point is calculated; if is less than a set curvature threshold, it is taken as a clustering initial seed point; S202, neighborhood search and dynamic threshold setting Calculate seed point Local density , , Seed point The set of neighborhood points, express The number of points, Let be the volume of the neighborhood point set; Setting dynamic distance threshold and dynamic normal angle threshold : ， wherein, d 0 is a base distance threshold, α is an adjustment coefficient for controlling sensitivity, is a base normal angle threshold, is an adjustment coefficient for controlling the degree of relaxation of the diagonal constraint; S203, clustering growth and end For a candidate neighboring point , if and and , it is added to the current clustering cluster and serves as a seed point for the next round of region growing, and the iteration is grown until it cannot continue, the current clustering cluster is recorded, and the next unclustering point is processed, and finally a plurality of target point cloud clusters are obtained; wherein, and are normal vectors of points and , and they are equal to the eigenvector direction corresponding to the smallest eigenvalue of the covariance matrix of the respective neighboring point set.

4. The method of claim 1, wherein, In step S3, the deviation information of the container truck includes lateral deviation, longitudinal deviation and directional deviation: Let the current pose of the truck in the ground coordinate system be , and the pose of the preset parking position be , wherein, and are the coordinates of the center of the truck in the X-axis and Y-axis directions, is the truck orientation angle, i.e. the included angle between the vehicle main shaft and the X-axis, and the preset parking position represents the ideal pose of the truck before the spreaders grab the container; then, Lateral deviation , represents the projection distance of the current pose of the container truck and the preset parking position in the direction perpendicular to the vehicle orientation direction, reflecting whether the container truck deviates from the center line; longitudinal deviation , which represents the position deviation of the current pose of the container truck and the preset parking position in the orientation direction, determines whether compensation is needed by advancing or retreating; direction deviation , represents the included angle between the current orientation of the container freight station and the preset parking orientation, and reflects whether the direction needs to be corrected by rotation.

5. The method of claim 4, wherein, In step S4: If , is a set lateral deviation tolerance, then a pose adjustment indication to adjust left or right is generated; If , is a set longitudinal deviation tolerance, then a forward or backward adjusted pose adjustment indication is generated; If , is a set orientation deviation tolerance, then a pose adjustment indication is generated that deflects left or right; The pose adjustment instruction or the deviation information is displayed to the driver of the container truck through a large screen on the side of the lane, or is directly pushed to the vehicle-mounted display module of the container truck; The vehicle-mounted display module includes: displaying the preset parking space as a rectangular parking space, and displaying the container truck as a rectangular area relative to the road surface, and displaying the relative position of the container truck and the preset parking space to the driver in real time through a video image.

6. The method of claim 1, wherein, In step S5, in order to cope with the local occlusion of the container point cloud caused by the lowering of the spreader, the current pose information of the container is obtained based on the historical complete point cloud data and the current local point cloud data of the container, comprising the steps of: The historical complete point cloud data of the container is stored before the lowering of the spreader, and the current local point cloud data of the container is obtained in real time as the lowering of the spreader causes point cloud occlusion; The current local point cloud data is registered with the historical complete point cloud data by the least mean square error method, and a rigid transformation matrix between the current local point cloud data and the historical complete point cloud data is calculated; The container pose corresponding to the current local point cloud data is obtained by mapping the container pose corresponding to the historical complete point cloud data using the rigid transformation matrix.

7. The method of claim 6, wherein the step of guiding and assisting the gripping and releasing of the box is performed by the tractor trailer. The rigid transformation matrix is calculated by using an ICP algorithm, including a rotation matrix R and a translation vector t : wherein, is the point cloud data of the container center in the historical complete point cloud data and is the point cloud data of the container center in the current local point cloud data , m is the number of matched points, and the container center coordinate corresponding to the historical complete point cloud data is represented as Further, by calculating the real-time relative position deviation of the spreader and the container or the lifting hole of the container, an adjustment instruction is generated to adjust the position of the spreader or the container truck, so as to realize the alignment and locking of the spreader lock head and the lifting hole of the container.

8. A system for guiding and assisting a truck driver in picking up and dropping off a container based on the method of any one of claims 1 to 7, characterized in that, The method comprises the following modules: A data acquisition and preprocessing module is configured to perform real-time scanning on the working area below the spreader by a laser radar sensor to obtain corresponding point cloud data, and to perform noise reduction processing and ground plane fitting on the point cloud data to obtain a foreground target point cloud set within the width of the container truck lane above the ground; The point cloud segmentation and target recognition module is configured to cluster and segment the foreground target point cloud set to obtain a plurality of target point cloud clusters, and identify actual categories of the target point cloud clusters according to size information of the container trucks and containers, and then obtain corresponding pose information of the target point cloud clusters according to three-dimensional bounding boxes of the container trucks and containers; The container truck alignment guiding control module is configured to calculate alignment deviation between the container truck and a preset parking position according to the real-time pose information of the container truck, and feed back the alignment deviation information or pose adjustment instructions generated according to the alignment deviation information to the driver of the container truck in real time, so as to guide the driver to complete the alignment parking of the container truck. The grab-and-drop box auxiliary positioning module is configured to calculate relative pose deviation between the spreader and the container according to real-time poses of the spreader and the container during the process that the spreader grabs the container, and feed back the real-time relative pose deviation to the spreader control system or the container truck in real time, so as to guide the spreader or the container to be carried by the container truck to adjust the pose, and realize alignment and locking of the spreader lock head and the container lifting hole.

9. The tractor guiding and auxiliary grab-and-pull box system as claimed in claim 8, wherein, The container truck alignment guiding control module also provides a standard communication protocol interface with a port automation system or a container truck automatic driving system, and is configured to send the calculated alignment deviation information or direct control instructions to a control module of the port automation system or the container truck automatic driving system, so as to realize automatic alignment parking operation of the container truck.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the container truck guiding and auxiliary grab-and-drop box method according to any one of claims 1-7.

Images