Container positioning method and device based on multi-frame fusion, automated gantry crane and medium

Abstract

The present application relates to the technical field of truck loading, and discloses a truck positioning method and device based on multi-frame fusion, an automated yard crane and a medium, which are applied to an automated yard crane provided with a trolley, a spreader and a laser acquisition device, and the method comprises the following steps: acquiring multi-frame truck point cloud data of the truck during lateral movement of the trolley to the truck; performing fusion processing on the multi-frame truck point cloud data to construct a reference model point cloud of the truck; acquiring single-frame point cloud data of the truck in real time during downward movement of the trolley towards the truck; and performing point cloud matching on the single-frame point cloud data and the reference model point cloud to determine pose information of the truck relative to the trolley. The present application can effectively solve the problems of positioning failure and excessive deviation caused by the truck being blocked, single-frame point cloud defects, truck movement and environmental interference under the condition of container unloading, greatly improve the precision and stability of truck positioning in the dynamic operation scene of the automated yard crane, and significantly improve the container unloading alignment accuracy.

Description

Technical Field

[0001] This invention relates to the field of truck loading technology, specifically to a truck positioning method, device, automated field bridge, and medium based on multi-frame fusion. Background Technology

[0002] In automated terminal crane operations, precise truck positioning is the core foundation for automated container loading and unloading and accurate container placement. However, when the crane is lowering containers, the spreader and container are suspended above the truck, creating a large area of ​​obstruction. LiDAR can only collect a small amount of localized point cloud data, resulting in scarce effective features. At the same time, the point cloud acquisition process is easily affected by environmental factors such as lighting, rain, snow, and debris in the yard, making the point cloud data prone to noise and missing data.

[0003] Currently, automated yard cranes commonly use real-time matching of single-frame point clouds from LiDAR for truck positioning. However, this method is highly dependent on the accuracy of single-frame LiDAR data acquisition; even minor hardware errors can directly affect the positioning results, resulting in extremely low fault tolerance. Furthermore, relying solely on real-time single-frame point clouds for feature recognition and matching is highly unstable. Insufficient effective features and fluctuations in point cloud quality can cause matching deviations, pose calculation failures, and a significant decrease in positioning accuracy. Especially under conditions of minor truck movement, single-frame point clouds are prone to feature loss and misalignment, further exacerbating positioning drift and calculation errors. This can easily lead to alignment errors, container placement failures, and other malfunctions, ultimately failing to meet the high-precision, high-stability unmanned operation requirements of automated yard cranes in complex yard conditions. Summary of the Invention

[0004] This invention provides a truck positioning method, device, automated yard crane, and medium based on multi-frame fusion, to solve the problem that the real-time matching of single-frame point clouds by existing lidar is difficult to adapt to the high-precision and high-stability unmanned operation requirements of automated yard cranes under complex yard conditions.

[0005] In a first aspect, the present invention provides a truck positioning method based on multi-frame fusion, applied to an automated yard crane. The automated yard crane is equipped with a laterally movable crane trolley, which is equipped with a spreader and a laser acquisition device. The method includes: acquiring multi-frame point cloud data of the truck corresponding to the truck during the lateral movement of the crane trolley from the yard to above the truck; fusing the multi-frame point cloud data to construct a reference model point cloud of the truck; acquiring real-time point cloud data of the truck during the lowering of the spreader towards the truck by the crane trolley to obtain single-frame point cloud data; and matching the single-frame point cloud data with the reference model point cloud to determine the position and pose information of the truck relative to the crane trolley.

[0006] This invention provides a multi-frame fusion-based truck positioning method. By collecting multi-frame point clouds of the truck during the lateral movement of the crane trolley and fusing them to construct a complete reference model point cloud, the method then uses real-time single-frame point clouds to match the reference model point cloud during the spreader lowering operation to solve the pose problem. By distinguishing between the unobstructed modeling stage and the obstructed alignment stage, this invention utilizes multi-frame data collected during the movement process to construct a complete and reliable truck reference model. This effectively solves the problems of positioning failure and excessive deviation caused by truck obstruction, incomplete single-frame point clouds, truck movement, and environmental interference during container placement. It exhibits extremely high environmental robustness and positioning accuracy, significantly improving the accuracy and stability of truck positioning in dynamic automated yard crane operations, significantly increasing the accuracy of container placement and alignment, adapting to complex terminal operating environments, and ensuring efficient and stable container placement and alignment operations.

[0007] In one optional implementation, acquiring multi-frame point cloud data of the container truck includes: controlling a laser acquisition device to acquire multi-frame point cloud data of the container truck, acquiring the acquired multi-frame initial point cloud data; determining the ground point cloud contained in each frame of initial point cloud data based on a multi-verification mechanism, and removing the ground point cloud to obtain object point cloud data; acquiring predetermined obstacle point clouds, and removing the obstacle point clouds contained in each frame of object point cloud data to obtain multi-frame container truck point cloud data, wherein the obstacle point cloud includes at least: container point cloud and spreader point cloud.

[0008] This invention, by sequentially removing ground point clouds and obstacle point clouds such as containers and spreaders from the initial point cloud during the acquisition of multi-frame container truck point cloud data, can accurately filter out various interfering point cloud data in the operation scenario. It effectively removes interference from irrelevant factors such as the ground, operating equipment, and containers waiting to be loaded and unloaded, thus obtaining clean and effective container truck point cloud data. This provides a high-quality data foundation for the subsequent construction of the global model point cloud, avoids model distortion caused by interfering point clouds, and further ensures the accuracy of subsequent container truck modeling and positioning.

[0009] In one optional implementation, the point cloud data of multiple frames of container trucks is fused to construct a reference model point cloud of the container truck, including: acquiring the position information of the crane trolley and determining the initial pose transformation matrix of each frame of container truck point cloud data based on the position information; performing global coordinate transformation on each frame of container truck point cloud data based on the initial pose transformation matrix to form at least one sliding window, the sliding window containing multiple consecutive frame point clouds; merging and fusing the multiple consecutive frame point clouds of the sliding window to obtain the local container truck point cloud corresponding to the sliding window; if there are multiple overlapping sliding windows, the local point clouds corresponding to each sliding window are spliced ​​and fused, and overlapping redundant points are removed to obtain the reference model point cloud.

[0010] This invention determines the initial pose transformation matrix based on the position information of the crane trolley, completes the global coordinate unification of multi-frame point clouds, and constructs a global model point cloud by sliding window block fusion and multi-window stitching to remove redundancy. It can effectively correct the coordinate deviation of multi-frame point clouds during the movement of the trolley, avoid the problem of local information loss in single point cloud fusion, and remove redundant point clouds in overlapping areas, improve the integrity and accuracy of the global model point cloud of the truck, reduce modeling errors, and provide high-precision benchmark model support for subsequent real-time point cloud matching and positioning.

[0011] In one optional implementation, the point clouds of multiple consecutive frames within a sliding window are merged and fused to obtain the local truck point cloud corresponding to the sliding window. This includes: constructing multi-frame joint voxels based on the sliding window; performing planar fitting on all cross-frame mixed points within each multi-frame joint voxel; performing singular value decomposition on the covariance matrix of the fitted point set; and selecting effective planar voxels based on the minimum eigenvalue obtained from the decomposition; constructing a global loss function based on the sum of the minimum eigenvalues ​​of all effective planar voxels within the sliding window; iteratively updating the frame-level pose transformation matrix of each frame within the sliding window based on the global loss function until the global loss function converges to a preset threshold; using the frame-level pose transformation matrix obtained after iterative convergence, performing global coordinate transformation on the truck point cloud data of each frame within the sliding window again; and merging all truck point cloud data within the sliding window to obtain the local truck point cloud corresponding to the sliding window.

[0012] This invention employs multi-frame joint voxel construction, planar fitting, and singular value decomposition to screen effective planar voxels. It then uses the sum of minimum eigenvalues ​​to construct a global loss function for iterative optimization of frame-level pose. This method accurately corrects pose deviations in each frame of point cloud data, effectively suppressing matching drift during multi-frame point cloud fusion. Compared to traditional direct stacking fusion methods, it significantly improves the accuracy of local point cloud fusion within a sliding window, further optimizing the detail accuracy and fit of the global model of the truck, and adapting to the dynamic movement of the field bridge vehicle during data acquisition.

[0013] In one optional implementation, point cloud matching is performed between single-frame point cloud data and reference model point cloud to determine the pose information of the truck relative to the crane trolley. This includes: performing block-based planar fitting on the reference model point cloud to determine the reference geometric surfaces corresponding to multiple preset regions on the truck; determining the ground point cloud included in the single-frame point cloud data based on a multi-verification mechanism, and removing the ground point cloud to obtain the point cloud data to be matched; performing real-time radar coordinate transformation on the reference model point cloud based on the initial matching pose transformation matrix, and performing nearest-neighbor association matching between the point cloud to be matched and the point clouds corresponding to each transformed reference geometric surface to construct a point cloud error function; iteratively updating the initial matching pose transformation matrix based on the point cloud error function until the point cloud error function converges to a preset threshold to obtain the optimal matching pose transformation matrix, and using the optimal matching pose transformation matrix as the six-degree-of-freedom pose information of the truck relative to the crane trolley.

[0014] This invention performs block fitting of reference geometric surfaces to the global model point cloud, and relies on proximity association matching and error function iterative optimization of the matching pose to achieve accurate pose solving under occlusion conditions. By replacing traditional scattered point cloud matching with structured reference surface matching, it can adapt to the operation scenario of large-area occlusion of container trucks and only residual point cloud in the container placement stage, effectively solving the matching deviation problem caused by insufficient feature of residual point cloud in a single frame, greatly improving the robustness and accuracy of container truck pose solving under occlusion conditions, and achieving high-precision pose output of six degrees of freedom.

[0015] In one optional implementation, after matching the single-frame point cloud data with the reference model point cloud to determine the pose information of the truck relative to the crane trolley, the method further includes: identifying feature points in the reference model point cloud to determine preset feature points of the truck, which are used for alignment between the spreader and the truck; determining the local position information of the preset feature points relative to the crane trolley based on the six degrees of freedom pose information of the truck relative to the crane trolley; and adjusting the attitude of the crane trolley and / or the spreader based on the local position information until the spreader and the truck are aligned.

[0016] This invention is based on the precise solution of the six-degree-of-freedom pose of the container truck, calculates the local position of the preset alignment feature points of the container truck, and adjusts the attitude of the trolley and the spreader accordingly. This transforms the overall pose positioning of the container truck into precise local alignment feature positioning, enabling accurate calibration of key alignment positions such as the spreader and the container truck lock head. This overcomes the shortcomings of overall positioning in meeting the requirements of fine alignment, thoroughly ensures the accurate docking of container unloading operations, reduces unloading deviation and the probability of operational failures, and improves the success rate of automated operations.

[0017] In one optional implementation, the ground point cloud is determined based on a multi-verification mechanism for each frame of initial point cloud data or a single frame of point cloud data, including: using the vertical height of the laser acquisition device as the statistical dimension, dividing the three-dimensional coordinate points of all point clouds contained in each frame of initial point cloud data or a single frame of point cloud data into intervals according to the vertical height, dividing continuous height intervals, and constructing a point cloud height distribution histogram; determining the number of point clouds contained in each height interval in the point cloud height distribution histogram, and taking the height interval with the largest number of point clouds as the main height range of ground point cloud distribution; expanding the main height range into upper and lower intervals based on a preset height threshold to determine the candidate height range of ground point clouds; performing neighbor point retrieval on the candidate ground point clouds within the candidate height range to obtain the corresponding nearest neighbor set, and calculating the three-dimensional covariance matrix of the nearest neighbor set; performing eigenvalue decomposition on the covariance matrix, and performing ground plane fitting on the candidate ground points based on the minimum eigenvalue after decomposition, and determining the ground point cloud based on the fitting result.

[0018] This invention employs a multi-verification mechanism combining histogram height coarse screening with plane fitting fine verification to identify ground point clouds. First, it quickly delineates candidate ground point clouds through height interval statistics, and then accurately fits the ground plane based on the covariance matrix eigenvalue decomposition. This mechanism can adapt to scenarios with slight undulations on the dock surface and interference from low-lying debris, accurately removing ground interference points in the point cloud at different operational stages, effectively improving the accuracy of ground point cloud segmentation, ensuring the quality of point cloud preprocessing, and providing a stable and clean point cloud data source for modeling and real-time matching and positioning.

[0019] Secondly, this invention provides a multi-frame fusion-based truck positioning device applied to automated yard cranes. The automated yard crane is equipped with a laterally movable crane trolley, which is equipped with a spreader and a laser acquisition device. The device includes: a first point cloud acquisition module, used to acquire multi-frame point cloud data of the truck corresponding to the truck during the lateral movement of the crane trolley from the yard to above the truck; a model point cloud construction module, used to fuse the multi-frame point cloud data of the truck to construct a reference model point cloud of the truck; a second point cloud acquisition module, used to acquire the truck's point cloud in real time during the process of the crane trolley lowering the spreader toward the truck to obtain single-frame point cloud data; and a truck pose determination module, used to match the single-frame point cloud data with the reference model point cloud to determine the truck's pose information relative to the crane trolley.

[0020] Thirdly, the present invention provides an automated field bridge, comprising: a controller and a lifting trolley, the lifting trolley being equipped with a lifting device and a laser acquisition device, the controller being connected to the lifting trolley, the lifting device and the laser acquisition device; the controller comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the multi-frame fusion-based truck positioning method of the first aspect or any corresponding embodiment described above.

[0021] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the multi-frame fusion-based card positioning method of the first aspect or any corresponding embodiment described above. Attached Figure Description

[0022] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of an application scenario according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of an automated field bridge according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the first process of a truck positioning method based on multi-frame fusion according to an embodiment of the present invention; Figure 4 This is a schematic diagram of multi-frame point cloud data acquisition for a truck positioning method based on multi-frame fusion according to an embodiment of the present invention; Figure 5 This is a schematic diagram of a reference template point cloud for a truck positioning method based on multi-frame fusion according to an embodiment of the present invention; Figure 6 This is a schematic diagram of single-frame point cloud data of the multi-frame fusion-based truck positioning method according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the second process of the multi-frame fusion-based truck positioning method according to an embodiment of the present invention; Figure 8 This is a schematic diagram of the overall processing flow of the truck positioning method based on multi-frame fusion according to an embodiment of the present invention; Figure 9 This is a schematic diagram of the third process of the multi-frame fusion-based truck positioning method according to an embodiment of the present invention; Figure 10 This is a structural block diagram of a truck positioning device based on multi-frame fusion according to an embodiment of the present invention; Figure 11 This is a schematic diagram of the hardware structure of the controller according to an embodiment of the present invention.

[0024] Explanation of reference numerals in the attached drawings: 2-Automated yard bridge; 201-Controller; 202-Lifting trolley; 203-Lifting device; 204-Laser acquisition device. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0027] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0028] As an optional application scenario of this invention, such as Figure 1 As shown, the automated control system developed for automated yard bridges may include at least one terminal device and at least one server. Figure 1 The system is illustrated in the example, which includes a computer 101, a mobile terminal 102, and a server 103, and the terminal devices such as the computer 101 and the mobile terminal 102 are connected to the server 103 through a network 110.

[0029] Specifically, the terminal device can be a smartphone, tablet, laptop, PDA, desktop computer, game console, smart TV, smart wearable device, in-vehicle terminal, VR (Virtual Reality) device, AR (Augmented Reality) device, etc. Server 103 can be a standalone physical server, a server cluster, a distributed system, or a cloud server providing cloud services. Network 110 can be a wired or wireless network, examples of which include, but are not limited to, the Internet, corporate intranet, local area network, wide area network, mobile communication network, and combinations thereof.

[0030] Furthermore, such as Figure 2 The diagram shows a schematic of the hardware components of the automated yard crane provided in an embodiment of the present invention. The automated yard crane 2 of this application is equipped with a controller 201 and a lifting trolley 202 that can move laterally along the crossbeam. The bottom of the lifting trolley 202 is fitted with a spreader 203 for grabbing and lifting containers. A laser acquisition device 204, such as a 3D lidar, is fixedly mounted on the lifting trolley 202 to collect real-time three-dimensional point cloud data of the trucks, containers, and yard ground below. The controller 201 is equipped with an automatic control system connected to the lifting trolley 202, the spreader 203, and the laser acquisition device 204, thereby automating the control of the lifting trolley 202 and the spreader 203 to achieve automated container loading and unloading operations for the trucks. In a specific hardware installation example, the laser acquisition device 204 can be fixedly installed at the bottom center of the crane trolley 202 and on the side near the spreader 203. This installation position can move laterally synchronously with the crane trolley 202. It has no structural obstruction and a wide field of view, and can continuously acquire complete point cloud data of the truck from multiple angles during the movement of the trolley. At the same time, it can acquire local residual point cloud data in real time during the lowering operation of the spreader, which is suitable for the full-process positioning acquisition requirements.

[0031] In this embodiment of the invention, taking the complete operation scenario of the automated yard crane 2 dropping containers from the yard to the truck as an example: During the operation, the automated yard crane 2 generates operation control instructions based on the automated control system, and controls the crane trolley 202 to carry the spreader 203 and the laser acquisition device 204 to move longitudinally from the yard area to the operation position of the truck; wherein, the truck is stationary on the side of the yard where the containers are stacked, waiting for the container to be dropped.

[0032] During operation, the crane trolley 202 is initially positioned above the yard. The spreader 203 is then lowered and lifted to pick up the container, subsequently moving laterally along with the container, spreader 203, and laser acquisition device 204 to the truck. During this lateral movement, the laser acquisition device 204 continuously acquires multi-frame point cloud data of the truck from multiple angles, completing the construction of a global model of the truck. Subsequently, the spreader 203 is lowered down, and real-time point cloud matching and positioning are used to achieve precise alignment and placement of the container and truck. The entire process relies on the point cloud data acquired by the laser acquisition device 204 to achieve high-precision real-time positioning of the truck, ensuring the stable and accurate completion of the automated container placement operation.

[0033] Therefore, this invention provides a multi-frame fusion-based container truck positioning method. By collecting point cloud data from multiple frames during the lateral movement of the trolley, a reference model of the container truck is constructed. This enables high-precision real-time positioning of the container truck based on the reference model during the unloading process, ensuring the stable and accurate completion of automated container unloading operations.

[0034] According to an embodiment of the present invention, a method for positioning a card based on multi-frame fusion is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0035] This embodiment provides a multi-frame fusion-based truck positioning method, which can be used in the aforementioned automated field bridge. Figure 3 This is a flowchart of a multi-frame fusion-based truck positioning method according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps: Step S301: During the process of the crane trolley moving laterally from the yard to above the container truck, acquire multi-frame point cloud data of the container truck.

[0036] Specifically, in this embodiment of the invention, the container yard is the operational area of ​​the terminal used for stacking and transferring containers, the container truck is a transport vehicle parked on the side of the container yard for transferring containers, and the crane trolley can move laterally along the crossbeams of the automated yard crane. After the automated container unloading operation is started, the automated control system outputs a movement control command, driving the crane trolley, carrying the spreader and laser acquisition device, to move smoothly laterally from the starting position of the container stacking in the container yard to the area directly above the stationary container truck. During the continuous movement of the crane trolley, the laser acquisition device maintains a continuous scanning acquisition state, continuously outputting multiple frames of raw point cloud data according to a preset acquisition frequency.

[0037] Because the crane trolley is in a moving state, the laser acquisition device can perform full-coverage acquisition of the truck body below from different horizontal positions and different scanning angles, such as... Figure 4 As shown, multiple frames of original point cloud data covering different areas of the truck and complementing each other are finally obtained, forming a multi-frame point cloud dataset that completely covers the main structure of the truck, providing sufficient data support for subsequent model construction.

[0038] Step S302: Perform fusion processing on the multi-frame point cloud data of the truck to construct the benchmark model point cloud of the truck.

[0039] Specifically, in this embodiment of the invention, the multi-frame point cloud data of the container truck consists of discrete point cloud data collected from different perspectives. A single frame of point cloud data can only reflect the local structural information of the container truck. Furthermore, if the spreader is lifting a container, the single frame of point cloud data may suffer from limitations in perspective, sporadic noise, and local missing information due to the container's obstruction. This embodiment of the invention addresses this by fusing and integrating the multi-frame discrete point cloud data of the container truck. It unifies, integrates, and optimizes the multi-frame point cloud data collected from different movement positions and scanning perspectives, thereby compensating for the information loss problem of a single frame of point cloud data and eliminating random noise interference from a single acquisition.

[0040] After multi-frame point cloud fusion processing, the embodiments of the present invention can construct a complete, regular, and accurate benchmark model point cloud that can realistically reproduce the overall shape and structural features of the truck, such as... Figure 5 As shown, the benchmark model point cloud is a pre-built standardized truck model, which does not include issues such as real-time operation occlusion and environmental interference, and can provide a fixed and accurate matching benchmark for subsequent real-time point cloud matching and positioning.

[0041] Step S303: During the process of the crane trolley lowering the spreader toward the truck, real-time point cloud data is collected from the truck to obtain single-frame point cloud data.

[0042] Specifically, in this embodiment of the invention, after the point cloud of the truck's reference model is constructed, the automated yard crane enters the container lowering and alignment operation stage. At this time, the crane trolley remains in the working position above the truck, and the automated control system drives the spreader to lower the grabbed container vertically downwards toward the truck body. During the dynamic operation of the spreader lowering, the laser acquisition device is simultaneously controlled to activate its real-time acquisition function, instantly scanning the area below the truck to acquire single-frame point cloud data under the current working conditions, such as... Figure 6 As shown. Due to the operating conditions, the spreader and container are suspended above the truck, which will obstruct the view of the truck body. Therefore, this single frame point cloud data is the partial residual point cloud data of the truck, which can reflect the actual position and attitude of the truck in real time.

[0043] Step S304: Match the single-frame point cloud data with the reference model point cloud to determine the pose information of the truck relative to the crane trolley.

[0044] Specifically, in this embodiment of the invention, the reference model point cloud is a pre-constructed complete standard point cloud model of the container truck, and the single-frame point cloud data is the local point cloud data of the container truck collected in real time under the deployment condition. By matching the two point clouds, the spatial position and attitude features of the real-time collected single-frame point cloud data can be compared and matched with the complete reference model point cloud. Through spatial correlation and iterative adaptation, the spatial transformation relationship between the real-time single-frame point cloud and the reference model can be solved. Finally, the six-degree-of-freedom pose information of the current container truck relative to the crane trolley can be accurately calculated, including the three-dimensional spatial position of the container truck and its pitch, roll, and yaw attitudes. This provides accurate pose data support for the subsequent precise alignment of the spreader and the container truck and automated container unloading operations, effectively adapting to the positioning requirements under occlusion conditions.

[0045] The multi-frame fusion-based truck positioning method provided in this invention collects multi-frame point clouds of the truck during the lateral movement of the crane trolley and fuses them to construct a complete reference model point cloud of the truck. Then, during the spreader lowering operation, the pose is solved by matching the real-time single-frame point cloud with the reference model point cloud. This invention distinguishes between the unobstructed modeling stage and the obstructed alignment operation stage, and uses multi-frame data collected during the movement process to construct a complete and reliable truck reference model. This effectively solves the problems of positioning failure and excessive deviation caused by truck obstruction, incomplete single-frame point clouds, truck movement, and environmental interference under container placement conditions. It possesses extremely high environmental robustness and positioning accuracy, thus significantly improving the accuracy and stability of truck positioning in dynamic operation scenarios of automated yard cranes, significantly improving the accuracy of container placement and alignment, adapting to complex terminal operating environments, and ensuring efficient and stable container placement and alignment operations.

[0046] This embodiment provides a multi-frame fusion-based truck positioning method, which can be used in the aforementioned automated field bridge. Figure 7This is a flowchart of a multi-frame fusion-based truck positioning method according to an embodiment of the present invention, such as... Figure 7 As shown, the process includes the following steps: Step S701: During the process of the crane trolley moving laterally from the yard to above the container truck, acquire multi-frame point cloud data of the container truck.

[0047] Specifically, step S701 includes: Step S7011: Control the laser acquisition device to perform multi-frame point cloud acquisition on the truck and obtain the acquired multi-frame initial point cloud data.

[0048] Specifically, in this embodiment of the invention, during the entire process of the crane trolley moving laterally from the stockyard to above the truck, the laser acquisition device remains continuously powered on and scanning, continuously acquiring three-dimensional point cloud data according to a preset fixed sampling frequency. Since the crane trolley is in a state of uniform lateral movement, such as... Figure 8 As shown, the laser acquisition device can perform full-coverage sampling of the working area below from different horizontal positions and scanning angles, obtaining multiple frames of continuous, complementary, and initial point cloud data covering the entire area of ​​the truck. This initial point cloud data includes point clouds of all scenes such as the truck, the ground, containers, spreaders, and yard debris, providing the raw data foundation for subsequent layered filtering and processing.

[0049] Step S7012: Based on the multi-verification mechanism, determine the ground point cloud contained in the initial point cloud data of each frame, and remove the ground point cloud to obtain the object point cloud data.

[0050] Specifically, in this embodiment of the invention, the initial point cloud data collected originally contains a large amount of ground point cloud data of the storage yard. The ground is a large area of ​​flat background interference and does not have the structural features of the truck body. If the ground point cloud is directly retained for subsequent multi-frame fusion modeling, a large amount of invalid background data will be mixed in, resulting in redundant point cloud data and a messy model base. This seriously interferes with the extraction of effective structural features of the truck, causing deformation and redundancy distortion in the subsequently constructed baseline model point cloud. At the same time, it will increase the computational load of subsequent plane fitting and pose iteration optimization, reducing modeling accuracy and matching stability. Therefore, this embodiment of the invention introduces a multi-verification mechanism to accurately peel off the ground background point cloud. The multi-verification mechanism combines height histogram statistics and plane fitting dual discrimination logic to accurately distinguish between the ground point cloud and the foreground object point cloud.

[0051] In some optional implementations, step S7012 above includes: Step a1: Using the vertical height of the laser acquisition device as the statistical dimension, divide the three-dimensional coordinate points of all point clouds contained in each frame of initial point cloud data or single frame of point cloud data into intervals according to the vertical height, divide the continuous height intervals, and construct a point cloud height distribution histogram.

[0052] Step a2: Determine the number of point clouds contained in each height interval of the point cloud height distribution histogram, and take the height interval with the largest number of point clouds as the main height range of the ground point cloud distribution.

[0053] Step a3: Based on a preset height threshold, expand the main height range into upper and lower intervals to determine the candidate height range of the ground point cloud.

[0054] Step a4: Perform a neighbor point search on the candidate ground point cloud within the candidate height range to obtain the corresponding nearest neighbor set, and calculate the three-dimensional covariance matrix of the nearest neighbor set.

[0055] Step a5: Perform eigenvalue decomposition on the covariance matrix, and perform ground plane fitting on the candidate ground points based on the minimum eigenvalue after decomposition. Determine the ground point cloud based on the fitting results.

[0056] Specifically, in this embodiment of the invention, the vertical height of the laser acquisition device is used as the sole statistical dimension. The height values ​​of all three-dimensional coordinate points within the initial point cloud of the current single frame are uniformly divided into several continuous, equidistant height intervals, thereby constructing a complete point cloud height distribution histogram. This histogram can intuitively reflect the dense distribution of point clouds at different height positions in the current scene. Since the ground of the storage yard is a large, continuous, flat area, the number of point clouds at its corresponding height position is much higher than that of other areas such as vehicle bodies and debris. Therefore, the number of point clouds contained in each height interval is counted one by one, and the height interval with the largest peak value and the densest aggregation of point clouds is determined as the main height range of the ground.

[0057] To accommodate height deviations caused by slight undulations in the dock surface, ground wear, and minor debris, this embodiment of the invention pre-configures a fixed height tolerance threshold, expands the selected main ground height range into upper and lower intervals, delineates a complete ground candidate height region, and determines all point clouds within this interval as candidate ground point clouds.

[0058] Furthermore, to avoid misjudgments caused by simple height-based filtering, all candidate ground point clouds within the candidate height range are precisely verified: For each candidate point, a nearest neighbor search is performed within its radius, resulting in a set of nearest neighbors for each candidate point. Based on the mean and deviation of the 3D spatial coordinates of this local point cloud set, a local 3D covariance matrix is ​​constructed. Singular Value Decomposition (SVD) is then performed on the covariance matrix to obtain the corresponding eigenvalues ​​and eigenvectors. The smallest eigenvalue corresponds to the planar normal of the local point cloud, representing the flatness of the local point cloud. This invention uses the feature vector corresponding to the minimum eigenvalue to fit the scene's ground plane, obtaining an accurate ground plane equation. Then, it calculates the spatial distance of all candidate points relative to the fitted ground plane, accurately identifying candidate points with distances less than a preset plane distance threshold as real ground point clouds. All real ground point clouds in each frame's initial point cloud data are then removed. After removing the ground point clouds, the remaining effective foreground point cloud data, including trucks, containers, and spreaders, constitutes the object point cloud data. This completely eliminates redundant interference from a large area of ​​ground background, retaining only the point cloud data relevant to the operational target, providing a clean and effective data foundation for subsequent high-precision truck model construction, feature extraction, and pose optimization.

[0059] Step S7013: Obtain the predetermined obstacle point cloud and remove the obstacle point cloud contained in each frame of object point cloud data to obtain multiple frames of truck point cloud data, wherein the obstacle point cloud includes at least: container point cloud and spreader point cloud.

[0060] Specifically, in this embodiment of the invention, after obtaining the object point cloud data, interfering obstacle point clouds in the scene are further removed. The types of fixed obstacles can be pre-determined through scene calibration and feature clustering, mainly including the point cloud of the container to be transported above and the point cloud of the lifting equipment under the crane trolley, or the container point cloud and lifting equipment point cloud can be determined through point cloud acquisition when the container is being lifted. Based on this, the container point cloud and lifting equipment point cloud in the object point cloud data are identified and filtered out by matching spatial distance thresholds, size features, and structural features, retaining only the complete and pure point cloud of the truck body. Finally, high-quality multi-frame truck point cloud data without ground interference or equipment occlusion interference is obtained. , This represents the number of frames, thereby ensuring the purity and validity of subsequent modeling data.

[0061] Step S702: Perform fusion processing on the multi-frame point cloud data of the truck to construct the benchmark model point cloud of the truck.

[0062] Specifically, step S702 includes: Step S7021: Obtain the position information of the crane trolley and determine the initial pose transformation matrix of the point cloud data of each frame of the truck based on the position information.

[0063] Specifically, in this embodiment of the invention, high-precision fusion of multi-frame point clouds is achieved by unifying initial coordinates, optimizing sliding window blocks, iterating pose correction, and removing redundancy through multi-window stitching. This constructs a reference model point cloud with a complete structure and higher accuracy corresponding to the truck, which is then used as a positioning reference.

[0064] Therefore, before multi-frame point cloud fusion, the lateral position information of the crane trolley is read from the real-time output of the field bridge odometer or the travel encoder. By combining the installation parameters of the laser acquisition device, the initial pose transformation matrix corresponding to each frame of point cloud acquisition time is calculated. The initial pose transformation matrix It includes preliminary rotation and translation relationships, which are used to initially unify the independent radar coordinate system point cloud of each frame into the same global coordinate system, realize the coarse alignment of multi-frame point clouds, solve the problem of coordinate inconsistency caused by the movement of the vehicle, and provide reliable initial values ​​for subsequent fine optimization.

[0065] Step S7022: Perform global coordinate transformation on each frame of card point cloud data according to the initial pose transformation matrix to form at least one sliding window, which contains multiple consecutive frames of point cloud.

[0066] Specifically, in this embodiment of the invention, the initial pose transformation matrix t' corresponding to each frame is used to sequentially pair the point cloud data of each frame of the card. A global coordinate transformation is performed to map the discrete multi-frame point clouds to a unified global coordinate system, thus obtaining the global coordinates of the truck point cloud. Simultaneously, a sliding window mechanism is employed to group the continuously acquired point clouds. Each sliding window contains a preset number of consecutive multi-frame point clouds. Sliding window grouping enables centralized optimization of local point clouds, avoiding error accumulation caused by global one-time fusion, and improving the precision and stability of local point cloud fusion. The formulas corresponding to global coordinate transformation and the sliding window are shown below:

[0067] in, The global coordinates of the truck point cloud. For collecting point cloud data, This represents the total number of frames within the sliding window.

[0068] Step S7023: Merge and fuse the point clouds of multiple consecutive frames of the sliding window to obtain the local point cloud of the truck corresponding to the sliding window.

[0069] Specifically, in this embodiment of the invention, after obtaining multiple sliding windows, the inter-frame pose of point clouds in multiple consecutive frames within any sliding window is finely corrected through voxel structuring, planar feature filtering, and loss function iterative optimization.

[0070] In some optional implementations, step S7023 above includes: Step b1: Construct multi-frame joint voxels based on a sliding window, perform plane fitting on all cross-frame mixed points within each multi-frame joint voxel, and perform singular value decomposition on the covariance matrix of the fitted point set. Filter effective planar voxels based on the minimum eigenvalue obtained from the decomposition.

[0071] Step b2: Construct a global loss function based on the sum of the minimum eigenvalues ​​of all valid planar voxels within the sliding window, and iteratively update the frame-level pose transformation matrix of each frame within the sliding window based on the global loss function until the global loss function converges to a preset threshold.

[0072] Step b3: Using the frame-level pose transformation matrix obtained after iterative convergence, the global coordinate transformation is performed again on the point cloud data of each frame of the truck within the sliding window, and all point cloud data of the truck within the sliding window are merged to obtain the local point cloud of the truck corresponding to the sliding window.

[0073] Specifically, in this embodiment of the invention, firstly, all cross-frame fused point clouds within the sliding window are divided into voxel meshes to construct multi-frame joint voxels, making the discrete point clouds form regular local structural units. Plane fitting is then performed on all mixed points within each joint voxel to characterize the planar structural features of the local point cloud. Furthermore, a three-dimensional covariance matrix is ​​constructed from the fitted point set, and singular value decomposition (SVD) is performed on the covariance matrix to obtain three sets of eigenvalues ​​and corresponding eigenvectors. The minimum eigenvalue is... The minimum eigenvalue represents the degree to which a local point cloud deviates from an ideal plane. The smaller the value, the better the planar flatness and the more stable the structure of the voxel. Based on this, effective planar voxels that meet the threshold condition are selected, and invalid voxels that are messy, distorted, or lack stable planar structures are eliminated, while retaining the effective structural features of the card.

[0074] In one alternative implementation, embodiments of the present invention summarize the minimum eigenvalues ​​of all valid planar voxels within the sliding window. A global loss function is constructed with the goal of minimizing the sum of minimum eigenvalues. This loss function can comprehensively characterize the smoothness and matching accuracy of multi-frame point cloud fusion within a sliding window. The frame-level pose transformation matrix corresponding to each frame point cloud within the window is continuously updated through gradient iteration to continuously correct inter-frame misalignment and matching deviation problems. When the change in the loss function is less than the preset convergence threshold or the maximum number of iterations is reached, iterative convergence is determined. At this time, the pose parameters of each frame reach the optimal state, completely eliminating the inter-frame positioning drift error generated during the vehicle's movement and acquisition process.

[0075] Furthermore, after iterative optimization, using the converged optimal frame-level pose transformation matrix as a benchmark, the original truck point cloud data for each frame within the sliding window is precisely calibrated again to replace the initial coarse alignment pose, achieving high-precision registration of multi-frame point clouds. All calibrated frame point clouds are then... By overlaying and merging, a local point cloud of the truck is obtained that is free from misalignment and drift and has a structure that fits the real truck body. This completes the fine-grained fusion modeling of a single sliding window.

[0076] Step S7024: If there are multiple overlapping sliding windows, the local point clouds corresponding to each sliding window are stitched and merged, and overlapping redundant points are removed to obtain the baseline model point cloud.

[0077] Specifically, in this embodiment of the invention, multiple overlapping and continuously distributed sliding windows are formed during the long-distance movement of the vehicle, and each sliding window corresponds to a high-precision local point cloud of the container truck. The local point clouds corresponding to all windows are spatially stitched together sequentially to achieve full coverage of the overall structure of the container truck. Simultaneously, redundant point clouds in the overlapping areas of the windows are subjected to redundancy removal and deduplication processing, deleting duplicate dense points and retaining structural feature points, ultimately obtaining a structurally complete, uniformly distributed, non-redundant, and high-precision point cloud of the container truck's baseline model. :

[0078] Benchmark Model Point Cloud It can provide a stable and standard model benchmark for real-time matching and positioning under subsequent occlusion conditions.

[0079] Step S703: During the process of the crane trolley lowering the spreader towards the truck, real-time point cloud data is collected from the truck to obtain a single frame of point cloud data. For details, please refer to [link to relevant documentation]. Figure 3 Step S303 of the illustrated embodiment will not be described again here.

[0080] Step S704: Perform point cloud matching between the single-frame point cloud data and the reference model point cloud to determine the pose information of the truck relative to the crane trolley. For details, please refer to [link to relevant documentation]. Figure 3 Step S304 of the illustrated embodiment will not be described again here.

[0081] The multi-frame fusion-based truck positioning method provided in this invention collects multi-frame point clouds of the truck during the lateral movement of the crane trolley and fuses them to construct a complete reference model point cloud of the truck. Then, during the spreader lowering operation, the pose is solved by matching the real-time single-frame point cloud with the reference model point cloud. This invention distinguishes between the unobstructed modeling stage and the obstructed alignment operation stage, and uses multi-frame data collected during the movement process to construct a complete and reliable truck reference model. This effectively solves the problems of positioning failure and excessive deviation caused by truck obstruction, incomplete single-frame point clouds, truck movement, and environmental interference under container placement conditions. It possesses extremely high environmental robustness and positioning accuracy, thus significantly improving the accuracy and stability of truck positioning in dynamic operation scenarios of automated yard cranes, significantly improving the accuracy of container placement and alignment, adapting to complex terminal operating environments, and ensuring efficient and stable container placement and alignment operations.

[0082] This embodiment provides a multi-frame fusion-based truck positioning method, which can be used in the aforementioned automated field bridge. Figure 9This is a flowchart of a multi-frame fusion-based truck positioning method according to an embodiment of the present invention, such as... Figure 9 As shown, the process includes the following steps: Step S901: During the lateral movement of the crane trolley from the yard to above the container truck, acquire multiple frames of point cloud data corresponding to the container truck. For details, please refer to [link to relevant documentation]. Figure 7 Step S701 of the illustrated embodiment will not be described again here.

[0083] Step S902 involves fusing the point cloud data from multiple frames of the truck to construct a baseline model point cloud for the truck. For details, please refer to [link to relevant documentation]. Figure 7 Step S702 of the illustrated embodiment will not be described again here.

[0084] Step S903: During the process of the crane trolley lowering the spreader towards the truck, real-time point cloud data is collected from the truck to obtain a single frame of point cloud data. For details, please refer to [link to relevant documentation]. Figure 7 Step S703 of the illustrated embodiment will not be described again here.

[0085] Step S904: Perform point cloud matching between the single frame point cloud data and the reference model point cloud to determine the pose information of the truck relative to the crane trolley.

[0086] Specifically, step S904 includes: Step S9041: Perform block-based planar fitting on the point cloud of the reference model to determine the reference geometric surfaces corresponding to multiple preset regions on the truck.

[0087] Specifically, in this embodiment of the invention, during the actual operation of lowering the spreader, the real-time collected single-frame point cloud data still suffers from local missing features and scattered effective features due to the obstruction caused by the container and the suspension of the spreader, making it impossible to directly solve for accurate pose through sparse feature point matching. Therefore, this embodiment of the invention achieves high-precision pose solving under residual point clouds by sequentially constructing block reference surfaces, preprocessing point clouds, matching nearest surfaces, and iterative optimization.

[0088] In some optional embodiments, the pre-constructed benchmark model point cloud of this invention is a complete, unobstructed, and high-precision standard point cloud model of a container truck, fully preserving the planar structural features of the container truck body. Based on the structural characteristics of the container truck body, multiple pre-defined regular regions are pre-divided, covering key alignment areas such as the container truck's bearing platform and side contact surfaces. Then, the model point cloud within each pre-defined region is independently segmented and planar fitted. The optimal planar equation for each region is solved through local point cloud covariance matrix decomposition, accurately extracting multiple flat, stable, and representative benchmark geometric surfaces. Each benchmark geometric surface solidifies the standard spatial structure and attitude characteristics of the container truck body, serving as a fixed comparison benchmark for subsequent real-time matching. Compared to scattered point cloud matching, this significantly improves matching stability under occlusion conditions.

[0089] Step S9042: Based on the multi-verification mechanism, determine the ground point cloud contained in the single frame point cloud data, and remove the ground point cloud to obtain the point cloud data to be matched.

[0090] Specifically, in this embodiment of the invention, the real-time acquired single-frame point cloud data also includes local point clouds of the container truck and ground point clouds of the storage yard. Ground point clouds are invalid background interference, and their participation in matching will reduce matching accuracy and cause pose calculation offset. Therefore, this embodiment of the invention uses the above-mentioned multi-verification mechanism to accurately remove ground point clouds: a height distribution histogram is constructed using the vertical height of the laser acquisition device as the statistical dimension to filter out the main height interval with the densest ground point clouds, and ground candidate areas are obtained by combining a preset height threshold; then, neighbor point retrieval, covariance matrix construction, and singular value decomposition are performed on the candidate area point clouds to accurately fit the real-time ground plane, and all ground point clouds in the single-frame point cloud are filtered and removed. The remaining valid local point clouds of the container truck are the pure point cloud data to be matched without background interference. This provides a high-quality data source for subsequent accurate surface matching.

[0091] Step S9043: Based on the initial matching pose transformation matrix, the reference model point cloud is transformed into radar coordinates in real time, and the point cloud to be matched is matched with the point cloud corresponding to each transformed reference geometric surface in the nearest correlation matching to construct the point cloud error function.

[0092] Specifically, in this embodiment of the invention, all valid sampling points in the point cloud to be matched are traversed, and an initial matching pose transformation matrix is ​​used. The reference model point cloud is transformed to the radar coordinate system, and the corresponding reference geometric surfaces are also transformed to the radar coordinate system. Based on this, and according to the principle of shortest spatial distance, the real-time acquired point cloud data to be matched are... By associating and matching the nearest reference geometric surface, a one-to-one correspondence between real-time points and standard planes is established, achieving precise point-to-surface matching constraints and effectively addressing the problem of insufficient cloud features in single-frame residual defects. This includes matching the pose transformation matrix. Used to characterize the spatial transformation relationship of the real-time point cloud to be matched relative to the geometric surface of the reference model, including spatial rotation parameters. With translation parameters .

[0093] Based on the spatial distance deviations from all associated matching points to the corresponding reference geometric surfaces, an overall point cloud error function is constructed as follows:

[0094] Among them, the magnitude of the error function generally represents the degree of fit between the real-time point cloud and the standard model under the current pose parameters.

[0095] Step S9044: Iteratively update the initial matching pose transformation matrix based on the point cloud error function until the point cloud error function converges to a preset threshold to obtain the optimal matching pose transformation matrix, and use the optimal matching pose transformation matrix as the six-degree-of-freedom pose information of the truck relative to the crane trolley.

[0096] Specifically, in this embodiment of the invention, after constructing the point cloud error function, the matching pose transformation matrix is ​​continuously corrected and updated through gradient iterative optimization. The overall matching error is continuously reduced, and the spatial alignment effect is continuously optimized. Iterative updates are performed repeatedly until the change in the error function is less than a preset convergence threshold. At this point, matching is considered complete and the pose parameters are optimal, thus obtaining the optimal matching pose transformation matrix. This effectively avoids noise interference from single-point matching and improves the robustness of pose solving.

[0097] It can be seen that the matching pose transformation matrix after iterative convergence The optimal spatial transformation parameters under the current operating conditions accurately characterize the mapping relationship between the model coordinate system and the real-time radar coordinate system. Therefore, the optimal matching pose transformation matrix... It can be used as the six-degree-of-freedom pose information of the truck relative to the crane trolley, that is, it covers three-dimensional translational position parameters and attitude parameters including pitch, roll and yaw three-dimensional angles, to achieve high-precision full-dimensional positioning output in occlusion, abnormal movement and complex environments.

[0098] Furthermore, the pre-constructed complete benchmark model point cloud can be transformed into global coordinates through the optimal matching pose transformation matrix, so that the standard truck model accurately matches the actual spatial position and attitude of the truck under the current real-time operating conditions. Step S905: Feature point recognition is performed on the reference model point cloud to determine the preset feature points of the container truck. The preset feature points are used for alignment between the spreader and the container truck.

[0099] Specifically, in this embodiment of the invention, the overall six-degree-of-freedom pose can only represent the overall position and attitude of the truck body, which cannot meet the refined operational requirements of precise container placement and lock hole alignment. Therefore, this embodiment of the invention pre-marks multiple preset feature points for alignment operations on the reference model point cloud. The feature point locations are selected from key operational positions such as the center of the truck's carrying platform, the lock head alignment reference point, and the side corner reference of the vehicle body. Each preset feature point is accurately located through model structural feature recognition, local curvature filtering, and structural intersection point extraction. Figure 8 As shown, these feature points are the core reference points for precise alignment of the spreader and calibration of the container, ensuring that subsequent fine-grained attitude adjustments are based on reliable data.

[0100] Step S906: Based on the six-degree-of-freedom pose information of the truck relative to the crane trolley, determine the local position information of the preset feature points relative to the crane trolley.

[0101] Specifically, in this embodiment of the invention, the solved overall six-degree-of-freedom pose of the truck is used as the global transformation reference. Combined with the relative coordinate positions of preset feature points in the reference model, spatial coordinate conversion and mapping are performed. For example, the coordinates of feature points in the model coordinate system are transformed to the equipment coordinate system of the crane trolley, accurately calculating the real-time local position information of each preset alignment feature point relative to the crane trolley. This yields refined, point-level alignment coordinate data, achieving an accuracy upgrade from "overall truck body positioning" to "precise local point positioning."

[0102] Step S907: Adjust the attitude of the lifting trolley and / or the spreader based on the local position information until the spreader is aligned with the truck.

[0103] Specifically, in this embodiment of the invention, the automated control system generates precise closed-loop adjustment control commands based on the real-time local position information of each preset feature point, and specifically compensates for alignment deviations. Depending on the actual deviation conditions, the lateral and longitudinal movement positions of the lifting trolley can be adjusted individually, and / or the lifting height, deflection attitude, and horizontal tilt angle of the spreader can be adjusted individually, achieving multi-dimensional collaborative fine-tuning and calibration. Continuous iterative adjustment and real-time comparison of alignment deviations are performed until the spreader lock head and the bottom of the container are perfectly aligned with the preset alignment benchmark of the container truck, completing the automated and precise alignment process. This effectively improves the success rate and alignment accuracy of container placement operations under complex obstruction and container truck movement conditions.

[0104] The multi-frame fusion-based truck positioning method provided in this invention collects multi-frame point clouds of the truck during the lateral movement of the crane trolley and fuses them to construct a complete reference model point cloud of the truck. Then, during the spreader lowering operation, the pose is solved by matching the real-time single-frame point cloud with the reference model point cloud. This invention distinguishes between the unobstructed modeling stage and the obstructed alignment operation stage, and uses multi-frame data collected during the movement process to construct a complete and reliable truck reference model. This effectively solves the problems of positioning failure and excessive deviation caused by truck obstruction, incomplete single-frame point clouds, truck movement, and environmental interference under container placement conditions. It possesses extremely high environmental robustness and positioning accuracy, thus significantly improving the accuracy and stability of truck positioning in dynamic operation scenarios of automated yard cranes, significantly improving the accuracy of container placement and alignment, adapting to complex terminal operating environments, and ensuring efficient and stable container placement and alignment operations.

[0105] This embodiment also provides a multi-frame fusion-based truck positioning device, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0106] This embodiment provides a truck positioning device based on multi-frame fusion, such as... Figure 10 As shown, it includes: The first point cloud acquisition module 1001 is used to acquire multiple frames of point cloud data of the container truck during the process of the crane trolley moving laterally from the yard to above the container truck.

[0107] The model point cloud construction module 1002 is used to fuse multi-frame point cloud data of the truck to construct the benchmark model point cloud of the truck.

[0108] The second point cloud acquisition module 1003 is used to acquire point cloud data of the container truck in real time during the process of the crane trolley lowering the spreader toward the container truck, and obtain single frame point cloud data.

[0109] The truck pose determination module 1004 is used to perform point cloud matching between single frame point cloud data and reference model point cloud to determine the pose information of the truck relative to the crane trolley.

[0110] The multi-frame fusion-based truck positioning device provided in this embodiment of the invention can execute the multi-frame fusion-based truck positioning method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the above modules and units are the same as in the corresponding embodiments described above, and will not be repeated here.

[0111] Figure 11 This is a schematic diagram of the structure of a controller provided in an embodiment of the present invention.

[0112] The following is a detailed reference. Figure 11 The diagram illustrates a structural schematic suitable for implementing a controller in an embodiment of the present invention. The controller may include a processor (e.g., a central processing unit, graphics processing unit, etc.) 1101, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1102 or a program loaded from memory 1108 into random access memory (RAM) 1103. The RAM 1103 also stores various programs and data required for controller operation. The processor 1101, ROM 1102, and RAM 1103 are interconnected via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0113] Typically, the following devices can be connected to I / O interface 1105: input devices 1106 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 1107 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; memory 1108 including, for example, magnetic tape, hard disk, etc.; and communication devices 1109. Communication device 1109 allows the controller to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 11 A controller with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown, and may alternatively implement or have more or fewer devices.

[0114] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 1109, or installed from a memory 1108, or installed from a ROM 1102. When the computer program is executed by the processor 1101, it performs the functions defined in the multi-frame fusion-based card positioning method of the embodiments of the present invention.

[0115] Figure 11 The controller shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0116] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the multi-frame fusion-based card positioning method shown in the above embodiments is implemented.

[0117] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0118] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A multi-frame fusion-based truck positioning method, applied to an automated yard crane, wherein the automated yard crane is equipped with a laterally movable lifting trolley, the lifting trolley being equipped with a lifting device and a laser acquisition device, characterized in that, The method includes: During the process of the crane trolley moving laterally from the yard to above the container truck, multiple frames of container truck point cloud data corresponding to the container truck are acquired; The multi-frame point cloud data of the truck is fused to construct a baseline model point cloud of the truck. During the process of the crane trolley lowering the spreader toward the truck, real-time point cloud data is collected from the truck to obtain single-frame point cloud data; The single-frame point cloud data is matched with the reference model point cloud to determine the pose information of the truck relative to the crane trolley.

2. The method according to claim 1, characterized in that, The acquisition of multi-frame point cloud data corresponding to the truck includes: The laser acquisition device is controlled to perform multi-frame point cloud acquisition on the truck to obtain the acquired multi-frame initial point cloud data. The ground point cloud contained in the initial point cloud data of each frame is determined based on a multi-verification mechanism, and the ground point cloud is removed to obtain the object point cloud data. Obtain predetermined obstacle point clouds, and remove obstacle point clouds contained in each frame of object point cloud data to obtain the multi-frame truck point cloud data, wherein the obstacle point clouds include at least: container point clouds and spreader point clouds.

3. The method according to claim 1, characterized in that, The step of fusing the multi-frame point cloud data of the truck to construct the reference model point cloud of the truck includes: Obtain the position information of the crane trolley, and determine the initial pose transformation matrix of each frame of truck point cloud data based on the position information; Based on the initial pose transformation matrix, global coordinate transformation is performed on each frame of the card point cloud data to form at least one sliding window, which contains multiple consecutive frames of point cloud data. The point clouds of multiple consecutive frames of the sliding window are merged and fused to obtain the local card point cloud corresponding to the sliding window; If there are multiple overlapping sliding windows, the local point clouds corresponding to each sliding window are stitched together and merged, and overlapping redundant points are removed to obtain the reference model point cloud.

4. The method according to claim 3, characterized in that, The step of merging and fusing multiple consecutive frame point clouds of the sliding window to obtain the local card point cloud corresponding to the sliding window includes: Based on the sliding window, a multi-frame joint voxel is constructed. Plane fitting is performed on all cross-frame mixed points within each multi-frame joint voxel. Singular value decomposition is performed on the covariance matrix of the fitted point set. Valid planar voxels are selected based on the minimum eigenvalue obtained from the decomposition. A global loss function is constructed based on the sum of the minimum eigenvalues ​​of all effective planar voxels within the sliding window, and the frame-level pose transformation matrix of each frame within the sliding window is iteratively updated based on the global loss function until the global loss function converges to a preset threshold. Using the frame-level pose transformation matrix obtained after iterative convergence, the global coordinate transformation is performed again on the point cloud data of each frame of the truck within the sliding window, and all point cloud data of the truck within the sliding window are merged to obtain the local point cloud of the truck corresponding to the sliding window.

5. The method according to claim 1, characterized in that, The step of matching the single-frame point cloud data with the reference model point cloud to determine the pose information of the truck relative to the crane trolley includes: The point cloud of the reference model is segmented into planes for fitting to determine the reference geometric surfaces corresponding to multiple preset regions on the truck. The ground point cloud contained in the single frame point cloud data is determined based on a multi-verification mechanism, and the ground point cloud is removed to obtain the point cloud data to be matched. Based on the initial matching pose transformation matrix, the reference model point cloud is transformed into radar coordinates in real time, and the point cloud to be matched is matched with the point cloud corresponding to each transformed reference geometric surface to construct a point cloud error function. The initial matching pose transformation matrix is ​​iteratively updated based on the point cloud error function until the point cloud error function converges to a preset threshold to obtain the optimal matching pose transformation matrix. The optimal matching pose transformation matrix is ​​then used as the six-degree-of-freedom pose information of the truck relative to the crane trolley.

6. The method according to claim 5, characterized in that, After performing point cloud matching between the single-frame point cloud data and the reference model point cloud to determine the pose information of the truck relative to the crane trolley, the method further includes: Feature point recognition is performed on the reference model point cloud to determine the preset feature points of the container truck. The preset feature points are used for alignment between the spreader and the container truck. Based on the six-degree-of-freedom pose information of the truck relative to the crane trolley, the local position information of the preset feature point relative to the crane trolley is determined; Adjust the posture of the lifting trolley and / or the spreading device based on the local position information until the spreading device is aligned with the truck.

7. The method according to any one of claims 2 or 5, characterized in that, The ground point cloud is determined based on a multi-verification mechanism, including: Using the vertical height of the laser acquisition device as the statistical dimension, the three-dimensional coordinate points of all points in each frame of initial point cloud data or the single frame of point cloud data are divided into intervals according to the vertical height, continuous height intervals are divided, and a point cloud height distribution histogram is constructed. The number of point clouds contained in each height interval of the point cloud height distribution histogram is determined, and the height interval with the largest number of point clouds is taken as the main height range of the ground point cloud distribution. Based on a preset height threshold, the main height range is expanded vertically to determine the candidate height range of the ground point cloud; For the candidate ground point cloud within the candidate height range, perform a neighbor point search to obtain the corresponding nearest neighbor set, and calculate the three-dimensional covariance matrix of the nearest neighbor set; The covariance matrix is ​​subjected to eigenvalue decomposition, and the candidate ground points are fitted to the ground plane based on the minimum eigenvalue after decomposition. The ground point cloud is determined based on the fitting result.

8. A multi-frame fusion-based truck positioning device, applied to an automated yard crane, wherein the automated yard crane is equipped with a laterally movable lifting trolley, the lifting trolley being equipped with a lifting device and a laser acquisition device, characterized in that, The device includes: The first point cloud acquisition module is used to acquire multiple frames of point cloud data of the container truck during the process of the crane trolley moving laterally from the yard to above the container truck. The model point cloud construction module is used to fuse the multi-frame card point cloud data to construct the reference model point cloud of the card. The second point cloud acquisition module is used to collect point cloud data of the truck in real time during the process of the crane trolley lowering the spreader toward the truck, and obtain single frame point cloud data. The truck pose determination module is used to perform point cloud matching between the single frame point cloud data and the reference model point cloud to determine the pose information of the truck relative to the crane trolley.

9. An automated field bridge, characterized in that, include: The controller and the lifting trolley are equipped with a lifting device and a laser acquisition device. The controller is connected to the lifting trolley, the lifting device and the laser acquisition device. The controller includes a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the multi-frame fusion-based truck positioning method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the multi-frame fusion-based truck positioning method according to any one of claims 1 to 7.

Images