Method and device for realizing bulk material quantitative loading based on three-dimensional perception and point cloud completion, processor and computer readable storage medium thereof

Abstract

The present application relates to a kind of based on three-dimensional perception and point cloud completion realizes bulk quantitative loading method, including the following steps: collecting original three-dimensional point cloud data of target vehicle and bulk and preprocessing;The three-dimensional point cloud data after preprocessing is parsed;Segment and extract the point cloud data of the heap inside carriage;Incomplete point cloud is handled by poisson surface reconstruction completion;Combining delaunay triangulation with projection integral method, the point cloud data of the heap that is completed is handled;Linkage control module carries out dynamic compensation to complete quantitative loading.The quantitative loading method, device, processor and computer readable storage medium thereof based on three-dimensional perception and point cloud completion of bulk of the present application can obtain carriage and heap point cloud information by relying on three-dimensional perception technology, combined with point cloud completion repair occlusion missing data, accurately measure real-time loading weight, ensure bulk automatic quantitative loading, effectively solve the problem of large measurement error, low intelligent degree of traditional loading.

Description

Technical Field

[0001] This invention relates to the field of intelligent logistics, and more particularly to the field of industrial automation technology. Specifically, it relates to a method, apparatus, processor, and computer-readable storage medium for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion. Background Technology

[0002] Bulk loading is a core link in industrial material transfer. Traditional loading relies on manual guidance, walkie-talkie commands, and visual estimation of material positions, resulting in low efficiency, poor loading accuracy, easy overloading and underloading, and low safety. Existing automated loading systems mostly use ultrasonic, two-dimensional vision, or simple distance measuring sensors, which are easily affected by light, dust, etc., making it difficult to obtain complete information from the site and unable to achieve high-precision measurement.

[0003] While loading solutions based on 3D LiDAR can acquire point cloud data, the point clouds on-site suffer from problems such as high noise, uneven density, and local missing data. Furthermore, the automation and accuracy of cargo compartment size detection are insufficient, and the point cloud of the material pile is difficult to separate from the cargo compartment structure. Weight calculation methods suffer from large errors and poor robustness, failing to meet the requirements for quantitative loading. The purpose of this invention is to provide a quantitative loading method for bulk materials based on 3D perception and point cloud completion, solving problems such as inaccurate perception, incomplete point clouds, large measurement errors, and low automation at the loading site. This method achieves stable and high-precision quantitative loading, thereby constructing an integrated automated loading system that combines perception, decision-making, and control. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method, apparatus, processor and computer-readable storage medium for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion, which is stable, highly accurate and widely applicable.

[0005] To achieve the above objectives, the present invention provides a method, apparatus, processor, and computer-readable storage medium for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion, as follows: The method for quantitative loading of bulk materials based on 3D perception and point cloud completion is characterized by the following steps: (1) Collect the original three-dimensional point cloud data of the target vehicle and bulk materials, and preprocess the original three-dimensional point cloud data; (2) Analyze the preprocessed 3D point cloud data to determine the vehicle compartment size and location data of the target vehicle; (3) Based on the car body size data and location data, segment and extract the point cloud data of the material pile inside the car body; (4) Perform Poisson surface reconstruction and completion processing on the residual point cloud data of the material pile to generate complete point cloud data of the material pile; (5) The complete material pile point cloud data is processed by combining Delaunay triangulation and projection integration to obtain real-time loading weight data; (6) Based on the real-time loading weight data, the linkage control module performs dynamic compensation to complete the quantitative loading.

[0006] Preferably, the preprocessing of the original 3D point cloud data in step (1) specifically includes the following steps: (1.1) Perform pass-through filtering on the original three-dimensional point cloud data, filter out irrelevant background point clouds according to the preset spatial truncation threshold, and retain the point cloud data of the effective area where the carriage and the internal material pile are located; (1.2) Spatiotemporal alignment and overlay are performed on the point cloud data after direct filtering using a multi-frame point cloud weighted fusion strategy based on a time sliding window; (1.3) Perform voxel mesh downsampling on the fused point cloud data to reduce the number of point clouds while preserving the geometric contours of the carriage and the material pile to the maximum extent, so as to obtain the preprocessed three-dimensional point cloud data.

[0007] Preferably, step (2) specifically includes the following steps: (2.1) Using the random sampling consistency algorithm with normal vector constraints, the floor plane of the carriage is identified by fitting the preprocessed three-dimensional point cloud data in the empty carriage state; (2.2) Using the plane of the car floor as a reference, calculate the normal vector of the floor, construct a unified coordinate system for the car, and normalize the point cloud of the entire car to the unified coordinate system of the car; (2.3) Project the point cloud of the base plate onto a two-dimensional plane, calculate the length and width of the carriage using the minimum bounding rectangle algorithm, and take the maximum value of the point cloud in the vertical direction as the height of the carriage; (2.4) By tracking the rear panel of the carriage in real time, the rear panel plane is fitted using the random sampling consistency algorithm, and the center point of the rear panel plane is used as the positioning reference to calculate the real-time distance from the rear panel to the center of the discharge port.

[0008] Preferably, step (3) specifically includes the following steps: (3.1) Using the rear panel plane of the car body detected during the empty car stage as a reference, the current loading point cloud and the empty car body point cloud are registered and calculated, and the current loading point cloud is transformed into the car body coordinate system to complete the coordinate alignment. (3.2) Construct a physical space constraint body for the car based on the detected car length, car width and car height. Combine the three-dimensional point cloud after coordinate alignment with the base plate reference surface to perform spatial trimming, remove the points outside the car and the baffle points, and retain the point cloud data of the material pile inside the car.

[0009] Preferably, step (4) specifically includes the following steps: (4.1) Construct a KD tree for the incomplete point cloud data of the material pile to perform fast index neighborhood search, and perform local surface smoothing fitting by moving least squares method; (4.2) Using the smoothed discrete point cloud data and its normal vector as input, the Poisson equation is solved by constructing an implicit function, and its isosurface is extracted to generate a three-dimensional surface model of a continuous surface. (4.3) The three-dimensional surface model is re-discretized into a point cloud by sampling the centroid coordinates, and the discretized point cloud is constrained and trimmed by combining the prior geometric structure of the carriage, and the over-completion points that exceed the physical boundary of the carriage are filtered out to obtain the complete material pile point cloud data.

[0010] Preferably, step (5) specifically includes the following steps: (5.1) Perform Delaunay triangulation on the surface of the completed stockpile point cloud data to generate a triangular mesh composed of multiple triangular faces; (5.2) Project the triangular faces in the triangular mesh onto the reference plane of the car floor to form triangular prisms, calculate the volume of each triangular prism and sum them up to obtain the total volume; (5.3) Multiply the total volume by the preset average bulk density of the bulk material to obtain the real-time loading weight data.

[0011] Preferably, step (6) specifically includes the following steps: (6.1) Transmit the real-time loading weight data and make a real-time judgment in combination with the preset system inertial lead amount; (6.2) If the real-time loading weight meets the condition of the target weight minus the system inertial lead, a stop command is triggered to complete the automatic quantitative loading of bulk materials.

[0012] The device for quantitative loading of bulk materials based on 3D perception and point cloud completion is characterized in that the device comprises: A processor is configured to execute computer-executable instructions; The memory stores one or more computer-executable instructions, which, when executed by the processor, implement the various steps of the method for quantitative loading of bulk materials based on 3D perception and point cloud completion.

[0013] The processor for realizing quantitative loading of bulk materials based on 3D perception and point cloud completion is characterized in that the processor is configured to execute computer-executable instructions, and when the computer-executable instructions are executed by the processor, the various steps of the above-mentioned method for realizing quantitative loading of bulk materials based on 3D perception and point cloud completion are implemented.

[0014] The computer-readable storage medium is characterized in that it stores a computer program thereon, which can be executed by a processor to implement the various steps of the above-described method for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion.

[0015] The present invention employs a method, apparatus, processor, and computer-readable storage medium for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion. It can acquire point cloud information of the truck body and material pile by relying on three-dimensional perception technology, and combine point cloud completion to repair occlusion and missing data, accurately calculate the real-time loading weight, ensure automated quantitative loading of bulk materials, and effectively solve the problems of large measurement error, occlusion measurement distortion, and low level of intelligence in traditional loading. Attached Figure Description

[0016] Figure 1 This is an algorithm flowchart of the method for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion according to the present invention.

[0017] Figure 2 This is a diagram showing the structural dimensions of a truck body for detecting the method of quantitative loading of bulk materials based on three-dimensional perception and point cloud completion, as presented in this invention.

[0018] Figure 3 This is a fitting effect diagram of the truck tailgate of the method for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion according to the present invention.

[0019] Figure 4 This diagram illustrates the process of acquiring point clouds of bulk materials in the method for quantitative loading based on three-dimensional perception and point cloud completion, as described in this invention.

[0020] Figure 5 This is a comparison image of the stockpile point cloud before and after completion of the method for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion according to the present invention.

[0021] Figure 6 This is a comparison chart of the loading measurement results before and after completion of the method for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion according to the present invention. Detailed Implementation

[0022] To more clearly describe the technical content of the present invention, the following description is provided in conjunction with specific embodiments.

[0023] The present invention provides a method for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion, comprising the following steps: (1) Collect the original three-dimensional point cloud data of the target vehicle and bulk materials, and preprocess the original three-dimensional point cloud data; (2) Analyze the preprocessed 3D point cloud data to determine the vehicle compartment size and location data of the target vehicle; (3) Based on the car body size data and location data, segment and extract the point cloud data of the material pile inside the car body; (4) Perform Poisson surface reconstruction and completion processing on the residual point cloud data of the material pile to generate complete point cloud data of the material pile; (5) The complete material pile point cloud data is processed by combining Delaunay triangulation and projection integration to obtain real-time loading weight data; (6) Based on the real-time loading weight data, the linkage control module performs dynamic compensation to complete the quantitative loading.

[0024] In a preferred embodiment of the present invention, the preprocessing of the original three-dimensional point cloud data in step (1) specifically includes the following steps: (1.1) Perform pass-through filtering on the original three-dimensional point cloud data, filter out irrelevant background point clouds according to the preset spatial truncation threshold, and retain the point cloud data of the effective area where the carriage and the internal material pile are located; (1.2) Spatiotemporal alignment and overlay are performed on the point cloud data after direct filtering using a multi-frame point cloud weighted fusion strategy based on a time sliding window; (1.3) Perform voxel mesh downsampling on the fused point cloud data to reduce the number of point clouds while preserving the geometric contours of the carriage and the material pile to the maximum extent, so as to obtain the preprocessed three-dimensional point cloud data.

[0025] In a preferred embodiment of the present invention, step (2) specifically includes the following steps: (2.1) Using the random sampling consistency algorithm with normal vector constraints, the floor plane of the carriage is identified by fitting the preprocessed three-dimensional point cloud data in the empty carriage state; (2.2) Using the plane of the car floor as a reference, calculate the normal vector of the floor, construct a unified coordinate system for the car, and normalize the point cloud of the entire car to the unified coordinate system of the car; (2.3) Project the point cloud of the base plate onto a two-dimensional plane, calculate the length and width of the carriage using the minimum bounding rectangle algorithm, and take the maximum value of the point cloud in the vertical direction as the height of the carriage; (2.4) By tracking the rear panel of the carriage in real time, the rear panel plane is fitted using the random sampling consistency algorithm, and the center point of the rear panel plane is used as the positioning reference to calculate the real-time distance from the rear panel to the center of the discharge port.

[0026] In a preferred embodiment of the present invention, step (3) specifically includes the following steps: (3.1) Using the rear panel plane of the car body detected during the empty car stage as a reference, the current loading point cloud and the empty car body point cloud are registered and calculated, and the current loading point cloud is transformed into the car body coordinate system to complete the coordinate alignment. (3.2) Construct a physical space constraint body for the car based on the detected car length, car width and car height. Combine the three-dimensional point cloud after coordinate alignment with the base plate reference surface to perform spatial trimming, remove the points outside the car and the baffle points, and retain the point cloud data of the material pile inside the car.

[0027] In a preferred embodiment of the present invention, step (4) specifically includes the following steps: (4.1) Construct a KD tree for the incomplete point cloud data of the material pile to perform fast index neighborhood search, and perform local surface smoothing fitting by moving least squares method; (4.2) Using the smoothed discrete point cloud data and its normal vector as input, the Poisson equation is solved by constructing an implicit function, and its isosurface is extracted to generate a three-dimensional surface model of a continuous surface. (4.3) The three-dimensional surface model is re-discretized into a point cloud by sampling the centroid coordinates, and the discretized point cloud is constrained and trimmed by combining the prior geometric structure of the carriage, and the over-completion points that exceed the physical boundary of the carriage are filtered out to obtain the complete material pile point cloud data.

[0028] In a preferred embodiment of the present invention, step (5) specifically includes the following steps: (5.1) Perform Delaunay triangulation on the surface of the completed stockpile point cloud data to generate a triangular mesh composed of multiple triangular faces; (5.2) Project the triangular faces in the triangular mesh onto the reference plane of the car floor to form triangular prisms, calculate the volume of each triangular prism and sum them up to obtain the total volume; (5.3) Multiply the total volume by the preset average bulk density of the bulk material to obtain the real-time loading weight data.

[0029] Preferably, step (6) specifically includes the following steps: (6.1) Transmit the real-time loading weight data and make a real-time judgment in combination with the preset system inertial lead amount; (6.2) If the real-time loading weight meets the condition of the target weight minus the system inertial lead, a stop command is triggered to complete the automatic quantitative loading of bulk materials.

[0030] The present invention relates to a device for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion, wherein the device comprises: A processor is configured to execute computer-executable instructions; The memory stores one or more computer-executable instructions, which, when executed by the processor, implement the various steps of the method for quantitative loading of bulk materials based on 3D perception and point cloud completion.

[0031] The processor of the present invention for realizing quantitative loading of bulk materials based on three-dimensional perception and point cloud completion is configured to execute computer-executable instructions. When the computer-executable instructions are executed by the processor, the various steps of the above-mentioned method for realizing quantitative loading of bulk materials based on three-dimensional perception and point cloud completion are implemented.

[0032] The computer-readable storage medium of the present invention stores a computer program thereon, which can be executed by a processor to implement the various steps of the above-described method for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion.

[0033] This invention discloses a method, system, and apparatus for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion. The method includes: acquiring raw point cloud data of the target vehicle and bulk materials using a three-dimensional lidar; preprocessing and parsing the data to determine the dimensions and position of the target vehicle's cargo compartment; segmenting and extracting point cloud data of the material pile inside the cargo compartment based on the cargo compartment data; generating a loading detection command when a loading operation start signal is detected, and acquiring the point cloud of the incomplete material pile under occlusion conditions; performing Poisson surface reconstruction and completion processing on the incomplete material pile point cloud, and obtaining real-time loading weight data by combining Delaunay triangulation and projection integration methods; dynamically matching loading control parameters and completing data interaction between the perception module and the PLC control module based on the real-time weight data and changes in the material pile morphology, thereby achieving quantitative loading.

[0034] The present invention provides a method for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion, comprising the following steps: 1. 3D point cloud data acquisition A 3D LiDAR is horizontally fixed in the upper left of the bulk material loading area. The LiDAR is used to scan the vehicle and its surrounding environment in real time in the target scene to obtain the original 3D point cloud data containing the vehicle and background objects.

[0035] 2. 3D point cloud data preprocessing The point cloud is subjected to pass-through filtering, point cloud fusion, and voxel downsampling to remove noise and redundant points, thereby improving the point cloud quality. The specific steps are as follows: Straight-through filtering: Based on the fixed area of ​​the loading operation, spatial cutoff thresholds are set for the X, Y, and Z dimensions to directly filter out irrelevant background point clouds such as the ground, distant equipment, and walls, and only retain the point clouds of the effective area where the car and internal material pile are located, thereby reducing interference from invalid data. Multi-frame point cloud fusion: A multi-frame point cloud weighted fusion strategy based on a time sliding window is adopted to perform spatiotemporal alignment and overlay of consecutive multi-frame point clouds, thereby improving the integrity and coverage of the point cloud. Voxel downsampling: Voxel mesh downsampling is performed on the fused point cloud to reduce the number of point clouds and the amount of computation while preserving the geometric contours of the carriage and the material pile to the maximum extent.

[0036] In summary, the 3D point cloud data preprocessing involves: parsing the acquired PCAP format raw point cloud into PLY format; converting the angle and distance data from the spherical coordinate system to the XYZ coordinate system in Cartesian coordinates to achieve 3D point cloud reconstruction; removing the background through pass-through filtering while retaining the effective range of X, Y, and Z points; employing a multi-frame fusion algorithm based on spatiotemporal weights to improve the problem of missing points in a single frame and enhance the integrity of the point cloud data; and using voxel downsampling to preserve the geometric structure and reduce computational load. 3. Inspection of the structural dimensions and positions of the carriage. For the preprocessed point cloud of the empty carriage, the RANSAC algorithm with normal vector constraints is used to fit the floor and identify the carriage floor plane. Using the detected carriage floor as a reference, the floor normal vector is calculated to construct a coordinate system. The floor height is normalized to 0, and a unified coordinate system for the carriage is constructed to achieve point cloud coordinate normalization. Then, the floor point cloud is projected onto a two-dimensional plane, and the length and width of the carriage are calculated using the minimum bounding rectangle algorithm. When calculating the height of the carriage, the maximum value in the Z-axis direction of the point cloud is taken as the final effective height of the carriage, so as to achieve accurate carriage size measurement.

[0037] By tracking the rear panel of the carriage in real time and using RANSAC to fit the plane, the current position of the vehicle is dynamically updated to ensure that the discharge port is always accurately positioned relative to the carriage. The center point of the rear panel plane is set as the longitudinal positioning reference of the vehicle. By calculating the real-time distance from the rear panel to the center of the discharge port, it is determined whether the vehicle is parked in place.

[0038] 4. Point cloud extraction of material pile based on prior constraints of carriage structure and point cloud registration Using the rear panel of the truck bed as a stable geometric feature, point cloud registration is performed on the rear panel to align the current loaded point cloud with the empty truck bed point cloud. Based on the detected length, width, and height of the truck bed, a physical space constraint body is constructed. Combined with the base plate reference plane, the 3D point cloud is spatially trimmed, removing points outside the truck bed and points on the rear panel, retaining only the point cloud inside the truck bed to obtain a clean material pile point cloud, thus solving the problems of occlusion and mixing.

[0039] 5. Point cloud completion of material piles based on surface reconstruction and geometric constraints Because the lidar's field of view is fixed, the point cloud of the material pile has varying degrees of incompleteness, and directly calculating the volume will produce significant deviations. To address this, KD tree neighborhood search and MLS moving least squares smoothing optimization are performed on the incomplete point cloud of the material pile; under the constraints of the carriage's geometry, a continuous surface is generated using Poisson surface reconstruction; and finally, a complete point cloud of the material pile with no incompleteness and uniform density is obtained through centroid coordinate sampling.

[0040] Specifically, this invention proposes a surface reconstruction-based point cloud completion method based on prior knowledge of carriage geometry, with the following steps: Step 1: KD-tree neighborhood search and MLS smoothing optimization A KD tree is constructed to quickly index the defect cloud of the material pile residue, and the local surface is smoothly fitted by moving least squares (MLS). Objective: To correct noise points, optimize point cloud normal vectors, and improve surface continuity.

[0041] Step 2: Poisson Surface Reconstruction Objective: To restore incomplete clouds to a continuous, closed, and complete three-dimensional surface, filling in the occluded and missing areas.

[0042] Principle: The point cloud is transformed into an implicit indicator function using the Poisson equation, and a continuous smooth surface is reconstructed.

[0043] Implementation: Taking discrete point cloud data as input, the Poisson equation is solved by constructing implicit functions, its isosurfaces are extracted, and finally a continuous surface model composed of triangular meshes is generated.

[0044] Step 3: Uniform resampling under carriage geometry constraints Objective: To avoid "over-completion" and ensure that the completion result is strictly located inside the carriage to guarantee physical rationality.

[0045] Implementation: After randomly and uniformly sampling the centroid coordinates to re-discretize the continuous surface into a point cloud, the point cloud is constrained and trimmed based on the prior geometry of the carriage to obtain the final completion result.

[0046] 6. Calculation of stockpile weight Objective: To perform high-precision volume and weight calculations on irregular material piles to meet the requirements of quantitative loading and metering in industrial applications.

[0047] Implementation: Delaunay triangulation is performed on the surface of the irregular material pile; the triangular mesh is projected onto the reference plane of the car floor to form triangular prisms; the volume of each triangular prism is calculated by integral of the triangular prisms and the total volume is obtained by summing them; the weight is converted by combining the bulk density, weight = volume × bulk density, and the weight data is output in real time.

[0048] 7. Result Output Based on the real-time calculation of the material pile weight, a stop command is issued, and the PLC is activated to complete the quantitative loading.

[0049] In specific embodiments of the present invention, the technical solution of the present invention will be described in detail below with reference to specific examples.

[0050] 1. 3D point cloud data acquisition Hardware deployment: The XunTeng RS-Bpearl LiDAR is fixedly installed on the left side of the road where vehicles pass to ensure that the scanning field of view covers the entire height of the vehicle and the surrounding environment.

[0051] The lidar is connected to an industrial computer via Ethernet to transmit point cloud data to the processing unit in real time.

[0052] Data acquisition process: When the vehicle enters the scanning area, the LiDAR performs a 3D scan of the target scene at a frequency of 10Hz to acquire raw point cloud data in PCAP format. The raw data includes the 3D coordinates (X, Y, Z) and reflection intensity of each point.

[0053] 2. 3D point cloud data preprocessing Straight-through filtering: Set the three-dimensional coordinate range as , , Extract the effective point cloud data within the operating area; Multi-frame point cloud fusion: Four frames of point cloud data from a continuous time series were selected for processing. The first three frames were used as reference benchmarks. By introducing spatiotemporal weights, the geometric information of the historical frames was transformed and mapped to the coordinate system of the current frame, ultimately constructing a more information-rich fused point cloud.

[0054] Voxel downsampling: The 3D point cloud space is divided into voxel grids of equal size, with a voxel threshold of 0.1m. A point is selected within each voxel to represent the distribution of the point set therein, and the remaining redundant points are discarded.

[0055] 3. Inspection of the structural dimensions and positions of the carriage. Perform RANSAC plane fitting with normal vector constraints on the point cloud of an empty carriage: (1) Randomly select 3 points to fit the plane, and calculate the angle between the plane normal vector and the Z-axis; (2) Only planes with an included angle < 15° are retained; (3) Set the number of iterations to 100, the distance threshold to 0.2m, and output the optimal base plane.

[0056] A unified coordinate system for the carriage is established with the center point of the base plate as the origin and the base plate normal vector as the Z-axis. The point cloud of the entire vehicle is then normalized to this coordinate system.

[0057] Project the point cloud of the base plate onto the XY plane, calculate the minimum bounding rectangle to obtain the length L and width W of the carriage; and calculate the maximum value of the point cloud along the Z axis to obtain the height H of the carriage.

[0058] Positioning is achieved by using the Y-coordinate of the center point of the truck's tailgate as the longitudinal position of the vehicle.

[0059] 4. Point cloud extraction of material pile based on prior constraints of carriage structure and point cloud registration Let the rear bumper plane detected during the empty vehicle phase be... Its normal vector is The center of the plane is During the loading stage, perform planar detection again on the current point cloud to obtain the back baffle plane. , thus obtaining the normal vector and the center point of the plane Calculate the rotation matrix by aligning the normal vectors. and translation vector Therefore, the rigid body transformation matrix can be obtained. Finally, the current loading point cloud is transformed into the car coordinate system, and the point cloud registration is completed. After the transformation, the point clouds acquired at different times are all unified into the car coordinate system established during the empty car stage.

[0060] Cut to fit the space constraints of the carriage: , , ,in, , These are the x and y coordinates of the center point of the carriage floor in the world coordinate system. The maximum superelevation is set to 0.2m.

[0061] 5. Point cloud completion of material piles based on surface reconstruction and geometric constraints Step 1: KD Tree Neighborhood Search and MLS Smoothing Optimization. The input point cloud of the material pile is smoothed and optimized, and the normal vector is calculated. Specific parameters are: the neighborhood search radius is set to 0.4m, and the maximum number of neighborhood points for normal vector estimation is set to 30.

[0062] Step 2: Poisson Surface Reconstruction. The smoothed point cloud and its normal vectors are used for implicit surface reconstruction, with the Poisson reconstruction depth set to 8. This parameter controls the octree partitioning level; a depth of 8 maintains high computational efficiency while ensuring the integrity of the material pile surface details, making it suitable for reconstructing continuous, smooth, and closed surfaces of bulk material piles.

[0063] Step 3: Uniform resampling under carriage geometry constraints. Uniformly sample the reconstructed triangular mesh to generate a completed point cloud, with 8 sampling points per triangle. Filter out "over-complete" points that exceed the spatial boundary of the original material pile point cloud. The final output is a complete, smooth, uniform, and physically sound completed material pile point cloud for subsequent volume calculations.

[0064] 6. Calculation of stockpile weight Triangulation: Based on the distribution characteristics of the bulk point cloud in this paper, the Delaunay triangulation is directly constructed using a stochastic incremental method.

[0065] Projection integral: Project the triangular facets onto the reference plane of the car floor to form a triangular prism. The sum of the volumes of all the triangular prisms is the volume of the material pile.

[0066] Weight conversion: The bulk density of bulk materials in their natural stacking state is approximately constant. The average bulk density of the processed bulk materials is taken as a constant value. Weight = Volume × Density.

[0067] 7. Results Output and Application The real-time loading weight is transmitted to the on-site PLC via TCP protocol, triggering the precise control logic of the loading equipment and automating the loading process.

[0068] Example verification: Test environment: A LiDAR system was installed in the bulk material loading area of ​​a company for on-site testing. The dimensions of the truck body structure were measured as follows: Figure 2 As shown, the position detection of the rear panel of the carriage is as follows: Figure 3 As shown, the point cloud of the material pile is obtained as follows: Figure 4 As shown, the point cloud completion of the material pile is as follows: Figure 5 As shown, the load measurement results before and after completion are as follows: Figure 6 As shown.

[0069] This technical solution is applied to an open "bulk material loading scenario," and one of its core technical features is the perception of dynamically entering truck bodies. This solution proposes a RANSAC algorithm with normal vector constraints, which can effectively eliminate interference from side walls and dust, enabling the autonomous construction and positioning of the truck body's coordinate system.

[0070] The point cloud completion method combining KD-trees with MLS smoothing, Poisson surface reconstruction, and carriage constraint resampling in this technical solution, as well as the technique of combining Delaunay triangulation with projection integral measurement and PLC-based loading, are not conventional techniques used by those skilled in the art. Although algorithms such as KD-trees and Poisson reconstruction are known tools in the field of computer vision, the combination logic of these algorithms and the collaborative improvement for specific industrial problems in this technical solution are innovative. Conventional Poisson reconstruction can lead to "over-closure," causing the completed point cloud to exceed the actual physical boundaries. This technical solution creatively proposes "uniform resampling under prior constraints of the carriage geometry," which uses the detected physical dimensions of the carriage walls as hard boundaries to perform real-time cropping of the completed point cloud, thus solving the "overflow" miscalculation problem caused by severe occlusion during the loading process.

[0071] This technical solution integrates a 0.15t control compensation lead with PLC linkage. This approach is based on the deep coupling decision of the inertia of falling bulk materials, the system's multi-threaded processing delay, and high-precision measurement results, achieving quantitative accuracy of "measuring while loading and dynamic compensation", and realizing industrial control from "sensing" to "quantitative execution".

[0072] This technical solution uses Poisson surface reconstruction, which infers a continuous smooth surface that conforms to physical properties by solving the Poisson equation, and can more accurately restore the missing shape of the raw material pile.

[0073] This technical solution uses the rear baffle, which is least affected by interference, as a reference to achieve dynamic alignment, ensuring that volume calculations are still based on a unified coordinate system even in the later stages when the base plate is obscured.

[0074] The core value of this technical solution lies in "single radar + algorithm completion", which uses Poisson reconstruction technology to "infer" invisible areas caused by limited viewing angle.

[0075] This technical solution compensates for the deficiencies of the hardware perspective through software algorithms, achieving high-precision measurement at low hardware cost, which is a completely different technical approach.

[0076] In the high-dust, high-vibration environment of bulk material loading, point cloud data suffers from severe non-uniformity and missing values. This technical solution deeply integrates KD tree neighborhood search, MLS smoothing, and Poisson reconstruction to extract stable topological features under dynamic, non-rigid material pile morphology, rather than relying on general 3D modeling. Its computational efficiency meets the requirements of industrial real-time control. The Delaunay triangulation combined with projection integration method employed in this solution has been experimentally verified to maintain a relative error within 3%. A 0.15t lead compensation addresses the system inertial error caused by the delayed characteristics of bulk material flow, thus constituting a complete industrial solution for quantitative loading.

[0077] For the specific implementation scheme of this embodiment, please refer to the relevant descriptions in the above embodiments, which will not be repeated here.

[0078] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0079] It should be noted that in the description of this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means at least two.

[0080] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0081] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution device. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0082] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The corresponding program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0083] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0084] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0085] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0086] The present invention employs a method, apparatus, processor, and computer-readable storage medium for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion. It can acquire point cloud information of the truck body and material pile by relying on three-dimensional perception technology, and combine point cloud completion to repair occlusion and missing data, accurately calculate the real-time loading weight, ensure automated quantitative loading of bulk materials, and effectively solve the problems of large measurement error, occlusion measurement distortion, and low level of intelligence in traditional loading.

[0087] In this specification, the invention has been described with reference to specific embodiments thereof. However, it will be apparent that various modifications and variations can be made without departing from the spirit and scope of the invention. Therefore, the specification and drawings should be considered illustrative rather than restrictive.

Claims

1. A method for quantitative loading of bulk materials based on 3D perception and point cloud completion, characterized in that, The method includes the following steps: (1) Collect the original three-dimensional point cloud data of the target vehicle and bulk materials, and preprocess the original three-dimensional point cloud data; (2) Analyze the preprocessed 3D point cloud data to determine the vehicle compartment size and location data of the target vehicle; (3) Based on the car body size data and location data, segment and extract the point cloud data of the material pile inside the car body; (4) Perform Poisson surface reconstruction and completion processing on the residual point cloud data of the material pile to generate complete point cloud data of the material pile; (5) The complete material pile point cloud data is processed by combining Delaunay triangulation and projection integration to obtain real-time loading weight data; (6) Based on the real-time loading weight data, the linkage control module performs dynamic compensation to complete the quantitative loading.

2. The method for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion according to claim 1, characterized in that, The preprocessing of the original 3D point cloud data in step (1) specifically includes the following steps: (1.1) Perform pass-through filtering on the original three-dimensional point cloud data, filter out irrelevant background point clouds according to the preset spatial truncation threshold, and retain the point cloud data of the effective area where the carriage and the internal material pile are located; (1.2) Spatiotemporal alignment and overlay are performed on the point cloud data after direct filtering using a multi-frame point cloud weighted fusion strategy based on a time sliding window; (1.3) Perform voxel mesh downsampling on the fused point cloud data to reduce the number of point clouds while preserving the geometric contours of the carriage and the material pile to the maximum extent, so as to obtain the preprocessed three-dimensional point cloud data.

3. The method for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion according to claim 1, characterized in that, Step (2) specifically includes the following steps: (2.1) Using the random sampling consistency algorithm with normal vector constraints, the floor plane of the carriage is identified by fitting the preprocessed three-dimensional point cloud data in the empty carriage state; (2.2) Using the plane of the car floor as a reference, calculate the normal vector of the floor, construct a unified coordinate system for the car, and normalize the point cloud of the entire car to the unified coordinate system of the car; (2.3) Project the point cloud of the base plate onto a two-dimensional plane, calculate the length and width of the carriage using the minimum bounding rectangle algorithm, and take the maximum value of the point cloud in the vertical direction as the height of the carriage; (2.4) By tracking the rear panel of the carriage in real time, the rear panel plane is fitted using the random sampling consistency algorithm, and the center point of the rear panel plane is used as the positioning reference to calculate the real-time distance from the rear panel to the center of the discharge port.

4. The method for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion according to claim 1, characterized in that, Step (3) specifically includes the following steps: (3.1) Using the rear panel plane of the car body detected during the empty car stage as a reference, the current loading point cloud and the empty car body point cloud are registered and calculated, and the current loading point cloud is transformed into the car body coordinate system to complete the coordinate alignment. (3.2) Construct a physical space constraint body for the car based on the detected car length, car width and car height. Combine the three-dimensional point cloud after coordinate alignment with the base plate reference surface to perform spatial trimming, remove the points outside the car and the baffle points, and retain the point cloud data of the material pile inside the car.

5. The method for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion according to claim 1, characterized in that, Step (4) specifically includes the following steps: (4.1) Construct a KD tree for the incomplete point cloud data of the material pile to perform fast index neighborhood search, and perform local surface smoothing fitting by moving least squares method; (4.2) Using the smoothed discrete point cloud data and its normal vector as input, the Poisson equation is solved by constructing an implicit function, and its isosurface is extracted to generate a three-dimensional surface model of a continuous surface. (4.3) The three-dimensional surface model is re-discretized into a point cloud by sampling the centroid coordinates, and the discretized point cloud is constrained and trimmed by combining the prior geometric structure of the carriage, and the over-completion points that exceed the physical boundary of the carriage are filtered out to obtain the complete material pile point cloud data.

6. The method for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion according to claim 1, characterized in that, Step (5) specifically includes the following steps: (5.1) Perform Delaunay triangulation on the surface of the completed stockpile point cloud data to generate a triangular mesh composed of multiple triangular faces; (5.2) Project the triangular faces in the triangular mesh onto the reference plane of the car floor to form triangular prisms, calculate the volume of each triangular prism and sum them up to obtain the total volume; (5.3) Multiply the total volume by the preset average bulk density of the bulk material to obtain the real-time loading weight data.

7. The method for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion according to claim 1, characterized in that, Step (6) specifically includes the following steps: (6.1) Transmit the real-time loading weight data and make a real-time judgment in combination with the preset system inertial lead amount; (6.2) If the real-time loading weight meets the condition of the target weight minus the system inertial lead, a stop command is triggered to complete the automatic quantitative loading of bulk materials.

8. A device for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion, characterized in that, The device includes: A processor is configured to execute computer-executable instructions; The memory stores one or more computer-executable instructions, which, when executed by the processor, implement the steps of the method for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion as described in any one of claims 1 to 7.

9. A processor for quantitative loading of bulk materials based on 3D perception and point cloud completion, characterized in that, The processor is configured to execute computer-executable instructions, which, when executed by the processor, implement the steps of the method for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program that can be executed by a processor to implement the various steps of the method for quantitative loading of bulk materials based on three-dimensional perception and point cloud completion as described in any one of claims 1 to 7.

Images