A method and system for autonomous stack-finding control of unloading trolley based on 3D scanning

Abstract

This application provides a method and system for autonomous stacking control of a 3D scanning unloading trolley. It involves acquiring temporal 3D point cloud frames containing the silo retaining wall and the material pile below, using the plane along the upper edge of the retaining wall in the first acquired point cloud frame as the initial reference plane. The point cloud along the upper edge of the retaining wall in subsequent point cloud frames is extracted, and the spatial pose change of the point cloud along the upper edge of the retaining wall in each frame relative to the initial reference plane is calculated and decomposed into a global motion component caused by the movement of the unloading trolley and a local distortion component caused by the vibration of the unloading trolley body. The fluctuation amplitude of the local distortion component is compared with a preset vibration baseline. When the fluctuation amplitude exceeds the vibration baseline, the distortion frame interval in the temporal 3D point cloud frame where motion blur occurs is identified based on the local distortion component. The target stacking point of the unloading trolley is identified within the distortion frame interval. In the technical solution provided in this application, the active material accumulation zone can be identified under the influence of local non-rigid distortion in the 3D point cloud.

Description

Technical Field

[0001] This application relates to the field of unloading control technology, and more specifically, to a method and system for autonomous stacking control of an unloading trolley based on three-dimensional scanning. Background Technology

[0002] In the industrial production of powder, granular and lumpy materials, unloading control is a key technology connecting storage, conveying and processing. Its goal is to achieve precise flow regulation and reliable start-stop while ensuring continuous and uniform material flow, and to prevent bridging, spraying or leakage. Traditional unloading relies on manual gates or fixed-speed equipment, which is difficult to adapt to process fluctuations. Modern unloading control integrates weighing sensors, variable frequency speed regulation and intelligent algorithms to achieve closed-loop control of mass flow rate or volumetric flow rate.

[0003] In existing unloading control, the unloading control first relies on weighing modules, level switches, or flow sensors to collect mass, level, or speed signals in real time during the unloading process. The controller compares the measured values ​​with the preset target flow rate value, calculates the deviation, and outputs adjustment commands according to the PID algorithm to adjust the opening of the unloading valve or the rotation speed of the feeder. However, in the autonomous stacking control of the unloading trolley based on 3D scanning, traditional methods usually identify the material stack based on the overall transformed point cloud. In actual working conditions, when the unloading trolley moves on the track, the gap between the wheels and the track, the impact of mechanical transmission, and the unevenness of the ground will induce non-negligible high-frequency low-amplitude vibrations in the vehicle body. This vibration causes non-uniform and non-rigid disturbances in the measurement coordinate system of the vehicle-mounted 3D scanner, resulting in local non-rigid distortion of the 3D point cloud. This makes it impossible for the unloading trolley to accurately identify the active material accumulation area. Therefore, how to identify the active material accumulation area under the influence of local non-rigid distortion of the 3D point cloud has become a difficult problem for the industry. Summary of the Invention

[0004] This application provides a method and system for autonomous stacking control of unloading trolley based on three-dimensional scanning, which can identify the active material stacking area under the influence of local non-rigid distortion generated by three-dimensional point cloud.

[0005] In a first aspect, this application provides an autonomous stacking control method for a discharge trolley based on three-dimensional scanning, comprising the following steps: Start the 3D scanner to continuously acquire time-series 3D point cloud frames containing the silo retaining wall and the material pile below, and use the plane above the retaining wall marked in the first acquired point cloud frame as the initial reference plane. Extract the point cloud above the retaining wall in each subsequent point cloud frame, calculate the spatial pose change of the point cloud above the retaining wall in each frame relative to the initial reference plane, and decompose the corresponding spatial pose change into a global motion component caused by the movement of the unloading trolley and a local distortion component caused by the vibration of the unloading trolley body. The fluctuation amplitude of the local distortion component is compared with a preset vibration baseline. When the fluctuation amplitude exceeds the vibration baseline, the distortion frame interval in the temporal three-dimensional point cloud frame where motion blur occurs is identified based on the local distortion component. Within the distorted frame interval, the spatial coordinate fluctuation sequence of the material pile surface points in the temporal three-dimensional point cloud frame is extracted. The active material accumulation area is identified through the spatial coordinate fluctuation sequence, and the projection center of the active material accumulation area on the horizontal plane is determined as the target pile-finding point of the unloading trolley.

[0006] In some embodiments, starting a 3D scanner to continuously acquire temporal 3D point cloud frames including the silo retaining wall and the material pile below, and using the plane above the retaining wall marked in the first acquired point cloud frame as the initial reference plane specifically includes: After the unloading trolley is moved to the preset starting scanning position, the 3D scanner is triggered to acquire the first frame of 3D point cloud data. The point cloud of the retaining wall area located on both sides of the scanning field of view is segmented from the first frame of 3D point cloud data based on the differences in geometric features between the retaining wall and the material pile. The edge point cloud that is in contact with the air is extracted from the upper part of the point cloud in each retaining wall area as the upper edge point cloud of the retaining wall. The point cloud along the extracted retaining wall is fitted to a spatial plane using the least squares method, and the plane equation coefficients of this spatial plane in the vehicle coordinate system are used as the initial reference plane.

[0007] In some embodiments, extracting the point cloud along the upper edge of the retaining wall in subsequent point cloud frames and calculating the spatial pose change of the point cloud along the upper edge of the retaining wall relative to the initial reference plane in each frame specifically includes: For each subsequent frame in the temporal 3D point cloud frame that is located after the first acquisition frame, the same segmentation method as that used to extract the point cloud along the upper edge of the retaining wall in the first frame is used to extract the current point cloud along the upper edge of the retaining wall from the subsequent frame. The point cloud along the current retaining wall is rigidly registered with the spatial plane containing the initial reference plane, and the optimal rotation matrix and optimal translation vector are solved to transform the point cloud along the current retaining wall to coincide with the initial reference plane. The optimal rotation matrix and the optimal translation vector are combined to form the spatial pose change of the point cloud along the frame retaining wall relative to the initial reference plane.

[0008] In some embodiments, decomposing the corresponding spatial pose change into a global motion component caused by the movement of the unloading trolley and a local distortion component caused by the vibration of the unloading trolley body specifically includes: The rotation matrix and translation vector in the spatial pose change are directly determined as the global motion components caused by the movement of the unloading trolley. The transformed point cloud is obtained by performing a rigid body transformation on the current retaining wall edge point cloud according to the global motion components. The signed distance from each point in the transformed point cloud to the initial reference plane is calculated, and the set of all signed distances is taken as the local distortion component caused by the vibration of the unloading trolley body.

[0009] In some embodiments, comparing the fluctuation amplitude of the local distortion component with a preset vibration baseline specifically includes: The root mean square value of all signed distances in the local distortion component is calculated as the current fluctuation amplitude; Read the pre-stored vibration baseline threshold, determine whether the current fluctuation amplitude is greater than the vibration baseline threshold, and if so, determine that there is abnormal vibration caused by material impact in the current unloading area.

[0010] In some embodiments, when the fluctuation amplitude exceeds the vibration baseline, identifying the distorted frame interval in the temporal 3D point cloud frame where motion blur occurs based on the local distortion component specifically includes: The difference between the maximum and minimum signed distances in the local distortion components is calculated as the distortion level value of the current frame; Determine whether the distortion level value exceeds a preset distortion threshold; if it does, mark the current frame as a distorted frame. Traverse each subsequent frame in the temporal 3D point cloud frame that is located after the first acquisition frame, and determine the starting frame number to the ending frame number corresponding to the consecutive frame sequence marked as distorted frames as the distorted frame interval.

[0011] In some embodiments, extracting the spatial coordinate fluctuation sequence of points on the surface of the material pile in the temporal three-dimensional point cloud frame within the distorted frame interval specifically includes: Within the distorted frame interval, the point cloud on the surface of the material pile is segmented from each frame of three-dimensional point cloud data. Using the point cloud of the material pile surface in the starting frame within the distorted frame interval as the reference point cloud, a point-to-point correspondence is established between the point cloud of the material pile surface in each subsequent frame and the point cloud of the material pile surface in the previous frame using the nearest neighbor search algorithm, thereby obtaining a set of matching point pairs between each pair of adjacent frames. Extract the spatial coordinate difference of each matching point pair from the set of matching point pairs of each pair of adjacent frames, arrange the spatial coordinate difference in order of frame number, and take the total set of spatial coordinate differences of matching point pairs of all frames in the distorted frame interval as the spatial coordinate fluctuation sequence.

[0012] Secondly, this application provides a three-dimensional scanning-based autonomous stack-finding control system for unloading trolleys, used to execute a three-dimensional scanning-based autonomous stack-finding control method for unloading trolleys. The system includes: The acquisition module is used to start the 3D scanner to continuously acquire time-series 3D point cloud frames containing the silo retaining wall and the material pile below, and to use the plane above the retaining wall marked in the first acquired point cloud frame as the initial reference plane. The processing module is used to extract the point cloud above the retaining wall in each subsequent point cloud frame, calculate the spatial pose change of the point cloud above the retaining wall in each frame relative to the initial reference plane, and decompose the corresponding spatial pose change into a global motion component caused by the movement of the unloading trolley and a local distortion component caused by the vibration of the unloading trolley body. The processing module is also used to compare the fluctuation amplitude of the local distortion component with a preset vibration baseline. When the fluctuation amplitude exceeds the vibration baseline, the distortion frame interval in the temporal three-dimensional point cloud frame where motion blur occurs is identified based on the local distortion component. The execution module is used to extract the spatial coordinate fluctuation sequence of the material pile surface points in the time-series three-dimensional point cloud frame within the distorted frame interval, identify the active material accumulation area through the spatial coordinate fluctuation sequence, and determine the projection center of the active material accumulation area on the horizontal plane as the target pile-finding point of the unloading trolley.

[0013] Thirdly, this application provides a computer device, the computer device including a memory and a processor, the memory storing code, the processor being configured to acquire the code and execute the above-described autonomous stacking control method for unloading trolleys based on three-dimensional scanning.

[0014] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described autonomous stacking control method for a three-dimensional scanning unloading trolley.

[0015] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects: The autonomous stacking control method and system for unloading trolleys based on 3D scanning provided in this application firstly starts a 3D scanner to continuously acquire temporal 3D point cloud frames containing the hopper retaining wall and the material pile below. The plane along the upper edge of the retaining wall calibrated in the first acquired point cloud frame is used as the initial reference plane. Secondly, the point cloud along the upper edge of the retaining wall in each subsequent point cloud frame is extracted, and the spatial pose change of the point cloud along the upper edge of the retaining wall in each frame relative to the initial reference plane is calculated. The corresponding spatial pose change is decomposed into a global motion component caused by the movement of the unloading trolley and a local component caused by the vibration of the unloading trolley body. Distortion components; then, the fluctuation amplitude of the local distortion components is compared with a preset vibration baseline. When the fluctuation amplitude exceeds the vibration baseline, the distortion frame interval in the temporal three-dimensional point cloud frame where motion blur occurs is identified based on the local distortion components; finally, within the distortion frame interval, the spatial coordinate fluctuation sequence of the material pile surface points in the temporal three-dimensional point cloud frame is extracted, the active material accumulation area is identified through the spatial coordinate fluctuation sequence, and the projection center of the active material accumulation area on the horizontal plane is determined as the target pile-finding point of the unloading trolley.

[0016] Therefore, this application can identify the active material accumulation area under the influence of local non-rigid distortion in 3D point clouds. First, by using the plane above the retaining wall calibrated in the first acquired point cloud frame as the initial reference plane, a fixed and physically stationary reference plane can be provided for the spatial comparison of all subsequent frames, eliminating the influence of the absolute pose change of the unloading trolley at different times on the consistency of point cloud data. Second, by extracting the point cloud above the retaining wall in each subsequent frame and calculating its spatial pose change relative to the initial reference plane, and then decomposing the change into global motion components and local distortion components, quantitative separation of the normal movement of the unloading trolley and the vibration interference of the vehicle body is achieved. This allows the non-rigid distortion caused by vibration to be independently extracted from the overall motion, avoiding the interference of trolley movement on vibration detection, and providing a pure distortion feature quantity for the accurate determination of subsequent abnormal vibrations. Then, by comparing the fluctuation amplitude of the local distortion component with the preset vibration baseline, and in the super-distortion component... When the baseline is established, the distorted frame intervals where motion blur occurs are identified, enabling adaptive detection and temporal localization of abnormal vibrations. This automatically filters out continuous frame ranges significantly affected by vibrations, eliminating interference from frames with no or weak distortion, effectively reducing computational redundancy in subsequent fluctuation analysis. Finally, by extracting the spatial coordinate fluctuation sequence of points on the material pile surface within the distorted frame intervals and using this sequence to identify the active material accumulation area, the horizontal projection center of this area is ultimately determined as the target accumulation point. This achieves the ability to inversely transform the vibration-induced point cloud distortion signal into a perception of the dynamic accumulation position of the material, directly capturing the flow and accumulation trend of the material under vibration excitation. This overcomes the technical deficiency of traditional static accumulation methods that cannot accurately locate the true pile height under vibration conditions, thereby improving the autonomous accumulation accuracy and robustness of the unloading trolley in a strong interference environment. In summary, the technical solution provided in this application can identify the active material accumulation area under the influence of local non-rigid distortion in three-dimensional point clouds. Attached Figure Description

[0017] Figure 1 This is an exemplary flowchart of an autonomous stacking control method for a 3D scanning-based unloading trolley, as shown in some embodiments of this application. Figure 2 This is an exemplary flowchart illustrating the determination of spatial pose changes according to some embodiments of this application; Figure 3 This is a schematic diagram of the structure of an autonomous stacking control system for a three-dimensional scanning unloading trolley, as shown in some embodiments of this application. Figure 4 This is a schematic diagram of the structure of a computer device for implementing an autonomous stacking control method for a three-dimensional scanning unloading trolley, according to some embodiments of this application. Detailed Implementation

[0018] To better understand the technical solution of this application, the technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] refer to Figure 1 This figure is an exemplary flowchart of an autonomous stacking control method for a 3D scanning-based unloading trolley, according to some embodiments of this application. The figure mainly includes the following steps: In step S101, the 3D scanner is started to continuously acquire time-series 3D point cloud frames containing the silo retaining wall and the material pile below, and the plane above the retaining wall marked in the first acquired point cloud frame is used as the initial reference plane.

[0020] In some embodiments, a 3D scanner is activated to continuously acquire temporal 3D point cloud frames containing the silo retaining wall and the material pile below. The plane along the upper edge of the retaining wall, as calibrated in the first acquired point cloud frame, is used as the initial reference plane, and the following steps are taken: After the unloading trolley is moved to the preset starting scanning position, the 3D scanner is triggered to acquire the first frame of 3D point cloud data. The point cloud of the retaining wall area located on both sides of the scanning field of view is segmented from the first frame of 3D point cloud data based on the differences in geometric features between the retaining wall and the material pile. The edge point cloud that is in contact with the air is extracted from the upper part of the point cloud in each retaining wall area as the upper edge point cloud of the retaining wall. The point cloud along the extracted retaining wall is fitted to a spatial plane using the least squares method, and the plane equation coefficients of this spatial plane in the vehicle coordinate system are used as the initial reference plane.

[0021] In practice, firstly, the unloading trolley moves to the starting scanning position based on the pre-calibrated travel limit feedback. This starting scanning position ensures that the field of view of the 3D scanner completely covers the upper edge of the silo retaining wall. Then, a trigger pulse is sent to the 3D scanner, which performs a full-area scan, generating a first frame of 3D point cloud data containing the silo retaining wall and the internal material pile. This 3D point cloud data consists of spatially discrete points, each described by its 3D coordinates in the sensor coordinate system. Secondly, after acquiring the first frame of 3D point cloud data, point cloud segmentation is performed based on the differences in geometric features between the silo retaining wall and the material pile. Specifically, points in the first frame of 3D point cloud data with consistent normal vector directions and a spatial distance less than a preset threshold are clustered into the same planar region. The vertical planar region where the two retaining walls are located is distinguished from the loose curved surface region where the material pile below is located, segmenting the retaining wall region point cloud located on the left and right sides of the scanning field of view. Here, the normal vector represents the orientation of the local surface of the point cloud, and the distance threshold is used to control the spatial connectivity of the point cloud. The retaining wall region point cloud refers to the point cloud corresponding to the retaining wall structure on both sides of the silo. The system extracts a set of discrete points on the surface, which collectively describe the spatial morphology of the retaining wall within the scanning field of view. Then, it extracts edge point clouds in contact with the air at the top of the point clouds in each retaining wall region as the upper edge point clouds of the retaining wall. Specifically, for each point cloud in the retaining wall region, it extracts boundary points where no other point clouds exist above it within an eight-neighborhood. These boundary points are connected according to spatial adjacency to form continuous edge lines, and the edge point clouds on these edge lines are then used as the upper edge point clouds of the retaining wall. This process utilizes the principle of height-direction neighborhood analysis in point cloud boundary extraction technology. Finally, all extracted upper edge point clouds of the retaining wall are combined into a point set, and the least squares method is used to fit this point set to a spatial plane. The least squares fitting process involves setting a spatial plane equation to be solved, where the coefficients of the plane equation are undetermined constants. The sum of squared signed distances from each point in the point set to the plane is calculated, and the optimal values ​​of each undetermined constant are obtained with the goal of minimizing this sum of squares. This yields the plane equation coefficients of the spatial plane, which are then used as the initial reference plane.

[0022] It should be noted that the initial reference plane in this application refers to the spatial plane orientation of the upper edge of the hopper retaining wall at the start of the unloading trolley's scanning. During the unloading trolley's piling, the hopper retaining wall is a fixed structural component of the hopper. During the unloading process, its spatial position remains stationary relative to the Earth coordinate system and does not undergo actual displacement due to material impact or trolley vibration. Therefore, it can serve as an ideal stationary reference in subsequent dynamic analysis, providing a reliable reference for identifying distorted frame intervals and material active accumulation areas.

[0023] In step S102, the point cloud above the retaining wall in each subsequent point cloud frame is extracted, the spatial pose change of the point cloud above the retaining wall in each frame relative to the initial reference plane is calculated, and the corresponding spatial pose change is decomposed into a global motion component caused by the movement of the unloading trolley and a local distortion component caused by the vibration of the unloading trolley body.

[0024] In some embodiments, reference Figure 2 As shown in the figure, this is an exemplary flowchart of determining the amount of spatial pose change according to some embodiments of this application. In this embodiment, the extraction of the point cloud above the retaining wall in each subsequent point cloud frame and the calculation of the amount of spatial pose change of the point cloud above the retaining wall in each frame relative to the initial reference plane can be achieved by the following steps: In step S1021, for each subsequent frame in the temporal three-dimensional point cloud frame that is located after the first acquisition frame, the same segmentation method as that used to extract the upper edge point cloud of the retaining wall in the first frame is used to extract the upper edge point cloud of the current retaining wall from the subsequent frame. In step S1022, the point cloud along the current retaining wall is rigidly registered with the spatial plane where the initial reference plane is located, and the optimal rotation matrix and optimal translation vector that transform the point cloud along the current retaining wall to coincide with the initial reference plane are solved. In step S1023, the optimal rotation matrix and the optimal translation vector are combined into the spatial pose change of the point cloud along the frame retaining wall relative to the initial reference plane.

[0025] In specific implementation, firstly, for each subsequent frame in the temporal 3D point cloud frame that is after the first acquisition frame, the current retaining wall upper edge point cloud is extracted from the subsequent frame using the same segmentation method as the extraction of the retaining wall upper edge point cloud in the first frame. This will not be elaborated further here. The current retaining wall upper edge point cloud refers to the set of spatially discrete points corresponding to the retaining wall upper edge in the subsequent frame. Secondly, the current retaining wall upper edge point cloud is rigidly registered with the spatial plane containing the initial reference plane. This rigid registration uses the iterative nearest point algorithm, i.e., the current retaining wall upper edge point cloud is set as the target point cloud, and the plane containing the initial reference plane is used as the reference plane. However, due to the lack of sufficient geometric constraints on the plane, in actual implementation, the initial reference plane is discretized into a set of reference points uniformly sampled from the retaining wall upper edge point cloud of the first frame. Then, for each point in the current retaining wall upper edge point cloud, the point with the closest Euclidean distance in the reference point set is found as the corresponding point, forming a point pair, and the... The least-squares singular value decomposition method is used to solve for the rigid body transformation that minimizes the sum of squared distances between corresponding points for all point pairs. This rigid body transformation is described by a rotation matrix and a translation vector. Then, the coordinate transformation of the current wall edge point cloud is performed according to this rigid body transformation, and the above process of finding the nearest point and solving the transformation is repeated until the difference between the two transformations is less than the convergence threshold. The iteration stops when the accumulated rigid body transformation is the optimal rotation matrix and optimal translation vector that transforms the current wall edge point cloud to coincide with the initial reference plane. The optimal rotation matrix describes the rotation angle around the three coordinate axes, and the optimal translation vector describes the translation along the three coordinate axes. Finally, the optimal rotation matrix and optimal translation vector are combined into a 4x4 homogeneous transformation matrix, and this homogeneous transformation matrix is ​​used as the spatial pose change of the current frame wall edge point cloud relative to the initial reference plane.

[0026] It should be noted that the spatial pose change in this application is used to characterize the overall displacement and rotation of the upper edge of the retaining wall in three-dimensional space during the movement of the trolley. As the unloading trolley moves, the 3D scanner moves with the trolley body, and the trolley body itself also experiences vibrations caused by material impact or uneven track. Both of these factors will cause changes in the spatial position and orientation of the point cloud in each subsequent frame relative to the initial reference plane of the first acquisition frame. The upper edge of the retaining wall, as a fixed structural reference, actually exhibits a pose change in the point cloud entirely due to the superposition of the rigid body motion and vibration of the trolley. Therefore, by calculating the spatial pose change between the point cloud of the upper edge of the retaining wall in the current frame and the initial reference plane, the combined effect of these two factors can be quantitatively extracted in the form of a rotation matrix and a translation vector, so as to achieve accurate perception of the active material accumulation area under vibration interference conditions.

[0027] In some embodiments, the decomposition of the corresponding spatial pose change into a global motion component caused by the movement of the unloading trolley and a local distortion component caused by the vibration of the unloading trolley body is achieved by the following steps: The rotation matrix and translation vector in the spatial pose change are directly determined as the global motion components caused by the movement of the unloading trolley. The transformed point cloud is obtained by performing a rigid body transformation on the current retaining wall edge point cloud according to the global motion components. The signed distance from each point in the transformed point cloud to the initial reference plane is calculated, and the set of all signed distances is taken as the local distortion component caused by the vibration of the unloading trolley body.

[0028] In specific implementation, firstly, the optimal rotation matrix and optimal translation vector obtained from rigid registration are used as global motion components caused by the movement of the unloading trolley. These global motion components represent the overall rigid body displacement of the unloading trolley body in three-dimensional space. Then, each point cloud in the current retaining wall's edge point cloud undergoes a rigid body transformation according to the global motion components. Specifically, the original coordinate vector of each point cloud is multiplied sequentially by the rotation matrix and then added to the translation vector to obtain the transformed new coordinates. The point cloud formed by all the transformed new coordinates is called the transformed point cloud. The transformed point cloud represents the spatial distribution remaining after deducting the trolley motion effect in the current frame's edge point cloud. Next, for each point in the transformed point cloud, its signed distance to the initial reference plane is calculated. The initial reference plane is a spatial plane, and the specific calculation method for the signed distance is to substitute the coordinates of the point into the plane equation of the initial reference plane. The plane equation is in the form of the dot product of the normal vector and the point coordinates plus a constant equal to zero. The calculated value is the signed distance, whose sign indicates which side of the plane the point is located on, and the absolute value represents the vertical distance. The signed distance refers to the signed vertical distance from a point in space to the reference plane. Since the initial reference plane is a spatial plane with a clear normal direction, and the upper edge of the retaining wall is a rigid structure physically located on this plane, after deducting the overall motion of the car, the points on the upper edge of the retaining wall should coincide completely with the initial reference plane. At this time, the signed distance is zero. Any deviation caused by vehicle vibration or scanning distortion will cause the point to leave the plane. At this time, the non-zero value of the signed distance and its sign can quantitatively describe the magnitude and direction of the deviation. The signed distances corresponding to each point in the transformed point cloud are combined into a numerical set according to the original order of the points, and this numerical set is used as the local distortion component.

[0029] It should be noted that, in this application, the local distortion component refers to the set of values ​​describing the degree of deviation of each point on the upper edge of the retaining wall relative to the initial reference plane. This deviation originates from the non-rigid deformation or scanning distortion caused by vehicle vibration. Since the upper edge of the actually stationary retaining wall should coincide completely with the initial reference plane, any non-zero signed distance reflects the instantaneous deformation or measurement error caused by vibration. The determination of the local distortion component can provide a time-domain range limit for extracting the spatial coordinate fluctuation sequence of the material pile surface points within the distortion frame interval, thereby achieving accurate capture of the dynamics of material accumulation under vibration interference conditions.

[0030] In step S103, the fluctuation amplitude of the local distortion component is compared with a preset vibration baseline. When the fluctuation amplitude exceeds the vibration baseline, the distortion frame interval in the temporal three-dimensional point cloud frame where motion blur occurs is identified based on the local distortion component.

[0031] In some embodiments, comparing the fluctuation amplitude of the local distortion component with a preset vibration baseline is achieved using the following steps: The root mean square value of all signed distances in the local distortion component is calculated as the current fluctuation amplitude; Read the pre-stored vibration baseline threshold, determine whether the current fluctuation amplitude is greater than the vibration baseline threshold, and if so, determine that there is abnormal vibration caused by material impact in the current unloading area.

[0032] Specifically, the calibration steps for the vibration baseline threshold in this embodiment are as follows: Under the condition that the unloading trolley is unloaded and there is no abnormal impact, the trolley is controlled to move at a constant speed along the track, and at least 200 frames of temporal three-dimensional point cloud are continuously collected; for each frame, the root mean square value of the local distortion component is calculated according to the methods of steps S102 and S103 to obtain a set of fluctuation amplitude samples; the arithmetic mean μ and standard deviation σ of the set of samples are calculated, and the vibration baseline threshold is set to μ+3σ; if the on-site working conditions change, the above calibration process can be repeated to update the threshold, which is not limited here.

[0033] In some embodiments, when the fluctuation amplitude exceeds the vibration baseline, the following steps are used to identify the distorted frame intervals in the temporal 3D point cloud frame that experience motion blur based on the local distortion components: The difference between the maximum and minimum signed distances in the local distortion components is calculated as the distortion level value of the current frame; Determine whether the distortion level value exceeds a preset distortion threshold; if it does, mark the current frame as a distorted frame. Traverse each subsequent frame in the temporal 3D point cloud frame that is located after the first acquisition frame, and determine the starting frame number to the ending frame number corresponding to the consecutive frame sequence marked as distorted frames as the distorted frame interval.

[0034] In practice, firstly, a set of all signed distance values ​​is extracted from the local distortion components corresponding to the current frame. A linear scan is performed on this set to find the maximum and minimum values. The maximum value is the element with the largest value in the set, and the minimum value is the element with the smallest value. Then, the maximum value is subtracted from the minimum value to obtain the difference, which is used as the distortion degree value of the current frame. This distortion degree value characterizes the maximum discrete range of deviation of each point along the retaining wall from the initial reference plane in the current frame, reflecting the extreme amplitude of non-rigid deformation caused by vibration. Then, a distortion threshold, which is a dimensionless distance threshold, is obtained through experimental calibration or empirical setting to distinguish between normal small vibrations and significant distortions sufficient to cause motion blur. The distortion level value of the current frame is compared with the distortion threshold. If the distortion level value is greater than the distortion threshold, the current frame is determined to have unacceptable distortion, and the frame number corresponding to the current frame is recorded in a distortion flag array, marking the current frame as a distorted frame; otherwise, it is marked as a non-distorted frame. Finally, each subsequent frame in the temporal 3D point cloud frame after the first acquisition frame is traversed, and the above operation of calculating the distortion level value and judging and comparing is repeated for each frame to obtain the distortion mark status of each frame. Then, a continuous region scan is performed on the distortion flag array, that is, starting from the first subsequent frame, all sequences in which distortion marks appear consecutively are searched, and the sequence number of the starting frame and the sequence number of the ending frame in each continuous sequence are extracted to determine the distorted frame interval corresponding to that sequence.

[0035] It should be noted that, in this application, the distortion frame interval refers to a range of frames that are continuous in time and all exhibit distortion. Because the abnormal vibration caused by material impact does not affect a single frame point cloud in isolation, but persists within a continuous time window, it causes the unloading trolley to vibrate or shake continuously. Consequently, multiple consecutive frames in the temporal 3D point cloud experience motion blur and spatial distortion. If only the distortion degree value of a single frame is used for analysis, it is easy to make misjudgments due to occasional noise spikes. At the same time, it is impossible to capture the dynamic evolution of material accumulation during the continuous vibration. Therefore, determining the distortion frame interval can provide a precise temporal range limit for the subsequent extraction of the spatial coordinate fluctuation sequence of the material pile surface points, ensuring that the fluctuation analysis is performed only within the frame segment where the vibration is continuous and significant. This avoids mixing normal frames that are not affected by vibration or transition frames with only minor distortion into the analysis, reducing computational redundancy and interference signals.

[0036] In step S104, within the distorted frame interval, the spatial coordinate fluctuation sequence of the material pile surface points in the temporal three-dimensional point cloud frame is extracted, the active material accumulation area is identified through the spatial coordinate fluctuation sequence, and the projection center of the active material accumulation area on the horizontal plane is determined as the target pile-finding point of the unloading trolley.

[0037] In some embodiments, the extraction of the spatial coordinate fluctuation sequence of points on the surface of the material pile in the temporal three-dimensional point cloud frame within the distorted frame interval is achieved by the following steps: Within the distorted frame interval, the point cloud on the surface of the material pile is segmented from each frame of three-dimensional point cloud data. Using the point cloud of the material pile surface in the starting frame within the distorted frame interval as the reference point cloud, a point-to-point correspondence is established between the point cloud of the material pile surface in each subsequent frame and the point cloud of the material pile surface in the previous frame using the nearest neighbor search algorithm, thereby obtaining a set of matching point pairs between each pair of adjacent frames. Extract the spatial coordinate difference of each matching point pair from the set of matching point pairs of each pair of adjacent frames, arrange the spatial coordinate difference in order of frame number, and take the total set of spatial coordinate differences of matching point pairs of all frames in the distorted frame interval as the spatial coordinate fluctuation sequence.

[0038] In specific implementation, firstly, the 3D point cloud data of each frame is traversed within the distorted frame interval. A segmentation operation based on normal vector changes is performed on each frame of 3D point cloud data. Specifically, the height value of each point in the point cloud is compared with a pre-defined ground surface. Points with height values ​​exceeding a threshold and exhibiting loose surface characteristics are classified as material pile surface point clouds. Then, a set of spatially discrete points corresponding to the material's upper surface is extracted from each frame, and this extracted set of spatially discrete points is used as the material pile surface point cloud. Secondly, the material pile surface point cloud of the starting frame within the distorted frame interval is used as a reference point cloud. For each subsequent frame's material pile surface point cloud, a nearest neighbor search algorithm is used to establish a point-to-point correspondence with the previous frame's material pile surface point cloud. That is, the nearest neighbor search algorithm finds the point with the closest Euclidean distance in the previous frame's material pile surface point cloud for each point in the current frame as the corresponding point. KD-trees are often used to accelerate queries. A KD-tree is a binary tree structure that partitions spatial points, reducing the search complexity for each point from linear to logarithmic. All found corresponding point pairs are grouped into a set of matching point pairs. Each point pair in the set contains the 3D coordinates of a point in the previous frame and the 3D coordinates of the corresponding point in the current frame. Finally, the spatial coordinate difference of each matching point pair is extracted from the set of matching point pairs of each pair of adjacent frames. That is, the coordinates of the current frame point are subtracted from the coordinates of the corresponding point in the previous frame to obtain a 3D vector. Each component of the 3D vector represents the displacement of the point between two adjacent frames. The spatial coordinate differences of all matching point pairs are arranged in order of frame number, and the total set of spatial coordinate differences generated by all adjacent frame pairs within the distorted frame interval is taken as the spatial coordinate fluctuation sequence.

[0039] It should be noted that the spatial coordinate fluctuation sequence in this application refers to the displacement trajectory of each spatial point on the surface of the material pile as a function of time during the duration of vibration. Within the distortion frame interval, the material will undergo dynamic responses such as flow, accumulation or collapse under continuous vibration excitation. These dynamic responses are manifested as the three-dimensional coordinate changes of each spatial point on the surface of the material pile between consecutive frames. This evolution process in the time domain cannot be captured by relying solely on a single frame of static point cloud. Therefore, it is necessary to use the spatial coordinate difference sequence of matching point pairs between adjacent frames to quantitatively describe the instantaneous displacement trajectory of each point on the surface of the material pile, so as to transform the point cloud distortion information under vibration environment into an effective perception of the dynamic accumulation position of the material.

[0040] In some embodiments, identifying the active material accumulation zone through the spatial coordinate fluctuation sequence and determining the projection center of the active material accumulation zone on the horizontal plane as the target accumulation point for the unloading trolley is achieved through the following steps: For each pair of matching points in the spatial coordinate fluctuation sequence, the spatial coordinate difference of the same spatial location point between different frames is accumulated to generate the cumulative displacement vector of that point. Calculate the magnitude of each cumulative displacement vector and mark points whose magnitude exceeds a preset magnitude threshold as active accumulation candidate points; All active stacking candidate points are clustered using Euclidean distance, and the spatial region covered by the cluster with the most points is identified as the active material stacking area. Project the spatial coordinates of all points in the active material accumulation area onto a horizontal plane, calculate the arithmetic mean center of the projected points, and determine the arithmetic mean center as the target accumulation point of the unloading trolley.

[0041] In practice, firstly, the spatial coordinate difference of each pair of adjacent frame matching points is obtained from the spatial coordinate fluctuation sequence. Each spatial coordinate difference is a three-dimensional vector, and the spatial location point corresponding to the difference (i.e., the three-dimensional coordinates of the point in the previous or current frame) is recorded. Then, the points are grouped according to their spatial coordinates, and all spatial coordinate differences generated between different adjacent frames belonging to the same physical location point are vector-accumulated. Specifically, a hash table with point coordinates as keys is established. For each matching point pair, its spatial coordinate difference is added to the cumulative vector corresponding to the key. Finally, each key corresponds to a cumulative displacement. The cumulative displacement vector represents the total displacement of a spatial location point from the start frame to the end frame within the entire distorted frame interval. Next, the magnitude of each cumulative displacement vector is calculated. The magnitude is the Euclidean norm of the vector, obtained by squaring the three components of the cumulative displacement vector, summing them, and then taking the square root. The magnitude of each spatial location point is compared with a pre-calibrated magnitude threshold. Spatial location points with a magnitude exceeding this threshold indicate that they have experienced significant net displacement during the entire vibration process. These points are marked as active accumulation candidate points, representing the impact of material on the surface of the material pile. The area affected by the significant movement is then analyzed. Euclidean distance clustering is performed on all active accumulation candidate points. This involves setting a distance radius threshold, starting with an unclassified point, and grouping all points within that radius into the same cluster. This process is repeated, expanding from the newly added point until the cluster can no longer absorb new points. The next unclassified point is then selected for further clustering, resulting in several clusters. Each cluster contains several spatially adjacent points. The number of points in each cluster is counted, and the spatial area covered by the cluster with the most points is identified as the material's main accumulation point. The active material accumulation zone represents the location where material flow is most concentrated and accumulation is most significant. Finally, the spatial coordinates of all points within the active material accumulation zone are projected onto a horizontal plane. The projection operation ignores the vertical coordinates (Z coordinate) of each point and only retains the horizontal coordinates (X and Y coordinates), resulting in a series of horizontal projection points. The arithmetic mean center of these horizontal projection points is calculated by summing the X coordinates of all projection points and dividing by the total number of points to obtain the average X coordinate. Similarly, the average Y coordinate is obtained. This arithmetic mean center is a point on the horizontal plane, and this point is determined as the target accumulation point for the unloading trolley.

[0042] It should be noted that, in this application, the target stacking point refers to the horizontal position that the unloading trolley should move to so that unloading operations can be carried out above that position.

[0043] Furthermore, in another aspect of this application, in some embodiments, this application provides an autonomous stacking control system for an unloading trolley based on three-dimensional scanning, referencing... Figure 3The figure is a schematic diagram of the structure of a three-dimensional scanning-based autonomous stacking control system for unloading trolleys according to some embodiments of this application. This three-dimensional scanning-based autonomous stacking control system includes: a data acquisition module 201, a processing module 202, and an execution module 203, which are described below: The acquisition module 201 in this application is mainly used to start the three-dimensional scanner to continuously acquire time-series three-dimensional point cloud frames including the silo retaining wall and the material pile below, and to take the plane above the retaining wall marked in the first acquired point cloud frame as the initial reference plane. Processing module 202, in this application, is mainly used to extract the upper edge point cloud of the retaining wall in each subsequent point cloud frame, calculate the spatial pose change of the upper edge point cloud of the retaining wall in each frame relative to the initial reference plane, and decompose the corresponding spatial pose change into a global motion component caused by the movement of the unloading trolley and a local distortion component caused by the vibration of the unloading trolley body. The processing module 202 is further configured to compare the fluctuation amplitude of the local distortion component with a preset vibration baseline, and when the fluctuation amplitude exceeds the vibration baseline, identify the distortion frame interval in the temporal three-dimensional point cloud frame where motion blur occurs based on the local distortion component. The execution module 203 in this application is mainly used to extract the spatial coordinate fluctuation sequence of the material pile surface points in the time-series three-dimensional point cloud frame within the distorted frame interval, identify the active material accumulation area through the spatial coordinate fluctuation sequence, and determine the projection center of the active material accumulation area on the horizontal plane as the target stacking point of the unloading trolley.

[0044] In addition, this application also provides a computer device, which includes a memory and a processor. The memory stores code, and the processor is configured to acquire the code and execute the above-described autonomous stacking control method for unloading trolleys based on three-dimensional scanning.

[0045] In some embodiments, reference Figure 4 The figure is a schematic diagram of the structure of a computer device implementing a three-dimensional scanning-based autonomous stack-finding control method for an unloading trolley, according to some embodiments of this application. The three-dimensional scanning-based autonomous stack-finding control method for an unloading trolley in the above embodiments can... Figure 4 The computer device shown is used to implement this, and the computer device includes at least one processor 301, a communication bus 302, a memory 303, and at least one communication interface 304.

[0046] The processor 301 can be a general-purpose central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more devices used to control the execution of the autonomous stack-seeking control method for the unloading trolley based on three-dimensional scanning in this application.

[0047] The communication bus 302 can be used to transmit information between the aforementioned components.

[0048] The memory 303 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disks or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 303 may exist independently and be connected to the processor 301 via the communication bus 302. The memory 303 may also be integrated with the processor 301.

[0049] The memory 303 stores program code for executing the scheme of this application, and its execution is controlled by the processor 301. The processor 301 executes the program code stored in the memory 303. The program code may include one or more software modules. In the above embodiments, the determination of the autonomous stack-finding control method of the unloading trolley based on three-dimensional scanning can be implemented by the processor 301 and one or more software modules in the program code in the memory 303.

[0050] Communication interface 304 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0051] In a specific implementation, as one example, a computer device may include multiple processors, each of which may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. Here, a processor may refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0052] The aforementioned computer device can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device can be a desktop computer, a portable computer, a network server, a handheld digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. This application does not limit the type of computer device.

[0053] In addition, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described autonomous stacking control method for unloading trolleys based on three-dimensional scanning.

[0054] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0055] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for autonomous stacking control of a discharge trolley based on three-dimensional scanning, characterized in that, Includes the following steps: Start the 3D scanner to continuously acquire time-series 3D point cloud frames containing the silo retaining wall and the material pile below, and use the plane above the retaining wall marked in the first acquired point cloud frame as the initial reference plane. Extract the wall edge point cloud from each subsequent point cloud frame, calculate the spatial pose change of the wall edge point cloud in each frame relative to the initial reference plane, and directly determine the rotation matrix and translation vector in the spatial pose change as the global motion component caused by the unloading trolley motion; perform a rigid body transformation on the current wall edge point cloud according to the global motion component to obtain the transformed point cloud, calculate the signed distance from each point in the transformed point cloud to the initial reference plane, and take the set of all signed distances as the local distortion component caused by the vibration of the unloading trolley body; The fluctuation amplitude of the local distortion component is compared with a preset vibration baseline. When the fluctuation amplitude exceeds the vibration baseline, the difference between the maximum and minimum values ​​of all signed distances in the local distortion component is calculated as the distortion degree value of the current frame. It is determined whether the distortion degree value exceeds a preset distortion threshold. If it does, the current frame is marked as a distorted frame. Each subsequent frame after the first acquisition frame in the temporal 3D point cloud frame is traversed, and the starting frame number to the ending frame number corresponding to the consecutive frame sequence marked as distorted frames is determined as the distorted frame interval. Within the distorted frame interval, the point cloud of the material pile surface is segmented from the 3D point cloud data of each frame. Using the point cloud of the material pile surface of the starting frame within the distorted frame interval as a reference point cloud, a nearest neighbor search algorithm is used to establish a point-to-point correspondence between the point cloud of the material pile surface of each subsequent frame and the point cloud of the material pile surface of the previous frame, resulting in a set of matching point pairs between each pair of adjacent frames. The spatial coordinate difference of each matching point pair is extracted from the set of matching point pairs of each pair of adjacent frames. The spatial coordinate difference is arranged in order of frame number, and the total set of spatial coordinate differences of matching point pairs across all frames within the distorted frame interval is used as the spatial coordinate fluctuation sequence. For each matching point pair in the spatial coordinate fluctuation sequence, the spatial coordinate difference of the same spatial location point between different frames is accumulated to generate a cumulative displacement vector for that point. The magnitude of each cumulative displacement vector is calculated, and points with a magnitude exceeding a preset magnitude threshold are marked as active stacking candidate points. All active stacking candidate points are clustered using Euclidean distance. The spatial area covered by the cluster with the most points is identified as the active material stacking area, and the projection center of the active material stacking area on the horizontal plane is determined as the target stacking point of the unloading trolley.

2. The method as described in claim 1, characterized in that, The 3D scanner is activated to continuously acquire temporal 3D point cloud frames containing the silo retaining wall and the material pile below. The plane along the upper edge of the retaining wall, as marked in the first acquired point cloud frame, is used as the initial reference plane. Specifically, this includes: After the unloading trolley is moved to the preset starting scanning position, the 3D scanner is triggered to acquire the first frame of 3D point cloud data. The point cloud of the retaining wall area located on both sides of the scanning field of view is segmented from the first frame of 3D point cloud data based on the differences in geometric features between the retaining wall and the material pile. The edge point cloud that is in contact with the air is extracted from the upper part of the point cloud in each retaining wall area as the upper edge point cloud of the retaining wall. The point cloud along the extracted retaining wall is fitted to a spatial plane using the least squares method, and the plane equation coefficients of this spatial plane in the vehicle coordinate system are used as the initial reference plane.

3. The method as described in claim 1, characterized in that, Extracting the point cloud along the upper edge of the retaining wall from subsequent point cloud frames and calculating the spatial pose change of the point cloud along the upper edge of the retaining wall relative to the initial reference plane in each frame specifically includes: For each subsequent frame in the temporal 3D point cloud frame that is located after the first acquisition frame, the same segmentation method as that used to extract the point cloud along the upper edge of the retaining wall in the first frame is used to extract the current point cloud along the upper edge of the retaining wall from the subsequent frame. The point cloud along the current retaining wall is rigidly registered with the spatial plane containing the initial reference plane, and the optimal rotation matrix and optimal translation vector are solved to transform the point cloud along the current retaining wall to coincide with the initial reference plane. The optimal rotation matrix and the optimal translation vector are combined to form the spatial pose change of the point cloud along the frame retaining wall relative to the initial reference plane.

4. The method as described in claim 1, characterized in that, The comparison of the fluctuation amplitude of the local distortion component with the preset vibration baseline specifically includes: The root mean square value of all signed distances in the local distortion component is calculated as the current fluctuation amplitude; Read the pre-stored vibration baseline threshold, determine whether the current fluctuation amplitude is greater than the vibration baseline threshold, and if so, determine that there is abnormal vibration caused by material impact in the current unloading area.

5. A three-dimensional scanning-based autonomous stacking control system for an unloading trolley, used to execute the three-dimensional scanning-based autonomous stacking control method for an unloading trolley as described in any one of claims 1 to 4, characterized in that, The system includes: The acquisition module is used to start the 3D scanner to continuously acquire time-series 3D point cloud frames containing the silo retaining wall and the material pile below, and to use the plane above the retaining wall marked in the first acquired point cloud frame as the initial reference plane. The processing module is used to extract the point cloud above the retaining wall in each subsequent point cloud frame, calculate the spatial pose change of the point cloud above the retaining wall in each frame relative to the initial reference plane, and decompose the corresponding spatial pose change into a global motion component caused by the movement of the unloading trolley and a local distortion component caused by the vibration of the unloading trolley body. The processing module is also used to compare the fluctuation amplitude of the local distortion component with a preset vibration baseline. When the fluctuation amplitude exceeds the vibration baseline, the distortion frame interval in the temporal three-dimensional point cloud frame where motion blur occurs is identified based on the local distortion component. The execution module is used to extract the spatial coordinate fluctuation sequence of the material pile surface points in the time-series three-dimensional point cloud frame within the distorted frame interval, identify the active material accumulation area through the spatial coordinate fluctuation sequence, and determine the projection center of the active material accumulation area on the horizontal plane as the target pile-finding point of the unloading trolley.

6. A computer device, characterized in that, The computer device includes a memory and a processor, the memory storing code, and the processor being configured to retrieve the code and execute the autonomous stacking control method for unloading trolleys based on three-dimensional scanning as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the autonomous stacking control method for the unloading trolley based on three-dimensional scanning as described in any one of claims 1 to 4.

Images