A warehouse material detection system and method

Abstract

The application relates to the technical field of warehouse material detection, and discloses a warehouse material detection system which comprises a crown block, a grab bucket, a scanning module, a PLC control module and an algorithm module. The scanning module comprises a laser scanner and a millimeter wave scanner, and the laser scanner and the millimeter wave scanner are fixedly arranged on the crown block. The PLC control module is used for controlling the crown block and the grab bucket to move. The algorithm module is used for carrying out noise reduction and fusion on the point clouds obtained by the laser scanner and the millimeter wave scanner, calculating the height of the target position material, and transmitting the height of the material to the PLC control module. The millimeter wave scanner is arranged, and the algorithm module is used for carrying out fusion on the point clouds obtained by the millimeter wave scanner and the laser scanner, so that the detection precision of the detection system is improved, and the automatic operation efficiency of the crown block in the environment with more dust is ensured.

Description

Technical Field

[0001] This invention relates to the field of warehouse material testing technology, and in particular to a warehouse material testing system and method. Background Technology

[0002] When overhead cranes perform automated operations in storage areas, they need to obtain the distribution of materials within the storage area through sensors. However, when operating in environments with a lot of dust, such as bituminous coal, the dust in the air can affect the accuracy of the sensors and reduce the efficiency of the overhead cranes' automated operations.

[0003] The existing invention patent with publication number CN115626570A discloses an automatic control system and method for grab buckets to realize the whole process of coke processing. It uses a combination of 2D fan-shaped laser and millimeter-wave sensor to realize material distribution detection. It uses grid-based automatic coke processing sub-software, intelligent multi-strategy automatic coke processing sub-software, multi-grab bucket collaborative sub-software, and automatic inventory sub-software to achieve remote automatic operation throughout the process. This keeps personnel away from the harmful environment on site, ensuring personnel health. At the same time, one person can operate multiple machines, saving manpower and improving efficiency. It realizes a remote, safe, automatic, and highly efficient coke processing process, which has high application value.

[0004] In the above technical solution, two detection methods are set up for material detection: 2D laser sensor and millimeter wave sensor. When there is little interference in the environment, 2D laser sensor is used to detect the material, while when there is a lot of interference in the environment, millimeter wave sensor is used to detect the material. However, since the wavelength of millimeter wave is large, its detection accuracy is poor, which will affect the accuracy of the grab position and the efficiency of material grabbing operation. Summary of the Invention

[0005] In view of this, the present invention proposes a storage material detection system and method, which uses an algorithm module to fuse point clouds obtained by millimeter-wave scanners and laser scanners, thereby improving the detection accuracy of the detection system.

[0006] The technical solution of this invention is implemented as follows: On one hand, this invention provides a warehouse material detection system, including an overhead crane, a grab bucket, a scanning module, a PLC control module, and an algorithm module, wherein...

[0007] The overhead crane includes a large carriage and a small carriage, with the small carriage slidably mounted on the large carriage;

[0008] The grab bucket is mounted on the trolley and can be displaced vertically relative to the trolley.

[0009] The scanning module includes a laser scanner and a millimeter-wave scanner, both of which are fixedly mounted on the vehicle.

[0010] The PLC control module is used to control the movement of the large trolley and the small trolley, control the vertical displacement of the grab bucket to grab materials, and control the laser scanner and the millimeter-wave scanner to perform scanning.

[0011] The algorithm module is used to reduce noise and fuse the point clouds acquired by the laser scanner and the millimeter-wave scanner, calculate the height of the material at the target location, and transmit the height of the material to the PLC control module.

[0012] Based on the above technical solutions, preferably, the laser scanner is located at the end of the vehicle, the millimeter-wave scanner is located in the middle of the vehicle, and the scanning direction of the millimeter-wave scanner is vertically downward.

[0013] Secondly, the present invention provides a method for detecting stored materials, employing the aforementioned stored material detection system, comprising the following steps:

[0014] S1, the PLC control module is used to control the movement of the large trolley and the small trolley, so that the grab bucket is positioned above the material grabbing position, and the laser scanner and the millimeter-wave scanner are turned on simultaneously to scan the stored materials and obtain laser point cloud and millimeter-wave point cloud;

[0015] S2, Establish a world coordinate system, and use the algorithm module to calibrate the laser point cloud and millimeter-wave point cloud into the world coordinate system respectively;

[0016] S3, use the algorithm module to perform noise reduction processing on millimeter-wave point cloud data in the world coordinate system;

[0017] S4, using the algorithm module, the denoised millimeter-wave point cloud is fused with the laser point cloud in the world coordinate system, and the de-coincidence algorithm is used to generate a de-pointed point cloud in the world coordinate system;

[0018] S5. Extract the Z-axis values ​​of each point in the defocusing cloud at the grab position, compare their magnitudes, define the largest Z-axis value as the material height at the grab position, and transmit the material height at the grab position to the PLC control module to control the grab bucket to grab the material.

[0019] Based on the above technical solutions, preferably, when the large vehicle and the small vehicle are in their initial positions, they can move along the first direction and the second direction on the horizontal plane, respectively, and the first direction and the second direction are perpendicular to each other;

[0020] In step S2, the projection point of the initial position of the trolley on the storage ground is the origin of the world coordinate system, the first direction is the positive direction of the X-axis of the world coordinate system, the second direction is the positive direction of the Y-axis of the world coordinate system, and the direction away from the storage ground is the positive direction of the Z-axis of the world coordinate system.

[0021] More preferably, in step S2, the laser point cloud is calibrated to the world coordinate system according to the installation position of the laser scanner on the vehicle;

[0022] More preferably, in step S2, calibrating the millimeter-wave point cloud to the world coordinate system includes the following steps:

[0023] S21, save the coordinates of the large vehicle in the world coordinate system and the coordinates of each point in the millimeter-wave point cloud, and use the material property characteristics to remove impurities and purify the millimeter-wave point cloud.

[0024] S22, calibrate the Euler angles of rotation to obtain the rotation matrix Mat_Rotate corresponding to the Euler angles. Multiply the coordinates of each point in the millimeter-wave point cloud by the rotation matrix Mat_Rotate, and add the coordinates of the vehicle in the world coordinate system to calibrate the millimeter-wave point cloud to the world coordinate system.

[0025] S23. Select feature point P1 of the millimeter-wave point cloud in the world coordinate system and feature point P2 of the laser point cloud in the world coordinate system corresponding to the position of P1. Calculate the translation matrix Mat_Tran = P2 - P1. Call the ICP registration algorithm to perform registration. The registration matrix is ​​Mat_ICP. The calibration matrix Mat = Mat_ICP * Mat_Tran. Use the calibration matrix Mat to calibrate the millimeter-wave point cloud in the world coordinate system.

[0026] More preferably, in step S22, calibrating the rotation Euler angles includes the following steps:

[0027] S221, Assume the Euler angles are X, and the Euler angle construction matrix is ​​Mat_E;

[0028] S222, define millimeter-wave point clouds with equal coordinate values ​​of the large vehicle as the same frame point cloud, subtract the coordinates of the large vehicle from the coordinates of each point in the same frame point cloud, multiply the coordinates with the Euler angle construction matrix Mat_E, and add the coordinates of the large vehicle to obtain the coordinate set Px of each point in the same frame point cloud;

[0029] S223, using a filtering algorithm, fit a plane to the points in the millimeter-wave point cloud located on the warehouse ground, calculate the average distance from Px to the fitted plane, and complete the calibration when the average distance is less than a threshold; otherwise, repeat S221-S223.

[0030] More preferably, in step S3, the millimeter-wave point cloud data denoising process includes the following steps:

[0031] S31. By analyzing the Doppler velocity, radar cross section, target type, dynamic attributes, and existence probability attributes of the millimeter-wave point cloud, points in the millimeter-wave point cloud with a velocity not greater than the maximum moving speed of the vehicle, a radar cross section within the standard range, a target type of obstacle, a dynamic attribute of stationary, and an existence probability greater than 80% are selected as valid points, and noisy points other than the valid points in the millimeter-wave point cloud are removed.

[0032] S32 uses a pass-through filter to filter out noise points in the millimeter-wave point cloud that are not in the material storage area by using coordinate values;

[0033] S33 uses a clustering algorithm to filter out outliers;

[0034] S34 uses the least squares smoothing algorithm to smooth the millimeter-wave point cloud;

[0035] S35 uses a material surface extraction algorithm based on material property features to determine the top surface point cloud and the slope point cloud, and removes aerial noise.

[0036] S36 uses an interpolation algorithm based on material features to supplement the material characteristics missing point cloud locations and construct the material surface;

[0037] S37, extract the point cloud near the surface from the millimeter-wave point cloud calibrated in S23;

[0038] S38 uses the least squares algorithm, interpolation algorithm, and triangulation algorithm to reconstruct the material surface.

[0039] More preferably, in step S35, the material surface extraction algorithm includes the following steps:

[0040] S351, based on the input grid size, forms multiple grids and distributes the millimeter-wave point cloud within the grids;

[0041] S352, each grid and its 8 surrounding grids form a 3*3 grid. Each point in the millimeter-wave point cloud within each grid is classified according to the difference in Z-axis values. Point clouds with Z-axis value differences less than a preset value are classified into the same category. If the number of points in the same category is less than a threshold, the point cloud of this category is marked as undefined. If the number of points in the same category is greater than the threshold, the point cloud of this category is marked as the top surface point cloud. The top surface point cloud of the grid located in the middle position of each grid is saved to the grid.

[0042] S353, check each undefined millimeter-wave point cloud one by one, compare its Z-axis value with the height of its two adjacent top surfaces, and if the Z-axis value of the undefined point cloud is between the heights of its two adjacent top surfaces, then mark this point cloud as a slope point cloud.

[0043] S354: For a mesh containing multiple types of top surface point clouds, compare the number of top surface point clouds of each type and retain the top surface point cloud with the most point clouds.

[0044] More preferably, in step S36, the interpolation algorithm includes the following steps:

[0045] S361, repeat steps S352-S353;

[0046] S362 specifies a blank grid as a grid that does not contain grids marked as top surface point clouds or slope point clouds. If a blank grid has top surface point clouds or slope point clouds on both sides, the height of the blank grid is calculated using an interpolation algorithm based on the height of the top surface point clouds or slope point clouds on both sides. If a blank grid has a top surface point cloud or slope point cloud on one side, the height of the blank grid is the average value of the top surface point cloud or slope point cloud on that side.

[0047] The storage material detection system and method of the present invention have the following advantages over the prior art:

[0048] (1) By setting up a millimeter-wave scanner and using an algorithm module to fuse the point clouds obtained by the millimeter-wave scanner and the laser scanner, the detection accuracy of the detection system is improved, and the efficiency of the crane's automated operation in a dusty environment is guaranteed.

[0049] (2) By placing the laser scanner at the end of the vehicle, other components such as the grab bucket can be prevented from obstructing the laser scanner, thereby increasing the scanning field of the laser scanner.

[0050] (3) By using a grid to extract the effective millimeter-wave point cloud and reprocessing the point cloud that was filtered out during the noise reduction process based on the effective point cloud, the accuracy of millimeter-wave point cloud noise reduction can be improved. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a front view of the overhead crane in a warehouse material detection system according to the present invention;

[0053] Figure 2 This is a flowchart of a warehouse material detection system according to the present invention;

[0054] Figure 3 This is a flowchart of a method for detecting stored materials according to the present invention.

[0055] The components include: 1. Overhead crane; 11. Large trolley; 12. Small trolley; 2. Grab bucket; 3. Scanning module; 31. Laser scanner; 32. Millimeter wave scanner; 4. PLC control module; 5. Algorithm module. Detailed Implementation

[0056] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0057] like Figure 1 As shown, a warehouse material detection system of the present invention includes an overhead crane 1, a grab bucket 2, a scanning module 3, a PLC control module 4, and an algorithm module 5.

[0058] The overhead crane 1 is used to control the movement of the grab bucket 2 on the horizontal plane. The overhead crane 1 includes a large trolley 11 and a small trolley 12. The small trolley 12 is slidably mounted on the large trolley 11, so that the grab bucket 2 moves directly above the material to be grabbed. When the large trolley 11 and the small trolley 12 are in the initial position (the small trolley 12 is located at one of the corners of the warehouse), they can move on the horizontal plane along the first direction and the second direction respectively, and the first direction and the second direction are perpendicular to each other.

[0059] The grab bucket 2 is used to grab materials. The grab bucket 2 is set on the trolley 12 and can move vertically with the trolley 12. When the trolley 11 and the trolley 12 move the grab bucket 2 to directly above the material to be grabbed, the grab bucket 2 can be moved downward to grab the material.

[0060] The scanning module 3 is used to detect and identify stored materials. The scanning module 3 includes a laser scanner 31 and a millimeter-wave scanner 32. Both the laser scanner 31 and the millimeter-wave scanner 32 are fixedly mounted on the trolley 11. As the trolley 11 moves, the scanning module 3 can detect and scan the materials in the warehouse and generate corresponding point clouds.

[0061] Furthermore, the laser scanner 31 can be placed at the end of the trolley 11, and the millimeter-wave scanner 32 can be placed in the middle of the trolley 11, with the scanning direction of the millimeter-wave scanner 32 being vertically downward. By setting the two separately, not only can interference between the two be prevented, but also other components such as the grab bucket 2 can be prevented from obstructing the laser scanner 31 or the millimeter-wave scanner 32, thereby increasing the field of view of the scanning module 3.

[0062] PLC control module 4 is the control mechanism of this detection system. PLC control module 4 is used to control the movement of the trolley 11 and the small trolley 12, control the vertical displacement of the grab bucket 2 to grab materials, and control the laser scanner 31 and the millimeter-wave scanner 32 to perform scanning.

[0063] Algorithm module 5 is used to reduce noise and fuse the point clouds acquired by laser scanner 31 and millimeter-wave scanner 32, calculate the height of the material at the target location, and transmit the height of the material to PLC control module 4. Then, PLC control module 4 controls grab bucket 2 to move downward by the height of the material to grab it. Although millimeter waves have stronger penetration than optical devices such as lasers and infrared, and can avoid scanning materials stored in the coal storage environment, the wavelength of millimeter waves is 1 to 10 mm, and its accuracy is low. Simply using millimeter waves to detect the height of the material will lead to excessive deviation in the detection results. This detection system uses algorithm module 5 to reduce noise and fuse the point clouds acquired by millimeter-wave scanner 32 and laser scanner 31, which can effectively improve its detection accuracy and ensure the automation efficiency of crane 1 and grab bucket 2.

[0064] The method of using the storage material detection system and method of the present invention is as follows:

[0065] S1, the PLC control module 4 is used to control the movement of the large trolley 11 and the small trolley 12 so that the grab bucket 2 is positioned above the material grabbing position, and the laser scanner 31 and the millimeter-wave scanner 32 are turned on simultaneously to scan the stored materials and obtain laser point cloud and millimeter-wave point cloud.

[0066] S2, establish a world coordinate system. The projection point of the initial position of the car 12 on the storage ground can be used as the origin of the world coordinate system. The first direction can be used as the positive direction of the X-axis of the world coordinate system, the second direction can be used as the positive direction of the Y-axis of the world coordinate system, and the direction away from the storage ground can be used as the positive direction of the Z-axis of the world coordinate system. Then, the laser point cloud and the millimeter wave point cloud can be calibrated into the world coordinate system using the algorithm module 5 respectively.

[0067] Since the laser scanner 31 is installed at the end of the trolley 11, its coordinates are inconsistent with those of the trolley 11, while the millimeter-wave scanner 32 is installed in the middle of the trolley 11, its coordinates are consistent with those of the trolley 11. The point cloud coordinates obtained by the laser scanner 31 and the millimeter-wave scanner 32 will deviate. Therefore, it is necessary to calibrate the laser point cloud to the world coordinate system according to the installation position of the laser scanner 31 on the trolley 11.

[0068] Calibrating laser point clouds to a world coordinate system is an existing and conventional technique, such as the calibration method used in the invention patent with publication number CN115128993A, which discloses a steel coil positioning, gripping, and warehousing system and its usage method. However, millimeter-wave point clouds differ from laser point clouds in that they have overlapping areas, so the calibration method for laser point clouds cannot be used to calibrate millimeter-wave point clouds. Calibrating millimeter-wave point clouds to a world coordinate system includes the following steps:

[0069] S21, save the coordinates of the large vehicle 11 in the world coordinate system and the coordinates of each point in the millimeter-wave point cloud, and use the material property characteristics to remove impurities and purify the millimeter-wave point cloud. The material property characteristics include the shape, color, and stacking method of the material. By comparing the obtained initial millimeter-wave point cloud with these characteristics, point clouds that are obviously different from the material property characteristics can be filtered out, thus achieving the initial screening of the millimeter-wave point cloud.

[0070] S22, calibrate the Euler angles of rotation to obtain the rotation matrix Mat_Rotate corresponding to the Euler angles. Multiply the coordinates of each point in the millimeter-wave point cloud by the rotation matrix Mat_Rotate, and add the coordinates of the large vehicle 11 in the world coordinate system to calibrate the millimeter-wave point cloud to the world coordinate system.

[0071] The calibration of rotational Euler angles includes the following steps:

[0072] S221, assuming the Euler angles are X, the Euler angle construction matrix is ​​Mat_E.

[0073] S222, define millimeter-wave point clouds with equal coordinate values ​​of the large vehicle 11 as the same frame point cloud. First, subtract the coordinates of the large vehicle 11 from the coordinates of each point in the same frame point cloud, then multiply it with the Euler angle construction matrix Mat_E, and add the coordinates of the large vehicle 11 to obtain the coordinate set Px of each point in the same frame point cloud.

[0074] S223: Using a filtering algorithm, fit a plane to the point in the millimeter-wave point cloud located on the warehouse ground, calculate the average distance from Px to the fitted plane. If the average distance is less than a threshold, the calibration is completed; otherwise, repeat S221-S223 until the assumed Euler angles are valid.

[0075] S23. Select feature point P1 of the millimeter-wave point cloud in the world coordinate system and feature point P2 of the laser point cloud in the world coordinate system corresponding to the position of P1. Calculate the translation matrix Mat_Tran = P2 - P1. Call the ICP registration algorithm to perform registration. The registration matrix is ​​Mat_ICP. The calibration matrix Mat = Mat_ICP * Mat_Tran. Use the calibration matrix Mat to calibrate the millimeter-wave point cloud in the world coordinate system.

[0076] S3. At this point, the millimeter-wave point cloud still contains some noise, which will affect the judgment of the material height. Therefore, it is necessary to use algorithm module 5 to perform noise reduction processing on the millimeter-wave point cloud data in the world coordinate system.

[0077] Compared to laser point cloud data, millimeter-wave point cloud data has fewer data points and lower accuracy. Furthermore, the point clouds tend to float on the curved surface of the material, making it impossible to obtain the material height through simple processing like laser point clouds. Noise reduction processing for millimeter-wave point cloud data includes the following steps:

[0078] S31. By analyzing the Doppler velocity (relative radial velocity, with velocities greater than the vehicle's moving speed being considered noise), radar cross section (RCS), target type (unclassified, noise, ground, penetrable, obstacle, invalid), dynamic attributes (stationary, moving), and existence probability of the millimeter-wave point cloud, points in the millimeter-wave point cloud with velocities not exceeding the maximum moving speed of the vehicle 11, RCS within the standard range, target type being an obstacle, dynamic attribute being stationary, and existence probability greater than 80% are selected as valid points, and noise points other than the valid points in the millimeter-wave point cloud are removed.

[0079] S32 uses a pass-through filter to filter out noise points in millimeter-wave point clouds that are not in the material storage area, i.e. point clouds whose X-axis and Y-axis values ​​are greater than the coordinate values ​​of the storage space, such as point clouds obtained by scanning the surrounding walls. This noise reduction method is an existing technology.

[0080] S33 uses a clustering algorithm to filter out outliers. First, it generates nuclei by regularly arranging regions and specifies the distance between the nuclei. Then, it specifies the distance between the point clouds and the minimum grain size. Smaller grains need to be integrated into the nearest larger grains. After obtaining the grain size and the crystallization range, it determines which point clouds are not representative and are outliers by factors such as color differences, differences in normals, and differences in distance. These outliers are then filtered out. This noise reduction method is an existing technology.

[0081] S34 uses a least squares smoothing algorithm to smooth the millimeter-wave point cloud. For each point in the millimeter-wave point cloud, the radius R of the search circle and the search step size S are set. A polynomial is used to fit the point P(x,y), the normal line L at P(x,y) is calculated, and the displacement is increased along the normal line L. The interpolation point coordinates Z = f(x,y) are calculated based on the Z coordinates of the points within the search range and the fitted polynomial f(x,y). This method is the existing technology.

[0082] S35. In the above steps, clustering and least squares smoothing algorithms can remove most of the noise points that are not near the material surface. However, since there are large-angle slopes when the material is piled up, and the number of point clouds on these large-angle slopes is very small, the above algorithm may filter out the point clouds on the slopes, affecting the detection of the material. Therefore, a material surface extraction algorithm based on material attribute features is used to determine the top surface point cloud and the slope point cloud, and remove aerial noise.

[0083] The material surface extraction algorithm includes the following steps:

[0084] S351, based on the input grid size, form multiple grids and distribute the millimeter-wave point cloud within the grids, that is, grid the millimeter-wave point cloud. For example, if the range of the millimeter-wave point cloud is 10m*20m, and the grid size is set to 0.5m*0.5m, then the range of the millimeter-wave point cloud is divided into a 20*40 grid.

[0085] S352: Each grid and its eight surrounding grids form a 3x3 grid. Points in the millimeter-wave point cloud within each grid are classified according to their Z-axis numerical differences. Points with Z-axis numerical differences less than a preset value are classified as belonging to the same category. Points in the same category represent the top surface of the scanned material or other object. If the number of points in the same category is less than a threshold, it means that this type of point cloud is not the top surface of the material, and it is marked as undefined. If the number of points in the same category is greater than the threshold, it means that this type of point cloud is highly likely to be the top surface of the material, and it is marked as top surface point cloud. The top surface point cloud located in the middle grid within each grid is saved to that grid. The point clouds in the eight surrounding grids are used to improve the accuracy of the point cloud in the middle grid. The accuracy of the point clouds in the eight surrounding grids can also be improved by the point clouds in their respective eight surrounding grids, thus preventing point cloud loss.

[0086] S353, In the above steps, the point cloud of the top surface of the material is obtained. When there is a difference in the height of the point cloud of the top surface of the two materials, it means that there is a slope between the top surfaces of the two materials. Then, check the millimeter-wave point cloud marked as undefined one by one, and compare its Z-axis value with the height of its two adjacent top surfaces. If the Z-axis value of the undefined point cloud is between the heights of its two adjacent top surfaces, it means that this point cloud is a point cloud at the transition position between the two top surfaces. Then, mark this point cloud as a slope point cloud.

[0087] S354. Due to the gaps in the material accumulation, millimeter waves can penetrate the material and form multiple top surface point clouds at the same material accumulation site. Of course, some top surface point clouds may also be point clouds of airborne dust. Since the number of points in the point cloud in these two cases will be less than the number of point clouds on the top surface of the material, for a grid containing multiple types of top surface point clouds, the number of multiple types of top surface point clouds is compared, and the top surface point cloud with the most point clouds is retained.

[0088] S36. In the above steps, the top surface point cloud and the slope point cloud have been determined. However, there will still be missing point clouds in many places. An interpolation algorithm based on material characteristics can be used to supplement the material characteristics at the locations where the point cloud is missing and construct the material surface.

[0089] The interpolation algorithm includes the following steps:

[0090] S361, Repeat steps S352-S353 to re-check the top surface point cloud and the slope point cloud to prevent errors caused by omissions or identification mistakes.

[0091] S362: Designate any grids that do not contain the markings for top or slope point clouds as blank grids. The material properties of the blank grids need to be supplemented. If there are top or slope point clouds on both sides of the blank grid, use the height of the top or slope point clouds on both sides as a reference and calculate the height of the blank grid using an interpolation algorithm. This ensures a smooth transition between the height of the blank grid and the heights of the top and slope point clouds on both sides. If there is a top or slope point cloud on one side of the blank grid, the height of the blank grid is the average value of the top or slope point cloud on that side, making the height of this blank grid consistent with the height of the top and slope point clouds on that side. Position this blank grid as an occluded point cloud region within the top or slope point cloud.

[0092] S37. In the original millimeter-wave point cloud, some valid point clouds will be filtered out in the algorithm due to attributes, distance, etc. To solve this problem, point clouds near the material surface can be extracted from the millimeter-wave point cloud after calibration in S23 to verify whether there are still valid point clouds in the filtered point clouds.

[0093] S38 uses the least squares algorithm, interpolation algorithm, and triangulation algorithm to perform secondary noise reduction on the original point cloud at the material surface in order to reconstruct the material surface.

[0094] S4. Using algorithm module 5, the denoised millimeter-wave point cloud from the above steps is fused with the laser point cloud in the world coordinate system. Using the de-coincidence algorithm, a de-pointed point cloud is generated in the world coordinate system. This de-pointed point cloud is the final point cloud.

[0095] S5, extract the Z-axis values ​​of each point in the defocusing cloud at the grab position, compare their magnitudes, define the largest Z-axis value as the material height at the grab position, and transmit the material height at the grab position to the PLC control module 4 to control the grab bucket 2 to grab the material.

[0096] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting stored materials, characterized in that: The warehouse material detection system includes an overhead crane (1), a grab bucket (2), a scanning module (3), a PLC control module (4), and an algorithm module (5). The overhead crane (1) includes a large trolley (11) and a small trolley (12), with the small trolley (12) slidably mounted on the large trolley (11). The grab bucket (2) is mounted on the small trolley (12) and can be displaced vertically relative to it. The scanning module (3) includes a laser scanner (31) and a millimeter-wave scanner (32). 32) All are fixedly installed on the large trolley (11); the PLC control module (4) is used to control the large trolley (11) and the small trolley (12) to move, control the grab bucket (2) to move in the vertical direction and grab the material, and control the laser scanner (31) and the millimeter-wave scanner (32) to scan; the algorithm module (5) is used to reduce noise and fuse the point cloud obtained by the laser scanner (31) and the millimeter-wave scanner (32), calculate the height of the material at the target position, and transmit the height of the material to the PLC control module (4); The laser scanner (31) is located at the end of the trolley (11), and the millimeter-wave scanner (32) is located in the middle of the trolley (11), with the scanning direction of the millimeter-wave scanner (32) being vertically downward. The method for testing stored materials includes the following steps: S1, use the PLC control module (4) to control the large trolley (11) and the small trolley (12) to move, so that the grab bucket (2) is above the material grabbing position, and simultaneously turn on the laser scanner (31) and the millimeter wave scanner (32) to scan the stored materials and obtain laser point cloud and millimeter wave point cloud; S2, establish a world coordinate system, and use the algorithm module (5) to calibrate the laser point cloud and millimeter wave point cloud into the world coordinate system respectively; S3, use the algorithm module (5) to perform noise reduction processing on the millimeter-wave point cloud data in the world coordinate system; S4, using the algorithm module (5) to fuse the denoised millimeter-wave point cloud with the laser point cloud in the world coordinate system, and using the de-coincidence algorithm to generate the de-point cloud in the world coordinate system; S5, extract the Z-axis values ​​of each point in the defocus cloud at the grab position, compare their sizes, define the largest Z-axis value as the material height at the grab position, and transmit the material height at the grab position to the PLC control module (4) to control the grab bucket (2) to grab the material.

2. The method for detecting stored materials as described in claim 1, characterized in that: When the large vehicle (11) and the small vehicle (12) are in their initial positions, they can move along the first direction and the second direction on the horizontal plane, respectively, and the first direction and the second direction are perpendicular to each other; In step S2, the projection point of the initial position of the trolley (12) on the storage ground is the origin of the world coordinate system, the first direction is the positive direction of the X-axis of the world coordinate system, the second direction is the positive direction of the Y-axis of the world coordinate system, and the direction away from the storage ground is the positive direction of the Z-axis of the world coordinate system.

3. The method for detecting stored materials as described in claim 2, characterized in that: In step S2, the laser point cloud is calibrated to the world coordinate system according to the installation position of the laser scanner (31) on the vehicle (11).

4. The method for detecting stored materials as described in claim 3, characterized in that, In step S2, calibrating the millimeter-wave point cloud to the world coordinate system includes the following steps: S21, save the coordinates of the large vehicle (11) in the world coordinate system and the coordinates of each point in the millimeter-wave point cloud, and use the material property characteristics to remove impurities and purify the millimeter-wave point cloud; S22, calibrate the Euler angles of rotation, obtain the rotation matrix Mat_Rotate corresponding to the Euler angles, multiply the coordinates of each point in the millimeter-wave point cloud by the rotation matrix Mat_Rotate, and add the coordinates of the large vehicle (11) in the world coordinate system to calibrate the millimeter-wave point cloud to the world coordinate system; S23. Select feature point P1 of the millimeter-wave point cloud in the world coordinate system and feature point P2 of the laser point cloud in the world coordinate system corresponding to the position of P1. Calculate the translation matrix Mat_Tran = P2-P1. Call the ICP registration algorithm to perform registration. The registration matrix is ​​Mat_ICP. The calibration matrix Mat = Mat_ICP*Mat_Tran. Use the calibration matrix Mat to calibrate the millimeter-wave point cloud in the world coordinate system.

5. The method for detecting stored materials as described in claim 4, characterized in that, In step S22, calibrating the rotation Euler angles includes the following steps: S221, Assume the Euler angles are X, and the Euler angle construction matrix is ​​Mat_E; S222, define millimeter-wave point clouds with equal coordinate values ​​of the large vehicle (11) as the same frame point cloud, subtract the coordinates of the large vehicle (11) from the coordinates of each point in the same frame point cloud, multiply it with the Euler angle construction matrix Mat_E, and add the coordinates of the large vehicle (11) to obtain the coordinate set Px of each point in the same frame point cloud; S223, using a filtering algorithm, fit a plane to the points in the millimeter-wave point cloud located on the warehouse ground, calculate the average distance from Px to the fitted plane, and complete the calibration when the average distance is less than a threshold; otherwise, repeat S221-S223.

6. The method for detecting stored materials as described in claim 5, characterized in that, In step S3, the millimeter-wave point cloud data denoising process includes the following steps: S31, by analyzing the Doppler velocity, radar cross section, target type, dynamic attributes and existence probability attributes of the millimeter-wave point cloud, select points in the millimeter-wave point cloud whose velocity is not greater than the maximum moving speed of the large vehicle (11), whose radar cross section is in the standard range, whose target type is an obstacle, whose dynamic attribute is stationary, and whose existence probability is greater than 80% as valid points, and remove noise points other than valid points in the millimeter-wave point cloud; S32 uses a pass-through filter to filter out noise points in the millimeter-wave point cloud that are not in the material storage area by using coordinate values; S33 uses a clustering algorithm to filter out outliers; S34 uses the least squares smoothing algorithm to smooth the millimeter-wave point cloud; S35 uses a material surface extraction algorithm based on material property features to determine the top surface point cloud and the slope point cloud, and removes aerial noise. S36 uses an interpolation algorithm based on material features to supplement the material characteristics missing point cloud locations and construct the material surface; S37, extract the point cloud near the surface from the millimeter-wave point cloud calibrated in S23; S38 uses the least squares algorithm, interpolation algorithm, and triangulation algorithm to reconstruct the material surface.

7. The method for detecting stored materials as described in claim 6, characterized in that, In step S35, the material surface extraction algorithm includes the following steps: S351, based on the input grid size, forms multiple grids and distributes the millimeter-wave point cloud within the grids; S352, each grid and its 8 surrounding grids form a 3*3 grid. Each point in the millimeter-wave point cloud within each grid is classified according to the difference in Z-axis values. Point clouds with Z-axis value differences less than a preset value are classified into the same category. If the number of points in the same category is less than a threshold, the point cloud of this category is marked as undefined. If the number of points in the same category is greater than the threshold, the point cloud of this category is marked as the top surface point cloud. The top surface point cloud of the grid located in the middle position of each grid is saved to the grid. S353, check each undefined millimeter-wave point cloud one by one, compare its Z-axis value with the height of its two adjacent top surfaces, and if the Z-axis value of the undefined point cloud is between the heights of its two adjacent top surfaces, then mark this point cloud as a slope point cloud. S354: For a mesh containing multiple types of top surface point clouds, compare the number of top surface point clouds of each type and retain the top surface point cloud with the most point clouds.

8. The method for detecting stored materials as described in claim 7, characterized in that, In step S36, the interpolation algorithm includes the following steps: S361, repeat steps S352-S353; S362 specifies a blank grid as a grid that does not contain grids marked as top surface point clouds or slope point clouds. If a blank grid has top surface point clouds or slope point clouds on both sides, the height of the blank grid is calculated using an interpolation algorithm based on the height of the top surface point clouds or slope point clouds on both sides. If a blank grid has a top surface point cloud or slope point cloud on one side, the height of the blank grid is the average value of the top surface point cloud or slope point cloud on that side.

Images