A method and device for detecting suspended objects by converting laser point cloud into depth image

Abstract

The present invention relates to a method and device for detecting suspended objects by converting laser point clouds into depth images, and belongs to the field of unmanned driving for stacking profiles or bars in steel plants. The method includes the steps of obtaining the original point cloud and driving position and configuring parameters, converting the original point cloud into a height table and filtering the data, converting the height table into a grayscale image and scaling and expanding it, performing statistical histogram and normalization operations, threshold segmentation and processing of the image interest layer, edge detection and contour search of the interest layer, matching the contour shape according to the contour moment and Hu moment, and determining the vertices of the suspended object. The present invention effectively improves the problem of decreased recognition rate of bar stacks due to deformation, and significantly improves the first small-angle scanning recognition rate of the lifting point and stacking point in the packaging area; the method is also suitable for improving the first scanning recognition rate of profiles and high-wire coils, and is easy to promote and apply.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned driving for stacking profiles or bars in steel plants, and particularly relates to a method and device for detecting suspended objects by converting laser point clouds into depth images. Background Art

[0002] With the advancement of automation and identification and positioning technologies, unmanned cranes for finished product hoisting in steel mills have become a trend. Profile bundles have a rectangular cross-section and are stacked in multiple layers, close together and in a forward direction. High-wire coils are hollow cylinders placed on their side and stacked in multiple layers, side by side, close together and in a rising seam pattern. Bar bundles have an elliptical cross-section and are stacked in multiple layers, overlapping in a crisscross pattern. Because profile bundles have a square top surface and are tall, and high-wire coils have a large diameter, stacking deformation has little impact on position detection. However, bar bundles are slender and easily bent, with a small height difference between layers. As the number of layers increases, stacking deformation becomes more pronounced, reducing the accuracy of stacking position recognition. While the applicant's previously filed applications for "A Method and System for Identifying and Calculating Coordinates Based on a Quasi-Plane Multi-Level Contour" and "A LiDAR Scanning Angle Switching and Point Cloud Sampling System and Method" address the problem of detecting the hoisting and stacking positions of these three types of steel, they also suffer from issues such as unintuitive point cloud data observation and significant deviations in finding the two vertices of the rectangular contour at the height of interest, other than the first and last vertices. This leads to large errors in the calculated width of the hoisted object (the deviation is even greater for the non-squareness of the top surface of the bar). While this error can be corrected by finding and calculating the width using five equally spaced rows or columns within the target contour area, it requires that the angle between the hoisted object and the laser scanning direction should not be too large. Therefore, overcoming the shortcomings of existing technologies is a pressing issue in this field. Summary of the Invention

[0003] The present application aims to solve the problems of detecting the lifting position of the unmanned vehicle's hoisted objects (bars, profile bundles or high-wire coils) in the packaging area and the stacking position in the stacking area, as well as calculating the scanning point positions in the packaging area and the stacking area. In addition, the application aims to solve the problems of the previous detection devices and methods, such as the non-intuitive display of point cloud data, the inability to cover the full stacking view in the large-angle scanning point cloud sampling, processing and display, the large deviation in searching the other two vertices of the rectangular outline of the height layer of interest of the hoisted object except for the first and last vertices, and the increase in the accuracy of stacking position recognition due to the large deformation of the bar stack as the number of layers increases, which requires the increase of dynamic angle correction of laser scanning and pan-tilt rotation direction. A method and device for detecting hoisted objects by converting laser point clouds into depth images are provided.

[0004] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0005] A method for detecting suspended objects by converting laser point clouds into depth images comprises the following steps:

[0006] Step (1), acquisition of original point cloud and driving position and parameter configuration: large-angle scanning of the stacking area, small-angle scanning of the stacking area and small-angle scanning of the packaging area, collecting point cloud data and driving position, and then configuring parameters according to these three scanning types, and dynamically updating the laser scanning angle and pan / tilt rotation angle correction coefficient;

[0007] Step (2), converting the original point cloud into an altimeter and filtering the data: converting the point cloud data obtained in step (1) into an altimeter, filtering out the data in the domain beyond the range of interest, the domain beyond the slope, and the domain beyond the jump, and then filtering the data in the altimeter for outliers;

[0008] Step (3), converting the altimeter into a grayscale image and scaling it: converting the altimeter into an original depth image, and then converting the depth image into a grayscale image; then scaling the grayscale image by the scaling factor, and adding multiple blank lines at the beginning and end to expand it;

[0009] Step (4), histogram statistics and normalization operation: perform histogram statistics and normalization on the grayscale image, and then find the continuous segment features of the histogram value to obtain the grayscale threshold of the interest layer and the corresponding height threshold of the interest layer;

[0010] Step (5), threshold segmentation and processing of the image interest layer: threshold segmentation is performed on the grayscale image processed in step (3) using the grayscale threshold of the interest layer, and the layer with grayscale values ​​within the upper and lower limits of the grayscale threshold of the interest layer is retained as the interest layer and an interest layer image is generated; then, the interest layer image is subjected to image morphology operations, filtering, and binarization processing in sequence;

[0011] Step (6), edge detection and contour search of the interest layer: edge detection is performed on the interest layer image processed in step (5), and then contour search is performed; then the length and area of ​​each contour are calculated, and filtering is performed according to the contour length and area as well as the contour area-to-length ratio respectively. The remaining contours are sorted from large to small by length and the contour length and corresponding contour number are stored in an array, and then the number of valid closed contours is determined;

[0012] Step (7) matches the contour shape according to the contour moment and Hu moment: calculate the contour moment of the effective closed contour, then use the contour moment to calculate the Hu moment, and then use the Hu moment and the Hu moment of the pre-stored contour shape sample of the suspended object to perform shape matching and filtering, filter out the contour of non-interest shapes, and then sort out the contour length and number array, and calculate the number of suspended objects currently stacked in the interest layer in the image;

[0013] Step (8), determination of the vertices of the suspended object: obtain the coordinates of the four vertices of the minimum rectangular outer frame of each valid rectangular outline; use the circular approximation method to find the points on the contour that are closest to the four vertices of the minimum rectangular outer frame of the contour; convert the coordinates of these four closest points on the contour to the coordinates corresponding to the altimeter; search for points whose height values ​​meet the height threshold of the interest layer in the neighborhood of these four coordinate points in the altimeter from near to far as the points to be used as the vertices of the suspended object;

[0014] Step (9), selection and processing of target contour: select the target contour according to the geometric features of the contour of interest, the relative position in the image and the effective value of the vertex in the corresponding height table; then calculate the three-dimensional coordinates (x, y, z) of the original point cloud corresponding to the four vertices of the target contour corresponding to the height table; then use the three-dimensional coordinates of the four vertices of the hoisted object to calculate the length, width, height, midpoint coordinates and placement angle of the hoisted object;

[0015] Step (10), calculation of the correction coefficients of the laser scanning angle and the pan-tilt rotation angle: select 20 points in the neighborhood of the four vertices of the suspended object and calculate the average height. Use the average height of the four vertex neighborhoods to construct a right triangle in the height direction and the laser scanning direction or the pan-tilt rotation direction. Use the height difference and the vertex spacing to calculate the arc tangent to calculate the new dynamic angle correction coefficient of the laser scanning and pan-tilt rotation directions.

[0016] Step (11), stacking state judgment and lifting and stacking position coordinates and angle calculation: the stacking state is judged according to the large and small angle stacking scanning types, the absolute coordinates (X, Y, Z) of the stacking midpoint, the relative coordinates (x, y, z) of the current stacking position and the length, width and height of the hoisted object, and the coordinates and placement angles of the lifting point and stacking point of the hoisted object are calculated and corrected;

[0017] Step (12), calculation of the vehicle position coordinates of the next scanning point: calculation and correction of the vehicle position coordinates of the next scanning point based on large and small angle stacking scans, stacking status and current stacking position;

[0018] Step (13), output of the position of the suspended object and the coordinates of the scanning point: the calculated coordinates of the lifting point and stacking point of the suspended object and the driving position of the next scanning point are transmitted to the PLC through the network, thereby controlling the driving to travel to the scanning point, lifting point and stacking point.

[0019] Furthermore, preferably, in step (1), the configured parameters include the coordinates of the current stack midpoint, the coordinates of the small-angle scanning point in the packaging area, the stacking spacing of the suspended objects, the scanning type, the laser scanning angle correction coefficient, the pan / tilt rotation angle correction coefficient, the outlier filtering neighborhood radius, the outlier filtering neighborhood point number threshold, the image row and column scaling ratio, the depth map to grayscale map ratio, the interest layer height threshold, the histogram interest layer grayscale ratio threshold, the histogram thresholding lower limit, the contour length threshold, the contour area threshold, the altimeter valid vertex search neighborhood radius, and the total number of altimeter rows and columns corresponding to the scanning type, the over-slant and jump height errors, and the filter spacing.

[0020] Furthermore, preferably, in step (2), the specific method of outlier filtering is:

[0021] According to the outlier filtering neighborhood radius r in the parameter configuration, the altimeter is traversed to calculate the number of neighboring points whose absolute value of the height difference between point (m,n) and its neighboring points within the neighborhood radius range [(mr)~(m+r), (nr)~(n+r)] is less than the over-slope and jump height errors. If the number of neighboring points of a point is less than the outlier filtering neighborhood point number threshold, it is considered an outlier and is deleted from the altimeter.

[0022] Furthermore, preferably, the specific method of step (4) is:

[0023] Perform histogram statistics and normalization on the grayscale image, retaining the values ​​whose grayscale value percentage is greater than the histogram threshold lower limit, and clearing those less than the lower limit;

[0024] Then find the continuous segments where the percentage of grayscale points is greater than the lower limit of the histogram threshold, record the continuous and discontinuous number of grayscale points in each segment, the maximum and minimum percentage of grayscale points in the segment, and the starting and ending values ​​of the segment;

[0025] Then, by comparing the maximum percentage of grayscale points in the segment with the grayscale percentage threshold of the histogram interest layer, valid segments are screened out. The specific method for screening valid segments is as follows: for the packaging area and the first layer of the stack, the valid segment is the segment with the largest percentage of grayscale points; for the second layer and above of the stack, all segments are sorted by the maximum percentage of grayscale points, and the segments with the maximum percentage of grayscale points greater than the grayscale percentage threshold of the histogram interest layer are retained as valid segments.

[0026] Finally, the segment with the smallest starting and ending values ​​in the valid segment is selected as the segment of interest. The ending and starting values ​​of the segment of interest are slightly expanded as the upper and lower limits of the grayscale threshold of the interest layer, and then divided by the depth map to grayscale map ratio to obtain the height threshold of the interest layer corresponding to the height table.

[0027] Furthermore, preferably, the specific method of step (6) is:

[0028] Perform Canny edge detection on the interest layer image processed in step (5), and then perform contour search;

[0029] Calculate the length and area of ​​each contour, and filter out contours whose length is less than the contour length threshold, whose area is less than the contour area threshold, and whose area-to-length ratio is less than the ratio of the two thresholds; for the remaining contours, store the contour length value in the contour length array, and store the contour number in the contour number array in the corresponding order;

[0030] Then the contour length array is sorted by contour length, and the contour number array is updated in the corresponding order; and then the number of valid closed contours is counted.

[0031] Furthermore, it is preferred that when the suspended object is detected next time, for large and small angle scanning of the stacking area, the laser scanning or pan-tilt rotation dynamic angle correction coefficient calculated in step (10) is used as the angle coefficient for correcting the upper surface of the stack and the laser scanning or pan-tilt rotation direction; for small angle scanning of the packaging area; the laser scanning dynamic angle correction coefficient calculated in step (10) is used as the angle coefficient for dynamically correcting the laser scanning direction, and the pan-tilt rotation still uses the manual correction coefficient;

[0032] In step (11), the stacking status includes five states: empty stack, full stack, not full stack but empty layer, not full stack but full layer, and not full stack and not full layer.

[0033] Furthermore, preferably, the following steps are also included: after selecting a valid closed rectangular interest contour, marking the interest layer rectangular contour and the minimum rectangular outer frame, and the four vertices of the interest contour corresponding to the vertices of the minimum rectangular outer frame into the original grayscale image for storage and display.

[0034] Furthermore, preferably, the following steps are also included: the placement angle is converted back into the three-dimensional long side xyz coordinate point set of the suspended object using the long side point set of the target contour in the grayscale image, and then a straight line is fitted in the (x, y) two-dimensional plane using this set of long side three-dimensional coordinate point sets to obtain the placement angle.

[0035] The present invention also provides a suspended object detection device for converting laser point cloud to depth image, comprising an embedded control module. When the embedded control module executes the program, the steps of the suspended object detection method for converting laser point cloud to depth image are implemented.

[0036] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of the suspended object detection method of converting laser point cloud to depth image as described above are implemented.

[0037] In step (1), after the laser radar is installed, the initial value of the laser scanning angle correction coefficient is calibrated according to the horizontal angle between the laser scanning direction and the ground plane of the stacking area and the packaging area, for example, -0.700°; the initial value of the pan-tilt rotation angle correction coefficient is calibrated according to the horizontal angle between the pan-tilt rotation direction and the ground plane of the stacking area and the packaging area, for example, -0.400°

[0038] In the present invention, when the vehicle is started, the stacking area is scanned at a large angle by default, and the scanning type is automatically switched according to the process.

[0039] In the present invention, the vertical coordinate of the grayscale histogram is the percentage of grayscale points, and the horizontal coordinate is the grayscale value 0~255. The percentage of points (vertical coordinate) of two consecutive grayscale values ​​(horizontal coordinate) in a segment is greater than the lower limit of the histogram threshold value, which is considered continuous. The conditions for judging a continuous segment are: (1) if the number of continuous points in the segment is 1~3, and there is more than 1 discontinuous point in the segment and at the end of the segment, the segment is considered to be over; (2) if there are 4 continuous points in the segment, and there are more than 2 discontinuous points in the segment and at the end of the segment, the segment is considered to be over; (3) if there are 5 continuous points in the segment, and there are more than 3 discontinuous points in the segment and at the end of the segment, the segment is considered to be over; (4) if there are more than 6 continuous points in the segment, and there are more than 5 discontinuous points in the segment and at the end of the segment, the segment is considered to be over.

[0040] In the present invention, the gray value point ratio refers to the ratio of the number of points corresponding to a certain layer height in the altimeter to the total number of points in the altimeter.

[0041] In the present invention, the segment with the smallest segment start value and end value is selected as the interesting segment, that is, the segment corresponding to the packaging area or the highest layer of the stack.

[0042] In the present invention, the end value and the start value of the interest segment are slightly expanded, that is, the segment start value is reduced by 1, and the segment end value is increased by 1, that is, the upper and lower limits of the grayscale values ​​corresponding to the height layer corresponding to the segment are relaxed a little.

[0043] In the present invention, the contour shape samples of the suspended object, such as the rectangular contour shape samples obtained by processing the laser point cloud to depth image method for the bundle of bar materials, can be rectangular, circular, and elliptical. In the present invention, after filtering and sorting according to the contour length and area, and the contour area-to-length ratio in step (6), the contour shape samples of the suspended object (such as the bundle of bar materials being rectangular) are matched, and the unmatched contours are deleted from the array. The deleted items in the array are shifted forward and then sorted.

[0044] In step (10) of the present invention, 20 points are selected in sequence from the vertex neighborhood range [row0~row4, column0~column4] containing the vertex (row0, column0, height0), and the average height value of the 18 points after removing the points with the maximum and minimum height values ​​other than the vertex is calculated, wherein the selected points should preferably include points in the range [row0~row3, column0~column3]. Among them, (row0, column0, height0) represents the coordinates of the vertices of the outline of the suspended object in the image, that is, (vertex row coordinate, vertex column coordinate, height value of the three-dimensional coordinate corresponding to the vertex); [row0~row4, column0~column4] represents the neighboring point area composed of the row where the vertex coordinate is located and its four adjacent rows and the column where the vertex is located and its four adjacent columns; [row0~row3, column0~column3] represents the neighboring point area composed of the row where the vertex is located and its three adjacent rows and the column where the vertex is located and its three adjacent columns; the rows row0~row4 and columns column0~column4 taken are all within the valid outline area of ​​the suspended object in the image.

[0045] To calculate and correct the coordinates of the lifting point and placement angle of the object, when the crane is lifting, the midpoint of the lower surface of the four electromagnetic suction cups (i.e., the electromagnetic crane) moves to align above the coordinates of the lifting point of the object. The lifting beam holding the four electromagnetic suction cups rotates to align the placement angle of the object and the direction of the long object, and then lowers it onto the object. This ensures that the electromagnetic suction cups are aligned with the object and securely hold it. The stacking point position coordinates and placement angle are the position and direction of the object to be placed in the current stack. The crane lifts the object from the packaging area (after lifting it to a safe height, the electromagnetic crane rotates back to 0 degrees) to the coordinates of the target stacking point. The electromagnetic crane rotates to the placement angle, then descends to the height of the stacking position coordinates, and then demagnetizes and lowers the object.

[0046] In the present invention, the placement angle is converted back into the xyz coordinate point set of the three-dimensional long side of the suspended object using the long side point set of the grayscale image target contour, and then a straight line is fitted in the (x, y) two-dimensional plane using this set of long side three-dimensional coordinate point sets to obtain the placement angle. Compared with the placement angle obtained by using the four three-dimensional vertices of the suspended object to form a right triangle to calculate the inverse tangent angle and the placement angle obtained by using the grayscale image target contour to fit the ellipse frame to obtain the axial angle, the method has better robustness and smaller error.

[0047] The method and device for detecting suspended objects by converting laser point clouds into depth images described in this application are implemented using the hardware and program of an embedded control module in an unmanned vehicle system. The embedded control module controls a three-dimensional laser radar scanning system via a network, performing low-angle scanning of bars, profile bundles, or high-wire coils in the packaging area, and high- and low-angle scanning of the stacking area. The system then collects raw point clouds and uses the position detection method and program to calculate the object's lifting position, stacking position, and placement angle, as well as the vehicle's position coordinates corresponding to the scanning points. These coordinates are then transmitted to a PLC via the network to control the automatic lifting of the vehicle.

[0048] The three-dimensional laser radar scanning system (referred to as laser radar) includes a pan-tilt head and a two-dimensional laser scanner, which are installed on the trolley. The rotation direction of the pan-tilt head is perpendicular to the trolley beam, and the laser scanning direction is parallel to the trolley beam (the direction can be adjusted according to the scene requirements). The laser radar is used as the reference xyz relative coordinate system origin, and the trolley, trolley and lifting mechanism are used as the reference xyz absolute coordinate system. The XYZ coordinate zero point is the midpoint of the bottom surface of the electromagnetic crane (or hook) when the trolley is at the right limit, the trolley is at the rear limit and the lifting mechanism is at the upper limit.

[0049] The device described in this application detects the position coordinates of the lifting points, stacking points and scanning points of profiles, bar bundles and high-wire coils (also known as coils or coils) through scanning, and transmits them to the PLC through the network to control the automatic lifting of the crane.

[0050] The embedded control module uses a higher-performance ARM Cortex-A53 quad-core chip and larger-capacity internal and external memory as the core unit for computing control and storage, and realizes network communication through a gigabit network card. It is equipped with an Ubuntu operating system, can run the OpenCV open source vision library, and cross-compile and debug with the host computer through the network or USB interface. Its power module has an AC220V input and a DC12V / 2A output.

[0051] This invention utilizes an embedded control module with greater memory and computing power. The point cloud sampling, processing, and display of a wide-angle scan of the stack area can cover the entire stack, resulting in more accurate initial stack status determination. Increasing the dynamic angle correction coefficient between the stack's top surface and the laser scanning and pan / tilt rotation directions helps mitigate the effects of stack deformation on recognition accuracy.

[0052] Building on previous work, this paper develops a method for detecting suspended objects by converting laser point clouds to depth images. This method effectively improves the accuracy of finding the four vertices of a rectangular outline at the height layer of interest for a bar, thereby improving the accuracy of detecting the bar's width and position. This allows for wider object placement angles and more accurate angle calculations. Furthermore, using the OpenCV library, the Canny edge detection and contour finding algorithms (the underlying edge detection algorithm includes first- and second-order derivative operators) detect the boundary transitions of suspended objects, achieving more robust matching than previous algorithms that compare adjacent point heights in rows or columns. This image-based method also enables detection of objects with shapes other than rectangles, such as circles and ellipses. Furthermore, the depth image-to-grayscale image conversion step for suspended object detection is also applicable to processing depth images captured by binoculars or depth cameras.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] (1) The present invention provides a method and device for detecting suspended objects by converting laser point cloud into depth image, which is highly practical, has low equipment cost, and is easy to implement;

[0055] (2) The point cloud data is converted into images for upload and display, with small storage space and transmission data volume. For example, a point cloud file with 300 rows and 720 columns of large-angle scanning occupies 2.532MB, and a grayscale image occupies only 225KB. The observation is intuitive and convenient for debugging and subsequent operation and maintenance. The device supports network remote debugging and program loading, and the algorithm operation is fast. The data calculation time for large-angle scanning is less than 300ms.

[0056] (3) Effectively improve the problem of reduced recognition rate caused by deformation of bar stacking. The recognition rate of the first small-angle scan at the lifting point and stacking point in the packaging area is increased from 93% to 95%, and the recognition rate of the second scan can reach 98%;

[0057] (4) The panoramic sampling and processing of the point cloud of the stack scanned at a large angle is beneficial for the subsequent lifting and loading from the stack; the depth image converted from the point cloud in the present invention is similar to the depth image processing process collected by the depth camera, which paves the way for the subsequent position detection using the multi-sensor fusion of the depth camera and the lidar. The multi-point installation detection of the depth camera on the ground, which is lower in cost than the lidar, is beneficial to improving the rhythm of the automatic operation of the vehicle and reducing the equipment cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 Schematic diagram of the framework of the suspended object detection device for converting laser point cloud to depth image according to the present invention;

[0059] Figure 2 This is a flow chart of the suspended object detection method of converting laser point cloud to depth image according to the present invention;

[0060] Figure 3The original grayscale image of a group of bars marked in the packaging area scanned at a small angle;

[0061] Figure 4 The original grayscale image of the two groups of bars in the second layer of the stack scanned at a small angle and marked;

[0062] Figure 5 The original grayscale image of a group of bars marked in the first layer of the stack scanned at a small angle;

[0063] Figure 6 The original grayscale image of two groups of bars marked in the second layer of the stacking area scanned at a large angle;

[0064] Figure 7 Original grayscale image of three groups of bars marked in the first layer of the stacking area scanned at a large angle DETAILED DESCRIPTION

[0065] The present invention is described in further detail below with reference to the embodiments.

[0066] Those skilled in the art will understand that the following examples are intended to illustrate the present invention only and should not be construed as limiting the scope of the present invention. Where specific techniques or conditions are not specified in the examples, the techniques or conditions described in the literature in the art or in the product specifications were used. Materials or equipment used without manufacturer identification are commercially available conventional products.

[0067] A method for detecting suspended objects by converting laser point clouds into depth images comprises the following steps:

[0068] Step (1), acquisition of original point cloud and driving position and parameter configuration: large-angle scanning of the stacking area, small-angle scanning of the stacking area and small-angle scanning of the packaging area, collecting point cloud data and driving position, and then configuring parameters according to these three scanning types, and dynamically updating the laser scanning angle and pan / tilt rotation angle correction coefficient;

[0069] Step (2), converting the original point cloud into an altimeter and filtering the data: converting the point cloud data obtained in step (1) into an altimeter, filtering out the data in the domain beyond the range of interest, the domain beyond the slope, and the domain beyond the jump, and then filtering the data in the altimeter for outliers;

[0070] Step (3), converting the altimeter into a grayscale image and scaling it: converting the altimeter into an original depth image, and then converting the depth image into a grayscale image; then scaling the grayscale image by the scaling factor, and adding multiple blank lines at the beginning and end to expand it;

[0071] Step (4), histogram statistics and normalization operation: perform histogram statistics and normalization on the grayscale image, and then find the continuous segment features of the histogram value to obtain the grayscale threshold of the interest layer and the corresponding height threshold of the interest layer;

[0072] Step (5), threshold segmentation and processing of the image interest layer: threshold segmentation is performed on the grayscale image processed in step (3) using the grayscale threshold of the interest layer, and the layer with grayscale values ​​within the upper and lower limits of the grayscale threshold of the interest layer is retained as the interest layer and an interest layer image is generated; then, the interest layer image is subjected to image morphology operations, filtering, and binarization processing in sequence;

[0073] Step (6), edge detection and contour search of the interest layer: edge detection is performed on the interest layer image processed in step (5), and then contour search is performed; then the length and area of ​​each contour are calculated, and filtering is performed according to the contour length and area as well as the contour area-to-length ratio respectively. The remaining contours are sorted from large to small by length and the contour length and corresponding contour number are stored in an array, and then the number of valid closed contours is determined;

[0074] Step (7) matches the contour shape according to the contour moment and Hu moment: calculate the contour moment of the effective closed contour, then use the contour moment to calculate the Hu moment, and then use the Hu moment and the Hu moment of the pre-stored contour shape sample of the suspended object to perform shape matching and filtering, filter out the contour of non-interest shapes, and then sort out the contour length and number array, and calculate the number of suspended objects currently stacked in the interest layer in the image;

[0075] Step (8), determination of the vertices of the suspended object: obtain the coordinates of the four vertices of the minimum rectangular outer frame of each valid rectangular outline; use the circular approximation method to find the points on the contour that are closest to the four vertices of the minimum rectangular outer frame of the contour; convert the coordinates of these four closest points on the contour to the coordinates corresponding to the altimeter; search for points whose height values ​​meet the height threshold of the interest layer in the neighborhood of these four coordinate points in the altimeter from near to far as the points to be used as the vertices of the suspended object;

[0076] Step (9), selection and processing of target contour: select the target contour according to the geometric features of the contour of interest, the relative position in the image and the effective value of the vertex in the corresponding height table; then calculate the three-dimensional coordinates (x, y, z) of the original point cloud corresponding to the four vertices of the target contour corresponding to the height table; then use the three-dimensional coordinates of the four vertices of the hoisted object to calculate the length, width, height, midpoint coordinates and placement angle of the hoisted object;

[0077] Step (10), calculation of the correction coefficients of the laser scanning angle and the pan-tilt rotation angle: select 20 points in the neighborhood of the four vertices of the suspended object and calculate the average height. Use the average height of the four vertex neighborhoods to construct a right triangle in the height direction and the laser scanning direction or the pan-tilt rotation direction. Use the height difference and the vertex spacing to calculate the arc tangent to calculate the new dynamic angle correction coefficient of the laser scanning and pan-tilt rotation directions.

[0078] Step (11), stacking state judgment and lifting and stacking position coordinates and angle calculation: the stacking state is judged according to the large and small angle stacking scanning types, the absolute coordinates (X, Y, Z) of the stacking midpoint, the relative coordinates (x, y, z) of the current stacking position and the length, width and height of the hoisted object, and the coordinates and placement angles of the lifting point and stacking point of the hoisted object are calculated and corrected;

[0079] Step (12), calculation of the vehicle position coordinates of the next scanning point: calculation and correction of the vehicle position coordinates of the next scanning point based on large and small angle stacking scans, stacking status and current stacking position;

[0080] Step (13), output of the position of the suspended object and the coordinates of the scanning point: the calculated coordinates of the lifting point and stacking point of the suspended object and the driving position of the next scanning point are transmitted to the PLC through the network, thereby controlling the driving to travel to the scanning point, lifting point and stacking point.

[0081] Specifically, in step (1), the configured parameters include the coordinates of the current stack midpoint, the coordinates of the small-angle scanning points in the packaging area, the stacking spacing of the suspended objects, the scanning type, the laser scanning angle correction coefficient, the pan / tilt rotation angle correction coefficient, the outlier filtering neighborhood radius, the outlier filtering neighborhood point number threshold, the image row and column scaling factor, the depth map to grayscale map scaling factor, the interest layer height threshold, the histogram interest layer grayscale ratio threshold, the histogram thresholding lower limit, the contour length threshold, the contour area threshold, the altimeter valid vertex search neighborhood radius, and the total number of altimeter rows and columns, the over-slant and jump height errors, and the filter spacing corresponding to the scanning type.

[0082] Specifically, in step (2), the specific method of outlier filtering is:

[0083] According to the outlier filtering neighborhood radius r in the parameter configuration, the altimeter is traversed to calculate the number of neighboring points whose absolute value of the height difference between point (m,n) and its neighboring points within the neighborhood radius range [(mr)~(m+r), (nr)~(n+r)] is less than the over-slope and jump height errors. If the number of neighboring points of a point is less than the outlier filtering neighborhood point number threshold, it is considered an outlier and is deleted from the altimeter.

[0084] Specifically, the specific method of step (4) is:

[0085] Perform histogram statistics and normalization on the grayscale image, retaining the values ​​whose grayscale value percentage is greater than the histogram threshold lower limit, and clearing those less than the lower limit;

[0086] Then find the continuous segments where the percentage of grayscale points is greater than the lower limit of the histogram threshold, record the continuous and discontinuous number of grayscale points in each segment, the maximum and minimum percentage of grayscale points in the segment, and the starting and ending values ​​of the segment;

[0087] Then, by comparing the maximum percentage of grayscale points in the segment with the grayscale percentage threshold of the histogram interest layer, valid segments are screened out. The specific method for screening valid segments is as follows: for the packaging area and the first layer of the stack, the valid segment is the segment with the largest percentage of grayscale points; for the second layer and above of the stack, all segments are sorted by the maximum percentage of grayscale points, and the segments with the maximum percentage of grayscale points greater than the grayscale percentage threshold of the histogram interest layer are retained as valid segments.

[0088] Finally, the segment with the smallest starting and ending values ​​in the valid segment is selected as the segment of interest. The ending and starting values ​​of the segment of interest are slightly expanded as the upper and lower limits of the grayscale threshold of the interest layer, and then divided by the depth map to grayscale map ratio to obtain the height threshold of the interest layer corresponding to the height table.

[0089] Specifically, the specific method of step (6) is:

[0090] Perform Canny edge detection on the interest layer image processed in step (5), and then perform contour search;

[0091] Calculate the length and area of ​​each contour, and filter out contours whose length is less than the contour length threshold, whose area is less than the contour area threshold, and whose area-to-length ratio is less than the ratio of the two thresholds; for the remaining contours, store the contour length value in the contour length array, and store the contour number in the contour number array in the corresponding order;

[0092] Then the contour length array is sorted by contour length, and the contour number array is updated in the corresponding order; and then the number of valid closed contours is counted.

[0093] Specifically, when the suspended object is detected next time, for large and small angle scanning of the stacking area, the laser scanning or pan-tilt rotation dynamic angle correction coefficient calculated in step (10) is used as the angle coefficient for correcting the upper surface of the stack and the laser scanning or pan-tilt rotation direction; for small angle scanning of the packaging area; the laser scanning dynamic angle correction coefficient calculated in step (10) is used as the angle coefficient for dynamically correcting the laser scanning direction, and the pan-tilt rotation still uses the manual correction coefficient;

[0094] In step (11), the stacking status includes five states: empty stack, full stack, not full stack but empty layer, not full stack but full layer, and not full stack and not full layer.

[0095] Specifically, the method further includes the following steps: after selecting a valid closed rectangular contour of interest, marking the rectangular contour of the interest layer, the minimum rectangular outer frame, and the four vertices of the interest contour corresponding to the vertices of the minimum rectangular outer frame into the original grayscale image for storage and display.

[0096] Specifically, the following steps are also included: the placement angle is converted back into the three-dimensional long side xyz coordinate point set of the suspended object using the long side point set of the grayscale target contour, and then the placement angle is obtained by fitting a straight line in the (x, y) two-dimensional plane using this set of long side three-dimensional coordinate point sets.

[0097] A suspended object detection device for converting laser point cloud to depth image includes an embedded control module. When the embedded control module executes the program, the steps of the suspended object detection method for converting laser point cloud to depth image are implemented.

[0098] A non-transitory computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned method for detecting suspended objects by converting laser point clouds into depth images.

[0099] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0100] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0101] Application examples include Figure 1 As shown, a 3D lidar scanning system and an embedded control module are installed on the traveling vehicle. The traveling vehicle has its own PLC module. The three are connected by wire through the network switch on the traveling vehicle. The network switch on the traveling vehicle is connected to the ground network switch through a wireless network module. Ground equipment (such as HMI) can communicate with the equipment on the traveling vehicle by connecting to the ground network switch.

[0102] A method for detecting suspended objects by converting laser point clouds into depth images is specifically implemented as follows:

[0103] Raw point cloud and vehicle position input: Based on the type of scan point reached by the vehicle (large-angle scan point in the stacking area, small-angle scan point in the stacking area, or small-angle scan point in the packaging area), the 3D LiDAR scanning system is controlled via network commands to perform a large-angle scan of the stacking area, a small-angle scan of the stacking area, or a small-angle scan of the packaging area. The collected point cloud data is sampled and stored in the raw point cloud data tables corresponding to the large- and small-angle scans (300 rows × 720 columns for large angle scans and 50 rows × 720 columns for small angle scans). A set of laser scanner point cloud data is collected for each pan-tilt rotation angle resolution and stored as a row in the table; that is, the rows in the table represent the pan-tilt rotation angle dimension, the columns represent the laser scanning angle dimension, and the values ​​in the table represent the laser point measurement distance dimension. Vehicle position input records the absolute coordinates of the scan point at which the vehicle is located, preparing the subsequent module to calculate the next small-angle scan point in the stacking area.

[0104] Parameter configuration: Configure the parameters of each function module based on the large-angle scanning stacking area, small-angle scanning stacking area, and small-angle scanning packaging area; for example, the coordinates of the current stack midpoint, the coordinates of the small-angle scanning point in the packaging area (the large-angle scanning point in the stacking area is calculated from the stack midpoint, and the small-angle scanning point in the stacking area (next time) is dynamically calculated), the stacking spacing of the suspended objects, the scanning type (including large-angle scanning of the stacking area, small-angle scanning of the stacking area, and small-angle scanning of the packaging area), the laser scanning and pan / tilt rotation angle correction coefficient, the total number of altimeter rows and columns corresponding to the scanning type, the over-skew and jump height error and filter spacing, the outlier filtering neighborhood radius and point count threshold, the image row and column scaling ratio, the depth map to grayscale conversion ratio, the interest layer height threshold, the histogram interest layer grayscale ratio threshold, the histogram thresholding lower limit, the contour length and area thresholds, and the altimeter valid vertex search neighborhood radius.

[0105] Over-slope and jump: In the height table, two adjacent points or a point set within the filter interval are traversed and analyzed by row (or column). If the difference between the height values ​​of the adjacent points is greater than the over-slope and jump height error, it is judged as a jump. If the difference between the maximum and minimum height values ​​of the first, last and midpoint of the segment within the filter interval is greater than the over-slope and jump height error, it is judged as over-slope.

[0106] The laser scanning angle and gimbal rotation angle correction coefficients are dynamically updated. The default value is manually corrected for the first time, and the last calculated value is used for subsequent times. After the laser radar is installed, the initial value of the laser scanning angle correction coefficient is calibrated according to the horizontal angle between the laser scanning direction and the ground plane of the stacking and packaging areas, for example, -0.700°. The initial value of the gimbal rotation angle correction coefficient is calibrated according to the horizontal angle between the gimbal rotation direction and the ground plane of the stacking and packaging areas, for example, -0.400°.

[0107] The original point cloud is converted into an altimeter: two memory spaces are allocated in the device of the present invention to store the altimeters corresponding to the large and small angle scans, so that the altimeter is mapped one-to-one with the original data table. Each point in the original data table is calculated for the xyz coordinate axis projection, and threshold filtering is performed using the original distance value, laser scanning angle α, pan-tilt rotation angle β, x, y, and z axis coordinate values ​​to filter out unmatched points; the z value of the remaining point (i.e., the height - the laser radar is used as the origin of the xyz relative coordinate system, and the x or y value can also be selected as needed) is stored in the position of the altimeter row and column mapping. Then, the height table is traversed row by row to filter out points with excessive inclination or jumps in height (in the height table, two adjacent points or a set of points within the filter interval are traversed and analyzed row by row (or column). If the difference between the height values ​​of adjacent points is greater than the excessive inclination and jump height error, it is judged as a jump. If the difference between the maximum and minimum height values ​​of the first and last points and the midpoint of the segment within the filter interval is greater than the excessive inclination and jump height error, it is judged as excessive inclination. Among them, when scanning the stacking area and packaging area at a small angle, if the row in the height table corresponds to the length of the bar bundle, the height error is 39mm and the filter interval is 18 when searching by row; if searching by column (corresponding to the width of the bar bundle), the height error is 35mm and the filter interval is 6). The height values ​​of points with continuous equal or gradually decreasing height errors are retained. Due to the large amount of calculation, the height table conversion is divided into four groups and four threads by row for synchronous (approximate) calculation.

[0108] Altimeter outlier filtering: Traverse the altimeter generated in the previous step and calculate the number of neighbors whose absolute value of the height difference between point (m,n) and its neighbors within the outlier filtering neighborhood radius r [(mr) to (m+r), (nr) to (n+r)] is less than the overslope and jump height errors. If the number of neighbors for a point is less than the outlier filtering neighborhood point threshold, the point is considered an outlier and filtered out of the altimeter, i.e., its height value is reset to zero. To shorten computation time, the altimeter outlier filtering process is divided into four groups per row, each threaded synchronously (approximately).

[0109] Convert the altimeter to grayscale: Convert the altimeter values ​​corresponding to the large and small angle scans generated in the previous step from 16-bit to 8-bit data according to the depth map to grayscale ratio, and then use cv::Mat to construct a grayscale image.

[0110] Grayscale image scaling and expansion: Enlarge the grayscale image generated in the previous step according to the image row and column scaling ratios of large and small angle scanning (divided into large and small angle scanning, that is, the large and small angles in the parameter configuration correspond to the image row and column scaling ratios), and then add 10 blank rows at the beginning and end of the grayscale image to expand it, so as to avoid the image occupying the first and last rows when stacking at a small angle, which is not conducive to closed contour search (see Figure 4 ), and the images are beautiful too.

[0111] Figure 4This is a low-angle scan of the stack's longitudinal layer of interest. The original grayscale image is converted from a LiDAR scan of the stack's midpoint. The two wide white areas in the image represent two groups (two bundles each) of bars placed horizontally in the first layer of the stack (the bars' lengths are displayed complete, but their widths are incomplete). The two narrow white areas in the image represent two groups of bars placed vertically in the second layer of the stack (the widths are displayed complete, but their lengths are incomplete). The image appears wide horizontally and narrow vertically because the height table for low-angle scanning uses 50 rows and 720 columns. For ease of image processing and aesthetically pleasing display, the image row and column scaling is 8 for rows and 2 for columns. The two marked gray-white rectangles represent the minimum rectangle of the object's layer of interest. The two marked gray-black groups of points 0 to 3 represent the four vertices of the object's layer of interest. The target contour is the group on the left in the image.

[0112] Statistical histogram analysis of the height layer of interest threshold: By performing histogram statistics and normalization on the grayscale image, the values ​​whose grayscale value points are greater than the histogram threshold lower limit (for example, the parameter is 1.0) are retained, and those less than the threshold value are cleared to zero.

[0113] Then find the continuous segments where the percentage of gray value points is greater than the lower limit of the histogram threshold, and record the continuous and discontinuous number of gray value points in each segment (the vertical axis of the gray histogram is the percentage of gray value points, and the horizontal axis is the gray value 0~255. In a segment, the percentage (vertical axis) of the points of two connected gray values ​​(horizontally) is greater than the lower limit of the histogram threshold, which is considered continuous. The conditions for judging a continuous segment are: (1) the number of continuous points in the segment is 1~3, and if there is more than one discontinuous point in the middle and at the end of the segment, the segment is considered to be over; (2) the segment (1) If there are 4 consecutive numbers in the segment and 2 or more discontinuous numbers in the segment and at the end of the segment, the segment is considered to be over; (2) If there are 5 consecutive numbers in the segment and 3 or more discontinuous numbers in the segment and at the end of the segment, the segment is considered to be over; (3) If there are 5 consecutive numbers in the segment and 3 or more discontinuous numbers in the segment and at the end of the segment, the segment is considered to be over; (4) If there are 6 or more consecutive numbers in the segment and 5 or more discontinuous numbers in the segment and at the end of the segment, the segment is considered to be over), the maximum and minimum values ​​of the proportion of grayscale points in the segment (the proportion of grayscale points - the proportion of the points corresponding to a certain layer of altitude in the altimeter to the total number of points in the altimeter), the starting value and the ending value of the segment.

[0114] Then, by comparing the maximum value of the grayscale value point ratio of the segment and the grayscale value ratio threshold of the histogram interest layer, the valid segments are screened out: the segment with the largest grayscale value point ratio is taken from the packaging area and the first layer of the stack. For the second layer and above of the stack, the maximum value of the grayscale value point ratio of each continuous segment is sorted, and the segments with the maximum value of the grayscale value point ratio greater than the grayscale value ratio threshold of the histogram interest layer are retained. On this basis, the segment with the smallest starting and ending values ​​is selected as the interest segment (that is, the corresponding packaging area or the highest layer of the stack - the height value is smaller), and the ending and starting values ​​of the interest segment are slightly expanded (the starting value of the segment is reduced by 1, and the ending value of the segment is increased by 1, that is, the upper and lower limits of the grayscale value corresponding to the height layer of the segment are relaxed a little) as the upper and lower limits of the grayscale threshold of the interest layer, and divided by the depth map to grayscale map ratio to obtain the height threshold of the interest layer corresponding to the height table.

[0115] Image Interest Layer Thresholding Segmentation and Processing: The grayscale image is thresholded and segmented using the grayscale threshold of the interest layer. Layers with grayscale values ​​within the upper and lower limits of the grayscale threshold are retained as the interest layer, generating an interest layer image. This interest layer image is then subjected to image morphological operations (such as opening, closing, dilation, or erosion, one or more times). An appropriate kernel size is selected to ensure that the overall outline of the suspended object does not expand or contract after the operation (for the kernel size, Size(11,11) is selected for the closing operation, and Size(3,3) is selected for the opening operation; both opening and closing operations include dilation and erosion, and the kernel size is the same), while still filling holes and eliminating protrusions. Interference points are then filtered out using filtering (such as median, mean, or Gaussian filtering), reducing the grayscale difference between pixels before binarization.

[0116] Edge detection and contour search on the interest layer: Perform Canny edge detection on the interest layer image, selecting appropriate low and high thresholds (e.g., 80 and 160) and kernel size (e.g., 3) to generate an edge image. Contour search is performed on the edge image. Input the edge image, set the contour extraction mode to retrieve only the outermost contour, and set the contour representation method to compress horizontal, vertical, and diagonal components, retaining only the last point. Output the contour point vector and contour tree structure vector.

[0117] Filter contours by length and area: Calculate the length and enclosing area of ​​each contour generated in the previous step, and filter out contours whose length is less than the contour length threshold (for example, 500), whose area is less than the contour area threshold (for example, 10000), and whose area-to-length ratio is less than the ratio of the two thresholds. For the remaining contours, store the contour length values ​​in the contour length array and the contour numbers in the contour number array in the corresponding order. Then, sort the contour length array by contour length, and update the contour number array in the corresponding order. Finally, count the number of valid closed contours.

[0118] Contour shape matching based on contour moment and Hu moment: Calculate the contour moment of each contour based on the contour length array, contour number array, and number of valid closed contours obtained in the previous step. Use the contour moment to calculate the Hu moment. Then, use the Hu moment to match the shape (e.g., rectangle, circle, or ellipse) of the suspended object contour samples (e.g., rectangular contour samples of the grayscale image of a bar bundle processed using the laser point cloud to depth image method). Filter out non-rectangular contours (bar bundles have rectangular contours). Reorder the filtered contour length array and contour number array to retain only the number that matches the actual number of stacked bundles (e.g., four bar bundles per layer in a bar stack).

[0119] Finding the Four Vertices of the Minimum Rectangular Box of a Valid Rectangular Outline: By finding the minimum rectangular box of a valid rectangular outline, the coordinates of the four vertices of the minimum rectangular box are obtained. Furthermore, by searching for the minimum rectangular box or elliptical bounding box of a valid rectangular outline, the placement angle of a rectangular object can be initially determined. The placement angle obtained using an elliptical bounding box has been shown to be more stable than that obtained using the minimum rectangular box. Similarly, searching for the minimum enclosing circle of a valid outline can be used to determine the center coordinates and radius of a circular object. Searching for an elliptical bounding box of a valid outline can be used to determine the center coordinates and major axis angle of an elliptical object.

[0120] The four vertices of the minimum rectangular box are approximated by a circle to the nearest points on the contour: The four vertices of the minimum rectangular box are not necessarily on the contour, so the four vertices are used as the center of the circle, and the radius of the circle is continuously increased to approximate the contour. The first intersection point (the closest point) between the circle and the contour is found as the vertex of the rectangular contour.

[0121] The coordinates of the four vertices of the rectangular outline are converted to the coordinate points corresponding to the altimeter: the row coordinates of the four vertices of the rectangular outline are first subtracted by 10 (the value after the subtraction operation is ≥ 0), and then the row and column coordinates are reduced to the row and column coordinates corresponding to the altimeter according to the image zoom ratio.

[0122] Search for vertices that meet the height domain value within the neighborhood of the four altimeter coordinate points: After the row and column coordinates of the four vertices of the rectangular outline are converted to altimeter coordinates, the height values ​​of these four coordinate points on the altimeter may not meet the height threshold of the interest layer. Therefore, it is necessary to search for the first point that meets the height threshold range of the interest layer within the neighborhood of these four coordinate points, starting from the nearest and then the farthest, as the vertex of the suspended object in the altimeter; if the neighborhood radius exceeds the effective vertex search neighborhood radius of the altimeter (for example, 3) and no matching point is found, the vertex search is considered to have failed; the larger the neighborhood radius when the vertex is found, the greater the error in the subsequent calculation of the length, width and coordinates of the suspended object. If the vertex search fails, the vertex coordinates will not be updated in this step, only a failure mark and prompt will be given. In the subsequent step of "calculating the three-dimensional coordinates (x, y, z) of the four vertices of the target contour corresponding to the original point cloud corresponding to the height table", this point will not be calculated. If one of the four vertices fails to be marked, the three-dimensional coordinates can be calculated using the other three vertices to obtain the coordinates of the midpoint and the length, width and height of the suspended object. If two or more vertices fail to be marked, it is directly considered that the suspended object identification detection has failed, and the lidar scan is restarted for calculation. If the suspended object fails to be scanned and calculated multiple times, a repair is reported.

[0123] The target profile is selected based on the geometric features, four vertices, and positions of the effective profile: For the packaging area and the stacking layer of interest, which is horizontal (parallel to the laser scanning direction) and small-angle scanning, since there is only one group of objects (such as a group of bars) in the scanning area, it is directly selected as the target profile. However, for large-angle scanning of stacking or small-angle scanning of stacking layers, which is longitudinal (perpendicular to the laser scanning direction), there may be multiple groups of object profiles in the height layer of interest. Therefore, the target profile should be selected and determined according to the geometric features of the profile and the positions of the four vertices in the stacking order, and the coordinates of the four vertices of the target profile corresponding to the height table should be given.

[0124] Among them, an exemplary explanation of selecting and determining the target contour according to the stacking order is as follows: for the stacking order of bar stacking, the first layer is horizontal, starting from the side away from the X coordinate zero position of the trolley. If two groups have been stacked, the contour with the larger row coordinates of the four vertices on the grayscale image (corresponding to the smaller X coordinate of the driving coordinate) is selected as the target contour to calculate the stacking position, and then the position is used to calculate the current stacking position; the second layer is vertical, starting from the side close to the Y coordinate zero position of the driving trolley. If three groups have been stacked, the contour with the smaller column coordinates of the four vertices on the grayscale image (corresponding to the larger Y coordinate of the driving coordinate) is selected as the target contour.

[0125] Calculate the three-dimensional coordinates (x, y, z) of the original point cloud corresponding to the four vertices of the target contour corresponding to the height table: After obtaining the coordinates of the four vertices of the target contour corresponding to the height table, look up the corresponding values ​​in the original data table and the row and column coordinates to calculate the corresponding gimbal rotation angle β and laser scanning angle α of the point, and then calculate the three-dimensional coordinates (x, y, z) of the point in the relative (lidar) coordinate system.

[0126] Use the three-dimensional coordinates of the four vertices to calculate the length, width, height and midpoint coordinates of the suspended object: After obtaining the three-dimensional coordinate values ​​of the four vertices of the suspended object relative to the (lidar) coordinate system, use the first ( Figure 3 Winning bid 1 point) tail ( Figure 3 3 points) vertex and a non-first and last vertex (such as Figure 3 The length 1 and width 1 of the object to be hoisted can be calculated based on the Pythagorean theorem of a right triangle (the angle of the object to be hoisted can also be calculated). Then, the first and last vertices and another non-first and last vertices (such as the attached Figure 3 (2 points in the bid) Calculate the length 2 and width 2 of the suspended object using the Pythagorean theorem for a right triangle. Use these two lengths and widths for verification or take their average. The x-coordinate of the suspended object's midpoint is calculated by adding the x-values ​​of the four vertices and dividing by 4. The y-coordinate of the suspended object's midpoint is calculated by adding and averaging the y-values ​​of the four vertices. The z-coordinate of the suspended object's midpoint is calculated by taking the average of the z-values ​​of the four vertices. The height of the suspended object is calculated by subtracting the z-value of the suspended object from the ground height. The coordinates of the suspended object's midpoint relative to the coordinates of the midpoint of the electromagnetic hoist (hook) bottom surface require a translation transformation using the coordinates of the relative (lidar) coordinate system.

[0127] Figure 3 In the figure, the white area represents the grayscale value corresponding to the height of each point on the top surface of a group (2 bundles) of horizontally placed bars in the packaging area scanned at a small angle; the marked gray-white rectangular box is the minimum rectangular box of the outline of the interest layer of the suspended object, and the marked gray-black points 0 to 3 are the four vertices of the outline of the interest layer of the suspended object; the small black dot marked in the middle of the minimum rectangular box is the midpoint of the outline. The position of the midpoint of the suspended object obtained by converting the midpoint of the outline to a three-dimensional coordinate point is different from the position of the midpoint of the suspended object calculated by converting the four vertices of the outline to a three-dimensional coordinate point. The midpoint of the suspended object obtained by converting the four vertices of the outline is not affected by image distortion and is more accurate.

[0128] Obtaining the angle of the object being hoisted: The angle of the object being hoisted detected by scanning the packaging area at a small angle is used to rotate the electromagnetic crane to align the long side of the object being hoisted (such as the long side of a bar), and high accuracy is required. The angle of the object being hoisted in the stacking area is mainly used to determine whether the current stacking layer is horizontal or vertical, and low accuracy is required. Method for obtaining the angle of the object being hoisted:

[0129] Method 1: Use the elliptical border of the outline as described in the "Find the Four Vertices of the Minimum Rectangular Frame of the Valid Rectangular Outline" step above to find the long axis angle minus 90 degrees as the placement angle of the object. This method may have errors due to the distortion of the grayscale image outline.

[0130] Method 2: Use the three-dimensional coordinates of the four valid vertices of the outline of the suspended object to calculate the average value of the inverse tangent of the quotient of the two right-angled sides of the right triangle formed by the two long side vertices. When only three valid vertices of the suspended object are found, the inverse tangent of the right triangle formed by the two valid vertices of the long side is calculated to obtain the angle.

[0131] Method 3: Find the long side points of the target contour and convert them back to the three-dimensional xyz coordinates of the long side points of the suspended object, and then fit a straight line in the (x, y) two-dimensional plane (z and other heights) from this set of long side three-dimensional coordinate points to obtain the placement angle. Specific implementation process: traverse the target contour point vector in order, find the order in which the four vertices of the contour appear, count the number of contour points of the segment between the starting point of the contour and the first vertex that appears, and the segment between two consecutive vertices, and store {vertex coordinates, sequence number, number of contour points of the segment} in the order of appearance into structure array 1[4]; then sort the contour points of the four vertex segments from large to small and store {vertex coordinates, sequence number, number of contour points of the segment} in order into structure array 2[4]; traverse the subscript i of structure array 1 from the second to the fourth, Find the subscript value i that matches "structure array 1[i].sequence number" and "structure array 2[0].sequence number", obtain the contour point set of the segment between "structure array 1[i].vertex coordinates" and "structure array 1[i-1].vertex coordinates" (that is, the point set of the long side of the contour), use this set of long side point sets to convert into the three-dimensional coordinate point set {(x, y, z)} of the suspended object, use the three-dimensional coordinate point set of the suspended object to fit a straight line in the xy coordinate plane (z is the same height in the point set) to calculate the angle of the long side of the suspended object as the placement angle of the suspended object.

[0132] The present invention prefers method three (high precision and good stability) for calculating the placement angle of the suspended objects in the packaging area; and prefers method one (qualitative judgment, good stability and easy implementation) for determining the stacking direction of each layer of the stack. Method two is used to verify the correctness of the conclusions of the first two methods and improve the accuracy of the output placement angle.

[0133] Use the height values ​​of the four vertices to calculate the dynamic angle correction coefficient of laser scanning and pan-tilt rotation: use the height average of 20 valid points in the neighborhood of each vertex in the four height tables (referring to selecting each of the four vertices in the corresponding neighborhood range) to replace the vertex z coordinate value (the vertex neighborhood range [row0~row4, column0~column4] contains the vertex (row0, column0, height0) and select 20 points in turn. After removing the maximum and minimum height points other than the vertex, the average height value of the 18 points is calculated. The selected points should preferably include the points in the range [row0~row3, column0~column3]). Use the three-dimensional coordinates (x, y, z) of the two vertices on the long side of the suspended object. The angle between the long side and the horizontal line is calculated as follows: if the object is horizontal (parallel to the laser scanning direction), it is equal to arctan(Δz / Δy); if the object is vertical (perpendicular to the laser scanning direction), it is equal to arctan(Δz / Δx). The average of the two angles with the horizontal line is used as the dynamic correction angle coefficient for the laser scanning or pan / tilt rotation direction. This is used when converting the original point cloud to a height table and calculating 3D coordinates. This method is primarily used to correct the angle coefficient between the long side of the object on the stack surface and the laser scanning or pan / tilt rotation direction when scanning stacking areas at high and low angles (the long side of the object is facing the laser scanning direction when stacked horizontally, and the long side of the object is facing the pan / tilt rotation direction when stacked vertically). For low-angle scanning of packaging areas, where only horizontal objects are present, dynamic correction only needs to be made to the laser scanning angle coefficient. The pan / tilt rotation still uses the default manual correction coefficient (due to the small width of the object, the dynamic angle correction coefficient can have large errors).

[0134] High- and low-angle scanning stacking status determination: The stacking status is determined based on the high- and low-angle stacking scan types, the absolute coordinates of the stack midpoint (X, Y, Z), the relative coordinates of the current stacking position (X, Y, Z), and the length, width, and height of the hoisted object. The absolute coordinates of the stack midpoint (X, Y, Z) are referenced to the vehicle coordinate system, while the relative coordinates of the stacking position (X, Y, Z) are referenced to the LiDAR coordinate system. Stacking status includes five states: empty, full, and not full (including empty, full, and not full layers), as well as the stacking orientation (horizontal and vertical) of the current layer. The calculation is performed by superimposing the absolute coordinates of the vehicle scanning point and the relative coordinates of the current stacking position to convert them into the stacking position corresponding to the vehicle absolute coordinate system. This is then compared with the stacking boundary to determine whether a layer is full or not. If a layer is full, it is moved up one layer to become empty. (If the stack status is determined to be full, the layer is moved up one layer to become empty, and the layer number is incremented by 1, and the z-coordinate is incremented by one layer height.) Empty pile judgment: when the altitude layer of interest is not identified within the upper limit of the stacking threshold after filtering out the ground altitude layer with the corresponding altimeter during large-angle scanning, the pile is judged to be empty; full pile judgment: when the altitude layer of interest is full and the altitude value is less than the lower limit of the stacking threshold, the pile is judged to be full.

[0135] Lifting and stacking position coordinates and angle correction for large and small angle scanning: When the stacking status is empty, the first layer of the stack is set to horizontal stacking by default. The stacking coordinates are given as the position of the first horizontal group of the stack, and the coordinates are corrected according to the actual stacking situation (the average deviation value of the position coordinates of the position point of the crane automatically lifting the hoisted object for more than 5 times and the given stacking point xyz coordinates is corrected to reduce the comprehensive deviation of the crane travel mechanism, crane position detection and object position detection). The stacking angle is set (that is, the electromagnetic crane rotates to the target angle) to 0 degrees. When the stacking status is that the layer is not full, the position coordinates of the stacking location are set according to the position of the previous bundle to be stacked in the current stacking and the stacking spacing of the suspended objects. The stacking angle is set to 0 degrees horizontally and 90 degrees vertically. The stacking coordinates and angle corrections are based on the actual stacking situation and the correction coefficient (the average deviation value (correction coefficient) of the position coordinates of the position point when the suspended objects are automatically lifted by the crane for more than 5 times and the given stacking point xyz coordinates is corrected to reduce the comprehensive deviation of the crane travel mechanism, the crane position detection and the suspended object position detection); when the stacking status is full, it is necessary to raise the layer for stacking (the layer will be empty after raising the layer). The stacking coordinates are the position of the first group on the previous layer. The placement angle needs to be reversed, with the horizontal setting set to 0 degrees and the vertical setting to 90 degrees, and a correction factor is added (horizontal stacking is defined as 0 degrees, and the placement angle is 0 degrees; vertical stacking is defined as 90 degrees, and the placement angle is 90 degrees; the correction factor is obtained by automatically lifting the objects by the crane and stacking them more than 5 times to see the average deviation between the actual stacking direction and the given placement angle, so as to reduce the rotation deviation of the crane rotation mechanism and the comprehensive deviation of the angle detection.); When the stacking status is full, the crane coordinates are set to a fixed waiting position, and the system sends a full stack prompt message to the remote operator.

[0136] Position correction of the next small-angle stacking scanning point for large and small-angle scanning: The small-angle stacking scanning point is also divided into the above four stacking states for coordinate setting and correction.

[0137] When the stack is empty, the first layer of the stack is set to horizontal stacking by default. The next small-angle stacking scan point is given the absolute coordinates (X, Y, Z) of the first stacking position in the stack, with the X coordinate minus the LiDAR scan distance (calculated by the middle angle of the stacking small-angle scan angle range and the stacking layer height, the same below). The placement angle (i.e., the electromagnetic crane rotates to this target angle) is 0 degrees. When the stack is not full, when stacking horizontally, the next small-angle stacking scan point position is calculated based on the absolute coordinates calculated by adding the position of the previous bundle to be stacked in the current stack to the stacking distance, and then subtracting the LiDAR scan distance from the X coordinate. The placement angle is 0 degrees. When stacking vertically, the next small-angle stacking scan point position is calculated based on the absolute coordinates of the current stack midpoint, and then subtracting the LiDAR scan distance from the X coordinate. The placement angle is 0 degrees. When the stacking status is full, the stacking is performed horizontally. The next small-angle stacking scan point position is given as the absolute coordinates (X, Y, Z) of the first stacking group, and the X coordinate is subtracted from the laser radar scanning distance. The placement angle is 0 degrees. When the stacking status is full, the next small-angle stacking scan point position is calculated based on the absolute coordinates of the current stacking midpoint and the X coordinate minus the laser radar scanning distance. The placement angle is 0 degrees. When the stacking status is full, the next small-angle stacking scan point position is calculated based on the absolute coordinates of the current stacking midpoint and the X coordinate minus the laser radar scanning distance. The driving coordinates are set to a fixed waiting position, and the system issues a full stack prompt message to the remote operator.

[0138] Processing of the contour of the interest layer, the minimum rectangular frame and the four vertex images: after a valid closed rectangular contour of the interest layer is selected, the coordinates of the four vertices of the minimum rectangular frame of each contour of the interest layer and the height table corresponding to the suspended object on the contour are converted into image coordinates, marked on the original grayscale image, and then saved as an image file, and then read out from the device for display.

[0139] The coordinates of the object's position and scanning points are output via the embedded control module network. The stacking area's small-angle scanning points, obtained from a large-angle scan of the stacking area, the stacking location and the next small-angle scanning point for stacking, and the object's pick-up point, obtained from a small-angle scan of the packaging area, are all transmitted to the crane's PLC. The large-angle scanning points for the stacking area and the small-angle scanning points for the packaging area are selected and sent by the operator through the remote HMI interface.

[0140] Figure 5 shows an original grayscale image of a group of bars marked during a low-angle scan of the first layer of a stack. This image is converted from a laser radar scan of the stack's previous group of bars at the next low-angle scan point. The white area in the image represents the grayscale values ​​corresponding to the heights of each point on the top surface of the previous group of horizontally placed bars. The marked off-white rectangle represents the minimum rectangle of the object's layer of interest, and the marked off-black points 0 to 3 represent the four vertices of the object's layer of interest.

[0141] The original grayscale image of two groups of bars marked on the second layer of the stacking area is scanned at a large angle. Figure 6 The image shows a high-angle scan of the stacking area, converted using a LiDAR scan directed at the stack's midpoint. The four horizontal white areas in the image represent four groups (two bundles each) of horizontally placed bars in the first layer of the stack (the bar lengths and widths are fully displayed). The two vertical white areas represent two groups of vertically placed bars in the second layer of the stack (the bar lengths and widths are fully displayed, and the image appears relatively balanced vertically and horizontally because the high-angle scan height table is 300 rows by 720 columns. For ease of image processing and aesthetically pleasing display, the image row and column scaling factors are 3 for rows and 2 for columns). The two gray-white vertical rectangles represent the minimum rectangle of the object's layer of interest. The two gray-black groups of points 0 to 3 represent the four vertices of the object's layer of interest. The target contour is the group on the left vertically in the image. It can be seen from the figure that the contour area has certain distortion (larger at the top and smaller at the bottom) and curvature. It can be analyzed that: (1) The placement angles of lifting and stacking obtained by fitting the elliptical frame of the target contour have certain distortion and deviation, but the angle range of the stacking and packaging area is small when scanning at a small angle, and the distortion is also small. The large angle scanning range is relatively stable when scanning at the midpoint of the stack. The placement angle error values ​​of these three scanning types are also stable and can be eliminated by correction. (2) The four vertices of the image contour are converted back to the three-dimensional coordinates xyz of the object being hoisted, so they are not affected by this distortion. (3) According to the idea of ​​item (2), the placement angle is converted back to the three-dimensional xyz coordinates of the long side points of the object being hoisted by using the long side points of the contour. Then, the placement angle obtained by fitting a straight line in the two-dimensional plane (x, y, z and other heights) by this set of three-dimensional coordinate points of the long side is more accurate. See the new content for details, which has been written into a claim.

[0142] Scan the original grayscale image of three groups of bars on the first layer of the stacking area at a large angle and add markings Figure 7 The image shows a high-angle scan of a stacking area, converted from a laser radar scan of three horizontally stacked groups of bars at the stack's midpoint. The three horizontal white areas in the image represent the three horizontally stacked groups of bars in the first layer of the stack (the bar lengths and widths are fully displayed). The three marked gray-white rectangles represent the minimum rectangles of the object's layer of interest, and the three marked gray-black points 0 to 3 represent the four vertices of the object's layer of interest. The target contour is the bottommost group in the image.

[0143] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the foregoing embodiments. The foregoing embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting suspended objects by converting laser point cloud to depth image, characterized in that: The steps include: Step (1), acquisition of original point cloud and driving position and parameter configuration: large-angle scanning of the stacking area, small-angle scanning of the stacking area and small-angle scanning of the packaging area, collecting point cloud data and driving position, and then configuring parameters according to these three scanning types, and dynamically updating the laser scanning angle and pan / tilt rotation angle correction coefficient; Step (2), converting the original point cloud into an altimeter and filtering the data: converting the point cloud data obtained in step (1) into an altimeter, filtering out the data in the domain beyond the range of interest, the domain beyond the slope, and the domain beyond the jump, and then filtering the data in the altimeter for outliers; Step (3), converting the altimeter into a grayscale image and scaling it: converting the altimeter into an original depth image, and then converting the depth image into a grayscale image; Then scale the grayscale image by the zoom factor and add multiple blank lines at the beginning and end to extend it; Step (4), histogram statistics and normalization operation: perform histogram statistics and normalization on the grayscale image, and then find the continuous segment features of the histogram value to obtain the grayscale threshold of the interest layer and the corresponding height threshold of the interest layer; Step (5), threshold segmentation and processing of the image interest layer: threshold segmentation is performed on the grayscale image processed in step (3) using the grayscale threshold of the interest layer, and the layer with grayscale values ​​within the upper and lower limits of the grayscale threshold of the interest layer is retained as the interest layer and an interest layer image is generated; then, the interest layer image is subjected to image morphology operations, filtering, and binarization processing in sequence; Step (6), edge detection and contour search of the interest layer: edge detection is performed on the interest layer image processed in step (5), and then contour search is performed; then the length and area of ​​each contour are calculated, and filtering is performed according to the contour length and area as well as the contour area-to-length ratio respectively. The remaining contours are sorted from large to small by length and the contour length and corresponding contour number are stored in an array, and then the number of valid closed contours is determined; Step (7) matches the contour shape according to the contour moment and Hu moment: calculate the contour moment of the effective closed contour, then use the contour moment to calculate the Hu moment, and then use the Hu moment and the Hu moment of the pre-stored contour shape sample of the suspended object to perform shape matching and filtering, filter out the contour of non-interest shapes, and then sort out the contour length and number array, and calculate the number of suspended objects currently stacked in the interest layer in the image; Step (8), determination of the vertices of the suspended object: obtain the coordinates of the four vertices of the minimum rectangular outer frame of each valid rectangular outline; use the circular approximation method to find the points on the contour that are closest to the four vertices of the minimum rectangular outer frame of the contour; convert the coordinates of these four closest points on the contour to the coordinates corresponding to the altimeter; search for points whose height values ​​meet the height threshold of the interest layer in the neighborhood of these four coordinate points in the altimeter from near to far as the points to be used as the vertices of the suspended object; Step (9), selection and processing of target contour: select the target contour according to the geometric features of the contour of interest, the relative position in the image and the effective value of the vertex in the corresponding height table; then calculate the three-dimensional coordinates (x, y, z) of the original point cloud corresponding to the four vertices of the target contour corresponding to the height table; then use the three-dimensional coordinates of the four vertices of the hoisted object to calculate the length, width, height, midpoint coordinates and placement angle of the hoisted object; Step (10), calculation of the correction coefficients of the laser scanning angle and the pan-tilt rotation angle: select 20 points in the neighborhood of the four vertices of the suspended object and calculate the average height. Use the average height of the four vertex neighborhoods to construct a right triangle in the height direction and the laser scanning direction or the pan-tilt rotation direction. Use the height difference and the vertex spacing to calculate the arc tangent to calculate the new dynamic angle correction coefficient of the laser scanning and pan-tilt rotation directions. Step (11), stacking state judgment and lifting and stacking position coordinates and angle calculation: the stacking state is judged according to the large and small angle stacking scanning types, the absolute coordinates (X, Y, Z) of the stacking midpoint, the relative coordinates (x, y, z) of the current stacking position and the length, width and height of the hoisted object, and the coordinates and placement angles of the lifting point and stacking point of the hoisted object are calculated and corrected; Step (12), calculation of the vehicle position coordinates of the next scanning point: calculation and correction of the vehicle position coordinates of the next scanning point based on large and small angle stacking scans, stacking status and current stacking position; Step (13), output of the position of the suspended object and the coordinates of the scanning point: the calculated coordinates of the lifting point and stacking point of the suspended object and the driving position of the next scanning point are transmitted to the PLC through the network, thereby controlling the driving to travel to the scanning point, lifting point and stacking point.

2. The method for detecting suspended objects by converting laser point clouds to depth images according to claim 1, characterized in that: In step (1), the configured parameters include the coordinates of the current stack center point, the coordinates of the small-angle scanning point in the packaging area, the stacking spacing of the suspended objects, the scanning type, the laser scanning angle correction coefficient, the pan / tilt rotation angle correction coefficient, the outlier filtering neighborhood radius, the outlier filtering neighborhood point number threshold, the image row and column scaling factor, the depth map to grayscale map scaling factor, the interest layer height threshold, the histogram interest layer grayscale ratio threshold, the histogram thresholding lower limit, the contour length threshold, the contour area threshold, the altimeter valid vertex search neighborhood radius, and the total number of altimeter rows and columns, the over-slant and jump height errors, and the filter spacing corresponding to the scanning type.

3. The method for detecting suspended objects by converting laser point clouds to depth images according to claim 1, characterized in that: In step (2), the specific method of outlier filtering is: According to the outlier filtering neighborhood radius r in the parameter configuration, the altimeter is traversed to calculate the number of neighboring points whose absolute value of the height difference between point (m,n) and its neighboring points within the neighborhood radius range [(mr)~(m+r), (nr)~(n+r)] is less than the over-slope and jump height errors. If the number of neighboring points of a point is less than the outlier filtering neighborhood point number threshold, it is considered an outlier and is deleted from the altimeter.

4. The method for detecting suspended objects by converting laser point clouds to depth images according to claim 1, characterized in that: The specific method of step (4) is: Perform histogram statistics and normalization on the grayscale image, retaining the values ​​whose grayscale value percentage is greater than the histogram threshold lower limit, and clearing those less than the lower limit; Then find the continuous segments where the percentage of grayscale points is greater than the lower limit of the histogram threshold, record the continuous and discontinuous number of grayscale points in each segment, the maximum and minimum percentage of grayscale points in the segment, and the starting and ending values ​​of the segment; Then, by comparing the maximum percentage of grayscale points in the segment with the grayscale percentage threshold of the histogram interest layer, valid segments are screened out. The specific method for screening valid segments is as follows: for the packaging area and the first layer of the stack, the valid segment is the segment with the largest percentage of grayscale points; for the second layer and above of the stack, all segments are sorted by the maximum percentage of grayscale points, and the segments with the maximum percentage of grayscale points greater than the grayscale percentage threshold of the histogram interest layer are retained as valid segments. Finally, the segment with the smallest starting and ending values ​​in the valid segment is selected as the segment of interest. The ending and starting values ​​of the segment of interest are slightly expanded as the upper and lower limits of the grayscale threshold of the interest layer, and then divided by the depth map to grayscale map ratio to obtain the height threshold of the interest layer corresponding to the height table.

5. The method for detecting suspended objects by converting laser point cloud to depth image according to claim 1, characterized in that: The specific method of step (6) is: Perform Canny edge detection on the interest layer image processed in step (5), and then perform contour search; Calculate the length and area of ​​each contour, and filter out contours whose length is less than the contour length threshold, whose area is less than the contour area threshold, and whose area-to-length ratio is less than the ratio of the two thresholds; for the remaining contours, store the contour length value in the contour length array, and store the contour number in the contour number array in the corresponding order; Then the contour length array is sorted by contour length, and the contour number array is updated in the corresponding order; and then the number of valid closed contours is counted.

6. The method for detecting suspended objects by converting laser point clouds to depth images according to claim 1, characterized in that: The next time the suspended object is detected, for large and small angle scanning of the stacking area, the laser scanning or pan-tilt rotation dynamic angle correction coefficient calculated in step (10) is used as the angle coefficient for correcting the upper surface of the stack and the laser scanning or pan-tilt rotation direction; for small angle scanning of the packaging area; the laser scanning dynamic angle correction coefficient calculated in step (10) is used as the angle coefficient for dynamically correcting the laser scanning direction, and the pan-tilt rotation still uses the manual correction coefficient; In step (11), the stacking status includes five states: empty stack, full stack, not full stack but empty layer, not full stack but full layer, and not full stack and not full layer.

7. The method for detecting suspended objects by converting laser point clouds to depth images according to claim 1, characterized in that: The method further includes the following steps: after selecting a valid closed rectangular interest contour, marking the interest layer rectangular contour, the minimum rectangular outer frame, and the four vertices of the interest contour corresponding to the vertices of the minimum rectangular outer frame into the original grayscale image for storage and display.

8. The method for detecting suspended objects by converting laser point clouds to depth images according to claim 1, characterized in that: The following steps are also included: the placement angle is converted back into the three-dimensional long side xyz coordinate point set of the suspended object using the long side point set of the grayscale target contour, and then the placement angle is obtained by fitting a straight line in the (x, y) two-dimensional plane using this set of long side three-dimensional coordinate point sets.

9. A suspended object detection device for converting laser point cloud to depth image, characterized in that: It includes an embedded control module, which implements the steps of the suspended object detection method of converting laser point cloud to depth image as described in any one of claims 1 to 8 when executing the program.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for detecting suspended objects by converting laser point cloud to depth image are implemented as described in any one of claims 1 to 8.

Images