An automatic material taking method, system and device for a stockyard stacker-reclaimer

Abstract

The application provides a stockyard stacker-reclaimer automatic material taking method, system and device, and belongs to the technical field of industrial automatic material taking, and comprises the following steps: obtaining stockpile point cloud data; adopting a deep learning target segmentation method to segment the stockpile point cloud data, and performing stockpile three-dimensional model reconstruction on the segmented stockpile point cloud data; determining stacker-reclaimer operation control parameters according to an automatic material taking strategy; controlling stacker-reclaimer movement according to the operation control parameters, and real-time detecting a stockpile boundary, adjusting a stacker-reclaimer operation rotation angle, and realizing stacker-reclaimer automatic material taking. The application can realize automatic material taking for a disordered and irregular stockpile, and realizes real-time detection in the material taking process, and improves material taking operation efficiency.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial automated material retrieving, and in particular relates to an automatic material retrieving method and system for a stockpile machine in a material yard. Background Art

[0002] As the primary operating equipment in a material yard, stackers and reclaimers primarily perform stacking and reclaiming operations. Operators operate the stacker and reclaimer from within a cab. However, the harsh on-site environment can be detrimental to the physical and mental health of operators. With the promotion of environmental protection and the need to improve operators' working environments, automated laser scanning stacker and reclaimer technology has been widely researched. However, the following challenges remain with automated reclaiming:

[0003] In actual production, trucks unload materials and cranes create piles, resulting in disorganized and chaotic material piles and complex on-site working conditions. Under these conditions, semi-automatic material handling is often the primary method. First, due to unclear material pile boundaries, it is impossible to accurately segment the material pile to be processed, making automated operation impossible. Second, due to the irregular and disorganized shape of the material pile, the left and right edges of the material pile are uneven, making it impossible to detect the material pile boundary in real time, reduce idle strokes, and improve work efficiency. Third, due to the irregular material pile, the material handling capacity is unstable even with the same process parameters, resulting in low handling efficiency.

[0004] In addition, since the material pile is very likely to collapse during the material retrieving process, when the material pile collapses, it is easy for the bucket wheel to eat too much material, causing the bucket wheel to be stuck and the bucket to be blocked. At this time, the only option is to suspend the material retrieving and clean up the collapsed material. There is currently no better response strategy. Summary of the Invention

[0005] In response to the defects or shortcomings in the existing technology, the present invention provides an automatic material retrieving method and system for a material yard stacker and reclaimer. For situations where the material pile is disorderly and irregular, automatic material retrieving can be achieved, and real-time detection can be carried out during the material retrieving process, thereby improving the efficiency of the material retrieving operation.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, an embodiment of the present invention provides an automatic material reclaiming method for a stacker-reclaimer in a material yard, comprising:

[0008] Step 1: Obtain the stockpile point cloud data;

[0009] Step 2: Use deep learning target segmentation method to segment the material pile point cloud data, and reconstruct the material pile 3D model based on the segmented material pile point cloud data;

[0010] Step 3: Determine the stacker-reclaimer operation control parameters based on the automatic reclaiming strategy;

[0011] Step 4: Control the movement of the stacker-reclaimer according to the operating control parameters, detect the pile boundary in real time, adjust the stacker-reclaimer operating rotation angle, and realize automatic material retrieving by the stacker-reclaimer.

[0012] In a second aspect, an embodiment of the present invention provides an automatic reclaiming system for a stacker-reclaimer in a material yard, comprising:

[0013] The image acquisition module is installed on both sides of the bucket wheel at the end of the stacker-reclaimer boom to obtain point cloud data of the material pile;

[0014] The segmentation and reconstruction module uses a deep learning target segmentation method to segment the stockpile point cloud data and reconstruct the stockpile 3D model based on the segmented stockpile point cloud data;

[0015] Operation control parameter acquisition module, used to determine the stacker-reclaimer operation control parameters according to the automatic reclaiming strategy;

[0016] The controller module controls the movement of the stacker-reclaimer according to the operating control parameters, detects the pile boundary in real time, adjusts the stacker-reclaimer's operating rotation angle, and realizes automatic material retrieving by the stacker-reclaimer.

[0017] In a third aspect, an embodiment of the present invention provides an automatic material retrieving device for a material yard stacker and reclaimer, comprising a laser scanner, an edge computer, a server, and an operating terminal, wherein the laser scanner, edge computer, server, and operating terminal are connected in sequence.

[0018] The laser scanner is installed on both sides of the bucket wheel at the end of the stacker-reclaimer boom for real-time detection of the pile;

[0019] The edge computer is set in the electrical control room of the stacker-reclaimer to perform real-time processing of laser scanner point cloud data and real-time acquisition of stacker-reclaimer status and motion control;

[0020] The server is set up inside the machine room, and the data processed by the edge computer is uploaded to it, and it also serves as the scheduling center for multiple stackers and reclaimers;

[0021] The operating terminal is set up in a centralized center and is used for human-computer interaction to implement the issuance of operation instructions, monitoring of the operation process, and viewing of operation data.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] 1. The present invention solves the problem of disorderly stacking of materials and adhesion between materials by accurately segmenting the material pile point cloud data, and avoids the situation where the material pile model to be worked cannot be accurately segmented due to unclear material pile boundaries, thereby providing basic data support for automatic material retrieving operations.

[0024] 2. The present invention solves the problem of many empty strokes caused by irregular material piles by using a rotating forward laser scanner to detect boundaries in real time and adjust the operation rotation angle in real time during the material retrieving process, thereby improving the efficiency of the material retrieving operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0026] Figure 1 This is a flow chart of the automatic material reclaiming method of the stacker-reclaimer in the first embodiment of the present invention;

[0027] Figure 2 This is a characteristic structure diagram of the point cloud training model in Example 1 of the present invention;

[0028] Figure 3 This is a connection block diagram of the automatic material taking device in the third embodiment of the present invention;

[0029] Figure 4 Schematic diagram of a boom pitch angle calculation model in Embodiment 1 of the present invention;

[0030] Figure 5 Schematic diagram of the initial rotation angle calculation model in Example 1 of the present invention; DETAILED DESCRIPTION

[0031] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0032] It should be noted that the terms used herein are for describing particular embodiments only and are not intended to limit the exemplary embodiments according to the present invention.

[0033] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0034] Example 1

[0035] A typical embodiment of the present invention is a method for automatically retrieving materials from a stockpile and reclaiming yard, comprising the following steps:

[0036] Step 1: Obtain the stockpile point cloud data;

[0037] Step 2: Use deep learning target segmentation method to segment the material pile point cloud data, and reconstruct the material pile 3D model based on the segmented material pile point cloud data;

[0038] Step 3: Determine the stacker-reclaimer operation control parameters based on the automatic reclaiming strategy;

[0039] Step 4: Control the movement of the stacker-reclaimer according to the operating control parameters, detect the pile boundary in real time, adjust the stacker-reclaimer operating rotation angle, and realize automatic material retrieving by the stacker-reclaimer.

[0040] In step 1, the pile is detected by laser scanners installed on both sides of the bucket wheel at the end of the stacker-reclaimer boom, and the point cloud data of the pile is obtained.

[0041] Because the piles are often cluttered and connected, the traditional clustering segmentation method can be used to roughly segment the pile point cloud data. This can separate the piles with clear boundaries. However, for piles that still have adhesions after rough segmentation, the deep learning target segmentation method is used to segment the pile point cloud data. The specific steps include:

[0042] S101: Collecting material pile point cloud data and converting the material pile point cloud data into a depth map;

[0043] S102: Using the stockpile depth map as a data set, perform data standardization;

[0044] S103: Model training is performed using a standard data set. The model feature structure is as follows: Figure 2 As shown;

[0045] S104: Randomly verify the model training accuracy until it meets the requirements;

[0046] S105: Inputting the roughly segmented point cloud depth map into the trained model to complete the accurate segmentation of the point cloud data;

[0047] In step 3, the stacker-reclaimer operation control parameters are determined according to the automatic reclaiming strategy, including:

[0048] Calculate the top area range. Based on the reconstructed 3D model of the pile, calculate the highest point of the 3D model and the area at the highest point, and determine the top-level reclaiming position and rotation angle of the stacker-reclaimer.

[0049] Specifically, according to the reconstructed stockpile three-dimensional model, a KD binary tree data structure is established based on point cloud data thereof, Z-axis data is taken as a basis, and Z-axis maximum value is solved through traversal screening, so as to determine the highest point of the stockpile. Assuming that the stockpile height is Zmax, in the height range of [Zmax-△Z, Z], a plane at the highest point of the stockpile is fitted by using a RANSAC random sample consensus algorithm, and an X / Y region range is solved at the plane by using an envelope algorithm. According to the highest point of the stockpile, the structure size of the stacker-reclaimer is combined, and the pitch angle of the stacker-reclaimer is determined by using a trigonometric function; similarly, according to the X / Y region range of the highest layer of the stockpile, the length of the boom of the stacker-reclaimer is combined, and the rotation angle and the walking position of the stacker-reclaimer on the horizontal plane are inversely solved by using a trigonometric function.

[0050] Stockpile layering. If the stockpile is a new stockpile, a fixed height can be directly divided according to the single-layer material taking capacity of the stacker-reclaimer; if the stockpile is taken or supplemented, the stockpile needs to be layered according to the original material taking step height, and the material taking step is distributed on one side of the material taking, and the material taking surface is a plane. In the range of the stockpile stacking area, a plurality of Z planes are fitted by using a random sample consensus algorithm (RANSAC), a suitable threshold value is set according to the fluctuation degree of the material taking plane, a plurality of Z planes are obtained, the Z planes at the fitting positions are verified by using the material taking layering height of the stacker-reclaimer, and the noise planes are removed.

[0051] Boundary solving. The rotation boundary angle of each material taking layer of the stacker-reclaimer is solved. The theoretical maximum left / right rotation angles a1 and a2 of the stacker-reclaimer are determined by using an AABB envelope box for each material taking boundary, and a proper rotation allowance a is added for improving safety, so that the actual maximum left / right rotation angles a1+a and a2+a of the stacker-reclaimer can be determined.

[0052] According to the material taking layer height and the front and rear boundary positions, the initial walking position and the boom pitch angle of the stacker-reclaimer are determined.

[0053] Specifically, as shown in Figure 4-Figure 5 the boom pitch angle Pitch=arcsin((H-h) / L) can be determined in combination with the boom size of the stacker-reclaimer and the stockpile height.

[0054] wherein H represents the stockpile height, h represents the height of the boom rotation point of the stacker-reclaimer from the ground, and L represents the length of the boom of the stacker-reclaimer.

[0055] The initial rotation angle Yaw=arcsin(Y1 / Lcos(Pitch)) is determined in combination with the left and right boundaries of the stockpile, the pitch angle and the boom size.

[0056] The initial walking position is determined according to the front and rear boundaries of the material pile, the rotation angle, the pitch angle and the size of the large arm, and the initial walking position is X = X1 - Lcos(Pitch)cos(Yaw);

[0057] The operating control parameters of the stacker-reclaimer include the walking position and the rotation angle of the top layer, the rotation boundary angle of the material taking layer, the initial walking position and the pitch angle of the large arm.

[0058] During the material taking process, the material pile is irregular, resulting in more idle stroke and reducing the material taking efficiency. In step 4, the boundary is detected in real time by using the rotary forward laser scanner during the material taking process, and the rotation angle is adjusted in real time. The specific steps are as follows:

[0059] (1) Data frame combination: real-time acquisition of laser scanner frame data, combination of N frames of data as the basis for boundary judgment;

[0060] (2) Data registration: registration of N frames of real-time acquisition data frames with N frames of data frames acquired at the previous time, coarse registration by using the FPFH algorithm, and fine registration by using the ICP algorithm. According to the registration coincidence degree, it is judged whether the boundary has been exceeded. After exceeding the boundary, continue to rotate the safety allowance angle a, otherwise stop rotating to avoid idle stroke.

[0061] In order to ensure the stability of the material taking process, the material flow detection method combining the bucket wheel current detection with the laser scanning is adopted. The linear relationship between the bucket wheel no-load current, the load current and the load amount is established, and the material taking amount can be roughly calculated through current detection. The laser scanner is installed above the belt, and the instantaneous flow is calculated by using the integral method. The corresponding model of the instantaneous material taking flow and the material taking operation parameters is established, the closed-loop feedback is realized, the key parameters such as the material taking rotation speed and the depth are adjusted in real time according to the material taking process, and the maximum efficiency taking without exceeding the rated load is realized.

[0062] For the problem of material collapse and blocked bucket, the real-time scanning is taken, the three-dimensional reconstructed material surface curve is compared with the standard material surface curve, it is judged whether the material is collapsed, if the material is collapsed, the rotation speed of the stacker-reclaimer is automatically adjusted and the stacker-reclaimer is moved, and the material taking amount is reduced. The blocked bucket is judged by the overload condition of the bucket wheel current, if the bucket is blocked, the stacker-reclaimer is reversely rotated, the load of the bucket wheel is reduced, and the bucket wheel is normally operated, and then the material taking is continued according to the material collapse strategy.

[0063] Example 2

[0064] The embodiment provides a material yard stacker-reclaimer automatic material taking system, which comprises:

[0065] An image acquisition module is arranged on both sides of the bucket wheel at the end of the large arm of the stacker-reclaimer, and is used for acquiring point cloud data of the material pile;

[0066] The segmentation and reconstruction module is used to accurately segment the material pile point cloud data and reconstruct the 3D model of the segmented material pile point cloud data;

[0067] Operation control parameter acquisition module, used to determine the stacker-reclaimer operation control parameters according to the automatic reclaiming strategy;

[0068] The controller module is used to control the movement of the stacker-reclaimer according to the operation control parameters to realize automatic material reclaiming.

[0069] Example 3

[0070] This embodiment provides an automatic material reclaiming device for a material yard stacker and reclaimer, such as Figure 3 As shown, it includes a laser scanner, an edge computer, a server and an operation terminal, and the laser scanner, edge computer, server and operation terminal are connected in sequence.

[0071] The laser scanner is installed on both sides of the bucket wheel at the end of the stacker-reclaimer boom to detect the pile in real time and obtain the point cloud data of the pile;

[0072] The edge computer is set in the electrical control room of the stacker-reclaimer to perform real-time processing of laser scanner point cloud data and real-time acquisition of stacker-reclaimer status and motion control;

[0073] The server is set up inside the machine room, and the data processed by the edge computer is uploaded to it, and it also serves as the scheduling center for multiple stackers and reclaimers;

[0074] The operating terminal is set up in the centralized control center and is used for human-computer interaction to implement the issuance of operation instructions, monitoring of the operation process, and viewing of operation data.

[0075] The advantage of this device architecture is that the processing of key data, calculation and issuance of device control instructions are completed at the device terminal, and safe operation can be achieved even when the server crashes during the operation process. Preprocessing and calculation of huge point cloud data at the device terminal can reduce the amount of ultra-long-distance data transmission and the computing pressure on the server.

[0076] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. A method for automatically retrieving materials by a stacker-reclaimer in a material yard, characterized in that: The following steps are involved: Obtaining stockpile point cloud data; The deep learning target segmentation method is used to segment the material pile point cloud data, and the segmented material pile point cloud data is used to reconstruct the material pile 3D model: Collect material pile point cloud data and convert it into a depth map; The stockpile depth map is used as the data set for data standardization; Model training using standard datasets; Randomly verify the model training accuracy until it meets the requirements; The point cloud depth map after rough segmentation is input into the trained model to complete the accurate segmentation of the point cloud data; Determine the stacker-reclaimer operation control parameters based on the automatic reclaiming strategy; The stacker-reclaimer movement is controlled according to the operating control parameters, and the stockpile boundary is detected in real time to adjust the stacker-reclaimer operating rotation angle to realize automatic material retrieving by the stacker-reclaimer.

2. The automatic material reclaiming method for a stacker-reclaimer in a material yard according to claim 1, characterized in that: The stacker-reclaimer operation control parameters include the top reclaiming running position and rotation angle, the reclaiming layer rotation boundary angle, the initial running position and the boom pitch angle.

3. The automatic material reclaiming method for a stacker-reclaimer in a material yard according to claim 1, characterized in that: The stacker-reclaimer operation control parameters are determined according to the automatic reclaiming strategy, specifically: Calculate the top area range: Based on the reconstructed 3D model of the stockpile, calculate the highest point of the 3D model and the area at the highest point, and determine the top-level reclaiming position and rotation angle of the stacker-reclaimer. Solve and determine the rotation boundary angle of each material taking layer; The 3D model of the stockpile is layered, and the boundaries of each layer are bounded using AABB envelopes to determine the theoretical maximum left / right rotation angles a1 and a2 of the stacker / reclaimer. To improve safety, an appropriate rotation margin a is added. At this point, the actual maximum left / right rotation angles a1+a and a2+a of the stacker / reclaimer can be determined. The initial travel position of the stacker-reclaimer and the pitch angle of the boom are determined based on the height of the reclaiming layer and the positions of the front and rear boundaries.

4. The automatic material reclaiming method for a stacker-reclaimer in a material yard according to claim 3, characterized in that: The three-dimensional model of the stockpile is layered as follows: If the stockpile is new, it can be directly divided into fixed heights according to the single-layer reclaiming capacity of the stacker-reclaimer. If the stockpile has been reclaimed or restocked, it needs to be layered according to the original reclaiming step height. The reclaiming steps are distributed on one side of the reclaiming, and the reclaiming surface is a plane. Within the stackable area of ​​the stockpile, a random sampling consistency algorithm is used to fit multiple Z planes. A suitable threshold is set according to the undulation of the reclaiming plane. After obtaining multiple Z planes, the Z planes at the fitting location are tested with the stacker-reclaimer reclaiming layer height to eliminate noise surfaces.

5. The automatic material reclaiming method for a material yard stacker and reclaimer according to claim 1, characterized in that: The real-time detection of the stockpile boundary and adjustment of the stacker-reclaimer operating rotation angle are specifically as follows: Data frame combination: real-time acquisition of laser scanner frame data, combining N frames of data as the basis for boundary judgment; Data registration: The N frames of data collected in real time are registered with the N frames of data collected at the previous moment. The FPFH algorithm is used for coarse registration, and the ICP algorithm is used for fine registration. Based on the registration overlap, it is determined whether the stockpile boundary has been exceeded. If the boundary has been exceeded, the stacker-reclaimer will continue to rotate by the safety margin angle a. Otherwise, the stacker-reclaimer will stop rotating to avoid empty travel.

6. An automatic reclaiming system for a stacker-reclaimer in a material yard, characterized in that: include: The image acquisition module is installed on both sides of the bucket wheel at the end of the stacker-reclaimer boom to obtain point cloud data of the material pile; The segmentation and reconstruction module uses deep learning target segmentation methods to segment the material pile point cloud data and reconstruct the material pile 3D model based on the segmented material pile point cloud data: Collect material pile point cloud data and convert it into a depth map; The stockpile depth map is used as the data set for data standardization; Model training using standard datasets; Randomly verify the model training accuracy until it meets the requirements; The point cloud depth map after rough segmentation is input into the trained model to complete the accurate segmentation of the point cloud data; Operation control parameter acquisition module, used to determine the stacker-reclaimer operation control parameters according to the automatic reclaiming strategy; The controller module controls the movement of the stacker-reclaimer according to the operating control parameters, detects the pile boundary in real time, adjusts the stacker-reclaimer's operating rotation angle, and realizes automatic material retrieving by the stacker-reclaimer.

7. An automatic material reclaiming device for a stacker-reclaimer in a material yard, used to implement the automatic material reclaiming method for a stacker-reclaimer in a material yard according to any one of claims 1 to 5, characterized in that: comprising a laser scanner and an edge computer, wherein the laser scanner is electrically connected to the edge computer; The laser scanner is installed on both sides of the bucket wheel at the end of the stacker-reclaimer boom to detect the pile in real time and obtain the point cloud data of the pile; The edge computer is set in the electrical control room of the stacker and reclaimer to perform real-time processing of laser scanner point cloud data and real-time collection and motion control of the stacker and reclaimer status.

8. The automatic reclaiming device for a stacker and reclaimer in a material field according to claim 7, characterized in that: It also includes a server and an operation terminal, wherein the server is electrically connected to the edge computer and the operation terminal respectively.

9. The automatic reclaiming device for a stacker and reclaimer in a material field according to claim 8, characterized in that: The server is set up inside the machine room, and the data processed by the edge computer is uploaded to it, and it also serves as the scheduling center for multiple stackers and reclaimers; The operating terminal is set up in the centralized control center and is used for human-computer interaction to implement the issuance of operation instructions, monitoring of the operation process, and viewing of operation data.

Images