An automatic material handling method and system for a gantry bucket wheel excavator based on laser point cloud data

Abstract

This invention discloses an automatic material handling method for a gantry bucket wheel excavator based on laser point cloud data, which can improve the accuracy and efficiency of material handling operations, enhance the safety and reliability of equipment operation, and save maintenance costs. It includes the following steps: S110, scanning the working area of ​​the gantry bucket wheel excavator with a laser scanner to obtain laser point cloud data of the coal pile to be handled; S120, constructing a three-dimensional model of the coal pile based on the laser point cloud data of the coal pile to be handled; S130, obtaining initial material handling parameters, including coal flow control parameters, trolley stepping parameters, and bucket wheel layer thickness parameters; S140, performing feature fusion correlation analysis on the initial material handling parameters and the three-dimensional model of the coal pile to be handled to obtain material handling parameter-coal pile three-dimensional fusion features; S150, determining a recommended slewing angle value and performing automatic control based on the material handling parameter-coal pile three-dimensional fusion features.

Description

Technical Field

[0001] This application relates to the field of gantry bucket wheel excavators, specifically to an automatic material handling method for gantry bucket wheel excavators based on laser point cloud data. The present invention also provides an automatic material handling system for gantry bucket wheel excavators based on laser point cloud data. Background Technology

[0002] Large stacker-reclaimer equipment in thermal power plants is mainly divided into cantilever bucket wheel stacker-reclaimers (including single-function cantilever bucket wheel stackers and cantilever bucket wheel reclaimers), gantry bucket wheel stacker-reclaimers, and circular stacker-reclaimers. Gantry bucket wheel stacker-reclaimers are large-scale mechanical equipment used for loading and unloading bulk materials (such as coal and ore), and are widely used in thermal power plants and other fields. However, due to the constantly changing shape and position of the coal pile, the reclaiming parameters of the bucket wheel stacker need to be adjusted according to real-time coal pile information, which places high demands on the bucket wheel stacker's control system. Currently, the reclaiming parameters of the bucket wheel stacker are mainly set manually, which suffers from problems such as large errors, low efficiency, and poor stability.

[0003] Specifically, for coal-fired power plants, blending coal is an effective way to improve economic efficiency. Different types of coal in thermal power plants are piled up or layered in the coal yard. After the boiler determines the coal stacking and blending plan based on the load and existing coal storage, it needs to guide the bucket wheel excavator to perform the correct coal stacking and unloading operations. However, since the bucket wheel excavator is operated manually on-site, especially given the complex coal storage structure in the coal yard, there is a possibility of unloading the wrong type of coal, which can cause the blending plan to fail, affecting economic efficiency and environmental emissions, and in severe cases, even causing coking and safety issues in the boiler.

[0004] Therefore, an optimized automatic material handling scheme for gantry bucket wheel excavators is desired. Summary of the Invention

[0005] To address the aforementioned technical problems, this application is proposed. This application provides an automatic material handling method for a gantry bucket wheel excavator based on laser point cloud data. This method can improve the accuracy and efficiency of material handling operations, enhance the safety and reliability of equipment operation, save maintenance costs, and simultaneously ensure the economic efficiency and safety of the co-firing scheme, thus optimizing production efficiency and safety.

[0006] An automatic material handling method for a gantry bucket wheel excavator based on laser point cloud data, characterized by comprising the following steps:

[0007] S110. The working area of ​​the gantry bucket wheel excavator is scanned by a laser scanner to obtain laser point cloud data of the coal pile to be removed, wherein the laser point cloud data is represented as (x, y, z, d), where x, y, z represent the spatial coordinates of each pixel and d represents the depth value of each pixel.

[0008] S120. Based on the laser point cloud data of the coal pile object to be removed, construct a three-dimensional model of the coal pile object to be removed.

[0009] S130. Obtain initial material handling parameters, wherein the initial material handling parameters include coal flow control parameters, trolley stepping parameters, and wheel bucket layer replacement thickness parameters.

[0010] S140. Perform feature fusion correlation analysis on the initial material extraction parameters and the three-dimensional model of the coal pile to be extracted to obtain the material extraction parameter-coal pile three-dimensional fusion features.

[0011] S150. Based on the material taking parameters and the three-dimensional fusion characteristics of the coal pile, determine the recommended rotation angle value and perform automatic control.

[0012] An automatic material handling system for a gantry bucket wheel excavator based on laser point cloud data, comprising:

[0013] The laser point cloud data acquisition module is used to scan the working area of ​​the gantry bucket wheel excavator with a laser scanner to obtain the laser point cloud data of the coal pile to be removed. The laser point cloud data is represented as (x, y, z, d), where x, y, z represent the spatial coordinates of each pixel and d represents the depth value of each pixel.

[0014] The three-dimensional model construction module is used to construct a three-dimensional model of the coal pile object based on the laser point cloud data of the coal pile object to be taken.

[0015] The initial material handling parameter acquisition module is used to acquire the initial material handling parameters, which include coal flow control parameters, trolley stepping parameters, and wheel bucket layer replacement thickness parameters.

[0016] The feature fusion and correlation analysis module is used to perform feature fusion and correlation analysis on the initial material extraction parameters and the three-dimensional model of the coal pile to be extracted, so as to obtain the material extraction parameter-coal pile three-dimensional fusion features; and

[0017] And a control module, used to determine the recommended rotation angle value and perform automatic control based on the material taking parameters-coal pile three-dimensional fusion characteristics.

[0018] Compared with existing technologies, the automatic material handling method and system for gantry bucket wheel excavators based on laser point cloud data provided in this application first scans the working area of ​​the gantry bucket wheel excavator with a laser scanner to obtain laser point cloud data of the coal pile to be handled. Then, based on the laser point cloud data of the coal pile, a three-dimensional model of the coal pile is constructed. Next, initial material handling parameters are obtained, including coal flow control parameters, trolley stepping parameters, and bucket wheel layer thickness parameters. Then, feature fusion and correlation analysis are performed on the initial material handling parameters and the three-dimensional model of the coal pile to obtain the material handling parameter-coal pile three-dimensional fusion feature. Finally, based on the material handling parameter-coal pile three-dimensional fusion feature, a recommended slewing angle value is determined and automatic control is implemented. This improves production efficiency and safety. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. The following drawings are not deliberately drawn to scale according to the actual size, but are intended to show the main idea of ​​this application.

[0020] Figure 1 This is a flowchart of an automatic material handling method for a portal bucket wheel excavator based on laser point cloud data, according to an embodiment of this application.

[0021] Figure 2 This is a schematic diagram of the architecture of the automatic material handling method for a portal bucket wheel excavator based on laser point cloud data according to an embodiment of this application;

[0022] Figure 3 This is a flowchart of sub-step S140 of the automatic material handling method for a portal bucket wheel excavator based on laser point cloud data according to an embodiment of this application;

[0023] Figure 4 This is a flowchart of sub-step S150 of the automatic material handling method for a portal bucket wheel excavator based on laser point cloud data according to an embodiment of this application;

[0024] Figure 5 This is a block diagram of an automatic material handling system for a portal bucket wheel excavator based on laser point cloud data, according to an embodiment of this application.

[0025] Figure 6 This is an application scenario diagram of the automatic material handling method for a portal bucket wheel excavator based on laser point cloud data according to an embodiment of this application. Detailed Implementation

[0026] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are also within the scope of protection of this application.

[0027] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not specifically singular and may include plural forms. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0028] While this application makes various references to certain modules of the systems according to embodiments of this application, any number of different modules can be used and run on user terminals and / or servers. The modules described are merely illustrative, and different aspects of the systems and methods may use different modules.

[0029] Flowcharts are used in this application to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0030] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0031] It is understandable that 3D imaging is a key technology for achieving intelligent and unmanned stacker-reclaimer equipment. 3D imaging technology not only provides "eyes" for equipment operation but also offers intuitive and reliable visual displays for manual monitoring. Current 3D imaging technologies generally utilize laser scanners, combining the spatial location data of the scanner to determine the distance from points on the stockpile surface to the laser scanner. Laser point cloud data is obtained through calculation, and then point cloud neighborhood relationships are established based on the existing point cloud data structure. Noise filtering and simplification methods for the point cloud data are studied to ultimately achieve complete and structured 3D point data of the stockpile. Finally, graphics processing technology is used to obtain the 3D coordinate values ​​of the stockpile surface, which are then displayed in 3D on the central control room interface, forming a complete real-time 3D imaging system for laser-scanned stockpiles.

[0032] Based on this, the technical concept of this application is to obtain laser point cloud data of the coal pile to be retrieved by scanning the working area of ​​the gantry bucket wheel excavator with a laser scanner, and to construct a three-dimensional model of the coal pile based on the laser point cloud data. Thus, given initial retrieving parameters, such as coal flow control parameters, trolley stepping parameters, and bucket wheel layer thickness parameters, a data processing and analysis algorithm is introduced at the back end to perform collaborative analysis of the laser point cloud data of the coal pile to be retrieved and the initial retrieving parameters. This allows the calculation of the initial position for the gantry bucket wheel excavator to retrieve coal, such as the slewing angle value. This parameter is then transmitted to the control system of the bucket wheel excavator to control the slewing angle value for retrieving coal. In this way, the retrieving parameters and operating posture of the gantry bucket wheel excavator can be adaptively adjusted based on the laser point cloud data to complete automated retrieving operations, thereby improving the accuracy and efficiency of retrieving operations, enhancing the safety and reliability of equipment operation, saving maintenance costs, and ensuring the economy and safety of the co-firing scheme, thus optimizing production efficiency and safety.

[0033] like Figure 1 and Figure 2 As shown, the automatic material handling method for a gantry bucket wheel excavator based on laser point cloud data according to an embodiment of this application includes the following steps:

[0034] S110, the working area of ​​the gantry bucket wheel is scanned by a laser scanner to obtain laser point cloud data of the coal pile to be removed, wherein the laser point cloud data is represented as (x, y, z, d), where x, y, z represent the spatial coordinates of each pixel and d represents the depth value of each pixel.

[0035] S120, Based on the laser point cloud data of the coal pile object to be removed, construct a three-dimensional model of the coal pile object to be removed;

[0036] S130, Obtain initial material handling parameters, wherein the initial material handling parameters include coal flow control parameters, trolley stepping parameters, and wheel bucket layer replacement thickness parameters;

[0037] S140, Perform feature fusion correlation analysis on the initial material extraction parameters and the three-dimensional model of the coal pile object to be extracted to obtain the material extraction parameter-coal pile three-dimensional fusion feature;

[0038] S150, based on the material taking parameters and the three-dimensional fusion characteristics of the coal pile, determine the recommended rotation angle value and perform automatic control.

[0039] Specifically, in the technical solution of this application, firstly, the working area of ​​the gantry bucket wheel excavator is scanned using a laser scanner to obtain laser point cloud data of the coal pile to be retrieved. The laser point cloud data is represented as (x, y, z, d), where x, y, and z represent the spatial coordinates of each pixel, and d represents the depth value of each pixel. It should be understood that the laser point cloud data, acquired through a laser scanner, contains a large amount of point cloud information; each point represents a pixel on the surface of the coal pile, possessing spatial coordinates and a depth value. Therefore, to better understand and describe the geometry and spatial distribution of the coal pile, the technical solution of this application further constructs a three-dimensional model of the coal pile based on the laser point cloud data of the coal pile to be retrieved. In other words, by processing and analyzing the laser point cloud data of the coal pile to be retrieved, these discrete point cloud data can be transformed into a three-dimensional model of the coal pile, thereby more intuitively presenting the shape and structure of the coal pile. This helps to more accurately understand the height, volume, surface curvature, and other information of the coal pile, providing a foundation for subsequent calculations and control.

[0040] Then, in order to determine the basic initial control parameters of the gantry bucket wheel excavator during automatic material handling, so as to more accurately perform adaptive control of subsequent material handling parameters and thus achieve an accurate and efficient material handling process, the technical solution of this application further obtains the initial material handling parameters, which include coal flow control parameters, trolley stepping parameters, and bucket wheel layer replacement thickness parameters.

[0041] In the automatic material handling process of a gantry bucket wheel excavator, both the initial material handling parameters and the three-dimensional model of the coal pile are crucial input information. The initial material handling parameters contain key initial parameter data controlling the material handling process of the gantry bucket wheel excavator, while the three-dimensional model of the coal pile provides information on its geometry and spatial distribution. Therefore, to better utilize these data parameters, they need to be fused to enable more precise control of the gantry bucket wheel excavator's operating parameters. Specifically, in the technical solution of this application, the initial material handling parameters and the three-dimensional model of the coal pile to be handled are further processed through a MetaNet fusion module containing an image encoder and a sequence encoder to obtain a three-dimensional feature map of the coal pile containing material handling parameter features. In particular, the MetaNet fusion module is a neural network structure comprising an image encoder and a sequence encoder. The image encoder processes the 3D model of the coal pile to be extracted, extracting the 3D feature distribution information from the model, i.e., capturing the visual features of the coal pile and converting them into image feature representations. The sequence encoder processes the initial extraction parameters, extracting the correlation feature information between the various data items in the initial extraction parameters and converting it into a sequence feature representation. Then, the feature distribution information of the 3D coal pile model and the correlation feature information of each data item in the initial extraction parameters are interactively fused to control the relevant characteristics of each feature channel, helping the network focus on specific parts of each feature channel, thereby depicting a 3D feature map of the coal pile that integrates the geometric feature information of the coal pile and the key feature correlation information of the extraction control parameters, including the features of the extraction parameters.

[0042] In step S140, feature fusion correlation analysis is performed on the initial material extraction parameters and the three-dimensional coal pile model of the coal pile object to be extracted to obtain the material extraction parameter-coal pile three-dimensional fusion features. This includes: passing the initial material extraction parameters and the three-dimensional coal pile model of the coal pile object to be extracted through a MetaNet fusion module containing an image encoder and a sequence encoder to obtain a three-dimensional coal pile feature map containing the material extraction parameter features as the material extraction parameter-coal pile three-dimensional fusion features.

[0043] More specifically, such as Figure 3 As shown, the initial material extraction parameters and the three-dimensional model of the coal pile to be extracted are processed by a MetaNet fusion module containing an image encoder and a sequence encoder to obtain a three-dimensional feature map of the coal pile containing material extraction parameter features, which is used as the material extraction parameter-coal pile three-dimensional fusion feature. The process includes the following steps:

[0044] S141, the three-dimensional model of the coal pile to be taken is passed through the image encoder of the MetaNet module to obtain the three-dimensional feature map of the coal pile to be taken.

[0045] S142, Encode the initial material picking parameters to obtain an initial material picking parameter encoding vector;

[0046] S143, the material initial parameter encoding vector is passed through the first convolutional layer of the sequence encoder in the MetaNet module and then linearly corrected by the ReLU function to obtain the linearly corrected material initial parameter associated feature vector;

[0047] S144, the linearly corrected material initial parameter associated feature vector is passed through the second convolutional layer of the sequence encoder in the MetaNet module and then processed by the Sigmoid function to obtain the activated material initial parameter associated feature vector.

[0048] S145, the three-dimensional feature map of the coal pile object to be extracted is weighted and fused along the channel dimension using the feature vector associated with the initial parameters of material extraction after activation to obtain the three-dimensional feature map of the coal pile containing the material extraction parameter features.

[0049] Furthermore, the three-dimensional feature map of the coal pile containing the material handling parameters is decoded to obtain a decoded value, which represents the recommended slewing angle value. In other words, the three-dimensional feature information of the coal pile containing the material handling parameters is decoded and regressed to control the slewing angle value of the gantry bucket wheel excavator. This allows for adaptive adjustment of the material handling parameters and operating posture of the gantry bucket wheel excavator based on laser point cloud data, enabling automated material handling operations, thereby improving the accuracy and efficiency of material handling, enhancing the safety and reliability of equipment operation, and saving maintenance costs.

[0050] In step S150, as Figure 4 As shown, based on the material handling parameters and the three-dimensional fusion characteristics of the coal pile, the recommended rotation angle value is determined and automatically controlled, including the following steps:

[0051] S151, the three-dimensional feature map of the coal pile containing the material taking parameters is decoded by a decoder to obtain a decoded value, the decoded value being used to represent the recommended rotation angle value;

[0052] S152, based on the decoded value, control the rotation angle of the gantry bucket wheel excavator to perform material handling.

[0053] Furthermore, in the technical solution of this application, the automatic material handling method for a gantry bucket wheel excavator based on laser point cloud data further includes a training step: for training the MetaNet fusion module containing an image encoder and a sequence encoder and the decoder.

[0054] In one example, the training steps include: acquiring training data, which includes training laser point cloud data of the coal pile object to be retrieved, training initial parameters for retrieving coal, and the true value of the recommended rotation angle; constructing a training 3D model of the coal pile object based on the training laser point cloud data of the coal pile object to be retrieved; passing the training initial parameters for retrieving coal and the training 3D model of the coal pile object to be retrieved through the MetaNet fusion module containing an image encoder and a sequence encoder to obtain a 3D feature map of the coal pile containing retrieving parameters; passing the training 3D feature map of the coal pile containing retrieving parameters through the decoder to obtain a decoding loss function value; and training the MetaNet fusion module containing an image encoder and a sequence encoder and the decoder based on the decoding loss function value and through directional propagation of gradient descent.

[0055] In a specific example, passing the training coal pile 3D feature map containing material extraction parameter features through the decoder to obtain the decoding loss function value includes: expanding the training coal pile 3D feature map containing material extraction parameter features along each feature matrix of the channel dimension to obtain the training coal pile 3D feature expansion vector containing material extraction parameter features; optimizing the feature distribution of the training coal pile 3D feature expansion vector containing material extraction parameter features to obtain an optimized training coal pile 3D feature expansion vector containing material extraction parameter features; and passing the optimized training coal pile 3D feature map containing material extraction parameter features through the decoder to obtain the decoding loss function value.

[0056] In the technical solution of this application, by using the training initial parameters for material extraction and the three-dimensional model of the coal pile object to be extracted through a MetaNet fusion module containing an image encoder and a sequence encoder, the resulting three-dimensional feature map of the coal pile containing the features of the extraction parameters can express the signal features of the local correlation features at the three-dimensional correlation scale of the training three-dimensional model of the coal pile object to be extracted, which combines the one-dimensional correlation scale of the initial parameters for material extraction and the three-dimensional correlation scale of the training three-dimensional model of the coal pile object. That is, the training three-dimensional feature map of the coal pile containing the features of the extraction parameters has the image semantic feature representation of the three-dimensional model of the coal pile at different correlation scales.

[0057] When the three-dimensional feature map of the coal pile containing the material extraction parameter features is decoded and regressed through the decoder, the three-dimensional feature map of the coal pile containing the material extraction parameter features is expanded into a three-dimensional feature expansion vector of the coal pile containing the material extraction parameter features. At the global scale, due to the difference in accuracy between the associated features at different associated scales, the training effect of the three-dimensional feature map of the coal pile containing the material extraction parameter features when trained through the decoder will be affected. Therefore, during the training process, the applicant of this application performs feature accuracy alignment on the three-dimensional feature expansion vector of the coal pile containing the material extraction parameter features, for example denoted as V, based on scale representation and inversion recovery.

[0058] In a specific example, optimizing the feature distribution of the trained 3D feature unfolded vector of the coal pile containing material extraction parameter features to obtain an optimized trained 3D feature unfolded vector of the coal pile containing material extraction parameter features includes: optimizing the feature distribution of the trained 3D feature unfolded vector of the coal pile containing material extraction parameter features using the following optimization formula to obtain the optimized trained 3D feature unfolded vector of the coal pile containing material extraction parameter features; wherein, the optimization formula is:

[0059]

[0060] Where V is the three-dimensional feature expansion vector of the coal pile containing material extraction parameter features, v i v′ is the feature value at the i-th position of the three-dimensional feature expansion vector V of the coal pile containing material extraction parameters, ||V||0 represents the zero norm of the three-dimensional feature expansion vector V of the coal pile containing material extraction parameters, L is the length of the three-dimensional feature expansion vector V of the coal pile containing material extraction parameters, and α is the weight hyperparameter. i It is the feature value at the i-th position of the three-dimensional feature expansion vector of the coal pile containing material extraction parameter features in the optimized training.

[0061] To address the accuracy discrepancy between scale-based high-dimensional feature encoding of signal features and associated feature editing, the feature accuracy alignment based on scale representation and inversion recovery is achieved by treating associated feature editing as an inversion embedding of high-dimensional feature encoding of signal features. This is accomplished by equipping the feature values, which serve as coded representations, with sparse distribution equalization based on scale representation, and by performing inversion recovery of associated details based on vector counting. This adaptive alignment of accuracy differences during training improves the training effect when the coal pile 3D feature map, which includes material handling parameter features, is decoded and regressed by the decoder. This enables adaptive control of the material handling parameters and operating posture of the gantry bucket wheel excavator, automating material handling operations. This improves the accuracy and efficiency of material handling operations, as well as the safety and reliability of equipment operation, saving maintenance costs. Simultaneously, it ensures the economy and reliability of the co-firing scheme, optimizing production efficiency and safety.

[0062] Further, the optimization training of the three-dimensional coal pile feature map containing material extraction parameter features is passed through the decoder to obtain the decoding loss function value, including: using the decoder to perform decoding regression on the optimization training of the three-dimensional coal pile feature map containing material extraction parameter features using the following decoding formula to obtain the decoding loss function value; wherein, the decoding formula is:

[0063]

[0064] Among them, F d The optimized training represents a 3D feature map of the coal pile containing material extraction parameter features, Y represents the decoding loss function value, and W represents the weight matrix. This represents matrix multiplication.

[0065] In summary, the automatic material handling method for gantry bucket wheel excavators based on laser point cloud data, as described in the embodiments of this application, has been clarified. It can improve the accuracy and efficiency of material handling operations, enhance the safety and reliability of equipment operation, save maintenance costs, and ensure the economy and safety of the co-firing scheme, thereby optimizing production efficiency and safety.

[0066] like Figure 5 As shown, the automatic material handling system 100 for a portal bucket wheel excavator based on laser point cloud data according to an embodiment of this application includes: a laser point cloud data acquisition module 110, a three-dimensional model construction module 120, a material handling initial parameter acquisition module 130, a feature fusion and correlation analysis module 140, and a control module 150.

[0067] The laser point cloud data acquisition module 110 is used to scan the working area of ​​the gantry bucket wheel excavator with a laser scanner to obtain the laser point cloud data of the coal pile to be removed. The laser point cloud data is represented as (x, y, z, d), where x, y, z represent the spatial coordinates of each pixel and d represents the depth value of each pixel.

[0068] The 3D model construction module 120 is used to construct a 3D model of the coal pile object based on the laser point cloud data of the coal pile object to be taken.

[0069] The initial material handling parameter acquisition module 130 is used to acquire the initial material handling parameters, which include coal flow control parameters, trolley stepping parameters, and wheel bucket layer replacement thickness parameters.

[0070] The feature fusion and correlation analysis module 140 is used to perform feature fusion and correlation analysis on the initial material extraction parameters and the three-dimensional model of the coal pile object to be extracted to obtain the material extraction parameter-coal pile three-dimensional fusion features.

[0071] The control module 150 is used to determine the recommended rotation angle value and perform automatic control based on the material taking parameters and the three-dimensional fusion characteristics of the coal pile.

[0072] In one example, in the above-mentioned automatic material handling system 100 for a portal bucket wheel excavator based on laser point cloud data, the feature fusion and correlation analysis module 140 is used to: use the initial material handling parameters and the three-dimensional model of the coal pile object to be handled to obtain a three-dimensional feature map of the coal pile containing material handling parameter features through a MetaNet fusion module containing an image encoder and a sequence encoder as the material handling parameter-coal pile three-dimensional fusion feature.

[0073] Here, those skilled in the art will understand that the specific functions and operations of each module in the aforementioned automatic material handling system 100 for gantry bucket wheel excavators based on laser point cloud data have been referenced above. Figures 1 to 4 The automatic material handling method for gantry bucket wheel excavators based on laser point cloud data has been described in detail, and therefore, its repeated description will be omitted.

[0074] As described above, the automatic material handling system 100 for a gantry bucket wheel excavator based on laser point cloud data according to the embodiments of this application can be implemented in various wireless terminals, such as servers with automatic material handling algorithms for gantry bucket wheel excavators based on laser point cloud data. In one example, the automatic material handling system 100 for a gantry bucket wheel excavator based on laser point cloud data according to the embodiments of this application can be integrated into a wireless terminal as a software module and / or a hardware module. For example, the automatic material handling system 100 for a gantry bucket wheel excavator based on laser point cloud data can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the automatic material handling system 100 for a gantry bucket wheel excavator based on laser point cloud data can also be one of many hardware modules of the wireless terminal.

[0075] Alternatively, in another example, the automatic material handling system 100 for the gantry bucket wheel excavator based on laser point cloud data and the wireless terminal can also be separate devices, and the automatic material handling system 100 for the gantry bucket wheel excavator based on laser point cloud data can be connected to the wireless terminal via wired and / or wireless networks, and transmit interactive information in accordance with an agreed data format.

[0076] Figure 6 This is an application scenario diagram of the automatic material handling method for a gantry bucket wheel excavator based on laser point cloud data according to an embodiment of this application. For example... Figure 6 As shown, in this application scenario, firstly, the laser point cloud data of the coal pile to be retrieved is acquired (e.g., Figure 6 As shown in the figure (D1) and the initial parameters for material handling (e.g., Figure 6As shown in D2), the initial material handling parameters include coal flow control parameters, trolley stepping parameters, and bucket wheel layer thickness parameters. Then, the laser point cloud data of the coal pile to be handled and the initial material handling parameters are input to a server (e.g., a gantry bucket wheel excavator automatic material handling algorithm based on laser point cloud data) deployed on the server. Figure 6 In the illustrated S), the server is able to use the automatic material handling algorithm for the gantry bucket wheel excavator based on laser point cloud data to process the laser point cloud data of the coal pile to be handled and the initial material handling parameters to obtain a decoded value representing the recommended slewing angle value.

[0077] In another example of this application, to achieve intelligent operation of the bucket wheel excavator, appropriate modifications are made to its software and hardware. The main hardware modifications include: adding a precise positioning device; laser 3D scanning and anti-collision detection equipment; upgrading the bucket wheel excavator's frequency converter; adding module I / O channels to the onboard PLC; upgrading fiber optic communication; installing a server and operator station; and adding video monitoring equipment and fiber optic communication to the bucket wheel excavator. Accordingly, several key technologies need to be implemented.

[0078] Regarding positioning technology, to achieve unmanned operation of large stacker-reclaimers, the first challenge is high-precision positioning. Different types of stacker-reclaimers require different positioning points. For example, cantilever bucket wheel excavators need positioning points for the main machine's travel, cantilever pitch, and cantilever rotation; circular stacker-reclaimers require positioning points for the rotation and pitch angles of the reclaiming arm and the rotation angle of the stacking arm. The overall requirement for intelligent unmanned stacker-reclaimers is high precision and reliable technology, with centimeter-level accuracy. Only by accurately measuring the main machine's travel positioning, cantilever pitch positioning, and cantilever rotation angle can the bucket wheel excavator's operating posture be accurately controlled. These three positioning data also form the basis for the stacking pattern calculation by the 3D laser scanner mounted on the cantilever.

[0079] Correspondingly, positioning can be achieved through encoder positioning technology and satellite positioning technology. Encoder positioning is a traditional technology where an encoder is mounted on the rotating shaft of the equipment via a coupling. The encoded signal is transmitted to a PLC controller via a specific cable, where the controller reads, programs, and applies the data to achieve physical positioning. Its theoretical positioning accuracy can be very high, but in practical applications, the accuracy varies significantly depending on the type of machinery. Its advantages include simple principle, low operating and maintenance costs, and wired communication, making it unaffected by external conditions such as obstructions. Its disadvantages include poor stability; the shaking of the machinery and slippage caused by braking can easily affect positioning accuracy. To reduce measurement errors caused by slippage, fixed calibration measurement points are installed at regular intervals during actual use. Calibration sensors typically use proximity switches and radio frequency switches. The bucket wheel excavator uses encoding for its travel and rotation positioning, but the cantilever's pitch positioning uses a high-precision elevation angle meter, achieving a positioning accuracy of 0.01 degrees. This allows for accurate calculation of the excavator's travel position, the cantilever's rotation angle, and the bucket wheel height.

[0080] Regarding 3D imaging technology, it is a key technology for realizing intelligent and unmanned stacker-reclaimer equipment. 3D imaging not only provides "eyes" for equipment operation but also offers intuitive and reliable visual displays for manual monitoring. Current 3D imaging technologies generally utilize laser scanners. Combining the spatial location data of the scanner with the "time-of-flight-distance theory," the distance from points on the stockpile surface to the laser scanner is calculated. Point cloud data is then obtained through calculation. Next, based on the existing point cloud data structure, point cloud neighborhood relationships are established, and methods for noise filtering and simplification of the point cloud data are studied, ultimately achieving complete and structured 3D point data of the stockpile. Finally, graphics processing technology is used to obtain the 3D coordinate values ​​of the stockpile surface, which are then displayed in 3D on the control room interface, forming a complete laser-scanned real-time 3D imaging system for the stockpile. To achieve real-time updates of the coal pile shape at the working face, a laser scanner needs to be installed on each side of the cantilever top (near the bucket wheel) to ensure the stock shape is up-to-date when the cantilever rotates left or right.

[0081] Regarding data communication technology, for stacker-reclaimers that haven't implemented intelligent unmanned operation, most data communication is completed internally. A small amount of data signals that need to be transmitted externally (such as signals to the coal conveying control system) is typically achieved through hardwiring, wireless communication, or fiber optic slip rings. However, if intelligent unmanned operation is to be implemented, a large amount of process processing data and image processing data will need to communicate with the operator station (server) located in the coal conveying control room, with high requirements for timeliness and stability. If the existing communication methods cannot meet these requirements, modifications are necessary. A common modification solution involves upgrading equipment such as cable reels and fiber optic slip rings, using power cables or control cables containing fiber optic cables, or adding fiber optic cables to cable drag chains. Ultimately, this allows all data signals to be transmitted via fiber optics, increasing data throughput, improving communication speed, and ensuring stable and reliable data communication.

[0082] Regarding flow control technology, before unmanned operation of stacker-reclaimer equipment, the coal flow rate was manually observed and controlled by the operator. After unmanned operation, human observation is no longer necessary, but the technology must keep pace. Automatic coal flow control technology is a crucial technology for the intelligent operation of bucket wheel excavators after the stacker-reclaimer equipment achieves intelligent unmanned operation. Its purpose is to prevent belt overload, and sometimes it's necessary for coal blending ratios. Some bucket wheel excavators have electronic belt scales installed on the cantilever belts, but due to the constant changes in the cantilever belt's pitch angle and the lack of calibration methods, these scales have extremely poor accuracy in practical use and cannot be used for measuring coal flow rate control. To achieve a relatively accurate and easy-to-maintain coal flow measurement method, a laser scanner can be installed above the cantilever belt. This method calculates the volume by scanning the surface shape of the coal flow, adding density estimation, and converting it into flow rate. The advantages of this method are low error drift, relative stability, and less need for frequent calibration. Its disadvantages are that the coal density requires empirical values, and severe cantilever belt misalignment can affect the accuracy of coal flow cross-sectional area calculation, requiring the installation of an effective correction device. In actual use, the current of the drive motor of the bucket wheel is introduced to participate in the coal flow control, and suitable control parameters need to be found during the actual debugging process.

[0083] Regarding the stacking and reclaiming algorithm, it is the core technology for realizing equipment intelligence, and it is divided into stacking algorithm and reclaiming algorithm. The reclaiming algorithm, based on the coal reclaiming plan (given the coal pile number), and given initial parameters such as coal flow control parameters, trolley stepping parameters, and bucket wheel layer thickness parameters, calculates the initial position (tail car position, cantilever slewing angle, and pitch angle), left and right slewing angles of the bucket wheel excavator based on the 3D model of the coal pile. These parameters are then transmitted to the bucket wheel excavator's control system. When the operator issues the start command, the bucket wheel excavator automatically moves to the initial position and begins reclaiming operations. The stacking algorithm, given the coal stacking plan, calculates the initial position (tail car position, cantilever slewing angle, and pitch angle), left and right slewing angles, and other automatic operating parameters of the bucket wheel excavator based on the 3D model of the stockpile (empty area coal stacking or replenishment stacking). These calculated parameters are then transmitted to the machine's control system, and the bucket wheel excavator performs automatic stacking operations as required.

[0084] Regarding protection technology, it is essential for achieving intelligent and unmanned operation of stacker-reclaimer equipment. Since the equipment no longer requires constant human monitoring after achieving intelligent and unmanned operation, obstacles and hazards that were traditionally identified manually must now be automatically identified by the equipment. Traditional protection functions primarily focus on protecting the equipment itself, such as limit protection, interlocking protection, over-temperature protection, and overcurrent protection. After achieving intelligent and unmanned operation, in addition to retaining these protections, more protection is needed for the equipment itself, other equipment, and personnel. This includes protection against hopper jamming, overload protection, protection against personnel and vehicle collisions, and communication interruption protection. The modification work requires adding distance detection devices, hopper jamming detection equipment and judgment logic programs, and coal flow detection equipment and algorithms. When two bucket wheel excavators are operating unmanned simultaneously, and there is a possibility of wheel-bucket (or cantilever) collision in adjacent coal yard processes, an alarm and automatic shutdown should be triggered based on the spatial position of the two bucket wheel excavators. In addition, if a bucket wheel excavator is operating automatically without human intervention and a pusher (coal yard shaping equipment) is running nearby, the automatic calculation and judgment of the safe distance, timely warning or even stopping operation is required. In this case, the pusher needs to be repositioned and its real-time position is sent to the unmanned bucket wheel excavator's calculation server.

[0085] Regarding video surveillance technology, before the implementation of intelligent and unmanned systems, equipment video surveillance mainly focused on monitoring the work scene from a wide angle and over a large area. Monitoring of detailed work points was generally done by the operator through "seeing and hearing". After the implementation of intelligent and unmanned systems, although all operations can be completed safely without human intervention, video surveillance is still indispensable as an auxiliary means of remote operation in emergency situations. Moreover, the monitoring points are more detailed than before the implementation of intelligent systems. For example, after the implementation of intelligent systems for bucket wheel excavators, about eight digital high-definition cameras have been added, including those for the bucket wheel, traveling trolley, boom, tail car, and slewing platform.

[0086] The intelligent system of the bucket wheel excavator integrates multiple technologies such as laser scanning, high-precision positioning, intelligent analysis, video processing, and intelligent control. After the operator issues the operation requirements of the bucket wheel stacker-reclaimer in the central control room, it automatically performs functions such as stacking, layer opening, layer changing, empty-field stacking, and replenishing stacks, realizing fully unmanned automatic stacking / reclaiming operations. This achieves the following objectives: increasing the degree of automation of the equipment, reducing labor costs, improving the accuracy and efficiency of stacking and reclaiming operations, thus achieving the goal of reducing manpower and increasing efficiency; avoiding electrical or mechanical damage caused by human error, improving the safety and reliability of equipment operation, and saving maintenance costs; and ensuring collision prevention throughout the material yard, in addition to the anti-collision detection switches on the machine, also through the central control room. Collision avoidance calculations determine the spatial coordinates of each stacker-reclaimer, enabling collision avoidance alerts and warnings, thus improving the system's safety level. With the large-scale completion of coal yard enclosure renovations, the coal yard environment has become more severe. The intelligent unmanned system for stacker-reclaimers can effectively improve the working environment of operators, reduce their workload, and decrease the probability of occupational diseases. The intelligent control of bucket wheel excavators and stacker-reclaimers is a crucial link in realizing the "one-click start" intelligent operation of coal-fired power plants. Together with digital coal yards and coal blending systems, it creates the technical conditions for the future automatic closed-loop operation of coal-fired power plants.

[0087] Furthermore, those skilled in the art will understand that aspects of this application can be described and illustrated through several patentable types or situations, including any new and useful combination of processes, machines, products, or substances, or any new and useful improvements thereof. Accordingly, aspects of this application can be implemented entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. All of the above hardware or software may be referred to as a “data block,” “module,” “engine,” “unit,” “component,” or “system.” Furthermore, aspects of this application may manifest as a computer product located on one or more computer-readable media, the product including computer-readable program code.

[0088] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in a common dictionary shall be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and not as having an idealized or highly formalized meaning, unless expressly defined herein.

[0089] The foregoing description is illustrative of the invention and should not be construed as limiting it. Although several exemplary embodiments of the invention have been described, those skilled in the art will readily understand that many modifications can be made to the exemplary embodiments without departing from the novel teachings and advantages of the invention. Therefore, all such modifications are intended to be included within the scope of the invention as defined in the claims. It should be understood that the foregoing description is illustrative of the invention and should not be construed as limiting it to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The invention is defined by the claims and their equivalents.

Claims

1. An automatic material handling method for a gantry bucket wheel excavator based on laser point cloud data, characterized in that, It includes the following steps: S110. The working area of ​​the gantry bucket wheel excavator is scanned by a laser scanner to obtain laser point cloud data of the coal pile to be removed, wherein the laser point cloud data is represented as (x, y, z, d), where x, y, z represent the spatial coordinates of each pixel and d represents the depth value of each pixel. S120. Based on the laser point cloud data of the coal pile object to be removed, construct a three-dimensional model of the coal pile object to be removed. S130. Obtain initial material handling parameters, wherein the initial material handling parameters include coal flow control parameters, trolley stepping parameters, and wheel bucket layer replacement thickness parameters. S140. Perform feature fusion correlation analysis on the initial material extraction parameters and the three-dimensional model of the coal pile to be extracted to obtain the material extraction parameter-coal pile three-dimensional fusion features. S150. Based on the material taking parameters and the three-dimensional fusion characteristics of the coal pile, determine the recommended rotation angle value and perform automatic control. In step S140, feature fusion correlation analysis is performed on the initial material extraction parameters and the three-dimensional coal pile model of the coal pile object to be extracted to obtain the material extraction parameter-coal pile three-dimensional fusion features. This includes: passing the initial material extraction parameters and the three-dimensional coal pile model of the coal pile object to be extracted through a MetaNet fusion module containing an image encoder and a sequence encoder to obtain a three-dimensional coal pile feature map containing material extraction parameter features as the material extraction parameter-coal pile three-dimensional fusion features. The initial material extraction parameters and the three-dimensional model of the coal pile to be extracted are processed by a MetaNet fusion module containing an image encoder and a sequence encoder to obtain a three-dimensional feature map of the coal pile containing material extraction parameter features, which is used as the material extraction parameter-coal pile three-dimensional fusion feature. The process includes the following steps: S141. The three-dimensional model of the coal pile to be taken is passed through the image encoder of the MetaNet fusion module to obtain the three-dimensional feature map of the coal pile to be taken. S142. Encode the initial material picking parameters to obtain an initial material picking parameter encoding vector; S143. The material initial parameter encoding vector is passed through the first convolutional layer of the sequence encoder in the MetaNet fusion module and then linearly corrected by the ReLU function to obtain the linearly corrected material initial parameter associated feature vector. S144. The linearly corrected material initial parameter associated feature vector is passed through the second convolutional layer of the sequence encoder in the MetaNet fusion module and then processed by the Sigmoid function to obtain the activated material initial parameter associated feature vector. S145. The three-dimensional feature map of the coal pile to be extracted is weighted and fused along the channel dimension using the feature vector associated with the initial parameters of material extraction after activation to obtain the three-dimensional feature map of the coal pile containing the material extraction parameter features. Based on the material handling parameters and the three-dimensional fusion characteristics of the coal pile, the recommended rotation angle value is determined and automatically controlled, including the following steps: S151. The three-dimensional feature map of the coal pile containing the material handling parameters is decoded using a decoder to obtain a decoded value, the decoded value being used to represent the recommended rotation angle value; and S152. Based on the decoded value, control the rotation angle of the gantry bucket wheel machine to perform material handling.

2. The automatic material handling method for a gantry bucket wheel excavator based on laser point cloud data according to claim 1, characterized in that... It also includes a training step for training the MetaNet fusion module containing the image encoder and the sequence encoder and the decoder.

3. The automatic material handling method for a gantry bucket wheel excavator based on laser point cloud data according to claim 2, characterized in that, The training steps include: Acquire training data, which includes training laser point cloud data of the coal pile to be removed, initial training parameters for material removal, and the actual value of the recommended rotation angle. Based on the training laser point cloud data of the coal pile to be retrieved, a three-dimensional model of the training coal pile is constructed. The training initial parameters for material extraction and the training 3D model of the coal pile object to be extracted are passed through the MetaNet fusion module containing an image encoder and a sequence encoder to obtain a 3D feature map of the coal pile containing the material extraction parameter features. The three-dimensional feature map of the coal pile, which includes the features of the material extraction parameters, is passed through the decoder to obtain the decoding loss function value; and The MetaNet fusion module, which includes an image encoder and a sequence encoder, and the decoder are trained based on the decoding loss function value and through directional propagation of gradient descent.

4. The automatic material handling method for a gantry bucket wheel excavator based on laser point cloud data according to claim 3, characterized in that, The three-dimensional feature map of the coal pile, which includes the material extraction parameter features, is passed through the decoder to obtain the decoding loss function value, including: The three-dimensional feature map of the coal pile containing the material extraction parameter features is expanded along each feature matrix of the channel dimension to obtain the expanded three-dimensional feature vector of the coal pile containing the material extraction parameter features. The feature distribution of the trained 3D feature unfolded vector of the coal pile, which includes material extraction parameter features, is optimized to obtain an optimized trained 3D feature unfolded vector of the coal pile, which includes material extraction parameter features; and The optimized training includes a three-dimensional feature map of the coal pile containing material extraction parameters, which is then passed through the decoder to obtain the decoding loss function value.

5. The automatic material handling method for a gantry bucket wheel excavator based on laser point cloud data according to claim 4, characterized in that, The optimized training of the three-dimensional feature map of the coal pile, which includes the features of the material extraction parameters, is passed through the decoder to obtain the decoding loss function value, including: The decoder is used to perform decoding regression on the 3D feature map of the coal pile containing material extraction parameter features during the optimized training, to obtain the decoding loss function value using the following decoding formula; wherein, the decoding formula is: ； in, This indicates that the optimized training includes a three-dimensional feature map of the coal pile containing material extraction parameter features. This represents the value of the decoding loss function. Represents the weight matrix. This represents matrix multiplication.

6. An automatic material handling system for a portal bucket wheel excavator based on laser point cloud data, applied to the automatic material handling method for a portal bucket wheel excavator based on laser point cloud data as described in claim 1, characterized in that... It includes: The laser point cloud data acquisition module is used to scan the working area of ​​the gantry bucket wheel excavator with a laser scanner to obtain the laser point cloud data of the coal pile to be retrieved. The laser point cloud data is represented as (x, y, z, d), where x, y, z represent the spatial coordinates of each pixel and d represents the depth value of each pixel. The three-dimensional model construction module is used to construct a three-dimensional model of the coal pile object based on the laser point cloud data of the coal pile object to be taken. The initial material handling parameter acquisition module is used to acquire the initial material handling parameters, which include coal flow control parameters, trolley stepping parameters, and wheel bucket layer replacement thickness parameters. The feature fusion and correlation analysis module is used to perform feature fusion and correlation analysis on the initial material extraction parameters and the three-dimensional model of the coal pile to be extracted, so as to obtain the material extraction parameter-coal pile three-dimensional fusion features; and And a control module, used to determine the recommended rotation angle value and perform automatic control based on the material taking parameters-coal pile three-dimensional fusion characteristics.

7. The automatic material handling system for a gantry bucket wheel excavator based on laser point cloud data according to claim 6, characterized in that: The feature fusion and correlation analysis module is used to use the initial material extraction parameters and the three-dimensional coal pile model of the coal pile object to be extracted through the MetaNet fusion module containing image encoder and sequence encoder to obtain a three-dimensional coal pile feature map containing material extraction parameter features as the material extraction parameter-coal pile three-dimensional fusion feature.

Images