Intelligent management and control system and method for bucket-wheel stacker-reclaimer based on digital twinning

Abstract

The invention relates to the technical field of bucket-wheel stacker reclaimers, and particularly discloses an intelligent management and control system and method for a bucket-wheel stacker reclaimer based on digital twinning, and the system employs a laser radar to collect the three-dimensional point cloud data of a material pile, builds a three-dimensional surface model of the material pile, achieves the real-time synchronization of the state of the material pile through a data twinning technology, and achieves the intelligent management and control of the bucket-wheel stacker reclaimer. The material pile three-dimensional surface model is abstracted into a plannable three-dimensional grid map, and then in combination with three-dimensional center point coordinates of a current mechanical arm of a bucket wheel, semantic query interaction analysis based on grid attribute information is carried out on all grid areas in the plannable three-dimensional grid map, so that the proper operation point position is intelligently determined; therefore, path planning is executed based on the target operation point, and intelligent control over the bucket-wheel stacker-reclaimer is achieved. In this way, the operation strategy can be adjusted in real time according to the material pile form change, resource allocation and the operation process are further optimized, and therefore the operation efficiency and safety of the bucket-wheel stacker-reclaimer are remarkably improved.

Description

Technical Field

[0001] The present application relates to the technical field of bucket wheel stacker reclaimers, and specifically to an intelligent management and control system and method for bucket wheel stacker reclaimers based on digital twins. Background Art

[0002] Bucket Wheel Stacker-Reclaimer (DTR), as the core equipment of bulk material storage and transportation system, is widely used in industrial scenarios such as ports, thermal power plants, steel mills, etc., to undertake the tasks of bulk material storage, retrieving and transportation. The traditional operation mode of bucket wheel stacker-reclaimer mainly relies on manual experience operation, and has problems such as low operation efficiency, poor accuracy, and insufficient adaptability to dynamic environment. It is also easy to cause safety hazards due to changes in the shape of the material pile or deviations in the posture of the equipment. With the advancement of Industry 4.0 and intelligent manufacturing, the demand for automation and intelligent upgrades of DTR is becoming increasingly urgent.

[0003] However, the existing intelligent control technology of bucket wheel stacker reclaimers focuses on the equipment status monitoring. As a dynamic interactive object in the operation process, the shape of the pile continues to change with the operation process, which leads to complexity and uncertainty in the selection of bucket wheel stacker reclaimer operation points and path planning. In recent years, digital twin technology has achieved real-time synchronization between physical equipment and twin models through virtual-real interactive interfaces, providing a data basis for intelligent decision-making in dynamic environments. However, how to transform complex operation scenes in physical space into plannable digital space and realize the selection of reclaiming operation points and path planning under multiple constraints is a key issue that needs to be solved in the current field of intelligent control of bucket wheel stacker reclaimers.

[0004] Therefore, an optimized digital twin-based intelligent management and control system and method for bucket wheel stacker reclaimer is expected. Summary of the invention

[0005] The embodiments of the present application aim to solve at least one of the technical problems existing in the prior art, and provide a bucket wheel stacker and reclaimer intelligent management and control system and method based on digital twin, which can adjust the operation strategy in real time according to the changes in the pile shape, further optimize the resource allocation and operation process, thereby significantly improving the operation efficiency and safety of the bucket wheel stacker and reclaimer.

[0006] On the one hand, an embodiment of the present application provides an intelligent management and control method for a bucket wheel stacker reclaimer based on digital twin, comprising:

[0007] Acquiring three-dimensional point cloud data of the material pile collected by the laser radar, and processing the three-dimensional point cloud data of the material pile to obtain a three-dimensional surface model of the material pile;

[0008] Using a data twin model to display a three-dimensional surface model of the stockpile;

[0009] Abstract the three-dimensional surface model of the stockpile into a three-dimensional grid map that can be planned;

[0010] Obtain the three-dimensional center point coordinates of the current robotic arm of the bucket wheel, and based on the three-dimensional center point coordinates of the current robotic arm of the bucket wheel, perform a search for job point semantic query responses based on grid attribute information for each grid in the three-dimensional grid map that can be planned to obtain a target point identification result, where the target point identification result is the corresponding grid of the stockpile job point in the three-dimensional grid map that can be planned;

[0011] Perform path planning based on the target point identification result.

[0012] Optionally, each grid in the three-dimensional grid map that can be planned includes position data and type data, and the type data includes free areas, non-passable areas, or dangerous areas.

[0013] Optionally, performing a search for job point semantic query responses based on grid attribute information for each grid in the three-dimensional grid map that can be planned based on the three-dimensional center point coordinates of the current robotic arm of the bucket wheel to obtain a target point identification result includes:

[0014] Perform semantic embedding encoding on each grid information in the three-dimensional grid map that can be planned to obtain a set of grid area information semantic embedding encoding vectors;

[0015] Perform low-dimensional embedding encoding on the three-dimensional center point coordinates of the current robotic arm of the bucket wheel to obtain a starting position low-dimensional embedding encoding vector;

[0016] Perform dynamic response encoding aggregation based on an adaptive attention mechanism on the starting position low-dimensional embedding encoding vector and the set of grid area information semantic embedding encoding vectors to obtain a target job point search response encoding vector;

[0017] Determine the target point identification result based on the target job point search response encoding vector.

[0018] Optionally, performing dynamic response encoding aggregation based on an adaptive attention mechanism on the starting position low-dimensional embedding encoding vector and the set of grid area information semantic embedding encoding vectors to obtain a target job point search response encoding vector includes:

[0019] Perform depth implicit feature extraction based on fully connected encoding on the starting position low-dimensional embedding encoding vector and each grid area information semantic embedding encoding vector in the set of grid area information semantic embedding encoding vectors to obtain a starting position feature depth implicit encoding vector and a set of grid area information semantic feature depth implicit encoding vectors;

[0020] Input each grid region information semantic feature depth implicit encoding vector in the set of the starting point position feature depth implicit encoding vector and the grid region information semantic feature depth implicit encoding vectors into a semantic response decision anchoring component to obtain a set of starting point position-grid region information semantic response anchoring encoding matrices;

[0021] Based on the feature contributions of each starting point position-grid region information semantic response anchoring encoding matrix in the set of the starting point position-grid region information semantic response anchoring encoding matrices, perform dynamic aggregation encoding on the set of the starting point position-grid region information semantic response anchoring encoding matrices to obtain the target operation point search response encoding vector.

[0022] Optionally, based on the feature contributions of each starting point position-grid region information semantic response anchoring encoding matrix in the set of the starting point position-grid region information semantic response anchoring encoding matrices, perform dynamic aggregation encoding on the set of the starting point position-grid region information semantic response anchoring encoding matrices to obtain the target operation point search response encoding vector, including:

[0023] Calculate the decision anchor adaptive splicing factors of each starting point position-grid region information semantic response anchoring encoding matrix in the set of the starting point position-grid region information semantic response anchoring encoding matrices to obtain a set of starting point position-grid region information decision anchor adaptive splicing factors;

[0024] Perform weight processing based on the Softmax function on the set of the starting point position-grid region information decision anchor adaptive splicing factors to obtain a set of starting point position-grid region information decision anchor adaptive splicing weight factors;

[0025] Fuse the set of the starting point position-grid region information semantic response anchoring encoding matrices based on the set of the starting point position-grid region information decision anchor adaptive splicing weight factors to obtain the target operation point search response encoding vector.

[0026] Optionally, calculating the decision anchor adaptive splicing factors of each starting point position-grid region information semantic response anchoring encoding matrix in the set of the starting point position-grid region information semantic response anchoring encoding matrices to obtain a set of starting point position-grid region information decision anchor adaptive splicing factors, including:

[0027] Based on the statistical eigenvalues of each starting position-grid area information semantic response anchoring coding matrix in the set of starting position-grid area information semantic response anchoring coding matrices, calculate the decision anchor adaptive splicing factor of each starting position-grid area information semantic response anchoring coding matrix to obtain the set of starting position-grid area information decision anchor adaptive splicing factors. The statistical eigenvalues include the maximum eigenvalue, eigenvalue variance, eigenvalue mean, and number of eigenvalues.

[0028] Optionally, based on the statistical eigenvalues of each starting position-grid area information semantic response anchoring coding matrix in the set of starting position-grid area information semantic response anchoring coding matrices, calculating the decision anchor adaptive splicing factor of each starting position-grid area information semantic response anchoring coding matrix to obtain the set of starting position-grid area information decision anchor adaptive splicing factors includes:

[0029] Taking the sum of the eigenvalue variance and the drift coefficient of the starting position-grid area information semantic response anchoring coding matrix as the numerator, and calculating the square of the difference between the maximum eigenvalue and the eigenvalue mean of the starting position-grid area information semantic response anchoring coding matrix multiplied by the number of its eigenvalues, and then adding the drift coefficient and twice the eigenvalue variance as the denominator to obtain the starting position-grid area information decision anchor adaptive splicing factor, where the drift coefficient is used to smooth the fluctuation of the eigenvalue distribution of the starting position-grid area information semantic response anchoring coding matrix.

[0030] Optionally, based on the target job point search response coding vector, determining the target point identification result includes:

[0031] Inputting the target job point search response coding vector into the job point identification module based on the classifier to obtain the target point identification result.

[0032] Optionally, inputting the target job point search response coding vector into the job point identification module based on the classifier to obtain the target point identification result includes:

[0033] Using the fully connected layer of the job point identification module based on the classifier to perform fully connected coding on the target job point search response coding vector to obtain the fully connected coded target job point search response coding vector;

[0034] Inputting the fully connected coded target job point search response coding vector into the Softmax classification function of the job point identification module based on the classifier to obtain the target point identification result, and the target point identification result is the corresponding grid of the stockpile job point in the plannable three-dimensional grid map.

[0035] On the other hand, the present application provides an intelligent control system for a bucket wheel stacker-reclaimer based on digital twin, including:

[0036] A three-dimensional point cloud data acquisition and processing module, configured to obtain three-dimensional point cloud data of a stockpile collected by a lidar, and process the three-dimensional point cloud data of the stockpile to obtain a three-dimensional surface model of the stockpile;

[0037] A three-dimensional surface model display module of the stockpile, configured to display the three-dimensional surface model of the stockpile by using a digital twin model;

[0038] A three-dimensional surface model abstraction module of the stockpile, configured to abstract the three-dimensional surface model of the stockpile into a three-dimensional grid map that can be planned;

[0039] A target point identification result acquisition module, configured to obtain the three-dimensional center point coordinates of the current robotic arm of the bucket wheel, and based on the three-dimensional center point coordinates of the current robotic arm of the bucket wheel, perform a semantic query response search for operation points based on grid attribute information on each grid in the three-dimensional grid map that can be planned to obtain a target point identification result, where the target point identification result is the corresponding grid of the stockpile operation point in the three-dimensional grid map that can be planned;

[0040] A path planning execution module, configured to perform path planning based on the target point identification result.

[0041] An intelligent control system and method for a bucket wheel stacker-reclaimer based on digital twin provided by the present application uses a lidar to collect three-dimensional point cloud data of a stockpile, constructs a three-dimensional surface model of the stockpile, and at the same time synchronizes the stockpile state in real time through digital twin technology, and abstracts the three-dimensional surface model of the stockpile into a three-dimensional grid map that can be planned. Furthermore, in combination with the three-dimensional center point coordinates of the current robotic arm of the bucket wheel, through semantic query interaction analysis of each grid area in the three-dimensional grid map that can be planned based on grid attribute information, the position of a suitable operation point is intelligently determined, and thus path planning is performed based on the target operation point, realizing intelligent control of the bucket wheel stacker-reclaimer. In this way, the operation strategy can be adjusted in real time according to the change of the stockpile shape, further optimizing the resource allocation and operation process, thereby significantly improving the operation efficiency and safety of the bucket wheel stacker-reclaimer. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic flowchart of an intelligent control method for a bucket wheel stacker-reclaimer based on digital twin and based on deep learning according to an embodiment of the present application;

[0043] Figure 2 It is a schematic diagram of data flow of an intelligent control method for a bucket wheel stacker-reclaimer based on digital twin and based on deep learning according to an embodiment of the present application;

[0044] Figure 3Schematic flowchart of step S4 in the intelligent control method of a bucket wheel stacker-reclaimer based on digital twin and deep learning according to an embodiment of the present application;

[0045] Figure 4 Schematic flowchart of step S43 in the intelligent control method of a bucket wheel stacker-reclaimer based on digital twin and deep learning according to an embodiment of the present application;

[0046] Figure 5 Schematic flowchart of step S433 in the intelligent control method of a bucket wheel stacker-reclaimer based on digital twin and deep learning according to an embodiment of the present application;

[0047] Figure 6 Schematic block diagram of the intelligent control system of a bucket wheel stacker-reclaimer based on digital twin according to an embodiment of the present application. Detailed implementation manners

[0048] Next, the technical solutions in the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts also belong to the scope of protection of the present application.

[0049] In view of the above technical problems, the technical concept of the present application is as follows: using lidar to collect three-dimensional point cloud data of the stockpile, constructing a three-dimensional surface model of the stockpile, and at the same time, through data twin technology, synchronizing the stockpile state in real time, and abstracting the three-dimensional surface model of the stockpile into a plannable three-dimensional grid map. Furthermore, in combination with the three-dimensional center point coordinates of the current robotic arm of the bucket wheel, through semantic query interaction analysis based on grid attribute information for each grid area in the plannable three-dimensional grid map, to intelligently determine the appropriate operation point position, and thus perform path planning based on the target operation point to achieve intelligent control of the bucket wheel stacker-reclaimer. In this way, the operation strategy can be adjusted in real time according to the change of the stockpile shape, further optimizing the resource allocation and operation process, thereby significantly improving the operation efficiency and safety of the bucket wheel stacker-reclaimer.

[0050] Figure 1 Schematic flowchart of the intelligent control method of a bucket wheel stacker-reclaimer based on digital twin and deep learning according to an embodiment of the present application. Figure 2 Data flow diagram of the intelligent control method of a bucket wheel stacker-reclaimer based on digital twin and deep learning according to an embodiment of the present application. As Figure 1 and Figure 2As shown, the intelligent control method for a bucket wheel stacker-reclaimer based on digital twin includes: S1, obtaining the three-dimensional point cloud data of the stockpile collected by a lidar, and processing the three-dimensional point cloud data of the stockpile to obtain a three-dimensional surface model of the stockpile. S2, using the digital twin model to display the three-dimensional surface model of the stockpile. S3, abstracting the three-dimensional surface model of the stockpile into a plannable three-dimensional grid map. S4, obtaining the three-dimensional center point coordinates of the current robotic arm of the bucket wheel, and based on the three-dimensional center point coordinates of the current robotic arm of the bucket wheel, performing a semantic query response search for operation points on each grid in the plannable three-dimensional grid map based on grid attribute information to obtain a target point identification result, where the target point identification result is the corresponding grid of the stockpile operation point in the plannable three-dimensional grid map. S5, performing path planning based on the target point identification result.

[0051] Exemplarily, in step S1, the three-dimensional point cloud data of the stockpile collected by the lidar is obtained, and the three-dimensional point cloud data of the stockpile is processed to obtain a three-dimensional surface model of the stockpile. It should be understood that in industrial production, the operation efficiency and safety of the bucket wheel stacker-reclaimer highly depend on the accurate perception of the stockpile. Since the shape of the stockpile is irregular and may change with the operation process, therefore, in order to achieve the intelligent control of the bucket wheel stacker-reclaimer, it is first necessary to obtain comprehensive and accurate information on the shape and position of the stockpile. The lidar emits laser beams and receives the reflected light, measures the time difference between the emission and reception of the laser, and calculates the distance between the sensor and each point on the object surface according to the speed of light, so as to obtain a large number of discrete three-dimensional coordinate points, constituting the three-dimensional point cloud data of the stockpile. The three-dimensional point cloud data can completely reflect the spatial form of the stockpile and provide an accurate spatial reference for the intelligent operation planning of the bucket wheel stacker-reclaimer.

[0052] Next, the three-dimensional point cloud data of the stockpile is processed to obtain a three-dimensional surface model of the stockpile. Since the original three-dimensional point cloud data presents as a discrete point set, the data is relatively scattered and lacks an intuitive geometric structure, which is not conducive to directly observing the overall shape and spatial distribution of the stockpile. Therefore, in this application, the three-dimensional point cloud data of the stockpile is processed to construct a three-dimensional surface model of the stockpile. In an embodiment of this application, the moving least squares method (MLS) is used to smooth the three-dimensional point cloud data to generate a continuous stockpile surface. The moving least squares method constructs a local polynomial fitting surface in the neighborhood of each point, smooths and fits the discrete point cloud data, and gradually constructs a continuous triangular grid model to more intuitively display the shape and spatial distribution of the stockpile and provide more accurate geometric information for subsequent intelligent operation planning.

[0053] Exemplarily, in step S2, the three-dimensional surface model of the stockpile is displayed using the digital twin model. It should be understood that considering that during the actual production process, operators need to monitor the status of the stockpile in real time to facilitate timely adjustment of the operation status of the bucket wheel stacker-reclaimer. Therefore, the present application further uses the digital twin model to display the three-dimensional surface model of the stockpile. The digital twin technology can create a virtual model in the virtual space that exactly corresponds to the physical stockpile, and through the real-time data transmission and synchronization mechanism, keep the status of the virtual model consistent with the physical stockpile. In this way, operators can comprehensively and intuitively grasp the real-time situation of the stockpile in the virtual environment without directly observing the actual stockpile, providing a more convenient and efficient monitoring means for the intelligent operation of the bucket wheel stacker-reclaimer.

[0054] Exemplarily, in step S3, the three-dimensional surface model of the stockpile is abstracted into a three-dimensional grid map that can be planned. It should be understood that although the three-dimensional surface model of the stockpile intuitively shows the shape of the stockpile, when performing path planning and operation point analysis, due to the complexity of its geometric shape, the calculation process is very complex and the calculation efficiency is low. To simplify the calculation and improve the efficiency and accuracy of path planning and operation point analysis, the present application further abstracts the three-dimensional surface model of the stockpile into a three-dimensional grid map that can be planned. In the three-dimensional grid map that can be planned, the three-dimensional space of the stockpile is divided into a series of cube grids of equal size, forming a three-dimensional grid structure. In one embodiment, each grid in the three-dimensional grid map that can be planned contains position data and type data, and the type data includes free areas, non-passable areas or dangerous areas. The position data is used to identify the specific position of the grid in the three-dimensional space for positioning analysis, while the type data is used to describe the attributes of the grid, such as free areas (i.e., areas where operations can be carried out), non-passable areas (such as obstacles, equipment foundations, etc.) or dangerous areas (such as areas with potential safety hazards). For example, by traversing the geometric data of the three-dimensional surface model of the stockpile, the relative position relationship between each grid and the stockpile is judged. If the grid is completely outside the stockpile and there are no other obstacles, it is marked as a free area. If the grid intersects with the stockpile or there are other obstacles that prevent the bucket wheel from passing through, it is marked as a non-passable area. If the grid has potential hazards, such as special environments like high temperature and high pressure, which may pose hazards to the bucket wheel or operators, it is marked as a dangerous area. In this way, the complex geometric shape of the stockpile is simplified into a grid structure that is easy to analyze and calculate, which can more conveniently analyze the feasibility of the path, quickly screen out areas suitable for bucket wheel operation, and improve the efficiency and accuracy of path planning.

[0055] Exemplarily, in step S4, the three-dimensional center point coordinates of the current mechanical arm of the bucket wheel are obtained, and based on the three-dimensional center point coordinates of the current mechanical arm of the bucket wheel, a search for a semantic query response of the working point based on the grid attribute information is performed on each grid in the plannable three-dimensional grid map to obtain a target point identification result, where the target point identification result is the corresponding grid of the stockpile working point in the plannable three-dimensional grid map.

[0056] In one embodiment, as Figure 3 shown, performing a search for a semantic query response of the working point based on the grid attribute information on each grid in the plannable three-dimensional grid map based on the three-dimensional center point coordinates of the current mechanical arm of the bucket wheel to obtain a target point identification result includes: S41, performing semantic embedding encoding on each grid information in the plannable three-dimensional grid map to obtain a set of grid region information semantic embedding encoding vectors. S42, performing low-dimensional embedding encoding on the three-dimensional center point coordinates of the current mechanical arm of the bucket wheel to obtain a starting position low-dimensional embedding encoding vector. S43, performing dynamic response encoding aggregation based on an adaptive attention mechanism on the starting position low-dimensional embedding encoding vector and the set of grid region information semantic embedding encoding vectors to obtain a target working point search response encoding vector. S44, determining the target point identification result based on the target working point search response encoding vector.

[0057] Exemplarily, in step S41, performing semantic embedding encoding on each grid information in the plannable three-dimensional grid map to obtain a set of grid region information semantic embedding encoding vectors. It should be understood that for each grid in the three-dimensional grid map, its position data and type data define the basic attributes of the grid, but these attribute information exist in a discrete and unstructured form, which is not conducive to efficient data interaction and analysis by a computer. In this regard, in this application, by performing semantic embedding encoding on each grid information in the plannable three-dimensional grid map, the position data and type data of the grid are converted into a structured continuous vector representation, thereby obtaining a set of grid region information semantic embedding encoding vectors. Specifically, the semantic embedding encoding technology is a method of converting discrete and unstructured data into a continuous and structured vector representation. By using the position data and type data of each grid as input and utilizing the pre-trained knowledge of a word embedding model (such as Word2Vec, GloVe, or BERT, etc.), the feature representation of each grid is learned, thereby generating the corresponding grid region information semantic embedding encoding vector. Through semantic embedding encoding, not only the basic attribute information of each grid is retained, but also the relative relationship and semantic connection between grids can be revealed through the spatial distribution of vectors in the high-dimensional semantic space, thereby providing a richer and more accurate data basis for subsequent working point determination and path planning.

[0058] Exemplarily, in step S42, a low-dimensional embedding encoding is performed on the three-dimensional center point coordinates of the current boom of the bucket wheel to obtain a low-dimensional embedding encoding vector of the starting position. It should be understood that the three-dimensional center point coordinates of the bucket wheel boom represent the specific position of the bucket wheel in the stockpile space and are the starting conditions for path planning and working point analysis. However, due to the high dimensionality and sparsity of the data of the three-dimensional center point coordinates, it is not conducive to joint analysis with the semantic embedding encoding vector of the grid area information. Therefore, the present application further performs a low-dimensional embedding encoding on the three-dimensional center point coordinates of the current boom of the bucket wheel to unify the data dimension, convert the three-dimensional center point coordinates into a structured continuous vector representation, and thus obtain a low-dimensional embedding encoding vector of the starting position, so as to facilitate subsequent effective joint analysis and path planning of the position of the current boom of the bucket wheel and the grid area information in the same feature space.

[0059] Exemplarily, in step S43, a dynamic response encoding aggregation based on an adaptive attention mechanism is performed on the set of the low-dimensional embedding encoding vector of the starting position and the semantic embedding encoding vector of the grid area information to obtain a target working point search response encoding vector. It should be understood that for the search for the target working point, factors such as the spatial relationship, accessibility, and working efficiency between the current position of the bucket wheel and each grid area need to be comprehensively considered. Therefore, in order to search for the target working point in the current working scenario, the present application further performs a dynamic response encoding aggregation based on an adaptive attention mechanism on the set of the low-dimensional embedding encoding vector of the starting position and the semantic embedding encoding vector of the grid area information to comprehensively consider the initial state of the bucket wheel boom and the overall distribution of the stockpile space to find the optimal solution. For example, a grid that is relatively close to the bucket wheel and is an idle area may be more suitable as the target working point.

[0060] In one embodiment, as Figure 4As shown, performing dynamic response coding aggregation based on an adaptive attention mechanism on the set of the low-dimensional embedded coding vectors of the starting point position and the semantic embedded coding vectors of the grid region information to obtain a target job point search response coding vector includes: S431. Respectively performing deep implicit feature extraction based on fully connected coding on the low-dimensional embedded coding vector of the starting point position and each semantic embedded coding vector of the grid region information in the set to obtain a set of starting point position feature deep implicit coding vectors and semantic feature deep implicit coding vectors of the grid region information. S432. Respectively inputting each semantic feature deep implicit coding vector in the set of the starting point position feature deep implicit coding vectors and the semantic feature deep implicit coding vectors of the grid region information into a semantic response decision anchoring component to obtain a set of starting point position-grid region information semantic response anchoring coding matrices. S433. Based on the feature contributions of each starting point position-grid region information semantic response anchoring coding matrix in the set of the starting point position-grid region information semantic response anchoring coding matrices, performing dynamic aggregation coding on the set of the starting point position-grid region information semantic response anchoring coding matrices to obtain the target job point search response coding vector.

[0061] Exemplarily, in step S431, in order to enhance the semantic expression capabilities of the starting point position information of the bucket wheel manipulator and each grid region information, the present application first performs deep implicit feature extraction based on fully connected coding on the low-dimensional embedded coding vector of the starting point position and each semantic embedded coding vector of the grid region information, so as to learn the global non-linear interaction relationship inside the original features through the non-linear mapping of the fully connected neural network, generate a deeper semantic feature representation, and obtain a set of starting point position feature deep implicit coding vectors and semantic feature deep implicit coding vectors of the grid region information. Specifically, the process of calculating the set of starting point position feature deep implicit coding vectors and semantic feature deep implicit coding vectors of the grid region information can be expressed by the formula as follows:

[0062] V 2 ={v 21 ,v 22 ,...,v 2i ,...,v 2n}

[0063]

[0064] where V 2 represents the set of semantic embedded coding vectors of the grid region information, v 21 , v 22 , v 2i and v 2nrespectively represent the 1st, 2nd, i-th, and n-th grid region information semantic embedding coding vectors in the set of grid region information semantic embedding coding vectors, where n is the number of vectors in the set of grid region information semantic embedding coding vectors, and W 1 represents the starting position information semantic feature weight matrix, and W 2 represents the grid region information semantic feature weight matrix, and b 1 represents the starting position information semantic feature bias term, and b 2 represents the grid region information semantic feature bias term, and V 1 represents the starting position low-dimensional embedding coding vector, sigmoid(·) represents the sigmoid activation function, and V d1 represents the starting position feature depth implicit coding vector, and v d2i represents v 2i corresponding grid region information semantic feature depth implicit coding vector.

[0065] Exemplarily, in step S432, further perform semantic interaction coding between the starting position feature depth implicit coding vector and each grid region information semantic feature depth implicit coding vector through the semantic response decision anchoring component, and calculate the outer product between vectors to capture the potential correlation between the starting position information of the bucket wheel manipulator and each grid region information, thereby generating a set of starting position-grid region information semantic response anchoring coding matrices. Specifically, the calculation process of the set of starting position-grid region information semantic response anchoring coding matrices can be expressed by the formula as follows:

[0066]

[0067] where, (·) T represents the transpose of the vector, represents vector multiplication, S represents the feature scale scaling factor, and M 12i represents V d1 and v dwi between the starting position-grid region information semantic response anchoring coding matrix.

[0068] Exemplarily, in step S433, in order to more precisely measure the possibility of each grid as a potential target operation point and dynamically aggregate the most responsive feature information, the present application introduces an adaptive weight learning mechanism to calculate the decision anchor adaptive splicing factor based on the feature distribution of the semantic response anchored coding matrix of each starting position-grid area information, and generate a set of decision anchor adaptive splicing factors for the starting position-grid area information. Here, the decision anchor adaptive splicing factor for the starting position-grid area information is used to reflect the semantic association strength of each grid area relative to the starting position of the bucket wheel manipulator and the dominance of the semantic interaction information between the two in the global context, and is the weight basis for subsequent dynamic response aggregation coding of features.

[0069] In one embodiment, as Figure 5 shown, based on the feature contributions of each starting position-grid area information semantic response anchored coding matrix in the set of starting position-grid area information semantic response anchored coding matrices, dynamically aggregating and coding the set of starting position-grid area information semantic response anchored coding matrices to obtain the target operation point search response coding vector includes: S4331, calculating the decision anchor adaptive splicing factor of each starting position-grid area information semantic response anchored coding matrix in the set of starting position-grid area information semantic response anchored coding matrices to obtain a set of decision anchor adaptive splicing factors for the starting position-grid area information. S4332, performing a weighting process based on the Softmax function on the set of decision anchor adaptive splicing factors for the starting position-grid area information to obtain a set of decision anchor adaptive splicing weight factors for the starting position-grid area information. S4333, fusing the set of starting position-grid area information semantic response anchored coding matrices based on the set of decision anchor adaptive splicing weight factors for the starting position-grid area information to obtain the target operation point search response coding vector.

[0070] In one embodiment, in step S4331, calculating the decision anchor adaptive splicing factor of each starting position-grid area information semantic response anchored coding matrix in the set of starting position-grid area information semantic response anchored coding matrices to obtain a set of decision anchor adaptive splicing factors for the starting position-grid area information includes: calculating the decision anchor adaptive splicing factor of each starting position-grid area information semantic response anchored coding matrix based on the statistical eigenvalue of each starting position-grid area information semantic response anchored coding matrix in the set of starting position-grid area information semantic response anchored coding matrices to obtain the set of decision anchor adaptive splicing factors for the starting position-grid area information, where the statistical eigenvalue includes the maximum eigenvalue, eigenvalue variance, eigenvalue mean, and number of eigenvalues.

[0071] In one embodiment, based on the statistical eigenvalue of each starting position-grid region information semantic response anchoring coding matrix in the set of starting position-grid region information semantic response anchoring coding matrices, calculating the decision anchor adaptive splicing factor of each starting position-grid region information semantic response anchoring coding matrix to obtain the set of starting position-grid region information decision anchor adaptive splicing factors, including: using the sum of the characteristic variance and the drift coefficient of the starting position-grid region information semantic response anchoring coding matrix as the numerator, and calculating the square of the difference between the maximum eigenvalue and the characteristic mean value of the starting position-grid region information semantic response anchoring coding matrix multiplied by the number of its eigenvalues, plus the drift coefficient and twice the characteristic variance as the denominator, to obtain the starting position-grid region information decision anchor adaptive splicing factor, where the drift coefficient is used to smooth the fluctuation of the characteristic distribution of the starting position-grid region information semantic response anchoring coding matrix. Specifically, this process can be represented by the formula:

[0072]

[0073] Where count(·) represents calculating the number of elements of the matrix, k represents the difference amplification coefficient, that is, the number of eigenvalues of the starting position-grid region information semantic response anchoring coding matrix, σ w represents the characteristic variance of the starting position-grid region information semantic response anchoring coding matrix, ∈ represents the drift coefficient of the starting position-grid region information semantic response anchoring coding matrix, μ represents the characteristic mean value of the starting position-grid region information semantic response anchoring coding matrix, max(·) is the maximum value function, E 12i represents M 12i corresponding starting position-grid region information decision anchor adaptive splicing factor, η is the intermediate transition representation value of the characteristic distribution balance state of the starting position-grid region information semantic response anchoring coding matrix, m ij is the jth eigenvalue of matrix M 12i and e is the natural constant.

[0074] Here, preferably, for the drift coefficient ∈ in the starting position-grid region information decision anchor adaptive splicing factor, for the state transition of the eigenvalue set distribution of the starting position-grid region information semantic response anchoring coding matrix from weak overall interpretability of the mean to strong local interpretability of the maximum value, the present application enhances the global dominance basis of the starting position-grid region information semantic response anchoring coding matrix by introducing the weak-to-strong interpretability generalization of the drift coefficient ∈.

[0075] Specifically, taking η as the intermediate state transition representation from weak interpretability to strong interpretability, for each eigenvalue m of the starting position-grid region information semantic response anchoring coding matrixij As the importance score of the starting position-grid area information semantic response anchoring coding matrix for the global smooth state transition, it is used to globally control the importance score weight of the intermediate state transition η relative to the global state transition, so as to realize the interpretable generalization inference of the weight basis of the starting position-grid area information decision anchor adaptive splicing factor.

[0076] Exemplarily, in step S4332, the Softmax function is used to weight the set of starting position-grid area information decision anchor adaptive splicing factors, that is, the Softmax function is used to normalize the set of starting position-grid area information decision anchor adaptive splicing factors into a weight set with probability distribution properties, and through the characteristics of the exponential function, the significant differences between the starting position-grid area information semantic response anchoring coding matrices are further amplified to enhance the distribution discrimination ability of features. Specifically, the calculation process of the set of starting position-grid area information decision anchor adaptive splicing weight factors can be expressed by the formula:

[0077] a 12i =softmax(E 12i )

[0078] where softmax(·) represents the normalized exponential function, and a 12i represents the starting position-grid area information decision anchor adaptive splicing weight factor of matrix M 12i .

[0079] Exemplarily, in step S4333, based on the generated weight distribution, the set of starting position-grid area information semantic response anchoring coding matrices is weighted and fused to obtain a global aggregated feature representation that fuses the semantic interaction information of each grid area relative to the starting position of the bucket wheel manipulator, and it is restored to a vector form through feature shape reshaping to generate the final target operation point search response coding vector. In this way, the spatial position relationship and accessibility information between the starting position of the bucket wheel manipulator and each grid area can be comprehensively considered, providing comprehensive and accurate data support for the subsequent determination of the target operation point. Specifically, the calculation process of fusing the set of starting position-grid area information semantic response anchoring coding matrices based on the set of starting position-grid area information decision anchor adaptive splicing weight factors to obtain the target operation point search response coding vector can be expressed by the formula:

[0080]

[0081] where, M cdenotes the starting position - grid area information semantic response anchoring encoding fusion matrix, reshape(·) denotes the feature shape reshaping function, and v c denotes the target operation point search response encoding vector.

[0082] Exemplarily, in step S44, based on the target operation point search response encoding vector, the target point identification result is determined. In one embodiment, determining the target point identification result based on the target operation point search response encoding vector includes: inputting the target operation point search response encoding vector into an operation point identification module based on a classifier to obtain the target point identification result, where the target point identification result is the grid corresponding to the stockpile operation point in the plannable three-dimensional grid map. Specifically, the operation point identification module performs classification processing on the target operation point search response encoding vector based on a pre-trained classifier model, and by analyzing the spatial position relationship, accessibility information, and operation efficiency factors contained in the target operation point search response encoding vector, identifies the grid area that best meets the requirements as the target operation point, thereby realizing the intelligent operation point selection of the bucket wheel robotic arm in complex operation scenarios.

[0083] In one embodiment, inputting the target operation point search response encoding vector into an operation point identification module based on a classifier to obtain the target point identification result includes: performing fully connected encoding on the target operation point search response encoding vector using the fully connected layer of the operation point identification module based on a classifier to obtain a fully connected encoded target operation point search response encoding vector. Inputting the fully connected encoded target operation point search response encoding vector into the Softmax classification function of the operation point identification module based on a classifier to obtain the target point identification result, where the target point identification result is the grid corresponding to the stockpile operation point in the plannable three-dimensional grid map.

[0084] Exemplarily, in step S5, path planning is performed based on the target point identification result. It should be understood that by using path planning algorithms such as the A* algorithm and the Dijkstra algorithm, in the three-dimensional grid map, according to the type of grid (idle area, non-passable area, or dangerous area) and the target point identification result, the optimal path from the starting point (the grid corresponding to the current position of the bucket wheel) to the target point is searched to ensure that the bucket wheel robotic arm can reach the target operation point efficiently and accurately for operation.

[0085] In summary, the intelligent control method for a bucket wheel stacker-reclaimer based on digital twin and deep learning according to the embodiments of the present application is elucidated. It uses lidar to collect three-dimensional point cloud data of the stockpile, constructs a three-dimensional surface model of the stockpile, and at the same time, through data twin technology, synchronizes the stockpile state in real time, and abstracts the three-dimensional surface model of the stockpile into a plannable three-dimensional grid map. Furthermore, in combination with the three-dimensional center point coordinates of the current robotic arm of the bucket wheel, through semantic query interaction analysis based on grid attribute information for each grid area in the plannable three-dimensional grid map, it intelligently determines the appropriate operation point position, and thus performs path planning based on the target operation point to achieve intelligent control of the bucket wheel stacker-reclaimer. In this way, the operation strategy can be adjusted in real time according to the change of the stockpile shape, further optimizing the resource allocation and operation process, and thus significantly improving the operation efficiency and safety of the bucket wheel stacker-reclaimer.

[0086] Figure 6 FIG. is a schematic block diagram of an intelligent control system for a bucket wheel stacker-reclaimer based on digital twin according to an embodiment of the present application. As Figure 6 shown, the intelligent control system 100 for a bucket wheel stacker-reclaimer based on digital twin includes: a three-dimensional point cloud data acquisition and processing module 110, configured to obtain three-dimensional point cloud data of a stockpile collected by a lidar, and process the three-dimensional point cloud data of the stockpile to obtain a three-dimensional surface model of the stockpile. A three-dimensional surface model display module 120 of the stockpile, configured to display the three-dimensional surface model of the stockpile by using a data twin model. A three-dimensional surface model abstraction module 130 of the stockpile, configured to abstract the three-dimensional surface model of the stockpile into a plannable three-dimensional grid map. A target point identification result acquisition module 140, configured to obtain the three-dimensional center point coordinates of the current robotic arm of the bucket wheel, and based on the three-dimensional center point coordinates of the current robotic arm of the bucket wheel, perform a semantic query response search for operation points for each grid in the plannable three-dimensional grid map based on grid attribute information to obtain a target point identification result, where the target point identification result is the corresponding grid of the stockpile operation point in the plannable three-dimensional grid map. A path planning execution module 150, configured to perform path planning based on the target point identification result.

[0087] Here, those skilled in the art can understand that the specific operations of each module in the above intelligent control system for a bucket wheel stacker-reclaimer based on digital twin have been introduced in detail in the description of the above Figures 1 to 5 intelligent control method for a bucket wheel stacker-reclaimer based on digital twin, and therefore, the repeated description thereof will be omitted.

[0088] Those of ordinary skill in the art will realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0089] It should be understood that the specific examples herein are only for helping those skilled in the art better understand the embodiments of this application, rather than limiting the scope of the embodiments of this application.

[0090] It should also be understood that in various embodiments of this application, the magnitudes of the serial numbers of the various processes do not mean the order of execution is prior or subsequent. The order of execution of the various processes should be determined by their functions and internal logics, and should not constitute any limitation to the implementation process of the embodiments of this application.

[0091] It should also be understood that the various implementation manners described in this specification can be implemented alone or in combination, and the embodiments of this application do not limit this.

[0092] Unless otherwise specified, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by those skilled in the technical field of this application. The terms used in this application are only for the purpose of describing specific embodiments, and are not intended to limit the scope of this application. The term "and / or" used in this application includes any and all combinations of one or more of the related listed items. The singular forms "a", "above", and "the" used in the embodiments of this application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0093] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the devices or units can be in electrical, mechanical, or other forms.

[0094] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or may be distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0095] In addition, each functional unit in various embodiments of the present application may be integrated into one processing unit, may exist separately as individual physical units, or two or more units may be integrated into one unit.

[0096] If the described function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0097] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed in the present application can easily think of changes or substitutions, which should all be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claimed rights.

Claims

1. An intelligent management and control method for a bucket wheel stacker and reclaimer based on digital twin, characterized in that: include: Acquiring three-dimensional point cloud data of the material pile collected by the laser radar, and processing the three-dimensional point cloud data of the material pile to obtain a three-dimensional surface model of the material pile; Using a data twin model to display a three-dimensional surface model of the stockpile; Abstracting the three-dimensional surface model of the stockpile into a plannable three-dimensional grid map; Acquire the three-dimensional center point coordinates of the current mechanical arm of the bucket wheel, and based on the three-dimensional center point coordinates of the current mechanical arm of the bucket wheel, perform a semantic query response search for the operation point based on the grid attribute information on each grid in the plannable three-dimensional grid map to obtain a target point identification result, wherein the target point identification result is a corresponding grid of the stockpile operation point in the plannable three-dimensional grid map; Path planning is performed based on the target point identification result.

2. The intelligent management and control method of bucket wheel stacker reclaimer based on digital twin according to claim 1 is characterized in that: Each grid in the plannable three-dimensional grid map contains position data and type data, wherein the type data includes an idle area, an impassable area or a dangerous area.

3. The intelligent management and control method of bucket wheel stacker reclaimer based on digital twin according to claim 2 is characterized in that: Based on the three-dimensional center point coordinates of the current manipulator of the bucket wheel, a semantic query response search of an operation point based on grid attribute information is performed on each grid in the plannable three-dimensional grid map to obtain a target point identification result, including: Performing semantic embedding coding on each grid information in the plannable three-dimensional grid map to obtain a set of semantic embedding coding vectors of grid area information; Performing low-dimensional embedding coding on the three-dimensional center point coordinates of the current manipulator of the bucket wheel to obtain a low-dimensional embedding coding vector of the starting point position; Performing dynamic response coding aggregation based on an adaptive attention mechanism on the set of the starting point position low-dimensional embedding coding vector and the grid area information semantic embedding coding vector to obtain a target operation point search response coding vector; The target point identification result is determined based on the target operation point search response coding vector.

4. The intelligent management and control method of bucket wheel stacker reclaimer based on digital twin according to claim 3 is characterized in that: The set of the starting point position low-dimensional embedding coding vector and the grid area information semantic embedding coding vector is subjected to dynamic response coding aggregation based on an adaptive attention mechanism to obtain a target operation point search response coding vector, including: Performing deep implicit feature extraction based on fully connected coding on each grid area information semantic embedded coding vector in the set of the starting point position low-dimensional embedded coding vector and the grid area information semantic embedded coding vector to obtain a set of starting point position feature deep implicit coding vector and grid area information semantic feature deep implicit coding vector; Inputting each grid area information semantic feature depth implicit coding vector in the set of the starting position feature depth implicit coding vector and the grid area information semantic feature depth implicit coding vector into the semantic response decision anchor component to obtain a set of starting position-grid area information semantic response anchor coding matrices; Based on the feature contribution of each starting position-grid area information semantic response anchor coding matrix in the set of starting position-grid area information semantic response anchor coding matrices, the set of starting position-grid area information semantic response anchor coding matrices is dynamically aggregated and encoded to obtain the target work point search response coding vector.

5. The intelligent management and control method of bucket wheel stacker reclaimer based on digital twin according to claim 4, characterized in that: Based on the feature contribution of each starting position-grid area information semantic response anchor coding matrix in the set of the starting position-grid area information semantic response anchor coding matrix, the set of the starting position-grid area information semantic response anchor coding matrix is ​​dynamically aggregated and encoded to obtain the target operation point search response coding vector, including: Calculating the decision anchor adaptive splicing factor of each starting position-grid area information semantic response anchor coding matrix in the set of starting position-grid area information semantic response anchor coding matrices to obtain a set of starting position-grid area information decision anchor adaptive splicing factors; Performing a weighting process based on a Softmax function on the set of starting position-grid region information decision anchor adaptive splicing factors to obtain a set of starting position-grid region information decision anchor adaptive splicing weight factors; Based on the set of starting position-grid area information decision anchor adaptive splicing weight factors, the set of starting position-grid area information semantic response anchor coding matrices is fused to obtain the target operation point search response coding vector.

6. The intelligent management and control method of bucket wheel stacker reclaimer based on digital twin according to claim 5, characterized in that: Calculating the decision anchor adaptive splicing factor of each starting position-grid area information semantic response anchor coding matrix in the set of starting position-grid area information semantic response anchor coding matrices to obtain a set of starting position-grid area information decision anchor adaptive splicing factors, including: Based on the statistical eigenvalues ​​of each starting position-grid area information semantic response anchor coding matrix in the set of starting position-grid area information semantic response anchor coding matrices, the decision anchor adaptive splicing factors of each starting position-grid area information semantic response anchor coding matrix are calculated to obtain the set of starting position-grid area information decision anchor adaptive splicing factors, and the statistical eigenvalues ​​include the maximum eigenvalue, eigenvariance, eigenmean and eigenvalue number.

7. The intelligent management and control method of bucket wheel stacker reclaimer based on digital twin according to claim 6, characterized in that: Based on the statistical eigenvalues ​​of each starting position-grid area information semantic response anchor coding matrix in the set of the starting position-grid area information semantic response anchor coding matrix, the decision anchor adaptive splicing factor of each starting position-grid area information semantic response anchor coding matrix is ​​calculated to obtain the set of the starting position-grid area information decision anchor adaptive splicing factors, including: The sum of the feature variance and the drift coefficient of the starting position-grid area information semantic response anchor coding matrix is ​​used as the numerator, and the square of the difference between the maximum eigenvalue and the feature mean of the starting position-grid area information semantic response anchor coding matrix is ​​calculated multiplied by the number of its eigenvalues, and then the drift coefficient and twice the feature variance are added as the denominator to obtain the starting position-grid area information decision anchor adaptive splicing factor, wherein the drift coefficient is used to smooth the feature distribution fluctuation of the starting position-grid area information semantic response anchor coding matrix.

8. The intelligent management and control method of bucket wheel stacker reclaimer based on digital twin according to claim 7, characterized in that: Determining the target point identification result based on the target operation point search response coding vector includes: The target operation point search response encoding vector is input into a classifier-based operation point identification module to obtain the target point identification result.

9. The intelligent management and control method of bucket wheel stacker reclaimer based on digital twin according to claim 8, characterized in that: Inputting the target operation point search response encoding vector into a classifier-based operation point identification module to obtain the target point identification result, including: Performing full-connection encoding on the target work point search response encoding vector using the fully-connected layer of the classifier-based work point identification module to obtain a fully-connected encoded target work point search response encoding vector; The fully connected encoded target operation point search response encoding vector is input into the Softmax classification function of the classifier-based operation point identification module to obtain a target point identification result, wherein the target point identification result is the corresponding grid of the stockpile operation point in the plannable three-dimensional grid map.

10. An intelligent management and control system for bucket wheel stacker reclaimer based on digital twin, characterized in that: include: A three-dimensional point cloud data acquisition and processing module is used to acquire the three-dimensional point cloud data of the material pile acquired by the laser radar, and process the three-dimensional point cloud data of the material pile to obtain a three-dimensional surface model of the material pile; A material pile three-dimensional surface model display module, used to display the material pile three-dimensional surface model using a data twin model; A three-dimensional surface model abstraction module for a pile of materials, used for abstracting the three-dimensional surface model of the pile of materials into a plannable three-dimensional grid map; A target point identification result acquisition module is used to obtain the three-dimensional center point coordinates of the current mechanical arm of the bucket wheel, and based on the three-dimensional center point coordinates of the current mechanical arm of the bucket wheel, perform a work point semantic query response search based on grid attribute information on each grid in the plannable three-dimensional grid map to obtain a target point identification result, wherein the target point identification result is a corresponding grid of the stockpile work point in the plannable three-dimensional grid map; The path planning execution module is used to execute path planning based on the target point identification result.

Images