A digital twin multi-agent collaborative control method for intelligent mining in mines

By applying multi-head attention mechanism algorithm and deep learning technology in intelligent mining of mines, digital twins are established for multi-agent collaborative control, solving the limitations of multi-objective detection and collaborative control in the existing technology, and realizing the working face of mine intelligent mining adaptive and efficient collaborative control.

CN115437257BActive Publication Date: 2024-11-15CHINA UNIV OF MINING & TECH (BEIJING)
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202211209148.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-09-30
Publication Date
2024-11-15
Estimated Expiration
2042-09-30

AI Technical Summary

Technical Problem

The existing technology has limitations in multi-object detection, tracking and identification and coordinated control in intelligent mining, and it is difficult to achieve effective coordinated control of multiple agents.

Method used

Using multi-head attention mechanism algorithm combined with deep learning, by establishing digital twins in the mining intelligent mining working surface, multi-integrated control of coal miners, hydraulic support and scraper conveyors is realized, and industrial Internet, Internet of Things and deep vision is used. The measurement technology conducts information interaction and feedback, and uses the multi-head attention mechanism and the full-connection layer training strategy to optimize the action value model.

Benefits of technology

The intelligent collaborative control capability of the mining intelligent mining work surface equipment has been improved, self-perception, self-learning, adaptation and self-control have been realized, and the robustness and efficiency of collaborative control of multiple agents has been improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115437257B_ABST
    Figure CN115437257B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for collaborative control of digital twin multi-agents for intelligent mining in a mine, and the system for implementing the method includes above-ground equipment and underground equipment. The method steps for implementing the system are as follows: (1) establishing digital twins of coal mining machines and hydraulic supports in a mine intelligent mining working face in Solidworks; (2) the mine hydraulic supports transmit sensor signals to a digital twin monitoring platform through industrial Ethernet or 5G wireless network through various sensors installed on the hydraulic supports; (3) the digital twin monitoring platform uses a multi-head attention mechanism algorithm to extract feature information from the coal mining machine and hydraulic support multi-agents; (4) after each agent information is extracted, a multi-head attention mechanism information exchange is performed with other agents, and a learning strategy training is performed using a DQN training model; (5) each sensor data is transmitted to the digital twin monitoring platform, and matched through the trained model strategy, thereby correcting the hydraulic support action.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a digital twin technology for intelligent mining in mines, and in particular to a digital twin multi-agent collaborative control method for intelligent mining in mines. Background Art

[0002] Intelligent mining technology in mines is developing towards informatization, intelligence and unmanned operation. Intelligent working face collaborative control technology is one of the key technologies for intelligent mining. As one of the "three machines" of intelligent mining working face, the realization of collaborative control of hydraulic support plays a vital role in intelligent coal mining.

[0003] At present, digital twin technology has been applied to the actual scenarios of intelligent mining in mines. The virtual mines and physical mines established by them can realize digital twin modeling, virtual-real interaction and virtual-real control of physical entities in mines through interactive mapping, synchronous feedback and symbiosis and co-intelligence. At the same time, in the research on intelligent operation and maintenance and fault diagnosis technology of mining equipment, some scholars have applied artificial intelligence technologies such as machine vision and deep learning algorithms to the field of equipment health management in intelligent mines. In terms of deep learning algorithms, convolutional neural networks, residual networks, LSTM and other methods are mainly used. However, the above methods still have certain limitations in multi-target detection, tracking and recognition and collaborative control in mines, and it is difficult to achieve multi-target tracking and multi-agent collaborative control. In recent years, the attention mechanism has made important breakthroughs in fields such as image and natural language processing. It has been proven to be beneficial to improving the performance of deep learning models and has good robustness in multi-target detection and recognition.

[0004] The attention mechanism algorithm is based on the fact that different parts of the human visual retina have different degrees of information processing capabilities, that is, the mechanism of visual acuity for target recognition and tracking. The use of multi-head attention can parallelly calculate and select multiple information from the input information, thereby realizing the detection and recognition of multiple targets. The multi-attention intention communication learning algorithm can represent the network information input of the intention of multiple agents, and use the attention mechanism to process the intention information and communication information at the same time, providing a useful innovative idea for multi-agent collaborative control technology. Summary of the invention

[0005] The technical problem to be solved by the present invention is: in response to the above-mentioned problems, a digital twin multi-agent collaborative control method for intelligent mining in mines is proposed, aiming to realize the collaborative control of multi-agents in the intelligent mining working face of mines.

[0006] The technical solution of the present invention is: to propose a digital twin multi-agent collaborative control method for intelligent mining in mines, and the system for implementing the method includes above-ground equipment and underground equipment, and the above-ground equipment communicates with the underground equipment through an optical link.

[0007] The well equipment includes an industrial Internet server and a digital twin monitoring platform.

[0008] The underground equipment includes a coal mining machine, a hydraulic support and a scraper conveyor.

[0009] The physical entity of the underground equipment is connected to the digital twin of the underground equipment through the mining Internet of Things to achieve information mapping interaction and feedback.

[0010] The coal mining machine is equipped with an inertial navigation system and an odometer, which are respectively used to collect the attitude information of the coal mining machine, including pitch angle, roll angle, heading angle and positioning information.

[0011] The push jack of the hydraulic support is equipped with a stroke sensor for monitoring the push position of the hydraulic support.

[0012] The top beam of the hydraulic support is equipped with a mining intrinsically safe height sensor for measuring the height of the coal mining working face.

[0013] The column bottom cavity of the hydraulic support is equipped with a pressure sensor for monitoring the top beam pressure and the top plate pressure.

[0014] The underground equipment is equipped with a deep vision measurement RGB-D camera to obtain the three-dimensional spatial posture information of the coal mining machine, hydraulic support and scraper conveyor of the intelligent mining working face of the mine.

[0015] The hydraulic support is equipped with a photoelectric positioning sensor and a wireless radio frequency transmission module, which is used for positioning identification and wireless communication of multiple intelligent bodies in the intelligent mining working face of a mine.

[0016] The digital twin monitoring platform is configured with a main controller, and the hydraulic support is configured with a sub-controller. Data is transmitted between the main controller and the sub-controller via an SPI interface. The support sub-controller is used to monitor the operating condition of the hydraulic support, and control the support to perform corresponding pushing, moving, side protection, top protection, and cyclic coal cutting actions. Information is transmitted between the support controllers via a CAN bus interface, a MODBUS bus interface, a PROFIBUS bus interface, an RS-485 bus interface, an industrial Ethernet interface, or a 5G wireless interface.

[0017] The digital twin multi-agent collaborative control method for intelligent mining in a mine comprises the following implementation steps:

[0018] Step 1: Use Solidworks to build digital twins of the coal mining machine, hydraulic support and scraper conveyor of the intelligent mining working face of the mine;

[0019] Step 2: The sensor signals of the hydraulic supports of the intelligent mining working face of the mine are transmitted to the digital twin monitoring platform through the industrial Ethernet network or the 5G wireless network to monitor the posture, operating status and relative position of the hydraulic supports and the coal mining machine;

[0020] Step 3, the digital twin monitoring platform uses a multi-head attention mechanism algorithm to extract feature information from the coal mining machine and hydraulic support multi-agent of the mine intelligent mining working face;

[0021] Step 4: After each agent extracts feature information, it uses a multi-head attention mechanism to interact with other agents, and then passes through a fully connected layer to obtain the state action value function of the agent. Finally, these action values ​​are weighted and added, and the DQN training model learning strategy is used for model training.

[0022] In step 5, the mine multi-agent uses the trained model strategy for matching, and optimizes and adjusts the error instructions, transmits them to the hydraulic support solenoid valve drive module, and then corrects the hydraulic support action.

[0023] The digital twin platform for intelligent mining in the mine adopts the attention mechanism algorithm to train the action value model of hydraulic support and coal mining machine, and its formula is:

[0024]

[0025] Among them: Att(Q,K,V) is the self-attention value, the weight is the distribution of the correlation or similarity between each sample and each piece of information; Q, K, V represent the query matrix, key vector matrix and value vector matrix respectively; is the activation function, usually softmax.

[0026] The digital twin multi-agent collaborative control method for intelligent mining in a mine further comprises the following steps:

[0027] Step 1: Install an inertial navigation system and odometer on the coal mining machine to collect the coal mining machine's attitude information in real time, including pitch angle, roll angle, and heading angle. Combined with the odometer data, the coal mining machine can be accurately positioned, and the straightness of the scraper conveyor can be obtained based on its running trajectory, providing data support for the mine intelligent mining face straightening control and hydraulic support push operation;

[0028] Step 2: According to the cutting action of the coal mining machine in the intelligent mining working face of the mine and the distance measured by the RGB-D camera, an RGB-D camera is installed for every five hydraulic supports, and the posture information of the coal mining machine and the hydraulic support collected by the RGB-D camera is transmitted to the digital twin monitoring platform, and the acquired video is processed to achieve real-time tracking of the intelligent mining equipment in the mine;

[0029] Step 3: The digital twin monitoring platform makes training decisions based on the received information and sends certain signals to the main controller, which then transmits the signals to the sub-controllers;

[0030] Step 4: Data instructions are transmitted from the sub-controller to the solenoid valve drive circuit of the hydraulic support, which drives the hydraulic circuit to execute the extension and retraction action of the jack. The digital twin monitoring platform integrates analysis and interactive feedback of the real-time operation data of the intelligent mining equipment in the intelligent mine, and automatically realizes the driving actions of following the machine, pushing the slide, pulling the frame, and moving the machine.

[0031] The digital twin multi-agent collaborative control method for intelligent mining in a mine, the steps of constructing the digital twin include:

[0032] Step 1: The physical entity of the intelligent mining working face of the mine is mapped to form a digital twin model through multi-dimensional mapping, and the physical entity of the intelligent mining working face of the mine is virtually mapped and simulated through model drive and data drive, and the mapping and simulation data are transmitted to the twin data center;

[0033] Step 2: Feedback the real-time data of the physical entity of the mine intelligent mining working face to the twin data center;

[0034] Step 3: Multi-source fusion of the twin data center’s mapping, simulation data, and real-time data;

[0035] In step 4, the twin data center performs deep learning and optimization iterations on the fused data, and uses a multi-head attention mechanism algorithm to train and optimize the model, thereby forming an optimal collaborative control strategy.

[0036] The beneficial effects of the present invention are:

[0037] This invention is based on digital twin technology and multi-head attention mechanism algorithm, and provides a method for collaborative control of digital twin multi-agents in intelligent mining of mines by constructing multiple digital twins for coal mining machines, hydraulic supports and scraper conveyors in intelligent mining working faces of mines. This invention solves the problems of weak intelligent collaborative ability, low intelligent control degree, self-perception, self-learning, self-adaptation and self-control in intelligent mining working faces of mines, and improves the intelligent collaborative control ability of equipment in intelligent mining working faces of mines. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 Schematic diagram of the system composition of the digital twin multi-agent collaborative control method for intelligent mining working face.

[0039] Figure 2 This is the overall framework diagram of the multi-agent attention intention communication algorithm.

[0040] Figure 3 This is the collaborative workflow diagram of the hydraulic support. DETAILED DESCRIPTION

[0041] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, specific implementations of the present invention are described in detail below with reference to the accompanying drawings.

[0042] Figure 1 This is a schematic diagram of the system composition of the digital twin multi-agent collaborative control method for intelligent mining working faces. Figure 1 As shown, the system for realizing the multi-agent collaborative control of intelligent mining in the above mine mainly includes: an intelligent mining working face hydraulic support (101), sensors on the hydraulic support (102), an RGB-D camera (103), a digital twin monitoring platform (104), an electromagnetic valve drive module (105), and a coal mining machine (106).

[0043] Among them, multiple hydraulic supports (101) are equipped with multiple sensors (102), such as travel sensors, height sensors, pressure sensors, and photoelectric positioning sensors. An inertial navigation system and an odometer are installed on the coal mining machine (106) to collect the attitude information (pitch angle, roll angle, and heading angle) of the coal mining machine in real time. Combined with the odometer data, the coal mining machine can be accurately positioned. The straightness of the scraper conveyor can be obtained according to its running trajectory, providing data support for the intelligent mining working face straightening control of the mine and the pushing operation of the hydraulic support. The hydraulic support information measured by these sensors and the RGB-D camera (103) can be transmitted to the digital twin monitoring platform (104) in real time through the industrial Ethernet network or the 5G wireless network. The digital twin monitoring platform (104) is a computing power platform, which includes functions such as data management, model simulation, learning prediction, calculation optimization, and decision autonomy. When the data is transmitted to the digital twin monitoring platform through the industrial Ethernet network or the mine 5G network and sent to the user domain after fusion processing, it is fed back to the terminal control through the user, and the digital quantity is transmitted to the controller, and the solenoid valve drive module (105) is controlled by the controller, thereby realizing intelligent control of the hydraulic support.

[0044] Figure 2 This is the overall framework diagram of the multi-agent attention intention communication algorithm. Figure 2As shown in the figure, the multi-agent attention intention communication algorithm is described as follows: (1) Through the processing of information features by the fully connected layer, it is proposed to add a public intention network for the agent at this stage. These intention networks are copied from the network with the best performance in the recent stage. The local observations and actions of the agent are processed by the GRU recurrent neural network to obtain state action information containing history. This part of information is extracted by the attention mechanism with the intention information of other agents obtained through the intention network to obtain state information containing intention information. (2) The information interaction channel of the multi-head attention mechanism is added between the agents. Each agent broadcasts the information to be interacted. The agent extracts multi-head attention information based on its own information and the interaction information received from other agents. Then the agent decision control information is input to the GRU unit. Finally, through information interaction and iterative optimization with other multi-agents, the coordinated control of multi-agents is realized. (3) The high-level information extracted through intention and communication is passed through a fully connected layer to obtain the local Q value function of the agent in the current state. A simple addition operation is performed to obtain the overall Q value function.

[0045] The steps of implementing the multi-agent communication learning algorithm of the hydraulic support include: (1) Processing information features and adding a public intention network to the agent. The local observations and actions of the agent are processed through the GRU recurrent neural network to obtain state action information containing history, and this part of information is extracted by the attention mechanism with the intention information of other agents obtained through the intention network to obtain state information containing intention information. (1) Add a communication channel of multi-head attention mechanism between multiple agents. Each agent broadcasts the information to be communicated. The agent extracts multi-head attention information based on its own information and the communication information received from other agents, so as to obtain information that is effective for its own decision and input it into the GRU unit. (3) In order to make the communication information more sufficient, the agent will communicate with other agents for multiple iterations before making a decision. (4) The high-level information extracted through intention and communication is passed through a fully connected layer to obtain the local Q value function of the agent in the current state, and a simple addition operation is performed to obtain the overall Q value function. (5) Use the DQN training mode to learn the strategy. To calculate the target Q value, its parameter θ′ will be replaced with the parameter of the Q value network at the corresponding moment at regular intervals. This will make the training process more stable and help convergence.

[0046] In the multi-agent collaborative environment of hydraulic support, agent i receives observations of the environment. Output corresponding actions through its own network After executing the action according to the current state, the environment gives a collective reward. The multi-agents in the mine can choose whether to use limited channels to communicate information. In order to make better use of the communication information, at different times, the strategy of the current agent has different concerns about other agents. Therefore, the attention model is selected for the communication information. By assigning different weights to the communication information of other agents, effective communication information can be extracted. Specifically, it includes: (1) Using the GRU neural network to process the communication between the multi-agents of the hydraulic support. The communication between the multi-agents can be cycled for k time steps. The state of the hidden layer of the network comes from the hidden layer state output of the GRU in the previous stage. Considering that it takes a certain amount of time for the multi-agents in the mine to send communication information in actual problems, if the communication information is processed synchronously, it is necessary to wait until all the information is sent each time, and the decision-making efficiency is bound to be low. Therefore, when realizing the communication information, asynchronous processing is generally adopted, that is, the communication information of the previous moment is accepted for processing at the current moment, and the communication information of the next moment is output. (2) The calculation formula of the communication information c is c = Multihead (Q, K, V) = Concat (head 1 ,head 2 …,head 8 ), where K, Q, and V have different situations according to the communication time and decision time. The algorithm adopts an eight-head attention mechanism. In addition, head i =Attention(A T W i Q ,A T W i K ,A T W i V ), where A T W i Q , A T W i K , A T W i V They are mapping weights respectively, and the matrix A is the output of each agent through the intention network attention model (a 0 t ,…,a n t ). (3) The calculation of the communication information c is divided into three cases: The first case is the first time that the agent interacts with the environment to make a decision. At t = k and k = 0, the agent has no communication information from the previous moment. The information input to the multi-head attention communication channel is (A t ,A t ,A t); The second case is when the agent is at t>0 and k=0. At this time, the communication information of the agent is given by the hidden layer vector of the previous decision moment. At k=0, the agent needs to use the intention observation information to extract attention. At this time, the input of the multi-head attention unit is (A t , ); The third case is when the communication time step k>0, the input is the hidden layer information of the previous communication time step at the current moment constitute.

[0047] Figure 3 This is a collaborative workflow diagram for intelligent mining face in mines. Figure 3 As shown in the figure, firstly, the real-time data is collected by the sensors of the coal mining machine and hydraulic support of the intelligent mining working face of the mine, and then the data is transmitted to the digital twin monitoring platform through the mine 5G network. The digital twin monitoring platform then screens the data and iterates the calculation to finally determine the time parameters of the hydraulic support autonomous collaborative operation. At the same time, the digital twin monitoring platform stores the original data and the calculation result data in the database. As the working face advances, these data and the newly generated data are used as new training samples to iteratively optimize the calculation model, so as to achieve dynamic optimization of the parameters.

[0048] The above content is a further detailed description of the present invention in combination with a specific preferred embodiment. It cannot be determined that the specific implementation methods of the present invention are limited to this. For ordinary technicians in the technical field to which the present invention belongs, several simple replacements and changes can be made without departing from the design concept of the present invention, which should be deemed to belong to the scope of protection involved in the claims submitted by the present invention.

Claims

1. A digital twin multi-agent collaborative control method for intelligent mining in a mine, wherein the equipment implementing the method includes above-ground equipment and underground equipment, and is characterized in that: The above-ground equipment includes an industrial Internet server and a digital twin monitoring platform; The underground equipment includes a coal mining machine, a hydraulic support and a scraper conveyor; The above-the-hole equipment communicates with the below-the-hole equipment via an optical link; The physical entity of the underground equipment is connected to the digital twin of the underground equipment through the mining Internet of Things to achieve information mapping interaction and feedback; The coal mining machine is equipped with an inertial navigation system and an odometer, which are respectively used to collect attitude information of the coal mining machine, including pitch angle, roll angle, heading angle and positioning information; The hydraulic support's push jack is equipped with a stroke sensor for monitoring the hydraulic support's push position; The top beam of the hydraulic support is equipped with a mining intrinsically safe height sensor for measuring the height of the coal mining working face; The bottom cavity of the column of the hydraulic support is equipped with a pressure sensor for monitoring the top beam pressure and the top plate pressure; The underground equipment is equipped with a deep vision measurement RGB-D camera to obtain the three-dimensional spatial posture information of the coal mining machine, hydraulic support and scraper conveyor of the intelligent mining working face of the mine; The hydraulic support is equipped with a photoelectric positioning sensor and a wireless radio frequency transmission module for positioning identification and wireless communication of multiple intelligent bodies in the intelligent mining working face of the mine; The digital twin monitoring platform is configured with a main controller, and the hydraulic support is configured with a sub-controller. The main controller and the sub-controller are used for data transmission through the SPI interface; the support sub-controller is used to monitor the operating conditions of the hydraulic support, and control the support to perform corresponding pushing, moving, side protection, top protection, and cyclic coal cutting actions. The support controllers are used to transmit information through a CAN bus interface, or a MODBUS bus interface, or a PROFIBUS bus interface, or an RS-485 bus interface, or an industrial Ethernet interface, or a 5G wireless interface; Its characteristics are also that The digital twin multi-agent collaborative control method for intelligent mining in a mine comprises the following implementation steps: Step 1: Use Solidworks to build digital twins of the coal mining machine, hydraulic support and scraper conveyor of the intelligent mining working face of the mine; Step 2: various sensor signals of the hydraulic support of the intelligent mining working face of the mine are transmitted to the digital twin monitoring platform through the industrial Ethernet network or the 5G wireless network to realize the monitoring of the hydraulic support posture, operating status and relative position of the coal mining machine; Step 3, the digital twin monitoring platform uses a multi-head attention mechanism algorithm to extract feature information from the coal mining machine and hydraulic support multi-agent of the mine intelligent mining working face; Step 4: After each agent extracts feature information, it uses a multi-head attention mechanism to interact with other agents, and then passes through a fully connected layer to obtain the state action value function of the agent. Finally, these action values ​​are weighted and added, and the DQN training model learning strategy is used for model training. In step 5, the mine multi-agent uses the trained model strategy for matching, and optimizes and adjusts the error instructions, transmits them to the hydraulic support solenoid valve drive module, and then corrects the hydraulic support action.

2. The method for multi-agent collaborative control of digital twins for intelligent mining in mines according to claim 1 is characterized in that: The digital twin platform for intelligent mining in mines uses the attention mechanism algorithm to train the action value model of hydraulic supports and coal mining machines. The formula is: Among them: Att(Q,K,V) is the self-attention value, the weight is the distribution of the correlation or similarity between each sample and each piece of information; Q, K, V represent the query matrix, key vector matrix and value vector matrix respectively; is the activation function, usually softmax.

3. The method for multi-agent collaborative control of digital twins for intelligent mining in mines according to claim 1 is characterized in that: Further comprising the steps of: Step 1: Install an inertial navigation system and odometer on the coal mining machine to collect the coal mining machine's attitude information in real time, including pitch angle, roll angle, and heading angle. Combined with the odometer data, the coal mining machine can be accurately positioned, and the straightness of the scraper conveyor can be obtained based on its running trajectory, providing data support for the mine intelligent mining face straightening control and hydraulic support push operation; Step 2: According to the cutting action of the coal mining machine in the intelligent mining working face of the mine and the distance measured by the RGB-D camera, an RGB-D camera is installed for every five hydraulic supports, and the posture information of the coal mining machine and the hydraulic support collected by the RGB-D camera is transmitted to the digital twin monitoring platform, and the acquired video is processed to achieve real-time tracking of the intelligent mining equipment in the mine; Step 3: The digital twin monitoring platform makes training decisions based on the received information and sends certain signals to the main controller, which then transmits the signals to the sub-controllers; Step 4: Data instructions are transmitted from the sub-controller to the solenoid valve drive circuit of the hydraulic support, which drives the hydraulic circuit to execute the extension and retraction action of the jack. The digital twin monitoring platform integrates analysis and interactive feedback of the real-time operation data of the intelligent mining equipment in the intelligent mine, and automatically realizes the driving actions of following the machine, pushing the slide, pulling the frame, and moving the machine.

4. The method for multi-agent collaborative control of digital twins for intelligent mining in mines according to claim 1 is characterized in that: The steps of constructing the digital twin include: Step 1: The physical entity of the intelligent mining working face of the mine is mapped into a digital twin model through multi-dimensional mapping. Through model-driven and data-driven methods, the physical entity of the intelligent mining working face of the mine is virtually mapped and simulated, and the mapping and simulation data are transmitted to the twin data center; Step 2: Feedback the real-time data of the physical entity of the mine intelligent mining working face to the twin data center; Step 3: Multi-source fusion of the twin data center’s mapping, simulation data, and real-time data; In step 4, the twin data center performs deep learning and optimization iterations on the fused data, and uses a multi-head attention mechanism algorithm to train and optimize the model, thereby forming an optimal collaborative control strategy.