Mine digital twin system roaming path planning method based on reinforcement learning
By constructing a deep reinforcement learning model based on reinforcement learning methods, the problem of long roaming path planning in the mine digital twin system was solved, and more efficient path planning was achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XIAN RES INST OF CHINA COAL TECH & ENG GRP CORP
- Filing Date
- 2022-12-29
- Publication Date
- 2026-06-09
AI Technical Summary
Existing methods for roaming path planning in mine digital twin systems are time-consuming and difficult to complete efficiently.
We employ a reinforcement learning-based approach, designing a training environment and constructing a deep reinforcement learning model, including an Actor module and a Critic module. We then utilize the RNN network of the GRU module for path planning to train the optimal roaming path.
It significantly shortened the roaming path planning time of the mine digital twin system and improved the efficiency of path planning.
Smart Images
Figure CN115952733B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of mine digital twins and relates to a roaming path planning method, specifically a roaming path planning method for a mine digital twin system based on reinforcement learning. Background Technology
[0002] The deep integration of new-generation information technology with coal mining technology provides key technological support for the technological transformation and development of intelligent mines, and will promote the innovative, green, safe, and efficient development of the coal industry. Digital twins, as the core technology of intelligent unmanned coal mining, are an organic combination of artificial intelligence theory and methods with digital technology. Digital twins are expected to solve the key technical challenges of interaction and shared intelligence between the physical and information worlds in intelligent unmanned coal mining, and are of great significance for promoting the intelligent construction of the entire life cycle of mines and the high-quality development of the coal industry.
[0003] A mine digital twin system can perfectly simulate the real environment underground. Therefore, one of the most critical functions of a mine digital twin system is immersive 3D visualization, i.e., digital twin system walkthrough. Typically, users specify the start point, intermediate points, and end point of the walkthrough within the digital twin system. This is an NP-complete problem, and solving it using traditional multi-objective optimization algorithms often consumes a significant amount of time. Summary of the Invention
[0004] In view of the shortcomings of the existing technology, the purpose of this invention is to provide a roaming path planning method for a mine digital twin system based on reinforcement learning, so as to solve the technical problem of long time consumption of existing roaming path planning methods.
[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:
[0006] A method for roaming path planning in a mine digital twin system based on reinforcement learning, comprising a training phase and an inference phase;
[0007] The training phase specifically includes the following steps:
[0008] Step 1: Establish a roaming path map based on the roamable locations and path relationships in the mine digital twin system, as a training environment;
[0009] Step 2: Specify the points that need to be traversed during the roaming of the mine digital twin system on the roaming path map;
[0010] Step 3: Use the actor critic framework to build a deep reinforcement learning model;
[0011] The deep reinforcement learning model includes an Actor module and a Critic module;
[0012] The Actor module is a pointer network, which contains two RNN modules, which serve as the encoding module and the decoding module, respectively.
[0013] The Critic module is a policy network;
[0014] Step 4: Input the points to be traversed into the encoding module of the pointer network. The decoding module of the pointer network outputs the optimal sequence of the points to be traversed and uses the negative of the time required for the optimal sequence as the reward for the sequence.
[0015] Step 5: Input the points to be traversed and the optimal sequence into the policy network, and output the quality of the sequence.
[0016] Step six: Based on the reward of the sequence obtained in step four and the quality of the sequence obtained in step five, update the Actor module through backpropagation;
[0017] Step 7: Increment the iteration count by 1, take the optimal sequence obtained in Step 4 as the point to be traversed, enter Step 4, until the maximum number of iterations is reached, and output the final Actor module.
[0018] The reasoning phase includes the following:
[0019] Input the points that need to be traversed during the roaming process of the mine digital twin system specified in step two into the final Actor module obtained in step seven. The Actor module provides the roaming path with the shortest current time based on the training environment.
[0020] This invention also includes the following technical features:
[0021] Both of the RNN modules described above use the GRU module.
[0022] Compared with the prior art, the beneficial technical effects of this invention are:
[0023] This invention designs an interactive training environment for the roaming locations and path relationships required by the mine digital twin system. Based on the actor-critic framework, a deep reinforcement learning model is constructed, and finally an Actor module that can provide roaming paths for the mine digital twin system is trained. Compared with traditional methods, this can significantly reduce the time for roaming path planning of the mine digital twin system and solve the technical problem of long time consumption in existing roaming path planning methods. Attached Figure Description
[0024] Figure 1 This is a schematic diagram of the deep reinforcement learning model structure in this invention.
[0025] Figure 2This is a schematic diagram of the roaming path of the mine digital twin system of the present invention;
[0026] The specific content of the present invention will be further explained in detail below with reference to the embodiments. Detailed Implementation
[0027] It should be noted that, unless otherwise specified, all components in this invention are those known in the art.
[0028] The following are specific embodiments of the present invention. It should be noted that the present invention is not limited to the following specific embodiments. All equivalent modifications made based on the technical solutions of this application fall within the protection scope of the present invention.
[0029] This invention provides a roaming path planning method for a mine digital twin system based on reinforcement learning, including a training phase and an inference phase;
[0030] The training phase specifically includes the following steps:
[0031] Step 1: Establish a roaming path map based on the roamable locations and path relationships in the mine digital twin system, as a training environment;
[0032] Step 2: Specify the points that need to be traversed during the roaming of the mine digital twin system on the roaming path map;
[0033] Step 3: Use the actor critic framework to build a deep reinforcement learning model;
[0034] Deep reinforcement learning models include Actor modules and Critic modules;
[0035] The Actor module is a pointer network, which contains two RNN modules, which serve as the encoding and decoding modules, respectively.
[0036] The Critic module is a policy network;
[0037] Step 4: Input the points to be traversed into the encoding module of the pointer network. The decoding module of the pointer network outputs the optimal sequence of the points to be traversed and uses the negative of the time required for the optimal sequence as the reward for the sequence.
[0038] Step 5: Input the points to be traversed and the optimal sequence into the policy network, and output the quality of the sequence.
[0039] Step six: Based on the reward of the sequence obtained in step four and the quality of the sequence obtained in step five, update the Actor module through backpropagation;
[0040] Step 7: Increment the iteration count by 1, take the optimal sequence obtained in Step 4 as the point to be traversed, enter Step 4, until the maximum number of iterations is reached, and output the final Actor module.
[0041] The reasoning phase includes the following:
[0042] Input the points that need to be traversed during the roaming process of the mine digital twin system specified in step two into the final Actor module obtained in step seven. The Actor module provides the roaming path with the shortest current time based on the training environment.
[0043] In the above technical solution, an interactive training environment was designed for the roamable locations and path relationships required by the mine digital twin system. A deep reinforcement learning model was constructed based on the actor-critic framework, and finally an Actor module that can provide roaming paths for the mine digital twin system was trained. Compared with traditional methods, this can significantly reduce the time for roaming path planning of the mine digital twin system and solve the technical problem of long time consumption in existing roaming path planning methods.
[0044] Set the maximum number of iterations according to different training environments.
[0045] See Figure 1 In a digital twin system for mines, "state" refers to the points that need to be traversed during the roaming process; "action" refers to the optimal sequence of points that need to be traversed; and "value" refers to the reward corresponding to the optimal sequence and the degree of quality of the sequence.
[0046] Specifically, both RNN modules use the GRU module.
Claims
1. A method for roaming path planning in a mine digital twin system based on reinforcement learning, characterized in that, Includes a training phase and an inference phase; The training phase specifically includes the following steps: Step 1: Establish a roaming path map based on the roamable locations and path relationships in the mine digital twin system, as a training environment; Step 2: Specify the points that need to be traversed during the roaming of the mine digital twin system on the roaming path map; Step 3: Use the actor critic framework to build a deep reinforcement learning model; The deep reinforcement learning model includes an Actor module and a Critic module; The Actor module is a pointer network, which contains two RNN modules, which serve as the encoding module and the decoding module, respectively. The Critic module is a policy network; Step 4: Input the points to be traversed into the encoding module of the pointer network. The decoding module of the pointer network outputs the optimal sequence of the points to be traversed and uses the negative of the time required for the optimal sequence as the reward for the sequence. Step 5: Input the points to be traversed and the optimal sequence into the policy network, and output the quality of the sequence. Step six: Based on the reward of the sequence obtained in step four and the quality of the sequence obtained in step five, update the Actor module through backpropagation; Step 7: Increment the iteration count by 1, take the optimal sequence obtained in Step 4 as the point to be traversed, enter Step 4, until the maximum number of iterations is reached, and output the final Actor module. The reasoning phase includes the following: Input the points that need to be traversed during the roaming process of the mine digital twin system specified in step two into the final Actor module obtained in step seven. The Actor module provides the roaming path with the shortest current time based on the training environment.
2. The method for roaming path planning in a mine digital twin system based on reinforcement learning as described in claim 1, characterized in that, Both of the RNN modules described above use the GRU module.