A rule and data hybrid driven unmanned mine truck path planning system and method

Abstract

The application relates to the technical field of path planning, and provides a rule and data hybrid driving unmanned mine truck path planning system, which comprises the following steps: constructing a node guide search rule, adopting a fixed step and a uniformly dispersed sampling steering angle to perform node exploration on an arbitrary current subnode to obtain an exploration subnode set of a next subnode with the fixed step, and generating a guide node by deep reinforcement learning and adding the guide node to the exploration subnode set; the cost of each exploration subnode in the exploration subnode set is evaluated, the exploration subnode with the lowest cost is obtained as the next subnode selected finally in a subsequent iteration search process, and meanwhile, when the distance from the current subnode to the terminal point is smaller than a set threshold, a parking curve from the current subnode to the terminal point is generated, so that a final global curve is obtained. The above technical scheme constructs a rule-data hybrid driving path planning framework, and realizes the complementary advantages of rules and learning methods.

Description

Technical Field

[0001] This invention relates to the technical field of path planning, and in particular to a rule- and data-driven path planning system and method for unmanned mining trucks. Background Technology

[0002] The deployment of unmanned mining trucks can effectively reduce labor costs and improve the safety and transportation efficiency of mine production. Among these, the path planning module, which plans collision-free paths for the unmanned mining trucks based on environmental information, is a crucial component of the unmanned mining truck autonomous driving system. In existing unmanned mining truck transportation processes, the routes from the parking lot to various loading platforms, dumping areas, and unloading areas are generally fixed and the roads are relatively flat. These routes can be recorded, and automated transportation is achieved through line following. However, after reaching the loading platforms and dumping areas, the destination of the mining truck changes according to operational needs, rendering line following inapplicable. Therefore, a path planning module is needed to plan efficient and safe production routes.

[0003] The mining area is unstructured, with irregularly distributed hillsides and various types of debris and water-filled pits, resulting in blurred feasible region boundaries and reduced path search efficiency. Furthermore, mining trucks must maintain high-precision position and orientation during loading and unloading operations to avoid safety risks. Therefore, path planning requires additional consideration of endpoint orientation constraints, further increasing the complexity of path planning.

[0004] Existing path search methods can be mainly divided into two technical approaches: rule-based and learning-based. Rule-based methods rely on manually defined search rules. However, due to the complex and varied mining environment, with irregularly distributed obstacles like loose rocks, these rules are difficult to accurately adapt to every scenario, leading to insufficient search efficiency. Learning-based methods utilize deep learning networks for path planning. While they can achieve good results on specific datasets, they suffer from insufficient interpretability, lack clear physical meaning, and face serious stability and generalization problems, making them unsuitable for path planning of unmanned mining trucks with stringent safety requirements.

[0005] Therefore, the problem that this technology needs to solve is how to integrate the self-evolutionary features of learning-based parameters into a rule-based path planning framework, and use rule constraints to ensure the stability and generalization of the neural network, thereby improving search efficiency while ensuring the quality and stability of the generated path, and making it applicable to diverse mining operation scenarios. Summary of the Invention

[0006] To address the aforementioned problems, the present invention aims to provide a rule- and data-driven path planning system and method for unmanned mining trucks. It constructs a rule- and data-driven path planning framework, achieving complementary advantages between rule-based and learning methods. Specifically, by constructing a search guidance strategy based on deep reinforcement learning, search efficiency and path quality are improved, overcoming the difficulties in parameter tuning and maintenance of rule-based algorithms. Furthermore, by selecting the optimal node using a hybrid A* framework and establishing an optimal node-guided reward feedback mechanism, the deep reinforcement learning strategy is guided and constrained, compensating for the insufficient stability and generalization of learning-based algorithms.

[0007] The above-mentioned objective of this invention is achieved through the following technical solutions:

[0008] A rule- and data-driven path planning system for unmanned mining trucks, comprising:

[0009] The hybrid A-star planning module is used to construct node-guided search rules. For any current child node, a set of exploration child nodes with a fixed step size and a uniformly discrete sampled turning angle is obtained by exploring the next child node with a fixed step size. Guiding nodes are generated and added to the set of exploration child nodes through deep reinforcement learning. The cost of each exploration child node in the set of exploration child nodes is evaluated, and the exploration child node with the lowest cost is selected as the next child node in each subsequent iteration search. At the same time, when the distance from the current child node to the destination is less than a set threshold, the berth curve from the current child node to the destination is generated, thereby obtaining the final global curve.

[0010] The guiding node generation module is used to design the state space and action space of the current child node, and to design the neural network structure for generating the guiding node. In each iteration, the state space is input into the neural network structure, and the Q value of the turning angle and step size in the action space of each group is output. The turning angle and step size with the highest Q value are selected to generate the guiding node, and a reward function is designed to guide the training of the neural network structure.

[0011] The path exploration framework design module is used to design the path exploration framework and the training strategy for deep reinforcement learning.

[0012] Furthermore, in the hybrid A-star planning module, the node-guided search rule is constructed. For any current child node, a fixed step size and a uniformly discrete sampling turning angle are used to explore the next child node with a fixed step size, resulting in the set of explored child nodes. The guiding node is then generated through deep reinforcement learning and added to the set of explored child nodes. Specifically:

[0013] After obtaining the current child node, the steering angle is... Perform uniform discrete sampling, for The calculation method is as follows:

[0014]

[0015] in, The boundary of the steering angle corresponds to the maximum front wheel steering angle that the mine car can perform, i is the index of the sampled steering angle, and n is the number of sampled steering angles;

[0016] Considering both forward and backward exploration cases, a fixed step size will be used. and the sampled steering angle Substituting into the following formula (2), we obtain the set of explored child nodes for the next child node. ;

[0017]

[0018] Wherein, the current child node is , The global coordinates of the current child node on the map. Let be the heading angle of the mining truck at the current child node, and let be any one of the exploration child nodes in the set of exploration child nodes. , To explore the global coordinates of child nodes on the map, The heading angle of the mining truck at the exploring sub-node. For a fixed step size of exploration, This refers to the wheelbase of the mining truck. The direction of exploration for a node, i.e., backward or forward;

[0019] The environmental information and the state of the current child node are input into the deep reinforcement learning network, and the guided step size is output. and steering angle Substitute this into formula (2) to generate the guiding node. The guiding node has a larger step size and lower cost, and is added to the set of exploratory child nodes. .

[0020] Furthermore, in the hybrid A-star planning module, the cost of each exploration sub-node in the set of exploration sub-nodes is evaluated, and the exploration sub-node with the lowest cost is selected as the next sub-node in each subsequent iteration search process. Specifically:

[0021] First, the validity of each exploration sub-node in the set of exploration sub-nodes is evaluated, meaning that the minecart cannot collide with environmental obstacles when it is located at the exploration sub-node. The circular path from the current sub-node to each exploration sub-node is sampled, and a rectangle is used to simulate the minecart at the corresponding discrete point position and orientation. The minecart is then finely divided into grids, and the corresponding grid on the global map for each subdivided grid is calculated. The global map is then checked to see if the corresponding grid is occupied. If it is occupied, it means a collision has occurred, i.e., the exploration sub-node is invalid, and it is removed from the set of exploration sub-nodes. If no collision occurs between any discrete points on the circular path, then the explored child node is valid.

[0022] Based on this, a cost evaluation function is established. To evaluate each exploration child node Specifically:

[0023]

[0024] in, The current cost function represents the actual cost incurred from the starting point to this explored child node. Let be the heuristic cost function, representing the cost of exploring a child node to the destination. These are the weighting coefficients. To the current child node The cost is based on five indicators, namely distance cost. The cost of reversing Cost of switching between forward and backward motion Turnaround Costs and the cost of turning around , This is the predicted distance from the explored child node to the destination.

[0025] Furthermore, in the hybrid A-star planning module, when the distance from the current sub-node to the endpoint is less than the set threshold, the berth curve from the current sub-node to the endpoint is generated, thereby obtaining the final global curve, specifically as follows:

[0026] If the current child node To the finish line The distance is less than the set threshold Generate from the current child node using Reeds-Shepp curves. To the finish line Several candidate loading berth curves are obtained, and the node cost on the berth curve is calculated by formula (3). The cost is sorted according to the cost, the path with the lowest cost is selected and reverse search is performed to obtain the final global path.

[0027] If all candidate loading berth curves collide, the process continues to the node exploration step.

[0028] Furthermore, in the guide node generation module, the state space and action space of the current child node are designed, specifically as follows:

[0029] The state space Designed as follows:

[0030] (4)

[0031] in, For the current child node, For the starting node information, For endpoint node information, The position and heading angle of the current child node relative to the starting node. The position and heading angle of the endpoint node relative to the current child node. The distance between the current child node and the endpoint node. Let be the minimum distance between the obstacle and the minecart in the i-th region;

[0032] The exploration parameters of the guiding node include step size and turning angle, and the corresponding action space is: Deep reinforcement learning employs the DQN algorithm, which uses a discrete action space and performs discrete sampling of the steering angle and step size.

[0033] Furthermore, in the guide node generation module, the neural network structure for generating the guide node is designed and generated as follows:

[0034] The DQN algorithm is trained using two networks with identical structures but different parameters to estimate the network. Used to calculate strategy selection Value and perform Iterative update of values, target network Used to calculate the next state in the time difference target TDTarget The loss function of the DQN algorithm is designed as follows:

[0035]

[0036] Where N is the sample size. For the target Q value, through the target network The time difference objective is calculated to obtain, The state calculated for the neural network structure Take action below Q value;

[0037] The target network and the estimation network All three fully connected layers are constructed using a network with 32 neurons. The outputs of the first two fully connected layers are input into the activation function and then output to the next fully connected layer. The activation function is set using PReLu. The last fully connected layer directly outputs the Q value of each action, including the turning angle and step size. Finally, the turning angle and step size with the highest Q value are selected as the exploration parameters of the final guide node and substituted into formula (2) to generate the guide node.

[0038] Furthermore, in the guide node generation module, a reward function is designed to guide the training of the neural network structure, specifically as follows:

[0039] The reward function includes a destination-oriented reward. Obstacle avoidance rewards Exploration Cost Smoothness reward Node selection rewards ;

[0040] The destination reward The definition is as follows:

[0041]

[0042] in, For a relatively large fixed reward constant, These are manually adjusted weighting coefficients. The current child node in this iteration To the endpoint node European distance, The guiding node generated by the neural network structure To the endpoint node Euclidean distance;

[0043] The obstacle avoidance reward The definition is as follows:

[0044]

[0045] in, and For adjustable weighting coefficients, As the distance threshold, when This is considered a collision and returns a large penalty constant. ,when When this occurs, it is considered a dangerous situation, and a large penalty function is returned. When this condition is met, it is considered a risk and a smaller penalty function is returned. When this time is considered safe, no penalty will be returned. This represents the obstacle avoidance reward in region i, with a total of n regions, and the total obstacle avoidance reward. Satisfy the following formula:

[0046]

[0047] The cost of exploration The definition is as follows:

[0048]

[0049] in, It is an adjustable penalty constant;

[0050] The smoothness reward The definition is as follows:

[0051]

[0052] in, and For adjustable weighting coefficients, For the generated boot node The corresponding steering angle, The current child node in the search iteration The corresponding steering angle;

[0053] The node selects a reward. The definition is as follows:

[0054]

[0055] in, It is an adjustable fixed reward constant. For the set of exploratory child nodes in this round of iterative search The steering angle corresponding to the best node selected in the final selection. These are adjustable weighting coefficients;

[0056] The reward function is:

[0057] .

[0058] Furthermore, in the path exploration framework design module, the path exploration framework is designed as follows:

[0059] Obtain the global grid map and the pose information of the start and end points, initialize the open set and the closed set, set the start point as a node and set the cost to 0, add it to the open set, and then enter the iterative search phase;

[0060] In each round of iterative search, the node with the lowest cost is first selected from the open set as the current child node, and the current child node is removed from the open set and added to the closed set.

[0061] Determine whether the distance between the current child node and the endpoint is not greater than a threshold. If the conditions are met, attempt to directly generate the loading docking path using the Reeds-Shepp curve. If successful, add the docking path to the closed set and end the iterative search. If unsuccessful, or if the distance to the destination is greater than a threshold... If the current child node is used for exploration, the next child node and the guiding node with a fixed step size are generated by the node-guided search rule and the deep reinforcement learning network, respectively, and added to the set of explored child nodes. Then, collision detection is performed on the explored child nodes in the set of explored child nodes to remove invalid child nodes and the cost of the child nodes is updated by the evaluation function. Finally, the child nodes are added to the open set to end the current round of iterative search.

[0062] After successful path planning and the end of the iterative search, all nodes of the planned path can be obtained through node backtracking.

[0063] Furthermore, in the path exploration framework design module, a deep reinforcement learning training strategy is designed, specifically as follows:

[0064] For training deep reinforcement learning, a suitable starting point and ending point are randomly selected on the map based on actual production data, and path planning is performed.

[0065] In the planning process, path planning parameters are optimized through reinforcement learning, thereby generating multiple sets of state transition sampling data, which are then added to the replay buffer.

[0066] During training, batches of data are randomly selected from the replay buffer, and the estimation network is updated according to the loss function. The parameters of the estimation network are determined after a certain number of iterations. The parameters are copied to the target network. Complete a learning process.

[0067] A rule- and data-driven unmanned mining truck route planning method, implemented using the rule- and data-driven hybrid approach described above, includes the following steps:

[0068] S1: Construct node-guided search rules. For any current child node, explore the node by using a fixed step size and a turning angle after uniform discrete sampling to obtain the exploration child node set of the next child node with a fixed step size. Then, generate a guiding node through deep reinforcement learning and add it to the exploration child node set.

[0069] S2: Evaluate the cost of each exploration sub-node in the set of exploration sub-nodes, and obtain the exploration sub-node with the lowest cost as the next sub-node to be selected in each round of iterative search.

[0070] S3: When the distance from the current child node to the endpoint is less than a set threshold, generate the berth curve from the current child node to the endpoint, thereby obtaining the final global curve.

[0071] Compared with the prior art, the beneficial effects of the present invention include:

[0072] (1) Improve the timeliness of route planning

[0073] The irregular distribution of gravel and mountain terrain in mining environments necessitates different exploration strategies based on environmental information. However, existing planning methods rely on singular exploration strategies and fixed parameters, leading to decreased search efficiency and lower path quality. This invention constructs a rule- and data-driven hybrid path planning framework and establishes a node exploration method based on a hybrid A*-deep reinforcement learning fusion. This effectively adjusts the exploration strategy according to different environments, thereby accelerating path search efficiency. Furthermore, because the generated nodes have lower costs, the generated paths are of higher quality.

[0074] (2) Reduced reliance on fixed exploration rules

[0075] Existing rule-based path planning methods address different operational scenarios by establishing various rules. However, simple rule establishment limits applicability, while complex rule design introduces numerous parameter debugging issues, making the rules difficult to maintain. This invention, based on building fundamental iterative exploration rules, generates adaptive guiding nodes through a deep reinforcement learning network, thereby accelerating exploration efficiency. Furthermore, this invention considers obstacle information in the state space of deep reinforcement learning, effectively handling complex and ever-changing mining scenarios. It eliminates the need for extensive manual debugging of exploration parameters, reducing reliance on fixed exploration rules.

[0076] (3) Provide guidance for policy convergence in deep reinforcement learning

[0077] Existing learning-based planning algorithms often directly establish a fitting relationship between the environment and the path, lacking clear physical meaning and guidance during training, resulting in generated paths that do not conform to the movement characteristics of mining trucks. This invention constructs a node selection reward feedback on the basis of a rule-based path planning framework, rewarding the exploration of rule-compliant paths during training. This can effectively guide the convergence of deep reinforcement learning policies, enabling them to generate nodes that are more rule-compliant and have lower costs.

[0078] (4) It ensures the stability and generalization of path planning.

[0079] The mining environment is complex and ever-changing, and existing planning algorithms all face challenges in stability and generalization. Rule-based algorithms are difficult to construct rules and have high parameter tuning costs, while learning-based algorithms suffer from unstable output and insufficient generalization, both posing difficulties for practical application. In contrast, this invention retains the rule-based exploration of hybrid A* systems during the exploration process and accelerates the node exploration process by generating guiding nodes based on adaptive exploration parameters through deep reinforcement learning. Furthermore, even if the guiding nodes fail, paths can still be generated through rule-based exploration, providing higher redundancy and ensuring the stability of the algorithm's output and its generalization ability across different scenarios.

[0080] (5) Experimental verification

[0081] On the grid map of the actual mining loading platform ( The algorithm was tested based on the starting and ending points in actual production data and compared with the existing Hybrid A-Star algorithm used for unmanned mining truck path planning. The solution time of this invention is 0.68 seconds, which is 80% less than that of Hybrid A-Star, and the path length is 579m, which is 4% less than that of Hybrid A-Star, demonstrating significant advantages.

[0082] The specific data is as follows:

[0083] On the grid map of the actual mining loading platform ( On the above, the solution time of this invention is 0.68 seconds and the path length is 579m. The solution time is reduced by 80% compared to Hybrid A-Star and the path length is reduced by 5% compared to Hybrid A-Star; the solution time is reduced by 60% compared to DQN (pure data-driven) and the path length is reduced by 2%, which is significantly better than other existing path planning methods.

[0084] In summary, this invention demonstrates through theoretical analysis and experimental data that the rule-data hybrid-driven path planning method can better leverage the complementary advantages of rule-based and data-driven methods, significantly improving search efficiency and path quality while ensuring stability and generalization. It is more suitable for the changing mining environment and has significant practical application value. Attached Figure Description

[0085] Figure 1This is a diagram illustrating the overall structure of the rule- and data-driven unmanned mining truck path planning system of the present invention.

[0086] Figure 2 This is a schematic diagram of the obstacle area division in this invention;

[0087] Figure 3 This is a flowchart of the rule-data hybrid driven unmanned mining truck path planning framework and training framework of the present invention;

[0088] Figure 4 This is a schematic diagram of the node-guided exploration rule process of the present invention;

[0089] Figure 5 This is an overall flowchart of the rule- and data-driven unmanned mining truck path planning method of the present invention. Detailed Implementation

[0090] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0091] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0092] This invention proposes a rule-data hybrid-driven path planning method. The method first establishes a node-guided hybrid A* planning module, then builds a deep reinforcement learning-based guiding node generation module on top of this. Finally, it establishes a rule-data hybrid-driven path exploration framework and training strategy, as detailed below:

[0093] 1. Node-guided hybrid A-satellite planning module

[0094] 1.1 Node-Guided Exploration Rules

[0095] First, a node-guided path exploration rule is constructed, and the exploration rule is parameterized and modeled to generate candidate child nodes for subsequent iterative search.

[0096] 1.2 Node Evaluation Method

[0097] For the child nodes generated in 1.1, the validity of the child nodes is determined based on collision detection, and a node evaluation method is constructed to evaluate the node cost.

[0098] 1.3 Parking Route Generation Method Based on Reeds-Shepp Curves

[0099] Since the loading operation path has strict constraints on the final pose, several candidate berthing paths are generated using Reeds-Shepp curves, and then sorted and filtered using an evaluation function to select the optimal path, thereby accelerating the search.

[0100] 2. Guiding node generation module based on deep reinforcement learning

[0101] 2.1 State Space Design

[0102] The relative positions of the start and end points and the distribution characteristics of obstacles are considered in the state space, and parameter modeling is performed to represent the environmental information of the agent and input it into the neural network.

[0103] 2.2 Neural Network Structure Design

[0104] Based on action modeling in 1.1 and state space modeling in 2.1, a corresponding neural network structure is constructed to realize the mapping from state to action.

[0105] 2.3 Reward Function Design

[0106] A reward function is constructed based on safety and efficiency, and an optimal node-guided reward feedback mechanism is established to guide and constrain the agent's policy training through rule-based path exploration.

[0107] 3. Rule-Data Hybrid Driven Path Exploration Framework and Training Strategy

[0108] 3.1 Path Exploration Framework Design

[0109] A rule- and data-driven path exploration framework is established, in which guide nodes are generated through a DQN network to accelerate path search efficiency.

[0110] 3.2 Training Strategy Design

[0111] Building upon 3.1, in offline training, the results of node selection during each search iteration are mapped to the reward function to guide the training of the DQN network.

[0112] The following is an illustration through specific examples:

[0113] First Embodiment

[0114] like Figure 1As shown, this embodiment provides a rule- and data-driven unmanned mining truck path planning system, including:

[0115] Hybrid A-Star planning module 1 is used to construct node-guided search rules. For any current child node, a set of exploration child nodes with a fixed step size and a uniformly discrete sampling turning angle is obtained by exploring the next child node with a fixed step size. Guiding nodes are generated and added to the set of exploration child nodes through deep reinforcement learning. The cost of each exploration child node in the set of exploration child nodes is evaluated and used to obtain the exploration child node with the lowest cost in each subsequent iteration search process as the final selected next child node. At the same time, when the distance from the current child node to the destination is less than a set threshold, the berth curve from the current child node to the destination is generated to obtain the final global curve.

[0116] The guiding node generation module 2 is used to design the state space and action space of the current child node, and to design the neural network structure for generating the guiding node. In each iteration, the state space is input into the neural network structure, and the Q value of the turning angle and step size in the action space of each group is output. The turning angle and step size with the highest Q value are selected to generate the guiding node, and a reward function is designed to guide the training of the neural network structure.

[0117] Module 3, Path Exploration Framework Design, is used to design a path exploration framework and a deep reinforcement learning training strategy.

[0118] The following is a description of each module:

[0119] 1. Node-guided hybrid A-satellite planning module

[0120] Node-guided exploration rules

[0121] Existing hybrid AlphaGo systems often employ fixed step sizes and turning angles for searching. However, in complex and variable environments, fixed parameters cannot guarantee that the cost of child nodes generated in each iteration will be minimized. Furthermore, directly replacing iterative search with learning-based methods can easily lead to path planning failures due to the instability of network outputs. Therefore, this paper proposes a node-guided search rule. It uses fixed step sizes and turning angles to explore nodes and obtain fixed child nodes, and then generates guiding nodes through deep reinforcement learning. These guiding nodes typically have longer step sizes and lower costs, thereby reducing the number of iterations, accelerating path planning, and improving path quality.

[0122] Specifically, since the minecart travels at low speed, the motion characteristics of the minecart can be characterized by a bicycle model in Hybrid A-Star, a clear boundary limit can be established for the steering angle, and a circular arc trajectory can be used for node exploration.

[0123] After obtaining the current child node, the steering angle is... Perform uniform discrete sampling, for The calculation method is as follows:

[0124]

[0125] in, The boundary of the steering angle corresponds to the maximum front wheel steering angle that the mine car can perform, i is the index of the sampled steering angle, and n is the number of sampled steering angles;

[0126] Considering both forward and backward exploration cases, a fixed step size will be used. and the sampled steering angle Substituting into the following formula (2), we obtain the set of explored child nodes for the next child node. The exploration child nodes in this set are generated by exploration rules, and the step size is relatively fixed.

[0127]

[0128] Wherein, the current child node is , The global coordinates of the current child node on the map. Let be the heading angle of the mining truck at the current child node, and let be any one of the exploration child nodes in the set of exploration child nodes. , To explore the global coordinates of child nodes on the map, The heading angle of the mining truck at the exploring sub-node. For a fixed step size of exploration, This refers to the wheelbase of the mining truck. The direction of exploration for a node, i.e., backward or forward;

[0129] The environmental information and the state of the current child node are input into the deep reinforcement learning network, and the guided step size is output. and steering angle Substitute this into formula (2) to generate the guiding node. The guiding node has a larger step size and lower cost, and is added to the set of exploratory child nodes. .

[0130] (2) Node evaluation method

[0131] In obtaining the set of child nodes to explore Next, the cost of child nodes needs to be evaluated to select the child node with the lowest cost as the current node in each subsequent iteration. Specifically, firstly, the validity of each exploration child node in the set of exploration child nodes is evaluated, meaning that the minecart cannot collide with environmental obstacles when it is located at the exploration child node. The circular path from the current child node to each exploration child node is sampled, and the minecart is simulated with rectangles at the corresponding discrete point positions and orientations. The minecart is then finely divided into grids, and the corresponding grid on the global map for each subdivided grid is calculated. The corresponding grid on the global map is then checked to see if it is occupied. If it is occupied, it means that a collision has occurred, i.e., the exploration child node is invalid, and it is removed from the set of exploration child nodes. If no collision occurs between any discrete points on the circular path, then the explored child node is valid.

[0132] Based on this, a cost evaluation function is established. To evaluate each exploration child node Specifically:

[0133]

[0134] in, The current cost function represents the actual cost incurred from the starting point to this explored child node. Let be the heuristic cost function, representing the cost of exploring a child node to the destination. These are the weighting coefficients. To the current child node The cost is based on five indicators, namely distance cost. The cost of reversing Cost of switching between forward and backward motion Turnaround Costs and the cost of turning around , This is the predicted distance from the explored child node to the destination.

[0135] Specifically, distance cost This represents the distance from the current node to its child nodes. (Reversing cost) For a fixed value, if the exploration direction from the current node to this child node is backward ( If it is a forward exploration, then it is accumulated; Then it is considered to be 0. Cost of switching between forward and backward motion. For fixed values, the current node's exploration direction and the child node's exploration direction ( If the exploration direction is the same (), then a fixed value is accumulated; If the cost is 0, then it is considered 0. The steering angle of the child node The absolute value is directly proportional, that is, The cost of shifting course The exploration turning angle of the current node is a fixed value. and the exploration turning angle of child nodes If they are different, then accumulate a fixed value; otherwise, treat it as 0. Heuristic function. This is the predicted distance from the current node to the destination. Due to obstacles, it cannot be directly predicted using Euclidean distance; instead, it uses... The method plans to calculate the distance between the current node's grid and the destination's grid.

[0136] (3) Parking route generation method based on Reeds-Shepp curve

[0137] If the current child node To the finish line The distance is less than the set threshold Generate from the current child node using Reeds-Shepp curves. To the finish line Several candidate loading berth curves are obtained, and the node cost on the berth curve is calculated by formula (3). The cost is sorted according to the cost, the path with the lowest cost is selected and reverse search is performed to obtain the final global path.

[0138] If all candidate loading berth curves collide, the process continues to the node exploration step.

[0139] 2. Guiding Node Generation Module Based on Deep Reinforcement Learning

[0140] (1) Design of state space and action space

[0141] Deep reinforcement learning determines the optimal action based on the input to the state space. Therefore, to enable deep reinforcement learning to generalize more effectively, the state space needs to be designed, fully considering the impact of environmental obstacles and the relative position of the endpoint on the planning process. Specifically, the state space... Designed as follows:

[0142] (4)

[0143] in, For the current child node, For the starting node information, For endpoint node information, The position and heading angle of the current child node relative to the starting node. The position and heading angle of the endpoint node relative to the current child node. The distance between the current child node and the endpoint node. Let be the minimum distance between the obstacle and the minecart in the i-th region.

[0144] For example, in this embodiment, such as Figure 2 As shown, to ensure the safe operation of mining trucks, it is essential to monitor the distance between the trucks and surrounding obstacles. Specifically, this paper divides the space around the mining trucks into eight zones based on angles. .use This represents the minimum distance between the obstacle and the minecart in the i-th region.

[0145] The output of deep reinforcement learning is the agent's action, which in this invention is embodied in the exploration parameters of the guiding node. The exploration parameters of the guiding node include step size and turning angle, and the corresponding action space is... Deep reinforcement learning employs the DQN algorithm, which uses a discrete action space and performs discrete sampling of the steering angle and step size. For example, in this embodiment, the sampling of the steering angle and step size is as follows:

[0146] , There are a total of 105 combinations.

[0147] (2) Neural network structure design

[0148] The DQN algorithm is trained using two networks with identical structures but different parameters to estimate the network... Used to calculate strategy selection Value and perform Iterative update of values, target network Used to calculate the next state in the time difference target TDTarget The loss function of the DQN algorithm is designed as follows:

[0149]

[0150] Where N is the sample size. For the target Q value, through the target network The time difference objective is obtained by calculation. The state calculated for the neural network structure Take action below Q value;

[0151] The target network and the estimation network All three fully connected layers are constructed using a network with 32 neurons. The outputs of the first two fully connected layers are input into the activation function and then output to the next fully connected layer. The activation function is set using PReLu. The last fully connected layer directly outputs the Q value of each action, including the turning angle and step size. Finally, the turning angle and step size with the highest Q value are selected as the exploration parameters of the final guide node and substituted into formula (2) to generate the guide node.

[0152] Here, Q-value is the output of the DQN network, related to both state and action, and used to evaluate the quality of a set of action states. Simply put, a deep reinforcement learning network evaluates each action based on the current state. This evaluation is equivalent to assessing the merits of taking that action in the current state, i.e., whether it's more likely to yield a greater reward. A high Q-value indicates that taking that action in the current state has a higher probability of obtaining a better reward, or is less likely to result in negative consequences such as penalties. This invention is based on the current state space... The steering angle and step size corresponding to the guiding node are determined. In order to better guide the search and improve the search efficiency, each action in the discrete action space is evaluated and the corresponding Q value is obtained. The steering angle and step size corresponding to the maximum Q value are selected and output.

[0153] (3) Reward function design

[0154] The path planning framework of this invention is an iterative search; therefore, in each iteration, the intelligent agent will... The information is input into the deep reinforcement learning network and outputs the action space. The Q value of each set of steering angle and step size actions is calculated, and the steering angle and step size with the highest Q value are selected for output, thereby generating the guide node according to formula (2). To effectively guide the training of an agent's policy, it is necessary to evaluate the merits of an action through environmental feedback after the agent completes it, and design corresponding rewards to guide the training of the deep reinforcement learning network. During training, the agent will generate guiding nodes in each iteration of the search process. The network receives corresponding rewards for its performance, thereby guiding the training of the network.

[0155] Specifically, when designing the reward function, it is necessary to ensure that the minecart reaches the target point as quickly as possible while maintaining a safe distance from obstacles. Therefore, the reward function designed in this invention includes a destination-oriented reward. Obstacle avoidance rewards Exploration Cost Smoothness reward Node selection rewards ;

[0156] It is a destination-oriented reward system. The purpose is to guide the minecart towards its destination. Therefore, a positive reward is given when the minecart approaches the destination, and a penalty is incurred when it moves away. Furthermore, if the minecart successfully connects directly to the destination via the Reeds-Shepp curve in the current search iteration, it is considered to have reached the destination and is given a fixed reward for reaching it. This destination-oriented reward... The definition is as follows:

[0157]

[0158] in, For a relatively large fixed reward constant, These are manually adjusted weighting coefficients. The current child node in this iteration To the endpoint node European distance, The guiding node generated by the neural network structure To the endpoint node Euclidean distance.

[0159] Triggering the Reeds-Shepp curve signifies successful path generation, thus yielding a substantial reward. Conversely, a failed Reeds-Shepp curve indicates that further exploration is required. During node exploration, the desired bootstrap node is... To get as close to the destination as possible, we include calculating the distance from the current node to the destination. Distance from the guide node to the destination The difference is used as a reward. If the guiding node If you are far from the finish line, the reward is negative; otherwise, the reward is positive.

[0160] It's an obstacle avoidance reward, set up The purpose is to avoid collisions between the minecart and surrounding obstacles due to insufficient distance. When designing the obstacle avoidance reward function, the distance between the obstacle and the minecart is considered. The safety status of mining trucks is divided into four categories: collision, danger, risk, and safe. The obstacle avoidance reward... The definition is as follows:

[0161]

[0162] in, and For adjustable weighting coefficients, As the distance threshold, when This is considered a collision and returns a large penalty constant. ,when When this occurs, it is considered a dangerous situation, and a large penalty function is returned. When this condition is met, it is considered a risk and a smaller penalty function is returned. When this time is considered safe, no penalty will be returned. This represents the obstacle avoidance reward in region i, with a total of n regions, and the total obstacle avoidance reward. Satisfy the following formula:

[0163]

[0164] To guide the minecart closer to its destination more quickly and prevent pointless exploration, this section sets a fixed exploration cost for each step. This cost is set as a negative number. The definition is as follows:

[0165]

[0166] in, It is an adjustable penalty constant;

[0167] Since turning the minecart incurs additional travel costs, this section includes a smoothness bonus. To guide the agent to reduce steering wheel rotation, the smoothness reward... The definition is as follows:

[0168]

[0169] in, and For adjustable weighting coefficients, For the generated boot node The corresponding steering angle, The current child node in the search iteration The corresponding steering angle;

[0170] Furthermore, to ensure that deep reinforcement learning generates the optimal turning angle—that is, the turning angle with the lowest cost and which is preferentially selected as the current node for the next round of search—this paper introduces a node selection reward. The training of deep reinforcement learning strategies is guided by a hybrid A* approach that selects whether to choose a mentor node. Specifically, a reward is given when the node generated by reinforcement learning is the best node in this expansion; otherwise, a penalty is imposed. The node selection reward... The definition is as follows:

[0171]

[0172] in, It is an adjustable fixed reward constant. For the set of exploratory child nodes in this round of iterative search The steering angle corresponding to the best node selected in the final selection. These are adjustable weighting coefficients.

[0173] Specifically, hybrid A-stars are evaluated using an evaluation function. The quality of each node is evaluated, and the optimal node with the minimum cost is selected through simulation. If... The node with the lowest cost is more likely to be chosen as the current node in each iteration of exploration (i.e., the point on the final path), and therefore will be given a fixed reward directly. Conversely, if exist If the node is not the node with the lowest cost, it means that it is more difficult to choose this node as a path point in actual exploration. Therefore, it is necessary to guide the deep learning network to select and generate a better guiding point, hence the calculation of the turning angle. With the selected node (i.e. The steering angle corresponding to the node with the minimum actual cost. The difference is used as a penalty.

[0174] In summary, the reward function is:

[0175] .

[0176] 3. Rule-data hybrid driven path exploration framework and training strategy path exploration framework design module 1

[0177] (1) Path exploration framework design

[0178] Path exploration framework such as Figure 3 As shown, the global grid map and the pose information of the start and end points are obtained, and the open set and closed set are initialized. The start point is set as a node and the cost is set to 0. It is then added to the open set, and then the iterative search phase is entered.

[0179] In each round of iterative search, the node with the lowest cost is first selected from the open set as the current child node, and the current child node is removed from the open set and added to the closed set.

[0180] Determine whether the distance between the current child node and the endpoint is not greater than a threshold. If the conditions are met, attempt to directly generate the loading docking path using the Reeds-Shepp curve. If successful, add the docking path to the closed set and end the iterative search. If unsuccessful, or if the distance to the destination is greater than a threshold... Then, the exploration is carried out through the current child node, first using, for example... Figure 4As shown, the node-guided search rule and the deep reinforcement learning network respectively generate the next child node and the guiding node with a fixed step size, and add them to the set of exploration child nodes. Then, collision detection is performed on the exploration child nodes in the set of exploration child nodes to remove invalid child nodes and the cost of child nodes is updated through the evaluation function. Finally, they are added to the open set to end the current round of iterative search. After the path planning is successful and the iterative search ends, all nodes of the planned path can be obtained by node backtracking.

[0181] (2) Training strategy design

[0182] For training deep reinforcement learning, a suitable starting point and ending point are randomly selected on the map based on actual production data, and path planning is performed.

[0183] In the planning process, path planning parameters are optimized through reinforcement learning, thereby generating multiple sets of state transition sampling data, which are then added to the replay buffer.

[0184] During training, batches of data are randomly selected from the replay buffer, and the estimation network is updated according to the loss function. The parameters of the estimation network are determined after a certain number of iterations. The parameters are copied to the target network. Complete a learning process.

[0185] Using two networks for training reduces the correlation between the current Q-value and the target Q-value to some extent, improving the algorithm's stability. During training, to improve generalization, the starting and ending points can be randomly fine-tuned, thus enriching the training data and further enhancing generalization. The pseudocode for the DQN algorithm is shown in Table 1 below.

[0186] Table 1 DQN Algorithm

[0187] 1. Initialize the prediction network with random network parameters.

[0188] 2. Use the same parameters to initialize the target network.

[0189] 3. Initialize the experience replay pool

[0190] 4. for sequence do

[0191] 5. Obtain the initial state of the environment

[0192] 6. for time step do

[0193] 7. Based on the current network Choose actions using a greedy strategy

[0194] 8. Perform actions and receive rewards. The environmental state becomes

[0195] 9. [The following is a list of items / items] Stored in the playback pool

[0196] 10. If there is sufficient data, sample data from it in batches.

[0197] 11. For each data point, compute using the target network.

[0198] 12. Update the network parameters to make as close as possible

[0199] 13. Every C updates make

[0200] This section describes the specific training algorithm for DQN, because the DQN algorithm ultimately trains and fits a dataset to obtain a complete set of data. The function, whose output is the Q-value, is an evaluation of the merits of taking action a in the current state s (i.e., whether it is possible to obtain more rewards and reduce penalties). In this invention, the actions are turning angle and step size. Therefore, in the specific implementation process, a corresponding Q-value is generated for each permutation and combination of turning angle and step size based on the current state s. The set of turning angles and step sizes with the highest Q-value is selected and used as the turning angle and step size output by the deep reinforcement learning network to generate the guiding node. .

[0201] The algorithm here is the training algorithm for the DQN network. For specific usage of DQN, please refer to [link / reference needed]. Figure 3 and Figure 4 Simply put, the state here... Includes state space This is a combination of the steering angle and the step size, i.e. The reward is in the corresponding state space. The following actions are adopted. The reward feedback in this training algorithm first uses a greedy strategy to output actions and then trains reverse path planning. Based on the results of each round of search iteration during training, the algorithm then... The data is stored in the experience replay pool and subjected to batch sampling training to update the network parameters and learn the optimal strategy.

[0202] Second Embodiment

[0203] like Figure 5As shown, this embodiment provides a rule- and data-driven unmanned mining truck route planning method executed by the rule- and data-driven unmanned mining truck route planning system as described in the first embodiment, including the following steps:

[0204] S1: Construct node-guided search rules. For any current child node, explore the node by using a fixed step size and a turning angle after uniform discrete sampling to obtain the exploration child node set of the next child node with a fixed step size. Then, generate a guiding node through deep reinforcement learning and add it to the exploration child node set.

[0205] S2: Evaluate the cost of each exploration sub-node in the set of exploration sub-nodes, and obtain the exploration sub-node with the lowest cost as the next sub-node to be selected in each round of iterative search.

[0206] S3: When the distance from the current child node to the endpoint is less than a set threshold, generate the berth curve from the current child node to the endpoint, thereby obtaining the final global curve.

[0207] A computer-readable storage medium stores computer code that, when executed, performs the methods described above. Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. This program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0208] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

[0209] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0210] It should be noted that the above embodiments can be freely combined as needed. The above description is only a preferred embodiment of the present invention. It should be pointed out that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A rule- and data-driven path planning system for unmanned mining trucks, characterized in that, include: The hybrid A-star planning module is used to construct node-guided search rules. For any current child node, a set of exploration child nodes with a fixed step size and a uniformly discrete sampled turning angle is obtained by exploring the next child node with a fixed step size. Guiding nodes are generated and added to the set of exploration child nodes through deep reinforcement learning. The cost of each exploration child node in the set of exploration child nodes is evaluated, and the exploration child node with the lowest cost is selected as the next child node in each subsequent iteration search. At the same time, when the distance from the current child node to the destination is less than a set threshold, the berth curve from the current child node to the destination is generated, thereby obtaining the final global curve. The guiding node generation module is used to design the state space and action space of the current child node, and to design the neural network structure for generating the guiding node. In each iteration, the state space is input into the neural network structure, and the Q value of the turning angle and step size in the action space of each group is output. The turning angle and step size with the highest Q value are selected to generate the guiding node, and a reward function is designed to guide the training of the neural network structure. The path exploration framework design module is used to design the path exploration framework and the training strategy for deep reinforcement learning. In the hybrid A-star planning module, the node-guided search rule is constructed. For any current child node, a fixed step size and a uniformly discrete sampling turning angle are used to explore the next child node with a fixed step size, resulting in the set of explored child nodes. The guiding node is then generated through deep reinforcement learning and added to the set of explored child nodes. Specifically: After obtaining the current child node, the steering angle is... Perform uniform discrete sampling, for The calculation method is as follows: in, The boundary of the steering angle corresponds to the maximum front wheel steering angle that the mine car can perform, i is the index of the sampled steering angle, and n is the number of sampled steering angles; Considering both forward and backward exploration cases, a fixed step size will be used. and the sampled steering angle Substituting into the following formula (2), we obtain the set of explored child nodes for the next child node. ; Wherein, the current child node is , The global coordinates of the current child node on the map. Let be the heading angle of the mining truck at the current child node, and let be any one of the exploration child nodes in the set of exploration child nodes. , To explore the global coordinates of child nodes on the map, The heading angle of the mining truck at the exploring sub-node. For a fixed step size of exploration, This refers to the wheelbase of the mining truck. The direction of exploration for a node, i.e., backward or forward; The environmental information and the state of the current child node are input into the deep reinforcement learning network, and the guided step size is output. and steering angle Substitute this into formula (2) to generate the guiding node. The guiding node has a larger step size and lower cost, and is added to the set of exploratory child nodes. ; In the guide node generation module, the state space and action space of the current child node are designed, specifically as follows: The state space Designed as follows: in, For the current child node, For the starting node information, For endpoint node information, The position and heading angle of the current child node relative to the starting node. The position and heading angle of the endpoint node relative to the current child node. The distance between the current child node and the endpoint node. Let be the minimum distance between the obstacle and the minecart in the i-th region; The exploration parameters of the guiding node include step size and turning angle, and the corresponding action space is: Deep reinforcement learning employs the DQN algorithm, which uses a discrete action space and performs discrete sampling of the steering angle and step size.

2. The rule- and data-driven unmanned mining truck path planning system according to claim 1, characterized in that, In the hybrid A-star planning module, the cost of each exploration sub-node in the set of exploration sub-nodes is evaluated. This evaluation is used to select the exploration sub-node with the lowest cost in each subsequent iteration search as the next selected sub-node. Specifically: First, the validity of each exploration sub-node in the set of exploration sub-nodes is evaluated, meaning that the minecart cannot collide with environmental obstacles when it is located at the exploration sub-node. The circular path from the current sub-node to each exploration sub-node is sampled, and a rectangle is used to simulate the minecart at the corresponding discrete point position and orientation. The minecart is then finely divided into grids, and the corresponding grid on the global map for each subdivided grid is calculated. The global map is then checked to see if the corresponding grid is occupied. If it is occupied, it means a collision has occurred, i.e., the exploration sub-node is invalid, and it is removed from the set of exploration sub-nodes. If no collision occurs between any discrete points on the circular path, then the explored child node is valid. Based on this, a cost evaluation function is established. To evaluate each exploration child node Specifically: in, The current cost function represents the actual cost incurred from the starting point to this explored child node. Let be the heuristic cost function, representing the cost of exploring a child node to the destination. These are the weighting coefficients. To the current child node The cost is based on five indicators, namely distance cost. The cost of reversing Cost of switching between forward and backward motion Turnaround Costs and the cost of turning around , This is the predicted distance from the explored child node to the destination.

3. The rule- and data-driven unmanned mining truck path planning system according to claim 2, characterized in that, In the hybrid A-star planning module, when the distance from the current sub-node to the endpoint is less than the set threshold, the berth curve from the current sub-node to the endpoint is generated, thereby obtaining the final global curve, specifically as follows: If the current child node To the finish line The distance is less than the set threshold Generate from the current child node using Reeds-Shepp curves. To the finish line Several candidate loading berth curves are obtained, and the node cost on the berth curve is calculated by formula (3). The cost is sorted according to the cost, the path with the lowest cost is selected and reverse search is performed to obtain the final global path. If all candidate loading berth curves collide, the process continues to the node exploration step.

4. The rule- and data-driven unmanned mining truck path planning system according to claim 3, characterized in that, In the guide node generation module, the neural network structure for generating the guide node is designed and generated as follows: The DQN algorithm is trained using two networks with identical structures but different parameters to estimate the network... Used to calculate strategy selection Value and perform Iterative update of values, target network Used to calculate the next state in the time difference target (TD Target) The loss function of the DQN algorithm is designed as follows: Where N is the sample size. For the target Q value, through the target network The time difference objective is calculated to obtain, The state calculated for the neural network structure Take action below Q value; The target network and the estimation network All three fully connected layers are constructed using a network with 32 neurons. The outputs of the first two fully connected layers are input into the activation function and then output to the next fully connected layer. The activation function is set using PReLu. The last fully connected layer directly outputs the Q value of each action, including the turning angle and step size. Finally, the turning angle and step size with the highest Q value are selected as the exploration parameters of the final guide node and substituted into formula (2) to generate the guide node.

5. The rule- and data-driven unmanned mining truck path planning system according to claim 4, characterized in that, In the guide node generation module, a reward function is designed to guide the training of the neural network structure, specifically as follows: The reward function includes a destination-oriented reward. Obstacle avoidance rewards Exploration Cost Smoothness reward Node selection rewards ; The destination reward The definition is as follows: in, For a relatively large fixed reward constant, These are manually adjusted weighting coefficients. The current child node in this iteration To the endpoint node European distance, The guiding node generated by the neural network structure To the endpoint node Euclidean distance; The obstacle avoidance reward The definition is as follows: in, and For adjustable weighting coefficients, As the distance threshold, when This is considered a collision and returns a large penalty constant. ,when When this occurs, it is considered a dangerous situation, and a large penalty function is returned. When this condition is met, it is considered a risk and a smaller penalty function is returned. When this time is considered safe, no penalty will be returned. This represents the obstacle avoidance reward in region i, with a total of n regions, and the total obstacle avoidance reward. Satisfy the following formula: The cost of exploration The definition is as follows: in, It is an adjustable penalty constant; The smoothness reward The definition is as follows: in, and For adjustable weighting coefficients, For the generated boot node The corresponding steering angle, The current child node in the search iteration The corresponding steering angle; The node selects a reward. The definition is as follows: in, It is an adjustable fixed reward constant. For the set of exploratory child nodes in this round of iterative search The steering angle corresponding to the best node selected in the final selection. These are adjustable weighting coefficients; The reward function is: 。 6. The rule- and data-driven unmanned mining truck path planning system according to claim 5, characterized in that, In the path exploration framework design module, the path exploration framework is designed as follows: Obtain the global grid map and the pose information of the start and end points, initialize the open set and the closed set, set the start point as a node and set the cost to 0, add it to the open set, and then enter the iterative search phase; In each round of iterative search, the node with the lowest cost is first selected from the open set as the current child node, and the current child node is removed from the open set and added to the closed set. Determine whether the distance between the current child node and the endpoint is not greater than a threshold. If the conditions are met, attempt to directly generate the loading docking path using the Reeds-Shepp curve. If successful, add the docking path to the closed set and end the iterative search. If unsuccessful, or if the distance to the destination is greater than a threshold... If the current child node is used for exploration, the next child node and the guiding node with a fixed step size are generated by the node-guided search rule and the deep reinforcement learning network, respectively, and added to the set of explored child nodes. Then, collision detection is performed on the explored child nodes in the set of explored child nodes to remove invalid child nodes and the cost of the child nodes is updated by the evaluation function. Finally, the child nodes are added to the open set to end the current round of iterative search. After successful path planning and the end of the iterative search, all nodes of the planned path can be obtained through node backtracking.

7. The rule- and data-driven unmanned mining truck path planning system according to claim 6, characterized in that, In the path exploration framework design module, a deep reinforcement learning training strategy is designed, specifically as follows: For training deep reinforcement learning, a suitable starting point and ending point are randomly selected on the map based on actual production data, and path planning is performed. In the planning process, path planning parameters are optimized through reinforcement learning, thereby generating multiple sets of state transition sampling data, which are then added to the replay buffer. During training, batches of data are randomly selected from the replay buffer, and the estimation network is updated according to the loss function. The parameters of the estimation network are determined after a certain number of iterations. The parameters are copied to the target network. Complete a learning process.

8. A rule- and data-driven unmanned mining truck route planning method executed by the rule- and data-driven unmanned mining truck route planning system as described in any one of claims 1-7, characterized in that, Includes the following steps: S1: Construct node-guided search rules. For any current child node, explore the node by using a fixed step size and a turning angle after uniform discrete sampling to obtain the exploration child node set of the next child node with a fixed step size. Then, generate a guiding node through deep reinforcement learning and add it to the exploration child node set. S2: Evaluate the cost of each exploration sub-node in the set of exploration sub-nodes, and obtain the exploration sub-node with the lowest cost as the next sub-node to be selected in each round of iterative search. S3: When the distance from the current child node to the endpoint is less than a set threshold, generate the berth curve from the current child node to the endpoint, thereby obtaining the final global curve.

Images