Open-pit truck dynamic path scheduling optimization method based on deep reinforcement learning

Abstract

This invention provides a dynamic route scheduling optimization method for open-pit mine trucks based on deep reinforcement learning, comprising the following steps: S1. Construction of static basic dataset and dynamic real-time dataset; S2. Construction of preprocessed data feature set and scheduling environment model; S3. Construction of a deep reinforcement learning model integrating improved Dijkstra's algorithm, improved deep Q-network, and deep deterministic policy gradient; S4. Offline training and iterative optimization of the deep reinforcement learning model; S5. Real-time data acquisition and preprocessing; S6. Real-time route decision and continuous action generation. This invention overcomes the limitations of single algorithms, achieving dynamic road condition adaptation, discrete-continuous decision collaboration, and intelligent response to abnormal scenarios. It effectively improves the transportation efficiency of open-pit mine trucks, reduces unit transportation fuel consumption and waiting time, adapts to the dynamic changes in mining area operation needs and equipment types, and provides integrated technical support for intelligent, low-cost, and highly reliable truck scheduling in open-pit mines.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and path planning technology, and in particular to a dynamic path scheduling optimization method for open-pit mine trucks based on deep reinforcement learning. Background Technology

[0002] Open-pit mining operations are characterized by complex road networks, dynamic changes in road conditions, and diverse operational tasks. Truck transportation scheduling is a core factor affecting the efficiency and cost of mining operations.

[0003] Existing technologies have significant limitations: traditional path planning algorithms, such as the standard Dijkstra algorithm, rely on fixed weights and cannot adapt to changes in road conditions and operational status in real time, resulting in paths that are easily out of touch with the actual scenario; single reinforcement learning algorithms can either only handle discrete path decisions or only optimize continuous action variables, making it difficult to achieve coordinated optimization of discrete path selection and continuous operational parameters; scheduling systems lack closed-loop feedback mechanisms, making it difficult for trained models to adapt to long-term dynamic changes in mining areas; and they have weak capabilities in responding to abnormal scenarios such as sudden road congestion, often relying on fixed contingency plans and lacking intelligent dynamic responses. These problems result in existing scheduling schemes lacking flexibility and decision-making accuracy, making it difficult to balance transportation efficiency, operational costs, and operational reliability. Summary of the Invention

[0004] This invention provides a dynamic route scheduling optimization method for open-pit mining trucks based on deep reinforcement learning. First, a preprocessed feature set and scheduling environment model are constructed through static basic data and dynamic real-time data acquisition. The core layer employs a three-algorithm collaborative mechanism: an improved Dijkstra algorithm generates a set of feasible paths adapted to real-time traffic conditions; IDQN accurately evaluates path value and outputs the optimal discrete path; and DDPG optimizes continuous operating parameters to feed back into path generation and value assessment. Model parameters are iteratively optimized through offline training, combined with real-time data acquisition to achieve dynamic decision generation. Simultaneously, an execution monitoring, feedback optimization, and hierarchical anomaly handling mechanism are established to ensure scheduling stability. Finally, through comprehensive effect evaluation and system upgrades and maintenance, the scheduling scheme achieves continuous optimization and long-term adaptation.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for dynamic route scheduling optimization of open-pit mine trucks based on deep reinforcement learning includes the following steps:

[0007] S1. Construct a static basic dataset and a dynamic real-time dataset;

[0008] S2. Based on the static basic dataset and the dynamic real-time dataset, construct a preprocessed data feature set and scheduling environment model;

[0009] S3. Based on the preprocessed data feature set and scheduling environment model, a deep reinforcement learning model is constructed that integrates the improved Dijkstra algorithm, the improved deep Q network, and the deep deterministic policy gradient. The improved Dijkstra algorithm is used to generate a dynamic feasible path set, the improved deep Q network is used to evaluate the value of each path in the dynamic feasible path set and output the value of the optimal path, and the deep deterministic policy gradient is used to output the path weight adjustment coefficient.

[0010] S4. Using historical feature data and scheduling environment model from the preprocessed data feature set, the deep reinforcement learning model is trained offline to obtain the optimal model parameters. During the training process, the dynamic feasible path set generated by the improved Dijkstra algorithm is used as the discrete action space of the improved deep Q network. The optimal path value output by the improved deep Q network is fed back to the deep deterministic policy gradient. The path weight adjustment coefficient output by the deep deterministic policy gradient feeds back into the improved Dijkstra algorithm to correct the dynamic weight of the road.

[0011] S5. Collect real-time data according to a preset cycle and perform preprocessing to obtain real-time preprocessed data;

[0012] S6. Input the real-time preprocessed data into the deep reinforcement learning model with the optimal model parameters, generate the real-time optimal path and continuous action instructions, and send them to the truck.

[0013] In this specification, the dynamic path scheduling optimization method for open-pit mine trucks based on deep reinforcement learning also includes S7: real-time monitoring of the execution process of the real-time optimal path and continuous action instructions, and collection of execution feedback data including path execution deviation, actual road conditions, and task completion quality.

[0014] In this specification, the method for dynamic path scheduling optimization of open-pit mine trucks based on deep reinforcement learning also includes S8: Based on execution feedback data, the road dynamic weight calculation rules, path value evaluation function and continuous action constraints in the deep reinforcement learning model are adjusted, and the model parameters are iteratively optimized online to form a closed-loop scheduling process of data acquisition-preprocessing-model training-real-time decision-execution feedback-parameter optimization.

[0015] In this specification, the three-algorithm bidirectional interaction mechanism of the deep reinforcement learning model in S3 is as follows: After the improved deep Q network outputs the optimal path value, it is fed back to the deep deterministic policy gradient to correct its value evaluation benchmark, and to the improved Dijkstra algorithm to dynamically adjust the screening threshold of the feasible path set. The path weight adjustment coefficients output by the deep deterministic policy gradient are simultaneously transmitted to the improved Dijkstra algorithm to correct the dynamic weights of the roads and to the improved deep Q network to optimize the feature dimensions of path value evaluation, forming a closed-loop interaction of the three algorithms.

[0016] In this specification, the specific process of generating a dynamic feasible path set using the improved Dijkstra algorithm in S3 is as follows: combining the preprocessed data feature set and the path weight adjustment coefficients output by the gradient of the deep deterministic strategy, the dynamic comprehensive weight of each road segment is calculated; based on the dynamic comprehensive weight, a dynamic feasible path set containing the shortest path and multiple suboptimal paths is selected, and the size of the path set is dynamically adjusted according to the current workload to ensure coverage of the main feasible routes.

[0017] In this specification, the path value evaluation mechanism of the improved deep Q-network in S3 is as follows: a multi-dimensional state vector is constructed using the path features extracted by the improved Dijkstra algorithm, the truck load status in the preprocessed data feature set, the loading and unloading point queuing data, and the path weight adjustment coefficients output by the gradient of the deep deterministic strategy as input; a Q-value function that integrates dynamic features is used to calculate the static value of the path and the real-time dynamic influence factors in a weighted manner, and the quantitative value score of each path is output, with the optimal path value being the value data corresponding to the path with the highest score.

[0018] In this specification, the output constraint and optimization mechanism of the deep deterministic policy gradient in S3 is as follows: the value range of the path weight adjustment coefficient must match the actual weight fluctuation law of the road, and take effect after being verified by the road physical constraints in the scheduling environment model, so as to avoid the path sudden change caused by the excessive weight adjustment amplitude; the value network uses the optimal path value output by the improved deep Q network as a reference benchmark to correct its own evaluation result of the path weight adjustment effect, and the policy network iteratively optimizes the output accuracy of the adjustment coefficient based on the corrected evaluation result.

[0019] In this manual, the specific optimization process for offline training in S4 is as follows: the historical feature data in the preprocessed data feature set is divided into training set and validation set according to time series. During training, batch training mode is adopted, and the model performance is verified through the validation set after each batch training. During the training iteration, the TD error and policy loss value of the deep deterministic policy gradient of the improved deep Q network are monitored in real time. When the TD error stabilizes in the preset range, the policy loss value tends to converge, and the path planning accuracy on the validation set reaches the preset standard in multiple consecutive iterations, training is stopped and the optimal model parameters are saved.

[0020] In this specification, the preprocessing operations for real-time data in S5 include: removing outliers from the collected real-time data and removing invalid data that exceeds the physically reasonable range; filling missing data by interpolation of adjacent period data, and controlling the deviation of the filled data within a preset range; calculating the real-time feature dimension based on the filled valid data; aligning the calculated real-time features with the feature dimension of the preprocessed data feature set to generate real-time preprocessed data with a uniform format.

[0021] In this specification, the static basic dataset in S1 includes road topology, road physical parameters, truck performance parameters, and basic parameters of the operation task, while the dynamic real-time dataset includes road status data, truck operation data, and loading / unloading point status data.

[0022] In summary, the present invention has at least the following beneficial effects:

[0023] 1. Enhanced dynamic adaptability: It can respond to dynamic changes in road conditions, operation status, and loading / unloading point load in real time, generating optimized decisions that fit the actual scenario and avoiding the limitations of fixed solutions.

[0024] 2. Enhanced decision-making synergy: Achieves deep synergy between discrete path selection and continuous operating parameters, improving the overall integrity and accuracy of scheduling decisions.

[0025] 3. Significantly improved robustness: A hierarchical anomaly handling mechanism has been established, which can intelligently respond to various abnormal scenarios such as road congestion and equipment failure, ensuring the stable operation of the dispatching system.

[0026] 4. Resource utilization optimization: By accurately planning routes and optimizing operating parameters, ineffective travel, waiting time and resource waste are reduced, thereby lowering the overall operating cost.

[0027] 5. Continuous optimization capability: Construct a closed loop of decision-making, execution, feedback, and optimization. By combining offline training with online iteration, the model can continuously adapt to long-term changes in the mining area and maintain the effectiveness of decision-making.

[0028] 6. Excellent scalability and adaptability: It supports flexible adaptation to changes in mining operation needs, equipment types, environmental protection requirements, etc., without the need for large-scale system reconstruction, and has a wide range of applications. Attached Figure Description

[0029] Figure 1 This is a flowchart illustrating the dynamic path scheduling optimization method for open-pit mine trucks based on deep reinforcement learning involved in this invention.

[0030] Figure 2 This is a schematic diagram of the data construction and preprocessing process involved in this invention.

[0031] Figure 3 This is a schematic diagram illustrating the model construction and offline training process involved in this invention.

[0032] Figure 4 This is a schematic diagram of the real-time decision generation process involved in this invention. Detailed Implementation

[0033] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0034] like Figure 1As shown, this embodiment provides a dynamic path scheduling optimization method for open-pit mine trucks based on deep reinforcement learning, including:

[0035] S1. Construction of Static Basic Dataset and Dynamic Real-Time Dataset; This step provides data support for the entire scheduling optimization method, and is divided into two categories according to data attributes: static basic data and dynamic real-time data.

[0036] 1.1 Construction of the static base dataset; Static base dataset This includes long-term, unchanging data such as parameters of fixed facilities in open-pit mines and truck performance parameters. The specific details are as follows:

[0037] Road topology data: Defining the road topology map Complete node set With edge set Node set Includes all mining areas, spoil heaps, and intersections, specifically Mining Area A, Mining Area B, Spoil Heap 1, Spoil Heap 2, Intersection 1, and Intersection 2; Edge Set Includes all road segments, specifically: Mining Area A → Intersection 1, Intersection 1 → Intersection 2, Intersection 2 → Spoil Dump 1, Mining Area A → Intersection 2, Mining Area B → Intersection 1, Intersection 1 → Spoil Dump 2, with no other nodes or road segments. Road physical parameters: Each road segment physical length (Unit: km), Design Speed ​​Limit (Unit: km / h), by edge set Record them one by one. Truck performance parameters: Truck fuel consumption coefficient per 100 kilometers (Unit: L / km), maximum load capacity, fuel tank capacity, minimum safe driving speed 10km / h, maximum permissible driving speed 40km / h. Basic operational parameters: ore production requirements for each mining area, capacity limits for each spoil heap, and operational efficiency of loading and unloading points. All static data were obtained through on-site surveys, equipment manual searches, and extraction from operational planning documents. After manual verification, the data is stored in a standardized format and will only be updated during facility renovations, equipment upgrades, or operational task adjustments.

[0038] 1.2 Construction of Dynamic Real-Time Datasets; Dynamic Real-Time Datasets It contains data reflecting the real-time job status, updated every 10 seconds, as detailed below:

[0039] Road condition data: for each road segment Real-time traffic coefficient The value ranges from [0,1], where 1 represents optimal road conditions and 0 represents impassable conditions; real-time congestion coefficient. The value range is [0,1], where 0 represents no congestion and 1 represents complete congestion. Data is collected from sensors and video surveillance equipment deployed along the road, and generated after data cleaning to remove outliers. Truck operation data includes: the real-time location node number of each truck, its real-time load status (0 for empty, 1 for fully loaded), the maximum driving distance supported by remaining fuel (in km), and its real-time speed, all uploaded in real-time through the truck's onboard terminal. Loading / unloading point status data includes: the real-time waiting queue length (per vehicle) at each target loading / unloading point, and its current operating status (normal or suspended). This data is collected from counting devices and the operation control system at the loading / unloading points. Dynamic data is preprocessed using edge computing to remove noise and outliers, ensuring the real-time performance and accuracy of the data.

[0040] S2. Construction of Preprocessed Data Feature Set and Scheduling Environment Model; This step constructs the preprocessed data feature set based on the dataset in S1. With scheduling environment model The data construction and preprocessing process is as follows: Figure 2 As shown.

[0041] 2.1 Construction of Preprocessed Data Feature Set; Preprocessed Data Feature Set This involves further processing of the S1 data, including key features for model training and inference. The specific construction process is as follows:

[0042] Road feature engineering: for each road segment Based on dynamic real-time data, the average road condition coefficient and average congestion coefficient within a 30-second sliding window are calculated. Combined with static physical length and design speed limit, a comprehensive feature vector of the road segment is generated for subsequent dynamic weight calculation. Truck status feature engineering: For each truck, based on real-time operating data, the maximum driving distance supported by remaining fuel is calculated, which is then combined with the remaining fuel and the fuel consumption coefficient per 100 kilometers. The shortest static distance between the current location and each target loading / unloading point is derived to generate a truck state feature vector. Operational scenario feature engineering: The waiting queue length and operational efficiency of each loading / unloading point are integrated to calculate the loading / unloading resource congestion index; combined with the mining area's production demand and the spoil heap capacity, operational task urgency features are generated to provide a basis for reward function design.

[0043] All features are standardized and mapped to the [0,1] interval to avoid the impact of dimensional differences on model training, ultimately forming a structured preprocessed data feature set. .

[0044] 2.2 Construction of the scheduling environment model; scheduling environment model Used to simulate dynamic scenarios of open-pit mine truck scheduling, providing an interactive environment for reinforcement learning models:

[0045] state space Defined as a set of state characteristics and environmental characteristics of all trucks, including key state variables such as the truck's current location node number, load status, maximum driving distance supported by remaining fuel, target loading / unloading point node number, average road congestion coefficient, average road condition coefficient, and loading / unloading point waiting queue length. Action Space This includes two categories: discrete actions and continuous actions. Discrete actions represent all feasible paths, while continuous actions are determined by driving speed and path weight adjustment coefficients. Driving speed values ​​range from [10, 40] km / h, and path weight adjustment coefficient values ​​range from [0.8, 1.2]. Reward function. The system is designed as a multi-objective, comprehensive reward mechanism to evaluate the merits of actions. The reward function is calculated as follows: Immediate Reward = Transportation Efficiency Reward - Congestion Penalty - Fuel Consumption Penalty - Waiting Penalty. Specifically, the transportation efficiency reward is positively correlated with the time and volume of goods transported by the truck; the congestion penalty is positively correlated with the average congestion coefficient of the truck's route; the fuel consumption penalty is positively correlated with the travel distance and road gradient; and the waiting penalty is positively correlated with the waiting time of the truck at loading / unloading points. The specific formula for calculating the reward function was determined through experimental tuning to ensure that it guides the model to learn the optimal scheduling strategy.

[0046] The scheduling environment model has the ability to be dynamically updated, based on the latest preprocessed data feature set every 10 seconds. Update the state space and reward calculation parameters to simulate the dynamic changes in a real scheduling scenario.

[0047] S3. Construction and initialization of a deep reinforcement learning model integrating improved DQN, DDPG, and improved Dijkstra; this step is based on the preprocessed data feature set output from S2. and scheduling environment model This paper constructs an integrated decision-making model that combines an improved deep Q-network, a deep deterministic policy gradient, and an improved Dijkstra's dynamic path planning algorithm. The improved Dijkstra generates a set of dynamically feasible paths that fit the real-time scenario, providing a high-quality discrete action space for IDQN and core state features for DDPG. IDQN accurately evaluates path value, outputs the optimal discrete path, and simultaneously provides feedback value information to correct DDPG's continuous action evaluation. DDPG optimizes driving speed and path weight adjustment coefficients, feeding back into the improved Dijkstra's path generation and IDQN's value calculation. These three deeply coupled algorithms jointly address the dual challenges of path candidate effectiveness and decision optimization accuracy in the dynamic environment of open-pit mines. The model construction and offline training process is as follows: Figure 3 As shown.

[0048] 3.1 Improved Dijkstra's dynamic path planning algorithm; The core innovation of the improved Dijkstra algorithm lies in the dynamic weight design and feasible path set generation. By integrating real-time traffic conditions, congestion status, fuel consumption costs and the continuously adjusted coefficients of DDPG output, it replaces the traditional fixed distance weights and single path output, ensuring that the path candidate set not only fits the real-time scenario, but also provides sufficient optimization space for subsequent reinforcement learning models.

[0049] 3.1.1 Algorithm Construction: Dynamic Weights and Feasible Path Set Generation

[0050] First, construct the road topology map of the open-pit mine. ,in The node set contains all mining areas, spoil heaps, and intersections, specifically mining area A, mining area B, spoil heap 1, spoil heap 2, intersection 1, and intersection 2. For edge sets, corresponding to all road segments within the mining area, specifically: mining area A → intersection 1, intersection 1 → intersection 2, intersection 2 → spoil heap 1, mining area A → intersection 2, mining area B → intersection 1, intersection 1 → spoil heap 2.

[0051] (1) Dynamic weight calculation; the dynamic weight of a road segment needs to comprehensively reflect traffic efficiency and cost, while also responding to the continuous optimization results of DDPG. The calculation formula is as follows: ;in for Constant connection node and road section The dynamic weight is unitless and is a comprehensive quantitative indicator of path cost. This is the weight adjustment factor output by DDPG, ranging from 0.8 to 1.2. It is a continuous action variable used to dynamically adjust the cost priority of road segments. The design logic of this formula is: the first term... Reflecting travel time costs, the longer the road, the lower the speed limit, the worse the road conditions, and the more severe the congestion, the higher this value, and the lower the traffic efficiency; the second item This reflects fuel consumption costs; the longer the road and the higher the fuel consumption coefficient, the larger this value will be. The introduction of this allows DDPG to dynamically intervene in path weights, when When the value is greater than 1, the overall cost of the corresponding road segment is amplified, reducing its probability of being selected; when... When the value is less than 1, the overall cost is reduced, increasing the probability of it being selected.

[0052] (2) Generation of feasible path set; The improved Dijkstra algorithm abandons the traditional single shortest path output and generates a feasible path set containing multiple suboptimal paths, providing sufficient discrete action space for IDQN. The specific steps are as follows:

[0053] 1. Initialization settings: Define the truck's current position as the starting node. The target loading / unloading point is the endpoint. Initialize the node distance array The distance to the starting node is set to 0, and the distances to all other nodes are set to infinity, representing that they are initially unreachable; the predecessor node array is initialized. All elements are empty, used to record the path trajectory; initialize the set of unvisited nodes. Its initial value is the node set. .

[0054] 2. Iteratively update node distances: set of unvisited nodes Select the node with the smallest distance. , take it from Remove nodes from the list and mark them as visited; iterate through the nodes. All adjacent nodes Calculate the distance from the starting node to the destination node. arrive The temporary distance, which is equal to the node's temporary distance. Distance and road section The sum of dynamic weights; if the temporary distance is less than the node Update the node based on the current distance. The distance is a temporary distance, and the node is recorded. The predecessor node is Repeat the above process until no node set has been visited. If empty, then the endpoint node is... The distance is Total weight of shortest path at time It has no unit and is the core indicator of the overall cost of the route.

[0055] 3. Shortest path backtracking: Starting from the destination node Begin by using the predecessor node array Tracing back in reverse until returning to the starting node. ,get Shortest path at time The path format is: starting node → intermediate node → ... → ending node.

[0056] 4. Feasible Path Filtering: Set the initial filtering threshold to... Based on road topology map The structure, examples of all nodes starting from the origin node. To the destination node Potential paths are identified; the total weight of each potential path is calculated, which is the sum of the dynamic weights of all road segments within the path; all paths with a total weight less than or equal to the initial screening threshold are selected to form [the path]. feasible path set at time .

[0057] feasible path set Its core function is to directly serve as the discrete action space of IDQN, while the path features it contains (total weight, length, congestion distribution, etc.) provide key inputs for the construction of the state vector of DDPG, thereby realizing improved positive data transfer from Dijkstra to IDQN and DDPG.

[0058] 3.1.2 Algorithm Training; The improved Dijkstra algorithm's gradient-free training process is based on dynamically adjusting the selection threshold of feasible paths through optimal path feedback from IDQN. This achieves reverse interaction between IDQN and the improved Dijkstra algorithm, ensuring a balance between the effectiveness and diversity of the path set. The adjustment rule for the selection threshold is determined based on the relationship between the total weight of the optimal path output by IDQN and the total weight of the shortest path: if the total weight of the optimal path output by IDQN is less than... This indicates that the current path set has excessive diversity and contains redundant paths. Therefore, the filtering threshold should be adjusted to... This reduces the number of redundant paths; if the total weight of the optimal path output by IDQN is between and The range indicates that the effectiveness and diversity of the current path set are in a balance, maintaining the screening threshold at [value missing]. If the total weight of the optimal path output by IDQN is greater than This indicates that the current path set lacks diversity, and the optimal path may not be globally optimal. Therefore, the filtering threshold should be adjusted to... This increases the number of candidate paths. The core significance of this dynamic adjustment mechanism is to enable the improved Dijkstra algorithm to respond to the decision results of IDQN, avoid the decision limitations caused by the solidification of the path set, and improve the adaptability of the path candidates.

[0059] 3.1.3 Algorithm Application; The improved Dijkstra algorithm is synchronized with the real-time data acquisition cycle of S5, executing the path set generation process every 10 seconds. This is based on the latest collected dynamic data (road condition coefficients, congestion coefficients, etc.) and the weight adjustment coefficients output by DDPG. Recalculate the dynamic weights of all road segments. And generate a new set of feasible paths according to the steps above. The newly generated set of feasible paths will be synchronized to IDQN to update its discrete action space; it will also be synchronized to DDPG to update the path features in its state vector. Furthermore, the algorithm receives the optimal path and its total weight from IDQN feedback, and adjusts the selection threshold for the next round of path generation according to the rules determined during the training phase, forming a closed-loop application process for the improved Dijkstra algorithm.

[0060] 3.2 Improved DQN (IDQN) Model; The IDQN model is designed for discrete path decision-making scenarios in open-pit mines. It introduces a priority experience replay mechanism and integrates the continuous action features of the DDPG. Its core function is to accurately evaluate the value of each path in the feasible path set, output the optimal discrete path, and feed the value evaluation results back to the DDPG to correct its continuous action value judgment, forming a positive interaction between IDQN and DDPG. It also receives the continuous actions of DDPG and the path set of the improved Dijkstra, optimizes its own decision-making accuracy, and forms a reverse interaction between DDPG and improved Dijkstra and IDQN.

[0061] 3.2.1 Model Construction

[0062] (1) State vector and action space; IDQN state vector Based on S2 state-space optimization, it integrates improved Dijkstra's path features and DDPG continuous actions, with a dimension of 8, and its specific structure is as follows:

[0063] Truck current location node number: value is the node set The node number in the data set is obtained from real-time data from S5; the truck's load status is 0 when empty and 1 when fully loaded, obtained from real-time data from S5; the average total weight of the feasible path set is calculated as the arithmetic mean of the total weights of all paths in the feasible path set, provided by the improved Dijkstra output; the average congestion coefficient of the feasible path set is calculated as the weighted average of the congestion coefficients of all road segments in the feasible path set, with the weight being the physical length of the corresponding road segment, calculated by the improved Dijkstra combined with real-time data; the maximum driving distance supported by the truck's remaining fuel is in km, obtained from real-time data from S5; the node number of the target loading / unloading point is a value from the node set. The node number in the DDPG output is provided by the task requirements; the suggested driving speed output by DDPG is in km / h and ranges from 10 to 40, and is output by the DDPG policy network; the weight adjustment coefficient output by DDPG is in the range of 0.8 to 1.2 and is output by the DDPG policy network.

[0064] IDQN's operational space The feasible path set generated by the improved Dijkstra is completely consistent with that generated by the improved Dijkstra. Each action corresponds to a specific path in the set of feasible paths, and the action dimension is equal to the number of paths in the set of feasible paths.

[0065] (2) Fusion Q-value function; The Q-value of traditional DQN depends only on the state and discrete actions, and cannot respond to the optimization results of continuous actions. This scheme introduces the continuous action characteristics of DDPG: suggesting driving speed and weight adjustment coefficients, and constructing a fusion Q-value function, so that the path value assessment can dynamically adapt to the optimization results of continuous actions. The formula is: ;in The path output by the IDQN master Q network Its value has no unit. This is the parameter set of the main Q network, which includes the network's weights and biases. The basic Q-value is dimensionless and is determined by the main Q-network through the state. and path The characteristics directly predict the static value of the path. The interaction weight between IDQN and DDPG is fixed at 0.3, with no unit, and is used to balance the contribution of the base Q value and the continuous action feature. The recommended driving speed output by DDPG. The average design speed limit for all road segments, expressed in km / h, is calculated as the arithmetic mean of the design speed limits for all road segments and is used to normalize the recommended driving speed. This is the speed adaptation coefficient during continuous operation. It has no unit. It is recommended that the closer the driving speed is to the average design speed limit, the closer this value is to 1, and the greater its positive contribution to the Q value. The weight adjustment coefficients for DDPG output. This represents the deviation of the weight adjustment coefficient in continuous action. It has no unit. The closer the weight adjustment coefficient is to 1, the less significant the adjustment. The smaller this value is, the smaller the negative impact on the Q value.

[0066] The core contribution of this fusion-type Q-value function lies in achieving deep coupling between discrete path value and continuous action characteristics, enabling IDQN's value assessment to not only consider the static characteristics of the path itself, but also respond to the continuous optimization results of DDPG, thereby improving the accuracy of value assessment.

[0067] (3) TD error and loss function; IDQN uses a priority-based empirical replay mechanism to improve training efficiency. TD error serves as the core basis for empirical priority, and its formula is: ;in for The TD error at time step 1 is dimensionless and is used to measure the deviation between the predicted value of the master Q network and the target Q value. The target Q value is dimensionless, and its formula is: ; for Execution path at all times The immediate reward is unitless and calculated according to the reward function of S2. This is a discount factor, fixed at 0.95, unitless, used to weigh the importance of immediate rewards against future rewards. This is the output of the target Q-network. Optimal path at any time Q value, This is the parameter set for the target Q-network. This is a task completion indicator; the value is 0 when the task is not completed and 1 when the task is completed.

[0068] The loss function of IDQN is calculated empirically based on priority-based empirical replay sampling, and the formula is as follows: ;in This represents the expectation of the sampled experience in the priority experience replay pool PER, where PER is used to store training experience. Sampling is performed according to priority, and the sampling probability is positively correlated with the TD error of the experience, ensuring that the training process prioritizes the use of high-value experience (experience with high TD error) and improves training efficiency.

[0069] 3.2.2 Model Training; The IDQN training process is deeply coupled with the improved Dijkstra and DDPG, with each training step involving multi-algorithm interaction logic. The specific steps are as follows:

[0070] 1. Experience collection: based on - Greedy strategy for choosing actions To explore the probability, the initial value is 0.9, and it decreases by 0.01 every 100 iterations until it reaches 0.1 (exploitation probability). When the action takes effect, a path is randomly selected from the action space; otherwise, the path with the largest fusion Q value is selected as the temporary optimal path.

[0071] 2. Experience storage: storing experience groups Stored into the priority experience replay pool PER, where and The continuous action features output by DDPG are used to reconstruct the state vector during subsequent training. The priority of this experience is also calculated, and the priority is the TD error plus the minimum value. This avoids situations where a priority of 0 prevents experience from being sampled.

[0072] 3. Batch Sampling: 64 experiences are extracted from PER according to sampling probability as batch training samples. The importance weight of each experience is recorded. The formula for calculating the importance weight is... ,in The total number of experiences in PER. For the first The sampling probability of the empirical evidence, This is the priority correction coefficient, initially set to 0.4, and gradually increased to 1.0 with each iteration, used to balance the bias in priority sampling.

[0073] 4. Parameter Update: Update the main Q-network parameters by minimizing the loss function using gradient descent. The updated formula is: ;in The learning rate for IDQN is fixed at a value of [value to be filled in]. , Indicates the parameters of the master Q network gradient, For the first The importance weight of each experience.

[0074] 5. Target network soft update: After every 50 iterations, adjust according to the soft update coefficient. Update target Q network parameters The updated formula is: The soft update mechanism updates the target Q-network parameters slowly to avoid excessive fluctuations in the target Q-value that could lead to training divergence, thus ensuring training stability.

[0075] 6. Multi-algorithm interaction and synchronization: This is the core of IDQN training, enabling closed-loop interaction with other algorithms. Every 20 iterations, it receives the latest feasible path set from the improved Dijkstra algorithm. Update its own operational space This ensures that the driving space is aligned with real-time road conditions; every 10 iterations, it receives the latest recommended driving speed output from DDPG. and weight adjustment coefficient Correct the state vector The corresponding components in the algorithm are used to optimize the calculation of the fusion Q value. After every 100 iterations, the current optimal path, i.e. the path with the largest fusion Q value, is fed back to the improved Dijkstra algorithm to adjust the filtering threshold for the next round of path generation.

[0076] 7. Convergence Criteria: The validation set, consisting of 20% of the training sample set, is generated from historical data of S1 and the environmental model of S2, satisfying three criteria for five consecutive rounds. The first criterion is an average TD error less than or equal to 0.5; the second criterion is an actual transportation efficiency of the optimal path greater than or equal to 800 tons / hour; and the third criterion is an average waiting time less than or equal to 5 minutes. Training stops when all three criteria are met, and the main Q-network parameters are saved. .

[0077] 3.2.3 Model Application; The core of IDQN application is to generate the optimal discrete path based on real-time state and output feedback to other algorithms. The specific process is as follows:

[0078] 1. Real-time state input: Receives real-time data updated by S5, combines the feasible path set features from the latest output of the improved Dijkstra method with the continuous actions from the latest output of DDPG, and constructs a real-time state vector. After processing according to the S2 normalization method, the data is input into the trained main Q-network with the following parameters: .

[0079] 2. Optimal Path Decision: Calculate the fusion Q-value of all paths in the action space and select the path with the largest Q-value as the optimal discrete path. If multiple paths have the same Q-value, select the path with the smallest total weight to ensure the uniqueness of the decision.

[0080] 3. Feedback to Improved Dijkstra: The optimal path and its total weight are fed back to the improved Dijkstra. The improved Dijkstra adjusts the filtering threshold for the next round of path generation according to the rules determined in the training phase, realizing the reverse interaction between IDQN and the improved Dijkstra.

[0081] 4. Feedback to DDPG: Extract the Q-value of the optimal path. This feedback is sent to the DDPG value network to revise the value assessment of continuous actions. The revision formula is as follows: ;in The output of the corrected DDPG value network. As the basic output of the DDPG value network, The weight for IDQN value feedback is fixed at 0.7, and is related to the weight in IDQN. Complementarity ensures a balance of contributions between the two, enabling positive interaction between IDQN and DDPG.

[0082] 3.3 Deep Deterministic Strategy Gradient (DDPG) Model; The DDPG model addresses continuous decision variables (driving speed, path weight adjustment coefficients) in open-pit mine scheduling. Through bidirectional interaction with IDQN and improved Dijkstra's algorithm, it achieves dynamic optimization of continuous actions. Its core function is to output path weight adjustment coefficients to the improved Dijkstra algorithm to correct path weight calculations; output suggested driving speeds to IDQN to optimize Q-value calculations, forming a positive interaction between DDPG and improved Dijkstra and IDQN; simultaneously, it receives Q-value feedback from IDQN to correct its own value assessment and receives path weight feedback from improved Dijkstra to adjust its strategy, forming a negative interaction between IDQN and improved Dijkstra and DDPG, ultimately achieving collaborative optimization of continuous actions and discrete paths.

[0083] 3.3.1 Model Construction

[0084] (1) State vector and action space; State vector of DDPG Based on the IDQN state vector, an improved Dijkstra's path weight bias and the optimal Q-value of IDQN are added, with a dimension of 9. The specific components are as follows: Truck current location node number: consistent with the corresponding component of the IDQN state vector; Truck load status: consistent with the corresponding component of the IDQN state vector; Average total weight of feasible path set: consistent with the corresponding component of the IDQN state vector; Average congestion coefficient of feasible path set: consistent with the corresponding component of the IDQN state vector; Maximum driving distance supported by the truck's remaining fuel: consistent with the corresponding component of the IDQN state vector; Target loading / unloading point node number: consistent with the corresponding component of the IDQN state vector; Standard deviation of total weight of feasible path set: calculated as the standard deviation of the total weight of all paths in the feasible path set, unitless, provided by the improved Dijkstra's calculation, reflecting the difference in path cost; Optimal path Q-value output by IDQN: provided by IDQN feedback, unitless; Waiting queue length of the target loading / unloading point: unit is vehicles, obtained from real-time data of S5, reflecting the congestion of loading / unloading resources.

[0085] DDPG's Action Space The continuous action space includes two core optimization variables: suggested driving speed and improved Dijkstra weight adjustment coefficients. The suggested driving speed is expressed in km / h and ranges from 10 to 40. It must meet road speed limit constraints, meaning the suggested driving speed must not exceed the maximum design speed limit of all road segments in the corresponding path. The improved Dijkstra weight adjustment coefficients range from 0.8 to 1.2 and are used to dynamically adjust the weights of road segments.

[0086] (2) Policy network and value network structure; DDPG includes a policy network (Actor) and a value network (Critic), both of which adopt a fully connected neural network structure.

[0087] Policy Network: Input is the state vector of DDPG The output is the optimal continuous action. The network structure is as follows: Input layer (9-dimensional) → Fully connected layer 1 (128 neurons, ReLU activation) → Fully connected layer 2 (64 neurons, ReLU activation) → Output layer (2-dimensional, tanh activation) → Action scaling. The output range of the tanh activation function is -1 to 1, which needs to be mapped to the target range through action scaling. The suggested mapping formula for driving speed is: The improved mapping formula for Dijkstra's weight adjustment coefficients is as follows: ,in and This represents the tanh activation value output by the policy network. The core formula of the policy network is: ;in This is the weight matrix from the input layer to the fully connected layer 1, with dimensions 9×128; This is the weight matrix for fully connected layers 1 to 2, with dimensions 128×64; This is the bias term for fully connected layer 1, with a dimension of 128; This is the bias term for fully connected layer 2, with a dimension of 64; The parameter set for the policy network contains , , , .

[0088] Value Network (Parameter Set) ): The input is the state vector of DDPG and continuous action Output as action value The network structure is as follows: State input layer (9-dimensional) → Action input layer (2-dimensional) → Concatenation layer (11-dimensional) → Fully connected layer 1 (128 neurons, ReLU activation) → Fully connected layer 2 (64 neurons, ReLU activation) → Output layer (1-dimensional, linear activation). The core formula of the value network is the value assessment formula that incorporates IDQN value feedback: ;in This is the weight matrix from the splicing layer to the fully connected layer 1, with dimensions 11×128; This is the weight matrix for fully connected layers 1 to 2, with dimensions 128×64; This is the bias term for fully connected layer 1, with a dimension of 128; This is the bias term for fully connected layer 2, with a dimension of 64; The parameter set of the value network, containing , , , ; This represents the concatenated vector of the state vector and the action vector. The optimal path Q value output by IDQN.

[0089] (3) Definition of loss function; Policy network loss function: The optimization objective of the policy network is to maximize the action value output by the value network. The loss function formula is: ;in This represents the expectation of the sampled experience in the ReplayBuffer, which stores the training experience of DDPG and has a capacity of 100,000 records. This loss function maximizes the value assessment of consecutive actions through gradient ascent, enabling the policy network to generate better proposed driving speeds and weight adjustment coefficients. Value Network Loss Function: The optimization objective of the value network is to minimize the deviation between the target value and the predicted value. The loss function formula is: ;in The target value is unitless, and the formula is: ; and These are the outputs of the target value network and the target policy network, respectively, and their parameter sets are... and Obtained through a soft update. For immediate rewards, shared with IDQN, calculated according to the reward function of S2.

[0090] 3.3.2 Model Training; The training process of DDPG is deeply coupled with IDQN and improved Dijkstra, and interactive experience is stored through an experience replay pool. The specific steps are as follows:

[0091] 1. Experience Collection: The policy network outputs continuous actions: suggested driving speed and weight adjustment coefficients, which are output to IDQN and Improved Dijkstra respectively. IDQN receives the suggested driving speed to update its own state vector; Improved Dijkstra receives the weight adjustment coefficients to calculate the dynamic weights of road segments. Simultaneously, DDPG receives the optimal path and optimal path Q-value from IDQN, and the feasible path set and total optimal path weight from Improved Dijkstra. Combined with real-time rewards, these are used to form experience groups. Store it in the experience replay pool ReplayBuffer.

[0092] 2. Batch Sampling: 64 experiences are randomly selected from the ReplayBuffer as batch training samples. The random sampling mechanism avoids training instability caused by experience correlation and improves the generalization ability of training.

[0093] 3. Parameter Update: Value network parameter update: Minimize the value network loss function using gradient descent. The update formula is: ;in The learning rate of the DDPG value network is fixed at a value of [value to be filled in]. , Indicates the parameters of the value network The gradient.

[0094] Policy network parameter update: Maximize value output through gradient ascent; the update formula is derived based on the chain rule. ; ;in The learning rate of the DDPG policy network is fixed at a value of [value to be filled in]. , Indicates the network parameters of the policy The gradient.

[0095] 4. Target network soft update: After every 50 iterations, the soft update coefficient is applied. The parameters of the target value network and the target policy network are updated using the following formula: ; The soft update mechanism keeps pace with IDQN to ensure that the target network parameters are updated slowly, avoiding excessive fluctuations in target value that could lead to training divergence.

[0096] 5. Multi-algorithm Interaction and Synchronization: This is the core of DDPG training, enabling closed-loop interaction with other algorithms. Every 10 iterations, the latest output weight adjustment coefficients and suggested driving speeds are synchronized to the improved Dijkstra and IDQN, triggering path weight recalculation and Q-value correction. Every 30 iterations, the system receives dynamic weight statistics of road segments from the improved Dijkstra: average weight and maximum weight deviation. If the maximum weight deviation is too large, it indicates that the adjustment range of the weight adjustment coefficients is unreasonable; in this case, the range of the weight adjustment coefficients is narrowed, such as from 0.8 to 1.2 to 0.9 to 1.1. Every 50 iterations, the system receives optimal path Q-value statistics from IDQN: average Q-value and Q-value variance. If the Q-value variance is too large, it indicates insufficient stability of the value assessment; in this case, the learning rate of the value network is reduced, such as from... Reduce to .

[0097] 6. Convergence Criteria: The validation set must satisfy three criteria for five consecutive rounds. The validation set is generated from historical data from S1 and the environment model from S2. The first criterion is that the value network loss is less than or equal to 5.0; the second criterion is that the absolute value of the policy network loss is less than or equal to 2.0; and the third criterion is that the transportation efficiency improvement after continuous action optimization is greater than or equal to 10%, which is obtained by comparing with the scenario without DDPG optimization. When all three criteria are satisfied, training stops, and the policy network parameters are saved. and value network parameters .

[0098] 3.3.3 Model Application; The core of DDPG application is to generate optimal continuous actions based on real-time states, which then feeds back into other algorithms to optimize decision-making. The specific process is as follows:

[0099] 1. Real-time state input: Receives real-time data updated by S5, and constructs a real-time state vector by combining the feasible path set features of improved Dijkstra's algorithm and the optimal path Q-value of IDQN. After being processed according to the S2 standardization method, it is input into the trained policy network.

[0100] 2. Optimal Continuous Action Generation: The policy network outputs initial continuous actions: initial suggested driving speed and initial weight adjustment coefficients. These need to be checked and adjusted based on constraints to ensure that the actions conform to the actual operational rules. The constraint checking and adjustment rules are as follows:

[0101] Speed ​​constraint: If the initial suggested speed is greater than the maximum design speed limit for all road segments in the corresponding optimal path, then the speed will be adjusted by multiplying the maximum design speed limit by the average congestion coefficient of the feasible path set. The adjustment formula is as follows: The higher the congestion coefficient, the lower the adjusted speed should be to ensure driving safety; if the initial recommended driving speed is less than 10km / h, then adjust it to 10km / h to ensure the minimum safe speed.

[0102] Weight adjustment coefficient constraint: If the initial weight adjustment coefficient is less than 0.8, it will be adjusted to 0.8; if the initial weight adjustment coefficient is greater than 1.2, it will be adjusted to 1.2, to ensure that the weight correction range is within a reasonable range and to avoid excessive adjustment that could lead to path generation distortion.

[0103] 3. Action Output and Positive Interaction: Output weight adjustment coefficients to the improved Dijkstra: As a core parameter of the improved Dijkstra's dynamic weight formula, it corrects the dynamic weights of all road segments, enabling the continuous actions of the DDPG towards the improved Dijkstra to intervene in path generation, making path generation more aligned with the optimization goals of continuous actions. Output suggested driving speeds to IDQN: As a component of the IDQN state vector, it participates in the calculation of the fusion-type Q-value, enabling the continuous actions of the DDPG towards IDQN to optimize the value assessment of discrete paths, improving the accuracy of value assessment.

[0104] 4. Reverse Feedback Reception and Action Adjustment: Receives Q-value deviation feedback from IDQN: Calculates the Q-value deviation, which is the difference between the optimal path Q-value from IDQN and the value assessment by DDPG. If the Q-value deviation is greater than 1.0, it indicates that IDQN considers the path more valuable. In the next iteration, the weight adjustment coefficient will be lowered by 5%, further reducing the weight of the corresponding path and increasing the probability of its selection. If the Q-value deviation is less than -1.0, it indicates that DDPG overestimated the path value. In the next iteration, the weight adjustment coefficient will be increased by 5%, reducing the probability of the path's selection. Receives weight deviation feedback from improved Dijkstra's algorithm: Calculates the weight deviation, which is the average absolute error between the dynamic weight of the road segment adjusted by DDPG and the actual comprehensive cost weight. The actual comprehensive cost weight is calculated from the feedback data from S7. If the weight deviation is greater than 0.1, it indicates that the weight adjusted by DDPG deviates too much from the actual weight. In this case, the range of the weight adjustment coefficient will be narrowed, and the adjustment magnitude will be reduced to ensure the rationality of the weight adjustment.

[0105] 3.4 Three-algorithm fusion and collaborative mechanism and overall output; closed-loop collaborative logic; the three algorithms form a complete closed-loop collaborative mechanism through bidirectional interaction and tripartite linkage, ensuring dynamic optimization and precise adaptation of decisions:

[0106] 1. Improve the bidirectional interaction between Dijkstra and IDQN: Improve Dijkstra to generate a set of feasible paths as the action space of IDQN, provide path features to construct the state vector of IDQN, and provide a basis for the discrete path decision of IDQN; IDQN outputs the optimal path and its total weight, driving the improved Dijkstra to dynamically adjust the screening threshold, optimize the diversity and effectiveness of the path set, and avoid the decision limitations caused by the solidification of the path set.

[0107] 2. Two-way interaction between IDQN and DDPG: IDQN outputs the optimal path Q-value, corrects the evaluation results of the DDPG value network, and improves the accuracy of continuous action value assessment; DDPG outputs suggested driving speed and weight adjustment coefficients, participates in IDQN's state construction and fusion Q-value calculation, so that IDQN's value assessment can respond to the optimization results of continuous actions and realize the value synergy between discrete paths and continuous actions.

[0108] 3. Improve the bidirectional interaction between Dijkstra and DDPG: DDPG outputs weight adjustment coefficients to correct and improve the dynamic weights of road segments in Dijkstra, so that path generation can respond to the optimization objectives of continuous actions; improve the statistical characteristics of the dynamic weights of road segments output by Dijkstra to adjust the action constraints of DDPG, ensure the rationality of continuous actions, and avoid excessive adjustment that leads to path generation distortion.

[0109] Final Output; The final output of S3 is the complete structure of the fusion model, training parameters, interaction interface specifications, and collaborative scheduling logic, providing the core foundation for subsequent steps, specifically including:

[0110] 1. Integration of Model Structure and Training Parameters: Improved Dijkstra Algorithm: Includes dynamic weight calculation model (with weight adjustment coefficient interface), feasible path set generation logic, and dynamic adjustment rules for screening thresholds; IDQN Model: Parameters of the main Q-network after training convergence. Target Q-network parameters Priority experience replay pool, containing historical interaction experience; DDPG model: parameters of the policy network after training convergence. Value network parameters Target policy network parameters Target value network parameters Experience replay pool, containing continuous action interaction experience.

[0111] 2. Algorithm Interaction Interface Specifications: Data Input Interface: Clearly define the input format, data type, and update frequency for S2 environment model parameters, including real-time data location, load capacity, and congestion coefficient for S5; Data Output Interface: Clearly define the optimal path output format for IDQN: node sequence + path features; the continuous action output format for DDPG: suggested driving speed + weight adjustment coefficient; improve the feasible path set output format for Dijkstra to ensure consistency and compatibility of data interaction.

[0112] 3. Cooperative scheduling logic script: Includes timing control rules for the interaction and synchronization of the three algorithms, such as updating the path set every 10 seconds, synchronizing parameters every 20 rounds of iteration, action constraint verification rules such as boundary adjustment of speed and weight coefficients, and convergence judgment criteria, to ensure the continued effectiveness of the multi-algorithm collaborative mechanism in subsequent steps.

[0113] S4. Offline Training Iteration and Optimization: This step is based on the fusion model built in S3. It uses the historical static data in S1 and the scheduling environment model in S2 to perform offline training iterations, optimize model parameters, and ensure that the model has good decision-making performance.

[0114] 4.1 Training data preparation; from the static base dataset of S1 Historical operational data from the past year was extracted, including historical road conditions, truck trajectories, and operational task records. Combined with the S2 scheduling environment model, a large number of simulated training samples were generated, covering different congestion scenarios, road condition scenarios, and operational task scenarios, ensuring the diversity and representativeness of the training samples. The training samples were divided into training and validation sets in a 7:3 ratio, with the validation set used to evaluate the model's generalization ability and the training set used for model parameter updates. All training samples underwent feature engineering and standardization according to S2's preprocessing standards.

[0115] 4.2 Iterative Training Process; Offline training iterations are performed according to the following steps until the model reaches the convergence condition:

[0116] 1. Model Initialization: Load the initial parameters of the fusion model built by S3, including the initial values ​​of the screening threshold of the improved Dijkstra, the initial parameters of the main Q network and target Q network of IDQN, the initial parameters of the policy network and value network of DDPG, and initialize the priority experience replay pool of IDQN and the experience replay pool of DDPG.

[0117] 2. Sample Batch Training: Randomly select a batch of samples (batch_size=64) from the training set and input them into the fusion model for training. The training process strictly follows the IDQN training process and DDPG training process defined in S3, including key steps such as experience collection, experience storage, batch sampling, parameter update, and target network soft update.

[0118] 3. Multi-algorithm collaborative update: In each round of iterative training, the closed-loop collaborative logic of the three algorithms is maintained. The improved Dijkstra adjusts the coefficients based on the weights output by DDPG to generate a set of feasible paths, IDQN evaluates the value of the paths and provides feedback on the optimal path, and DDPG adjusts its continuous actions based on the value feedback from IDQN and the path characteristics of the improved Dijkstra. The three algorithms constrain and correct each other.

[0119] 4. Model Evaluation and Parameter Tuning: After every 100 training iterations, the model performance is evaluated using a validation set. Evaluation metrics include average transportation efficiency, average waiting time, average fuel consumption, and model loss value. If the model performance does not meet the preset metrics, the learning rate, discount factor, and interaction weights are adjusted. Once the hyperparameters are equal, continue iterative training; if the preset target is reached, record the current model parameters as candidate optimal parameters.

[0120] 5. Training Termination Criteria: Offline training will stop when the evaluation metrics of the validation set for 5 consecutive rounds meet the following conditions: ① Average TD error ≤ 0.5; ② Actual transportation efficiency of the optimal path ≥ 800 tons / hour; ③ Average waiting time ≤ 5 minutes; ④ Value network loss ≤ 5.0; ⑤ Absolute value of policy network loss ≤ 2.0; ⑥ Transportation efficiency improvement after continuous action optimization ≥ 10%.

[0121] 4.3 Determining the Optimal Model Parameters: After offline training, the set of parameters with the best performance on the validation set is selected from all candidate optimal parameters as the final offline training parameters. This includes the improved Dijkstra's selection threshold adjustment rule and the main Q-network parameters of IDQN. With the target Q network parameters DDPG policy network parameters With value network parameters The data is then stored in the model parameter library to provide a foundation for subsequent real-time scheduling.

[0122] S5. Real-time data acquisition and preprocessing; This step connects with the construction of the dynamic real-time dataset in S1, realizing the continuous acquisition, cleaning and preprocessing of real-time data, providing data support for real-time decision-making in S6, with an acquisition cycle of once every 10 seconds.

[0123] 5.1 Real-time data acquisition; Through sensor networks deployed in the mining area, truck-mounted terminals, loading and unloading point control systems, video surveillance equipment, etc., the following data is collected in real time: Road status data: for each road segment Real-time traffic coefficient Real-time congestion coefficient The data is collected every 10 seconds to ensure real-time data accuracy. Truck operation data: the real-time location of each truck, mapped to a set of nodes. The system includes: node number; real-time load capacity, mapped to a load capacity status of 0 or 1; remaining fuel level, calculated as the maximum supported driving distance; and real-time driving speed, uploaded in real-time by the onboard terminal to the data acquisition platform. Loading / unloading point status data includes: real-time waiting queue length and current operation status (normal / paused) at each loading / unloading point, collected by the counting sensors and operation control system at the loading / unloading points. All data acquisition devices have been time-synchronized to ensure data from different sources is consistent in timestamps, avoiding decision-making errors caused by data misalignment.

[0124] 5.2 Real-time data preprocessing; The following preprocessing operations are performed on the collected real-time data:

[0125] Data cleaning: Remove outliers such as data with a congestion coefficient greater than 1 or less than 0, and data whose location is not in the node set. The data was processed; missing values ​​were filled using the mean of the previous 5 periods to ensure data validity. Data transformation: real-time truck locations were converted into node sets. The node number in the code converts the remaining fuel into the maximum supported driving distance: remaining fuel ÷ fuel consumption coefficient per 100 kilometers. The load capacity is converted into a binary state of 0 (empty) or 1 (fully loaded) to ensure that the data format is consistent with the model input requirements. Feature calculation: Path features such as the average total weight, average congestion coefficient, and total weight standard deviation of the feasible path set are calculated to support the construction of state vectors for IDQN and DDPG. The preprocessed real-time data is stored in a standardized format in the real-time database and simultaneously pushed to the S6 real-time decision module to provide real-time input for the model.

[0126] S6. Real-time Route Decision and Continuous Action Generation; This step, based on the fusion model of S3, the offline trained optimal parameters of S4, and the real-time preprocessed data of S5, generates the real-time optimal route and continuous action instructions for each truck to guide truck operations. The real-time decision generation process is as follows: Figure 4 As shown.

[0127] 6.1 Real-time State Construction: Extract all current state data from the S5 real-time database, and combine them with the state vector definitions of IDQN and DDPG in S3 to construct the real-time state vector for each truck. and .

[0128] The construction of the state vector strictly follows the definition of S3, ensuring that the value of each component is accurate and in the correct format. For example, the state vector of IDQN. It includes eight components: the truck's current location node number, load status, average total weight of the feasible path set, average congestion coefficient of the feasible path set, maximum driving distance supported by remaining fuel, target loading / unloading point node number, suggested driving speed output by DDPG, and weight adjustment coefficient output by DDPG; DDPG's state vector. exist Based on this, the standard deviation of the total weight of the feasible path set and the optimal path Q value output by IDQN are added, for a total of 9 components.

[0129] 6.2 Optimal Path and Continuous Action Generation; The optimal decision is generated according to the following process:

[0130] 1. Improved Dijkstra path set generation: Based on the dynamic weights of road segments at the current moment, the algorithm calculates feasible paths using the real-time traffic coefficients and congestion coefficients from S5 and the weight adjustment coefficients output by DDPG; it then executes the improved Dijkstra algorithm to generate the feasible path set for the current moment. It is synchronized to IDQN as its discrete action space.

[0131] 2. IDQN Optimal Path Decision: The constructed real-time state vector... Input the trained IDQN main Q network and compute the set of feasible paths. For each path, the fused Q-value is used, and the path with the largest Q-value is selected as the optimal discrete path. If multiple paths have the same Q value, the path with the smallest total weight is selected.

[0132] 3. DDPG optimal continuous action decision: The constructed real-time state vector... Input a trained DDPG policy network, output initial continuous actions: suggested driving speed and path weight adjustment coefficients, adjusted after constraint verification.

[0133] Generate the final optimal consecutive actions: Recommended travel speed With weight adjustment coefficient .

[0134] 4. Decision result integration and instruction issuance: Integrate the optimal discrete path output by IDQN. The optimal continuous actions output by DDPG are integrated into a complete dispatch instruction, which includes a sequence of path nodes, a suggested driving speed range, and a congestion response threshold during the journey (set based on the average congestion coefficient of the feasible path set). The dispatch instruction is sent to the on-board terminal of the corresponding truck via a wireless communication network, and simultaneously synchronized to the monitoring platform of the mining area dispatch center.

[0135] 6.3 Closed-loop decision-making process; Real-time path decision-making and continuous action generation are executed cyclically every 10 seconds, forming a closed-loop process: 1. Receive the latest real-time preprocessed data pushed by S5; 2. Improve the Dijkstra algorithm to generate a new set of feasible paths. 3. IDQN and DDPG output optimal decisions based on the new state vector; 4. Issue scheduling instructions and monitor their execution; 5. Collect execution feedback data such as actual driving speed, path execution deviation, and real-time congestion changes to provide a basis for the next round of decision-making.

[0136] The decision-making process includes a decision delay tolerance mechanism. If real-time data is not updated in time due to network latency, the decision result of the previous cycle is automatically used and marked as a temporary instruction. Once the data is updated, the decision is recalculated and a formal instruction is issued immediately to ensure the continuity of truck operation.

[0137] S7. Execution Monitoring and Data Feedback: This step monitors the execution process of scheduling instructions in real time, collects execution feedback data, provides data support for online iterative optimization in S8 and anomaly handling in S9, and realizes a closed loop of decision-making-execution-feedback-optimization.

[0138] 7.1 Multi-dimensional execution monitoring; Multi-dimensional execution monitoring is achieved through the mine area dispatch center monitoring platform, truck-mounted terminals, road sensors, and loading / unloading point monitoring equipment:

[0139] Truck execution status monitoring: Real-time tracking of each truck's actual driving path and optimal path. Comparison of actual driving speed and recommended driving speed Compare and track changes in remaining fuel level and load status (empty → fully loaded / fully loaded → empty), recording route deviations such as deviation points and duration. Dynamic road condition monitoring: Real-time monitoring of road condition coefficients for all road segments. Congestion coefficient Changes, focus on monitoring the optimal path If the congestion coefficient suddenly increases by more than 0.7 or the road condition coefficient suddenly decreases by less than 0.3 on the route traveled, it will be immediately marked as an abnormal state. Loading / unloading point execution status monitoring: Real-time statistics are collected on the actual waiting time of trucks at loading / unloading points and the completion time of loading / unloading operations. These are compared with preset operational efficiency indicators, and cases of waiting time exceeding 10 minutes or abnormal operational efficiency (below 80% of the preset value) are recorded. Monitoring data is collected every 5 seconds and displayed in real-time on the dispatch center platform, allowing dispatchers to monitor the overall execution status in real time.

[0140] 7.2 Feedback Data Collection and Processing; The collected feedback data is divided into three categories, preprocessed, and stored in the feedback database, while simultaneously being pushed to the online iterative optimization module of S8:

[0141] Decision execution deviation data: including path deviation distance, driving speed deviation, and decision command response delay, used to correct the model's decision adaptability. State change feedback data: including the actual comprehensive cost weight of road segments. The actual waiting queue length changes at loading and unloading points and the actual remaining fuel consumption rate of trucks are used to optimize and improve Dijkstra's dynamic weight calculation and DDPG value assessment. Task completion quality data, including single-trip task completion time, actual transportation efficiency, and actual fuel consumption, are used to verify the rationality of the reward function and the effectiveness of the model's decisions. Feedback data preprocessing includes: deviation calibration, such as removing equipment errors from driving speed deviations; data standardization, mapping to the [0,1] interval; and outlier removal, such as extreme fuel consumption data caused by equipment failure.

[0142] S8. Online Iterative Optimization; This step, based on the feedback data from S7, performs online iterative optimization of the fusion model, continuously adjusting model parameters to adapt the model to dynamic changes in the mining area, such as long-term road condition trends, work task adjustments, and truck performance degradation, ensuring the continuous optimization of scheduling decisions.

[0143] 8.1 Online Optimization Trigger Conditions; Three types of online optimization trigger conditions are set. Meeting any one of these conditions initiates the iterative optimization process: **Timed Trigger:** A full parameter optimization is performed every 2 hours, updating the model based on feedback data from the past 2 hours to avoid frequent iterations impacting real-time decision-making efficiency. **Threshold Trigger:** Emergency optimization is immediately initiated when the average transportation efficiency decreases by more than 5% over 10 consecutive decision cycles, the average driving speed deviation exceeds 8 km / h, or the average deviation between the actual comprehensive road cost weight and the model's predicted weight exceeds 0.1. **Scenario Trigger:** When significant changes occur in the mining area, such as the addition of new mining areas / spoil dumps, road closures due to construction, or a sudden increase in workload exceeding 50%, a specific optimization is manually triggered to retrain the model to adapt to the new scenario.

[0144] 8.2 Online Iterative Optimization Process; The online iterative optimization process is based on the optimal parameters trained offline and uses incremental training to avoid model performance degradation:

[0145] 1. Feedback data sampling: Extract the feedback data corresponding to the trigger conditions from the feedback database, mix it with a portion of the offline training samples in a certain proportion to form an online training sample set, with online data accounting for 30% and offline data accounting for 70%, to ensure the continuity of sample distribution.

[0146] 2. Model parameter fine-tuning: Loads the currently effective model parameters, IDQN's... DDPG and We used an online training sample set for mini-batch iterative training, with a batch size of 32 and 50 iterations. During training, the learning rate was adjusted to 1 / 5 of that used in offline training to avoid drastic parameter fluctuations.

[0147] 3. Multi-algorithm collaborative calibration: Improved Dijkstra algorithm: based on actual comprehensive road cost weights The coefficient ratios in the dynamic weight calculation formula are corrected to improve the accuracy of weight prediction. IDQN model: Based on driving speed deviation and path execution deviation data, the interaction weights in the fusion Q-value function are adjusted. The fluctuation range is ±0.05, making the Q-value assessment more closely reflect the actual execution effect. DDPG model: Based on the value feedback of actual continuous actions, the output constraints of the policy network are fine-tuned, such as the weight adjustment coefficients. The range of values ​​can be adjusted to reduce operational deviations.

[0148] 4. Optimization effect verification: Use real-time data from the last 10 decision cycles to verify the performance of the optimized model. If the optimized transportation efficiency improves by ≥3% or the deviation index decreases by ≥10%, save the new parameters and replace the currently effective parameters; otherwise, abandon this optimization and keep the original parameters unchanged.

[0149] 5. Parameter backtracking mechanism: After each optimization, historical parameters are saved. If the model performance degrades after optimization and the transportation efficiency decreases by ≥5%, the previous optimal parameters are immediately backtracked to ensure system stability.

[0150] In some embodiments, a surface fitting algorithm (SFA) is incorporated into S3 to handle the nonlinear coupling relationships of multi-dimensional dynamic features, addressing the problem that traditional linear weighted fusion cannot accurately characterize the interactive effects of dynamic factors such as road conditions, truck status, and loading / unloading point loads. It outputs two core feature fusion coefficients by performing nonlinear fitting on high-dimensional dynamic features: path weight fusion coefficients. Used to correct and improve the dynamic weight calculation of Dijkstra's algorithm, enhancing the adaptability of path generation to nonlinear features; Q-value evaluates the fusion coefficient. This is used to optimize the fusion-type Q-value function of IDQN, enhancing the correlation between discrete path value assessment and dynamic features. Simultaneously, SFA receives continuous action feedback from DDPG, weight bias feedback from improved Dijkstra's algorithm, and value assessment error feedback from IDQN, dynamically adjusting the fitting parameters to achieve bidirectional closed-loop interaction with the other three algorithms, forming a collaborative logic of nonlinear feature fusion, multi-algorithm decision correction, and feedback optimization fitting.

[0151] 1. Model building;

[0152] (1) Definition of input and output; Input feature vector Key dynamic features influencing path decisions are selected, with a dimension of 6. All features are S2 standardized and mapped to [0,1], as defined below:

[0153] ; Average road condition coefficient for all road segments within the feasible path set output by Dijkstra (improved). The weighted average of the road segments (with the weight being the length of the road segment). Average congestion coefficient of road segments (all road segments within the feasible path set) The weighted average of the road segments (with the weight being the length of the road segment). Truck load status (consistent with IDQN state vector, 0 = empty, 1 = fully loaded); Average waiting queue length at loading / unloading points; : The normalized recommended driving speed output by DDPG ( ); : Weight adjustment factor of DDPG output ( , directly use its original value.

[0154] Output fusion coefficient: : Path weight fusion coefficient, used to correct and improve Dijkstra's dynamic weights; Q-values ​​evaluate the fusion coefficients and are used to correct the fusion-type Q-value function of IDQN. The output is achieved through multivariate quadratic surface fitting, and the fitting function outputs two coefficients simultaneously to ensure the consistency of feature fusion.

[0155] (2) Multivariate quadratic surface fitting function; a multivariate quadratic surface is used as the fitting model, which takes into account both nonlinear fitting ability and computational efficiency. The function form is as follows:

[0156] ;

[0157] ( SFA is the fitting parameter for path weight fusion. For constant terms, The coefficient of the linear term, These are the coefficients of the quadratic cross term; ( SFA evaluates the fitting parameters for fusion based on Q-value; , The fitting error term follows a mean of 0 and a variance of 1. , The normal distribution represents model noise; the quadratic cross term : To characterize the nonlinear interaction between different dynamic features, such as the coupled impact of "congestion coefficient × waiting queue length" on path value.

[0158] (3) Parameter constraints and initialization; Fitting parameter constraints: coefficients of all linear terms coefficient of the quadratic term To avoid the output coefficients from being too large and exceeding the reasonable range;

[0159] Initial parameter settings: Preprocessed feature set using the least squares method Preliminary fitting was performed using historical dynamic feature data to obtain initial parameters. , Ensure the initial output coefficients , This does not affect the initial decision-making process.

[0160] 2. Model Training; The training process of SFA is deeply coupled with improved Dijkstra, IDQN, and DDPG. The training data comes from the decision results and feedback data of other algorithms. The loss function is designed to reflect the interaction error of multiple algorithms. The training process is as follows:

[0161] (1) Training data preparation; Sample set construction: from preprocessed feature set Extracting historical dynamic feature vectors The following feedback data was used to construct training samples. Optimal weighted fusion coefficient : Derived from the improved Dijkstra's actual dynamic weights and predicted dynamic weights. ,in The weight of the actual comprehensive cost of the road segment fed back by S7. To improve the prediction weights calculated by Dijkstra's original formula; optimal Q-value fusion coefficients. Derived from the ideal Q-value and actual Q-value of IDQN, ,in For the ideal target Q value, This is the output of the original Q-value function of IDQN. Sample splitting: The samples are divided into training and validation sets in a 7:3 ratio, with a total of no less than 100,000 samples covering different congestion, road conditions, and work scenarios.

[0162] (2) Loss function definition; The loss function of SFA consists of two parts: self-fitting error and multi-algorithm interaction error, ensuring that the optimization direction of the fusion coefficient is consistent with the decision objectives of other algorithms: Weighted fusion loss : Characterizes the deviation between the predicted coefficients and the optimal coefficients. , Number of samples; Q-value fusion loss : Characterizes the deviation of the Q-value fusion coefficient, Interaction loss : Characterizes the impact of fusion coefficients on other algorithms' errors. ,in For IDQN's TD error, To improve Dijkstra's weight error; interactive weights. The importance of balancing self-fitting and multi-algorithm interaction.

[0163] (3) Training steps are synchronized with the interaction of multiple algorithms;

[0164] Parameter initialization: Load initial fitting parameters , Initialize the experience replay pool Capacity of 50,000 records, storage samples .

[0165] Batch sampling: from 32 samples were randomly selected from the data to form a batch training dataset.

[0166] Gradient descent optimization: Minimizing the loss function using the Adam optimizer. Update the fitted parameters:

[0167] ;in For SFA learning rate, , Loss function pairs , The gradient.

[0168] Multi-algorithm interactive feedback: After every 20 iterations, the TD error fed back from IDQN is received. :like This indicates a large bias in the Q-value assessment, necessitating a temporary increase. The weight is multiplied by 1.2, and optimization is prioritized. Every 30 iterations, receive the weight error from the improved Dijkstra feedback. :like Temporarily increase The weight is multiplied by 1.2, and optimization is prioritized. ;

[0169] Every 50 iterations, receive continuous action deviation feedback from DDPG. :like This indicates that the dynamic feature fitting is insufficient. To improve the nonlinear fitting ability, the gradient weight of the quadratic cross term is increased by multiplying it by 1.1.

[0170] Convergence criteria: The validation set satisfies the following metrics for 5 consecutive rounds:

[0171] Weighted fusion coefficient error ( (Number of validation set samples); Q-value fusion coefficient error Interaction loss Once the conditions are met, save the optimal fitting parameters. , .

[0172] 3. Model Application; SFA is applied synchronously with the real-time decision-making cycle, every 10 seconds. Its core function is to generate fusion coefficients and interact with other algorithms. The process is as follows:

[0173] (1) Real-time feature input and coefficient generation: Extract dynamic features from the real-time preprocessed data of S5 and construct the real-time input vector. Ensure the feature dimensions are consistent with those during the training phase; load the optimal fitting parameters. , Substitute the values ​​into the fitting function to calculate the initial fusion coefficients:

[0174] Coefficient constraint adjustment: If Then set it to 0.9, if Then set it to 1.1; Set it to 0.8. Set it to 1.2 to avoid over-correction.

[0175] (2) Bidirectional interaction with other algorithms; SFA corrects the core formulas of other algorithms by outputting fusion coefficients, and at the same time receives feedback data from other algorithms to optimize its own parameters, forming 4 sets of bidirectional interaction relationships:

[0176] SFA↔ Improved Dijkstra:

[0177] Positive interaction, SFA → Improved Dijkstra: By embedding an improved Dijkstra's dynamic weight formula, the overall cost of road segments is corrected. ;in The weight adjustment coefficients for DDPG output. The nonlinear feature fusion coefficients output by SFA are used together to correct the weights, so that path generation simultaneously responds to continuous action optimization and nonlinear feature coupling.

[0178] Reverse interaction, improving Dijkstra's approach to SFA: improving Dijkstra's calculation of the actual weights of road segments. With the corrected prediction weights deviation Feedback is sent to SFA to update the fitted parameters: ;in To provide feedback on the learning rate, if Then, by adjusting the coefficient of the quadratic term... Enhance the fitting accuracy of corresponding feature interactions.

[0179] SFA↔IDQN:

[0180] Positive interaction, SFA→IDQN: will Embedding IDQN in a fusion-based Q-value function enhances the sensitivity of value assessment to nonlinear features: ;in By scaling up or down the contribution of continuous action characteristics to the Q-value, the value assessment can simultaneously consider both linear continuous action and nonlinear dynamic characteristics.

[0181] Reverse interaction, IDQN→SFA: IDQN will convert TD error Feedback is sent to the SFA to correct the fitting parameters of the Q-value fusion coefficients: ;like This indicates a large deviation in the Q-value assessment. Adjustments can be made to address this. Optimize feature fusion accuracy.

[0182] SFA↔DDPG:

[0183] Forward interaction, SFA→DDPG: Input features of SFA (DDPG recommended normalized driving speed values) , so that the fitting coefficient , Strongly correlated with continuous action; simultaneously, SFA output As a new component in the DDPG state vector, it optimizes continuous action decisions: ;in This reflects the effect of nonlinear feature fusion and helps DDPG adjust the weight adjustment coefficients more accurately. .

[0184] Reverse interaction, DDPG→SFA: DDPG calculates the improvement rate of transportation efficiency after continuous action optimization. , To optimize efficiency, For baseline efficiency, feedback is given to the SFA: ,but , Temporarily reduce the correction range of the fusion coefficient to avoid overfitting that could lead to failure of continuous action optimization.

[0185] SFA↔ Three-Algorithm Collaboration: SFA, as the core of nonlinear feature fusion, participates in the closed-loop interaction of the three algorithms: the optimal path Q-value output by IDQN. Modify the DDPG value network; DDPG output The Dijkstra weights are modified and improved, the path set of the Dijkstra output is improved, and the IDQN action space is corrected, while SFA is improved through... , This provides nonlinear feature support for each interaction step, ultimately forming a four-way closed-loop collaborative mechanism of improved Dijkstra → IDQN → DDPG → SFA → improved Dijkstra, with the core collaborative formula as follows:

[0186] ;in To improve Dijkstra's basic weight function, For the continuous action characteristic function of IDQN, This is the basic policy function for DDPG. The four functions are nested together and their parameters are passed between each other, achieving deep synergy between discrete-continuous decision-making and nonlinear feature fusion.

[0187] 4. Core Contributions; Nonlinear Feature Fusion Capability: Through multivariate quadratic surface fitting, it solves the coupling influence of dynamic features such as "road conditions-congestion-loading / unloading" that traditional linear weighting cannot characterize, making path weights and value assessments more closely aligned with real-world scenarios; Multi-Algorithm Collaborative Enhancement: As a "feature fusion hub" in a four-way closed loop, it provides nonlinear correction coefficients for other algorithms while receiving multi-dimensional feedback to optimize itself, improving the adaptability of the entire fusion model to dynamic scenarios; Improved Decision Accuracy: Through Adjust path weights to reduce the generation of invalid paths; through By revising the Q-value assessment, the probability of misjudging the optimal path is reduced, indirectly improving transportation efficiency and reducing fuel consumption.

Claims

1. A dynamic path scheduling optimization method for open-pit mine trucks based on deep reinforcement learning, characterized in that, Includes the following steps: S1. Construct a static basic dataset and a dynamic real-time dataset; S2. Based on the static basic dataset and the dynamic real-time dataset, construct a preprocessed data feature set and scheduling environment model; S3. Based on the preprocessed data feature set and scheduling environment model, a deep reinforcement learning model is constructed that integrates the improved Dijkstra algorithm, the improved deep Q network, and the deep deterministic policy gradient. The improved Dijkstra algorithm is used to generate a dynamic feasible path set, the improved deep Q network is used to evaluate the value of each path in the dynamic feasible path set and output the value of the optimal path, and the deep deterministic policy gradient is used to output the path weight adjustment coefficient. S4. Using historical feature data and scheduling environment model from the preprocessed data feature set, the deep reinforcement learning model is trained offline to obtain the optimal model parameters. During the training process, the dynamic feasible path set generated by the improved Dijkstra algorithm is used as the discrete action space of the improved deep Q network. The optimal path value output by the improved deep Q network is fed back to the deep deterministic policy gradient. The path weight adjustment coefficient output by the deep deterministic policy gradient feeds back into the improved Dijkstra algorithm to correct the dynamic weight of the road. S5. Collect real-time data according to a preset cycle and perform preprocessing to obtain real-time preprocessed data; S6. Input the real-time preprocessed data into the deep reinforcement learning model with the optimal model parameters, generate the real-time optimal path and continuous action instructions, and send them to the truck; The three-algorithm bidirectional interaction mechanism of the deep reinforcement learning model in S3 is as follows: After the improved deep Q network outputs the optimal path value, it is fed back to the deep deterministic policy gradient to correct its value evaluation benchmark, and to the improved Dijkstra algorithm to dynamically adjust the screening threshold of the feasible path set. The path weight adjustment coefficients output by the deep deterministic policy gradient are simultaneously transmitted to the improved Dijkstra algorithm to correct the dynamic weights of the roads and to the improved deep Q network to optimize the feature dimensions of path value evaluation, forming a closed-loop interaction of the three algorithms. The specific process of generating a dynamic feasible path set in the improved Dijkstra algorithm in S3 is as follows: combining the preprocessed data feature set and the path weight adjustment coefficient of the gradient output of the deep deterministic strategy, the dynamic comprehensive weight of each road segment is calculated; based on the dynamic comprehensive weight, a dynamic feasible path set containing the shortest path and multiple suboptimal paths is selected, and the size of the path set is dynamically adjusted according to the current workload to ensure coverage of the main feasible routes. The path value evaluation mechanism of the improved deep Q-network in S3 is as follows: a multi-dimensional state vector is constructed using the path features extracted by the improved Dijkstra algorithm, the truck load status and loading / unloading point queuing data in the preprocessed data feature set, and the path weight adjustment coefficients output by the gradient of the deep deterministic strategy as input; a Q-value function that integrates dynamic features is used to calculate the static value of the path and the real-time dynamic influence factors in a weighted manner, and the quantitative value score of each path is output, with the optimal path value being the value data corresponding to the path with the highest score; The output constraint and optimization mechanism of the deep deterministic policy gradient in S3 is as follows: the value range of the path weight adjustment coefficient matches the actual weight fluctuation law of the road, and takes effect after being verified by the road physical constraints in the scheduling environment model, so as to avoid the path sudden change caused by excessive weight adjustment. The value network uses the optimal path value output by the improved deep Q network as a reference benchmark to correct its own evaluation result of the path weight adjustment effect. The policy network iteratively optimizes the output accuracy of the adjustment coefficient based on the corrected evaluation result.

2. The method for dynamic path scheduling optimization of open-pit mine trucks based on deep reinforcement learning according to claim 1, characterized in that, It also includes S7: real-time monitoring of the execution process of the real-time optimal path and continuous action instructions, and collection of execution feedback data including path execution deviation, actual road conditions, and task completion quality.

3. The method for dynamic path scheduling optimization of open-pit mine trucks based on deep reinforcement learning according to claim 2, characterized in that, It also includes S8: Based on execution feedback data, adjust the road dynamic weight calculation rules, path value evaluation function and continuous action constraints in the deep reinforcement learning model, perform online iterative optimization of model parameters, and form a closed-loop scheduling process of data collection-preprocessing-model training-real-time decision-execution feedback-parameter optimization.

4. The method for dynamic path scheduling optimization of open-pit mine trucks based on deep reinforcement learning according to claim 1, characterized in that, The specific optimization process for offline training in S4 is as follows: the historical feature data in the preprocessed data feature set is divided into training set and validation set according to time series. During training, batch training mode is adopted, and the model performance is verified by the validation set after each batch training. During the training iteration, the TD error and policy loss value of the deep deterministic policy gradient of the improved deep Q network are monitored in real time. When the TD error stabilizes in the preset range, the policy loss value tends to converge, and the path planning accuracy on the validation set reaches the preset standard in multiple consecutive iterations, training is stopped and the optimal model parameters are saved.

5. The method for dynamic path scheduling optimization of open-pit mine trucks based on deep reinforcement learning according to claim 1, characterized in that, The preprocessing operations for real-time data in S5 include: removing outliers from the collected real-time data and removing invalid data that exceeds the physical reasonable range; filling missing data by interpolation of adjacent period data, and controlling the deviation of the filled data within a preset range; calculating the real-time feature dimension based on the filled valid data; aligning the calculated real-time features with the feature dimension of the preprocessed data feature set to generate real-time preprocessed data with a uniform format.

6. The method for dynamic path scheduling optimization of open-pit mine trucks based on deep reinforcement learning according to claim 1, characterized in that, The static basic dataset in S1 includes road topology, road physical parameters, truck performance parameters, and basic parameters of the operation task, while the dynamic real-time dataset includes road status data, truck operation data, and loading / unloading point status data.

Images