Intelligent dispatching method and system for underground mining truck fleet
By employing graph attention networks and reinforcement learning methods based on continuous action spaces, the problem of topological complexity in the scheduling of truck fleets in underground mines was solved, enabling smooth and efficient vehicle scheduling and improving transportation efficiency and energy consumption management.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- UNIV OF SCI & TECH BEIJING
- Filing Date
- 2025-12-16
- Publication Date
- 2026-05-08
AI Technical Summary
Existing methods for dispatching truck fleets in underground mines are ill-suited to effectively handle complex topologies such as "one-way, two-way, single-channel" and "passing chambers," resulting in rigid vehicle control, high energy consumption, significant mechanical wear, and an inability to make precise decisions.
By employing graph attention networks and reinforcement learning methods with continuous action spaces, and combining them with the dynamic window method to design a shaping reward function, a continuous acceleration command for the agent is constructed to achieve smooth and efficient vehicle scheduling.
It improves transportation efficiency, reduces traffic conflicts, lowers energy consumption, achieves smoother and more refined vehicle control decisions, and enhances the intelligence level of fleet management.
Smart Images
Figure CN121998292A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of underground mine transportation scheduling technology, and in particular to an intelligent scheduling method and system for underground mining truck fleets based on graph attention networks and continuous action reinforcement learning. Background Technology
[0002] With the development of modern mining technology, intelligentization and automation have become core trends in mine operations. In this process, underground mines, due to their complex operating environment, place extremely high demands on the coordinated operation of multiple pieces of equipment. The production cycle of underground mining involves multiple closely linked stages such as mining, excavation, transportation, and hoisting. Among these, the transportation system is the key lifeline connecting each production stage and ensuring production continuity.
[0003] Underground mining truck fleets are the main force responsible for transportation tasks. However, unlike the network structure of open-pit mines or traditional manufacturing, the transportation network of underground mines has a unique topology: it mainly consists of narrow, long, single-lane, two-way roadways with extremely limited space. To solve the problem of two-way passing, a limited number of passing chambers are set up in the roadways. At the same time, the main roadway and branch roads leading to different loading nodes form multiple intersection nodes. This "long, straight, and narrow" topology, which relies on specific passing points, makes vehicle scheduling extremely complex, and it is prone to traffic conflicts, operational deadlocks, and efficiency bottlenecks caused by ineffective waiting.
[0004] Existing research on underground mining truck fleet management, such as vehicle-by-vehicle scheduling or global scheduling strategies, often relies on static rules or traditional operations research optimization, which struggles to effectively address real-time fluctuations in transportation tasks and dynamic environmental changes. These methods fail to fully leverage the potential of modern autonomous mining trucks, especially intelligent vehicles capable of accurately responding to scheduling commands and flexibly adjusting their speeds.
[0005] In recent years, reinforcement learning (RL), as a powerful machine learning method, has demonstrated significant advantages in solving complex sequential decision-making problems. Reinforcement learning allows agents to learn optimal policies through trial and error in their interactions with the environment, aiming to maximize cumulative rewards (such as maximizing transportation efficiency or minimizing waiting time). This self-learning and policy iteration characteristic makes it well-suited for handling highly dynamic and uncertain complex problems such as underground mine scheduling.
[0006] While reinforcement learning has shown promise in scheduling, existing research primarily focuses on mesh-structured scheduling in open-pit mines or general-purpose automated guided vehicles (AGVs). These scenarios have relatively simple topologies and a limited range of conflict types. Currently, no research has explored in depth how to apply reinforcement learning to solve the complex scheduling challenges unique to underground mines, such as "single-lane, two-way, single-passage" and "passing chambers."
[0007] More importantly, in the few existing attempts at reinforcement learning scheduling, the action space design generally employs discretization. Specifically, the agent's decisions are restricted to a few high-level, discrete instructions, such as "go forward," "stop," "enter the chamber," or "wait."
[0008] This discrete action space design has the following drawbacks: vehicle control is black-and-white. For example, when meeting oncoming traffic, the agent can only choose between "complete stop" or "full speed ahead." This "abrupt start and stop" control method not only results in abrupt and uneven operation but also wastes a significant amount of time. In many scenarios where a slight deceleration and passing could be achieved, discrete control forces the vehicle to wait unnecessarily for extended periods. Moreover, this leads to high vehicle energy consumption and significant wear and tear; frequent complete stops and restarts are the main sources of vehicle energy consumption and also exacerbate mechanical wear and tear on the vehicle (such as the engine and braking system).
[0009] Furthermore, discrete actions cannot make refined decisions or handle sophisticated interactions. For example, it cannot express decisions that are closer to human driving intelligence, such as "decelerate at 70% speed" or "slightly accelerate to get through the intersection before the other vehicle," resulting in stiff and inefficient vehicle control. Summary of the Invention
[0010] To address the problem of rigidity in traditional discrete motion control, this invention provides an intelligent scheduling method and system for underground mining truck fleets. By introducing a continuous motion space and combining graph attention networks and dynamic window method for shaping rewards, intelligent, smooth, and efficient scheduling of underground mining truck fleets can be achieved.
[0011] To solve the above-mentioned technical problems, the present invention provides the following technical solution: On the one hand, an intelligent dispatching method for underground mining truck fleets is provided, the method comprising the following steps: S1. Construct a scenario model of the underground mine transportation system and obtain the topology of the underground mine; S2. Design the observation space to obtain the physical and logical states of each vehicle, as well as the global task state; S3. A graph attention network is used as a feature extractor for reinforcement learning. Adjacency relationships are dynamically constructed based on the observed spatial state data, and neighbor node information is aggregated through the attention mechanism to extract collaborative features for input to the policy network. S4. A policy network is constructed based on a reinforcement learning algorithm with a continuous action space. The agent acquires the current cooperative features at each time step and outputs continuous actions. The action space of the agent is defined as a continuous acceleration command. S5. Design a shaping reward function inspired by the dynamic window method to guide the agent's learning and achieve optimized vehicle scheduling; wherein, the shaping reward function includes a distance penalty term and a speed reward term.
[0012] Optionally, in step S1, the topology of the underground mine includes: loading nodes, unloading nodes, two-way single-vehicle road sections, passing chambers for passing vehicles, and lane intersection nodes.
[0013] Optionally, in step S2, the physical state of each vehicle includes: normalized vehicle position and normalized vehicle speed. The logical state of each vehicle includes: whether it is loaded, whether it has been assigned a task, and its role in intersection conflicts; where the role in intersection conflicts refers to whether the vehicle plays the role of a "yielding vehicle" or a "first-mover vehicle" in an intersection conflict, as well as the distance and speed of the other vehicle; The global task status includes: the normalized value of the remaining task quantity at each load point.
[0014] Optionally, in step S3, a graph attention network is used as a feature extractor. At each time step, the distance matrix between vehicles is calculated based on the real-time positions of all vehicles, and the adjacency matrix and edge index are dynamically constructed based on the preset perception radius. Using the vehicle's own state features as node features, the input is processed into a multi-layer graph attention convolution. The graph attention mechanism enables the agent to dynamically evaluate the importance of different neighboring vehicles, and aggregates weighted information based on importance to obtain graph embedding features. The graph embedding features are concatenated with the global task state features of the observation space to obtain the collaborative features.
[0015] Optionally, the vehicle's own state features include physical state features and logical state features. First, they are embedded through a linear layer. Then, the embedded data and edge indices are fed into two GAT convolutional layers. A dropout layer is added between the first and second GAT convolutional layers. The output of the second GAT convolutional layer is obtained by global mean pooling to get graph embedding features. Then, it is concatenated with global task state features to obtain high-dimensional collaborative features through the output feature layer.
[0016] Optionally, in step S4, the action space of the agent is defined as a continuous and standardized acceleration command. The agent outputs a specific acceleration value at each decision step, which can be directly substituted into the vehicle's physical kinematics formula in the environment.
[0017] Optionally, in step S5, the distance penalty term is used to ensure safety, specifically including: calculating the minimum distance d_obs between the current vehicle and other vehicles, and when d_obs is less than the perception range, applying a penalty that is inversely proportional to the minimum distance; The speed bonus is used to encourage efficiency, specifically by giving a positive reward based on the ratio of the vehicle's current speed to its maximum speed, self.max_speed.
[0018] On the other hand, an intelligent dispatching system for underground mining truck fleets is provided for implementing the method described in any of the above claims, the system comprising: The scene construction module is used to build a scene model of the underground mine transportation system and obtain the topology of the underground mine. The state acquisition module is used to design the observation space, acquire the physical and logical states of each vehicle, as well as the global task state; Feature extractors are used to process observed spatial state data; Among them, a graph attention network is used as a feature extractor for reinforcement learning. Adjacency relationships are dynamically constructed based on the observed spatial state data, and neighbor node information is aggregated through the attention mechanism to extract collaborative features for input policy network. The policy network is used to make decisions based on the output of the feature extractor, and outputs a series of actions. Among them, a policy network is constructed based on a reinforcement learning algorithm with a continuous action space. The agent acquires the current cooperative features at each time step and outputs continuous actions. The action space of the agent is defined as a continuous acceleration command. The reward function design module is used to design a shaping reward function inspired by the dynamic window method to guide the agent's learning and achieve optimized vehicle scheduling; wherein, the shaping reward function includes a distance penalty term and a speed reward term.
[0019] On the other hand, an electronic device is provided, the electronic device comprising: processor; The memory stores computer-readable instructions, which, when loaded and executed by the processor, implement the steps of the intelligent dispatching method for underground mining truck fleets described above.
[0020] On the other hand, a computer-readable storage medium is provided, wherein program code is stored in the computer-readable storage medium, and the program code can be invoked by a processor to execute the steps of the intelligent scheduling method for underground mining truck fleet described above.
[0021] The beneficial effects of the technical solution provided by this invention include at least the following: In this embodiment of the invention, the action space of the agent is defined as a continuous set of acceleration commands, enabling the vehicle to perform smooth and precise dynamic control. To achieve this goal, on the one hand, a graph attention network is constructed as a feature extractor, enabling it to dynamically learn the topological relationships and interaction intentions between vehicles in complex environments; on the other hand, a shaping reward function inspired by the dynamic window method is designed to guide the agent in learning how to balance speed and safe distance in the continuous action space during training, thereby optimizing its driving strategy. Case studies have verified that this method can effectively improve transportation efficiency, reduce conflicts, and achieve efficient and smooth continuous vehicle control. Attached Figure Description
[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0023] Figure 1 This is a flowchart of an intelligent dispatching method for an underground mining truck fleet provided by an embodiment of the present invention; Figure 2 This is a schematic diagram of a transportation system provided in an embodiment of the present invention; Figure 3 This is a topology diagram of the working surface of the transportation system provided in an embodiment of the present invention; Figure 4 This is a flowchart illustrating the working principle of the graph attention network provided in this embodiment of the invention. Figure 5 This is a Markov decision-making flowchart for continuous actions provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of an intelligent dispatching system for underground mining truck fleet provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0025] This invention provides an intelligent dispatching method for underground mining truck fleets. This method can be implemented using electronic equipment, which can be a terminal or a server. Figure 1 As shown, the processing flow of this method may include the following steps:
[0026] S1. Construct a scenario model of the underground mine transportation system and obtain the topology of the underground mine.
[0027] refer to Figure 2 As shown, this invention focuses on underground metal mine transportation systems, which mainly consist of ore bodies, mining faces, central ore passes, loading-transporting-unloading machinery, underground mining trucks, crushers, underground ore bins, hoisting equipment, etc. For ease of analysis, the complex transportation system is simplified into a topological model, such as... Figure 3 As shown, the environment includes loading and unloading nodes, two-way single-vehicle road sections (including main road sections and branch road sections), passing chambers, and road intersections (lane intersection nodes). Main road sections such as S1-S3-S5-S7-S9 are relatively long, reaching several hundred to over a thousand meters; branch road sections such as S6, S8, and S10 are shorter. The operational nodes include loading and unloading nodes, while non-operational nodes include passing chambers (S2, S4) and road intersections (intersections between main road sections and branch road sections). This environment is used for the training and interaction of reinforcement learning agents.
[0028] S2. Design the observation space to obtain the physical and logical states of each vehicle, as well as the global task state.
[0029] For each vehicle, the physical state includes: normalized vehicle position and normalized vehicle speed; the logical state includes: whether it is loaded, whether it is assigned a task, and the role of the vehicle in intersection conflict.
[0030] In this embodiment of the invention, to handle intersection conflicts, the observation space specifically includes a "role" feature. The intersection conflict role refers to whether a vehicle plays the role of a "yielding vehicle" or a "first-mover vehicle" in an intersection conflict, as well as the distance and speed of the other vehicle.
[0031] The global task status includes: the normalized value of the remaining task quantity at each load point.
[0032] S3. A graph attention network is used as a feature extractor for reinforcement learning. Adjacency relationships are dynamically constructed based on the observed spatial state data, and neighbor node information is aggregated through the attention mechanism to extract collaborative features for input policy network.
[0033] To make intelligent decisions in a complex, continuous action space, the agent must have a deep understanding of the dynamic environment of multi-vehicle interactions. This invention employs a Graph Attention Network (GAT) as a feature extractor for deep reinforcement learning (RL) to handle the dynamic interaction relationships between vehicles. In this feature extractor, at each time step, the system calculates the distance matrix between vehicles based on the real-time positions of all vehicles, and dynamically constructs an adjacency matrix and edge index based on a preset perception radius. The vehicle's own features (including physical and logical state features, such as speed and whether it is loaded) are used as node features and input into a multi-layer graph attention convolution. The graph attention mechanism enables the agent to dynamically evaluate the importance of different neighboring vehicles and aggregate weighted information according to importance to obtain a highly condensed graph embedding feature. The graph embedding feature vector represents the agent's understanding of the current global traffic situation. It is then concatenated with the global task state features in the observation space to obtain a high-dimensional collaborative feature, which serves as the input to the reinforcement learning algorithm's policy network, providing a high-quality basis for its continuous action decisions.
[0034] like Figure 4 As shown, the flattened observation state data is input into the feature extractor. The system first parses the position of each vehicle; then, it calculates the distance matrix between vehicles and dynamically constructs an adjacency matrix and edge index based on a preset sensing radius. This achieves... Figure 4 The steps involved in "calculating vehicle positions" and "constructing adjacency matrices" are as follows. The vehicle's own features are used as node features, and are first embedded through a linear layer (corresponding to...). Figure 4 (The "node embedding layer" is mentioned here). Subsequently, the embedded data and edge indices are fed into two GAT convolutional layers, with a dropout layer added between the first and second GAT convolutional layers (corresponding to...). Figure 4 In the "Dropout" (or "Dropout"), the output of the second GAT convolutional layer is processed by global mean pooling (corresponding to...). Figure 4 The graph embedding feature is obtained by "global mean pooling". This graph embedding feature is then concatenated with the global state features in the observation space (such as the remaining task amount) (corresponding to...). Figure 4 (This involves "concatenating global features"). Finally, the concatenated vector is passed through the output feature layer to obtain a high-dimensional collaborative feature vector (corresponding to...). Figure 4 The "output feature vector" will be used for subsequent policy and value calculations.
[0035] S4. A policy network is constructed based on a reinforcement learning algorithm with a continuous action space. The agent acquires the current cooperative features at each time step and outputs continuous actions, where the agent's action space is defined as a continuous acceleration command.
[0036] Here, the action space of the agent is defined as a continuous, standardized acceleration command. At each decision step, the agent outputs no longer a "category" but a specific acceleration value, which can be directly substituted into the vehicle's physical kinematics formula in the environment.
[0037] This design enables vehicles to perform smooth, precise dynamic speed adjustments, such as "slightly decelerating to avoid an obstacle" or "smoothly accelerating to pass," rather than the traditional discrete actions of "abrupt starts and stops." This significantly improves transportation efficiency, reduces energy consumption, and makes the driving behavior of intelligent agents more closely resemble the real physical world.
[0038] S5. Design a shaping reward function inspired by the dynamic window method to guide the agent's learning and achieve optimized vehicle scheduling; wherein, the shaping reward function includes a distance penalty term and a speed reward term.
[0039] To guide the agent in learning within a continuous action space, this invention designs a shaping reward inspired by the Dynamic Window Method (DWA). This reward is calculated in real-time within the environment and aims to guide the agent to balance efficiency and safety. Specifically, it includes: Distance penalty (for safety): Calculate the minimum distance d_obs between the current vehicle and other vehicles. When d_obs is less than the perception range, apply a penalty that is inversely proportional to the minimum distance. This penalty strongly inhibits consecutive actions that lead to dangerous approach.
[0040] Speed Bonus (used to encourage efficiency): A positive reward is given based on the ratio of the vehicle's current speed to its maximum speed self.max_speed.
[0041] This shaping reward system forces the agent to continuously solve an optimization problem during the learning process: how to maximize efficiency rewards while minimizing risk. This effectively guides the agent to find efficient and safe driving strategies in the continuous action space.
[0042] In addition, the reward function also includes rewards for successful task completion, rewards for successful subtask completion, collision penalties, and timeout penalties.
[0043] The scheduling method of the present invention follows Figure 5 The Markov decision process framework is shown. At each time step, the agent obtains the current "observed state" from the environment and outputs a "continuous action". After the environment executes the action, it generates a "new state" and provides a "reward". The agent learns and adjusts its policy based on the reward.
[0044] To address the abrupt control issues caused by discrete actions, this invention employs a continuous action space, defined as the continuous acceleration values corresponding to each truck. These output continuous acceleration values are then directly used in the vehicle's physics update formula. This design enables smooth acceleration and deceleration control of the vehicle.
[0045] Accordingly, embodiments of the present invention also provide an intelligent dispatching system for underground mining truck fleets. Figure 6 This is a structural block diagram illustrating an intelligent dispatching system for an underground mining truck fleet, according to an exemplary embodiment. Figure 6 As shown, the system includes:
[0046] Scene construction module 201 is used to construct a scene model of an underground mine transportation system and obtain the topology of the underground mine. The state acquisition module 202 is used to design the observation space, acquire the physical and logical states of each vehicle, and the global task state; Feature extractor 203 is used to process observed spatial state data; Among them, a graph attention network is used as a feature extractor for reinforcement learning. Adjacency relationships are dynamically constructed based on the observed spatial state data, and neighbor node information is aggregated through the attention mechanism to extract collaborative features for input policy network. Policy network 204 is used to make decisions based on the output of the feature extractor and output continuous actions; Among them, a policy network is constructed based on a reinforcement learning algorithm with a continuous action space. The agent acquires the current cooperative features at each time step and outputs continuous actions. The action space of the agent is defined as a continuous acceleration command. The reward function design module 205 is used to design a shaping reward function inspired by the dynamic window method to guide the agent to learn and achieve optimized vehicle scheduling; wherein, the shaping reward function includes a distance penalty term and a speed reward term.
[0047] For ease of explanation, Figure 6 Only the main components of the system are shown. The system of this embodiment can be used to perform... Figure 1 The technical solutions of the method embodiments shown are similar in principle and in effect, and will not be described again here.
[0048] This invention provides an intelligent scheduling method and system for underground mining truck fleets based on graph attention networks and continuous action reinforcement learning. The method defines the agent's action space as continuous acceleration commands, enabling vehicles to perform smooth and precise dynamic control. To achieve this, a graph attention network is constructed as a feature extractor, allowing it to dynamically learn the topological relationships and interaction intentions between vehicles in complex environments. Furthermore, a shaping reward function inspired by the dynamic window method is designed to guide the agent in learning how to balance speed and safe distance in the continuous action space during training, thereby optimizing its driving strategy. Case studies have verified that this method can effectively improve transportation efficiency, reduce conflicts, and achieve efficient and smooth continuous vehicle control.
[0049] In an exemplary embodiment, the present invention also provides an electronic device, the electronic device comprising: processor; The memory stores computer-readable instructions, which, when loaded and executed by the processor, implement the steps of the intelligent dispatching method for underground mining truck fleets described above.
[0050] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 7 As shown, the electronic device 300 may include a first processor 3001 and a memory 3002. Optionally, the electronic device 300 may also include a transceiver 3003. The first processor 3001, memory 3002, and transceiver 3003 may be connected via a communication bus. The memory 3002 stores computer-readable instructions, which, when executed by the first processor 3001, implement the steps of the intelligent dispatching method for underground mining truck fleets described above.
[0051] In a specific implementation, as one example, the first processor 3001 may include one or more CPUs, for example... Figure 7 CPU0 and CPU1 are shown in the diagram.
[0052] In a specific implementation, as one example, the electronic device 300 may also include multiple processors, for example... Figure 7 The first processor 3001 and the second processor 3004 are shown in the diagram. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).
[0053] The memory 3002 is used to store the software program that executes the present invention, and is controlled by the first processor 3001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.
[0054] The transceiver 3003 is used to communicate with network devices or with terminal devices.
[0055] Optionally, the transceiver 3003 may include a receiver and a transmitter. The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.
[0056] Optionally, the transceiver 3003 can be integrated with the first processor 3001 or exist independently and be coupled to the first processor 3001 through the interface circuit of the electronic device 300. This embodiment of the invention does not specifically limit this.
[0057] It should be noted that, Figure 7 The structure of the electronic device 300 shown is not intended to limit the electronic device. Actual electronic devices may include more or fewer components than shown, or combine certain components, or have different component arrangements. Furthermore, the technical effects of the electronic device 300 can be understood by referring to the technical effects of the above-described method embodiments, and will not be repeated here.
[0058] In an exemplary embodiment, the present invention also provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the steps of the intelligent scheduling method for underground mining truck fleets described above. For example, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc.
[0059] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.
[0060] The use of terms such as "an embodiment," "an embodiment," "an exemplary embodiment," and "some embodiments" in the specification indicates that the described embodiment may include a specific feature, structure, or characteristic, but not every embodiment necessarily includes that specific feature, structure, or characteristic. Furthermore, when a specific feature, structure, or characteristic is described in connection with an embodiment, implementing such a feature, structure, or characteristic in conjunction with other embodiments (whether explicitly described or not) should be within the knowledge of those skilled in the art.
[0061] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0062] In the embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0063] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0064] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0065] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0066] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.
[0067] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for intelligent dispatching of underground mining truck fleets, characterized in that, Includes the following steps: S1. Construct a scenario model of the underground mine transportation system and obtain the topology of the underground mine; S2. Design the observation space to obtain the physical and logical states of each vehicle, as well as the global task state; S3. A graph attention network is used as a feature extractor for reinforcement learning. Adjacency relationships are dynamically constructed based on the observed spatial state data, and neighbor node information is aggregated through the attention mechanism to extract collaborative features for inputting into the policy network. S4. A policy network is constructed based on a reinforcement learning algorithm with a continuous action space. The agent acquires the current cooperative features at each time step and outputs continuous actions. The action space of the agent is defined as a continuous acceleration command. S5. Design a shaping reward function inspired by the dynamic window method to guide the agent's learning and achieve optimized vehicle scheduling; wherein, the shaping reward function includes a distance penalty term and a speed reward term.
2. The intelligent dispatching method for underground mining truck fleets according to claim 1, characterized in that, In step S1, the topology of the underground mine includes: loading nodes, unloading nodes, two-way single-vehicle road sections, passing chambers for passing vehicles, and lane intersection nodes.
3. The intelligent dispatching method for underground mining truck fleets according to claim 1, characterized in that, In step S2, the physical state of each vehicle includes: normalized vehicle position and normalized vehicle speed. The logical state of each vehicle includes: whether it is loaded, whether it has been assigned a task, and its role in intersection conflicts; where the role in intersection conflicts refers to whether the vehicle plays the role of a "yielding vehicle" or a "first-mover vehicle" in an intersection conflict, as well as the distance and speed of the other vehicle; The global task status includes: the normalized value of the remaining task quantity at each load point.
4. The intelligent dispatching method for underground mining truck fleets according to claim 1, characterized in that, In step S3, a graph attention network is used as a feature extractor. At each time step, the distance matrix between vehicles is calculated based on the real-time positions of all vehicles, and the adjacency matrix and edge index are dynamically constructed based on the preset perception radius. Using the vehicle's own state features as node features, the input is processed into a multi-layer graph attention convolution. The graph attention mechanism enables the agent to dynamically evaluate the importance of different neighboring vehicles, and aggregates weighted information based on importance to obtain graph embedding features. The graph embedding features are concatenated with the global task state features of the observation space to obtain the collaborative features.
5. The intelligent dispatching method for underground mining truck fleets according to claim 4, characterized in that, The vehicle's own state features include physical state features and logical state features. First, they are embedded through a linear layer. Then, the embedded data and edge indices are fed into two GAT convolutional layers. A dropout layer is added between the first and second GAT convolutional layers. The output of the second GAT convolutional layer is obtained by global mean pooling to get graph embedding features. Then, it is concatenated with global task state features to obtain high-dimensional collaborative features through the output feature layer.
6. The intelligent dispatching method for underground mining truck fleets according to claim 1, characterized in that, In step S4, the action space of the agent is defined as a continuous and standardized acceleration command. The agent outputs a specific acceleration value at each decision step, which can be directly substituted into the vehicle's physical kinematics formula in the environment.
7. The intelligent dispatching method for underground mining truck fleets according to claim 1, characterized in that, In step S5, the distance penalty term is used to ensure safety, specifically including: calculating the minimum distance d_obs between the current vehicle and other vehicles; when d_obs is less than the perception range, applying a penalty that is inversely proportional to the minimum distance. The speed bonus is used to encourage efficiency, specifically by giving a positive reward based on the ratio of the vehicle's current speed to its maximum speed, self.max_speed.
8. An intelligent dispatching system for underground mining truck fleets, used to implement the method as described in any one of claims 1 to 7, characterized in that, The system includes: The scene construction module is used to build a scene model of the underground mine transportation system and obtain the topology of the underground mine. The state acquisition module is used to design the observation space, acquire the physical and logical states of each vehicle, as well as the global task state; Feature extractors are used to process observed spatial state data; Among them, a graph attention network is used as a feature extractor for reinforcement learning. Adjacency relationships are dynamically constructed based on the observed spatial state data, and neighbor node information is aggregated through the attention mechanism to extract collaborative features for input policy network. The policy network is used to make decisions based on the output of the feature extractor, and outputs a series of actions. Among them, a policy network is constructed based on a reinforcement learning algorithm with a continuous action space. The agent acquires the current cooperative features at each time step and outputs continuous actions. The action space of the agent is defined as a continuous acceleration command. The reward function design module is used to design a shaping reward function inspired by the dynamic window method to guide the agent's learning and achieve optimized vehicle scheduling; wherein, the shaping reward function includes a distance penalty term and a speed reward term.
9. An electronic device, characterized in that, The electronic device includes: processor; A memory storing computer-readable instructions that, when loaded and executed by the processor, implement the method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 7.