Signal-free intersection unmanned mine vehicle cooperative control method and system

Abstract

The application provides a kind of intersection unmanned mine car cooperative control method and system without signal light, comprising: based on conflict point model, the internal area of intersection is constructed;Based on the internal area of intersection, an optimization area is constructed;Mine car enters optimization area, sends the car state information and pass request to intersection manager;Based on conflict table, intersection manager judges whether there is potential conflict between mine car;According to first-in first-out and first-in first-out principle, the priority of mine car is judged;Based on priority analysis result, intersection manager sends the pass instruction and pass priority information to the mine car that has sent pass request;A deep reinforcement learning model is constructed, a deep learning network is trained based on DDPG algorithm and parameters are updated, and the acceleration of each mine car in intersection is adjusted.The application realizes safe passage of unmanned mine car in irregular intersection scene without signal light, and effectively improves the passage efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent driving, in particular, to a signal-free intersection unmanned mine truck cooperative control method and system. BACKGROUND

[0002] Researching on mine area unmanned driving is of great significance to improve the driving economy, safety and transportation efficiency of mine area traffic system. As the intersection that bears the functions of merging, crossing and splitting of mine trucks, it is very easy to cause traffic congestion due to the lack of signal control, thereby reducing the passing speed of mine trucks, increasing the delay time and ultimately affecting the transportation efficiency of the whole mine area traffic. Therefore, the cooperative control method of mine trucks at the signal-free intersection of mine area is crucial to the passing safety and efficiency.

[0003] Patent document CN111047871A (application number: CN201911132245.9) discloses a human-machine driving hybrid traffic cooperative control system and method based on vehicle networking. The system includes a plurality of cooperative controllers corresponding to the number of intersection paths. Each cooperative controller is arranged beside the signal light of the corresponding intersection and the intersection path affected by the signal light is the cooperative control area targeted by the cooperative controller. The cooperative controller includes a collection unit, a processing unit and a control unit connected in turn. The vehicle networking and V2I technology are used to realize the communication between the cooperative control system and manned vehicles and unmanned vehicles. The information collection unit obtains effective information from the communication network. After analysis and processing by the processing unit, the control signal for the unmanned vehicle is obtained and transmitted to the control unit. After the control unit takes over the unmanned vehicle entering the cooperative control area, the unmanned vehicle is controlled to pass through the congested intersection according to the control signal given by the processing unit, which can well solve the problem of unmanned vehicles passing through congested intersections.

[0004] The current control method of signal-free intersection mine trucks mainly includes centralized control and distributed control. Centralized control means that the intersection manager collects the information of all mine trucks in the intersection area, and adjusts the speed of each truck after calculation and processing. Distributed control means that each truck collects the information of other mine trucks within a certain range, combines with the state of the vehicle itself, and makes a strategy that is most beneficial to the vehicle itself, so as to avoid collision with other mine trucks. However, most of the existing researches are aimed at regular crossroads scenes, and their methods are not suitable for irregular intersection scenes. In the irregular intersection scene of mine area, how to improve the mine truck passing throughput and eliminate the passing conflict has become one of the key problems in the research of mine area unmanned transportation technology. SUMMARY

[0005] In view of the defects in the prior art, the purpose of the present application is to provide a signal-free intersection unmanned mine truck cooperative control method and system.

[0006] The signal-free intersection unmanned mine vehicle cooperative control method provided by the application comprises the following steps:

[0007] Step 1: Based on the conflict point model, the intersection internal region is constructed.

[0008] Step 2: Based on the intersection internal region, the optimization region is constructed.

[0009] Step 3: The mine vehicle enters the optimization region and sends the self-vehicle state information and the passing request to the intersection manager.

[0010] Step 4: Based on the conflict table, the intersection manager judges whether there is a potential conflict between the mine vehicles.

[0011] Step 5: According to the first-in-first-out and first-in-first-out principles, the mine vehicle passing priority is judged.

[0012] Step 6: Based on the priority analysis result, the intersection manager sends the passing instruction and the passing priority information to the mine vehicle that has sent the passing request.

[0013] Step 7: A deep reinforcement learning model is constructed, the deep learning network is trained based on the DDPG algorithm and the parameters are updated, and the acceleration of each mine vehicle in the intersection is adjusted.

[0014] Preferably, when the intersection internal region is constructed, first, based on the conflict point model, the mine vehicle is made to travel along a fixed trajectory, and all possible trajectory conflict regions are analyzed; then, the intersection internal region boundary is drawn on each entrance lane, and the requirement is that all trajectory conflict regions are inside the drawn boundary, and the region surrounded by all the boundaries is taken as the intersection internal region; finally, the boundary is extended outward by 100m according to the boundary, and the region between the two boundaries is taken as the optimization region.

[0015] Preferably, after receiving the passing permission instruction and the passing priority, the mine vehicle enters the intersection internal region, otherwise it needs to be parked outside the intersection internal region boundary and wait.

[0016] Preferably, the step 7 comprises:

[0017] First, the mine area intersection simulation environment is constructed using the SUMO software, and the MADDPG algorithm neural network is initialized, wherein each agent has a strategy network and a target strategy network with the same structure but different parameters, and an evaluation network and a target evaluation network;

[0018] Secondly, the state information of all mine cars is acquired, including speed, position and lane information, after training, the action of each mine car at the next moment is output, and the corresponding reward and punishment are obtained, the current state, the next state of all agents, the action taken and the reward obtained are stored in the experience pool;

[0019] Finally, during the training process, samples are extracted from the experience pool based on the priority experience replay method, the evaluation network is trained, the parameters of the evaluation network are updated by minimizing the loss function, and the corresponding policy network parameters are updated according to the gradient descent method, and the target policy network and the target evaluation network are updated by using soft update.

[0020] Preferably, during the training process, the action space of each mine car agent includes continuous actions of [-2, 2]m / s 2 The reward function of each mine car agent is the same, as follows:

[0021] r = r1 + r2 + r3 + r4

[0022] Wherein, r3 = -α * t, t represents the time of the mine car passing through the intersection area, and α represents the weight; r4 = -β * w, w represents the waiting time of the mine car, and β represents the weight;

[0023] If a collision occurs, a strong negative reward of -100 is given, if the intersection area is passed, a strong positive reward of 50 is given, in addition, the total time of the mine car passing through the intersection area and the waiting time are minimized to improve the traffic efficiency and ensure fairness;

[0024] The priority experience replay refers to that the current state, the next state, the action taken and the reward obtained of all mine car agents are stored in the experience pool, which is represented by an array: (x, x', a1, a2,..., a n , r1, r2,..., r n The evaluation network preferentially extracts important experiences for training, thereby accelerating convergence;

[0025] The joint action-value function y and the joint action-value function output by the evaluation network are used The formula for solving the loss L is as follows:

[0026]

[0027]

[0028] In the formula, is the joint action-value function output by the target evaluation network, and r i jis the reward value of the agent i at the j moment, μ' is the target policy network of the lagging update, and S is the sample number;

[0029] The policy network parameter update is performed by using the gradient descent method:

[0030]

[0031] The target policy network and target evaluation network parameter update formula of the soft update method with the balance factor η are as follows:

[0032]

[0033]

[0034] In the formula, is the target policy network, is the target evaluation network.

[0035] The unmanned mine vehicle cooperative control system for the intersection without signal lights provided by the application comprises:

[0036] Module M1: based on the conflict point model, the internal area of the intersection is constructed;

[0037] Module M2: based on the internal area of the intersection, the optimization area is constructed;

[0038] Module M3: the mine vehicle enters the optimization area, and sends the self-vehicle state information and the passing request to the intersection manager;

[0039] Module M4: based on the conflict table, the intersection manager judges whether there is a potential conflict between the mine vehicles;

[0040] Module M5: according to the first-in-first-out and first-in-first-out principles, the passing priority of the mine vehicle is judged;

[0041] Module M6: based on the priority analysis result, the intersection manager sends the passing instruction and the passing priority information to the mine vehicle which has sent the passing request;

[0042] Module M7: a deep reinforcement learning model is constructed, the deep learning network is trained based on the DDPG algorithm and the parameters are updated, and the acceleration of each mine vehicle in the intersection is adjusted.

[0043] Preferably, when the internal area of the intersection is constructed, first, based on the conflict point model, the mine vehicle is required to drive along the fixed track, and all possible track conflict areas are analyzed; then, the internal area boundary of the intersection is drawn on each entrance lane, which requires that all track conflict areas are inside the drawn boundary, and the area surrounded by all the boundaries is taken as the internal area of the intersection; finally, the boundary is expanded outward by 100 m according to the boundary, which is taken as the optimization area boundary, and the area between the two boundaries is the optimization area.

[0044] Preferably, after receiving the instruction of passing and the priority, the mine car drives into the internal area of the intersection, otherwise it needs to stop and wait outside the boundary of the internal area of the intersection.

[0045] Preferably, the module M7 comprises:

[0046] Firstly, a simulation environment of the mine intersection is constructed using the SUMO software, and the neural network of the MADDPG algorithm is initialized, wherein each agent has a strategy network and a target strategy network with the same structure but different parameters, and an evaluation network and a target evaluation network;

[0047] Secondly, the state information of all mine cars is obtained, including the speed, position and lane information, after training, the action of each mine car at the next moment is outputted, and the corresponding rewards and punishments are obtained, and the current state, the next state, the action taken and the rewards and punishments obtained by all agents are stored in the experience pool;

[0048] Finally, during the training process, samples are extracted from the experience pool based on the priority experience replay method, the evaluation network is trained, the parameters of the evaluation network are updated by minimizing the loss function, the corresponding strategy network parameters are updated according to the gradient descent method, and the target strategy network and the target evaluation network are updated by using soft update.

[0049] Preferably, during the training process, the action space of each mine car agent includes continuous actions of [-2, 2] m / s 2 , and the reward function of each mine car agent is the same, as shown below:

[0050] r = r1 + r2 + r3 + r4

[0051] wherein, r3 = -α * t, t represents the time of the mine car passing through the intersection area, and α represents the weight; r4 = -β * w, w represents the waiting time of the mine car, and β represents the weight;

[0052] If a collision occurs, a strong negative reward of -100 is given, and if the mine car passes through the intersection area, a strong positive reward of 50 is given, in addition, the total time of the mine car passing through the intersection area and the waiting time are minimized to improve the passing efficiency and ensure fairness;

[0053] The priority experience replay refers to that the current state, the next state, the action taken and the rewards and punishments obtained by all mine car agents are stored in the experience pool, which is represented by an array: (x, x', a1, a2,..., a n , r1, r2,..., r n ), and the evaluation network preferentially extracts important experiences for training, thereby accelerating the convergence;

[0054] The target joint action-value y and the joint action-value function output by the evaluation network are utilized The formula of the loss L is as follows:

[0055]

[0056]

[0057] wherein, is the joint action-value function output by the target evaluation network, r i j is the reward value of the agent i at the j moment, mu' is the target policy network of the lagging update, and S is the sample number;

[0058] The policy network parameter update is performed by using the gradient descent method:

[0059]

[0060] The target policy network and the target evaluation network parameter update formula of the soft update method with the balance factor η are as follows:

[0061]

[0062]

[0063] wherein, is the target policy network, is the target evaluation network.

[0064] Compared with the prior art, the present application has the following beneficial effects:

[0065] The present application constructs the intersection internal region and the optimization region, solves the problem that there is no standard intersection internal region and optimization region for irregular intersections, and when the mine car enters the optimization region, the intersection manager judges whether there is a potential conflict based on the conflict table, so that the conflict judgment is more simple and accurate, according to the first-in first-out and first-in first-out principles, the priority module judges the mine car passing priority, effectively improves the mine car passing efficiency, based on the priority analysis result, the intersection manager sends the passing instruction and the passing priority information, so that the mine car can pass in order, and the problems of congestion and signal failure caused by too many mine cars in the intersection can be avoided, based on the deep reinforcement learning model, the acceleration of the mine car in the intersection is adjusted, compared with the traditional mathematical programming method, it can be used in irregular intersection scenes, and the vehicle speed can be adjusted according to the real traffic flow, and the mine car passing safety is improved. BRIEF DESCRIPTION OF DRAWINGS

[0066] Other features, objects and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments, made with reference to the accompanying drawings.

[0067] Figure 1 is a flowchart of a cooperative control technology of unmanned mine trucks in a signal-free intersection scene;

[0068] Figure 2 is a schematic diagram of the construction process of the intersection internal region and the optimization region;

[0069] Figure 3 is a conflict representation intention;

[0070] Figure 4 is a MADDPG algorithm block diagram. DETAILED DESCRIPTION

[0071] The present application will be described in detail below with specific embodiments. The following examples will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present application. These are within the scope of the present application.

[0072] Example 1:

[0073] As shown in Figure 1 , the present application provides a cooperative control method of unmanned mine trucks in a signal-free intersection scene, comprising the following steps:

[0074] Step 1, constructing an intersection internal region: based on a conflict point model, constructing an intersection internal region.

[0075] Step 2, constructing an optimization region: based on the intersection internal region, constructing an optimization region.

[0076] As shown in Figure 2 , specifically, when constructing the intersection internal region, first, based on the conflict point model, i.e. making the mine truck travel along a fixed trajectory, analyzing all possible trajectory conflict regions. Then, draw the intersection internal region boundary on each entrance lane, which requires that all trajectory conflict regions are within the drawn boundary, and the region surrounded by all boundaries is the intersection internal region. Finally, expand 100m outward according to the boundary as the optimization region boundary, and the region between the two boundaries is the optimization region.

[0077] Step 3, the mine truck enters the optimization region and sends the vehicle state information and passing request to the intersection manager, so that the intersection manager collects the mine truck information to realize centralized control. The vehicle state information includes speed, position, and lane information.

[0078] Step 4, the intersection manager judges the conflict possibility: based on the conflict table, the intersection manager judges whether there is a potential conflict between the mine trucks.

[0079] As Figure 3 shown, specifically, first, based on the conflict point model, all possible trajectory conflict areas are analyzed. Then, a conflict table is drawn up, i.e. the conflict areas that each lane can pass through. According to the lane information of each mine car, the intersection manager can quickly and accurately determine whether there is a potential conflict between mine cars.

[0080] Step 5, priority module determines the priority of mine car passing: according to the first-in-first-out and first-to-first-out principle, the priority module determines the priority of mine car passing, realizes the orderly and efficient passing of mine cars through the intersection.

[0081] The first-in-first-out and first-to-first-out principle refers to giving priority to the passing priority of the mine car that first enters the internal area of the intersection. Assuming that there are three mine cars V1, V2 and V3, if V1 first enters the internal area of the intersection, the intersection manager will give priority to the passing right of V1, and if V1 and V2 have potential trajectory conflicts, the mine car that arrives at the conflict area first will have priority.

[0082] Step 6, intersection manager sends passing instruction and priority information: based on the priority analysis result, the intersection manager sends the passing instruction and priority information to the mine car that has sent the passing request. After receiving the instruction and priority information allowing passing, the mine car can enter the internal area of the intersection, otherwise, it needs to be parked outside the boundary of the internal area of the intersection and wait.

[0083] Step 7, speed adjustment: on the basis of single-agent deep reinforcement learning, multiple agents, i.e. multiple unmanned mine cars, are introduced, based on DDPG algorithm, the deep learning network is trained and the parameters are updated, the acceleration of each mine car in the internal area of the intersection is adjusted, and the passing conflict of multiple unmanned mine cars is resolved.

[0084] As Figure 4 shown, specifically, first, use SUMO software to build a mine intersection simulation environment, initialize the MADDPG algorithm neural network, wherein each agent has a strategy network and a target strategy network with the same structure but different parameters, as well as an evaluation network and a target evaluation network.

[0085] Secondly, obtain the state information of all mine cars, including speed, position and lane information. After training, output the action of each mine car at the next time, and obtain the corresponding reward and punishment. Store the current state, next state, action taken and reward and punishment obtained by all agents in the experience pool.

[0086] Finally, during the training process, the method based on the priority experience replay extracts samples from the experience pool to train the evaluation network. The parameters of the evaluation network are updated by minimizing the loss function, and the corresponding policy network parameters are updated according to the gradient descent method. The target policy network and the target evaluation network are updated using soft update.

[0087] It should be noted that during the training process, the simulation environment of the mine intersection is drawn in the SUMO software, and the state space of each mine car agent is obtained, including speed, position and lane information.

[0088] During the training process, the action space of each mine car agent includes continuous actions of [-2, 2] m / s. 2

[0089] During the training process, the reward and punishment function of each mine car agent is the same, as shown below:

[0090] r = r1 + r2 + r3 + r4

[0091] wherein, r3 = -α * t, t represents the time of the mine car passing through the intersection area, and α represents the weight; r4 = -β * w, w represents the waiting time of the mine car, and β represents the weight.

[0092] Specifically, if a collision occurs, a strong negative reward of -100 is given, and if the intersection area is passed, a strong positive reward of 50 is given. In addition, the total time of the mine car passing through the intersection area and the waiting time are minimized to improve the traffic efficiency and ensure fairness.

[0093] During the training process, the priority experience replay refers to the storage of the current state, the next state, the action taken and the reward obtained of all mine car agents in the experience pool, which is represented by an array: (x, x', a1, a2,..., a n , r1, r2,..., r n ). The evaluation network will preferentially extract important experiences for training, thereby accelerating convergence.

[0094] During the training process, the joint action-value y output by the target joint action-value function and the evaluation network is used to solve the loss L formula as follows:

[0095]

[0096]

[0097] wherein, is the joint action-value function output by the target evaluation network, and r i j ​​is the reward value of the agent i at the j moment, μ' is the target policy network of the lagging update, and S is the sample number.

[0098] In the training process, the gradient descent method is used for policy network parameter update.

[0099]

[0100] In the training process, the target policy network and the target evaluation network parameter update formula of the soft update method with the balance factor η are as follows:

[0101]

[0102]

[0103] In the formula, is the target policy network, is the target evaluation network.

[0104] Embodiment 2:

[0105] The application also provides a signal lamp-free intersection unmanned mine car cooperative control system, which can be realized by executing the flow steps of the signal lamp-free intersection unmanned mine car cooperative control method, that is, the signal lamp-free intersection unmanned mine car cooperative control method can be understood by those skilled in the art as the preferred embodiment of the signal lamp-free intersection unmanned mine car cooperative control system.

[0106] According to the signal lamp-free intersection unmanned mine car cooperative control system provided by the application, the modules include: module M1: constructing an intersection internal region based on a conflict point model; module M2: constructing an optimization region based on the intersection internal region; module M3: a mine car entering the optimization region and sending self-car state information and a passing request to an intersection manager; module M4: the intersection manager judging whether there is a potential conflict between mine cars based on a conflict table; module M5: judging the passing priority of the mine car according to the first-come-first-served and first-in-first-out principles; module M6: the intersection manager sending passing instructions and passing priority information to the mine car that has sent a passing request based on the priority analysis result; and module M7: constructing a deep reinforcement learning model, training a deep learning network based on a DDPG algorithm and updating parameters, and adjusting the acceleration of each mine car in the intersection.

[0107] When constructing the internal region of the intersection, first, based on the conflict point model, let the mine car run along the fixed track, and analyze all possible track conflict regions; then, draw the internal region boundary of the intersection on each entrance lane, which requires that all track conflict regions are within the drawn boundary, and the region surrounded by all boundaries is the internal region of the intersection; finally, expand 100m outward according to the boundary as the optimized region boundary, and the region between the two boundaries is the optimized region.

[0108] After receiving the instruction to pass and the priority, the mine car enters the internal region of the intersection, otherwise it needs to be parked outside the internal region boundary of the intersection to wait.

[0109] The module M7 comprises: first, using SUMO software to construct a mine intersection simulation environment, initializing the MADDPG algorithm neural network, wherein each agent has a strategy network and a target strategy network with the same structure but different parameters, and an evaluation network and a target evaluation network; second, obtaining the state information of all mine cars, including speed, position and lane information, after training, outputting the action of each mine car at the next moment, and obtaining the corresponding reward and punishment, storing the current state, the next state, the action taken and the reward and punishment obtained by all agents into the experience pool; finally, in the training process, based on the priority experience replay method, samples are extracted from the experience pool, the evaluation network is trained, the parameters of the evaluation network are updated by minimizing the loss function, the corresponding strategy network parameters are updated according to the gradient descent method, and the target strategy network and the target evaluation network are updated by using soft update.

[0110] During the training process, the action space of each mine car agent includes [-2 ~ 2] m / s 2 continuous action, and the reward and punishment function of each mine car agent is the same, as follows:

[0111] r = r1 + r2 + r3 + r4

[0112] wherein, r3 = -α * t, t represents the time of the mine car passing through the intersection region, and α represents the weight; r4 = -β * w, w represents the waiting time of the mine car, and β represents the weight;

[0113] If a collision occurs, a strong negative reward of -100 is given, and if the mine car passes through the intersection region, a strong positive reward of 50 is given, in addition, the total time of the mine car passing through the intersection region and the waiting time are minimized to improve the passing efficiency and ensure fairness;

[0114] The priority experience replay refers to storing the current state, the next state, the action taken and the reward and punishment obtained by all mine car agents in the experience pool, which is represented by an array: (x, x', a1, a2,..., an, r1, r2,..., rn). n ​n ), the evaluation network is trained by evaluating the important experience of the network to accelerate the convergence;

[0115] The target joint action-value y and the joint action-value function output by the evaluation network The formula for solving the loss L is as follows:

[0116]

[0117]

[0118] In the formula, is the joint action-value function output by the target evaluation network, r i j is the reward value of the agent i at time j, μ' is the target policy network of the lagging update, and S is the number of samples;

[0119] The policy network parameter is updated by using the gradient descent method:

[0120]

[0121] The target policy network and the target evaluation network parameter update formula of the soft update method with the balance factor η are as follows:

[0122]

[0123]

[0124] In the formula, is the target policy network, is the target evaluation network.

[0125] Those skilled in the art know that in addition to implementing the system, device and each module thereof provided by the present application in the form of pure computer readable program code, the same program can be realized by logically programming the method steps to make the system, device and each module thereof provided by the present application in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers. Therefore, the system, device and each module thereof provided by the present application can be considered as a hardware component, and the modules included therein for implementing various programs can also be considered as structures within the hardware component; the modules for implementing various functions can also be considered as both software programs for implementing methods and structures within hardware components.

[0126] The specific embodiments of the present application are described above. It needs to be understood that the present application is not limited to the specific embodiments described above, and various changes or modifications can be made by those skilled in the art within the scope of the claims, which does not affect the essential content of the present application. The embodiments of the present application and the features in the embodiments can be combined with each other at will without conflict.

Claims

1. A method for cooperative control of unmanned mining trucks at traffic light-free intersections, characterized in that, include: Step 1: Construct the internal area of ​​the intersection based on the conflict point model; Step 2: Construct an optimization area based on the internal area of ​​the intersection; Step 3: The mining truck enters the optimization area and sends its own vehicle status information and a pass request to the intersection manager; Step 4: Based on the conflict table, the intersection manager determines whether there are potential conflicts between mining trucks; Step 5: Determine the priority of mine truck passage based on the first-in-first-out and first-in-first-out principles; Step 6: Based on the priority analysis results, the intersection manager sends passage instructions and passage priority information to the mining trucks that have already sent passage requests; Step 7: Construct a deep reinforcement learning model, train the deep learning network based on the DDPG algorithm and update the parameters to adjust the acceleration of each mining truck in the intersection; When constructing the internal area of ​​the intersection, firstly, based on the conflict point model, the mining truck is driven along a fixed trajectory to analyze all possible trajectory conflict areas; then, the boundary of the internal area of ​​the intersection is drawn on each entrance lane, requiring that all trajectory conflict areas are inside the drawn boundary, and the area enclosed by all boundaries is the internal area of ​​the intersection; finally, based on this boundary, it is extended outward by 100m as the boundary of the optimization area, and the area between the two boundaries is the optimization area. After receiving permission to pass and priority, the mining truck enters the inner area of ​​the intersection; otherwise, it must stop and wait outside the boundary of the inner area of ​​the intersection. Step 7 includes: First, a simulation environment for the mining area intersection was constructed using SUMO software, and the MADDPG algorithm neural network was initialized. Each agent has a policy network and a target policy network with the same structure but different parameters, as well as an evaluation network and a target evaluation network. Secondly, acquire the status information of all mining trucks, including speed, position and lane information. After training, output the action of each mining truck in the next moment and obtain corresponding rewards and penalties. Store the current state, next state, action taken and reward / penalty of all agents into the experience pool. Finally, during the training process, samples are drawn from the experience pool based on the priority experience replay method to train the evaluation network. The parameters of the evaluation network are updated by minimizing the loss function, and the parameters of the corresponding policy network are updated according to the gradient descent method. Soft update is used to update the target policy network and the target evaluation network. During training, the motion space of each mining truck agent includes For consecutive actions, the reward and penalty functions of each mining truck agent are the same, as shown below: in, , ; , This indicates the time it takes for the mine car to pass through the intersection area. Indicates weight; , Indicates the waiting time for the mine cart. Indicates weight; If a collision occurs, a strong negative reward of -100 is given; if the vehicle passes through an intersection area, a strong positive reward of 50 is given. In addition, the total time and waiting time of the minecart in the intersection area are minimized to improve traffic efficiency and ensure fairness. The prioritized experience replay refers to the storage of the current state, next state, actions taken, and rewards / penalties of all mining truck agents in the experience pool, represented by an array: The evaluation network prioritizes extracting more important experiences for training, thereby accelerating convergence; Using the joint action-value y of the objective and the joint action-value function of the evaluation network output The formula for calculating the loss L is as follows: In the formula, It is the joint action-value function output by the target evaluation network. It is the reward value of agent i at time j. It is a target policy network with delayed updates, where S is the number of samples; Updating policy network parameters using gradient descent: The balance factor is The parameter update formulas for the target policy network and target evaluation network in the soft update method are as follows: In the formula, It is a target policy network. It is a target evaluation network.

2. A cooperative control system for unmanned mining trucks at traffic light-free intersections, characterized in that: include: Module M1: Constructs the internal area of ​​the intersection based on the conflict point model; Module M2: Constructs an optimized area based on the internal area of ​​the intersection; Module M3: When a mining truck enters the optimization area, it sends its own vehicle status information and a pass request to the intersection manager; Module M4: Based on the conflict table, the intersection manager determines whether there are potential conflicts between mining trucks; Module M5: Determines the priority of minecart passage based on the first-in-first-out (FIFO) and first-in-first-out (FIFO) principles; Module M6: Based on the priority analysis results, the intersection manager sends passage instructions and passage priority information to mining trucks that have already sent passage requests; Module M7: Construct a deep reinforcement learning model, train a deep learning network based on the DDPG algorithm and update parameters to adjust the acceleration of each mining truck within the intersection; When constructing the internal area of ​​the intersection, firstly, based on the conflict point model, the mining truck is driven along a fixed trajectory to analyze all possible trajectory conflict areas; then, the boundary of the internal area of ​​the intersection is drawn on each entrance lane, requiring that all trajectory conflict areas are inside the drawn boundary, and the area enclosed by all boundaries is the internal area of ​​the intersection; finally, based on this boundary, it is extended outward by 100m as the boundary of the optimization area, and the area between the two boundaries is the optimization area. After receiving permission to pass and priority, the mining truck enters the inner area of ​​the intersection; otherwise, it must stop and wait outside the boundary of the inner area of ​​the intersection. The module M7 includes: First, a simulation environment for the mining area intersection was constructed using SUMO software, and the MADDPG algorithm neural network was initialized. Each agent has a policy network and a target policy network with the same structure but different parameters, as well as an evaluation network and a target evaluation network. Secondly, acquire the status information of all mining trucks, including speed, position and lane information. After training, output the action of each mining truck in the next moment and obtain corresponding rewards and penalties. Store the current state, next state, action taken and reward / penalty of all agents into the experience pool. Finally, during the training process, samples are drawn from the experience pool based on the priority experience replay method to train the evaluation network. The parameters of the evaluation network are updated by minimizing the loss function, and the parameters of the corresponding policy network are updated according to the gradient descent method. Soft update is used to update the target policy network and the target evaluation network. During training, the motion space of each mining truck agent includes For consecutive actions, the reward and penalty functions of each mining truck agent are the same, as shown below: in, , ; , This indicates the time it takes for the mine car to pass through the intersection area. Indicates weight; , Indicates the waiting time for the mine cart. Indicates weight; If a collision occurs, a strong negative reward of -100 is given; if the vehicle passes through an intersection area, a strong positive reward of 50 is given. In addition, the total time and waiting time of the minecart in the intersection area are minimized to improve traffic efficiency and ensure fairness. The prioritized experience replay refers to the storage of the current state, next state, actions taken, and rewards / penalties of all mining truck agents in the experience pool, represented by an array: The evaluation network prioritizes extracting more important experiences for training, thereby accelerating convergence; Using the joint action-value y of the objective and the joint action-value function of the evaluation network output The formula for calculating the loss L is as follows: In the formula, It is the joint action-value function output by the target evaluation network. It is the reward value of agent i at time j. It is a target policy network with delayed updates, where S is the number of samples; Updating policy network parameters using gradient descent: The balance factor is The parameter update formulas for the target policy network and target evaluation network in the soft update method are as follows: In the formula, It is a target policy network. It is a target evaluation network.

Images