Traffic light-free intersection control method and equipment based on deep reinforcement learning

Abstract

The present invention discloses a control method and device for intersections without signal lights based on deep reinforcement learning. The method uses a deep reinforcement learning algorithm to coordinate and control the conflict-free release of intersections, avoids conflict release by judging conflict points, achieves fairness in vehicle release through the design of a reward function, and achieves maximum release on this basis. The release action of the target intersection without signal lights at the current moment is calculated by interactively updating the trained DQN network, which reduces the average waiting time of vehicles and increases traffic efficiency, thereby solving the problem of vehicle congestion at intersections without signal lights. Compared with the existing method for releasing intersections without signal lights based on deep reinforcement learning, the present invention uses a deep learning framework to formulate the intersection coordination problem, eliminates the need to record the occupation status of the conflict area grid, and reduces the complexity of the calculation. At the same time, the overall analysis of the intersection without signal lights also fully utilizes the coordination potential of the intersection.

Description

Technical Field

[0001] The present invention relates to the field of urban traffic control technology, and in particular to a method and device for controlling a non-signalized intersection based on deep reinforcement learning. Background Art

[0002] In urban transportation systems, intersections are where conflicting traffic flows are forced to share limited spatial and temporal resources, often causing the most congestion. To reduce congestion and improve traffic efficiency, various intelligent intersection management methods have been proposed. Existing intersection management research focuses on two main scenarios: signalized intersections and unsignalized intersections.

[0003] Currently, there are three main approaches to managing intersections without signal lights: reservation-based, planning-based, and optimization-based. Reservation-based methods centrally control vehicle behavior, with each vehicle reserving a time-space block at the intersection. A centralized controller manages reservations based on a first-come, first-served basis. This method is currently the most widely used approach to achieving collision-free multi-vehicle traffic at intersections without signal lights. Planning-based methods plan the spatiotemporal trajectories of vehicles to resolve conflicts and avoid collisions. Optimization-based methods centrally model vehicle behavior and solve optimization problems within collision avoidance constraints.

[0004] However, these methods either rely on overly conservative decision-making to ensure driving safety. In real-world driving, the long waits caused by yielding to other vehicles' safety are unacceptable. Alternatively, they acquire vehicle information and calculate the optimal passage sequence. However, due to the collision-free constraints and the complexity of the required maneuvers, this requires processing large amounts of data and is complex.

[0005] The simplest formulation without collision constraints is the collision set (CS) strategy. The basic rule is that no two vehicles with overlapping paths should be in the intersection at the same time, thus eliminating potential collisions from all conflict-based approaches. This strategy offers advantages only in terms of low-complexity implementation and fails to fully exploit the coordination potential of intersections. Furthermore, reservation-based approaches divide the conflict area into multiple grids. As long as two vehicles do not occupy the same grid at the same time, there is no conflict. This approach can make better use of the conflict area, but requires recording the occupation status of each grid in the conflict area, increasing computational complexity.

[0006] In addition, with the development of artificial intelligence, machine learning has begun to be applied to intelligent transportation systems. Deep reinforcement learning (DRL), as the learning system closest to the human brain, can efficiently solve complex decision-making problems. However, in current research on intelligent transportation systems, most deep reinforcement learning algorithms are used to optimize the phase and timing of traffic lights in isolated or grouped environments. Their application in intersections without signal lights is relatively rare. Moreover, among the few applications at intersections without signal lights, some are not used to control the release of vehicles at the intersection, but rather to optimize a parameter in the process to facilitate the calculation of release control instructions by other algorithms. This has failed to effectively apply deep reinforcement learning to the application scenario of intersections without signal lights to solve the problem of collision-free vehicle passage and improve intersection traffic efficiency. Summary of the Invention

[0007] Based on the problems raised in the background technology, the purpose of the present invention is to provide a control method and device for intersections without signal lights based on deep reinforcement learning, which uses a deep reinforcement learning algorithm to coordinate and control the conflict-free release of intersections, avoids conflict release by judging the conflict points, and enables conflict-free release of vehicles. The fairness of vehicle release and the maximum release on this basis are achieved through the design of the reward function. The release action of the target intersection without signal lights at the current moment is calculated by interactively updating the trained DQN network, which reduces the average waiting time of vehicles and increases traffic efficiency, thereby solving the problem of vehicle congestion at intersections without signal lights.

[0008] The present invention is achieved through the following technical solutions:

[0009] The first aspect of the present invention provides a method for controlling a non-signalized intersection based on deep reinforcement learning, comprising:

[0010] S1. Constructing an agent and an environment, wherein the environment includes a conflict area and a lane area, and the conflict area includes a conflict point;

[0011] S2. Observe the environment and obtain the overall state s of the environment at time t t , the agent selects state s through the ε-greedy strategy t The release action a at the next time t t ;

[0012] S3, the agent executes the release action a t , obtain the conflict situation of the conflict point at time t;

[0013] S4. Determine the reward r at time t based on the conflict situation t , and obtain the state s of the entire environment at time t+1 t+1 ;

[0014] S5, change the state s t, release action a t , reward r t , status s t+1 Integrate into experience and store it in an experience replay pool, and use the experience in the experience replay pool to train the agent;

[0015] S6. Repeat steps S2 to S5 to train the environment and agent interaction, and use the training results to update the DQN network;

[0016] S7. Use the updated DQN network to calculate the release action of the intersection without traffic lights at the current moment.

[0017] In the above technical solution, an environment including conflict zones and lane zones is constructed, and the lane zones and conflict zones are observed and analyzed as a whole. Compared with the existing technology, the overall state of the intersection is observed instead of the state of each vehicle locally. This fully utilizes the coordination potential of the intersection while ensuring collision-free passage of individual vehicles.

[0018] The release action at the intersection state at time t is selected by the ε-greedy strategy, and then the release action is used to determine whether there is a conflict between the vehicles that choose the release action at the conflict point. The reward at time t and the environment state at the next moment are determined based on the conflict situation of the intersection. t , release action a t , reward r t , status s t+1 The experience is integrated into the experience replay pool, and the agent is trained through this cycle, and the training results are used to update the DQN network.

[0019] At this time, by using the updated DQN network to calculate the release action of the intersection without signal lights at the current moment, on the basis of fair vehicle release, the maximum conflict-free release of the intersection without signal lights is achieved, the average waiting time of vehicles is reduced, and the traffic efficiency is increased.

[0020] In an optional embodiment, the lane area includes at least 12 roads.

[0021] In an optional embodiment, the environment is observed and the overall state s of the environment at time t is obtained. t Specifically include:

[0022] Observe the road conditions of 12 roads and use a 12-element vector N to represent the road conditions of the 12 roads respectively;

[0023] If the first vehicle on the ith road (0≤i≤11) arrives at the conflict zone, the vector N corresponding to the ith road is i Recorded as the convoy length of the i-th road; otherwise, the vector N corresponding to the i-th roadi Recorded as 0;

[0024] At this time, the state of the entire environment at time t s t Indicated as s t ={N0(t),N1(t),…,N 10 (t),N 11 (t)}.

[0025] In an alternative embodiment, the agent selects the state s by the ε-greedy strategy t The release action a at the next time t t Specifically include:

[0026] A 12-element vector M is used to represent the release actions of the 12 roads respectively;

[0027] If the i-th road is released, the vector M corresponding to the i-th road is i Recorded as 1; otherwise, the vector M corresponding to the i-th road i Recorded as 0;

[0028] At this time, the release action at time t is a t Represented as a t ={M0(t),M1(t),…,M 10 (t),M 11 (t)}.

[0029] In an optional embodiment, the agent performs the release action a t Obtaining the conflict situation of the conflict point at time t specifically includes: calculating the time when the vehicles arrive at the conflict point, if multiple vehicles arrive at the conflict point at the same time, there is a conflict, otherwise there is no conflict.

[0030] In an optional embodiment, the reward r at time t is determined according to the conflict situation. t Previously included:

[0031] Construct the reward function at time t, which is constructed as follows:

[0032]

[0033] Among them, r(t) is the reward function at time t, r n (t) is the fairness reward function, r f (t) is the reward function of the intersection release capability.

[0034] In an optional embodiment, the fairness reward function is expressed as follows:

[0035] r f (t)=∑(Ni (t) / N max )

[0036] Among them, N i (t) represents the number of vehicles in the lane released at time t, N max Indicates the maximum capacity of a single lane.

[0037] In an optional embodiment, the reward function of the intersection release capability is expressed as follows:

[0038] r n (t)=V(t) / V max

[0039] Among them, V(t) represents the number of vehicles released at time t, V max Indicates the maximum number of vehicles that can be released.

[0040] A second aspect of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a method for controlling a non-signalized intersection based on deep reinforcement learning is implemented.

[0041] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for controlling a non-signaled intersection based on deep reinforcement learning.

[0042] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0043] By using the updated DQN network to calculate the release action at the current moment at the intersection without signal lights, the maximum conflict-free release at the intersection without signal lights is achieved on the basis of fair vehicle release, which reduces the average waiting time of vehicles and increases traffic efficiency.

[0044] Compared with the existing deep reinforcement learning method for releasing traffic at unsignaled intersections, the present invention uses a deep learning framework to formulate intersection coordination problems, eliminating the need to record the occupation status of conflict area grids and reducing computational complexity. At the same time, analyzing the unsignaled intersection as a whole also fully taps the intersection's coordination potential. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required for use in the examples. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be considered as limiting the scope. A person of ordinary skill in the art can also derive other relevant drawings based on these drawings without inventive effort. In the drawings:

[0046] Figure 1 Flowchart of the non-signalized intersection control method based on deep reinforcement learning provided in Example 1 of the present invention;

[0047] Figure 2 This is an architecture diagram of the non-signalized intersection control method based on deep reinforcement learning provided in Example 1 of the present invention;

[0048] Figure 3 A schematic diagram of conflict points in a conflict zone provided in Example 1 of the present invention;

[0049] Figure 4 This is a structural diagram of an electronic device provided in Example 2 of the present invention. DETAILED DESCRIPTION

[0050] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with examples and drawings. The exemplary embodiments of the present invention and their descriptions are only used to explain the present invention and are not intended to limit the present invention.

[0051] Example 1

[0052] Figure 1 Flowchart of the non-signalized intersection control method based on deep reinforcement learning provided in Example 1 of the present invention, as shown in FIG. Figure 1 As shown in the figure, the control method for non-signaled intersections based on deep reinforcement learning is as follows:

[0053] Step S1: Construct an agent and an environment, wherein the environment includes a conflict area and a lane area, and the conflict area includes a conflict point.

[0054] Among them, the agent is the role of learner and decision maker in the DQN network. In the present invention, the agent is used to interact with the environment based on the release action, the conflict situation of the conflict point and the reward; the conflict point is the point where vehicles may collide in an intersection without signal lights, and the conflict point is located in the conflict zone.

[0055] It should be noted that, in the embodiment of the present invention, it is assumed that the vehicle has changed its driving route before reaching the conflict zone and is moving at a constant speed. Figure 3 A schematic diagram of conflict points in the conflict zone provided by Example 1 of the present invention is shown in FIG. Figure 3 As shown in Figure 2, vehicles going straight left and right may collide with vehicles going straight up and down in the conflict zone when passing through an intersection without signal lights, for example Figure 3 In the embodiment of the present invention, there are 16 such conflict points.

[0056] It is important to note that the deep reinforcement learning algorithms used in existing technologies for intersections without signal lights are not used to control the release of vehicles at intersections. Instead, they optimize a parameter in the process to facilitate the calculation of release control instructions by other algorithms. This fails to effectively apply deep reinforcement learning to the application scenario of intersections without signal lights to solve the problem of collision-free vehicle passage and improve intersection traffic efficiency. In contrast, the present invention observes the overall state of the intersection, rather than locally observing the state of each vehicle. The conflict zone and lane area of ​​the intersection are observed and analyzed as the environment. Compared with existing technologies, this not only facilitates the collision-free passage of individual vehicles, but also fully utilizes the coordination potential of the intersection.

[0057] In an optional embodiment, the lane area is three lanes, at least 12 roads, and the three lanes are used for vehicles to turn left, go straight, and turn right respectively.

[0058] Step S2: Observe the environment and obtain the overall state s of the environment at time t t , the agent selects state s through the ε-greedy strategy t The release action a at the next time t t .

[0059] Among them, the ε-greedy strategy is a deterministic strategy, which means that when the agent makes a decision, there is a small positive number ∈ (<1) probability of randomly selecting an unknown action, and the remaining 1-∈ probability of selecting the action with the largest value among the actions that have been selected. In the embodiment of the present invention, the ε-greedy strategy π (a t |s t )Select the release action for the intersection.

[0060] Specifically, observe the environment and obtain the overall state s of the environment at time t t The method is as follows:

[0061] The road conditions of 12 roads are observed, and a 12-element vector N is used to represent the road conditions of the 12 roads respectively.

[0062] If the first vehicle on the ith road (0≤i≤11) arrives at the conflict zone, the vector N corresponding to the ith road is i Recorded as the convoy length of the i-th road; otherwise, the vector N corresponding to the i-th road i Recorded as 0.

[0063] At this time, the overall state of the environment at time t s t Indicated as s t ={N0(t),N1(t),…,N 10 (t),N 11 (t)}.

[0064] The lead vehicle refers to the first vehicle on the road traveling towards the conflict zone.

[0065] It should be noted that this invention considers the overall state of the intersection to demonstrate intersection coordination capabilities. If the leading vehicle on a road heading toward the conflict zone has not yet reached the conflict zone, it indicates that a collision at the conflict point in that lane will not occur at time t, so the vector corresponding to that road is recorded as 0. If the leading vehicle on a road reaches the conflict zone, it indicates that the vehicles on that road may collide at the conflict point at time t, so the length of the convoy on that road is recorded.

[0066] In the embodiment of the present invention, the vehicle conditions on 12 roads are used as parameters of the environment to describe the overall state of the environment at time t.

[0067] Specifically, the agent selects state s through the ε-greedy strategy t The release action a at the next time t t Specifically include:

[0068] A 12-element vector M is used to represent the release actions of the 12 roads.

[0069] If the i-th road is released, the vector M corresponding to the i-th road is i Recorded as 1; otherwise, the vector M corresponding to the i-th road i Recorded as 0.

[0070] At this time, the release action at time t is a t Represented as a t ={M0(t),M1(t),…,M 10 (t),M 11 (t)}.

[0071] It should be noted that, in the embodiment of the present invention, it is assumed that the vehicles maintain a constant speed, and the method for determining whether to pass the intersection without conflict using the DNQ algorithm to coordinate and control the vehicles is to determine whether to release the leading vehicle arriving at the intersection.

[0072] Since there are 12 roads in the embodiment of the present invention, a 12-element vector M is used to represent the release actions of the 12 roads. Each road has two situations: release and non-release. If the i-th road is released, the vector M corresponding to the i-th road is i Recorded as 1; otherwise, the vector M corresponding to the i-th road i Recorded as 0. Therefore, there are 2 intersections in total. 12 = 4096 actions. The action selected at time t can be described as a t ={M0(t),M1(t),…,M10 (t),M 11 (t)}, at this time a t Indicates the release status of 12 roads.

[0073] Step S3: The agent executes the release action a t , and obtain the conflict situation of the conflict point at time t.

[0074] Among them, the conflict points are Figure 3 As shown, there is a possibility of collision between straight-moving vehicles, between turning vehicles, and between straight-moving vehicles. Since the embodiment of the present invention assumes that the vehicles have already changed lanes and are moving at a constant speed in this environment, the vehicle trajectories are fixed, and the collision points are also fixed. The collision points can be used to determine whether the vehicles will collide at the intersection.

[0075] Specifically, the agent performs the release action a t Obtaining the conflict situation of the conflict point at time t specifically includes: calculating the time when the vehicles arrive at the conflict point, if multiple vehicles arrive at the conflict point at the same time, there is a conflict, otherwise there is no conflict.

[0076] It's important to note that the existing technology uses a reservation-based method to divide the conflict area into multiple grids. As long as two vehicles don't occupy the same grid at the same time, there's no conflict. This method can make better use of the conflict area, but it requires recording the occupation status of each grid in the conflict area, increasing computational complexity. In contrast, the present invention calculates the time it takes for vehicles to arrive at the conflict point. If multiple vehicles arrive at the conflict point simultaneously, there's a conflict; otherwise, there's no conflict. This eliminates the need to divide the conflict area into multiple grids and record the occupation status of each grid, reducing computational complexity and improving efficiency compared to the existing technology.

[0077] Step S4: Determine the reward r at time t according to the conflict situation. t , and obtain the state s of the entire environment at time t+1 t+1 .

[0078] Among them, the reward r at time t is determined according to the conflict situation t Before that, we need to build a reward function. The method to build a reward function is as follows:

[0079] First, since the reward function needs to be designed based on the optimization goal, in this invention, the optimization goal is to reduce the average queue time and increase traffic efficiency. Therefore, it is necessary to first determine the maximum number of vehicles that can be released at each time. The reward function for intersection release capacity is expressed as follows:

[0080] r n (t)=V(t) / V max

[0081] Among them, V(t) represents the number of vehicles released at time t, V max Indicates the maximum number of vehicles that can be released.

[0082] Secondly, the release of vehicles needs to be fair. It is not advisable to have a queue on a certain road. Therefore, the queue length is used as the reward function for fairness. The fairness reward function is expressed as follows:

[0083] r f (t)=∑(N i (t) / N max )

[0084] Among them, N i (t) represents the number of vehicles in the lane released at time t, N max Indicates the maximum capacity of a single lane.

[0085] Finally, the maximum number of vehicles released each time and the queue length are normalized as the fairness reward function to obtain the reward function at time t. The reward function at time t is expressed as follows:

[0086]

[0087] Among them, r(t) is the reward function at time t, r n (t) is the fairness reward function, r f (t) is the reward function of the intersection release capability.

[0088] Step S5: Set the state s t , release action a t , reward r t , status s t+1 The experience is integrated into the experience replay pool and stored in the experience replay pool, and the agent is trained using the experience in the experience replay pool.

[0089] in, Figure 2 The architecture diagram of the non-signalized intersection control method based on deep reinforcement learning provided in Example 1 of the present invention is as follows: Figure 2 As shown, the state s t , release action a t They are represented by 12-dimensional vector N and 12-dimensional vector M respectively and integrated. At the same time, the reward r at time t obtained from the environment is t and the state s at time t+1 t+1 They are integrated into the experience replay pool, and the experience in the experience replay pool is used to train the agent.

[0090] Step S6: Repeat the environment-agent interaction training from step S2 to step S5, and use the training results to update the DQN network.

[0091] Step S7: Use the updated DQN network to calculate the release action of the intersection without traffic lights at the current moment.

[0092] At this time, by using the updated DQN network to calculate the release action of the intersection without signal lights at the current moment, on the basis of fair vehicle release, the maximum conflict-free release of the intersection without signal lights is achieved, the average waiting time of vehicles is reduced, and the traffic efficiency is increased.

[0093] Compared with the existing deep reinforcement learning method for releasing traffic at a non-signaled intersection, the present invention uses a deep learning framework to formulate the intersection coordination problem, eliminating the need to record the occupation status of the conflict area grid, thereby reducing the complexity of the calculation; at the same time, it also fully taps the coordination potential of the intersection.

[0094] Example 2

[0095] Figure 4 This is a schematic diagram of the structure of an electronic device provided in Example 2 of the present invention, such as Figure 4 As shown, the electronic device includes a processor 21, a memory 22, an input device 23 and an output device 24; the number of processors 21 in the computer device can be one or more. Figure 4 In the figure, a processor 21 is taken as an example; the processor 21, memory 22, input device 23 and output device 24 in the electronic device can be connected by a bus or other means. Figure 4 The bus connection is taken as an example.

[0096] Memory 22, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules. Processor 21 executes the software programs, instructions, and modules stored in memory 22 to perform various electronic device functions and data processing, thereby implementing the deep reinforcement learning-based unsignalized intersection control method of Example 1.

[0097] The memory 22 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data created based on the use of the terminal, etc. Furthermore, the memory 22 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state memory device. In some instances, the memory 22 may further include a memory remotely located relative to the processor 21, and these remote memories may be connected to the electronic device via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0098] The input device 23 can be used to receive the ID and password input by the user, etc. The output device 24 is used to output the network configuration page.

[0099] Example 3

[0100] Embodiment 3 of the present invention further provides a computer-readable storage medium, wherein the computer-executable instructions, when executed by a computer processor, are used to implement the non-signalized intersection control method based on deep reinforcement learning as provided in embodiment 1.

[0101] An embodiment of the present invention provides a storage medium containing computer-executable instructions, and its computer-executable instructions are not limited to the method operations provided in Example 1, but can also execute related operations in the signal-free intersection control method based on deep reinforcement learning provided in any embodiment of the present invention.

[0102] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A non-signalized intersection control method based on deep reinforcement learning, characterized in that: include: S1. Constructing an agent and an environment, wherein the environment includes a conflict area and a lane area, and the conflict area includes a conflict point; S2. Observe the environment and obtain the overall state s of the environment at time t t , the agent selects state s through the ε-greedy strategy t The release action a at the next time t t ; S3, the agent executes the release action a t , obtain the conflict situation of the conflict point at time t; S4. Determine the reward r at time t based on the conflict situation t , and obtain the state s of the entire environment at time t+1 t+1 ; S5, change the state s t , release action a t , reward r t , status s t+1 Integrate into experience and store it in an experience replay pool, and use the experience in the experience replay pool to train the agent; S6. Repeat steps S2 to S5 to train the environment and agent interaction, and use the training results to update the DQN network; S7. Use the updated DQN network to calculate the release action at the current moment at the intersection without a signal light. Observe the environment and obtain the overall state s of the environment at time t t Specifically include: Observe the road conditions of 12 roads and use a 12-element vector N to represent the road conditions of the 12 roads respectively; If the leading vehicle on the ith road arrives at the conflict zone, the vector N corresponding to the ith road is i Recorded as the convoy length of the i-th road; otherwise, the vector N corresponding to the i-th road i Recorded as 0; where 0≤i≤11; At this time, the state of the entire environment at time t s t Indicated as s t ={N0(t),N1(t),…,N 10 (t),N 11 (t)}; The leading vehicle refers to the first vehicle traveling on the road towards the conflict zone; The reward function of intersection release capability is expressed as follows: r n (t)=V(t) / V max Among them, V(t) represents the number of vehicles released at time t, V max Indicates the maximum number of vehicles that can be released; Using queue length as the fairness reward function, the fairness reward function is expressed as follows: r f (t)=∑(N i (t) / N max ) Among them, N i (t) represents the number of vehicles in the lane released at time t, N max Indicates the maximum capacity of a single lane; The maximum number of vehicles released each time and the queue length are normalized as the fairness reward function to obtain the reward function at time t. The reward function at time t is expressed as follows: Among them, r(t) is the reward function at time t, r n (t) is the fairness reward function, r f (t) is the reward function of the intersection release capability.

2. The method for controlling intersections without signal lights based on deep reinforcement learning according to claim 1, characterized in that: The lane area has at least 12 roads.

3. The method for controlling intersections without signal lights based on deep reinforcement learning according to claim 1, characterized in that: The agent selects state s through the ε-greedy strategy t The release action a at the next time t t Specifically include: A 12-element vector M is used to represent the release actions of the 12 roads respectively; If the i-th road is released, the vector M corresponding to the i-th road is i Recorded as 1; otherwise, the vector M corresponding to the i-th road i Recorded as 0; At this time, the release action at time t is a t Represented as a t ={M0(t),M1(t),…,M 10 (t),M 11 (t)}.

4. The method for controlling intersections without signal lights based on deep reinforcement learning according to claim 1, characterized in that: The agent performs the release action a t Obtaining the conflict situation of the conflict point at time t specifically includes: calculating the time when the vehicles arrive at the conflict point, if multiple vehicles arrive at the conflict point at the same time, there is a conflict, otherwise there is no conflict.

5. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for controlling an intersection without signal lights based on deep reinforcement learning as described in any one of claims 1 to 4 is implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for controlling a non-signalized intersection based on deep reinforcement learning as described in any one of claims 1 to 4 is implemented.

Images