Variable lane and signal cooperative control method, system, equipment and medium

Abstract

The invention discloses a variable lane and signal cooperative control method, system and device and a medium, and relates to the technical field of variable lane control, and the method comprises the steps: obtaining the traffic state data of an intersection; inputting the traffic state data into a reinforcement learning model to generate a switching strategy of the variable lane; the switching strategy of the variable lane comprises the steps of switching the state of the variable lane into left turning or straight going, and keeping three signal period durations; according to the switching strategy of the variable lane, performing corresponding timing on the signal; according to the method, collaborative optimization of the variable lane and signal control is realized, and the time and space resources of the intersection are optimized, so that the resources can be reasonably allocated, and the traffic efficiency of the intersection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of variable lane control, and in particular to a variable lane and signal coordinated control method, system, device and medium. Background Art

[0002] With the rapid growth of the number of motor vehicles in my country, the contradiction between road traffic supply and demand has become increasingly prominent, especially at intersections, where the imbalance between straight and left-turn traffic flows can easily lead to traffic congestion. Figure 1 It can be seen that the queue of left-turning vehicles at the current intersection is short, while the queue of straight-moving vehicles is long. In this case, if two left-turn lanes are fixed, it will cause a waste of lane resources. At the same time, the long queue caused by the lack of straight-moving lanes will directly affect the traffic efficiency of adjacent lanes and upstream lanes, causing congestion at the entire intersection, and even the congestion will further spread to the surrounding areas, thereby affecting the traffic capacity of the entire urban road network.

[0003] As a means of traffic management, variable lane technology effectively combines time and space optimization, and can dynamically adjust lane functions according to traffic demand, thereby improving the efficiency of road resource use. At present, the control method of variable guide lanes mainly adopts traditional manual or experience control methods. Generally, the variable lane function is manually switched based on experience and historical data, or a control scheme is formed through research to switch the variable lane function at a fixed time. According to experience or historical data rules, the control scheme of pre-setting the switching time of the variable guide lane function has a better control effect on the traffic state with regular changes in traffic flow.

[0004] However, the traditional manual or timed control method of variable lanes has limitations. When road traffic conditions change rapidly or abnormal traffic incidents occur, although a single variable lane setting can dynamically adjust the number of lanes to adapt to different traffic flow requirements, this method is difficult to make timely adjustments and cannot completely solve the problem of traffic congestion. Summary of the invention

[0005] In view of the deficiency of the existing technology that a single variable lane setting is difficult to solve traffic congestion, the present invention proposes a variable lane and signal coordinated control method, system, equipment and medium, which coordinately optimizes the variable lane control and signal timing scheme through a variable lane control strategy based on reinforcement learning, thereby solving the problems existing in the existing technology.

[0006] A variable lane and signal coordinated control method, comprising the following steps:

[0007] Obtaining real-time traffic status data at the intersection; the traffic status data includes the position and speed of each vehicle, the queue length of vehicles in the variable lane, and the duration of the green light in the signal controller;

[0008] A reinforcement learning model is constructed; the reinforcement learning model includes a state space, an action space and a reward function; the state space is used to record the position and speed of each vehicle and the length of the lane queue in real time; the action space is used to make decisions on the variable lane according to the position and speed of each vehicle and the length of the queue of vehicles in the variable lane, and the decision includes switching the state of the variable lane to turn left or go straight; the reward function is used to set a reward value based on the queue length of vehicles in each lane of the current intersection, the waiting time of vehicles and the number of stops, and feed the reward value back to the action space to generate an expected decision;

[0009] Inputting real-time traffic status data into the reinforcement learning model to generate a switch decision for a variable lane;

[0010] The signal controller is timed through the switching decision of the variable lane; when the variable lane is switched to straight driving, the green light duration of the straight driving phase in the variable lane direction is extended by t seconds, and the green light duration of the left turn phase is reduced by t seconds; when the variable lane is switched to left turn, the green light duration of the left turn phase in the variable lane direction is extended by t seconds, and the green light duration of the straight driving phase is reduced by t seconds.

[0011] Furthermore, the state space is used to record the position and speed of each vehicle and the lane queue length in real time, specifically including the following steps:

[0012] Create a two-layer two-dimensional matrix to store traffic status; the first layer of the matrix records the specific location of each vehicle in the detection area, and the second layer of the matrix is ​​used to record the speed of the vehicle;

[0013] When a vehicle enters the detection area, the distance from the vehicle to the detection area and the distance from the vehicle to the intersection are obtained in real time, and the ratio of each distance to the vehicle length is calculated to determine the column number of the vehicle in the matrix, and then the specific position of each vehicle is recorded; when there is a vehicle at a certain position, the corresponding position value of the vehicle position in the matrix is ​​set to 1, and the speed information of the vehicle is recorded in the speed matrix at the position; the specific expression is:

[0014]

[0015] in: represents the position of the nth vehicle in lane 1 at time step t; represents the speed of the nth vehicle in lane 1 at time step t; Indicates the traffic status of lane 1; s t Indicates the traffic status of the entire intersection;

[0016] By looking at each row of the vehicle position matrix, the queue length of the lane is obtained.

[0017] Furthermore, the reward function is used to set the reward value according to the queue length, vehicle waiting time and number of stops of vehicles in the variable lane of the current intersection, which is specifically expressed as:

[0018]

[0019] Among them, k1, k2, k3 are weight coefficients, is the sum of the lengths of the vehicle queues in each lane before the intersection, is the sum of the waiting time of vehicles before the intersection, i is the lane number of the four directions of the intersection; is the sum of the number of vehicle stops, and j is the vehicle number.

[0020] Furthermore, a normalization processing method is used to determine the values ​​of the queue lengths of each lane before the intersection, the waiting time of vehicles before the intersection, and the number of vehicle stops within the range of 0-1, and when calculating the reward value, the queue lengths of each lane before the intersection, the waiting time of vehicles, and the number of vehicle stops are multiplied by a negative value. The smaller the sum of the queue lengths of each lane before the intersection, the sum of the waiting time of vehicles, and the sum of the number of vehicle stops, the greater the reward, and vice versa.

[0021] Furthermore, the method further includes training the reinforcement learning model, wherein the training steps specifically include:

[0022] First, initialize the intersection state and the hyperparameters, including learning rate, discount factor, training batch, sampling size, experience pool size, and network Q(s,a,w) parameters; initialize the action;

[0023] The environment state is input into the reinforcement learning model, and the action is output according to the state, and the action is applied to the environment; after the action is taken, the environment state changes, the reward value is calculated, and the environment state, action, and reward value are stored in the experience pool;

[0024] When the number of samples in the experience pool reaches the threshold, samples are randomly drawn from the experience pool for training and learning. The loss is calculated by the gradient descent method, and the target network parameters are updated at regular intervals. After multiple iterative updates, the training of the reinforcement learning model is completed.

[0025] The present invention also includes a variable lane and signal coordinated control system, including:

[0026] An acquisition module is used to acquire real-time traffic status data of the intersection; the traffic status data includes the position and speed of each vehicle, the queue length of vehicles in the variable lane, and the duration of the green light in the signal controller;

[0027] A model building module is used to build a reinforcement learning model; the reinforcement learning model includes a state space, an action space and a reward function; the state space is used to record the position and speed of each vehicle and the length of the lane queue in real time; the action space is used to make decisions on the variable lane according to the position and speed of each vehicle and the length of the queue of vehicles in the variable lane, and the decision includes switching the state of the variable lane to turn left or go straight; the reward function is used to set a reward value based on the queue length of vehicles in each lane of the current intersection, the waiting time of vehicles and the number of stops, and feed the reward value back to the action space to generate an expected decision;

[0028] A strategy generation module, used to input real-time traffic status data into the reinforcement learning model to generate a switch decision for a variable lane;

[0029] The collaborative control module is used to time the signal controller through the switching decision of the variable lane; when the variable lane is switched to straight driving, the green light duration of the straight driving phase in the variable lane direction is extended by t seconds, and the green light duration of the left turn phase is reduced by t seconds; when the variable lane is switched to left turn, the green light duration of the left turn phase in the variable lane direction is extended by t seconds, and the green light duration of the straight driving phase is reduced by t seconds.

[0030] The present invention also includes a variable lane and signal coordinated control computer device, including: a memory, a processor and a computer program stored in the memory, and the processor implements the steps of the variable lane and signal coordinated control method when executing the computer program.

[0031] Furthermore, a readable storage medium is provided, wherein the readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, they are used to execute the steps of the variable lane and signal coordinated control method.

[0032] The present invention provides a variable lane and signal coordinated control method, system, device and medium, which have the following beneficial effects:

[0033] The present invention constructs the state space, action space and reward function of reinforcement learning, and aims to provide a variable lane control scheme according to the real-time traffic status and the reward value set by the queue length of vehicles in each lane of the current intersection, the waiting time of vehicles and the number of parking times, and to time the signal controller through the control scheme of the variable lane to achieve the coordinated optimization of the variable lane and signal control; through these two steps, the time and space resources of the intersection are optimized, and the resources can be reasonably allocated to improve the traffic efficiency of the intersection. By effectively combining time and space optimization, the intersection road resources can be dynamically allocated according to the real-time traffic status, thereby improving the time and space utilization of traffic resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of the unbalanced phenomenon of left-turn and straight-through traffic flow in the background technology of the present invention;

[0035] Figure 2 This is a schematic diagram of a reinforcement learning structure in an embodiment of the present invention;

[0036] Figure 3 This is a schematic diagram of the structure of an intelligent agent in an embodiment of the present invention;

[0037] Figure 4 This is a collaborative control flow chart in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0039] The present invention proposes a method for coordinated control of variable lanes and signals, constructs a variable lane control agent, designs the agent's state space, action space and reward function, and the method aims to provide a variable lane control plan based on real-time traffic conditions. Then, a signal timing optimization method based on reinforcement learning is studied, a signal timing agent is constructed, and the agent's state space, action space and reward function are designed. The reinforcement learning signal timing optimization model is trained separately according to the functions of the variable guide lanes. A two-layer control model for variable lanes and signal control at intersections is designed, the upper layer is a variable guide lane function decision model, and the lower layer optimizes the signal timing plan according to the function of the variable guide lane, thereby realizing coordinated optimization of variable lanes and signal control. Through these two steps, the time and space resources of the intersection are optimized, resources can be reasonably allocated, and the traffic efficiency of the intersection is improved. Specifically, the following steps are included:

[0040] S1. A two-layer control approach is used to establish a collaborative control model of variable directional lane adaptive control and signal timing optimization using reinforcement learning.

[0041] S2. Design the three elements of reinforcement learning, including state space, action space and reward function.

[0042] Reinforcement learning algorithms rely on interaction with the environment and require continuous trial and error learning to optimize action generation strategies, such as Figure 2 Specifically, the agent obtains state information from the environment, takes actions based on the information, and acts on the environment. When the state of the environment changes, the agent calculates the reward after taking the action based on the changes in the environment.

[0043] The state space is defined as follows: First, a two-layer two-dimensional matrix is ​​created to store the traffic state. The first layer of the matrix accurately records the specific position of each vehicle in the detection area, while the second layer of the matrix is ​​used to record the speed of the vehicle. This data acquisition method can not only obtain more information, but also does not have a lot of redundant information like directly inputting images, which can speed up training.

[0044]

[0045] Where: represents the position of the nth vehicle in lane 1 at time step t; represents the speed of the nth vehicle in lane 1 at time step t; Indicates the traffic status of lane 1; s t Indicates the traffic status of the entire intersection.

[0046] For example, if the length of each lane in the detection area is 200 meters and the length of the vehicle is 5 meters, each lane is divided into 40 grids, each grid is regarded as a "cell", each "cell" has space to accommodate a vehicle, and the collected speed and position information are recorded in this matrix. When a vehicle enters the detection area, the system will obtain the distance of the vehicle entering the detection area and the distance of the vehicle from the intersection in real time. By calculating the ratio of these distances to the length of the vehicle, the column number of the vehicle in the matrix can be determined, and then combined with the row number to accurately record the specific position of each vehicle. If there is a vehicle at a certain position, the value of the corresponding position in the vehicle position matrix is ​​set to 1, and the speed information of the vehicle is recorded in the speed matrix at that position. By looking at each row of the vehicle position matrix, the queue length information of the lane can be obtained. In order to simplify the data calculation process, the vehicle speed is normalized. If there is no vehicle at the corresponding position of the lane, the element values ​​of the corresponding position in the vehicle position matrix and the speed matrix are set to 0. The information contained in the final state space includes vehicle position, vehicle speed, and lane queue length.

[0047] The action space is defined as:

[0048] The reinforcement learning agent takes actions at each decision step. The variable-guide lane agent takes two actions: the variable-guide lane function switches to a left turn and the variable-guide lane function switches to going straight.

[0049] The signal timing agent optimizes the timing plan according to the function of the current variable oriented lane after the variable oriented lane agent takes action. The signal timing agent has three actions: keep the current signal duration unchanged; extend the green light duration of the straight phase in the variable lane direction and reduce the green light duration of the left turn phase accordingly; extend the green light duration of the left turn phase in the variable lane direction and reduce the green light duration of the straight phase. In order to avoid traffic congestion in other directions caused by too long green light time in a certain phase, the upper and lower limits of the green light duration are set, and each time before adjusting the green light duration, it is checked whether it is within the upper and lower limits.

[0050] The reward function is defined as:

[0051] The calculation of the reward consists of three parts: first, the sum of the queue lengths of each lane in front of the intersection at the decision time, which is an indicator that can directly reflect the traffic pressure at the intersection; second, the sum of the accumulated waiting time of vehicles; and finally, the sum of the number of stops in each lane at the decision time is also considered.

[0052]

[0053] Where k1, k2, k3 are weight coefficients, is the sum of the lengths of the vehicle queues in each lane before the intersection, is the sum of the waiting time of vehicles before the intersection, i is the lane number of the four directions of the intersection; is the sum of the number of vehicle stops, and j is the vehicle number.

[0054] In order to avoid affecting the reward calculation and thus the decision-making of the intelligent agent due to the excessive value of a certain indicator, the queue length, waiting time and number of stops are normalized. After normalization, the values ​​of queue length, waiting time and number of stops are all in the range of 0-1. When calculating the reward, these three will be multiplied by a negative coefficient. The smaller the queue length and other indicators are, the greater the reward is, and vice versa.

[0055] S3. In a single intersection environment, the variable guide lane adaptive control decision model is first trained, and then the signal timing model is trained according to the function of the variable guide lane (left turn or straight going).

[0056] S3.1. First, initialize the intersection state and the hyperparameters, including learning rate, discount factor, training batch, sampling size, experience pool size, and network Q(s,a,w) parameters; initialize the action.

[0057] S3.2. Preprocess the environment state and input it into the model. Output the action according to the state and apply the action to the environment. The action is taken according to the greedy strategy. After the action is taken, the environment changes. The reward is calculated and the state, action and reward are stored in the experience pool.

[0058] S3.3. When the number of samples in the experience pool reaches the threshold, samples are randomly drawn from the experience pool for training and learning, the loss is calculated by the gradient descent method, and the target network parameters are updated at regular intervals.

[0059] S4. After the model training is completed, the variable guide lane adaptive control model and the signal timing model are saved, and a two-layer control is used to realize the variable guide lane and signal timing optimization coordinated control model. The upper model is the variable guide lane adaptive control model, and the lower model is the signal timing optimization model. When the function of the variable guide lane changes, the signal timing will optimize the timing plan according to whether the lane function is straight or left turn, so as to realize coordinated control, such as Figure 3 shown.

[0060] S4.1. First, keep the initial signal timing plan unchanged and monitor the traffic status in real time.

[0061] S4.2, the upper variable guide lane decision model determines whether the variable lane function needs to be switched based on the information obtained. If the function needs to be switched, the switching operation is performed immediately. After each switching of the variable lane function, three signal cycles must be maintained before the function can be switched again.

[0062] S4.3. When the function of the variable guide lane changes, new traffic status information is obtained again. This information will be used as the input of the signal control model. The signal timing model will give a new timing plan based on the current function of the variable guide lane and the traffic status.

[0063] S4.4. According to the result of the timing scheme optimization, the traffic status information is obtained again and used as the input of the variable guide lane decision model to determine whether the variable guide lane function needs to be switched.

[0064] S4.5. Repeat the above process.

[0065] Simulation experiment:

[0066] The algorithm evaluation indicators used in the present invention are: (1) Average queue length: This indicator reflects the number of vehicles waiting to pass on the road. The shorter the average queue length, the fewer vehicles are waiting on the road, reflecting the smoothness of traffic at the intersection. (2) Average waiting time: refers to the average waiting time required for vehicles to pass through the intersection. (3) Average delay time: This indicator measures the difference between the actual driving speed of the vehicle and the ideal speed. (4) Traffic volume: It directly reflects the number of vehicles passing through the intersection in the same time.

[0067] SUMO is used to build an intersection simulation scene, and signal lights, detectors and other facilities are configured in the road network. The Python Traci interface is used as a bridge to connect the control algorithm to the SUMO simulation environment. During the experiment, the agent interacts with the environment to obtain the intersection state, which is first input into the variable guide lane decision model. The model switches the function of the variable guide lane according to the real-time traffic state data. When the lane function changes, the traffic state will also change. The original signal timing plan is no longer applicable. The signal timing model adjusts the green light time of the corresponding phase according to the current lane function to optimize the timing plan and apply it to the intersection. Complete a complete decision-making process.

[0068] The present invention first proposes a variable guide lane adaptive control algorithm based on reinforcement learning. The algorithm aims to adapt to complex and changeable traffic flows through intelligent decision-making and switch the function of the variable guide lane according to the real-time traffic status. The collaborative control algorithm optimizes the signal timing scheme based on the adaptive control of the variable guide lane. By giving a reasonable timing scheme according to the function of the variable guide lane, the algorithm further exerts the function of the variable guide lane. Compared with various control algorithms such as induction control and timing control methods, the comparison results are shown in Table 1.

[0069] Table 1 Performance comparison of variable guide lane and signal optimization collaborative control method

[0070]

[0071]

[0072] From the comparison results, it can be seen that the VL-TLC (Variable Lane and Signal Optimization Cooperative Control) algorithm performs better. In the left-turn scenario, compared with other algorithms, the average waiting time is reduced by 17.9% to 24.1%, the average delay time is reduced by 9.4% to 20.3%, the average queue length is shortened by 22.1% to 34.7%, and the traffic volume is increased by 4.5% to 21.9%. In the straight-through scenario, the average waiting time is reduced by 18% to 47%, the average delay time is reduced by 17.5% to 44.9%, the average queue length is shortened by 35.9% to 57.9%, and the traffic volume is increased by 8.8% to 22.4%.

[0073] The collaborative control algorithm optimizes the signal timing plan based on the variable guide lane control. By giving a reasonable timing plan based on the function of the variable guide lane, the time and space resources of the intersection are more fully utilized, effectively reducing the delay of the intersection and improving the traffic efficiency of the intersection.

[0074] Based on the same inventive concept, the present invention also proposes a variable lane and signal coordinated control system, comprising:

[0075] The acquisition module is used to obtain real-time traffic status data of the intersection; the traffic status data includes the position and speed of each vehicle and the lane queue length.

[0076] A model building module is used to build a reinforcement learning model; the reinforcement learning model includes a state space, an action space and a reward function; the state space is used to record the position and speed of each vehicle and the length of the lane queue in real time; the action space is used to make decisions on the variable lanes according to the position and speed of each vehicle and the length of the queue of each lane, and the decision includes switching the state of the variable lane to turn left or go straight, and maintaining the duration of three signal cycles; the reward function is used to set a reward value based on the queue length of each entrance lane of the current intersection, the vehicle waiting time and the number of stops, and feed the reward value back to the action space to generate an expected decision.

[0077] The strategy generation module is used to input real-time traffic status data into the reinforcement learning model to generate variable lane switching decisions.

[0078] The collaborative control module is used to time the signal controller through the switching decision of the variable lane; when the variable lane is switched to straight driving, the green light duration of the straight driving phase in the variable lane direction is extended by t seconds, and the green light duration of the left turn phase is reduced by t seconds; when the variable lane is switched to left turn, the green light duration of the left turn phase in the variable lane direction is extended by t seconds, and the green light duration of the straight driving phase is reduced by t seconds.

[0079] Based on the same inventive concept, the present invention also proposes a variable lane and signal coordinated control computer device, including: a memory, a processor, and a computer program stored in the memory. When the processor executes the computer program, the steps of the variable lane and signal coordinated control method are implemented.

[0080] Based on the same inventive concept, the present invention also proposes a readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, they are used to execute the steps of the variable lane and signal coordinated control method.

[0081] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A variable lane and signal coordinated control method, characterized in that: The following steps are involved: Obtaining real-time traffic status data at the intersection; the traffic status data includes the position and speed of each vehicle, the queue length of vehicles in the variable lane, and the duration of the green light in the signal controller; Constructing a reinforcement learning model; the reinforcement learning model includes a state space, an action space, and a reward function; the state space is used to record the position and speed of each vehicle and the length of the lane queue in real time; The action space is used to make decisions on the variable lanes according to the position and speed of each vehicle and the queue length of the vehicles in the variable lanes, and the decision includes switching the variable lane state to turn left or go straight; the reward function is used to set the reward value according to the queue length of vehicles in each lane of the current intersection, the waiting time of the vehicles and the number of stops, and feed the reward value back to the action space to generate an expected decision; Inputting real-time traffic status data into the reinforcement learning model to generate a switch decision for a variable lane; The signal controller is timed by the switching decision of the variable lane; when the variable lane is switched to straight driving, the green light duration of the straight driving phase in the variable lane direction is extended by t seconds, and the green light duration of the left turning phase is reduced by t seconds; When the variable lane switches to a left turn, the green light duration of the left turn phase in the variable lane direction is extended by t seconds, while the green light duration of the straight phase is reduced by t seconds.

2. The variable lane and signal coordinated control method according to claim 1, characterized in that: The state space is used to record the position and speed of each vehicle and the lane queue length in real time, and specifically includes the following steps: Create a two-layer two-dimensional matrix to store traffic status; the first layer of the matrix records the specific location of each vehicle in the detection area, and the second layer of the matrix is ​​used to record the speed of the vehicle; When a vehicle enters the detection area, the distance from the vehicle to the detection area and the distance from the vehicle to the intersection are obtained in real time, and the ratio of each distance to the vehicle length is calculated to determine the column number of the vehicle in the matrix, and then the specific position of each vehicle is recorded; when there is a vehicle at a certain position, the corresponding position value of the vehicle position in the matrix is ​​set to 1, and the speed information of the vehicle is recorded in the speed matrix at the position; the specific expression is: in: represents the position of the nth vehicle in lane 1 at time step t; represents the speed of the nth vehicle in lane 1 at time step t; Indicates the traffic status of lane 1; s t Indicates the traffic status of the entire intersection; By looking at each row of the vehicle position matrix, the queue length of the lane is obtained.

3. The variable lane and signal coordinated control method according to claim 1, characterized in that: The reward function is used to set the reward value based on the queue length, vehicle waiting time and number of stops of the variable lane vehicles at the current intersection, which is specifically expressed as: Among them, k1, k2, k3 are weight coefficients, is the sum of the lengths of the vehicle queues in each lane before the intersection, is the sum of the waiting time of vehicles before the intersection, i is the lane number of the four directions of the intersection; is the sum of the number of vehicle stops, and j is the vehicle number.

4. The variable lane and signal coordinated control method according to claim 3, characterized in that: The normalization processing method is adopted to determine the values ​​of the queue length of each lane before the intersection, the waiting time of vehicles before the intersection, and the number of vehicle stops within the range of 0-1, and when calculating the reward value, the queue length of each lane before the intersection, the waiting time of vehicles, and the number of vehicle stops are multiplied by a negative value. The smaller the sum of the queue lengths of each lane before the intersection, the sum of the waiting time of vehicles, and the sum of the number of vehicle stops, the greater the reward, and vice versa.

5. The variable lane and signal coordinated control method according to claim 1, characterized in that: The reinforcement learning model is further trained, wherein the training steps specifically include: First, initialize the intersection state and the hyperparameters, including learning rate, discount factor, training batch, sampling size, experience pool size, and network Q(s,a,w) parameters; initialize the action; The environment state is input into the reinforcement learning model, and the action is output according to the state, and the action is applied to the environment; after the action is taken, the environment state changes, the reward value is calculated, and the environment state, action, and reward value are stored in the experience pool; When the number of samples in the experience pool reaches the threshold, samples are randomly drawn from the experience pool for training and learning. The loss is calculated by the gradient descent method, and the target network parameters are updated at regular intervals. After multiple iterative updates, the training of the reinforcement learning model is completed.

6. A variable lane and signal coordinated control system, characterized in that: include: An acquisition module is used to acquire real-time traffic status data of the intersection; the traffic status data includes the position and speed of each vehicle, the queue length of vehicles in the variable lane, and the duration of the green light in the signal controller; A model building module, used to build a reinforcement learning model; the reinforcement learning model includes a state space, an action space and a reward function; the state space is used to record the position and speed of each vehicle and the lane queue length in real time; The action space is used to make decisions on the variable lanes according to the position and speed of each vehicle and the queue length of the vehicles in the variable lanes, and the decision includes switching the variable lane state to turn left or go straight; the reward function is used to set the reward value according to the queue length of vehicles in each lane of the current intersection, the waiting time of the vehicles and the number of stops, and feed the reward value back to the action space to generate an expected decision; A strategy generation module, used to input real-time traffic status data into the reinforcement learning model to generate a switch decision for a variable lane; A collaborative control module is used to time the signal controller through the switching decision of the variable lane; when the variable lane is switched to straight driving, the green light duration of the straight driving phase in the variable lane direction is extended by t seconds, and the green light duration of the left turning phase is reduced by t seconds; When the variable lane switches to a left turn, the green light duration of the left turn phase in the variable lane direction is extended by t seconds, while the green light duration of the straight phase is reduced by t seconds.

7. A variable lane and signal coordinated control computer device, characterized in that: include: A memory, a processor, and a computer program stored in the memory, wherein the processor implements the steps of the variable lane and signal coordinated control method described in any one of claims 1 to 5 when executing the computer program.

8. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which includes program instructions. When the program instructions are executed by a processor, they are used to execute the steps of the variable lane and signal coordinated control method described in any one of claims 1-5.

Images