Intelligent two-way single-lane reconstruction and extension signal control method and system

Abstract

The invention provides an intelligent two-way single-lane reconstruction and extension signal management and control method and system, and belongs to the technical field of road traffic planning. Comprising the following contents; establishing a two-way single-lane simulation road network, and arranging a signal lamp and a visual identification unit at an entrance lane of a reconstruction and extension construction area of the simulation road network; an intelligent agent based on a deep reinforcement learning algorithm D3QN-PER is constructed; in the environment of the simulation road network, the intelligent agent is trained based on a reward function, and the reward function is based on the number of vehicles running at the speed lower than the set running speed in the last time step in the reconstruction and extension construction area; and regulating and controlling traffic control signal lamps of the reconstruction and extension construction area through the trained intelligent agent, and selecting an optimal signal lamp state for output. According to the method, after the simulation environment is established, the positions and the motion states of the vehicles are considered, the intelligent agent is trained by reducing the number of the vehicles running at a low speed in the reconstruction and extension construction area, and the vehicles in the two-way single lane are guided to pass.

Description

Technical Field

[0001] The present invention relates to the technical field of road traffic planning, and in particular to an intelligent two-way single lane reconstruction and expansion signal control method and system. Background Art

[0002] With the rapid development of my country's social economy, residents' consumption capacity continues to rise, urban scale continues to expand, and residents' travel needs are also growing. Many grade road planning has been completed ahead of schedule, especially some two-way two-lane low-grade roads, which are facing the problem of reconstruction and expansion. When a two-way two-lane occupies one lane due to reconstruction and expansion, resulting in traffic congestion, the narrow passing space will significantly reduce traffic safety and traffic capacity, which cannot match the people's growing needs for a better life. Therefore, traffic control around the reconstruction and expansion construction area has become a major key project in the current transportation and infrastructure fields.

[0003] Traditional traffic signal control methods are often based on timing control or inductive control. These methods are not effective in dealing with complex and changeable traffic flows in construction areas and are difficult to effectively alleviate traffic congestion. Therefore, providing an intelligent two-way single lane reconstruction and expansion signal control method and system to adapt to the traffic characteristics under reconstruction and expansion scenarios and improve road traffic efficiency has become an urgent problem to be solved in the current transportation field. Summary of the invention

[0004] In view of this, the present invention proposes an intelligent two-way single lane reconstruction and expansion signal control method and system, which can improve the traffic efficiency problem of the two-way single lane during construction and effectively alleviate congestion.

[0005] On the one hand, the present invention provides an intelligent two-way single lane reconstruction and expansion signal control method, comprising the following steps: S1: Establish a simulated road network with two-way single lanes, and set up signal lights and visual recognition units at the entrance of the reconstruction and expansion construction area of ​​the simulated road network; S2: Build an intelligent agent based on the deep reinforcement learning algorithm D3QN-PER; S3: In the simulated road network environment, the agent is trained based on a reward function, wherein the reward function is based on the number of vehicles traveling at a speed lower than a set driving speed in the reconstruction and expansion construction area in the previous time step; S4: Based on the above technical solution, preferably, the construction of the intelligent agent described in step S2 includes the construction of a state set and the construction of an action set. The state set is constructed based on an improved discrete traffic state encoding method DTSE, and the action set adopts a binary action method to construct an action set of an intelligent agent based on a deep reinforcement learning algorithm. A ; The constructed intelligent agent interacts with the simulated road network.

[0006] Preferably, the state set is constructed as follows: the entrance road of the reconstruction and expansion construction area is divided into four cells of different lengths according to the distance from the parking line of the single lane entering the construction area and the supervision distance of the visual recognition unit from near to far [ s 1 , s 2 , s 3 , s 4 ], the lengths of each cell are 10%, 20%, 30% and 40% of the supervision distance of the visual recognition unit respectively; based on the vehicle information on each cell, an agent state set based on the deep reinforcement learning algorithm is constructed ,in For the i The first state variable of the cell state set, For the i The second state variable of the cell state set, For the i The third state variable of the cell state set, i =1, 2, 3, 4.

[0007] More preferably, i The first state variable of the cell state set , is at the entrance of the renovation and expansion construction area, and the center point of the vehicle is located in the cell i The number of vehicles on i The second state variable of the cell state set , is constructed based on the cell where the vehicle is located and the vehicle speed at the entrance of the renovation and expansion construction area. The second state variable The expression is , Indicates that the center point of the vehicle is located in the cell i On k The absolute value of the vehicle's speed; i The expression of the third state variable of the cell state set is , The center point of the vehicle is located in the cell i On k The absolute value of the vehicle's acceleration.

[0008] Preferably, construct an action set of an intelligent agent based on a deep reinforcement learning algorithm A , is to generate a binary action set by combining the vehicle positions and motion states in the entrance lanes on both sides of the construction area A={0,1}, wherein the entrance road of the reconstruction and expansion construction area has a first end and a second end that are relatively set, and the state of the signal light at the first end of the reconstruction and expansion construction entrance is green, and the state of the signal light at the second end is red, if the action of the intelligent agent at the current moment is 1, then within the action step, the signal light at the first end is switched to yellow and lasts for a first time, and then switched to a red light state, and when the signal lights at the first end and the signal lights at the second end both maintain red for a second time, the traffic light at the second end is switched to green, and the signal light at the first end remains unchanged; conversely, when the state of the signal light at the second end of the reconstruction and expansion construction entrance is green, and the state of the signal light at the first end is red, if the action of the intelligent agent at the current moment is 1, then within the action step, the signal light at the second end is switched to yellow and lasts for a first time, and then switched to a red light state, and when the signal lights at the first end and the signal lights at the second end both maintain red for a second time, the traffic light at the first end is switched to green, and the signal light at the second end remains unchanged; If the action of the agent at the current moment is 0, the current states of the traffic lights at the first end and the second end are maintained unchanged within the action step.

[0009] Further preferably, the reward function described in step S3 is to make the current action step length t The reward value is R t , the expression of the reward function is R t = -num ( v i < v th ), num ( v i < v th ) indicates that within the previous action step, the vehicle speed on the lane is less than the set driving speed v th The average number of vehicles.

[0010] Further preferably, the content of step S4 is to obtain real-time road traffic information of the two-way single lane reconstruction and expansion construction area, and determine the optimal action through the trained intelligent agent using the D3QN-PER model, so that the traffic lights at the first and second ends of the reconstruction and expansion construction entrance output the output state corresponding to the optimal action of the intelligent agent.

[0011] Preferably, the establishment of the simulated road network of two-way single lane in step S1 is to establish the simulated road network through the SUMO simulation platform and perform simulation interaction.

[0012] Further preferably, the content of the simulation interaction is: taking the area between the parking lines at the first end and the second end of the renovation and expansion construction area as the unit perception area, initializing the traffic simulation environment, preheating the environment, allowing simulated vehicles to fill the road, using the Traci interface to realize the interaction between the intelligent agent and the simulation environment, obtaining the interaction data of each simulated vehicle within each action step, and inputting the vehicle's interaction data into the intelligent agent based on the deep reinforcement learning algorithm D3QN-PER.

[0013] On the other hand, the present invention provides an intelligent two-way single lane reconstruction and expansion signal control system, which is used to execute the above-mentioned intelligent two-way single lane reconstruction and expansion signal control method, including: Signal lights are respectively arranged at the first end and the second end of the entrance road of the reconstruction and expansion construction area, and are used to output the indication signal of the road passing or stopping; The visual recognition unit is installed on the top of the signal light, facing the entrance of the reconstruction and expansion construction area, and is used to obtain images of the entrance of the construction area and the single lane in the construction area; The main control unit is connected to the visual recognition units at the first and second ends of the entrance road of the reconstruction and expansion construction area, receives images transmitted by each visual recognition unit, recognizes vehicle information, analyzes the speed and number of target vehicles, uses the trained intelligent agent to judge the target vehicle entering the reconstruction and expansion construction area through the D3QN-PER algorithm, selects the optimal action, and outputs the optimal action to the signal light; at the same time, when the active unit finds that there is a traffic jam in the single lane of the construction area and it cannot be solved by passing vehicles, it uses the trained intelligent agent to adjust in time; The power supply unit is electrically connected to the signal light, the visual recognition unit and the main control unit respectively, and is used to provide working power.

[0014] The intelligent two-way single lane reconstruction and expansion signal control method and system provided by the present invention has the following beneficial effects compared with the prior art: The method establishes a simulation scenario for agent training; constructs an agent based on a deep reinforcement learning algorithm, wherein the construction includes the construction of a state set and an action set of the agent, and the construction of the state set takes into account the position and motion state of vehicles in the entrance roads on both sides of the construction area; in the simulation scenario, the agent is trained based on a reward function, wherein the setting of the reward function takes into account guiding the agent to reduce the number of low-speed vehicles; through the trained agent, the two-way single-lane traffic signal lights in the reconstruction and expansion construction area are regulated, so that the position and motion state of vehicles in the entrance roads on both sides of the construction area are integrated into the state set construction of the reinforcement learning agent, and the setting of the reward function for training takes into account the low-speed vehicles between different time steps, so that the trained agent can more effectively clear the roads around the reconstruction and expansion construction area and alleviate congestion when deriving the signal light control strategy. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A flowchart of the method steps of an intelligent two-way single lane reconstruction and expansion signal control method and system of the present invention; Figure 2 A schematic diagram of a simulated road network of an intelligent two-way single lane reconstruction and expansion signal control method and system of the present invention; Figure 3 A schematic diagram of an average vehicle queue length curve in a two-way single lane construction area of ​​an intelligent two-way single lane reconstruction and expansion signal control method and system of the present invention; Figure 4 A curve diagram of average vehicle travel time in a two-way single lane construction area of ​​an intelligent two-way single lane reconstruction and expansion signal control method and system of the present invention; Figure 5 A schematic diagram of an average vehicle waiting time curve in a two-way single lane construction area of ​​an intelligent two-way single lane reconstruction and expansion signal control method and system of the present invention; Figure 6 A schematic diagram of an average vehicle CO2 emission curve in a two-way single lane construction area of ​​an intelligent two-way single lane reconstruction and expansion signal control method and system of the present invention; Figure 7 A schematic diagram of an average vehicle fuel consumption curve in a two-way single lane construction area of ​​an intelligent two-way single lane reconstruction and expansion signal control method and system of the present invention; Figure 8 A schematic diagram of a deep reinforcement learning framework of an intelligent two-way single lane reconstruction and expansion signal control method and system of the present invention; Fig. 9 A schematic diagram of a network framework of a deep reinforcement learning model of an intelligent two-way single lane reconstruction and expansion signal control method and system of the present invention; Fig.10 The present invention is a structural schematic diagram of the system composition of an intelligent two-way single lane reconstruction and expansion signal control method and system. DETAILED DESCRIPTION

[0017] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] like Figure 1 As shown, on the one hand, the present invention provides an intelligent two-way single lane reconstruction and expansion signal control method, comprising the following steps: S1: Establish a simulated road network with two-way single lanes, and set up traffic lights and visual recognition units at the entrance of the reconstruction and expansion construction area of ​​the simulated road network.

[0019] The specific content of step S1 is as follows: Figure 2 As shown, a simulated road network is established through the SUMO simulation platform, and simulated interaction is performed. There are two-way lanes at both ends of the simulated road network, but there is a reconstruction and expansion construction area in the middle of the road. Within this area, only one lane is available for traffic. This embodiment uses the open source SUMO simulation platform for traffic simulation, and simulates various important factors such as traffic flow and speed based on real-time information. Compared with on-site real-vehicle training, the SUMO-based simulation solution has the advantages of rich scenarios, high safety, low cost, and fast training. In addition, SUMO simulation realizes offline learning, which can meet the requirements of large samples and real-time information interaction required for deep reinforcement learning training, and quickly migrates for different scenarios, using real traffic information to carry out large-scale training, thereby promoting the in-depth development and application of vehicle-road collaborative technology. The visual recognition unit can be a camera, and the traffic light has the function of switching between yellow, red, and green colors.

[0020] The content of the simulation interaction is: take the area between the parking lines at the first end and the second end of the renovation and expansion construction area as the unit perception area, initialize the traffic simulation environment, preheat the environment, make the simulated vehicles fill the road, use the Traci interface to realize the interaction between the intelligent agent and the simulation environment, obtain the interaction data of each simulated vehicle in each action step, and input the vehicle's interaction data into the intelligent agent based on the deep reinforcement learning algorithm D3QN-PER.

[0021] S2: Build an intelligent agent based on the deep reinforcement learning algorithm D3QN-PER; The construction of the intelligent agent in step S2 includes the construction of the state set and the construction of the action set. The state set is constructed based on the improved discrete traffic state encoding method DTSE, and the action set adopts the binary action method to construct the action set of the intelligent agent based on the deep reinforcement learning algorithm. A ; The constructed intelligent agent interacts with the simulated road network.

[0022] The state set is constructed as follows: the entrance road of the expansion and reconstruction construction area is divided into four cells of different lengths according to the distance from the parking line of the single lane entering the construction area and the supervision distance of the visual recognition unit from near to far. s 1 , s 2 , s 3 , s 4 ], the lengths of each cell are 10%, 20%, 30% and 40% of the supervision distance of the visual recognition unit respectively; assuming that the supervision distance of the visual recognition unit is 20 meters in front of the parking line m , then the four cells of different lengths correspond to 0-2 of the regulatory distance respectively. m , 2-6 m , 6-12 m and 13-20 m .

[0023] Based on the vehicle information on each cell, a state set of intelligent agents based on deep reinforcement learning algorithm is constructed, namely the DTSE state set ,in is the first state variable of the DTSE state set, For the i The second state variable of the cell state set, For the i The third state variable of the cell state set, i =1, 2, 3, 4.

[0024] The first state variable of the DTSE state set , is the number of vehicles whose center points are located in cell i at the entrance of the renovation and expansion construction area; it should be noted that the number of vehicles in a cell reflects the traffic density. The greater the density, the greater the traffic efficiency within a certain range. After exceeding a certain range, the traffic efficiency will decrease. By reasonably adjusting the number of vehicles in cells with different lengths from the stop line, the average length of vehicle queues in front of the entrance of the construction area can be effectively reduced to a certain extent, alleviating congestion problems. Figure 3 It is a schematic diagram of the average vehicle queue length curve in the two-way single lane construction area according to this embodiment.

[0025] Second state variable of DTSE state set , is constructed based on the cell where the vehicle is located and the vehicle speed at the entrance of the renovation and expansion construction area. The second state variable The expression is , Indicates that the center point of the vehicle is located in the cell i On kThe absolute value of the vehicle's speed; it should be noted that the vehicle speed directly reflects the efficiency of traffic flow. The faster the speed, the greater the traffic efficiency. In other words, this embodiment considers the average speed of vehicles in a two-way single-lane construction area, and effectively improves the efficiency of urban ramp traffic and alleviates traffic congestion. Figure 4 is a schematic diagram of an average vehicle travel time curve in a two-way single lane construction zone according to this embodiment, Figure 5 4 is a schematic diagram of an average vehicle waiting time curve in a two-way single lane construction zone according to this embodiment.

[0026] The expression of the third state variable of the DTSE state set is , The center point of the vehicle is located in the cell i On k The absolute value of the vehicle's acceleration. It should be noted that frequent acceleration and deceleration of vehicles will affect riding comfort, increase power and fuel consumption, and carbon dioxide emissions. The traffic light control strategy of the intelligent agent in this embodiment can effectively reduce the total acceleration and deceleration of vehicles in the construction area compared to the fixed-time traffic lights. Figure 6 is a schematic diagram of an average vehicle CO2 emission curve in a two-way single lane construction zone according to this embodiment, Figure 7 It is a schematic diagram of the average vehicle fuel consumption curve in a two-way single lane construction area according to this embodiment.

[0027] In general, this solution considers traffic signal control in a two-way single lane reconstruction and expansion area, including reducing environmental impact, improving safety level and traffic efficiency. Therefore, in order to comprehensively evaluate the performance of the D3QN-PER agent in this embodiment, five indicators, including average vehicle CO2 emissions, average fuel consumption, average vehicle queue length, average waiting time, and average travel time, are selected as the basis for evaluation.

[0028] The specific evaluation data is shown in Table 1. Compared with the fixed timing model, the deep reinforcement learning D3QN-PER model used in this embodiment has a better control effect on the two-way single-lane traffic lights in the construction area: the average waiting time for vehicles entering the two-way single-lane reconstruction and expansion area is only 5.010s, and the average queue length is only 0.834 vehicles, which is less than 25% of the fixed signal timing in the baseline method, and the effect is particularly significant. Its average CO2 emissions are 326.749 , the average fuel consumption is 104.222 The average travel time is 86.792s, and the degree of optimization is also the highest. Compared with the fixed timing method, there is a more obvious optimization, reaching about 78% of the fixed signal timing.

[0029] Table 1

[0030] Constructing an action set for an intelligent agent based on a deep reinforcement learning algorithm A , is to generate a binary action set by combining the vehicle positions and motion states in the entrance lanes on both sides of the construction area A ={0,1}, where the entrance road of the reconstruction and expansion construction area has a first end and a second end that are relatively set. When the traffic light state of the first end of the reconstruction and expansion construction entrance is green and the traffic light state of the second end is red, if the action of the intelligent agent at the current moment is 1, then within the action step, the traffic light at the first end is switched to yellow and lasts for a first time length, and then switched to red. When the traffic lights at the first end and the traffic lights at the second end both remain red for a second time length, the traffic light at the second end is switched to green, and the traffic light at the first end remains unchanged; otherwise, the traffic light at the reconstruction and expansion construction entrance is switched to green, and the traffic light at the first end remains unchanged. When the traffic light at the second end of the extension construction entrance is green and the traffic light at the first end is red, if the action of the agent at the current moment is 1, then within the action step, the traffic light at the second end switches to yellow and lasts for the first time, and then switches to red. When the traffic lights at the first end and the second end both remain red for the second time, the traffic light at the first end is switched to green, and the traffic light at the second end remains unchanged; the sum of the first time duration T1 of the yellow light and the second time duration T2 of the red light satisfies: T1+T2=( L / v 15% )×1.5, where L Indicates the length of the renovation and expansion construction area. v 15% Indicates the 15% speed, that is, only 15% of the vehicles have a speed lower than this speed. T1+T2 indicates the time interval for the alternating switching of the green lights at both ends of the construction entrance when the action of the intelligent agent is 1 at the current moment.

[0031] If the action of the agent at the current moment is 0, the current states of the traffic lights at the first end and the second end are maintained unchanged within the action step.

[0032] S3: In the simulated road network environment, the agent is trained based on a reward function, wherein the reward function is based on the number of vehicles traveling at a speed lower than a set driving speed in the reconstruction and expansion construction area in the previous time step.

[0033] The reward function described in step S3 is to make the current action step length t The reward value is R t , the expression of the reward function is R t = -num ( v i < v th ), num( v i < v th ) indicates that within the previous action step, the vehicle speed on the lane is less than the set driving speed v th In this embodiment, the vehicle speed is set to v th The given value of is 1m / s. Setting the reward function can guide the agent to reduce the number of low-speed vehicles and use its negative value as the reward value. Figure 8 It is a schematic diagram of the deep reinforcement learning D3QN algorithm framework in the two-way single lane construction zone signal control model of this embodiment.

[0034] like Fig. 9 As shown, in this embodiment, the environment is initialized and the training of the intelligent agent is realized through the Traci interface. First, the camera sends the image to the visual processing chip; the visual processing chip recognizes the target in the image and obtains the coordinates of all recognized vehicles in the Cartesian coordinate system of the real world through the BEV perspective conversion technology. At the same time, the vehicle's forward direction is determined by combining the sequential images taken by the infrared night vision camera, and the vehicle's speed and acceleration are calculated. The agent generates a DTSE state matrix based on the vehicle information on the entrance lane input by the camera. Then, the DTSE matrix is ​​passed to the D3QN algorithm framework. Q network and Target network, Q Whether the network output switches the phase action corresponding to Q Value. The agent Q Value The strategy selects actions and executes actions to interact with the simulation environment to obtain a four-tuple , and the four-tuple Stored in the experience replay pool.

[0035] ; When there are enough experience trajectories in the experience replay pool, batch extraction is performed, the Target network is used to calculate the TD target, the decision made by the combined Q network is updated to the Target network, and then the Target network is updated by multi-step weighted averaging, which reduces the degree of bootstrapping to a certain extent. After the agent has trained for a certain number of rounds, the model reaches convergence and the simulation training model is saved.

[0036] It should be noted that different road construction areas will lead to different degrees of road closures. Therefore, the service distances of two-way single lanes are not the same. In order to maximize efficiency, it is necessary to determine the yellow light time and full red time of the phase combination before training the D3QN-PER model, and improve the green signal ratio while ensuring that there will be no difficulty in meeting when two vehicles in opposite directions enter the construction area at the same time.

[0037] S4: Using the trained intelligent agent, the traffic control lights in the reconstruction and expansion construction area are regulated and the optimal state of the traffic lights is selected for output.

[0038] The content of step S4 is to obtain real-time road traffic information of the two-way single lane reconstruction and expansion construction area, and determine the optimal action through the trained intelligent agent using the D3QN-PER model, so that the traffic lights at the first and second ends of the reconstruction and expansion construction entrance output the output state corresponding to the optimal action of the intelligent agent.

[0039] On the other hand, the present invention provides an intelligent two-way single lane reconstruction and expansion signal control system, which is used to execute the above-mentioned intelligent two-way single lane reconstruction and expansion signal control method, including: a signal light, a visual recognition unit, a main control unit and a power supply unit. Fig.10 As shown, the main control unit is set in the chassis, and the power supply unit can be set in the chassis or on the top of the signal light. The chassis can be set on the signal light, such as before the approach road of the construction section, and the main body of the chassis on the right side of the motor vehicle driver can be made by sheet metal welding, and the body is mainly sprayed with yellow and black stripes.

[0040] The signal lights are respectively arranged at the first end and the second end of the entrance road of the reconstruction and expansion construction area, facing the driving direction of the vehicle, and are used to output the indication signal of road passing or stopping; the signal lights have three light phases, namely yellow, red and green LED lights, and the signal lights light up or extinguish the corresponding LED lights according to the output of the active unit.

[0041] The visual recognition unit is installed on the top of the traffic light, facing the direction of the entrance road of the renovation and expansion construction area, and is used to obtain images of the entrance to the construction area and the single lane in the construction area; the visual recognition unit has communication and interaction capabilities, and sends the obtained vehicle images to the outside; the visual recognition unit usually includes an infrared night vision camera and a visual processing chip. The visual recognition unit includes an infrared night vision camera, which obtains the image in front of the expansion area and transmits it to the main control unit. The infrared night vision camera obtains daytime images in the 940nm infrared band. At night, the infrared night vision camera cooperates with the 940nm infrared fill light to collect nighttime images. The infrared night vision camera can be in the style of a ball camera or a gun camera. The infrared night vision camera is connected to the visual processing chip in communication; the infrared night vision camera obtains images of the roads around the renovation and expansion construction area, and sends the images to the visual processing chip; the visual processing chip identifies the target in the image, and obtains the coordinates of all identified vehicles in the real-world Cartesian coordinate system through the BEV perspective conversion technology. At the same time, combined with the sequential images taken by the infrared night vision camera, the vehicle's forward direction is determined, and the vehicle's speed and acceleration are calculated. The coordinate conversion process is common knowledge in this field and will not be repeated here. This step can be divided into two steps: target recognition and target processing acquisition. The target here refers to all vehicles on the lanes around the expansion area.

[0042] The main control unit is connected to the visual recognition units at the first and second ends of the entrance road of the reconstruction and expansion construction area, receives images transmitted by each visual recognition unit, recognizes vehicle information, analyzes the speed and number of target vehicles, uses the trained intelligent agent to judge the target vehicle entering the reconstruction and expansion construction area through the D3QN-PER algorithm, selects the optimal action, and outputs the optimal action to the signal light; at the same time, when the active unit finds that there is a traffic jam in the single lane of the construction area and it cannot be solved by passing vehicles, it uses the trained intelligent agent to adjust in time; The power supply unit is electrically connected to the signal light, the visual recognition unit, the main control unit and the information interaction unit respectively, and is used to provide working power.

[0043] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An intelligent two-way single lane reconstruction and expansion signal control method, characterized in that: The steps include: S1: Establish a simulated road network with two-way single lanes, and set up signal lights and visual recognition units at the entrance of the reconstruction and expansion construction area of ​​the simulated road network; S2: Build an intelligent agent based on the deep reinforcement learning algorithm D3QN-PER; S3: In the simulated road network environment, the agent is trained based on a reward function, wherein the reward function is based on the number of vehicles traveling at a speed lower than a set driving speed in the reconstruction and expansion construction area in the previous time step; S4: Using the trained intelligent agent, the traffic control lights in the reconstruction and expansion construction area are regulated and the optimal state of the traffic lights is selected for output.

2. According to claim 1, a smart two-way single lane reconstruction and expansion signal control method is characterized in that: The construction of the intelligent agent in step S2 includes the construction of a state set and an action set. The state set is constructed based on the improved discrete traffic state encoding method DTSE, and the action set adopts a binary action method to construct the action set of the intelligent agent based on the deep reinforcement learning algorithm. A ; The constructed intelligent agent interacts with the simulated road network.

3. According to claim 2, a smart two-way single lane reconstruction and expansion signal control method is characterized in that: The state set is constructed as follows: the entrance road of the expansion and reconstruction construction area is divided into four cells of different lengths according to the distance from the parking line of the single lane entering the construction area and the supervision distance of the visual recognition unit from near to far. s 1 , s 2 , s 3 , s 4 ], the lengths of each cell are 10%, 20%, 30% and 40% of the supervision distance of the visual recognition unit respectively; based on the vehicle information on each cell, an agent state set based on the deep reinforcement learning algorithm is constructed ,in For the i The first state variable of the cell state set, For the i The second state variable of the cell state set, For the i The third state variable of the cell state set, i =1, 2, 3, 4.

4. The intelligent two-way single lane reconstruction and expansion signal control method according to claim 3 is characterized in that: No. i The first state variable of the cell state set , is at the entrance of the renovation and expansion construction area, and the center point of the vehicle is located in the cell i The number of vehicles on i The second state variable of the cell state set , is constructed based on the cell where the vehicle is located and the vehicle speed at the entrance of the renovation and expansion construction area. The second state variable The expression is , Indicates that the center point of the vehicle is located in the cell i On k The absolute value of the vehicle's speed; i The expression of the third state variable of the cell state set is , The center point of the vehicle is located in the cell i On k The absolute value of the vehicle's acceleration.

5. The intelligent two-way single lane reconstruction and expansion signal control method according to claim 2 is characterized in that: Constructing an action set for an intelligent agent based on a deep reinforcement learning algorithm A , is to generate a binary action set by combining the vehicle positions and motion states in the entrance lanes on both sides of the construction area A ={0,1}, wherein the entrance road of the reconstruction and expansion construction area has a first end and a second end that are relatively set, and the state of the signal light at the first end of the reconstruction and expansion construction entrance is green, and the state of the signal light at the second end is red, if the action of the intelligent agent at the current moment is 1, then within the action step, the signal light at the first end is switched to yellow and lasts for a first time, and then switched to a red light state, and when the signal lights at the first end and the signal lights at the second end both maintain red for a second time, the traffic light at the second end is switched to green, and the signal light at the first end remains unchanged; conversely, when the state of the signal light at the second end of the reconstruction and expansion construction entrance is green, and the state of the signal light at the first end is red, if the action of the intelligent agent at the current moment is 1, then within the action step, the signal light at the second end is switched to yellow and lasts for a first time, and then switched to a red light state, and when the signal lights at the first end and the signal lights at the second end both maintain red for a second time, the traffic light at the first end is switched to green, and the signal light at the second end remains unchanged; If the action of the agent at the current moment is 0, the current states of the traffic lights at the first end and the second end are maintained unchanged within the action step.

6. The intelligent two-way single lane reconstruction and expansion signal control method according to claim 5 is characterized in that: The reward function described in step S3 is to make the current action step length t The reward value is R t , the expression of the reward function is R t = -num ( v i < v th ), num ( v i < v th ) indicates that within the previous action step, the vehicle speed on the lane is less than the set driving speed v th The average number of vehicles.

7. The intelligent two-way single lane reconstruction and expansion signal control method according to claim 5 is characterized in that: The content of step S4 is to obtain real-time road traffic information of the two-way single lane reconstruction and expansion construction area, and determine the optimal action through the trained intelligent agent using the D3QN-PER model, so that the traffic lights at the first and second ends of the reconstruction and expansion construction entrance output the output state corresponding to the optimal action of the intelligent agent.

8. The intelligent two-way single lane reconstruction and expansion signal control method according to claim 2 is characterized in that: The establishment of the simulated road network of two-way single lane described in step S1 is to establish the simulated road network through the SUMO simulation platform and perform simulation interaction.

9. The intelligent two-way single lane reconstruction and expansion signal control method according to claim 8 is characterized in that: The content of the simulation interaction is: take the area between the parking lines at the first end and the second end of the renovation and expansion construction area as the unit perception area, initialize the traffic simulation environment, preheat the environment, make the simulated vehicles fill the road, use the Traci interface to realize the interaction between the intelligent agent and the simulation environment, obtain the interaction data of each simulated vehicle in each action step, and input the vehicle's interaction data into the intelligent agent based on the deep reinforcement learning algorithm D3QN-PER.

10. An intelligent two-way single lane reconstruction and expansion signal control system, used to execute the intelligent two-way single lane reconstruction and expansion signal control method according to any one of claims 1 to 9, characterized in that: include: Signal lights are respectively arranged at the first end and the second end of the entrance road of the reconstruction and expansion construction area, and are used to output the indication signal of road passing or stopping; The visual recognition unit is installed on the top of the signal light, facing the entrance of the reconstruction and expansion construction area, and is used to obtain images of the entrance of the construction area and the single lane in the construction area; The main control unit is connected to the visual recognition units at the first and second ends of the entrance road of the reconstruction and expansion construction area, receives images transmitted by each visual recognition unit, recognizes vehicle information, analyzes the speed and number of target vehicles, uses the trained intelligent agent to judge the target vehicle entering the reconstruction and expansion construction area through the D3QN-PER algorithm, selects the optimal action, and outputs the optimal action to the signal light; at the same time, when the active unit finds that there is a traffic jam in the single lane of the construction area and it cannot be solved by passing vehicles, it uses the trained intelligent agent to adjust in time; The power supply unit is electrically connected to the signal light, the visual recognition unit and the main control unit respectively, and is used to provide working power.

Images