Expressway inside and outside width regulation and control induction method and device

Abstract

The invention belongs to the technical field of intelligent traffic management and control, and particularly relates to an expressway inside and outside width regulation and control induction method and equipment. The method comprises the following steps: S1, acquiring traffic state parameters; s2, the traffic state parameters are input into a trained intelligent agent with an Actor network, the intelligent agent outputs induction action information, the induction action information comprises an induction direction and an induction destination, and real-time inter-amplitude induction regulation and control are achieved; and training the intelligent agent according to the traffic state parameter, the induction action information, the reward value of the intelligent agent and the new traffic state parameter. According to the invention, the flexibility of the internal and external amplitude division management and control road section is increased, the emergency response capability is improved, the road resource waste is reduced, the travel efficiency of travelers is improved, the management and control scheme is free from dependence on artificial experience, and a full-parameterized inter-amplitude regulation and control scheme is formed.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent traffic control, and in particular relates to a method and device for regulating and inducing inner and outer widths of a highway. Background Art

[0002] The purpose of the present invention is to solve the problem that the use of static signs and markings to guide vehicles will lead to unbalanced traffic flow between the inner and outer lanes and insufficient utilization of road resources, and propose a method and device for regulating and inducing the inner and outer lanes of expressways. Through calculation and analysis, dynamic solid and dotted line state information is output to induce vehicles to change lanes, so as to balance the reasonable utilization of road resources.

[0003] In order to achieve the above-mentioned object of the invention, the present invention provides the following technical solutions:

[0004] A method for regulating and inducing the inner and outer widths of a highway comprises the following steps:

[0005] S1, obtaining traffic status parameters, wherein the traffic status parameters include flow rate, vehicle density and average vehicle speed of road sections;

[0006] S2, inputting the traffic state parameters into a trained intelligent agent with an Actor network, and the intelligent agent outputs induced action information, wherein the induced action information includes an induced direction and an induced destination, so as to realize real-time range-to-range induced control;

[0007] The method also includes the steps of training the agent having the Actor network:

[0008] S21, the output of the induced action information causes the traffic state parameter to change, thereby obtaining a new traffic state parameter;

[0009] S21, calculating the reward value of the agent according to the new traffic state parameter;

[0010] S23, training the intelligent agent according to the traffic state parameters, the induced action information, the reward value of the intelligent agent and the new traffic state parameters.

[0011] As a preferred solution, during the training of the Actor network, a Critic network is used as an evaluation network, and the induced action information evaluation value of the intelligent agent is calculated through traffic state parameters and induced action information; the parameters of the Actor network are updated according to the induced action information evaluation value.

[0012] As a preferred solution, step S22 specifically includes the following steps:

[0013] S220, calculating an inner and outer range balance index according to the new traffic state parameter;

[0014] S221, calculating the reward value of the internal and external width balance index according to the internal and external width balance index, and taking the inverse of the sum of the internal and external width balance indexes of each road section as the reward value of the intelligent agent.

[0015] As a preferred solution, the reward value calculation formula is: Among them, r t is the reward value, m is the road segment number, M is the maximum value of the road segment number, γ m It is an indicator of internal and external balance.

[0016] As a preferred solution, the calculation formula of the internal and external amplitude balance index is:

[0017]

[0018] Among them, m is the section number of the road, γ m is the ratio of the inner and outer width saturation of the mth section of road, Q 2m-1 , Q 2m are the flow rates of the inner and outer sections of the mth section of road, respectively, and C 2m-1 , C 2m They are the inner and outer capacities of the mth road section respectively.

[0019] As a preferred solution, the agent includes a solid-dashed line agent and a dotted-solid line agent;

[0020] The solid and dotted line agent is used to determine whether the outer-travel vehicle should be guided to drive into the inner-travel area at the solid and dotted line. The action variables output by the solid and dotted line agent include the medium-distance destination and the medium- and long-distance guidance directions.

[0021] Virtual and real line agent: used to determine whether the vehicle driving in the inner lane of the virtual and real line guidance should drive out of the outer lane; the action variables output by the virtual and real line agent include the destination and the guidance direction.

[0022] As a preferred solution, the induced action information includes multiple groups of information, each group of information corresponds to a section number of the road, each group of information functions independently, and the output induced action information realizes distributed real-time inter-section induced control.

[0023] As a preferred solution, the location where the induced action information is released includes the starting point and midpoint of the real and virtual line and the real and virtual line segments where the vehicle performs the lane transition.

[0024] Based on the same concept, a dynamic and static fusion variable information release device is also proposed, including a processor and a splicing screen;

[0025] The processor uses any of the above-mentioned methods for inducing the inner and outer width control of expressways to output the inducing action information;

[0026] The induced action information is displayed through the spliced ​​screen.

[0027] As a preferred solution, the direction and destination displayed on the spliced ​​screen are variable, including variable information of dynamic and static fusion at the solid and dotted lines and dynamic and static fusion width control marks at the solid and dotted lines:

[0028] The objects guided by the variable information signs of dynamic and static fusion at the solid and dotted lines include medium and long-distance destinations. By changing the medium-distance destination, the objects entering the inner width are adjusted, and the arrow is changed to a variable guide arrow with two forms, front and left front, to control whether the medium and long-distance at the variable information signs of dynamic and static fusion at the solid and dotted lines enter the inner width. The variable information information board is used below the sign to explain the road conditions, so as to assist the driver to understand the variable information signs of dynamic and static fusion at the solid and dotted lines.

[0029] The dynamic and static fusion inter-lane control signs at the dotted and solid lines guide vehicles traveling in the inner lane to enter the outer lane through the variable information board hanging below.

[0030] Compared with the prior art, the present invention has the following beneficial effects: the use of the method and device of the present invention increases the flexibility of the inner and outer section control sections, improves the ability to respond to emergencies, reduces the waste of road resources, improves the travel efficiency of travelers, gets rid of the control plan's reliance on manual experience, and forms a fully parameterized inter-section control plan. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 The dashed line and the dashed and solid lines in the background technology are schematic diagrams for the vehicle to perform outside-to-inside and inside-to-outside changes respectively;

[0032] Figure 2 This is a schematic diagram of the dynamic and static fusion mark at the solid and dotted line in Example 1;

[0033] Figure 3 This is a flow chart of a method for regulating and inducing the inner and outer sections of a highway in Example 1;

[0034] Figure 4 This is a schematic diagram of model updating and correction in Example 1;

[0035] Figure 5 This is a schematic diagram of parameter quantization in the simulation environment in Example 1. DETAILED DESCRIPTION

[0036] The present invention is further described in detail below in conjunction with test examples and specific implementation methods. However, this should not be understood as the scope of the above subject matter of the present invention being limited to the following embodiments, and all technologies realized based on the content of the present invention belong to the scope of the present invention.

[0037] The main idea of ​​the present invention is to obtain the inducing action information including the inducing direction and the inducing destination according to the traffic status parameters, set the dynamic and static fusion signs in the transition area between the inner and outer widths (solid and dotted lines, solid and dotted lines), present the inducing action information on the fusion signs, conduct inner and outer width traffic induction, and achieve inner and outer width traffic balance.

[0038] Example 1

[0039] A flow chart of a method for inducing regulation between inner and outer widths of a highway Figure 3 As shown, the following steps are included: S1, obtaining traffic state parameters, wherein the traffic state parameters include the flow rate, vehicle density and average speed of the road section; S2, inputting the traffic state parameters into a trained intelligent agent with an Actor network, and the intelligent agent outputs induced action information, wherein the induced action information includes an induced direction and an induced destination, so as to realize real-time inter-section induced control.

[0040] Step S2 is the core step of the present invention. It is the key to output accurate induced action information for the intelligent agent with the Actor network. Therefore, it is necessary to train the intelligent agent with the Actor network to improve the accuracy of the intelligent agent's output information.

[0041] The agent with the Actor network is trained and updated, and the model update and correction diagram is shown in Figure 4 As shown, the specific steps include:

[0042] S21, the output of the induced action information causes the traffic state parameter to change, thereby obtaining a new traffic state parameter;

[0043] S22, calculating the reward value of the agent according to the new traffic state parameter;

[0044] S23, training the intelligent agent according to the traffic state parameters, the induced action information, the reward value of the intelligent agent and the new traffic state parameters.

[0045] Furthermore, in the process of training the Actor network, a Critic network is used as an evaluation network, and an induced action information evaluation value of the intelligent agent is calculated through traffic state parameters and induced action information; and the parameters of the Actor network are updated according to the induced action information evaluation value.

[0046] Wherein, step S22 specifically includes the following steps:

[0047] S220, calculating an inner and outer range balance index according to the new traffic state parameter;

[0048] S221, calculating the reward value of the internal and external width balance index according to the internal and external width balance index, and taking the inverse of the sum of the internal and external width balance indexes of each road section as the reward value of the intelligent agent.

[0049] In the process of the agent outputting the induction information in real time, the reinforcement learning framework is used to learn to observe the current state, choose to output the appropriate action, and adjust its own strategy according to the reward value obtained after executing the action. This process is carried out continuously, and the agent learns an optimal strategy through interaction with the environment to obtain the maximum cumulative reward value. Specifically, the reinforcement learning framework is used to construct the induction environment between the inner and outer widths, with the road traffic flow operation status in each period as the state variable, and the information released by each dynamic and static fusion sign as the action. In a certain state, the agent needs to make a decision and take a certain action. The traveler chooses to drive in the inner or outer width according to the induction information, and then enters the next state, and so on. The reciprocation of state and action constitutes the main part of reinforcement learning. The opposite number of the deviation degree of the balance index is used as the reward. The purpose of reinforcement learning is to maximize the total reward, that is, the balance of the inner and outer widths.

[0050] Taking the balance of inner and outer traffic flow as the optimization goal, the inner and outer balance index is constructed. The inner and outer balance index is γ m is the ratio of the inner and outer width saturation of the mth section of road, Q 2m-1 , Q 2m are the flow rates of the inner and outer sections of the mth section of road, respectively, and C 2m-1 , C 2m are the inner and outer capacities of the mth section of road respectively. The objective function is to minimize the deviation of the overall balance index of the road: Traffic capacity refers to the maximum hourly flow rate that highway facilities are expected to pass under the design service level. The maximum hourly flow rate is calculated by looking up the table based on design indicators such as design speed and road width.

[0051] The input to the agent is the traffic state parameter, which is represented by s t =[s t，1 ,s t，2 , ..., s t，N ], where N is the number of road segments, s t，i Represents the traffic flow state of the i-th road section, including flow, speed, and density. Flow, speed, and density are state variables throughout the entire method steps. The state variables flow, average speed, and density affect each other. The internal and external balance index is the reward function of the intelligent agent, and the state variables affect the calculation of the reward value.

[0052] As a preferred solution of the present invention, two types of agents are defined, including solid-dashed line agents and dashed-solid line agents;

[0053] The real and dotted line agent is used to determine whether the outer-width vehicle is induced to drive into the inner width at the real and dotted line; the action variables output by the real and dotted line agent include the medium-distance destination and the medium- and long-distance induced directions. The real and dotted line agent is responsible for inducing the outer-width vehicle to drive into the inner width at the real and dotted line. The action variables of the real and dotted line agent include the medium-distance destination and the medium- and long-distance induced directions, that is, solving [π, ζ, η], where π represents the corresponding number of the medium-distance destination, ζ, η represent the medium-distance destination and the medium- and long-distance induced directions, respectively, and the value range is [1,0], representing entering the inner width and keeping driving on the outer width, respectively.

[0054] Virtual and real line agent: used to determine whether the vehicle driving on the inner side of the virtual and real line drives out of the outer side; the action variables output by the virtual and real line agent include the destination and the induced direction. The virtual and real line agent is responsible for determining whether the vehicle driving on the inner side of the virtual and real line drives into the outer side. The action variables of the virtual and real line agent include the destination and the induced direction, that is, solving [ρ, σ], ρ represents the corresponding number of the destination, σ represents the induced direction, and the value range is [1,0], which respectively represents entering the outer side and keeping driving on the inner side.

[0055] Each agent uses an Actor-Critic architecture, where:

[0056] Actor network: It is the strategy generation network for each agent. The input of each agent’s Actor network is the global traffic state s t =[s t，1 ,s t，2 , ..., s t，N ], which includes the traffic status of all road sections. This allows each agent to take into account the global traffic flow when making decisions. Based on the global state, the Actor network outputs the speed limit action a of the agent t，i , and applied to the environment.

[0057] Critic network: As a policy evaluation network, the Critic network is also based on the global state and the actions of all agents. t =[a t，1 , a t，2 , ..., a t，K ], calculate the Q value of each agent to evaluate the value of the current action.

[0058] At each time step t, each agent’s Actor network observes the global traffic state parameter s t Select the induced action a t，i , and applies the action to the traffic environment. The traffic environment changes to a new state s under the intervention of the induced action t+1 , and calculate the corresponding reward value r t, according to the traffic state parameter s t , induced action information a t，i , the agent's reward value r t and the new traffic state parameter s t+1 , the Critic network is used to update the parameters of the agent's Actor network, so that the agent with the Actor network can be updated in real time and accurately obtain the induced action information.

[0059] The execution phase includes the following steps:

[0060] Step 1: Observe the global state

[0061] At each time t, the agent system observes the state s of the entire traffic network t =[s t，1 ,s t，2 , ..., s t，N ];

[0062] Step 2: Action selection for each agent

[0063] Actor network for each agent i Based on the global traffic status t Select the corresponding induction action

[0064] The solid-dashed line agent and the dotted-solid line agent generate different speed limit control actions according to their respective strategies, which depends on the policy network of each agent.

[0065] Step 3: Apply actions and environment updates

[0066] Each agent i imposes a speed-limiting action a t，i To the corresponding road environment. The traffic environment is updated to the new state s according to the actions of all agents t+1 .

[0067] Step 4: Calculate Rewards

[0068] For each agent i, according to the action applied and the new state s t+1 , calculate the reward value r of all agents t .

[0069] Step 5: Storing Experience

[0070] The current interaction data (s t , a t , r t ,s t+1 ) is stored in the global experience pool for subsequent training.

[0071] The training phase includes the following steps:

[0072] Step 1: Sampling empirical data

[0073] Randomly sample a batch of experience (s) from the global experience pool t , a t , r t ,s t+1 ) is used to update the agent's Actor and Critic networks.

[0074] Step 2: Critic Network Update

[0075] Critic network for each agent i Based on the global state s t and all agents’ actions a t , calculate the value estimate of the strategy

[0076] Update the Critic network parameters φ through TD error i , to minimize the loss function, the loss function is calculated as follows:

[0077]

[0078] Where γ is the discount factor, s t is the global traffic state, s t+1 is the new state after the action is applied, a t is the action of the agent, a t+1 is the action of the next new agent, is the estimated value of the agent’s strategy, r t is the agent’s reward value, It's expectation.

[0079] As a preferred solution, the Actor network of each agent i It can also be updated using the policy gradient method. The main steps include: calculating the policy gradient of each agent based on the Q value of the Critic network:

[0080]

[0081] in, is the policy gradient, θ i It is a parameter used to update the Actor network, so that it can generate a better rate limit strategy. It's expectation.

[0082] Example 2

[0083] Based on the same concept, a dynamic and static fusion variable information release device is also proposed, including a processor and a splicing screen;

[0084] The processor outputs the induced action information using any one of the highway inner and outer width control induction methods described in Example 1; the induced action information is displayed through a spliced ​​screen. The direction and destination displayed on the spliced ​​screen are variable, including variable dynamic and static fusion information at the solid and dotted lines and dynamic and static fusion width control signs at the solid and dotted lines.

[0085] The dynamic and static fusion variable information sign at the solid dotted line guides the objects including the medium and long distance destinations. By changing the medium distance destination, the object entering the inner width is adjusted, and the arrow is changed to a variable guide arrow with two forms, front and left front, to control whether the medium and long distance at the dynamic and static fusion variable information sign at the solid dotted line enters the inner width. The variable information information board below the sign explains the road conditions to assist the driver in understanding the dynamic and static fusion variable information sign at the solid dotted line. Figure 2 shown.

[0086] The dynamic and static fusion variable information release equipment is set at the starting point and midpoint of the real and virtual line and the virtual and real line segments where the vehicle switches between widths; the dynamic and static fusion variable information release equipment splices the LED screen to make the direction and destination variable.

[0087] The dynamic and static fusion inter-lane control signs at the dotted and solid lines guide the inner-lane vehicles to enter the outer lane through the variable information board hanging below. Figure 2 As shown, on the splicing screen, the bottom line uses text prompts: the outer radiation traffic volume is heavy, and then arrows are used above to indicate that because the outer radiation traffic volume is heavy, the vehicle needs to change lanes in the direction indicated by arrows. The top is the destination corresponding to the lane. For example, if "Chang'an" is displayed, it means that the lane is heading towards Chang'an, and if "Shenzhen" is displayed, it means that the lane is heading towards Shenzhen. The owner can change lanes in time according to the prompt information on the splicing screen to balance the traffic flow in each lane, which is conducive to improving traffic efficiency.

[0088] Example 3

[0089] Furthermore, in order to obtain parameters and verify the feasibility of the method, the present embodiment also provides the construction of a simulation environment, and designs an intelligent agent in the simulation environment. The specific steps of the construction of the simulation environment and the design of the intelligent agent include:

[0090] Step 1: Calibration of the inter-amplitude conversion probability model

[0091] Using historical vehicle trajectory data, we select features related to the transition between frames, including: vehicle position l x , l y , speed v x 、v y , acceleration a, current position (inner or outer) B, distance to destination D, distance to the end point of the dashed line or dashed line Dtrans , the consistency between the current lane and the information board guidance scheme C, and the distance between the front and rear vehicles in the target width d front , d rear , and the speed difference between the front and rear vehicles of the target range Δv front , Δv rear , the traffic density in the area is K.

[0092] Set the target variable y for the inter-amplitude conversion, whose value is 1 (internal amplitude to external amplitude), -1 (external amplitude to internal amplitude), 0 (no conversion). Perform model training based on the loss function. After the inter-amplitude conversion probability model is calibrated, the inter-amplitude conversion probability model is used for the simulation of step 2.

[0093] Step 2: Vehicle Agent Design and Road Agent Design

[0094] Create multiple vehicle agents in the simulation environment, each representing a vehicle. Initialize the state of each vehicle agent, including position, speed, acceleration, current section, lane, destination, etc. Use the section transition probability model as the behavior rule of the vehicle agent.

[0095] The road is divided into M sections according to the marking type, and the road is abstracted into N road agents based on the inner and outer widths, where N = 2 × M. Each road agent has a number N and a road length L. n , where P n (Inner width P n =0 outer width P n =1), marking type M n (Double solid line M n =0, solid dashed line M n =1, dashed and solid lines M n =-1) and other attributes. Each road agent is responsible for monitoring the traffic flow and density within its jurisdiction. The schematic diagram of parameter quantization in the simulation environment is as follows Figure 5 shown.

[0096] In each simulation step, the state of each vehicle agent is updated, and its next action is calculated based on its current state and surrounding environment information (occupancy of the front and rear road agents, speed difference, etc.); the traffic density and flow of each road agent are updated according to the vehicle agent action.

[0097] Model application: The trained intelligent agent is applied to the real traffic environment, with the traffic status (flow, density, speed) of each road section collected by radar as the state variable, and the display content of the dynamic and static signs fused as the action variable.

[0098] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for regulating and inducing the inner and outer widths of a highway, characterized in that: The following steps are involved: S1, obtaining traffic status parameters, wherein the traffic status parameters include flow rate, vehicle density and average vehicle speed of road sections; S2, inputting the traffic state parameters into a trained intelligent agent with an Actor network, and the intelligent agent outputs induced action information, wherein the induced action information includes an induced direction and an induced destination, so as to realize real-time range-to-range induced control; The method also includes the steps of training the agent having the Actor network: S21, the output of the induced action information causes the traffic state parameter to change, thereby obtaining a new traffic state parameter; S21, calculating the reward value of the agent according to the new traffic state parameter; S23, training the intelligent agent according to the traffic state parameters, the induced action information, the reward value of the intelligent agent and the new traffic state parameters.

2. A method for inducing control of inner and outer widths of a highway as claimed in claim 1, characterized in that: In the process of training the Actor network, a Critic network is used as an evaluation network, and an induced action information evaluation value of the intelligent agent is calculated through traffic state parameters and induced action information; and the parameters of the Actor network are updated according to the induced action information evaluation value.

3. A method for inducing control of inner and outer widths of a highway as claimed in claim 1, characterized in that: Step S22 specifically includes the following steps: S220, calculating an inner and outer range balance index according to the new traffic state parameter; S221, calculating the reward value of the internal and external width balance index according to the internal and external width balance index, and taking the inverse of the sum of the internal and external width balance indexes of each road section as the reward value of the intelligent agent.

4. A method for regulating and inducing the inner and outer widths of a highway as claimed in claim 3, characterized in that: The reward value calculation formula is: ; Among them, r t is the reward value, m is the road segment number, M is the maximum value of the road segment number, It is an indicator of internal and external balance.

5. A method for inducing control of inner and outer widths of a highway as claimed in claim 4, characterized in that: The calculation formula of the internal and external range balance index is: Where m is the section number of the road, is the ratio of the inner and outer width saturation of the mth section of road, Q 2m-1 , Q 2m are the flow rates of the inner and outer sections of the mth section of road, respectively, and C 2m-1 , C 2m They are the inner and outer capacities of the mth road section respectively.

6. A method for inducing control of inner and outer widths of a highway as claimed in claim 1, characterized in that: The intelligent agent includes a solid-dashed line intelligent agent and a dotted-solid line intelligent agent; The solid and dotted line agent is used to determine whether the outer-travel vehicle should be guided to drive into the inner-travel area at the solid and dotted line. The action variables output by the solid and dotted line agent include the medium-distance destination and the medium- and long-distance guidance directions. Virtual and real line agent: used to determine whether the vehicle driving in the inner lane of the virtual and real line guidance should drive out of the outer lane; the action variables output by the virtual and real line agent include the destination and the guidance direction.

7. A method for inducing control of inner and outer widths of a highway as claimed in claim 6, characterized in that: The induced action information includes multiple groups of information, each group of information corresponds to a section number of the road, and each group of information plays a role independently. The output induced action information realizes distributed real-time inter-section induced control.

8. The method for inducing control of inner and outer widths of a highway according to claim 1, characterized in that: The location where the induced action information is released includes the starting point and midpoint of the real and virtual line and the real and virtual line segments where the vehicle performs the inter-width conversion.

9. Dynamic and static fusion variable information publishing equipment, characterized in that: Including processor and splicing screen; The processor uses a highway inner and outer width control induction method according to any one of claims 1 to 8 to output the induction action information; The induced action information is displayed through the spliced ​​screen.

10. The dynamic and static fusion variable information publishing device according to claim 9, characterized in that: The direction and destination displayed on the spliced ​​screen are variable, including variable information of dynamic and static fusion at the real and dotted lines and the dynamic and static fusion width control mark at the real and dotted lines: The objects guided by the variable information signs of dynamic and static fusion at the solid and dotted lines include medium and long-distance destinations. By changing the medium-distance destination, the objects entering the inner width are adjusted, and the arrow is changed to a variable guide arrow with two forms, front and left front, to control whether the medium and long-distance at the variable information signs of dynamic and static fusion at the solid and dotted lines enter the inner width. The road conditions are explained in conjunction with the variable information board below the sign to assist the driver in understanding the variable information signs of dynamic and static fusion at the solid and dotted lines. The dynamic and static fusion inter-lane control signs at the dotted and solid lines guide vehicles traveling in the inner lane to enter the outer lane through the variable information board hanging below.

Images