Highway variable speed limit control method of multi-agent cooperation under complex environment

Abstract

The application discloses a highway variable speed limit control method for multi-agent cooperation in a complex environment and belongs to the technical field of highway traffic speed limit control. The method comprises the following steps: dividing a highway into multiple control sections, acquiring traffic state data of the multiple control sections in each control period and weather information and event information corresponding to the control sections; calculating a scenario perception feature vector for each control section; constructing a multi-agent cooperative variable speed limit control model; training an Actor network and a Critic network based on a cooperative reward to obtain a trained Actor network; in online control, calling the trained Actor network in each control period, outputting a speed limit action according to the traffic state data and the scenario perception feature vector of each control section, converting the speed limit action into a speed limit control instruction and issuing the speed limit control instruction to a variable speed limit sign, a roadside unit or a traffic control platform for execution.

Description

Technical Field

[0001] This invention belongs to the field of highway traffic speed limit control technology, specifically involving a variable speed limit control method for highways with multi-agent collaboration in complex environments. Background Technology

[0002] Variable speed limit control on highways affects vehicle speed and traffic flow by adjusting the speed limit values ​​of different road sections, and is an important means of alleviating congestion and reducing the risk of accidents.

[0003] In multi-segment corridor scenarios, multi-agent reinforcement learning has been used to treat each controlled segment as an agent, achieving corridor-level coordinated control through centralized training / distributed execution. However, existing multi-segment variable speed limit coordination methods generally suffer from two problems: First, the physical coupling mechanism between environmental features and traffic state features is ignored. Existing technologies typically treat weather (e.g., rainfall) or events (e.g., accidents) as independent feature dimensions, simply concatenating them with traffic state (speed, flow) before inputting them into the neural network. However, from the perspective of traffic flow dynamics, severe weather or accidents are not simply additional information, but rather alter the basic graph properties of the road (e.g., reduced maximum capacity, critical density drift). Simple concatenation cannot allow the network to understand the physical nature of this "time-varying parameter," leading to the model's difficulty in capturing the nonlinear drift of traffic flow distribution under complex conditions with multiple factors, thus causing control strategies to fail. Second, the problem of spatial pseudo-correlation in long-distance coordination. Existing attention-based coordination methods often only focus on the similarity of data features, ignoring the physical attenuation characteristics of shock wave propagation in highway traffic flow. This makes it easy for the model to establish incorrect strong connections between road segments that are spatially distant and physically weakly correlated. This not only increases computational redundancy but may also introduce distant noise that interferes with the stability of local control. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, the present invention aims to provide a multi-agent collaborative variable speed limit control method for highways in complex environments. This method encodes weather and event information into continuous scenario features that can describe the intensity and range of influence. It introduces a feature linear modulation mechanism into the multi-agent collaborative Actor network to enable the policy to respond to complex environments. Furthermore, it improves the allocation and stability of the multi-segment collaborative learning process by weighting the local rewards in the neighborhood through a spatial attention collaborative mechanism related to distance decay to form collaborative rewards.

[0005] To achieve the above objectives, the specific technical solution of the present invention is as follows: The method for multi-agent cooperative variable speed limit control on highways in complex environments includes the following steps: Step 1: Divide the highway into multiple control sections, obtain traffic status data of multiple control sections in each control cycle, as well as weather information and event information corresponding to the control sections; Step 2: Based on the weather and event information from Step 1, calculate the scenario-aware feature vector for each controlled road segment. The scenario-aware feature vector includes the affected length ratio, visibility loss rate, capacity loss rate, and road surface friction coefficient. Step 3: Construct a multi-agent cooperative variable speed limit control model. The multi-agent cooperative variable speed limit control model includes an Actor network, a Critic network, a feature linear modulation module, and a spatial attention collaboration module. Step 4: Train the Actor network and Critic network based on collaborative rewards to obtain the trained Actor network; Step 5: During online control, the trained Actor network is invoked in each control cycle. Based on the traffic status data and context-aware feature vectors of each controlled road segment, speed limit actions are output and converted into speed limit control commands, which are then sent to variable speed limit signs, roadside units, or traffic control platforms for execution.

[0006] Furthermore, step 1 includes the following: (1) Obtain the traffic status data, which includes the density, average speed and flow rate of each controlled road segment at each sampling time; (2) Obtain the weather information and event information, the weather information including visibility and rainfall intensity level or snowfall intensity level, and the event information including the length of the affected area of ​​an accident or construction and the number of closed lanes; (3) Align the traffic status data with the weather information and event information according to the sampling time and the control road segment to form time-series input data for training and online control.

[0007] Furthermore, the context-aware feature vector in step 2 is composed of the following features: The percentage of affected length is used to characterize the extent of the impact of an accident or construction on the controlled road section; Visibility loss rate is used to characterize the degree of low visibility; Capacity loss rate is used to characterize the degree of decrease in traffic capacity caused by an event; The road surface friction coefficient is used to characterize changes in road surface adhesion conditions caused by rain and snow.

[0008] Furthermore, the specific details of step 3 are as follows: Step 3.1: The Actor network extracts intermediate features from the traffic state data; Step 3.2: The feature linear modulation module generates scaling and offset parameters corresponding to the intermediate features based on the context-aware feature vector, and performs element-wise scaling and offset modulation on the intermediate features; Step 3.3: The Actor network outputs the speed limit action corresponding to the control section based on the modulated intermediate features; Step 3.4: The spatial attention collaboration module determines the collaboration weight between road segments based on the joint features of each control road segment. The collaboration weight includes a normalized physical distance decay factor that decreases as the spatial distance between road segments increases, and is used to weight and aggregate the local rewards of neighboring control road segments to obtain the collaboration reward.

[0009] Furthermore, the neighborhood control segment includes control segments whose spatial distance from the target control segment is less than a preset threshold; the local reward is composed of a weighted combination of the following indicators: average segment speed, density level, total flow, speed fluctuation index, and density fluctuation index.

[0010] Furthermore, in the training phase of step 4, offline training is carried out in the simulation platform. The training process for each training cycle is as follows: the input is constructed according to steps 1 and 2; the Actor network outputs a speed-limiting action and applies it to the environment to obtain the next state; the local reward and cooperative reward are calculated according to step 3; the parameters of the Critic network and Actor network are updated based on the samples of "state-action-cooperative reward-next state", and the iteration continues until the policy converges to obtain the trained Actor network.

[0011] Furthermore, in step 5, the speed limiting action satisfies the following constraints: (1) The speed limit action shall not be lower than the preset minimum speed limit and not higher than the preset maximum speed limit. The preset minimum speed limit is 60 km / h and the preset maximum speed limit is 120 km / h. In extreme cases (e.g., visibility <50m, serious accidents, etc.), the minimum speed limit may be set to less than 60 km / h.

[0012] (2) The speed limit change range of adjacent control cycles shall not exceed the preset maximum change range, which is 10 km / h.

[0013] (3) The speed limit difference between adjacent controlled road sections shall not exceed the preset maximum speed limit difference, which is 10 km / h.

[0014] Compared with the prior art, the present invention has the following advantages: (1) By quantifying the physical impact of weather and events on the traffic system through context-aware feature vectors, the problem of missing physical coupling mechanism caused by simple splicing of environmental features in existing technologies is solved, enabling the control strategy to accurately respond to the dynamic changes in road capacity under complex environments.

[0015] (2) The characteristic linear modulation mechanism is adopted. The compression effect and flow cutoff effect of environmental factors on traffic capacity are simulated by scaling parameters and offset parameters, respectively. The basic traffic flow graph model adapted to the current environment is dynamically reconstructed at the feature level, which improves the robustness and generalization ability of the model under complex working conditions with multiple superimposed factors.

[0016] (3) The introduction of a spatial attention coordination mechanism that integrates physical distance attenuation factors effectively suppresses the interference of noise from distant road segments on local strategies, solves the problem of spatial pseudo-correlation caused by the traditional attention mechanism ignoring the physical laws of traffic flow, and improves the stability and rationality of multi-segment coordinated control. Attached Figure Description

[0017] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0019] This invention is a variable speed limit control method for highways in complex environments. The "complex environment" refers to the scenario in which weather and events affect highway operation. Weather effects include low visibility, rain or snowfall; event effects include accidents or construction; and it includes scenarios where the above factors occur individually or in combination.

[0020] like Figure 1 As shown, a multi-agent cooperative variable speed limit control method for highways in complex environments includes the following steps: (a) Step 1: Divide the highway into multiple control sections, obtain traffic status data of multiple control sections in each control cycle, as well as weather information and event information corresponding to the control sections.

[0021] The highway corridor to be controlled is divided into multiple control segments according to mileage. Each control segment corresponds to a variable speed limit control unit (variable speed limit sign, roadside unit, or control object in the traffic control platform). Each control segment corresponds to an intelligent agent to achieve parallel decision-making and collaborative control of multiple segments. The control cycle can be set according to engineering requirements (e.g., 10s, 30s, 60s, etc.), and each cycle completes "data acquisition - feature calculation - action output - command issuance - status feedback".

[0022] Traffic status data for each controlled road segment is collected from the traffic detection system. This data includes at least density, average speed, and flow rate. Simultaneously, weather and event information corresponding to the controlled road segments are also collected. Weather information includes visibility, rainfall intensity level, or snowfall intensity level, while event information includes at least the length of the affected area and the number of closed lanes due to accidents or construction. The traffic detection system may include inductive loop detectors, microwave radar, video detection, floating car data, etc. Weather information may come from roadside weather stations or meteorological service interfaces; event information may come from event detection systems or management platforms. The traffic status data, weather information, and event information are aligned according to "sampling time – controlled road segment" to form time-series input data for training and online control.

[0023] (ii) Step 2: Based on the weather and event information in Step 1, calculate the scenario perception feature vector for each controlled road segment. The scenario perception feature vector includes: the proportion of affected length, the visibility loss rate, the capacity loss rate, and the road surface friction coefficient.

[0024] In one alternative implementation, for each control segment During the control cycle Calculate context-aware feature vectors .

[0025] (1) Affected length ratio Used to characterize the impact range of an accident or construction on a controlled road section, it is determined by the ratio of the length of the event's impact interval to the total length of the corresponding road section. Let the road section... Total length is (Each section is typically 5km long), the length of the accident / construction impact zone is... ,but: , Among them, no accidents / construction time .

[0026] (2) Visibility loss rate Used to characterize the degree of low visibility, it is determined based on the degree to which real-time visibility decreases relative to reference visibility. Let the real-time visibility be... Reference visibility is (Determined by the low visibility level threshold, usually taken as 1000m), then: .

[0027] (3) Capacity loss rate It is used to characterize the degree of capacity reduction caused by an event. It is determined based on the ratio of the difference between the baseline capacity and the actual capacity after the event. The actual capacity can be obtained by adjusting the baseline capacity in combination with the effective passable length, the number of closed lanes, or the severity of the event.

[0028] Assume the road segment's baseline capacity is The total usable length is The effective passable length is Severity of the incident Then the bottleneck capacity can be defined as: , Further introduce multi-factor superposition correction coefficient Under a single environmental factor In complex scenarios involving "rain / snow + accidents," considering the additional increase in headway due to driver anxiety, the following settings are implemented: (The values ​​here are for illustrative purposes only; this coefficient can be dynamically adjusted based on the actual road geometry and historical accident statistics); therefore, the actual available capacity is: , The capacity loss rate is: .

[0029] (4) The road surface friction coefficient is used to characterize the changes in road surface adhesion conditions caused by rain and snow. It is obtained by correcting the friction coefficient of dry road surface according to the rainfall intensity level or snowfall intensity level.

[0030] Let the coefficient of friction of the dry road surface be... Rainfall / snowfall intensity levels are mapped to normalized intensity. , The rain and snow impact coefficients are respectively ,but: , In summary, the context-aware feature vector can be represented as: .

[0031] (III) Step 3: Construct a multi-agent cooperative variable speed limit control model. The multi-agent cooperative variable speed limit control model includes an Actor network, a Critic network, a feature linear modulation module, and a spatial attention cooperative module.

[0032] (1) Traffic state vector. For road segments In the cycle Constructing traffic state vectors: , in, These represent density, average speed, and flow rate, respectively. Further, the traffic state vector is concatenated with the context-aware feature vector to obtain joint features: .

[0033] (2) Actor Network and Feature Linear Modulation Module. The Actor Network extracts intermediate features from the traffic state vector: , in, This represents the traffic state feature extraction subnetwork. The feature linear modulation module generates scaling parameters using the context-aware feature vector as input. With offset parameter : , The intermediate features are then element-wise scaled and offset to obtain the modulated features: , in, This indicates element-wise multiplication. The physical significance of using feature linear modulation lies in the fact that traditional neural networks, which process inputs through additive bias, struggle to simulate the multiplicative effect of environmental factors on system parameters in a traffic system. In this invention, a scaling parameter is generated to simulate the compression effect of environmental factors (such as reduced road surface adhesion coefficient due to heavy rain) on road capacity—that is, scaling the traffic flow feature space as a whole; while an offset parameter is used to simulate the flow cutoff or speed benchmark reduction caused by environmental factors (such as lane closures due to accidents)—that is, translating the feature space. Through modulation operations, a basic traffic flow graph model adapted to the current adverse environment is dynamically reconstructed at the feature level, thereby solving the problem of poor versatility of a single model under various weather or event conditions.

[0034] Actor networks output rate-limiting actions based on modulation features: .

[0035] (3) Speed ​​limiting action constraints: To ensure the executability and smoothness of speed limiting control, the following three types of constraints are applied to the speed limiting action output by the Actor: ① Upper and lower limit constraints: The speed limiting action shall not be lower than the preset minimum speed limit. And not exceeding the preset maximum speed limit : .

[0036] ② Variation range between adjacent control cycles: Let the speed limit finally issued in the previous control cycle be... The preset maximum change range is Then the following is required: .

[0037] ③ Speed ​​limit difference between adjacent control road segments: Let the set of control road segments adjacent to road segment i be . The preset maximum speed difference is Then the following is required: .

[0038] (4) Spatial Attention Coordination Module. Assume a road segment... The set of neighboring road segments is Road section and The center distance is Neighboring road sections are usually For distances not exceeding 20km, attention scores are generated based on joint features, and a distance decay term is introduced. , in, These are the query vector and key vector obtained by the joint feature linear mapping. The dimension of the key vector. The normalized road segment spacing, This is the distance attenuation coefficient, typically set to 0.5. The purpose of introducing the distance attenuation term is that the congestion propagation on highways follows the physical law of shock wave propagation, and its correlation decreases exponentially with increasing spatial distance. Ordinary attention mechanisms calculate weights based solely on the similarity of data features, which can easily assign high weights to two road segments that are similar in state but extremely far apart (physically unaffected). By introducing the distance attenuation bias, the interference of far-end noise on the local policy gradient is suppressed.

[0039] Collaborative weights are obtained by normalizing neighborhood scores based on attention scores: .

[0040] (5) Local rewards and synergistic rewards. Local rewards consist of a weighted combination of average speed, density level, total flow, speed fluctuation index, and density fluctuation index of the road segment: , in, For free flow velocity, For the density of blockage, Let i be the traffic flow of segment i. The maximum theoretical flow rate of the corridor and These are the velocity variance and density variance, respectively. and This is an empirical constant used for normalization (taking the historical maximum variance value). This is the weighting coefficient, and its value is usually 1. , , , , The collaborative reward is obtained by weighting and aggregating the local rewards of neighboring road segments using collaborative weights. , in, The weighting factor for local rewards and collaborative rewards is usually... Set to 0.6. The collaborative reward is used to update the Actor / Critic parameters during the offline training phase; during the online inference phase, the Actor network input is traffic state data and context-aware feature vectors, and the collaborative reward does not need to be used as input.

[0041] (iv) Step 4: Train the Actor network and Critic network based on collaborative rewards to obtain the trained Actor network.

[0042] During the training phase, offline training can be conducted on a simulation platform. In each training cycle, the following steps are performed: the input is constructed based on steps 1 and 2; the Actor network outputs a speed-limiting action and applies it to the environment to obtain the next state; the local reward and cooperative reward are calculated based on step 3; the parameters of the Critic network and Actor network are updated based on the "state-action-cooperative reward-next state" sample, and the iteration continues until the policy converges, resulting in the trained Actor network.

[0043] (v) Step 5: During online control, the trained Actor network is invoked in each control cycle. Based on the traffic status data and scenario perception feature vectors of each controlled road segment, the speed limit action is output and converted into a speed limit control command, which is then sent to the variable speed limit sign, roadside unit or traffic control platform for execution.

[0044] During the online control phase, current traffic status data, weather information, and event information are collected in each control cycle to calculate the scenario perception feature vector for each controlled road segment. The traffic status data and scenario perception feature vectors are input into the trained Actor network to output speed limit actions. Constraints are applied to the speed limit actions (minimum / maximum speed limit, change range between adjacent cycles, speed limit difference between adjacent road segments). The speed limit actions are converted into speed limit control commands and issued to variable speed limit signs, roadside units, or traffic control platforms for execution. After the control cycle ends, the traffic status after execution is collected to enter the next control cycle, achieving continuous closed-loop control.

[0045] Speed ​​limiting actions must satisfy the following constraints: ① The speed limit action shall not be lower than the preset minimum speed limit and not higher than the preset maximum speed limit. The preset minimum speed limit is 60 km / h and the preset maximum speed limit is 120 km / h. In extreme cases (e.g., visibility <50m, serious accidents, etc.), the minimum speed limit may be set to be lower than 60 km / h.

[0046] ② The speed limit change range between adjacent control cycles shall not exceed the preset maximum change range, which is 10 km / h.

[0047] ③ The speed limit difference between adjacent controlled road sections shall not exceed the preset maximum speed limit difference, which is 10 km / h.

[0048] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-agent cooperative variable speed limit control method for highways in complex environments, characterized in that, Includes the following steps: Step 1: Divide the highway into multiple control sections, obtain traffic status data of multiple control sections in each control cycle, as well as weather information and event information corresponding to the control sections; Step 2: Calculate the scenario perception feature vector for each controlled road segment. The scenario perception feature vector includes the affected length ratio, visibility loss rate, capacity loss rate, and road surface friction coefficient. Step 3: Construct a multi-agent cooperative variable speed limit control model. This model includes an Actor network, a Critic network, a feature linear modulation module, and a spatial attention coordination module. Details are as follows: Step 3.1: The Actor network extracts intermediate features from the traffic state data; for road segments... In the cycle Constructing traffic state vectors: ,in, Let density, average speed, and flow rate represent these respectively. Further, the traffic state vector is concatenated with the context-aware feature vector to obtain joint features: ; Step 3.2: The feature linear modulation module generates scaling and offset parameters corresponding to the intermediate features based on the context-aware feature vector, and performs element-wise scaling and offset modulation on the intermediate features; Step 3.3: The Actor network outputs the speed limit action corresponding to the control section based on the modulated intermediate features; Step 3.4: The spatial attention coordination module determines the coordination weight between road segments based on the joint features of each control road segment. The coordination weight includes a normalized physical distance decay factor that decreases as the spatial distance between road segments increases, and is used to weight and aggregate the local rewards of the neighboring control road segments to obtain the coordination reward. Set up road section The set of neighboring road segments is Road section and The center distance is Attention scores are generated based on joint features, and a distance decay term is introduced: ,in, These are the query vector and key vector obtained by the joint feature linear mapping. The dimension of the key vector. The normalized road segment spacing, The distance decay coefficient is used; the collaborative weights are obtained by normalizing the scores within the neighborhood based on the attention scores. ; Step 4: Train the Actor network and Critic network based on collaborative rewards to obtain the trained Actor network; Step 5: During online control, the trained Actor network is invoked in each control cycle. Based on the traffic status data and context-aware feature vectors of each controlled road segment, speed limit actions are output and converted into speed limit control commands, which are then sent to variable speed limit signs, roadside units, or traffic control platforms for execution.

2. The method for multi-agent cooperative variable speed limit control on highways in complex environments according to claim 1, characterized in that, Step 1 includes the following: (1) Obtain the traffic status data, which includes the density, average speed and flow rate of each controlled road segment at each sampling time; (2) Obtain the weather information and event information, the weather information including visibility and rainfall intensity level or snowfall intensity level, and the event information including the length of the affected area of ​​an accident or construction and the number of closed lanes; (3) Align the traffic status data with the weather information and event information according to the sampling time and the control road segment to form time-series input data for training and online control.

3. The method for multi-agent cooperative variable speed limit control on highways in complex environments according to claim 1, characterized in that, The context-aware feature vector in step 2 consists of the following features: The percentage of affected length is used to characterize the extent of the impact of an accident or construction on the controlled road section; Visibility loss rate is used to characterize the degree of low visibility; Capacity loss rate is used to characterize the degree of decrease in traffic capacity caused by an event; The road surface friction coefficient is used to characterize changes in road surface adhesion conditions caused by rain and snow.

4. The method for multi-agent cooperative variable speed limit control on highways in complex environments according to claim 1, characterized in that, The neighborhood control segment includes control segments whose spatial distance from the target control segment is less than a preset threshold; the local reward is composed of a weighted combination of the following indicators: average segment speed, density level, total flow, speed fluctuation index, and density fluctuation index.

5. The method for multi-agent cooperative variable speed limit control on highways in complex environments according to claim 1, characterized in that, In the training phase of step 4, offline training is carried out in the simulation platform. The training process of each training cycle is as follows: the input is constructed according to steps 1 and 2; the Actor network outputs a speed-limiting action and applies it to the environment to obtain the next state; the local reward and cooperative reward are calculated according to step 3; the parameters of the Critic network and Actor network are updated based on the "state-action-cooperative reward-next state" sample, and the iteration continues until the policy converges to obtain the trained Actor network.

6. The method for multi-agent cooperative variable speed limit control on highways in complex environments according to claim 1, characterized in that, In step 5, the speed limiting action satisfies the following constraints: (1) The speed limit action shall not be lower than the preset minimum speed limit and not higher than the preset maximum speed limit; (2) The speed limit change range between adjacent control cycles does not exceed the preset maximum change range; (3) The speed limit difference between adjacent controlled road sections shall not exceed the preset maximum speed limit difference.

Images