A city-level multi-scenario intelligent traffic collaborative management and control and credible simulation method based on reinforcement learning

Abstract

The application discloses a kind of city-level multi-scene intelligent traffic collaborative management and simulation method based on reinforcement learning, comprising the following steps: S1, city-level multi-scene multi-source traffic data is collected, and after time-space alignment and abnormal cleaning, a standardized data set is obtained;S2, a simulation rule and model library suitable for mixed traffic flow are established, and a road network efficiency quantitative analysis system is constructed;S3, a multi-agent space-time attention reinforcement learning model is constructed, and the agent is trained to generate a regional-level collaborative traffic signal control scheme, which is input into a simulation platform for credible verification;S4, the scheme and model are iteratively optimized according to the simulation results to form a scenario-adaptive control template;S5, the optimized template is deployed to an edge computing node to achieve fast dynamic response, and federated continuous iteration is performed in the cloud.The application solves the key technical problems of city-wide traffic collaborative governance difficulty, low simulation credibility of control scheme and weak adaptability of simulation platform.

Description

Technical Field

[0001] This invention relates to the fields of reinforcement learning and intelligent traffic control technology, specifically to a city-level multi-scenario intelligent traffic collaborative management and reliable simulation method based on reinforcement learning. Background Technology

[0002] Urban traffic signal control is a key technology for improving road traffic efficiency, alleviating regional congestion, and ensuring urban operational order, directly impacting the overall efficiency of urban transportation systems and the public's travel experience. Intelligent analysis of road network operation status, traffic participant behavior characteristics, and changing scenario factors to automatically generate traffic signal control schemes that balance efficiency and safety has become an important research direction in the field of intelligent transportation. Highly efficient intelligent traffic collaborative management and control systems can accurately identify traffic operation patterns, evaluate the effectiveness of different control strategies, and continuously optimize control parameters based on changes in actual scenarios, thereby improving the operational efficiency of urban road networks and the level of intelligent traffic management.

[0003] Existing urban traffic control and optimization methods mainly fall into three categories: fixed rule-based methods, traffic flow statistical modeling-based methods, and machine learning-based methods. Fixed rule-based methods primarily rely on preset timing schemes, human experience, or historical templates to control traffic signals; they are simple to implement but lack flexibility. Traffic flow statistical modeling-based methods optimize signals by analyzing indicators such as traffic flow, queue length, and delays, but their adaptability to complex and ever-changing urban traffic scenarios is limited. The introduction of reinforcement learning, graph neural networks, and digital twin simulation technologies provides new technical pathways for city-level traffic collaborative control. However, existing methods still have significant shortcomings in dealing with large-scale urban road networks, mixed traffic of various types of participants, dynamic switching between multiple scenarios, and the reliable verification of control strategies.

[0004] The main shortcomings of existing city-level intelligent traffic management technologies are: 1) When dealing with large-scale road networks and multi-intersection collaborative control tasks, it is difficult to balance computational efficiency and regional collaborative effects, resulting in slow optimization of control strategies; 2) They lack dynamic adaptability to various scenarios such as daily commuting, large-scale events, holidays, and emergencies, making it difficult to quickly switch and adjust control schemes according to real-time traffic status changes; 3) For the common urban scenario of mixed traffic flow of motor vehicles, non-motor vehicles, and pedestrians, existing methods do not adequately characterize the propagation of traffic conflicts, behavioral disturbances, and congestion evolution, leading to discrepancies between simulation results and actual operating conditions; 4) Existing methods generally lack the ability to reliably verify and iteratively optimize reinforcement learning decision results with digital twin simulation platforms, making it difficult to guarantee the stability, reliability, and continuous adaptive optimization effect of control strategies in real road network deployments. Summary of the Invention

[0005] To address the aforementioned shortcomings in existing technologies, the purpose of this invention is to provide a city-level multi-scenario intelligent traffic collaborative management and reliable simulation method based on reinforcement learning. This method can solve the problems of existing urban traffic management methods lacking the ability to dynamically and collaboratively adjust traffic conditions across multiple scenarios, resulting in low regional traffic efficiency, slow congestion response speed, and insufficient adaptability of control strategies.

[0006] To achieve the aforementioned objectives, the present invention employs the following technical solution: a city-level multi-scenario intelligent traffic collaborative management and reliable simulation method based on reinforcement learning, comprising the following steps: S1. Collect city-level multi-scenario, multi-source traffic data and obtain a standardized dataset through preprocessing; S2. Establish a simulation rules and model library adapted to mixed traffic flow, and construct a road network efficiency quantitative analysis system; S3. Construct a multi-agent spatiotemporal attention reinforcement learning model, train the agents to generate a regional-level collaborative traffic signal control scheme, and input it into the simulation platform to complete the credible verification. S4. Iteratively optimize the scheme and model based on the simulation results to form a scenario-adaptive control template; S5. Deploy the optimized template to edge computing nodes to achieve rapid dynamic response and continuous federation in the cloud.

[0007] Further, step S1 includes the following sub-steps: S101. Based on the city-level multi-scenario traffic management needs, comprehensively collect core data, including but not limited to: road network basic data, real-time traffic flow data, traffic signal operation data, traffic participant behavior data, and scenario-related data. Among them, road network basic data includes core road attributes and spatial coordinate information, including road topology, number of lanes, and road grade; real-time traffic flow data includes key vehicle-related indicators collected by static and mobile sensing devices; traffic signal operation data includes real-time and historical data of traffic lights at each intersection; traffic participant behavior data includes key data related to drivers and pedestrians; and scenario-related data includes information on daily peak hour distribution, number of participants in large-scale events, and the impact range of sudden accidents. S102. According to the data quality control standards, outlier removal is performed on the collected multi-source traffic data to obtain preliminary cleaned data. The outlier determination formula is as follows: Outlier(x, Scene, t) = Where Outlier(x, Scene, t) is the outlier determination result, where 1 represents an outlier and 0 represents normal data; x is the data sample value, μ Scene, t φ represents the average value of the data at the corresponding scenario and time point. Sceneσ is the scene-adaptive threshold coefficient. Scene, t This represents the standard deviation of the data in the corresponding scenario. S103. Based on the constraints of temporal continuity, spatial proximity and scene similarity, perform spatiotemporal attention imputation of missing values ​​to obtain complete traffic data; S104. Based on the spatiotemporal unification standard, perform spatiotemporal alignment on the complete data. The spatiotemporal confidence alignment formula is as follows: Φ i (t) = ω1*Δx i (t)+ω2*Δy i (t)+ω3*Δτ i (t)+ω4*(1-C i (t)) Where, Δx i (t) and Δy i (t) represents the spatial position deviation, Δτ i (t) represents the timestamp offset, C i (t) represents the credibility of the i-th type of data source at time t, and ω1 to ω4 are the alignment weights. i When (t) is less than the spatiotemporal alignment tolerance threshold, the corresponding data is determined to have completed spatiotemporal alignment; S105. Based on the dynamic topology construction rules, construct a dynamic adjacency matrix to obtain the road network traffic spatiotemporal graph. The formula for calculating matrix elements is: A ij (t) = Dist(i, j) -1 *exp (- )*(1+θ2*Corr ij (t)+θ3*RiskFlow ij (t)) Where Dist(i, j) is the physical distance between node i and node j, and ΔPhase ij (t) represents the signal phase difference between the two intersections at time t, τ travel,ij (t) represents the expected free-flow arrival time of the vehicle between the two nodes, Corr ij (t) represents the traffic flow correlation coefficient, RiskFlow ij (t) represents the conflict propagation influence coefficient, and θ1, θ2, and θ3 are adjustment parameters; S106. According to the feature extraction rules, the dynamic adjacency matrix is ​​input into the spatiotemporal graph representation learning network, and spatiotemporal graph feature fusion encoding is performed to obtain a low-dimensional standardized state matrix that contains green light continuity information, conflict propagation and scene context information.

[0008] Furthermore, the specific process of learning the spatiotemporal graph representation in step S106 is as follows: (1) Based on the signal coordination requirements, the phase coordination relationship between adjacent intersections is quantified to obtain the phase coupling strength between nodes. The calculation formula is as follows: P ij (t) = max(0, 1- )*I green (t) Among them, P ij (t) represents the phase coupling strength between node i and node j at time t, T arrive,ij (t) represents the predicted arrival time of the target traffic flow released from upstream intersection i to downstream intersection j, where T is the arrival time. match,j (t) represents the phase matching time window at downstream intersection j, I green (t) is the phase indicator function; (2) Jointly assess the queuing overflow risk and trajectory overlap between adjacent nodes to obtain the conflict propagation constraint strength C between nodes. ij (t), the calculation formula is: C ij (t) = ω c *exp(η )+(1-ω c *Overlap ij (t) Among them, C ij (t) represents the conflict propagation constraint strength between node i and node j at time t, L queue, j (t) represents the real-time queue length extending from downstream intersection j to road segment ij, L link, ij (t) represents the physical length of the road segment connecting node i and node j, Overlap ij (t) represents the degree of overlap in trajectory occupancy between motor vehicles and non-motor vehicles or pedestrians on road segment ij, ω c η is the weighting coefficient between queue overflow risk and the degree of overlap in trajectory occupancy, and η is the queue overflow sensitivity parameter. (3) Based on the phase coupling strength and the conflict propagation constraint strength, the dynamic adjacency matrix constructed in step S105 is jointly modified to obtain the joint encoding weight, calculated using the following formula: W ij (t) = A ij (t)* Among them, W ij (t) represents the joint encoding weights of node i and node j at time t, A ij (t) represents the elements of the dynamic adjacency matrix; (4) Normalize the joint coding weights to obtain the normalized propagation weights. The calculation formula is as follows: ij (t) = in, ij (t) represents the normalized node propagation weight, and N(i) represents the set of neighboring nodes of node i. (5) The traffic observation features of neighboring nodes are weighted and aggregated using normalized propagation weights, and then jointly encoded with the participant behavior features and scene embedding features of the current node to obtain the structured feature representation of the node. (6) Normalize and map the structured feature representation of the nodes to obtain a standardized state matrix that simultaneously contains signal coordination information, conflict risk information, and scene context information. The calculation formula is as follows: S i (t) = Norm(H i (t), M i (t), E i (t)) Among them, S i H(t) is the standardized state vector of node i at time t. i (t) represents the traffic structure characteristics of the node, M i (t) represents the behavioral characteristics of node participants, E i (t) represents the node scene embedding feature, and Norm() is the normalization mapping function.

[0009] The beneficial effects of the above-mentioned further scheme are as follows: By introducing phase coupling strength and conflict propagation constraint strength to jointly correct the dynamic graph, the signal coordination relationship and conflict propagation relationship between intersections can be more accurately characterized. On this basis, feature aggregation and normalization coding are completed, reducing the complexity of high-dimensional heterogeneous data expression, which helps to improve the ability to express traffic conditions and the accuracy of subsequent reinforcement learning decisions.

[0010] Furthermore, step S2 includes the following sub-steps: S201. Based on the core characteristics of mixed traffic flow, formulate core simulation rules for mixed traffic scenarios involving motor vehicles, non-motor vehicles, and pedestrians. For scenarios involving mixed traffic of motor vehicles and non-motor vehicles at large intersections, pedestrian crossings, bus stops, and temporary pick-up and drop-off of ride-hailing vehicles, formulate conflict avoidance mechanisms based on dynamic game theory and trajectory prediction to obtain a set of simulation rules adapted to mixed traffic flow. S202. Based on the behavioral characteristics of traffic participants, calculate the conflict risk for any two traffic participants at time t to obtain the conflict risk value, and the conflict risk Risk between node i and node j. ij The formula for calculating (t) is: Risk ij (t) = [α*O ij (t)+β*Gij (t)]*(1+γ*B ij (t)+δ*P ij (t)) + ζ*H ij (t) Among them, O ij (t) represents the overlap of the predicted trajectory, G ij (t) represents the conflict intensity of the prevailing game, B ij (t) represents the intensity of encroachment on the boundary between motor vehicles and non-motor vehicles, P ij (t) represents the pedestrian priority disturbance intensity, H ij (t) represents the historical high-risk behavior memory item, and α, β, γ, δ, and ζ represent the risk fusion weights; S203. Conduct a joint assessment of the traffic benefits, waiting losses, conflict risks, local queuing disturbances and congestion shock wave spread effects of traffic participants after performing each candidate action, and dynamically adjust the comprehensive benefits of each candidate action according to the risk level under the current mixed traffic scenario to obtain the benefit assessment results corresponding to each candidate action. S204. Based on the benefit evaluation results of each candidate action, select the action with the largest comprehensive benefit as the action to be executed by the traffic participant at time t, and obtain the dynamic decision result of the traffic participant. S205. Based on the behavioral characteristics of traffic participants, formulate an appropriate micro-behavioral model to obtain a micro-behavioral model that adapts to mixed traffic flow. S206. Based on scenarios such as mixed traffic of motorized and non-motorized vehicles at large intersections, bus stops, and temporary passenger pick-up and drop-off for ride-hailing vehicles, a joint correction is performed on the uncorrected congestion propagation velocity to obtain the congestion propagation velocity. The calculation formula is as follows: v' w (t) = v w (t)*[1-β1 ]*[1-β2 ]*[1+β3G release (t)] Among them, v' w (t) represents the corrected congestion propagation speed, v w (t) represents the original congestion propagation speed without considering the effects of non-motorized vehicle infiltration, temporary lane occupation, and signal release capacity. non-motor (t) represents the real-time sensed non-motorized vehicle infiltration flow, W lane W is the total width of the mixed-traffic lane for motor vehicles and non-motor vehicles. occ (t) represents the equivalent physical width for parked vehicles or temporary road occupancy, L ref For reference queue length, G release (t) represents the effective green light release ratio within the current signal cycle, and β1, β2, and β3 are correction coefficients; S207. Based on the corrected congestion propagation speed, the congestion propagation impact boundary and congestion dissipation time are dynamically evaluated to obtain the congestion propagation evolution results. S208. Based on the principle of modular integration, the core simulation rules, dynamic decision-making results of traffic participants, micro-behavioral models, and mixed traffic flow congestion propagation models are integrated to obtain a dynamic model library that adapts to different traffic scenarios. S209. Quantitatively analyze traffic efficiency, congestion control, safety risks, and collaborative control effectiveness, construct a road network efficiency quantitative analysis system, and obtain road network efficiency evaluation results for simulation assessment and subsequent reinforcement learning training.

[0011] The beneficial effects of the above-mentioned further scheme are as follows: by establishing simulation core rules, conflict risk models, dynamic decision-making mechanisms, micro-behavioral models and congestion propagation models that are adapted to the characteristics of mixed traffic flow, the traffic operation mechanism under mixed conditions of motor vehicles, non-motor vehicles and pedestrians can be more realistically reflected. Furthermore, by constructing a road network efficiency quantitative analysis system, not only is the scenario adaptability and credibility of the simulation results improved, but reliable support is also provided for subsequent reinforcement learning training and control scheme verification.

[0012] Furthermore, the micro-behavioral model in step S205 specifically includes: (1) Vehicle following model considering driving emotions: The calculation formula is as follows: a n (t) = a max *[1- ]*exp(-k ) Among them, a n (t) represents the acceleration of the nth vehicle at time t, a max For maximum acceleration, v n (t) represents the current velocity, v des (t) represents the desired velocity, s ∗ (t) represents the dynamic safety following distance, s n (t) represents the actual following distance, k represents the emotional sensitivity coefficient, and L queue (t) represents the current queue length, L max This represents the maximum capacity of the intersection. (2) Phase Coordination Lane Changing Decision Model: Based on the difference in traffic utility between the target lane and the current lane, the risk cost of lane changing, and the degree of matching between the vehicle and the target green light phase when it arrives at the downstream intersection, the probability of successful lane changing is dynamically evaluated to obtain a lane changing decision result that meets the requirements of local safety and downstream signal coordination. (3) Non-motorized vehicle weaving model: Based on the operating characteristics of non-motorized vehicles under mixed traffic conditions, the non-motorized vehicle speed, lateral interference intensity, and available clearance length are jointly corrected to obtain the dynamic following distance of the non-motorized vehicle. The formula for calculating the dynamic following distance of the non-motorized vehicle is as follows: S non-motor (t) = s0+0.5*λ1*v non-motor (t)+Inter side (t)-Gap front (t) Among them, S non-motor (t) represents the dynamic following distance of the non-motorized vehicle at time t, s0 is the basic safety distance, and v non-motor (t) represents the speed of the non-motorized vehicle, Inter side (t) represents the lateral interference intensity, Gap front (t) represents the length of the available gap ahead, and λ1 is the non-motorized vehicle behavior correction coefficient; (4) Pedestrian crossing group model: Based on the characteristics of group crossing behavior, the speed of individual crossing, group size, degree of interference from oncoming vehicles and cross-section occupancy effect are jointly corrected to obtain the group crossing speed and group crossing time, and thus obtain the characteristics of group crossing behavior. (5) Bus Priority Control Model: Based on the bus's operating status when approaching the intersection, the distance of the bus from the intersection, the bus's passenger load level, and the current traffic light phase are jointly determined to obtain the bus priority triggering result. The red light shortening time is dynamically adjusted based on the urgency of bus priority and downstream queuing pressure. The formula for determining the bus priority triggering condition is as follows: Priority bus (t) Among them, Priority bus (t) is the bus priority trigger signal, distance(t) is the distance of the bus from the intersection, Load bus (t) represents the passenger load level of public transport, D th L is the distance threshold. th The passenger capacity threshold is given, and Phase(t) is the current traffic light phase. (6) Ride-hailing passenger pick-up and drop-off model: Based on the characteristics of ride-hailing temporary stops, the number of passengers picking up and dropping off, the degree of roadside interference, the stopping scenario, the duration of passenger pick-up and drop-off on the road, the intensity of road occupation, and the transfer capacity of adjacent lanes are jointly corrected to obtain the duration of passenger pick-up and drop-off and the capacity of affected lanes, and thus obtain the characteristics of the impact of ride-hailing temporary stops.

[0013] The beneficial effects of the above-mentioned further scheme are as follows: by characterizing the typical behavioral features of motor vehicles, non-motor vehicles, pedestrians, buses and ride-hailing vehicles respectively, it is possible to more comprehensively reflect the disturbances of car following, lane changing, crossing the street, priority passage and temporary stopping in mixed traffic flow, which helps to improve the accuracy of micro-simulation and provide more granular support for conflict risk identification and control decision optimization.

[0014] Furthermore, the specific analysis process of the road network efficiency quantitative analysis system in step S209 is as follows: (1) Based on the traffic efficiency calculation method, the average delay time of intersections, the average travel speed of road segments and the utilization rate of road network capacity are calculated to obtain the traffic efficiency index. (2) Calculate the congestion duration, congestion range and delay index, and combine the congestion propagation model of mixed traffic flow in step S206 to identify the congestion propagation and dissipation process to obtain congestion control indicators. (3) Based on the safety risk assessment method, the number of intersection conflicts, the frequency of rapid acceleration and the frequency of rapid deceleration are calculated to obtain the safety risk index; (4) Using the collaborative control evaluation method, the phase coordination degree of adjacent intersections, the continuity of green light passage and the regional collaborative release effect are calculated to obtain the collaborative control index; (5) The weighted fusion evaluation method is adopted to weight and fuse traffic efficiency indicators, congestion control indicators, safety risk indicators and collaborative control indicators to obtain the comprehensive evaluation value of road network efficiency.

[0015] The beneficial effects of the above-mentioned further scheme are as follows: by comprehensively and quantitatively evaluating traffic efficiency, congestion control, safety risks and collaborative control effects, the operation status of the road network and the merits of the control scheme can be reflected from multiple dimensions, providing a unified evaluation basis for simulation verification, model training and scheme optimization, thereby improving the scientificity and consistency of simulation evaluation, model training and scheme optimization.

[0016] Furthermore, step S3 includes the following sub-steps: S301. Based on the requirements of multi-agent collaborative management and control, construct a multi-agent reinforcement learning model, with each intersection corresponding to an agent, to obtain a regional multi-agent collaborative control model. S302. Based on the traffic propagation relationship, conflict propagation relationship, scene consistency, phase coupling relationship, and queuing pressure difference between intersections, the neighborhood information of each agent is weighted and aggregated to obtain a cooperative state representation. The aggregation weight calculation formula is as follows: ψ ij (t) = Where, ψ ij (t) represents the aggregation weight of node i to node j at time t, Corrij (t) represents the traffic flow association strength, Phase ij (t) represents the phase coupling strength, Scene ij (t) represents the scene consistency strength, Gap ij (t) represents the queuing pressure difference between nodes; S303. Based on the aggregation weight, the state information of neighboring nodes is weighted and fused, and combined with the local state information of the current node, the collaborative state representation of the intersection agent is obtained: ij (t) = h i (t)+ in, ij (t) represents the cooperative state of intersection i at time t, h i (t) is the local state latent vector of intersection i, h j (t) is the local state latent vector of intersection j; S304. According to the model training specifications, initialize the reinforcement learning model parameters, and prioritize the use of samples according to the empirical sample scheduling rules. Prioritize the training of samples with large time difference errors, high collision risks, and obvious scene boundary features. Use the average relative change rate of reward as the convergence criterion. When the average relative change rate of reward for a preset number of consecutive iterations is not greater than the preset threshold, stop training to obtain a training process that meets the convergence requirements. S305. Construct a reliable twin constraint reward function based on the training objective, and train the multi-agent spatiotemporal attention reinforcement learning model to obtain the trained collaborative control agent. The reward function satisfies: R i (t) = Among them, R i (t) represents the reward value of intersection i at time t, ΔEff i (t) represents the efficiency improvement, Cred i (t) represents the consistency score between the digital twin simulation results and the measured traffic conditions, Coord i (t) represents the regional cooperative gain, and Delay i (t) represents the delay cost, Queue i (t) represents the queuing cost, Risk. i (t) represents the risk cost; S306. Based on the standardized state matrix obtained in step S1, the trained cooperative control agent is input into the regional road network for interactive decision-making, outputs the optimal signal control action, generates a regional-level cooperative traffic signal control scheme covering multiple adjacent intersections, and optimizes the phase difference coordination logic of adjacent intersections to obtain the initial control scheme. S307. According to the trusted verification requirements, the initial control scheme is input into the digital twin-based simulation platform to complete the trusted verification of the scheme. The simulation parameter configuration is expressed as follows: SimParam = {Step, RecordFreq, SceneType, λ sim} Where Step is the simulation step size, RecordFreq is the data recording frequency, SceneType is the scene type defined in step S2, and λ sim For simulation accuracy coefficients; S308. According to the simulation data recording rules, record the delays, speeds, congestion status, and implementation effects of the scheme during the simulation process to obtain the simulation verification results. The simulation data record is expressed as follows: SimData = {Delay(t), Speed(t), Congest(t), PhaseExec(t)} Where Delay(t) is the intersection delay time at time t, Speed(t) is the road segment speed at time t, Congest(t) is the congestion status at time t, and PhaseExec(t) is the signal control scheme execution effect at time t.

[0017] The beneficial effects of the above-mentioned further scheme are as follows: by constructing a regional multi-agent reinforcement learning model and incorporating the traffic propagation relationship, scene consistency, phase coupling relationship and queuing pressure difference between intersections into the collaborative state modeling, the joint perception and collaborative decision-making capabilities among multiple intersections can be enhanced. Furthermore, by combining the trusted twin constraint reward and simulation verification mechanism, the effectiveness, deployability and adaptability to complex dynamic traffic scenarios of the initial control scheme can be improved.

[0018] Furthermore, the specific process for training the multi-agent spatiotemporal attention reinforcement learning model in step S305 is as follows: (1) Based on the empirical sample scheduling rules, the sample time difference error, minimum collision time, scene scarcity, and scene switching boundary characteristics are jointly evaluated to obtain the sample sampling priority. The calculation formula is as follows: Priority(s, a) = (|TD_Error|+ )*(1+Rare(s)+Shift(s)) Where TD_Error is the temporal difference error of the sample, TTC(s, a) is the minimum collision time after performing action a in state s, Rare(s) is the scarce scene indicator, Shift(s) is the scene switching boundary indicator, and ε is a minimal constant to prevent the denominator from being zero. (2) Based on the sampling priority, the samples in the experience playback library are sorted and stratified for retention, so that high-risk samples, scarce scene samples and scene boundary samples can obtain a higher retention level. (3) Allocate sample sampling probabilities according to sample sampling priority, and prioritize the use of experience replay samples so that high-priority samples can obtain a higher training call frequency. (4) Enhance the input traffic flow state by applying scene disturbances to obtain the enhanced traffic flow characteristics. The calculation formula is as follows: Q aug (t) = Q(t)*(1+Bias(t)+Resist(t)) Among them, Q aug (t) represents the enhanced flow characteristics, Q(t) represents the original flow characteristics, Bias(t) represents the scene offset term, and Resist(t) represents the weather impedance term; (5) Based on the topology regularization rule, the connections between nodes with weak correlations in the dynamic traffic graph are randomly suppressed to obtain the topology-constrained training graph. The formula for calculating the edge drop probability is: p ij (t) = p0(1- ) Where, p ij (t) represents the edge drop probability between node i and node j at time t, p0 is the basic drop rate, max(A(t)) is the maximum weight value in the dynamic adjacency matrix at time t, and m is a minimal constant to prevent the denominator from being zero.

[0019] The beneficial effects of the above-mentioned further scheme are as follows: by prioritizing training on high-risk samples, scarce scene samples and scene boundary samples, and combining scene perturbation enhancement and topology regularization, the model's learning ability for complex scenes and extreme working conditions can be improved, and the training stability, generalization performance and the adaptability of collaborative control strategies in real-world scenarios can be enhanced.

[0020] Furthermore, step S4 includes the following sub-steps: S401. Based on the trusted verification result of step S3, set the optimization triggering conditions and obtain the optimization triggering result. The formula for determining the triggering conditions is: Trigger Among them, Trigger is the optimization trigger signal, 1 indicates that optimization is started and 0 indicates that it is not started; Similarity is the matching degree between simulation data and actual collected data; DelayGain is the delay optimization magnitude; SafeScore is the extreme scenario adaptability score; θ1 is the simulation matching degree trigger threshold; θ2 is the delay optimization trigger threshold; and θ3 is the safety adaptation trigger threshold. S402. Based on the optimized triggering results, the signal control parameters and simulation parameters are adjusted in a targeted manner to obtain the adjusted parameters. The adjustment range includes at least the phase difference, green ratio, signal period, conflict risk threshold and congestion propagation model correction coefficient. S403. Based on the adjusted parameters, substitute them into the multi-agent spatiotemporal attention reinforcement learning model of step S3, and jointly update the control parameter correction, simulation model correction, scene adaptation correction and risk control correction to obtain the optimized joint parameter set. S404. Based on the preset evaluation criteria, repeat the training and simulation verification, iteratively correct the joint parameter set, and obtain an optimized solution that meets the requirements of credibility, performance and security. S405. Based on the scenario classification rules, the optimization scheme is solidified and classified into daily commuting, large-scale events, holidays and emergency scenarios to obtain scenario adaptation and control templates.

[0021] The beneficial effects of the above-mentioned further solutions are as follows: By setting optimized trigger conditions and jointly iteratively correcting control parameters, simulation parameters, and scenario adaptation parameters, it is possible to identify the sources of deviation between the control scheme and the actual road network state in a timely manner and continuously optimize them. On this basis, repeated training and simulation verification can continuously improve the reliability, traffic efficiency, and safety of the scheme, and form a reusable template suitable for different traffic scenarios, thereby improving the efficiency of subsequent deployment.

[0022] Furthermore, step S5 includes the following sub-steps: S501. Based on the actual deployment requirements of the traffic control system, deploy the scenario adaptation control template obtained in step S4 to the edge-side signal controller to obtain the edge-side executable template set. S502. Based on the real-time traffic flow collection mechanism, the road network operation status is continuously monitored to obtain the road network status monitoring results; S503. Based on the road network status monitoring results, quickly match the scenario-adaptive control templates to obtain the candidate template scoring results; S504. Based on the candidate template scoring results, select the template with the highest comprehensive score as the template to be executed; S505. Based on the traffic mutation rate, equipment status and conflict risk change rate, edge cases are identified to obtain template switching trigger results; S506. Based on the long-term optimization rules, evaluate the effectiveness of the deployed solutions to obtain a solution effectiveness score. The calculation formula is as follows: E = Where E is the scheme effectiveness score, M is the number of samples in the evaluation period, and ΔDelay m Let ΔSpeed ​​be the delay reduction rate for the m-th sample. m Let ΔCongest be the velocity increase rate of the m-th sample. m Let w1, w2, and w3 be the congestion relief rate for the m-th sample, and w1, w2, and w3 be the weighting coefficients. S507. Based on edge cases and edge-side incremental parameters, aggregate parameter increments from various regions in the cloud, and combine them with scarce scenario correction information, road network structure change information, and event disturbance correction information to continuously update the control template in a federated manner, thereby obtaining the updated scenario template. S508. Based on the updated scenario template, continuously revise the traffic control strategy in the actual road network to obtain a continuous adaptive learning result that adapts to the evolution of the actual road network.

[0023] The beneficial effects of the above-mentioned further solutions are as follows: by deploying the scenario-adaptive control template to the edge signal controller, and combining it with real-time monitoring, template matching, trigger switching, effectiveness evaluation and cloud federation continuous update mechanism, it is possible to achieve rapid response and continuous evolution of traffic control strategies. This not only improves the local handling capability under sudden traffic disturbances, but also enhances the cross-regional experience sharing and long-term adaptive optimization capability, thereby improving the stability and intelligence level of actual road network operation.

[0024] In summary, the beneficial effects of this invention are as follows: (1) In terms of traffic data processing and status construction, by uniformly collecting, cleaning, filling and aligning basic road network data, real-time traffic flow data, signal operation data, traffic participant behavior data and scene-related data, more complete, accurate and reliable traffic status information can be provided. (2) By constructing a dynamic adjacency matrix and introducing phase coupling strength and conflict propagation constraint strength for spatiotemporal graph feature fusion, the collaborative relationship between multiple intersections and the characteristics of mixed traffic conflict propagation can be more effectively characterized, providing a more comprehensive state representation for multi-agent reinforcement learning; (3) Based on the self-built mixed traffic simulation rules and multiple micro-behavioral models, the diverse driving characteristics of motor vehicles, non-motor vehicles, pedestrians, buses and ride-hailing vehicles are fully reproduced, which fits the actual working conditions of urban mixed traffic. Combined with a multi-dimensional road network efficiency evaluation system, the authenticity of the simulation results is improved, and a credible evaluation benchmark is established for the verification of control schemes. (4) In the training process of multi-agent spatiotemporal attention reinforcement learning, not only is the effectiveness and credibility of the control strategy improved by the credible twin constraint reward function, but the training efficiency and convergence speed are also improved by sample priority scheduling, scene perturbation enhancement and topology regularization. (5) In the model training stage, high-value sample priority learning, traffic data perturbation enhancement and topology regularization optimization strategies are adopted to specifically make up for the shortage of samples in extreme scenarios, reduce the risk of model overfitting, accelerate the model convergence rate, and significantly improve the generalization and adaptation ability of the agent in unconventional scenarios such as sudden congestion and large gatherings. (6) By leveraging the cloud-edge collaborative architecture to implement solidified management and control templates, the edge terminal can quickly match and switch control strategies based on real-time monitoring to achieve instantaneous road condition handling. The cloud aggregates incremental parameters from various regions to carry out federated iterative optimization and continuously iterates and updates scenario templates to achieve long-term adaptive iterative upgrades of road network management and control based on road conditions.

[0025] This invention can generate collaborative control schemes that adapt to urban-level multi-scenario traffic operation environments, and has higher traffic efficiency, better congestion mitigation effect and stronger dynamic adaptive capability in actual road networks.

[0026] Other advantages, objectives, and features of the present invention will be set forth in detail in the following description. Through a thorough study of the following text, those skilled in the art will be able to clearly recognize these advantages and features and gain valuable lessons from the practice of the present invention. Attached Figure Description

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein: Figure 1 This is a flowchart of the process of the present invention; Figure 2 This is a schematic diagram of the regional road network and key node numbering in an embodiment of the present invention; Figure 3 This is a flowchart of the multi-source traffic data processing and spatiotemporal map construction process of the present invention; Figure 4 This is a diagram of the multi-agent spatiotemporal attention reinforcement learning and trusted simulation architecture of the present invention; Figure 5 This is a diagram illustrating the edge deployment and cloud-based federated update architecture of the scene template in this invention. Figure 6 , 7 This is a diagram of the main functional interface of the system of the present invention. Detailed Implementation

[0028] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0029] like Figure 1 As shown, the present invention provides a city-level multi-scenario intelligent traffic collaborative management and reliable simulation method based on reinforcement learning, comprising the following steps: Step S1: Collect city-level multi-scenario, multi-source traffic data and obtain a standardized dataset through preprocessing; In a specific embodiment provided here, a regional road network in the core urban area of ​​a city is selected as the implementation object, such as... Figure 2 As shown, the area includes 12 adjacent intersections, 3 main roads, 4 secondary roads, and several branch roads, covering typical traffic scenarios such as weekday morning and evening rush hours, large event departures, holidays, and local accident disturbances. The collected data comes from video detectors, geomagnetic detectors, radar detectors, and traffic signal control systems. The data time granularity is 5 seconds, the historical sample duration is 90 days, and the raw data volume is approximately 2.6TB.

[0030] The following section will explain the overall process of this invention by dynamically generating a regional-level cooperative traffic signal control scheme using this method: First, city-level multi-scenario, multi-source traffic data is collected and preprocessed, such as... Figure 3 As shown, step S1 includes the following sub-steps: S101. Based on the city-level multi-scenario traffic management needs, comprehensively collect core data, including but not limited to: road network basic data, real-time traffic flow data, traffic signal operation data, traffic participant behavior data, and scenario-related data.

[0031] In this embodiment, the core data specifically includes: basic road network data, including 12 intersections, 46 approach lanes and 44 exit lanes, road grades, connectivity, and spatial coordinates; real-time traffic flow data, including vehicle speed, flow rate, density, and dynamic trajectory. Taking a weekday evening rush hour scenario as an example, the average flow rate on the main road is 1520 veh / h, the average vehicle speed is 31.6 km / h, and the average queue length is 48m; traffic signal operation data, including the 4-phase timing scheme for each intersection, phase switching records, and an average cycle of 120s; participant behavior data, including following distance of motor vehicles, lane changing frequency, pedestrian crossing speed, and non-motorized vehicle weaving behavior; and scenario-related data, including the morning rush hour from 7:30 to 9:00, the number of people leaving a large event (approximately 23,000), the accident impact radius of approximately 450m, and information on rainy weather scenarios. S102. According to the data quality control standards, outlier removal is performed on the collected multi-source traffic data to obtain preliminary cleaned data. The outlier determination formula is as follows: Outlier(x, Scene, t) = Where Outlier(x, Scene, t) is the outlier determination result, where 1 represents an outlier and 0 represents normal data; x is the data sample value, μ Scene, t φ represents the average value of the data at the corresponding scenario and time point. Scene σ is the scene-adaptive threshold coefficient. Scene, t This represents the standard deviation of the data in the corresponding scenario. In this embodiment, taking the main road vehicle speed data at 18:07:35 during the evening rush hour on a weekday as an example, the sample value x collected at a certain moment is 51 km / h, and the mean value μ corresponding to the scene and time is... Scene, t =32 km / h, standard deviation σ Scene, t =6 km / h, scene adaptive threshold coefficient φ Scene =2.5, then we have: Outlier(51, Weekday Evening Rush Hour, 18:07:35) = >2.5*6 Since 19 > 15, Outlier(51, Weekday Evening Peak, 18:07:35) = 1, meaning this sample is determined to be an outlier and is removed. S103. Based on the constraints of temporal continuity, spatial proximity and scene similarity, perform spatiotemporal attention imputation of missing values ​​to obtain complete traffic data; In this embodiment, step S103 specifically involves: if the video detector at intersection I5 fails between 8:15:10 and 8:15:20, then the traffic flow changes and historical scenes from adjacent intersections I4 and I6 during the same period are used to complete the data. For example, within this 10-second time window, the measured traffic flow at intersection I4 is 1465 veh / h, the measured traffic flow at intersection I6 is 1510 veh / h, and the average traffic flow for historical similar weekday evening peak segments is 1490 veh / h. After spatiotemporal attention filling, the estimated traffic flow for intersection I5 within this time window is 1492 veh / h, thus obtaining complete traffic data. S104. Based on the spatiotemporal unification standard, perform spatiotemporal alignment on the complete data. The spatiotemporal confidence alignment formula is as follows: Φ i (t) = ω1*Δx i (t)+ω2*Δy i (t)+ω3*Δτ i(t)+ω4*(1-C i (t)) Where, Δx i (t) and Δy i (t) represents the spatial position deviation, Δτ i (t) represents the timestamp offset, C i (t) represents the credibility of the i-th type of data source at time t, and ω1 to ω4 are the alignment weights. i When (t) is less than the spatiotemporal alignment tolerance threshold, the corresponding data is determined to have completed spatiotemporal alignment; In this embodiment, taking the spatiotemporal alignment process of the second type of floating car trajectory data at 08:16:00 during the weekday morning rush hour as an example, the normalized spatial position deviation Δx2(08:16:00) = 0.10, Δy2(08:16:00) = 0.08, the normalized timestamp deviation Δτ2(08:16:00) = 0.05, the confidence level C2(08:16:00) = 0.92, and taking the alignment weight ω1 = ω2 = ω3 = ω4 = 0.25, then we have Φ2(08:16:00) = 0.25*0.10+0.25*0.08+0.25*0.05+0.25*(1-0.92) = 0.0775 If the spatiotemporal alignment tolerance threshold is set to 0.10, then 0.0775 < 0.10, therefore the data is determined to have completed spatiotemporal alignment. S105. Based on the dynamic topology construction rules, construct a dynamic adjacency matrix to obtain the road network traffic spatiotemporal graph. The formula for calculating matrix elements is: A ij (t) = Dist(i, j) -1 *exp(- )*(1+θ2*Corr ij (t)+θ3*RiskFlow ij (t)) Where Dist(i, j) is the physical distance between node i and node j, and ΔPhase ij (t) represents the signal phase difference between the two intersections at time t, τ travel,ij (t) represents the expected free-flow arrival time of the vehicle between the two nodes, Corr ij (t) represents the traffic flow correlation coefficient, RiskFlow ij (t) represents the conflict propagation influence coefficient, and θ1, θ2, and θ3 are adjustment parameters; In this embodiment, taking the adjacent intersections I3 and I4 in the main road corridor at 18:15:30 during the evening rush hour on a weekday as an example, the distance between the two nodes Dist(3, 4) = 320m, and the signal phase difference ΔPhase 34(18:15:30) = 18s, the expected free-flow arrival time τ of the vehicle between the two nodes. travel, 34 (18:15:30) = 20s, traffic flow correlation coefficient Corr 34 (18:15:30)=0.62, RiskFlow (Conflict Propagation Impact Coefficient) 34 Given (18:15:30) = 0.28, parameters θ1 = 10, θ2 = 0.5, θ3 = 0.4, the dynamic adjacency matrix element values ​​of intersection 3 and intersection 4 at 18:15:30 are: A 34 (18:15:30) = 1 / 320*exp(- )*(1+0.5*0.62+0.4*0.28) ≈ 0.00364 Similarly, by calculating the adjacency relationships between the 12 intersections in the region one by one, a 12*12 dynamic adjacency matrix can be obtained; S106. According to the feature extraction rules, the dynamic adjacency matrix is ​​input into the spatiotemporal graph representation learning network, and spatiotemporal graph feature fusion encoding is performed to obtain a low-dimensional standardized state matrix containing green light passage continuity information, conflict propagation and scene context information. In this embodiment, step S106 specifically involves: inputting the dynamic adjacency matrix obtained in step S105 into the spatiotemporal graph representation learning network, constructing a 64-dimensional state vector for each intersection node, and finally outputting a standardized state matrix with a dimension of 12*64, which includes green light continuity information, conflict propagation information, and scene context information.

[0032] The specific process of the spatiotemporal graph representation learning method in step S106 is as follows: (1) Based on the signal coordination requirements, the phase coordination relationship between adjacent intersections is quantified to obtain the phase coupling strength between nodes. The calculation formula is as follows: P ij (t) = max(0, 1- )*I green (t) Among them, P ij (t) represents the phase coupling strength between node i and node j at time t, T arrive,ij (t) represents the predicted arrival time of the target traffic flow released from upstream intersection i to downstream intersection j, where T is the arrival time. match,j (t) represents the phase matching time window at downstream intersection j, I green (t) is the phase indicator function; In this embodiment, taking adjacent intersections 3 and 4 in the main road corridor at 18:15:30 during the evening rush hour on a weekday as an example, according to the signal control system records, the phase difference between intersections 3 and 4 is 18s; according to the floating car trajectory and road segment travel time statistics, the predicted arrival time offset of a vehicle from intersection 3 to intersection 4 is 19s; according to the current signal timing scheme of intersection 4, the phase matching time window is 12s, and the target phase at this moment is green, therefore I green (18:15:30)=1, then the phase coupling strength between nodes is: P 34 (18:15:30) = max(0, 1- ) * 1 = 0.9167 (2) Jointly assess the queuing overflow risk and trajectory overlap between adjacent nodes to obtain the conflict propagation constraint strength C between nodes. ij (t), the calculation formula is: C ij (t) = ω c *exp(η )+(1-ω c *Overlap ij (t) Among them, C ij (t) represents the conflict propagation constraint strength between node i and node j at time t, L queue, j (t) represents the real-time queue length extending from downstream intersection j to road segment ij, L link, ij (t) represents the physical length of the road segment connecting node i and node j, Overlap ij (t) represents the degree of overlap in trajectory occupancy between motor vehicles and non-motor vehicles or pedestrians on road segment ij, ω c η is the weighting coefficient between queue overflow risk and the degree of overlap in trajectory occupancy, and η is the queue overflow sensitivity parameter. In this embodiment, taking the queue propagation situation at intersection 4 extending to road segment 3-4 at 18:15:30 as an example, the real-time queue length L queue,4 (18:15:30) = 60m, physical length of road segment L link,34 (18:15:30) = 300m, Trajectory overlap 34 (18:15:30) = 0.35, queuing overflow risk weighting coefficient ω c =0.6, queue overflow sensitivity parameter η=1.2, then the constraint strength for conflict propagation between nodes: C 34 (18:15:30) = 0.6*exp(1.2* ) + (1 - 0.6) * 0.35 = 0.9026 (3) Based on the phase coupling strength and the conflict propagation constraint strength, the dynamic adjacency matrix constructed in step S105 is jointly modified to obtain the joint encoding weight, calculated using the following formula: W ij (t) = A ij (t)* Among them, W ij (t) represents the joint encoding weights of node i and node j at time t, A ij (t) represents the elements of the dynamic adjacency matrix; In this embodiment, based on the dynamic adjacency matrix element A obtained in step S105... 34 (18:15:30)=0.00364, the phase coupling strength P obtained in step S106(1) 34 (18:15:30)=0.9167, and the conflict propagation constraint strength C obtained in step S106(2) 34 If (18:15:30) = 0.9026, then the joint coding weight is: W 34 (18:15:30) = 0.00364* ≈ 0.00175 (4) Normalize the joint coding weights to obtain the normalized propagation weights. The calculation formula is as follows: ij (t) = in, ij (t) represents the normalized node propagation weight, and N(i) represents the set of neighboring nodes of node i. (5) The traffic observation features of neighboring nodes are weighted and aggregated using normalized propagation weights, and the participant behavior features and scene embedding features of the current node are jointly encoded to obtain the structured feature representation of the node. (6) Normalize and map the structured feature representation of the nodes to obtain a standardized state matrix that simultaneously contains signal coordination information, conflict risk information, and scene context information. The calculation formula is as follows: S i (t) = Norm(H i (t), M i (t), E i (t)) Among them, S i H(t) is the standardized state vector of node i at time t. i(t) represents the traffic structure characteristics of the node, M i (t) represents the behavioral characteristics of node participants, E i (t) represents the node scene embedding feature, and Norm() is the normalization mapping function.

[0033] S2. Establish a simulation rules and model library adapted to mixed traffic flow, and construct a road network efficiency quantitative analysis system; In a specific embodiment provided here, step S2 is mainly used to construct simulation rules, conflict risk models, dynamic decision-making mechanisms, micro-behavior models, congestion propagation models, and road network efficiency quantitative analysis systems that are adapted to the characteristics of mixed traffic flow, thereby providing a reliable simulation foundation for subsequent reinforcement learning training and scheme verification.

[0034] Step S2 includes the following sub-steps: S201. Based on the core characteristics of mixed traffic flow, formulate core simulation rules for mixed traffic scenarios involving motor vehicles, non-motor vehicles, and pedestrians. For scenarios involving mixed traffic of motor vehicles and non-motor vehicles at large intersections, pedestrian crossings, bus stops, and temporary pick-up and drop-off of ride-hailing vehicles, formulate conflict avoidance mechanisms based on dynamic game theory and trajectory prediction to obtain a set of simulation rules adapted to mixed traffic flow. S202. Based on the behavioral characteristics of traffic participants, calculate the conflict risk for any two traffic participants at time t to obtain the conflict risk value, and the conflict risk Risk between node i and node j. ij The formula for calculating (t) is: Risk ij (t) = [α*O ij (t)+β*G ij (t)]*(1+γ*B ij (t)+δ*P ij (t)) + ζ*H ij (t) Among them, O ij (t) represents the overlap of the predicted trajectory, G ij (t) represents the conflict intensity of the prevailing game, B ij (t) represents the intensity of encroachment on the boundary between motor vehicles and non-motor vehicles, P ij (t) represents the pedestrian priority disturbance intensity, H ij (t) represents the historical high-risk behavior memory item, and α, β, γ, δ, and ζ represent the risk fusion weights; In this embodiment, step S202 specifically involves: taking the interaction process of motor vehicle number 1 and non-motor vehicle number 2 within the conflict zone of intersection 6 at 18:12:20 as an example, predicting the trajectory overlap degree O. 12 (18:12:20) = 0.42, the conflict intensity G in the prevailing game. 12(18:12:20)=0.38, Intensity of encroachment on the boundary between motor vehicles and non-motor vehicles B 12 (18:12:20)=0.25, pedestrian priority disturbance intensity P 12 (18:12:20) = 0.30, historical high-risk behavior memory item H 12 (18:12:20)=0.15, given the risk fusion weights α=0.8, β=0.7, γ=0.4, δ=0.5, ζ=0.3, then the conflict risk value at this moment is: Risk 12 (18:12:20) = [0.8*0.42+0.7*0.38]*(1+0.4*0.25+0.5*0.3)+0.3*0.15 =0.7975 S203. Conduct a joint assessment of the traffic benefits, waiting losses, conflict risks, local queuing disturbances and congestion shock wave spread effects of traffic participants after performing each candidate action, and dynamically adjust the comprehensive benefits of each candidate action according to the risk level under the current mixed traffic scenario to obtain the benefit assessment results corresponding to each candidate action. S204. Based on the benefit evaluation results of each candidate action, select the action with the largest comprehensive benefit as the action to be executed by the traffic participant at time t, and obtain the dynamic decision result of the traffic participant. S205. Based on the behavioral characteristics of traffic participants, formulate an appropriate micro-behavioral model to obtain a micro-behavioral model that adapts to mixed traffic flow. The microscopic behavior model in step S205 specifically includes: (1) Vehicle following model considering driving emotions: The calculation formula is as follows: a n (t) = a max *[1- ]*exp(-k ) Among them, a n (t) represents the acceleration of the nth vehicle at time t, a max For maximum acceleration, v n (t) represents the current velocity, v des (t) represents the desired velocity, s ∗ (t) represents the dynamic safety following distance, s n (t) represents the actual following distance, k represents the emotional sensitivity coefficient, and L queue (t) represents the current queue length, L max This represents the maximum capacity of the intersection. (2) Phase Coordination Lane Changing Decision Model: Based on the difference in traffic utility between the target lane and the current lane, the risk cost of lane changing, and the degree of matching between the vehicle and the target green light phase when it arrives at the downstream intersection, the probability of successful lane changing is dynamically evaluated to obtain a lane changing decision result that meets the requirements of local safety and downstream signal coordination. (3) Non-motorized vehicle weaving model: Based on the operating characteristics of non-motorized vehicles under mixed traffic conditions, the non-motorized vehicle speed, lateral interference intensity, and available clearance length are jointly corrected to obtain the dynamic following distance of the non-motorized vehicle. The formula for calculating the dynamic following distance of the non-motorized vehicle is as follows: S non-motor (t) = s0+0.5*λ1*v non-motor (t)+Interside(t)-Gap front (t) Among them, S non-motor (t) represents the dynamic following distance of the non-motorized vehicle at time t, s0 is the basic safety distance, and v non-motor (t) represents the speed of the non-motorized vehicle, Interside(t) represents the intensity of the lateral disturbance, and Gap front (t) represents the length of the available gap ahead, and λ1 is the non-motorized vehicle behavior correction coefficient; (4) Pedestrian crossing group model: Based on the characteristics of group crossing behavior, the speed of individual crossing, group size, degree of interference from oncoming vehicles and cross-section occupancy effect are jointly corrected to obtain the group crossing speed and group crossing time, and thus obtain the characteristics of group crossing behavior. (5) Bus Priority Control Model: Based on the bus's operating status when approaching the intersection, the distance of the bus from the intersection, the bus's passenger load level, and the current traffic light phase are jointly determined to obtain the bus priority triggering result. The red light shortening time is dynamically adjusted based on the urgency of bus priority and downstream queuing pressure. The formula for determining the bus priority triggering condition is as follows: Priority bus (t) Among them, Priority bus (t) is the bus priority trigger signal, distance(t) is the distance of the bus from the intersection, Load bus (t) represents the passenger load level of public transport, D th L is the distance threshold. th The passenger capacity threshold is given, and Phase(t) is the current traffic light phase. (6) Ride-hailing passenger pick-up and drop-off model: Based on the characteristics of ride-hailing temporary stops, the number of passengers picking up and dropping off, the degree of roadside interference, the stopping scenario, the duration of passenger pick-up and drop-off on the road, the intensity of road occupation, and the transfer capacity of adjacent lanes are jointly corrected to obtain the duration of passenger pick-up and drop-off and the capacity of affected lanes, and thus obtain the characteristics of the impact of ride-hailing temporary stops.

[0035] S206. Based on scenarios such as mixed traffic of motorized and non-motorized vehicles at large intersections, bus stops, and temporary passenger pick-up and drop-off for ride-hailing vehicles, a joint correction is performed on the uncorrected congestion propagation velocity to obtain the congestion propagation velocity. The calculation formula is as follows: v' w (t) = v w (t)*[1-β1 ]*[1-β2 ]*[1+β3G release (t)] Among them, v' w (t) represents the corrected congestion propagation speed, v w (t) represents the original congestion propagation speed without considering the effects of non-motorized vehicle infiltration, temporary lane occupation, and signal release capacity. non-motor (t) represents the real-time sensed non-motorized vehicle infiltration flow, W lane W is the total width of the mixed-traffic lane for motor vehicles and non-motor vehicles. occ (t) represents the equivalent physical width for parked vehicles or temporary road occupancy, L ref For reference queue length, G release (t) represents the effective green light release ratio within the current signal cycle, and β1, β2, and β3 are correction coefficients; In this embodiment, step S206 specifically involves: taking the mixed traffic section where the main road and the branch road intersect at 22:05:30 after a large event as an example, without correcting the congestion propagation speed v w (22:05:30) = 5.2 m / s, real-time sensed non-motorized vehicle infiltration flow Q non-motor (22:05:30) = 180 pcu / m, total width W of mixed traffic lanes lane =7.5m, the equivalent road width W occupied by parked vehicles occ (22:05:30) = 2.0m, current queue length L queue (22:05:30) = 90m, reference queue length L ref =150m, the proportion of green lights effectively released during the current signal cycle G release (22:05:30)=0.55, correction coefficients β1=0.004, β2=0.25, β3=0.3, then the corrected congestion propagation speed is: v w (22:05:30) = 5.2*[1-0.004* ]*[1-0.25* ]*[1+0.3*0.55] ≈ 4.47m / s S207. Based on the corrected congestion propagation speed, the congestion propagation impact boundary and congestion dissipation time are dynamically evaluated to obtain the congestion propagation evolution results. S208. Based on the principle of modular integration, the core simulation rules, dynamic decision-making results of traffic participants, micro-behavioral models, and mixed traffic flow congestion propagation models are integrated to obtain a dynamic model library that adapts to different traffic scenarios. S209. Quantitatively analyze traffic efficiency, congestion control, safety risks, and collaborative control effectiveness, construct a road network efficiency quantitative analysis system, and obtain road network efficiency evaluation results for simulation assessment and subsequent reinforcement learning training.

[0036] The road network efficiency quantitative analysis system in step S209 specifically includes: (1) Based on the traffic efficiency calculation method, the average delay time of intersections, the average travel speed of road segments and the utilization rate of road network capacity are calculated to obtain the traffic efficiency index. In this embodiment, the traffic efficiency calculation method adopts conventional traffic engineering statistical methods in the field. The average delay time at intersections is calculated based on the average controlled delay of vehicles within the assessment period. The average travel speed of road segments is calculated based on the road segment length and the average travel time. The road network capacity utilization rate is calculated based on the ratio of actual traffic flow to corresponding capacity. (2) Calculate the congestion duration, congestion range and delay index, and combine the congestion propagation model of mixed traffic flow in step S206 to identify the congestion propagation and dissipation process to obtain congestion control indicators. (3) Based on the safety risk assessment method, the number of intersection conflicts, the frequency of rapid acceleration and the frequency of rapid deceleration are calculated to obtain the safety risk index; In this embodiment, the safety risk assessment method is a comprehensive evaluation method based on the statistics of traffic conflict events and abnormal driving behavior. The number of intersection conflicts, the frequency of rapid acceleration and the frequency of rapid deceleration are standardized and then fused according to preset risk weights to obtain safety risk indicators. (4) Using the collaborative control evaluation method, the phase coordination degree of adjacent intersections, the continuity of green light passage and the regional collaborative release effect are calculated to obtain the collaborative control index; (5) The weighted fusion evaluation method is adopted to weight and fuse traffic efficiency indicators, congestion control indicators, safety risk indicators and collaborative control indicators to obtain the comprehensive evaluation value of road network efficiency.

[0037] In this embodiment, the weighted fusion evaluation method first standardizes the traffic efficiency index, congestion control index, safety risk index, and collaborative control index, and then sums the weights of each index according to preset weights to obtain the comprehensive evaluation value of road network efficiency. The preset weights are set based on the calibration results of historical operation data, expert experience, or regional traffic control targets, and the sum of each weight is 1.

[0038] S3. Construct a multi-agent spatiotemporal attention reinforcement learning model, train the agents to generate a regional-level collaborative traffic signal control scheme, and input it into the simulation platform to complete the credible verification. In this embodiment, as Figure 4 As shown, step S3 specifically involves defining the 12 intersections within the area as 12 control agents, with each agent corresponding to one intersection. The state input dimension is 64, and the action space includes control actions such as phase switching, green light extension, green light shortening, and phase difference adjustment. Each agent shares the network structure but corresponds to different local state inputs, thus forming a regional multi-agent collaborative control architecture.

[0039] Step S3 includes the following sub-steps: S301. Based on the requirements of multi-agent collaborative management and control, construct a multi-agent reinforcement learning model, with each intersection corresponding to an agent, to obtain a regional multi-agent collaborative control model. S302. Based on the traffic propagation relationship, conflict propagation relationship, scene consistency, phase coupling relationship, and queuing pressure difference between intersections, the neighborhood information of each agent is weighted and aggregated to obtain a cooperative state representation. The aggregation weight calculation formula is as follows: ψ ij (t) = Where, ψ ij (t) represents the aggregation weight of node i to node j at time t, Corr ij (t) represents the traffic flow association strength, Phase ij (t) represents the phase coupling strength, Scene ij (t) represents the scene consistency strength, Gap ij (t) represents the queuing pressure difference between nodes; In this embodiment, step S302 specifically involves: taking the aggregation weight calculation of node 4 for its neighboring node 3 at 18:15:30 as an example, the traffic flow association strength Corr... 43 (18:15:30)=0.62, Phase coupling strength (Phase) 43 (18:15:30) = 0.73, Scene consistency strength 43 (18:15:30)=0.58, Conflict propagation coupling Risk 43(18:15:30) = 0.41, Gap in queuing pressure between nodes 43 (18:15:30)=0.25, then the aggregate weight of node 4 with respect to its neighbor node 3 is obtained: ψ 43 (18: 15: 30) = ≈ 1.163 S303. Based on the aggregation weight, the state information of neighboring nodes is weighted and fused, and combined with the local state information of the current node, the collaborative state representation of the intersection agent is obtained: ij (t) = h i (t)+ in, ij (t) represents the cooperative state of intersection i at time t, h i (t) is the local state latent vector of intersection i, h j (t) is the local state latent vector of intersection j; In this embodiment, step S303 specifically involves: based on the aggregation weight ψ obtained in step S302... 43 (18:15:30) The state information of neighboring nodes is weighted and fused, and combined with the local state information of node 4 at 18:15:30, to obtain the cooperative state representation of intersection 4 at 18:15:30. 43 (18: 15: 30), thereby enhancing the joint perception capabilities of queuing spillover, phase coordination and conflict propagation at adjacent intersections; S304. According to the model training specifications, initialize the reinforcement learning model parameters, and prioritize the use of samples according to the empirical sample scheduling rules. Prioritize the training of samples with large time difference errors, high collision risks, and obvious scene boundary features. Use the average relative change rate of reward as the convergence criterion. When the average relative change rate of reward for a preset number of consecutive iterations is not greater than the preset threshold, stop training to obtain a training process that meets the convergence requirements. In this embodiment, step S304 specifically involves initializing the reinforcement learning training parameters as follows: the experience replay library capacity is 225,428 samples, the batch training sample size is 128, the learning rate is 0.0005, the discount factor is 0.95, the maximum number of training rounds is 50,000 rounds, and the average relative change rate of return over 200 consecutive rounds is less than 10%. -4 When the model is considered to have converged; S305. Construct a reliable twin constraint reward function based on the training objective, and train the multi-agent spatiotemporal attention reinforcement learning model to obtain the trained collaborative control agent. The reward function satisfies: R i (t) = Among them, R i (t) represents the reward value of intersection i at time t, ΔEff i (t) represents the efficiency improvement, Cred i (t) represents the consistency score between the digital twin simulation results and the measured traffic conditions, Coord i (t) represents the regional cooperative gain, and Delay i (t) represents the delay cost, Queue i (t) represents the queuing cost, Risk. i (t) represents the risk cost; The multi-agent spatiotemporal attention reinforcement learning model training in step S305 specifically includes: (1) Based on the empirical sample scheduling rules, the sample time difference error, minimum collision time, scene scarcity, and scene switching boundary characteristics are jointly evaluated to obtain the sample sampling priority. The calculation formula is as follows: Priority(s, a) = (|TD_Error|+ )*(1+Rare(s)+Shift(s)) Where TD_Error is the temporal difference error of the sample, TTC(s, a) is the minimum collision time after performing action a in state s, Rare(s) is the scarce scene indicator, Shift(s) is the scene switching boundary indicator, and ε is a minimal constant to prevent the denominator from being zero. In this embodiment, taking training sample number 257 in the experience playback library as an example, this sample corresponds to the control state of intersection 7 at 18:21:10 during the evening rush hour on a weekday, with a time difference error TD_Error. 257 =0.42, in state s 257 Minimum collision time TTC(s) after executing action a3 257 , a3)=2.5s, Rare(s) is the scene scarcity indicator 257 )=1, Scene transition boundary indicator Shift(s) 257 Given that ) = 0.5 and the minimum constant ε = 0.01, the sample sampling priority is: Priority(s 257 , a3) = (0.42+ )*(1+1+0.5) = 2.046 This means that the sample has a higher training call priority; (2) Based on the sampling priority, the samples in the experience playback library are sorted and stratified for retention, so that high-risk samples, scarce scene samples and scene boundary samples can obtain a higher retention level. In this embodiment, in the experience replay library with a capacity of 225,428 samples, the top 20% of samples are classified as high-priority retention layer, the middle 50% of samples are classified as regular retention layer, and the remaining samples are classified as low-priority layer. In subsequent training, samples from the high-priority retention layer are extracted first. (3) Allocate sample sampling probabilities according to sample sampling priority, and prioritize the use of experience replay samples so that high-priority samples can obtain a higher training call frequency. (4) Enhance the input traffic flow state by applying scene disturbances to obtain the enhanced traffic flow characteristics. The calculation formula is as follows: Q aug (t) = Q(t)*(1+Bias(t)+Resist(t)) Among them, Q aug (t) represents the enhanced flow characteristics, Q(t) represents the original flow characteristics, Bias(t) represents the scene offset term, and Resist(t) represents the weather impedance term; (5) Based on the topology regularization rule, the connections between nodes with weak correlations in the dynamic traffic graph are randomly suppressed to obtain the topology-constrained training graph. The formula for calculating the edge drop probability is: p ij (t) = p0(1- ) Where, p ij (t) represents the edge drop probability between node i and node j at time t, p0 is the basic drop rate, max(A(t)) is the maximum weight value in the dynamic adjacency matrix at time t, and m is a minimal constant to prevent the denominator from being zero.

[0040] In this embodiment, taking the connection between node 3 and node 4 at 18:15:30 during the evening rush hour on a weekday as an example, the dynamic adjacency matrix element A 34 (18:15:30) = 0.00364, the maximum weight value in the dynamic adjacency matrix at this moment is max(A(18:15:30)) = 0.00820, the basic discard rate is p0 = 0.2, and the minimum constant is m = 0.001. Therefore, the edge discard probability is: p 34 (18:15:30) = 0.2*(1- ) ≈ 0.121 That is, the edge between node 3 and node 4 has a random inhibition probability of about 0.121 in this training round, thereby reducing the risk of overfitting the model to local strong connection structures; S306. Based on the standardized state matrix obtained in step S1, the trained cooperative control agent is input into the regional road network for interactive decision-making, outputs the optimal signal control action, generates a regional-level cooperative traffic signal control scheme covering multiple adjacent intersections, and optimizes the phase difference coordination logic of adjacent intersections to obtain the initial control scheme. S307. According to the trusted verification requirements, the initial control scheme is input into the digital twin-based simulation platform to complete the trusted verification of the scheme. The simulation parameter configuration is expressed as follows: SimParam = {Step, RecordFreq, SceneType, λ sim} Where Step is the simulation step size, RecordFreq is the data recording frequency, SceneType is the scene type defined in step S2, and λ sim For simulation accuracy coefficients; S308. According to the simulation data recording rules, record the delays, speeds, congestion status, and implementation effects of the scheme during the simulation process to obtain the simulation verification results. The simulation data record is expressed as follows: SimData = {Delay(t), Speed(t), Congest(t), PhaseExec(t)} Where Delay(t) is the intersection delay time at time t, Speed(t) is the road segment speed at time t, Congest(t) is the congestion status at time t, and PhaseExec(t) is the signal control scheme execution effect at time t.

[0041] In this embodiment, steps S306 to S308 specifically involve: inputting the standardized state matrix obtained in step S1 into the trained cooperative control agent, and outputting regional-level cooperative traffic signal control actions. For example, in a weekday evening rush hour scenario at 18:15:30, a cooperative control scheme is given for the main corridor intersection: "the phase difference is adjusted from 20s to 18s, the green light in the main direction is extended by 6s, and the green light in the secondary direction is shortened by 4s". Subsequently, the scheme is input into a digital twin simulation platform for verification. The simulation parameters are: Step=1s, RecordFreq=5s, SceneType=weekday evening rush hour, λ sim =0.92. Simulation results show that the average delay in the region is reduced by 12%, the average speed in the main corridor is increased by 8%, and the number of sudden deceleration events at key conflict points is reduced by 15%.

[0042] S4. Iteratively optimize the scheme and model based on the simulation results to form a scenario-adaptive control template; In this embodiment, step S4 specifically involves: based on the credible verification results of step S3, jointly optimizing the signal control parameters, simulation parameters, and scenario adaptation parameters to form a control template suitable for different urban traffic scenarios. If the simulation verification results do not meet the preset credibility, performance, or security requirements, the iterative optimization process is initiated.

[0043] Step S4 includes the following sub-steps: S401. Based on the trusted verification result of step S3, set the optimization triggering conditions and obtain the optimization triggering result. The formula for determining the triggering conditions is: Trigger Among them, Trigger is the optimization trigger signal, 1 indicates that optimization is started and 0 indicates that it is not started; Similarity is the matching degree between simulation data and actual collected data; DelayGain is the delay optimization magnitude; SafeScore is the extreme scenario adaptability score; θ1 is the simulation matching degree trigger threshold; θ2 is the delay optimization trigger threshold; and θ3 is the safety adaptation trigger threshold. In this embodiment, step S401 specifically involves: Assuming that after a simulation verification in the scenario of a large event ending at 22:18:00, the similarity between the simulation data and the actual collected data is Similarity=0.82, the delay optimization magnitude DelayGain=0.11, the extreme scenario adaptability score SafeScore=0.88, and the thresholds are θ1=0.85, θ2=0.10, and θ3=0.90 respectively, then since Similarity<θ1 and SafeScore<θ3, we have: Trigger = 1 This initiates the optimization process; S402. Based on the optimized triggering results, the signal control parameters and simulation parameters are adjusted in a targeted manner to obtain the adjusted parameters. The adjustment range includes at least the phase difference, green ratio, signal period, conflict risk threshold and congestion propagation model correction coefficient. S403. Based on the adjusted parameters, substitute them into the multi-agent spatiotemporal attention reinforcement learning model of step S3, and jointly update the control parameter correction, simulation model correction, scene adaptation correction and risk control correction to obtain the optimized joint parameter set. S404. Based on the preset evaluation criteria, repeat the training and simulation verification, iteratively correct the joint parameter set, and obtain an optimized solution that meets the requirements of credibility, performance and security. S405. Based on the scenario classification rules, the optimization scheme is solidified and classified into daily commuting, large-scale events, holidays and emergency scenarios to obtain scenario adaptation and control templates.

[0044] In this embodiment, steps S402 to S405 are as follows: After determining the trigger optimization in step S401, the phase difference of the main corridor intersection is adjusted from 20s to 18s, the green light ratio is adjusted from 0.42 to 0.46, the signal period is adjusted from 120s to 110s, and the congestion propagation model correction coefficient β2 is finely adjusted from 0.25 to 0.22. Then, the reinforcement learning model is retrained and the simulation verification is repeated. After 3 rounds of iteration, the regional average delay is further reduced from the original 53s to 49s, the simulation matching degree is improved to 0.88, and the extreme scenario adaptability score is improved to 0.92, reaching the preset standard. The optimized solution is solidified into 4 types of templates according to the scenario, namely: "Weekday morning rush hour template", "Large event dispersal template", "Rainy day holiday template" and "Local accident emergency template", and saved to the scenario adaptation control template library.

[0045] S5. Deploy the optimized template to edge computing nodes to achieve rapid dynamic response and continuous federation in the cloud; In this embodiment, step S5 specifically involves: Figure 5 As shown, step S5 mainly includes edge-side deployment of the scene adaptation and control template, real-time status monitoring, template fast matching, template switching triggering, long-term effect evaluation, and continuous cloud federation updates. The scene adaptation and control template formed in step S4 is deployed to the edge-side signal controller and combined with the cloud federation aggregation mechanism to form a continuous optimization closed loop. Step S5 includes the following sub-steps: S501. Based on the actual deployment requirements of the traffic control system, deploy the scenario adaptation control template obtained in step S4 to the edge-side signal controller to obtain the edge-side executable template set. S502. Based on the real-time traffic flow collection mechanism, the road network operation status is continuously monitored to obtain the road network status monitoring results; S503. Based on the road network status monitoring results, quickly match the scenario-adaptive control templates to obtain the candidate template scoring results; S504. Based on the candidate template scoring results, select the template with the highest comprehensive score as the template to be executed; S505. Based on the traffic mutation rate, equipment status and conflict risk change rate, edge cases are identified to obtain template switching trigger results; In this embodiment, steps S501 to S505 are as follows: Three edge computing nodes are selected in the regional road network to control 12 intersections respectively. Each edge node can manage four adjacent intersections. When the regional traffic flow mutation rate reaches 23%, the queue growth rate reaches 18%, and the critical intersection conflict risk change rate reaches 0.27 during the period from 22:10:00 to 22:15:00 after a large event ends, the edge side automatically switches the current control template from "weekday commuting template" to "large event end template" according to the candidate template scoring results, and completes the switch within two signal cycles. S506. Based on the long-term optimization rules, evaluate the effectiveness of the deployed solutions to obtain a solution effectiveness score. The calculation formula is as follows: E = Where E is the scheme effectiveness score, M is the number of samples in the evaluation period, and ΔDelay m Let ΔSpeed ​​be the delay reduction rate for the m-th sample. m Let ΔCongest be the velocity increase rate of the m-th sample. m Let w1, w2, and w3 be the congestion relief rate for the m-th sample, and w1, w2, and w3 be the weighting coefficients. S507. Based on edge cases and edge-side incremental parameters, aggregate parameter increments from various regions in the cloud, and combine them with scarce scenario correction information, road network structure change information, and event disturbance correction information to continuously update the control template in a federated manner, thereby obtaining the updated scenario template. S508. Based on the updated scenario template, continuously revise the traffic control strategy in the actual road network to obtain a continuous adaptive learning result that adapts to the evolution of the actual road network.

[0046] In this embodiment, steps S507 and S508 are specifically as follows: the incremental parameters generated by the three edge nodes in the large-scale event dispersal scenario are uploaded to the cloud. The cloud aggregates the parameters of each region and updates the "large-scale event dispersal template" by combining scarce scenario correction information, road network structure change information and event disturbance correction information. The updated template is then sent back to the edge nodes so that it can be directly called in subsequent similar scenarios, thereby obtaining a continuous adaptive learning result that adapts to the actual road network evolution.

[0047] In summary, by applying this invention to multi-intersection road networks in urban core areas, regional collaborative traffic signal control schemes can be dynamically generated under various scenarios, including daily peak hours, short-term traffic congestion after large events, holiday tidal travel, and localized accident disturbances. Reliable verification is then achieved through a digital twin simulation platform. Compared to traditional fixed timing methods or single-point adaptive control methods, this invention can more effectively improve the traffic efficiency of regional road networks, reduce the risk of congestion spread and conflict, and enhance the real-time response capability, reliability, and continuous adaptive optimization capability of the control strategy in complex multi-scenario traffic environments.

[0048] It should be recognized that embodiments of the present invention can be implemented or carried out by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable storage medium. The method can be implemented using standard programming techniques—including implementation in a computer program on a non-transitory computer-readable storage medium configured to allow the computer to operate in a specific and predefined manner—according to the methods and drawings described in the specific embodiments. Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with the computer system. However, if desired, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. Furthermore, for this purpose, the program can run on a programmed application-specific integrated circuit (ASIC).

[0049] Furthermore, the procedures described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by the context. The procedures described herein (or variations and / or combinations thereof) may be executed under the control of one or more computer systems configured with executable instructions, and may be implemented by hardware or a combination thereof as code (e.g., executable instructions, one or more computer programs, or one or more applications) that commonly executes on one or more processors. The computer program comprises a plurality of instructions executable by one or more processors.

[0050] Furthermore, the method can be implemented in any suitable type of computing platform, including but not limited to personal computers, minicomputers, master models, workstations, networked or distributed computing environments, standalone or integrated computer platforms, or communication with charged particle tools or other imaging devices. Aspects of the invention can be implemented as machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optical read and / or write storage medium, RAM, ROM, etc., such that it is readable by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the processes described herein. Furthermore, the machine-readable code, or portions thereof, can be transmitted via wired or wireless networks. The invention described herein includes these and other different types of non-transitory computer-readable storage media when such media comprises instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor. When programmed according to the methods and techniques described in the invention, the invention also includes the computer itself.

[0051] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A city-level multi-scenario intelligent traffic collaborative management and reliable simulation method based on reinforcement learning, characterized in that, Includes the following steps: S1. Collect city-level multi-scenario, multi-source traffic data and obtain a standardized dataset through preprocessing; S2. Establish a simulation rules and model library adapted to mixed traffic flow, and construct a road network efficiency quantitative analysis system; S3. Construct a multi-agent spatiotemporal attention reinforcement learning model, train the agents to generate a regional-level collaborative traffic signal control scheme, and input it into the simulation platform to complete the credible verification. S4. Iteratively optimize the scheme and model based on the simulation results to form a scenario-adaptive control template; S5. Deploy the optimized template to edge computing nodes to achieve rapid dynamic response and continuous federation in the cloud.

2. The city-level multi-scenario intelligent traffic collaborative management and reliable simulation method based on reinforcement learning according to claim 1, characterized in that, Step S1 includes the following sub-steps: S101. Based on the city-level multi-scenario traffic management needs, comprehensively collect core data, including but not limited to: road network basic data, real-time traffic flow data, traffic signal operation data, traffic participant behavior data, and scenario-related data. Among them, road network basic data includes core road attributes and spatial coordinate information; real-time traffic flow data includes key vehicle-related indicators collected through static and mobile sensing devices; traffic signal operation data includes real-time and historical data of traffic lights at each intersection; traffic participant behavior data includes key data related to drivers and pedestrians; and scenario-related data includes information on daily peak hour distribution, number of participants in large-scale events, and the impact range of sudden accidents. S102. According to the data quality control standards, outlier removal is performed on the collected multi-source traffic data to obtain preliminary cleaned data. The outlier determination formula is as follows: Outlier(x, Scene, t) = Where Outlier(x, Scene, t) is the outlier determination result, where 1 represents an outlier and 0 represents normal data; x is the data sample value, μ Scene, t φ represents the average data for the corresponding scenario and time period. Scene σ is the scene-adaptive threshold coefficient. Scene, t This represents the standard deviation of the data in the corresponding scenario. S103. Based on the constraints of temporal continuity, spatial proximity and scene similarity, perform spatiotemporal attention imputation of missing values ​​to obtain complete traffic data; S104. Based on the spatiotemporal unification standard, perform spatiotemporal alignment on the complete data. The spatiotemporal confidence alignment formula is as follows: Where, Δx i (t) and Δy i (t) represents the spatial position deviation, Δτ i (t) represents the timestamp offset, C i (t) represents the credibility of the i-th type of data source at time t, and ω1 to ω4 are the alignment weights. i When (t) is less than the spatiotemporal alignment tolerance threshold, the corresponding data is determined to have completed spatiotemporal alignment; S105. Based on the dynamic topology construction rules, construct a dynamic adjacency matrix to obtain the road network traffic spatiotemporal graph. The formula for calculating matrix elements is: Where Dist(i, j) is the physical distance between node i and node j, and ΔPhase ij (t) represents the signal phase difference between the two intersections at time t, τ travel,ij (t) represents the expected free-flow arrival time of the vehicle between the two nodes, Corr ij (t) represents the traffic flow correlation coefficient, RiskFlow ij (t) represents the conflict propagation influence coefficient, and θ1, θ2, and θ3 are adjustment parameters; S106. According to the feature extraction rules, the dynamic adjacency matrix is ​​input into the spatiotemporal graph representation learning network, and spatiotemporal graph feature fusion encoding is performed to obtain a low-dimensional standardized state matrix that contains green light continuity information, conflict propagation and scene context information.

3. The city-level multi-scenario intelligent traffic collaborative management and reliable simulation method based on reinforcement learning according to claim 2, characterized in that, In step S106, the specific process of spatiotemporal graph representation learning is as follows: (1) Based on the signal coordination requirements, the phase coordination relationship between adjacent intersections is quantified to obtain the phase coupling strength between nodes. The calculation formula is as follows: Among them, P ij (t) represents the phase coupling strength between node i and node j at time t, T arrive,ij (t) represents the predicted arrival time of the target traffic flow released from upstream intersection i to downstream intersection j, where T is the arrival time. match,j (t) represents the phase matching time window at downstream intersection j, I green (t) is the phase indicator function; (2) Jointly assess the queuing overflow risk and trajectory overlap between adjacent nodes to obtain the conflict propagation constraint strength C between nodes. ij (t), the calculation formula is: Among them, C ij (t) represents the conflict propagation constraint strength between node i and node j at time t, L queue, j (t) represents the real-time queue length extending from downstream intersection j to road segment ij, L link, ij (t) represents the physical length of the road segment connecting node i and node j, Overlap ij (t) represents the degree of overlap in trajectory occupancy between motor vehicles and non-motor vehicles or pedestrians on road segment ij, ω c η is the weighting coefficient between queue overflow risk and the degree of overlap in trajectory occupancy, and η is the queue overflow sensitivity parameter. (3) Based on the phase coupling strength and the conflict propagation constraint strength, the dynamic adjacency matrix constructed in step S105 is jointly modified to obtain the joint encoding weight, calculated using the following formula: Among them, W ij (t) represents the joint encoding weights of node i and node j at time t, A ij (t) represents the elements of the dynamic adjacency matrix; (4) Normalize the joint coding weights to obtain the normalized propagation weights. The calculation formula is as follows: ij (t) = in, ij (t) represents the normalized node propagation weight, and N(i) represents the set of neighboring nodes of node i. (5) The traffic observation features of neighboring nodes are weighted and aggregated using normalized propagation weights, and the participant behavior features and scene embedding features of the current node are jointly encoded to obtain the structured feature representation of the node. (6) Normalize and map the structured feature representation of the nodes to obtain a standardized state matrix that simultaneously contains signal coordination information, conflict risk information, and scene context information. The calculation formula is as follows: S i (t) = Norm(H i (t), M i (t), E i (t)) Among them, S i H(t) is the standardized state vector of node i at time t. i (t) represents the traffic structure characteristics of the node, M i (t) represents the behavioral characteristics of node participants, E i (t) represents the node scene embedding feature, and Norm() is the normalization mapping function.

4. The city-level multi-scenario intelligent traffic collaborative management and reliable simulation method based on reinforcement learning according to claim 1, characterized in that, Step S2 includes the following sub-steps: S201. Based on the core characteristics of mixed traffic flow, formulate core simulation rules for mixed traffic scenarios involving motor vehicles, non-motor vehicles, and pedestrians. For scenarios involving mixed traffic of motor vehicles and non-motor vehicles at large intersections, pedestrian crossings, bus stops, and temporary pick-up and drop-off of ride-hailing vehicles, formulate conflict avoidance mechanisms based on dynamic game theory and trajectory prediction to obtain a set of simulation rules adapted to mixed traffic flow. S202. Based on the behavioral characteristics of traffic participants, calculate the conflict risk for any two traffic participants at time t to obtain the conflict risk value, and the conflict risk Risk between node i and node j. ij The formula for calculating (t) is: Among them, O ij (t) represents the overlap of the predicted trajectory, G ij (t) represents the conflict intensity of the prevailing game, B ij (t) represents the intensity of encroachment on the boundary between motor vehicles and non-motor vehicles, P ij (t) represents the pedestrian priority disturbance intensity, H ij (t) represents the memory item of historical high-risk behavior, and α, β, γ, δ, and ζ represent the risk fusion weights; S203. Conduct a joint assessment of the traffic benefits, waiting losses, conflict risks, local queuing disturbances and congestion shock wave spread effects of traffic participants after performing each candidate action, and dynamically adjust the comprehensive benefits of each candidate action according to the risk level under the current mixed traffic scenario to obtain the benefit assessment results corresponding to each candidate action. S204. Based on the benefit evaluation results of each candidate action, select the action with the largest comprehensive benefit as the action to be executed by the traffic participant at time t, and obtain the dynamic decision result of the traffic participant. S205. Based on the behavioral characteristics of traffic participants, formulate an appropriate micro-behavioral model to obtain a micro-behavioral model that adapts to mixed traffic flow. S206. Based on scenarios such as mixed traffic of motorized and non-motorized vehicles at large intersections, bus stops, and temporary passenger pick-up and drop-off for ride-hailing vehicles, a joint correction is performed on the uncorrected congestion propagation velocity to obtain the congestion propagation velocity. The calculation formula is as follows: Among them, v' w (t) represents the corrected congestion propagation speed, v w (t) represents the original congestion propagation speed without considering the effects of non-motorized vehicle infiltration, temporary lane occupation, and signal release capacity. non-motor (t) represents the real-time sensed non-motorized vehicle infiltration flow, W lane W is the total width of the mixed-traffic lane for motor vehicles and non-motor vehicles. occ (t) represents the equivalent physical width for parked vehicles or temporary road occupancy, L ref For reference queue length, G release (t) represents the effective green light release ratio within the current signal period, and β1, β2, and β3 are correction coefficients; S207. Based on the corrected congestion propagation speed, the congestion propagation impact boundary and congestion dissipation time are dynamically evaluated to obtain the congestion propagation evolution results. S208. Based on the principle of modular integration, the core simulation rules, dynamic decision-making results of traffic participants, micro-behavioral models, and mixed traffic flow congestion propagation models are integrated to obtain a dynamic model library that adapts to different traffic scenarios. S209. Quantitatively analyze traffic efficiency, congestion control, safety risks, and collaborative control effectiveness, construct a road network efficiency quantitative analysis system, and obtain road network efficiency evaluation results for simulation assessment and subsequent reinforcement learning training.

5. The city-level multi-scenario intelligent traffic collaborative management and reliable simulation method based on reinforcement learning according to claim 4, characterized in that, The microscopic behavior model in step S205 includes: (1) Vehicle following model considering driving emotions: The calculation formula is as follows: Among them, a n (t) represents the acceleration of the nth vehicle at time t, a max For maximum acceleration, v n (t) represents the current velocity, v des (t) represents the desired velocity, s ∗ (t) represents the dynamic safety following distance, s n (t) represents the actual following distance, k represents the emotional sensitivity coefficient, and L queue (t) represents the current queue length, L max This represents the maximum capacity of the intersection. (2) Phase Coordination Lane Changing Decision Model: Based on the difference in traffic utility between the target lane and the current lane, the risk cost of lane changing, and the degree of matching between the vehicle and the target green light phase when it arrives at the downstream intersection, the probability of successful lane changing is dynamically evaluated to obtain a lane changing decision result that meets the requirements of local safety and downstream signal coordination. (3) Non-motorized vehicle weaving model: Based on the operating characteristics of non-motorized vehicles under mixed traffic conditions, the non-motorized vehicle speed, lateral interference intensity, and available clearance length are jointly corrected to obtain the dynamic following distance of the non-motorized vehicle. The formula for calculating the dynamic following distance of the non-motorized vehicle is as follows: Among them, S non-motor (t) represents the dynamic following distance of the non-motorized vehicle at time t, s0 is the basic safety distance, and v non-motor (t) represents the speed of the non-motorized vehicle, Interside(t) represents the intensity of the lateral disturbance, and Gap front (t) represents the length of the available gap ahead, and λ1 is the non-motorized vehicle behavior correction coefficient; (4) Pedestrian crossing group model: Based on the characteristics of group crossing behavior, the speed of individual crossing, group size, degree of interference from oncoming vehicles and cross-section occupancy effect are jointly corrected to obtain the group crossing speed and group crossing time, and thus obtain the characteristics of group crossing behavior. (5) Bus Priority Control Model: Based on the bus's operating status when approaching the intersection, the distance of the bus from the intersection, the bus's passenger load level, and the current traffic light phase are jointly determined to obtain the bus priority triggering result. The red light shortening time is dynamically adjusted based on the urgency of bus priority and downstream queuing pressure. The formula for determining the bus priority triggering condition is as follows: Priority bus (t) Among them, Priority bus (t) is the bus priority trigger signal, distance(t) is the distance of the bus from the intersection, Load bus (t) represents the bus passenger load level, D th L is the distance threshold. th The passenger capacity threshold is given, and Phase(t) is the current traffic light phase. (6) Ride-hailing passenger pick-up and drop-off model: Based on the characteristics of ride-hailing temporary stops, the number of passengers picking up and dropping off, the degree of roadside interference, the stopping scenario, the duration of passenger pick-up and drop-off on the road, the intensity of road occupation, and the transfer capacity of adjacent lanes are jointly corrected to obtain the duration of passenger pick-up and drop-off and the capacity of affected lanes, and thus obtain the characteristics of the impact of ride-hailing temporary stops.

6. The city-level multi-scenario intelligent traffic collaborative management and reliable simulation method based on reinforcement learning according to claim 4, characterized in that, The specific process of the road network efficiency quantitative analysis system in step S209 is as follows: (1) Based on the traffic efficiency calculation method, the average delay time of intersections, the average travel speed of road segments and the utilization rate of road network capacity are calculated to obtain the traffic efficiency index. (2) Calculate the congestion duration, congestion range and delay index, and combine the congestion propagation model of mixed traffic flow in step S206 to identify the congestion propagation and dissipation process to obtain congestion control indicators. (3) Based on the safety risk assessment method, the number of intersection conflicts, the frequency of rapid acceleration and the frequency of rapid deceleration are calculated to obtain the safety risk index; (4) Using the collaborative control evaluation method, the phase coordination degree of adjacent intersections, the continuity of green light passage and the regional collaborative release effect are calculated to obtain the collaborative control index; (5) The weighted fusion evaluation method is adopted to weight and fuse traffic efficiency indicators, congestion control indicators, safety risk indicators and collaborative control indicators to obtain the comprehensive evaluation value of road network efficiency.

7. The city-level multi-scenario intelligent traffic collaborative management and reliable simulation method based on reinforcement learning according to claim 1, characterized in that, Step S3 includes the following sub-steps: S301. Based on the requirements of multi-agent collaborative management and control, construct a multi-agent reinforcement learning model, with each intersection corresponding to an agent, to obtain a regional multi-agent collaborative control model. S302. Based on the traffic propagation relationship, conflict propagation relationship, scene consistency, phase coupling relationship, and queuing pressure difference between intersections, the neighborhood information of each agent is weighted and aggregated to obtain a cooperative state representation. The aggregation weight calculation formula is as follows: ψ ij (t) = Where, ψ ij (t) represents the aggregation weight of node i to node j at time t, Corr ij (t) represents the traffic flow association strength, Phase ij (t) represents the phase coupling strength, Scene ij (t) represents the scene consistency strength, Gap ij (t) represents the queuing pressure difference between nodes; S303. Based on the aggregation weight, the state information of neighboring nodes is weighted and fused, and combined with the local state information of the current node, the collaborative state representation of the intersection agent is obtained: ij (t) = h i (t)+ in, ij (t) represents the cooperative state of intersection i at time t, h i (t) is the local state latent vector of intersection i, h j (t) is the local state latent vector of intersection j; S304. According to the model training specifications, initialize the reinforcement learning model parameters, and prioritize the use of samples according to the empirical sample scheduling rules. Prioritize the training of samples with large time difference errors, high collision risks, and obvious scene boundary features. Use the average relative change rate of reward as the convergence criterion. When the average relative change rate of reward for a preset number of consecutive iterations is not greater than the preset threshold, stop training to obtain a training process that meets the convergence requirements. S305. Construct a reliable twin constraint reward function based on the training objective, and train the multi-agent spatiotemporal attention reinforcement learning model to obtain the trained collaborative control agent. The reward function satisfies: R i (t) = Among them, R i (t) represents the reward value of intersection i at time t, ΔEff i (t) represents the efficiency improvement, Cred i (t) represents the consistency score between the digital twin simulation results and the measured traffic conditions, Coord i (t) represents the regional cooperative gain, and Delay i (t) represents the delay cost, Queue i (t) represents the queuing cost, Risk. i (t) represents the risk cost; S306. Based on the standardized state matrix obtained in step S1, the trained cooperative control agent is input into the regional road network for interactive decision-making, outputs the optimal signal control action, generates a regional-level cooperative traffic signal control scheme covering multiple adjacent intersections, and optimizes the phase difference coordination logic of adjacent intersections to obtain the initial control scheme. S307. According to the trusted verification requirements, the initial control scheme is input into the digital twin-based simulation platform to complete the trusted verification of the scheme. The simulation parameter configuration is expressed as follows: SimParam = {Step, RecordFreq, SceneType, λ sim } Where Step is the simulation step size, RecordFreq is the data recording frequency, SceneType is the scene type defined in step S2, and λ sim For simulation accuracy coefficients; S308. According to the simulation data recording rules, record the delays, speeds, congestion status, and implementation effects of the scheme during the simulation process to obtain the simulation verification results. The simulation data record is expressed as follows: SimData = {Delay(t), Speed(t), Congest(t), PhaseExec(t)} Where Delay(t) is the intersection delay time at time t, Speed(t) is the road segment speed at time t, Congest(t) is the congestion status at time t, and PhaseExec(t) is the signal control scheme execution effect at time t.

8. The city-level multi-scenario intelligent traffic collaborative management and reliable simulation method based on reinforcement learning according to claim 1, characterized in that, The specific training process for the multi-agent spatiotemporal attention reinforcement learning model in step S305 is as follows: (1) Based on the empirical sample scheduling rules, the sample time difference error, minimum collision time, scene scarcity, and scene switching boundary characteristics are jointly evaluated to obtain the sample sampling priority. The calculation formula is as follows: Where TD_Error is the temporal difference error of the sample, TTC(s, a) is the minimum collision time after performing action a in state s, Rare(s) is the scarce scene indicator, Shift(s) is the scene switching boundary indicator, and ε is a minimal constant to prevent the denominator from being zero. (2) Based on the sampling priority, the samples in the experience playback library are sorted and stratified for retention, so that high-risk samples, scarce scene samples and scene boundary samples can obtain a higher retention level. (3) Allocate sample sampling probabilities according to sample sampling priority, and prioritize the use of experience replay samples so that high-priority samples can obtain a higher training call frequency. (4) Enhance the input traffic flow state by applying scene disturbances to obtain the enhanced traffic flow characteristics. The calculation formula is as follows: Among them, Q aug (t) represents the enhanced flow characteristics, Q(t) represents the original flow characteristics, Bias(t) represents the scene offset term, and Resist(t) represents the weather impedance term; (5) Based on the topology regularization rule, the connections between nodes with weak correlations in the dynamic traffic graph are randomly suppressed to obtain the topology-constrained training graph. The formula for calculating the edge drop probability is: p ij (t) = p0(1- ) Where, p ij (t) represents the edge drop probability between node i and node j at time t, p0 is the basic drop rate, max(A(t)) is the maximum weight value in the dynamic adjacency matrix at time t, and m is a minimal constant to prevent the denominator from being zero.

9. The city-level multi-scenario intelligent traffic collaborative management and reliable simulation method based on reinforcement learning according to claim 1, characterized in that, Step S4 includes the following sub-steps: S401. Based on the trusted verification result of step S3, set the optimization triggering conditions and obtain the optimization triggering result. The formula for determining the triggering conditions is: Trigger Among them, Trigger is the optimization trigger signal, 1 indicates that optimization is started and 0 indicates that it is not started; Similarity is the matching degree between simulation data and actual collected data; DelayGain is the delay optimization magnitude; SafeScore is the extreme scenario adaptability score; θ1 is the simulation matching degree trigger threshold; θ2 is the delay optimization trigger threshold; and θ3 is the safety adaptation trigger threshold. S402. Based on the optimized triggering results, the signal control parameters and simulation parameters are adjusted in a targeted manner to obtain the adjusted parameters. The adjustment range includes at least the phase difference, green ratio, signal period, conflict risk threshold and congestion propagation model correction coefficient. S403. Based on the adjusted parameters, substitute them into the multi-agent spatiotemporal attention reinforcement learning model of step S3, and jointly update the control parameter correction, simulation model correction, scene adaptation correction and risk control correction to obtain the optimized joint parameter set. S404. Based on the preset evaluation criteria, repeat the training and simulation verification, iteratively correct the joint parameter set, and obtain an optimized solution that meets the requirements of credibility, performance and security. S405. Based on the scenario classification rules, the optimization scheme is solidified and classified into daily commuting, large-scale events, holidays and emergency scenarios to obtain scenario adaptation and control templates.

10. A city-level multi-scenario intelligent traffic collaborative management and reliable simulation method based on reinforcement learning according to claim 1, characterized in that, Step S5 It includes the following steps: S501. Based on the actual deployment requirements of the traffic control system, deploy the scenario adaptation control template obtained in step S4 to the edge-side signal controller to obtain the edge-side executable template set. S502. Based on the real-time traffic flow collection mechanism, the road network operation status is continuously monitored to obtain the road network status monitoring results; S503. Based on the road network status monitoring results, quickly match the scenario-adaptive control templates to obtain the candidate template scoring results; S504. Based on the candidate template scoring results, select the template with the highest comprehensive score as the template to be executed; S505. Based on the traffic mutation rate, equipment status and conflict risk change rate, edge cases are identified to obtain template switching trigger results; S506. Based on the long-term optimization rules, evaluate the effectiveness of the deployed solutions to obtain a solution effectiveness score. The calculation formula is as follows: E = Where E is the scheme effectiveness score, M is the number of samples in the evaluation period, and ΔDelay m Let ΔSpeed ​​be the delay reduction rate for the m-th sample. m Let ΔCongest be the velocity increase rate of the m-th sample. m Let w1, w2, and w3 be the congestion relief rate for the m-th sample, and w1, w2, and w3 be the weighting coefficients. S507. Based on edge cases and edge-side incremental parameters, aggregate parameter increments from various regions in the cloud, and combine them with scarce scenario correction information, road network structure change information, and event disturbance correction information to continuously update the control template in a federated manner, thereby obtaining the updated scenario template. S508. Based on the updated scenario template, continuously revise the traffic control strategy in the actual road network to obtain a continuous adaptive learning result that adapts to the evolution of the actual road network.

Images