A method for intelligent regulation of buffer zones in densely populated situations

Abstract

This invention relates to the field of passenger flow management, specifically to an intelligent control method for buffer zones in densely populated situations. The method includes: 1. Determining key observation areas; 2. Constructing a single-input multi-output discrete-time state-space predictive dynamic identification model (SSM) for multi-step prediction; 3. Constructing a state estimator to feed back measured output information to the state update process, recursively updating and correcting the internal state of the system; 4. Constructing a model predictive controller (MPC) to perform multi-step rolling predictions of future outputs in the prediction time domain based on the updated internal state of the system; 5. Constructing a DDQN agent to calculate the risk of exceeding the target or going out of bounds; 6. Completing a gradual switching from the initial to the later stages; 7. Iterating online on the parameters of the DDQN evaluation network and the target network. Through preheating guidance and soft-constraint rolling optimization of model predictive control, the safety of the control process is ensured.

Description

Technical Field

[0001] This invention relates to the field of passenger flow management, specifically to an intelligent control method for buffer zones in densely populated situations. Background Technology

[0002] With the rapid development of urban rail transit, subway stations, as typical passenger distribution points, often experience long queues at turnstile entrances and overcrowding in the station hall due to short-term surges in passenger flow. This easily leads to key facilities such as turnstiles operating under continuous high load. If reasonable passenger flow management measures are not implemented in a timely manner, the risk of stampedes in bottleneck areas will significantly increase and disrupt station operations. Currently, station management personnel generally set up passenger flow buffer zones composed of fences upstream of the turnstile entrances. By extending the pedestrian walking path, disorderly passenger flow is transformed into orderly queues, thereby regulating the flow rate into bottleneck areas. However, existing buffer fences are mostly static and fixed layouts. Their operation often relies on the experience of management personnel and lacks quantitative theoretical basis, making it difficult for the facilities to continuously match dynamic changes in passenger flow: when passenger flow is low, it can easily cause unnecessary loss of passage efficiency, while when passenger flow surges, it cannot effectively alleviate congestion due to a lack of timely adjustment. In addition, while fixed layouts can achieve local peak reduction, they can easily cause congestion to migrate spatially, leading to the formation of new high-density gathering points or passage blockages at the buffer zone entrances.

[0003] To improve the intelligence level of traffic control, existing research attempts to introduce model predictive control or reinforcement learning for dynamic regulation of passenger flow management facilities. However, the performance of model predictive controllers often highly depends on the accuracy of the predictive model, and they tend to exhibit insufficient robustness when facing dynamic crowd systems with strong uncertainties and nonlinear disturbances. While reinforcement learning possesses strong adaptive potential, its trial-and-error exploration characteristics require extensive random interactions with the environment in the early stages of training, which can easily lead to extreme control actions. This poses a serious risk of congestion exceeding limits in scenarios like subway stations where safety and stability requirements are extremely high. Therefore, how to achieve adaptive dynamic regulation of buffer facilities while ensuring passenger flow safety constraints, effectively suppressing the transfer of congestion between different areas, and balancing traffic efficiency and control stability, is a pressing technical challenge in the field of urban rail transit passenger flow management. Summary of the Invention

[0004] To address the problems existing in the prior art, the purpose of this invention is to provide an intelligent control method for buffer zones under densely populated conditions that can achieve adaptive dynamic control of buffer zone facilities while ensuring passenger flow safety constraints, effectively suppress the transfer of congestion between different areas, and take into account both traffic efficiency and control stability.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is: an intelligent control method for buffer zones in densely populated situations, comprising the following steps: Step 1: Use pedestrian dynamics simulation software to model passenger flow in densely populated areas. Based on the terrain characteristics of densely populated areas, set up passenger flow buffer zones with specific flow guidance structures composed of fences in the upstream area at bottleneck locations. Determine the bottleneck locations and passenger flow buffer zones as key observation areas. Step 2: Using a data-driven system identification approach, learn the dynamic mapping relationship between facility control input and passenger flow density response in key areas, apply multiple sets of discrete operation mode excitations to the fence facility, collect control inputs at each sampling time and passenger flow density outputs in key observation areas, and construct a single-input-multiple-output discrete-time state-space prediction dynamic identification model (SSM) for multi-step prediction. Step 3: At each sampling moment, the current passenger flow density measurement value and the discrete operation mode executed at the previous moment are obtained in real time, a state estimator is constructed, and based on the SSM, the measurement output information is fed back to the state update process to recursively update and correct the internal state of the system. Step 4: Construct a Model Predictive Controller (MPC) that includes a single-sided target tracking term for passenger flow density, a penalty term for changes in control input, a penalty term for soft constraint out-of-bounds, and a dual-region consistency term. Use the identified SSM as the internal prediction model of the MPC, and use the discrete opening and closing combination mode of the buffer zone entrance facility as the control decision variable. Based on the updated internal state of the system, perform multi-step rolling prediction of future output in the prediction time domain. Step 5: Using an approximate Markov decision process with feature extension, construct a deep dual-Q network reinforcement learning controller (DDQN) agent that integrates MPC prediction information. Introduce future prediction terms and use the internally built SSM to calculate future predictions. If the risk of exceeding the target or going out of bounds occurs within a step, evaluate the network output to determine the optimal action in the current state; Step 6: During the warm-up phase, the control input generated by the MPC is used as the actual control action to provide the agent with stable decision reference and high-quality interaction samples. After the warm-up phase, the candidate control action output by the MPC is selected according to the probability decaying over time, and the control action output by the DDQN is selected according to the complementary probability to complete the gradual switch from the initial strong security guidance to the mid-to-late stage autonomous learning optimization. Step 7: Dynamically send actual control actions to the entrance of the controllable fence facility to adaptively adjust the exit passenger flow rate, thereby controlling the crowd density at the bottleneck location and the entrance area of ​​the buffer zone. The system state transition and passenger flow movement status are fed back to the control algorithm in real time to calculate the immediate reward and store it in the experience playback pool. The parameters of the DDQN evaluation network and the target network are updated and iterated online using the experience playback mechanism and the time-series differential target error to realize the iterative optimization and adaptive evolution of the control strategy.

[0006] The aforementioned intelligent control method for buffer zones in densely populated situations, in step 2, uses a pseudo-random binary sequence (PRBS) as the control input signal for the system identification experiment. The structure of the SSM is as follows: , , in, For discrete-time indexing, Let be the system state vector. The control input mode for opening and closing the entrance to the passenger flow buffer zone. Indicates the system at the 1st Output at any moment The state space matrix, To estimate the gain for the state, This is the noise disturbance term.

[0007] The aforementioned intelligent control method for buffer zones in densely populated situations, in step 3, constructs a state estimator to recursively update and correct the internal state of the system using real-time acquired passenger flow density measurements. At each sampling time, given the control input from the previous time step... Measurement output at the current time In this case, the SSM identified by the system includes: Step 3-1: Given the control input at the previous time step and the measured output at the current time step, calculate the one-step predicted output based on the state-space identification model as follows: ,in, Indicates based on Known information at any given time The one-step prediction value output at each time step. Indicates based on Information obtained at a given time One-step prediction of state at time step. express Constantly control input; Step 3-2: Define the error term and extract the prediction error of the current time step as follows: ,in, express Output the prediction error at all times. express Actual measured output at any given moment; Step 3-3: Update the state using a linear correction method, feeding the measurement output information back into the state update process to obtain the state estimate for the next time step: .

[0008] The above-mentioned intelligent control method for buffer zones in densely populated situations includes step 4 as follows: Step 4-1: Keep the input outside the control time domain constant, and assume the prediction time domain is... Control time domain is The stacked prediction matrix for model predictive control is constructed as follows:

[0009] in, This represents the stacked output vector in the prediction time domain. Represents the control sequence vector within the control time domain. The matrix is ​​a prediction matrix from state to output. For control gain matrix; Step 4-2: In the rolling optimization solution process, the multi-objective cost function is defined as: ,in, For passenger flow density, a one-sided target tracking item, To control the penalty term for input changes, This is a soft constraint out-of-bounds penalty item. This is a term with consistent density across two regions; Step 4-3: The controller explicitly introduces hard constraints, including input increments, while optimizing the cost function. Output soft constraints and output difference soft constraints Multiple types of constraints; Step 4-4: Predict future time domain based on current state estimation The system output is used to solve the optimal control sequence online under the various constraints, and the first control variable in the sequence is applied to the fence of the buffer zone to complete the rolling optimization and closed-loop control of the system.

[0010] The above-mentioned intelligent control method for buffer zones in densely populated situations includes step 5 as follows: Step 5-1: Extract the state-space representation of the system's current features containing information about delayed actions, and define an approximate Markov decision process and reinforcement learning framework. ,in, and Normalized passenger flow density for key areas The action performed in the previous moment; Step 5-2: Define the time frame The initial cumulative discount reward is: The corresponding optimal action value function satisfies the Bellman optimality equation as follows: ,in, This represents the cumulative discount reward obtained starting from time k. This represents the immediate reward obtained at step k. Indicates the discount factor. The optimal action value function is used to characterize the state. s Take action below a The maximum expected cumulative discount return that can be obtained by subsequently following the optimal strategy; s Indicates the current state. a Indicates the current action. Indicates the next state. This represents a candidate action for the next state. This represents the immediate reward obtained after performing the current action. Indicates that given the current state s and actions a Conditional expectation under given conditions This indicates the value of the candidate action that maximizes long-term benefits in the next state. Step 5-3: By weighting and summing the five sub-indicators, a composite reward function integrating the fusion model prediction control cost function structure and the prediction model information is constructed as follows: ,in, Indicates the first k Total reward obtained in each step , , , and These represent the weights of the target tracking term, the safety soft constraint term, the dual-region equilibrium consistency term, the action smoothing term, and the future prediction term, respectively. The target tracking reward is used to reflect the degree of deviation of the system output from the target threshold. This indicates that soft-constraint rewards are used to penalize high-risk congestion states that exceed the safety upper bound. The dual-region equilibrium consistency reward is used to suppress excessive differences in passenger flow density between two key regions. This indicates that the smoothness reward is used to suppress frequent switching of control actions between adjacent time steps. Indicates the first k Always looking towards the future The reward for predicting the future is obtained by making predictions step by step.

[0011] The above-mentioned intelligent control method for buffer zones in densely populated situations includes step 6 as follows: Step 6-1: When the number of rounds is... During the warm-up phase, the actual actions are performed using the control input generated by the model predictive controller, defined as... , , Indicates the system at the 1st k The control actions actually performed at each decision-making moment The model predicts the controller in the first... k The control action output at each decision moment; Step 6-2: When the number of rounds is... After the preheating phase, control is allocated between MPC and DDQN based on the probability decaying over time during annealing. At each decision moment, the system allocates control based on probability. Choose to execute MPC actions based on probability. The scheduling selection probability function of MPC is defined as follows: (The original text contains several typographical errors and inconsistencies, making a precise translation difficult. A more accurate translation would require the full context.) ,in, The initial control selection probability after preheating. The minimum probability is preset. To determine the step size for the decay rate; Step 6-3: When the hybrid strategy determines that the DDQN agent should output the control action, it adopts... - The greedy strategy makes an online trade-off between environmental exploration and value exploitation, defining the exploration probability as monotonically decaying over time: ,in, This represents the exploration probability at the k-th decision time. This represents the initial exploration probability after the warm-up phase ends. This represents the minimum probability of exploration. Indicates the duration of the preheating phase. The decay step size represents the probability of exploration. k Indicates the current decision-making moment; Step 6-4: By generating in Uniform random numbers within the interval and Make a judgment: if Then the control action output by the model predictive controller will be executed. ;like Then the deep dual-Q network agent outputs the control action.

[0012] The aforementioned intelligent control method for buffer zones in densely populated situations is characterized in that, in step 6-4, when a control action is output by a deep dual-Q network agent, if Then, actions are uniformly and randomly selected from the discrete action set A to explore the environment; if Then, the evaluation network is used to calculate the action value, and the optimal action that maximizes the action value evaluation function is selected as the current control action, that is: ,in, This represents the action chosen by the DDQN agent at the k-th decision time. Indicates the first k The system state at each decision moment This represents the action value function used to evaluate the network output. This indicates the evaluation of network parameters.

[0013] The above-mentioned intelligent control method for buffer zones in densely populated situations includes step 7, which includes: Step 7-1: Obtain state transitions and passenger flow status feedback in real time through the simulation interaction channel, and convert the experience tuples generated at each interaction step into a real-time state transition and passenger flow status feedback. Store in the experience replay pool Among them This is the round end marker; Step 7-2: The deep dual-Q network calculates the temporal difference TD target error using sampled mini-batch samples. The target error is... ,in, This represents the index of the transferred sample sampled from the experience replay pool. For the first Instant rewards for each sample This is the end-of-round flag, indicating the state at the next moment. If it is a terminated state, then ,otherwise , and These represent the current state and the state at the next moment, respectively. In the state The action to be performed This represents the action space of the controllable fence discrete operation mode of the passenger flow buffer zone. Indicates the state Lower Action Space Candidate actions in To evaluate network parameters, For target network parameters; Step 7-3: Employ the dynamic adaptive Huber loss function To handle errors and improve the stability of training systems against random perturbations and extreme samples in the target population, ,in, This is the clipping threshold; Step 7-4: In the case of mini-batch training, the overall objective function of the batch loss is defined as follows: ,in, Indicates about parameters Batch loss function, This indicates the number of samples contained in a batch. Indicates the first l The temporal difference error corresponding to each transferred sample. Indicates the first l The loss function value corresponding to each sample; Step 7-5: Combine the Adam optimizer with adaptive adjustment of the learning step size based on the first and second moment estimates to evaluate the network parameters. Perform online updates and periodically synchronize parameters to the target network according to a preset number of steps. This enables iterative optimization and adaptive evolution of control strategies.

[0014] The aforementioned intelligent control method for buffer zones in densely populated situations also includes: Step 8: Construct a comprehensive evaluation index system for the dynamic control scheme of passenger flow buffer zones, which includes four dimensions: congestion risk, passage end time, number of switching times, and density difference, to assess the congestion status of key areas.

[0015] The aforementioned intelligent control method for buffer zones under dense crowd conditions, wherein the congestion risk index simultaneously characterizes congestion intensity and congestion duration, and a graded risk function is used to evaluate the passenger flow status for time-varying passenger flow density, step 8 includes: Step 8-1: Define the hierarchical risk penalty function for the w-th output at time k as follows: When passenger flow density When, the penalty value is 0; when At that time, the penalty value is ;when At that time, the penalty value is ; Step 8-2: Calculate the comprehensive congestion risk by combining time accumulation and regional weighting, and define it as follows: ,in, This represents the overall congestion risk function. and Let represent the weight coefficients corresponding to the first and second key regions, respectively, and N represent the number of sampling steps in the evaluation time domain. and They represent the first k Passenger flow density in the first and second key areas at each sampling time. This represents the graded risk penalty function corresponding to passenger flow density. This represents the sampling time interval between adjacent sampling moments. This indicates that the travel time contribution at each sampling time point is accumulated over the entire evaluation time domain; Step 8-3: Calculate the time it takes for pedestrians to completely cross different areas, and define the average crossing time index as... ,in and These represent the end times when all pedestrians have passed through the gate entrance area and the buffer zone entrance area, respectively. Step 8-4: Directly incorporate the discrete switching number of the control input fence opening and closing combination mode into the evaluation. The fewer the switching number, the more stable the control command and the smaller the disturbance caused by pedestrian path reselection. Step 8-5: Extract the output density difference of different key regions As an evaluation indicator, it monitors the fluctuations and concentration of density differences, quantifies the balanced distribution of passenger flow across areas, and identifies and mitigates the spatial bias and congestion risks associated with high-density populations. The beneficial effects of the intelligent regulation method for buffer areas in densely populated situations proposed in this invention are as follows: through the preheating guidance of model predictive control and soft constraint rolling optimization, it is ensured that the regulation process never exceeds the safety red line; at the same time, the deep double-Q network realizes closed-loop adaptive optimization for complex passenger flow and static response using interactive data.

[0016] The design of consistency items and hierarchical risk assessment indicators effectively suppressed the extreme migration of congestion from the gate entrance to the buffer zone entrance, and achieved a balanced distribution of risk between the two areas.

[0017] By introducing control input switching penalties and annealing scheduling mechanisms, the facility switching frequency is significantly reduced compared to traditional methods, and by 63.32% compared to MPC, greatly reducing transient disturbances to passenger behavior.

[0018] The design of a reward function that integrates prediction information and a Hubbar loss function enables the agent to converge faster and achieve higher rewards within a limited number of training rounds, significantly improving the deployment efficiency of the algorithm in real-world engineering environments. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the steps of the intelligent control method provided in the embodiment of the present invention; Figure 2 This is an overall control framework diagram of the intelligent regulation method provided in the embodiments of the present invention; Figure 3 This is a schematic diagram of the geometric structure design of the passenger flow buffer zone and the distribution of key observation areas in an embodiment of the present invention; Figure 4 This is a fitting verification diagram of the system identification model in this embodiment of the invention for the one-step predicted passenger flow density output of the entrance gate area; Figure 5This is a fitting verification diagram of the system identification model in this embodiment of the invention for the 20-step predicted passenger flow density output of the entrance area of ​​the gate; Figure 6 This is a fitting verification diagram of the system identification model in this embodiment of the invention for the 40-step predicted passenger flow density output of the entrance area of ​​the gate; Figure 7 This is a fitting verification diagram of the system identification model in this embodiment of the invention for the 60-step predicted passenger flow density output of the entrance gate area; Figure 8 This is a fitting verification diagram of the system identification model in this embodiment of the invention for the one-step predicted passenger flow density output of the passenger flow buffer entrance area; Figure 9 This is a fitting verification diagram of the system identification model's output of the predicted passenger flow density in the entrance area of ​​the passenger flow buffer zone in this embodiment of the invention. Figure 10 This is a fitting verification diagram of the system identification model in this embodiment of the invention for the 40-step predicted passenger flow density output of the passenger flow buffer entrance area; Figure 11 This is a fitting verification diagram of the system identification model's output of the predicted passenger flow density in the entrance area of ​​the passenger flow buffer zone in this embodiment of the invention. Figure 12 This is a comparison chart showing the control effects of the regulation method and the comparison method provided in this embodiment of the invention on the dynamic evolution of passenger flow density in region A; Figure 13 This is a comparison chart showing the control effects of the regulation method and the comparison method provided in this embodiment of the invention on the dynamic evolution of passenger flow density in region B; Figure 14 This is a statistical comparison chart of the control method and the comparison method provided in this embodiment of the invention in terms of the number of switching times and maintenance characteristics of the discrete operation mode of the fence; Figure 15 This is a box plot comparing the control method and the comparison method provided in this embodiment of the invention on the distribution of passenger flow density difference in key areas; Figure 16 This is a comparison diagram of the cumulative reward value evolution and convergence of the hybrid control strategy and traditional reinforcement learning in the training process in this embodiment of the invention. Detailed Implementation

[0020] To enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention will be described below in conjunction with specific embodiments and accompanying drawings.

[0021] Example 1 This embodiment takes a subway station as an example to explain the intelligent control method for buffer zones in densely populated situations. Similarly, this method is also applicable to various public places with turnstiles.

[0022] Figure 1 This is a flowchart illustrating an intelligent control method for subway station passenger flow buffer zones based on model predictive control guided reinforcement learning, provided as an embodiment of this disclosure. This invention proposes an intelligent control method for buffer zones under densely populated conditions. Specifically, it includes... Figure 1 The following steps are shown.

[0023] Step 1: Use pedestrian dynamics simulation software to model passenger flow in densely populated areas. Based on the terrain characteristics of densely populated areas, set up passenger flow buffer zones with specific flow guidance structures composed of fences in the upstream area at bottleneck locations. Determine the bottleneck locations and passenger flow buffer zones as key observation areas.

[0024] Scene modeling and key area definition target typical bottleneck locations such as subway station entrance gates, where large influxes of passengers in a short period often lead to long queues at the gate entrances and overcrowding in the station hall. This embodiment aims to regulate the flow rate into the bottleneck area by setting up a passenger flow buffer zone composed of fencing facilities upstream of the gate entrance, thereby extending the pedestrian walking path and transforming disordered passenger flow into an orderly queue.

[0025] This embodiment uses pedestrian dynamics simulation software to model passenger flow at a station. The software uses a social force model to model specific pedestrian movement behavior. Pedestrian movement is determined by the resultant force of self-driving force, interaction forces between people, interaction forces between people and obstacles, and random forces. Its dynamic equation is expressed as: Among them, self-driving force It describes the pedestrian's tendency to walk at the desired speed and in the desired direction: ,in, For pedestrian quality, For the desired speed, Let the desired direction be the unit vector. This refers to the reaction time.

[0026] Interaction forces between pedestrians Psychological repulsion friction with the body composition: ,in, The sum of the pedestrian radii. For pedestrian distance, This is a contact activation mechanism. The force between the pedestrian and the obstacle. Similar definition.

[0027] Furthermore, pedestrians choose their routes based on a minimum cost model, with the cost function being: ,in, Distance to destination For the expected speed, For the expected queuing time, Cost of geometric components.

[0028] Step 2: Using a data-driven system identification approach, learn the dynamic mapping relationship between facility control input and passenger flow density response in key areas, apply multiple sets of discrete operating mode excitations to the fence facility, collect control inputs at each sampling time and passenger flow density outputs in key observation areas, and construct a single-input-multiple-output discrete-time state-space prediction dynamic identification model (SSM) for multi-step prediction.

[0029] Traditional mechanistic modeling is insufficient to meet the requirements of real-time multi-step predictive control given the highly nonlinear and stochastic fluctuations in passenger flow evolution within subway stations. Therefore, this embodiment employs a data-driven system identification method to directly learn the dynamic mapping relationship between control input and passenger flow density response.

[0030] To fully stimulate the dynamic characteristics of the system, this embodiment uses a pseudo-random binary sequence (PRBS) as the control input signal for the identification experiment. Pseudo-random binary sequences have the characteristics of a wide spectral distribution and can effectively stimulate the system with fewer prior assumptions, revealing the complex dynamic evolution laws within the system. This allows for a more complete revelation of the system's internal dynamic characteristics, significantly improving the identification model's ability to represent dynamic behavior in different frequency bands, enhancing the predictive model's identifiability and generalization ability under different passenger flow tidal fluctuation scenarios, thereby improving the system's identifiability and model generalization ability.

[0031] In the simulation environment, by applying the PRBS signal, the system randomly generates 25 sets of experimental data. The system's discrete control inputs... The output corresponds to the three fence opening and closing modes at the entrance of the passenger flow buffer zone. Density of entrance gate area Density of the buffer zone entrance region Of these 25 sets of data, 20 were used for parameter estimation and 5 were used for validation and evaluation.

[0032] Considering the sensitivity of online solution for model predictive control to computational efficiency, this embodiment adopts a single-input multiple-output (SSM) model. , ,in, For discrete-time indexing; This is the system state vector, used to characterize the internal evolution process of the system; The control input mode for opening and closing the entrance to the passenger flow buffer zone; Indicates the system at the 1st The output at any given time includes passenger flow density components for the subway station gate entrance area and the buffer zone entrance area; This is the state space matrix, used to characterize the evolution of the system state and the dynamic mapping relationship between input and output; Estimate the gain for the state; The noise disturbance term describes the combined effects of uncertainties such as the randomness of crowd movement, measurement errors, and unmodeled dynamics. This model has a simple structure, significantly reducing the solution burden of online optimization and meeting the requirements for engineering deployability.

[0033] Parameter identification employs a two-stage strategy of subspace identification fusion to minimize prediction error. The final calculated true system state space matrix is ​​as follows:

[0034] .

[0035] To quantify the model's performance, it was compared with ARMAX, ARX, and NLARX models. The results show that the SSM model... and The goodness of fit reached 91.02% and 93.76% respectively, with a mean square error of [missing value]. Especially in multi-step prediction, the SS model suffers from a high final prediction error (FPE). Akaike Information Criteria (AIC) ) and Bayesian Information Criterion (BIC) The model significantly outperforms other polynomial and nonlinear models in all aspects, demonstrating the best long-term predictive stability. This identification model provides a highly reliable predictive foundation for subsequent rolling optimization solutions and reward function calculations.

[0036] Step 3: At each sampling moment, the current passenger flow density measurement value and the discrete operation mode executed at the previous moment are obtained in real time, a state estimator is constructed, and based on the SSM, the measurement output information is fed back to the state update process to recursively update and correct the internal state of the system.

[0037] State estimation and internal state recursive update: In this embodiment, since the internal state of the subway station passenger flow control system cannot be directly measured by sensors, in order to improve the accuracy and robustness of state estimation and output prediction, it is necessary to construct a state estimator to recursively update and correct the internal state of the system using real-time passenger flow density measurements.

[0038] At each sampling time, given the control input from the previous time step... Measurement output at the current time In this case, based on the SSM identified by the system, the specific recursive calculation steps are as follows.

[0039] First, given the control input at the previous time step and the measured output at the current time step, the one-step prediction value of the output at the current time step is calculated based on the state-space identification model. : .

[0040] Secondly, define the error term and extract the prediction error of the current time step. : .

[0041] Finally, a linear correction method is used to update the state, and the measurement output information is fed back into the state update process to obtain the state estimate for the next time step. : .

[0042] In the above formula, Indicates based on Known information at any given time The one-step prediction value output at each time step. express Output the prediction error at all times. express Real-time measured output Indicates based on Information obtained at a given time One-step prediction of state at time step. express Constantly control the input. Let A represent the state matrix of the state-space model, B represent the input matrix of the state-space model, C represent the output matrix of the state-space model, and D represent the direct transfer matrix of the state-space model. K This is the state estimation gain matrix.

[0043] By using measurement error feedback and system state updates to correct model predictions in real time at each sampling moment, it is possible to effectively suppress error accumulation and state drift caused by random disturbances in the population system, noise, and model uncertainty.

[0044] Step 4: Construct a Model Predictive Controller (MPC) that includes a single-sided target tracking term for passenger flow density, a penalty term for changes in control input, a penalty term for soft constraint out-of-bounds behavior, and a dual-region consistency term. Use the identified SSM (Site Stream Management System) as the internal predictive model of the MPC, and the discrete opening and closing combination mode of the buffer zone entrance facility as the control decision variable. Based on the updated internal state of the system, in the prediction time domain... The system performs multi-step rolling prediction of future outputs and solves for the optimal control sequence under constraints.

[0045] In this embodiment, the identified SSM is used as the internal prediction model, and multi-step prediction and control sequence optimization are unified into a rolling optimization framework by constructing a stacked prediction matrix. Let the prediction time domain be... Control time domain is The stacked prediction expression for model predictive control is as follows: ,in, This represents the stacked output vector in the prediction time domain; This represents the control sequence vector within the control time domain; The matrix is ​​a prediction matrix from state to output. This reflects the effect of the predicted time-domain output on the system state; The control gain matrix is ​​used to characterize the cumulative effect of the control sequence on the predicted output within the control time domain.

[0046] To ensure that a complete prediction output can still be generated when the number of prediction steps exceeds the control time domain, a processing method is adopted that keeps the input outside the control time domain unchanged, that is, when the number of prediction steps... At that time, keep the future control input as the last decision quantity: This approach enables multi-step forecasting without adding additional decision variables, while maintaining the interpretability and stability of the control sequence at the end of the forecast. It captures the long-term trend of passenger flow evolution without introducing additional decision variables, while ensuring the interpretability of the control sequence at the end of the forecast.

[0047] In the rolling optimization solution process, a multi-objective cost function is defined. .

[0048] The cost function is specifically composed of four concatenated parts: First, there is the passenger flow density one-sided target tracking item. The one-sided over-limit error is defined as Indicates the future number The step is the one-sided over-limit error vector; Q is the one-sided target tracking error weight matrix for passenger flow density; This is the predicted value of the output at the (k+d)th step based on the state at time k; This is the desired threshold density vector. This term only penalizes cases where the predicted passenger flow density exceeds the target threshold, allowing passenger flow to fluctuate naturally within a safe range.

[0049] Secondly, there is the penalty for changes in control input. ,in, It is used to constrain and control the input increment. R is the control input change penalty weight matrix. This term is used to suppress frequent switching or large jumps in discrete control modes, improving the smoothness and executability of the control sequence.

[0050] Next is a soft constraint boundary violation penalty term that allows the system output to exceed controllable limits within a small range and applies a nonlinear penalty. ,in, Indicates the future number Step 1 The upper bounds of the output; Indicates the future number Step 1 The lower bounds of the output; and These represent the soft constraint weights corresponding to the upper and lower bounds of the w-th output, respectively. This term allows the system output to exceed the bounds slightly within a controllable range, and highlights the risk state with a larger degree of bounds exceedance through a squared penalty.

[0051] Finally, the dual-region density consistency term ,in, Weights for the density consistency term in the two regions; This is the allowable density difference threshold between the two key areas. This parameter is used to constrain the passenger flow density difference between the two key areas to remain within an acceptable range, thereby reducing the risk of extreme spatial offset of passenger flow load and congestion migration.

[0052] Constraints cover hard constraints in discrete mode of control input Hard constraints on the rate of change of control input In addition, soft constraints on output are introduced. ,in and These represent the upper and lower bounds of the output, respectively. By transforming the originally stringent hard constraints into penalized soft constraints, the system can still find feasible solutions under extreme conditions such as sudden increases in passenger flow. This avoids the problem of overly conservative control or no solution caused by forcibly satisfying the target value, and improves the robustness of the system in highly uncertain scenarios.

[0053] During the solution phase, the controller explicitly introduces multiple types of constraints while optimizing the cost function, specifically including hard constraints on the input increment. Output soft constraints and output difference soft constraints Predicting the future based on the current state. The system output is used to solve the optimal control sequence online under the above constraints, and the first control variable in the sequence is applied to the adjustable facility of the buffer, thereby completing the rolling optimization and closed-loop control of the system.

[0054] Step 5: Using an approximate Markov decision process with feature extension, construct a deep dual-Q network reinforcement learning controller (DDQN) agent that integrates MPC prediction information. Introduce future prediction terms and use the internally built SSM to calculate future predictions. If the risk of exceeding the target or going out of bounds occurs within a step, evaluate the network output to determine the optimal action in the current state.

[0055] In this embodiment, in order to compensate for the non-Markovianness caused by time delay and partial observability in the subway station crowd management system, an approximate Markov decision process with feature extension is used for modeling.

[0056] The system defines the state space as follows: ,in and Normalized passenger flow density for key areas The action performed at the previous moment is used to explicitly characterize the input lag effect and pattern inertia of the system. Introducing historical action information is to explicitly characterize the input lag effect and pattern inertia of the system, thereby compensating for the non-Markovian nature caused by some observability and control feedback time delay in the subway station crowd management system.

[0057] The goal of reinforcement learning is to maximize cumulative discounted returns. An optimal action value function is introduced to evaluate the expected reward obtained by following the policy after taking an action in a given state. The corresponding optimal action value function satisfies the Bellman optimality equation. It is used to evaluate the maximum long-term benefit that can be obtained by taking the open-close mode decision under a specific state, thereby guiding the strategy to achieve adaptive passage optimization under security constraints.

[0058] in, This represents the cumulative discounted reward obtained starting from time k. This represents the immediate reward obtained at step k; This represents the discount factor, used to characterize the weight of future returns relative to current returns; The optimal action value function is used to characterize the state. s Take action below a The maximum expected cumulative discount return that can be obtained by subsequently following the optimal strategy; s Indicates the current state. a Indicates the current action. Indicates the next state. This represents a candidate action for the next state. This represents the immediate reward obtained after performing the current action. Indicates that given the current state s and actions a Conditional expectation under given conditions This indicates the value of the candidate action that maximizes long-term benefits in the next state.

[0059] To guide the agent in adaptive optimization under safety constraints, the system constructs a composite reward function that integrates the model predictive control cost function structure with predictive model information. ,in, Indicates the first k Total reward earned per step; , , , and These represent the weights of the target tracking term, the safety soft constraint term, the dual-region equilibrium consistency term, the action smoothing term, and the future prediction term, respectively. The target tracking reward is used to reflect the degree of deviation of the system output from the target threshold; This indicates that soft-constraint rewards are used to penalize high-risk congestion states that exceed the safety upper bound; The dual-region equilibrium consistency reward is used to suppress excessive differences in passenger flow density between two key regions. The action smoothing reward is used to suppress frequent switching of control actions between adjacent time steps; Indicates the first k Always looking towards the future The reward for making predictions in the future is obtained by taking steps to make predictions, thereby improving the foresight of the strategy.

[0060] This composite reward function comprehensively quantifies the overall performance of the control action by weighting and summing five sub-indicators: target tracking, safety soft constraint, regional equilibrium consistency, action smoothing, and future prediction.

[0061] In each specific calculation, the target tracking item Positive and negative deviation segmented penalty design Nonlinear penalties are imposed on states that deviate from the expected threshold; soft safety constraints. Strengthen penalties for behaviors exceeding the maximum permissible density to ensure the safety of the red line; dual-region equilibrium consistency item A dimensionless penalty for exceeding the regional density difference limit is introduced to constrain the passenger flow load of the two regions to maintain a balance; motion smoothing term The difference in fence patterns executed at adjacent time points is penalized to reduce policy jitter.

[0062] Meanwhile, to enhance the foresight of the strategy, the system introduces future prediction terms. This step utilizes an internally built-in SSM to quickly calculate the future. The risk of exceeding the target or going beyond the boundary may occur within the step. This represents the number of forward prediction steps in the future prediction term, used to calculate the future risk reward. It can be set as needed. The future prediction term uses the identified state-space model to calculate the predicted reward for multiple future sampling steps. This term focuses on whether there will be risks of exceeding the target or going out of bounds in the future, enhancing the agent's foresight and guiding the strategy to take mild adjustment measures in advance to prevent the accumulation of risks caused by the continuous increase in the number of passengers.

[0063] By integrating the aforementioned immediate and future reward signals, the DDQN agent uses an evaluation network to output the optimal action for the current state. This allows for the approximation of the optimal strategy under multi-objective collaboration.

[0064] Step 6: During the warm-up phase, the control input generated by the MPC is used as the actual control action to provide the agent with stable decision reference and high-quality interaction samples. After the warm-up phase, the candidate control action output by the MPC is selected according to the probability decaying over time, and the control action output by the DDQN is selected according to the complementary probability, thus completing the gradual switch from the initial strong security guidance to the mid-to-late stage autonomous learning optimization.

[0065] In this embodiment, to avoid extreme control actions caused by random exploration in the initial stage of the deep double-Q network, the system implements an annealing-based hybrid scheduling strategy. During the preset warm-up phase, i.e., the number of steps in the current round... At that time, the actual actions are executed entirely using the control inputs generated by the model predictive controller, and the actual actions are executed entirely according to the model predictive control baseline. Defined as , , Indicates the system at the 1st k The control actions actually performed at each decision-making moment The model predicts the controller in the first... k The control action output at each decision moment. This formula indicates that during the warm-up phase, the system directly uses the control action predicted by the model from the controller output as the actual action to be executed at the current moment. This step aims to provide the system with a stable and feasible initial operating trajectory and to accumulate high-quality interaction samples for subsequent network training.

[0066] When the preheating phase is over, that is when At that time, the system allocates control between MPC and DDQN based on a probability that decays over time through annealing. At each decision time, the system allocates control based on probability... Choose to execute MPC actions based on probability. The DDQN action is selected to achieve a gradual transition from initial strong security guidance to later autonomous learning and optimization. The scheduling selection probability function of the MPC is defined as follows: ,in, This represents the initial MPC selection probability after preheating. The lowest probability for selecting the MPC control strategy. For the decay step size scale, This refers to the duration of the preheating phase.

[0067] When the hybrid strategy determines that the DDQN agent should output the control action, the system further adopts... - A greedy strategy makes an online trade-off between environmental exploration and value exploitation. The exploration probability, which decays monotonically over time, is defined as: ,in, This represents the exploration probability at the k-th decision moment; This represents the initial exploration probability after the warm-up phase ends; This represents the minimum probability of exploration. Indicates the duration of the preheating phase; The decay step size represents the probability of exploration. k This represents the current decision-making moment. The formula indicates that after the warm-up phase, as the decision-making moment increases, the exploration probability increases from the initial value... Gradually decay to minimum value This allows the agent to retain the necessary random exploration ability in the early stages of training, and gradually enhance its utilization of the evaluation network's output actions in the later stages of training, thereby improving the stability and convergence efficiency of the control strategy.

[0068] The system generates in Uniform random numbers within the interval and Make a judgment: if Execute the control actions output by MPC; if Then the DDQN agent outputs a control action. Furthermore, when the DDQN agent outputs a control action, if... Then, actions are uniformly and randomly selected from the discrete action set to explore the environment; if Then, the evaluation network is used to calculate the action value, and the action that maximizes the action value function is selected as the current control action, that is: ,in, Indicates the first k At each decision moment, the DDQN agent chooses its actions. Represents a set of discrete actions. Indicates the first k The system state at each decision moment This represents the action value function used to evaluate the network output. This indicates the evaluation of network parameters.

[0069] Furthermore, its determination process can be expressed as: by generating in Uniform random numbers within the interval Make a judgment: if The agent then randomly selects actions from a set of discrete actions to explore the environment; if The agent then uses an evaluation network to calculate the value of actions and selects the optimal action that maximizes the action value evaluation function as the current control action, i.e.: ,in, This represents the action chosen by the DDQN agent at the k-th decision time. Indicates the first k The system state at each decision moment; This represents the action value function used to evaluate the network output. This represents the evaluation of network parameters. This strategy allows the agent to maintain necessary stochastic exploration capabilities in the early stages of training and gradually enhances its utilization of the evaluation network's output actions in the later stages of training, thereby improving the stability and convergence efficiency of the control strategy.

[0070] Through the aforementioned phased scheduling and probability decay mechanism, the system ensured that the early stages of training were conducted within a safe boundary, and that the later stages of training were a smooth transition from strong initial safety guidance to autonomous learning and optimization.

[0071] Step 7: Dynamically send actual control actions to the entrance of the controllable fence facility to adaptively adjust the exit passenger flow rate, thereby controlling the crowd density at the bottleneck location and the entrance area of ​​the buffer zone. The system state transition and passenger flow movement status are fed back to the control algorithm in real time to calculate the immediate reward and store it in the experience playback pool. The parameters of the DDQN evaluation network and the target network are updated and iterated online using the experience playback mechanism and the time-series differential target error to realize the iterative optimization and adaptive evolution of the control strategy.

[0072] The system sends the final selected control actions to the controllable fence entrance actuator of the subway station's passenger flow buffer zone. By adjusting the entrance opening and closing mode, it adaptively adjusts the exit passenger flow rate, thereby achieving closed-loop control of the crowd density in the gate and buffer zone entrance areas. During the execution of control commands, the system acquires state transitions and passenger flow movement status feedback in real time through a simulation interaction channel, and processes the experience tuples generated at each interaction step. Store in the experience replay pool Among them This is the round termination marker.

[0073] During the parameter update phase, the deep dual-Q network calculates the temporal difference (TD) target error using sampled mini-batch samples, and its formula is defined as follows: ,in, This represents the index of the transferred sample sampled from the experience replay pool. For the first Instant rewards for each sample This is a round termination flag used to indicate whether the simulation round has ended; if the state at the next moment... If it is a terminated state, then ,otherwise , and These represent the current state and the state at the next moment, respectively. In the state The action to be performed This represents the action space of the controllable fence discrete operation mode of the passenger flow buffer zone. Indicates the state Lower Action Space Candidate actions in To evaluate network parameters, These are the target network parameters.

[0074] To improve the stability of the training system against random perturbations and extreme samples, the system employs a dynamic adaptive Huber loss function to handle the aforementioned errors. ,in, This is the cropping threshold. When the error is within the threshold... When the threshold is within the range, a quadratic term is used, and when the threshold is exceeded, a linear term is used, thereby improving the training stability of the population system under random perturbations and extreme samples.

[0075] Specific loss items exist Time calculation is ,exist Time calculation is .

[0076] Subsequently, the parameters were updated. In the case of mini-batch training, the overall objective function of the batch loss is defined as follows: ,in, Indicates about parameters Batch loss function; Indicates the number of samples contained in a batch; Indicates the first l The temporal difference error corresponding to each transferred sample; Indicates the first l The loss function value corresponding to each sample. And based on the batch loss function, its parameters are calculated. The gradient provides a basis for subsequent parameter updates.

[0077] Its gradient formula is: ,in, g Represents the loss function Regarding parameters gradient, Indicates the parameter The gradient calculation operator. Global norm pruning is used to prevent gradient explosion.

[0078] Finally, the system incorporates the Adam optimizer to adaptively adjust the learning step size based on first-order and second-order moment estimates, thus evaluating the network parameters. Perform online updates and periodically synchronize parameters to the target network according to a preset number of steps. This enables iterative optimization and adaptive evolution of the control strategy.

[0079] Step 8: Construct a comprehensive evaluation index system for the dynamic control scheme of passenger flow buffer zones, which includes four dimensions: congestion risk, passage end time, number of switching times, and density difference, to assess the congestion status of key areas.

[0080] To quantitatively evaluate the effect of dynamic control schemes for passenger flow buffer zones on congestion in key areas, a comprehensive evaluation index system was constructed, which includes four dimensions: congestion risk, passage end time, number of switching operations, and density difference.

[0081] The congestion risk indicator uses a tiered risk penalty function. To depict: when density No penalty is imposed when the risk level is in the low-risk zone; penalties are imposed when the risk level enters the medium-risk zone. Time-linear penalty, entering the high-risk zone A quadratic penalty is applied to mitigate the overall congestion risk. The system is derived by accumulating and weighting the risk levels of gate area A and buffer area B over time. This system comprehensively reflects the combined performance of different control strategies in terms of safety control, traffic efficiency loss, implementation stability, and dual-area equalization.

[0082] Tiered risk penalty function Defined as: ,in, and These are the thresholds for medium-risk and high-risk zones, respectively.

[0083] For time-varying passenger flow density, the system performs risk classification assessment, defining the risk penalty function for the w-th output at time k as follows: When passenger flow density When, the penalty value is 0; when At that time, the penalty value is ;when At that time, the penalty value is Based on this, the system calculates comprehensive congestion risk through time accumulation and regional weighting, defined as follows: This is used to reflect the overall risk level under the significance of dual-regional collaboration, among which, This represents the overall congestion risk function. The smaller the value, the lower the overall risk of overcrowding; and These represent the weight coefficients corresponding to the first and second key regions, respectively; N represents the number of sampling steps in the evaluation time domain. and They represent the first k Passenger flow density in the first and second critical areas at each sampling time; This represents the graded risk penalty function corresponding to passenger flow density; Indicates the sampling time interval between adjacent sampling times; This indicates that the travel time contribution at each sampling time point is accumulated over the entire evaluation time domain. When and A larger value indicates a higher cost of passage in the corresponding area.

[0084] The travel time contributions of the two key areas at each sampling time are weighted and accumulated, and this is used to measure the travel efficiency level of the system in the entire evaluation time domain.

[0085] Regarding traffic efficiency, the system calculates the time it takes for pedestrians to completely cross different areas, defining the average crossing time index as follows: ,in and These represent the end times when all pedestrians have passed through the gate entrance area and the buffer zone entrance area, respectively.

[0086] In terms of control stability, the system directly incorporates the discrete switching number of the control input fence opening and closing combination mode into the evaluation. The fewer the switching number, the more stable the control command and the smaller the disturbance caused by pedestrian path reselection.

[0087] In terms of spatial load balancing, the system extracts the output density difference between different key areas. As an evaluation indicator, by monitoring the fluctuations and concentration of this difference, the system can quantify the balance of passenger flow distribution across areas, thereby effectively identifying and suppressing the spatial bias and congestion risks of high-density crowds.

[0088] This embodiment proposes a reinforcement learning-based intelligent regulation framework guided by model predictive control (MPC). First, considering the nonlinear, time-varying, strongly coupled, and randomly disturbed characteristics of passenger flow evolution in buffer zones, a dynamic passenger flow prediction model suitable for multi-step prediction is established using a system identification method. Evaluation metrics for measuring the model's accuracy and generalization ability are provided. Subsequently, the dynamic passenger flow prediction model is used as the prediction model for MPC, and a two-layer control framework integrating MPC and DDQN for buffer zone facilities is constructed, such as... Figure 2As shown in the figure, MPC performs online rolling optimization under constraints and outputs candidate strategies for buffer control that meet feasibility and safety requirements; the DDQN agent learns the optimal strategy by interacting with the environment and adaptively corrects the impact of model errors and changes in operating conditions through its inherent compensation mechanism.

[0089] Example 2 In the specific implementation scenario modeling, this embodiment takes a subway station as the object and constructs a full-size simulation model including the concourse level, platform level, and main passages based on pedestrian dynamics simulation software. The specific scene layout and passenger flow buffer structure design are shown in the figure below. Figure 3 As shown. The buffer zone has a channel-shaped center and S-shaped sides, with an area of ​​72 m². 2 The buffer zone is 10 m long and 7.2 m wide, with an internal corner passage width of 1.2 m. Different colored arrows in the diagram represent the main flow patterns that pedestrians may form after entering the buffer zone from different entrances. Two key observation areas, Area A and Area B, are defined: ① Area A: The gate entrance area, which is prone to queuing and congestion in front of the gates and is a typical congestion risk point within the station; ② Area B: The buffer zone entrance area, which is significantly affected by the opening and closing of the buffer zone entrance and passenger flow dynamics. It is prone to passenger accumulation and passageway blockage under inappropriate control, and is a potential new bottleneck area after the introduction of the buffer zone. This is achieved by adjusting the four controllable barriers at the buffer zone entrance. , , , The system can dynamically adjust the release rhythm of the buffer zone exit.

[0090] The system identification model's fit validation performance on the validation dataset is as follows: Figure 4 -Figure 7 and Figures 8-11 As shown in the figure, the density of the entrance area of ​​the entry gate is displayed. Density of passenger flow buffer zone entrance area Performance under 1-step, 20-step, 40-step, and 60-step prediction conditions. It can be observed that under the 1-step prediction condition, all models can track output changes well, and the performance differences between models are not significant. This indicates that for short-term prediction, the main dynamic characteristics of the system can be captured by multiple model structures, and the impact of structural differences on prediction accuracy is limited. However, as the prediction step number increases further from 20 steps to 40 and then 60 steps, the performance differences between different models gradually become apparent. ARX and ARMAX-like multinomial models accumulate errors faster in multi-step prediction, their ability to track local peaks and troughs decreases, and their long-term prediction stability is relatively insufficient. The NLARX model performs well in short-term prediction, but its prediction performance declines more significantly as the prediction step number increases, manifested as worse fitting of local dynamic segments and faster decay of the fit. The SSM model shows more overall stability under multi-step prediction, maintaining trend consistency and dynamic pattern continuity well. The fit decreases relatively slowly as the prediction step length increases, demonstrating superior long-term prediction stability. Among them, ARX is an autoregressive model with exogenous input, ARMAX is an autoregressive moving average model with exogenous input, NLARX is a nonlinear autoregressive model with exogenous input, and SSM is a state space model.

[0091] The effects of different control methods on passenger flow management after being applied to buffer facilities, for example Figure 12 and Figure 13 As shown, the dynamic evolution of passenger flow density in regions A and B, along with the corresponding heatmaps, are illustrated. Experimental results demonstrate that all three buffer control strategies can suppress passenger flow density fluctuations in both regions to a certain extent, keeping the overall system operation within an acceptable range and effectively reflecting the characteristics of random fluctuations and periodic surges in passenger flow in real-world scenarios. Further comparison reveals that the MPC-guided DDQN hybrid strategy proposed in this paper can more effectively maintain passenger flow density in the two key regions within low and medium-risk ranges for most of the time period. It also exhibits stronger peak-suppression capabilities and smaller exceedance amplitudes during periods of rising passenger flow density, demonstrating superior dual-region coordinated control effects, particularly in locally amplified regions.

[0092] To quantify the feasibility and stability of the evaluation strategy, this embodiment statistically analyzed the switching characteristics of the discrete operation modes of the fence. Specific results are as follows: Figure 14As shown in the figure, statistical data indicates that the MPC-guided DDQN method proposed in this application performs best in terms of the number of switching events, with a 63.32% reduction in mode switching frequency compared to traditional MPC and a 40.48% reduction compared to traditional DDQN. This result demonstrates that the MPC-guided DDQN method is more stable at the decision-making level, significantly reducing frequent jumps in discrete instructions while ensuring control effectiveness, thereby improving the executability of the strategy and the stability of system operation. Furthermore, the MPC-guided DDQN method maintains a relatively higher number of times in mode 1. Considering that the organizational method corresponding to mode 1 is simpler, more stable, and requires less intervention, it helps maintain the continuity of pedestrian flow and reduces the loss of traffic efficiency caused by frequent adjustments.

[0093] The distribution of regional passenger flow density difference in the embodiments of this application compared with the fixed mode is as follows: Figure 15 As shown, the concentration of the difference distribution directly reflects the control strategy's ability to manage spatial load balancing. Figure 15 It can be observed that, by comparing fixed buffer modes 1-3, a clear conclusion can be drawn: Mode 1... The fact that the overall passenger flow density is on the same order of magnitude as under the closed-loop control strategy indicates that under Mode 1, the passenger flow density in the two areas is more likely to change synchronously, and the buffer zone's effect on the passenger flow density at the gate entrance area is more inclined to smooth and distribute the load. However, under Mode 2, the difference in passenger flow density between the two areas increases significantly, suggesting that this mode may exacerbate the imbalance of passenger flow load between the two areas within a certain period; especially under Mode 3, its... The significantly increased and more dispersed passenger flow indicates a strong divergence in passenger density between the two areas, with a more pronounced density shift or accumulation effect. The system sacrifices passenger density at the entrance of the buffer zone to improve passenger density at another entrance gate, thus creating a new high-risk point. Therefore, the introduction of buffer zones must be coupled with reasonable control strategies; otherwise, passenger density imbalances or even congestion shifts can easily occur. Compared to the fixed mode, under the three closed-loop control strategies... The overall level is low and the fluctuation range is relatively concentrated, indicating that these control strategies can maintain a relative balance of passenger flow density in the two areas under dynamic fluctuations in passenger flow, avoiding significant imbalances in passenger flow density caused by improper facility regulation. Among them, the passenger flow density difference distribution under the MPC-guided DDQN method is more robust.

[0094] Finally, the evolution trend and convergence characteristics of the cumulative reward value during reinforcement learning training are as follows: Figure 16As shown in the figure, by comparing the training curves and the box plots of the final reward levels, it can be found that the MPC-guided DDQN method exhibits more significant advantages than the traditional DDQN method, mainly in three aspects: the reward level in the early stage of training, the faster convergence trend, and the smaller reward within the same number of rounds. First, in the early stage of training, the reward of MPC-guided DDQN is significantly higher than that of traditional DDQN. This is because the control method designed in this paper introduces MPC as the baseline policy at the beginning of the learning stage. The agent does not explore a large amount of action space from a completely random perspective, but rather imitates and explores under constraints based on the feasible solutions given by MPC. This allows for the generation of a more reasonable control sequence from the initial stage, avoiding the blind trial and error common in the early stages of traditional DDQN. Second, in terms of the trend of reward evolution with rounds, the reward of the MPC-guided DDQN method increases faster, the rise process is smoother, and it exhibits better training efficiency and convergence characteristics. In contrast, traditional DDQN maintains a large fluctuation range during a long training period, indicating that its policy update relies more on the sample accumulation brought by random exploration, resulting in relatively low learning efficiency and a greater tendency for training instability.

[0095] Furthermore, at the end of the 400th training round, the final reward level of the MPC-guided DDQN is still significantly better than that of the traditional DDQN, and its fluctuation amplitude is smaller in the magnified region, maintaining a higher reward range. The box plot corresponding to the magnified region shows that the average reward of the MPC-guided DDQN method in the last 100 rounds is smaller than that of DDQN in the same number of rounds, indicating superior control performance and verifying the correctness of the control results and evaluation metrics. Given a training budget, the MPC-guided DDQN can learn a better and more stable control policy, achieving better long-term cumulative returns. It should be noted that as the number of training rounds increases further, the reward of the traditional DDQN may continue to improve and gradually approach a better level. However, in practical engineering applications, training costs and trial-and-error risks are often limited; therefore, the MPC-guided DDQN, which achieves higher rewards and faster convergence with the same number of training rounds, has greater practical significance and application value.

[0096] The above embodiments are merely illustrative of the structural concept and features of the present invention, intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly, and should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made based on the essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for intelligent regulation of buffer zones in densely populated situations, characterized in that, Includes the following steps: Step 1: Use pedestrian dynamics simulation software to model passenger flow in densely populated areas. Based on the terrain characteristics of densely populated areas, set up passenger flow buffer zones with specific flow guidance structures composed of fences in the upstream area at bottleneck locations. Determine the bottleneck locations and passenger flow buffer zones as key observation areas. Step 2: Using a data-driven system identification approach, learn the dynamic mapping relationship between facility control input and passenger flow density response in key areas, apply multiple sets of discrete operation mode excitations to the fence facility, collect control inputs at each sampling time and passenger flow density outputs in key observation areas, and construct a single-input-multiple-output discrete-time state-space prediction dynamic identification model (SSM) for multi-step prediction. Step 3: At each sampling moment, the current passenger flow density measurement value and the discrete operation mode executed at the previous moment are obtained in real time, a state estimator is constructed, and based on the SSM, the measurement output information is fed back to the state update process to recursively update and correct the internal state of the system. Step 4: Construct a Model Predictive Controller (MPC) that includes a single-sided target tracking term for passenger flow density, a penalty term for changes in control input, a penalty term for soft constraint out-of-bounds behavior, and a dual-region consistency term. Use the identified SSM (Site Stream Management System) as the internal predictive model of the MPC, and the discrete opening and closing combination mode of the buffer zone entrance facility as the control decision variable. Based on the updated internal state of the system, in the prediction time domain... The system performs multi-step rolling prediction of future outputs and solves for the optimal control sequence under constraints. Step 5: Using an approximate Markov decision process with feature extension, construct a deep dual-Q network reinforcement learning controller (DDQN) agent that integrates MPC prediction information. Introduce future prediction terms and use the internally built SSM to calculate future predictions. If the risk of exceeding the target or going out of bounds occurs within a step, evaluate the network output to determine the optimal action in the current state; Step 6: During the warm-up phase, the control input generated by the MPC is used as the actual control action to provide the agent with stable decision reference and high-quality interaction samples. After the warm-up phase, the candidate control action output by the MPC is selected according to the probability decaying over time, and the control action output by the DDQN is selected according to the complementary probability to complete the gradual switch from the initial strong security guidance to the mid-to-late stage autonomous learning optimization. Step 7: Dynamically send actual control actions to the entrance of the controllable fence facility to adaptively adjust the exit passenger flow rate, thereby controlling the crowd density at the bottleneck location and the entrance area of ​​the buffer zone. The system state transition and passenger flow movement status are fed back to the control algorithm in real time to calculate the immediate reward and store it in the experience playback pool. The parameters of the DDQN evaluation network and the target network are updated and iterated online using the experience playback mechanism and the time-series differential target error to realize the iterative optimization and adaptive evolution of the control strategy.

2. The intelligent control method for buffer zones in densely populated situations according to claim 1, characterized in that, In step 2, a pseudo-random binary sequence (PRBS) is used as the control input signal for the system identification experiment. The structure of the SSM is as follows: ， ， in, For discrete-time indexing, Let be the system state vector. The control input mode for opening and closing the entrance to the passenger flow buffer zone. Indicates the system at the 1st Output at any moment The state space matrix, To estimate the gain for the state, This is the noise disturbance term.

3. The intelligent control method for buffer zones in densely populated situations according to claim 2, characterized in that, In step 3, a state estimator is constructed to recursively update and correct the internal state of the system using real-time passenger flow density measurements. At each sampling time, given the control input from the previous time step... Measurement output at the current time In this case, the SSM identified by the system includes: Step 3-1: Given the control input at the previous time step and the measured output at the current time step, calculate the one-step predicted output based on the state-space identification model as follows: ,in, Indicates based on Known information at any given time The one-step prediction value output at each time step. Indicates based on Information obtained at a given time One-step prediction of state at time step. express Constantly control input; Step 3-2: Define the error term and extract the prediction error of the current time step as follows: ,in, express Output the prediction error at all times. express Actual measured output at any given moment; Step 3-3: Update the state using a linear correction method, feeding the measurement output information back into the state update process to obtain the state estimate for the next time step: .

4. The intelligent control method for buffer zones in densely populated situations according to claim 3, characterized in that, Step 4 includes: Step 4-1: Keep the input outside the control time domain constant, and assume the prediction time domain is... Control time domain is The stacked prediction matrix for model predictive control is constructed as follows: ,in, This represents the stacked output vector in the prediction time domain. Represents the control sequence vector within the control time domain. The matrix is ​​a prediction matrix from state to output. For control gain matrix; Step 4-2: In the rolling optimization solution process, the multi-objective cost function is defined as: ,in, For passenger flow density, a one-sided target tracking item, To control the penalty term for input changes, This is a soft constraint out-of-bounds penalty item. This is a term with consistent density across two regions; Step 4-3: The controller explicitly introduces hard constraints, including input increments, while optimizing the cost function. Output soft constraints and output difference soft constraints Multiple types of constraints; Step 4-4: Predict future time domain based on current state estimation The system output is used to solve the optimal control sequence online under the various constraints, and the first control variable in the sequence is applied to the fence of the buffer zone to complete the rolling optimization and closed-loop control of the system.

5. The intelligent control method for buffer zones in densely populated situations according to claim 3, characterized in that, Step 5 includes: Step 5-1: Extract the state-space representation of the system's current features containing information about delayed actions, and define an approximate Markov decision process and reinforcement learning framework. ,in, and Normalized passenger flow density for key areas The action performed in the previous moment; Step 5-2: Define the time frame The initial cumulative discount reward is: The corresponding optimal action value function satisfies the Bellman optimality equation as follows: ,in, This represents the cumulative discount reward obtained starting from time k. This represents the immediate reward obtained at step k. Indicates the discount factor. The optimal action value function is used to characterize the state. s Take action below a The maximum expected cumulative discount return that can be obtained by subsequently following the optimal strategy; s Indicates the current state. a Indicates the current action. Indicates the next state. This represents a candidate action for the next state. This represents the immediate reward obtained after performing the current action. Indicates that given the current state s and actions a Conditional expectation under given conditions This indicates the value of the candidate action that maximizes long-term benefits in the next state. Step 5-3: By weighting and summing the five sub-indicators, a composite reward function integrating the fusion model prediction control cost function structure and the prediction model information is constructed as follows: ,in, Indicates the first k Total reward obtained in each step , , , and These represent the weights of the target tracking term, the safety soft constraint term, the dual-region equilibrium consistency term, the action smoothing term, and the future prediction term, respectively. The target tracking reward is used to reflect the degree of deviation of the system output from the target threshold. This indicates that soft-constraint rewards are used to penalize high-risk congestion states that exceed the safety upper bound. The dual-region equilibrium consistency reward is used to suppress excessive differences in passenger flow density between two key regions. This indicates that the smoothness reward is used to suppress frequent switching of control actions between adjacent time steps. Indicates the first k Always looking towards the future The reward for predicting the future is obtained by making predictions step by step.

6. The intelligent control method for buffer zones in densely populated situations according to claim 3, characterized in that, Step 6 includes: Step 6-1: When the number of rounds is... During the warm-up phase, the actual actions are performed using the control input generated by the model predictive controller, defined as... , , Indicates the system at the 1st k The control actions actually performed at each decision-making moment The model predicts the controller in the first... k The control action output at each decision moment; Step 6-2: When the number of rounds is... After the preheating phase, control is allocated between MPC and DDQN based on the probability decaying over time during annealing. At each decision moment, the system allocates control based on probability. Choose to execute MPC actions based on probability. The scheduling selection probability function of MPC is defined as follows: (The original text contains several typographical errors and inconsistencies, making a precise translation difficult. A more accurate translation would require the full context.) ,in, The initial control selection probability after preheating. The minimum probability is preset. To determine the step size for the decay rate; Step 6-3: When the hybrid strategy determines that the DDQN agent should output the control action, it adopts... - The greedy strategy makes an online trade-off between environmental exploration and value exploitation, defining the exploration probability as monotonically decaying over time: ,in, This represents the exploration probability at the k-th decision time. This represents the initial exploration probability after the warm-up phase ends. This represents the minimum probability of exploration. Indicates the duration of the preheating phase. The decay step size represents the probability of exploration. k Indicates the current decision-making moment; Step 6-4: By generating in Uniform random numbers within the interval and Make a judgment: if Then the control action output by the model predictive controller will be executed. ;like Then the deep dual-Q network agent outputs the control action.

7. The intelligent control method for buffer zones in densely populated situations according to claim 6, characterized in that, In step 6-4, when the deep dual-Q network agent outputs a control action, if Then, actions are uniformly and randomly selected from the discrete action set A to explore the environment; if Then, the evaluation network is used to calculate the action value, and the optimal action that maximizes the action value evaluation function is selected as the current control action, that is: ,in, This represents the action chosen by the DDQN agent at the k-th decision time. Indicates the first k The system state at each decision moment This represents the action value function used to evaluate the network output. This indicates the evaluation of network parameters.

8. The intelligent control method for buffer zones in densely populated situations according to claim 6, characterized in that, Step 7 includes: Step 7-1: Obtain state transitions and passenger flow status feedback in real time through the simulation interaction channel, and convert the experience tuples generated at each interaction step into a real-time state transition and passenger flow status feedback. Store in the experience replay pool Among them This is the round end marker; Step 7-2: The deep dual-Q network calculates the temporal difference TD target error using sampled mini-batch samples. The target error is... ,in, This represents the index of the transferred sample sampled from the experience replay pool. For the first Instant rewards for each sample This is the end-of-round flag, indicating the state at the next moment. If it is a terminated state, then ,otherwise , and These represent the current state and the state at the next moment, respectively. In the state The action to be performed This represents the action space of the controllable fence discrete operation mode of the passenger flow buffer zone. Indicates the state Lower Action Space Candidate actions in To evaluate network parameters, For target network parameters; Step 7-3: Employ the dynamic adaptive Huber loss function To handle errors and improve the stability of training systems against random perturbations and extreme samples in the target population, ,in, This is the clipping threshold; Step 7-4: In the case of mini-batch training, the overall objective function of the batch loss is defined as follows: ,in, Indicates about parameters Batch loss function, This indicates the number of samples contained in a batch. Indicates the first l The temporal difference error corresponding to each transferred sample. Indicates the first l The loss function value corresponding to each sample; Step 7-5: Combine the Adam optimizer with adaptive adjustment of the learning step size based on the first and second moment estimates to evaluate the network parameters. Perform online updates and periodically synchronize parameters to the target network according to a preset number of steps. This enables iterative optimization and adaptive evolution of control strategies.

9. The intelligent control method for buffer zones in densely populated situations according to claim 1, characterized in that, Also includes: Step 8: Construct a comprehensive evaluation index system for the dynamic control scheme of passenger flow buffer zones, which includes four dimensions: congestion risk, passage end time, number of switching times, and density difference, to assess the congestion status of key areas.

10. The intelligent control method for buffer zones in densely populated situations according to claim 9, characterized in that, The congestion risk index simultaneously characterizes congestion intensity and congestion duration. For time-varying passenger flow density, a graded risk function is used to assess passenger flow status. Step 8 includes: Step 8-1: Define the hierarchical risk penalty function for the w-th output at time k as follows: When passenger flow density When, the penalty value is 0; when At that time, the penalty value is ;when At that time, the penalty value is ; Step 8-2: Calculate the comprehensive congestion risk by combining time accumulation and regional weighting, and define it as follows: ,in, This represents the overall congestion risk function. and Let represent the weight coefficients corresponding to the first and second key regions, respectively, and N represent the number of sampling steps in the evaluation time domain. and They represent the first k Passenger flow density in the first and second key areas at each sampling time. This represents the graded risk penalty function corresponding to passenger flow density. This represents the sampling time interval between adjacent sampling moments. This indicates that the travel time contribution at each sampling time point is accumulated over the entire evaluation time domain; Step 8-3: Calculate the time it takes for pedestrians to completely cross different areas, and define the average crossing time index as... ,in and These represent the end times when all pedestrians have passed through the gate entrance area and the buffer zone entrance area, respectively. Step 8-4: Directly incorporate the discrete switching number of the control input fence opening and closing combination mode into the evaluation. The fewer the switching number, the more stable the control command and the smaller the disturbance caused by pedestrian path reselection. Step 8-5: Extract the output density difference of different key regions As an evaluation indicator, it monitors the fluctuations and concentration of density differences, quantifies the balance of passenger flow distribution in different areas, and identifies and suppresses the spatial bias and congestion transfer risks of high-density populations.

Images