A self-feedback passenger flow control method and device based on deep reinforcement learning

Abstract

The present application provides a self-feedback passenger flow control method and device based on deep reinforcement learning. The self-feedback passenger flow control method based on deep reinforcement learning includes: obtaining the historical time series data, simulation data and short-term prediction data of the station at the current time point; obtaining passenger flow characteristics based on the historical time series data, simulation data and short-term prediction data; obtaining the station facility capacity load evaluation based on the passenger flow characteristics; obtaining the passenger flow control strategy, which includes a variety of control behaviors; updating the current station passenger flow control decision plan based on the station facility capacity load evaluation and the passenger flow control strategy. The present application evaluates the station facility capacity load at the current time point based on the historical time series data, simulation data and short-term prediction data of the passenger flow. On the one hand, it can understand the capacity utilization of the key facilities and equipment of the current station, and on the other hand, it can update the intelligent adjustment of the station passenger flow control decision plan based on the feedback of the capacity utilization of the key facilities and equipment of the current station.

Description

Technical Field

[0001] The present application belongs to the field of rail transit technology, and in particular relates to a self-feedback passenger flow control method based on deep reinforcement learning and a self-feedback passenger flow control device based on deep reinforcement learning. Background Art

[0002] my country's rail transit industry is in a period of rapid development. The scale of existing lines and high-speed rail networks continues to expand, but the phenomenon of congestion at rail stations is prominent, which often leads to excessive congestion in stations, resulting in the failure to exert the design capacity, and also seriously affecting station safety and passenger satisfaction, which has attracted great attention from the government and the public at rail stations.

[0003] At present, the selection of flow control sites, the setting of flow control time range and flow control intensity in operation management mainly rely on the on-site work experience of the management department, and lack of reasonable scientific explanation and verification. When encountering sudden large passenger flows caused by large-scale activities, abnormal climate and emergencies, due to the lack of effective passenger flow monitoring means, passenger flow trend forecasting and early warning assessment technical means, there is also a lack of clear and appropriate handling strategies, which makes it easy to miss the best handling time, increase congestion and create the risk of passenger stampede.

[0004] Therefore, it is desired to have a technical solution to overcome or at least alleviate at least one of the above-mentioned defects of the prior art. Summary of the invention

[0005] The purpose of this application is to provide a self-feedback passenger flow control method based on deep reinforcement learning to solve at least one of the above-mentioned problems.

[0006] In a first aspect of the present application, a self-feedback passenger flow control method based on deep reinforcement learning is provided, wherein the self-feedback passenger flow control method based on deep reinforcement learning comprises:

[0007] Obtain historical time series data, simulation data, and short-term prediction data for the current time point of the station;

[0008] Acquire passenger flow characteristics according to the historical time series data, simulation data and short-term prediction data;

[0009] Obtaining a station passenger flow score according to the passenger flow characteristics;

[0010] Acquire a passenger flow control strategy, wherein the passenger flow control strategy includes multiple control behaviors;

[0011] The station passenger flow control decision plan at the current time point is updated according to the station passenger flow score and the passenger flow control strategy.

[0012] Optionally, the obtaining of historical time series data, simulation data and short-term prediction data of the current time point of the station includes:

[0013] Get the historical time series data of the station at the current time point;

[0014] Acquire simulation data of the station at the current time point according to the historical time series data;

[0015] Get the feedback processing result of the previous time point of the station's current time point;

[0016] The short-term prediction data of the station at the current time point is obtained according to the simulation data and the feedback processing result.

[0017] Optionally, the acquiring of historical time series data of the current time point of the station includes:

[0018] Get image information;

[0019] Obtain pedestrian trajectory information based on image information;

[0020] The historical time series data is counted according to the pedestrian trajectory information.

[0021] Optionally, the acquiring simulation data of the current time point of the station according to the historical time series data includes:

[0022] Generate station model;

[0023] Generate process logic;

[0024] Draw a logical flow chart based on process logic;

[0025] Generate simulation control logic according to the logic flow chart;

[0026] The station model is simulated according to the simulation control logic and the historical time series data, so as to obtain simulation data at the current time point.

[0027] Optionally, the acquiring the short-term prediction data of the current time point of the station according to the simulation data and the feedback processing result includes:

[0028] Build LSTM model;

[0029] The simulation data and the feedback processing result are input into the LSTM model and the output information of the LSTM model is obtained as short-term prediction data.

[0030] Optionally, acquiring passenger flow characteristics according to the historical time series data, simulation data and short-term prediction data includes:

[0031] The features of the historical time series data, simulation data and short-term prediction data are extracted as passenger flow features through the deep learning network SAE.

[0032] Optionally, obtaining a station passenger flow score according to the passenger flow characteristics includes:

[0033] The station passenger flow score is obtained by the following formula:

[0034] in,

[0035] v is the evaluation result, and the passenger flow characteristic value is z, z max is the flow rate and velocity standard; M is the penalty value.

[0036] Optionally, obtaining a station passenger flow control decision plan at a current time point according to the station passenger flow score and the passenger flow control strategy includes:

[0037] Obtaining a state-action transition probability, where the state-action transition probability represents a probability of performing a corresponding control action according to the station passenger flow score at a current time point;

[0038] Obtaining an action-state transition probability, wherein the action-state transition probability represents a probability that the score changes within a predetermined time after performing a corresponding control action;

[0039] Generate station passenger flow control decision optimization strategy;

[0040] The current station passenger flow control decision plan is updated according to the state-action transition probability, the action-state transition probability and the station passenger flow control decision optimization strategy.

[0041] The present application also provides a self-feedback passenger flow control device based on deep reinforcement learning, and the self-feedback passenger flow control device based on deep reinforcement learning includes:

[0042] A data acquisition module, the data acquisition module is used to obtain historical time series data, simulation data and short-term prediction data of the station at the current time point;

[0043] A passenger flow feature extraction module, the passenger flow feature extraction module is used to obtain passenger flow features according to the historical time series data, simulation data and short-term prediction data;

[0044] A station passenger flow scoring module, the station passenger flow scoring module is used to obtain a station passenger flow score according to the passenger flow characteristics;

[0045] A passenger flow control strategy acquisition module, the passenger flow control strategy acquisition module is used to acquire a passenger flow control strategy, the passenger flow control strategy includes a variety of control behaviors;

[0046] An updating module, wherein the updating module is used to update the current station passenger flow control decision plan according to the station passenger flow score and the passenger flow control strategy.

[0047] The present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, the above-mentioned self-feedback passenger flow control method based on deep reinforcement learning is implemented.

[0048] The present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement the above-mentioned self-feedback passenger flow control method based on deep reinforcement learning.

[0049] This application has at least the following beneficial technical effects:

[0050] This application scores the station passenger flow at the current time point based on historical time series data, simulation data and short-term prediction data. On the one hand, it can understand the current station passenger flow situation, and on the other hand, it can intelligently adjust the station passenger flow control decision plan in real time according to the current station passenger flow situation. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a flow chart of a self-feedback passenger flow control method based on deep reinforcement learning provided in one embodiment of the present application.

[0052] Figure 2 It is an exemplary structural diagram of an electronic device that can implement the self-feedback passenger flow control method based on deep reinforcement learning provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical scheme and advantages of the implementation of this application clearer, the technical scheme in the embodiment of this application will be described in more detail below in conjunction with the drawings in the embodiment of this application. In the drawings, the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The described embodiments are part of the embodiments of this application, not all of them. The embodiments described below with reference to the drawings are exemplary and are intended to be used to explain this application, and should not be construed as limitations on this application. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. The embodiments of this application are described in detail below in conjunction with the drawings.

[0054] It should be noted that, in the description of the present invention, the terms “first” and “second” are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0055] Figure 1 It is a flowchart of an image tag acquisition method provided by one embodiment of the present application.

[0056] like Figure 1 The self-feedback passenger flow control method based on deep reinforcement learning shown includes:

[0057] Step 1: Obtain the historical time series data, simulation data and short-term prediction data of the passenger flow at the current time point of the station;

[0058] Step 2: Obtain passenger flow characteristics based on historical passenger flow time series data, simulation data, and short-term forecast data;

[0059] Step 3: Obtain a comprehensive evaluation of the station facility capacity load based on passenger flow characteristics;

[0060] Step 4: Obtain a passenger flow control strategy, which includes multiple control behaviors;

[0061] Step 5: Update the current station passenger flow control decision plan based on the comprehensive evaluation of station facility capacity load and passenger flow control strategy.

[0062] This application conducts a comprehensive evaluation of the station facility capacity load at the current time point based on the historical passenger flow time series data, simulation data and short-term forecast data. On the one hand, it can understand the current capacity utilization of key station facilities and equipment, and on the other hand, it can update the intelligent adjustment of the station passenger flow control decision plan based on the feedback of the current capacity utilization of key station facilities and equipment.

[0063] In this embodiment, step 1: obtaining the historical time series data, simulation data and short-term prediction data of the passenger flow at the current time point of the station includes:

[0064] Step 11: Obtain the historical time series data of the station at the current time point;

[0065] Step 12: Obtain simulation data of the current time point of the station based on historical time series data;

[0066] Step 13: Obtain the feedback processing result of the previous time point of the current time point of the station;

[0067] Step 14: Obtain the short-term prediction data of the station at the current time point based on the simulation data and the feedback processing results.

[0068] In this embodiment, step 11: obtaining the historical time series data of passenger flow at the current time point of the station includes:

[0069] Step 111: Obtain image information;

[0070] Step 112: Obtain pedestrian trajectory information according to the image information;

[0071] Step 113: Count historical time series data based on pedestrian trajectory information.

[0072] For example, image information can be obtained in the following ways:

[0073] The original images are collected by cameras at key points in the station (including: entrance and exit gates and security checkpoints, platforms, transfer passages such as stairs, walkways and escalators). These cameras should be installed as high as possible, their shooting angles should be as close to right angles as possible, and they should have good resistance to light interference. All collected images are converted into grayscale images for processing.

[0074] The following methods can be used to obtain pedestrian trajectory information based on image information:

[0075] Random Hough transform is used to identify and distinguish the dark, nearly circular area in the image that is close to the size of a human head, and the fitting circle of the passenger's head in the station is obtained, and the trajectory of its center position is tracked. By analyzing the movement trajectory of passengers, they are classified into boarding trajectory, alighting trajectory and transfer trajectory in various directions. The analysis method is mainly to determine whether the pedestrian movement trajectory passes through the calibrated boundary in the scene, that is, to determine whether the line connecting the starting point and the end point of the pedestrian trajectory intersects with the boundary, so as to count the pedestrian flow.

[0076] The specific method of counting historical time series data according to the pedestrian trajectory information is as follows: counting pedestrian flow and calculating flow speed according to pedestrian flow and trajectory, and storing the result data in a database as historical time series data.

[0077] In this embodiment, step 12: obtaining simulation data of the current time point of the station according to the historical time series data includes:

[0078] Step 121: Generate a station model.

[0079] For example, before building a station model, you need to import the CAD floor plans of each floor of the station into AnyLogic software as the model base map, and set the scale of the graphics and the scale of the objects. Then draw the obstacles in the picture, such as the boundaries, columns, service windows, security check areas, escalator boundaries and other obstacles in the station, which are implemented using the Wall component. Escalators and other embedded modules can be set directly. Taking escalators as an example, the main setting process of this type of model is:

[0080] Step 1211: Double-click the rectangular area and draw in the escalator area;

[0081] Step 1212: drag in the ped Area Descriptor, and select the dragged in rectangular area in Area;

[0082] Step 1213: If it is an elevator, set Is moving and at speed. If not, only set Throughout.

[0083] Step 1214: If you need to set the slope, select rectangular area and check slope to set it.

[0084] Step 122: Generate process logic. For example, the generated process logic is as follows:

[0085] Step 1221: Design the process logic. AnyLogic uses flowcharts to drive the entire simulation. As a discrete event simulation, subway station pedestrian simulation can be summarized into four sub-steps: generation - destination selection - going - leaving (disappearance). The specific implementation steps are as follows:

[0086] Set the process function implementation objects, including target lines, target areas, travel paths, service points, service queues, etc., and set the names and corresponding attributes. When setting the arrival or departure of pedestrians, set it through the Target Line. Pedestrians will appear or leave from the Target Line. The Target Line is both the entry and exit line of each entrance and exit and the entrance and exit of each escalator; secondly, set service queue lines (Service with Lines) where services are needed, such as gates, security checks, ticket sales and other areas, and then set the relevant attributes of the service and the queue form.

[0087] Step 1222: Prepare a logic flow chart and design the process logic of the following stations based on the actual logic of pedestrians.

[0088] The main logic of pedestrian entry is: pedestrians enter the station (generation) - pedestrians make a choice (direct security check / ticket purchase) - pedestrians who are directly checked receive security check services - all pedestrians receive station gate services.

[0089] The logic for pedestrians to wait for the train is: pedestrians pass through the gate - select the escalator - go to the platform escalator - pass through the station passage - select the platform door - go to the waiting room - board the train (disappears).

[0090] The logic of pedestrian transfer is: pedestrians get off the bus (generate) - pedestrians choose the exit route - pass through the station passage - choose the transfer route - choose the transfer area - go to the waiting area - board the bus (disappear).

[0091] The main logic of pedestrian exit is: pedestrians get off the bus (generate) - pedestrians choose the exit route - pass through the station passage - choose the escalator - choose the exit gate - accept the gate service - go to the exit (disappear).

[0092] Step 123: Draw a logic flow chart according to the process logic; specifically, after designing the process, draw a logic flow chart in combination with the functional objects in the model.

[0093] Step 124: Generate simulation control logic according to the logic flow chart;

[0094] Step 125: Simulate the station model according to the simulation control logic and the historical time series data to obtain simulation data at the current time point.

[0095] In this embodiment, step 14: obtaining the short-term prediction data of the current time point of the station according to the simulation data and the feedback processing result includes:

[0096] Step 141: constructing an LSTM model; specifically, step 141 includes:

[0097] Step 1411: Construct an LSTM model containing three continuous loop structures. Each LSTM layer contains a loop structure. It is sufficient to input only one loop unit into the current layer. Each loop structure has two outputs. The number of self-updates of a single loop structure of the LSTM structure is determined only by the dimension of the input data and the number of loops. That is, the total number of parameters of the current layer LSTM only needs to calculate one loop unit, and does not need to calculate the total number of multiple continuous units, so as to improve the computational efficiency of the system.

[0098] Step 1412: Set the input gate, forget gate, output gate and memory cell state for the LSTM cell.

[0099] Input gate: determines how much of the historical data input at the current moment needs to be saved to the unit state. These historical data are the data results verified by the simulation model mentioned in step S2 and the feedback processing results mentioned in step S6.

[0100] Forget gate: determines how much of the simulation result feedback from the previous moment needs to be retained until the current moment and substituted into the loop for recalculation. The forget gate needs to discard some information so that the time span of each batch of data is the same, and monitor the data scale to prevent the explosion of calculation amount, and set the proportion parameter σ of discarded data (its value range is [0,1]).

[0101] Output gate: Analyze the short-term prediction results of each loop layer of the LSTM system, and decide how much of the current unit state timing information and prediction information needs to be output to the current result or returned to the hidden layer. At the same time, this output is used to update the short-term prediction results stored in the memory unit of this layer.

[0102] Memory unit: ensures that the gradient of the updated optimization is always passed and ensures faster convergence, controlled by a state function.

[0103] Step 1413: Apply the output h of the feedback processing result of the last moment t-1 and the current simulation data input x t , get f through the forget gate t , the formula is as follows:

[0104] f t =σ(W f ·[h t-1 , x t ]+b f )

[0105] Step 1414: Apply the output h of the feedback processing result of the previous moment t-1 and the current simulation data input x t , get i through the input gate t , and obtain the temporary state of the memory unit at the current moment through the unit state The formula is as follows:

[0106] i t =σ(W i ·[h t-1 , x t ]+b i )

[0107]

[0108] Step 1415: Apply the memory cell state C of the previous cell structure t-1 , the forget gate output f of this cycle t , input gate output i t And the output of the memory cell state of this cycle Get the current state of the memory unit C t The formula is as follows:

[0109]

[0110] Step 1416: Apply the output h of the previous moment t-1 and the current data input x t , get o through the output gate t The process, combined with the current cell state C t and t Get the final output h t The formula is as follows:

[0111] o t =σ(W o ·[h t-1 , x t ]+b o )

[0112] h t =o t *tanh(C t )

[0113] Step 142: Input the simulation data and the feedback processing result into the LSTM model and obtain the output information of the LSTM model as short-term prediction data. Specifically, when the LSTM model iteration reaches a reasonable accuracy or a specified iteration number of 2000 generations, the short-term prediction data is obtained, and the result at this moment is retained for the next short-term prediction.

[0114] In this embodiment, step 2: obtaining passenger flow characteristics based on historical time series data, simulation data, and short-term prediction data includes:

[0115] The features of the historical time series data, simulation data and short-term prediction data are extracted as passenger flow features through the deep learning network SAE.

[0116] For example, step 21: select key areas in the station, such as escalators, entrances and exits, and transfer passages, and obtain the above-mentioned historical time series data, simulation data, and short-term prediction data for each different area to form a heterogeneous data source.

[0117] Step 22: To address the inconsistency problem of heterogeneous data sources, use the deep learning network SAE to extract features.

[0118] The network consists of Encoder and Decoder, and its structure is:

[0119] Table 1 Deep learning network structure

[0120]

[0121] The training parameters are set as:

[0122] Table 2 SAE network parameters

[0123]

[0124] According to the stochastic gradient descent method, the passenger flow characteristics of each area can be obtained after 2000 iterations.

[0125] In this embodiment, step 3: obtaining a station passenger flow score according to the passenger flow characteristics includes:

[0126] The station passenger flow score is obtained by the following formula:

[0127] in,

[0128] v is the evaluation result, and the passenger flow characteristic value is z, zmax is the flow rate and velocity standard; M is the penalty value.

[0129] The loop ends when no M appears in the scores of all areas.

[0130] In this embodiment, the control behaviors include opening more gates, adding diversion personnel, activating flow-limiting fences, closing gates, and lifting flow-limiting measures.

[0131] In this embodiment, step 5: obtaining the current station passenger flow control decision plan according to the station passenger flow score and the passenger flow control strategy includes:

[0132] Step 51: Obtaining the state-action transition probability, where the state-action transition probability represents the probability of performing a corresponding control action according to the station passenger flow score at the current time point;

[0133] Step 52: Obtaining an action-state transition probability, wherein the action-state transition probability represents a probability that the score changes within a predetermined time after performing a corresponding control action;

[0134] Step 53: Generate station passenger flow control decision optimization strategy;

[0135] Step 54: Update the current station passenger flow control decision plan according to the state-action transition probability, the action-state transition probability and the station passenger flow control decision optimization strategy.

[0136] For example, the overall optimization process of station passenger flow management is expressed as a state-action chain: {s0, a0, s1, a1, …, s t-1 , a t-1 ,s t Two state transitions are set in this chain: one is the state-action transition probability, which is determined by the flow and flow rate management strategy; the other is the action-state transition probability, which is determined by the data environment composed of historical data, real-time data, simulation data, and short-term forecast data. Among them, the action methods include opening more gates, adding diversion personnel, activating flow-limiting fences, closing gates, and lifting flow-limiting measures. The time consumed by each measure is its most important feature. And the corresponding time length. In order to prevent the problem of excessive calculation of subsequent iterations, the time length is generated by a discrete alternative set.

[0137] State-action conversion requires the facility manager to choose the best action within his or her cognitive scope according to the current state (i.e. the expected value of the fastest return adjustment of the current passenger flow under the condition of safety), that is, to establish the environmental state value s t The mapping from the probability distribution or probability density function to a set of actions (each action has a certain probability of being executed, and the shorter the time consumption, the greater the probability of being executed) is expressed as follows:

[0138]

[0139] According to Markov property (i.e. the current behavior is only related to the current state, and has nothing to do with the state before the current state), the formula is simplified to:

[0140]

[0141] This architecture uses the calculation results of the above formula to execute actions. And because the set of actions is discrete and finite, the behavior selection problem is transformed into a multi-classification problem and solved using machine learning methods.

[0142] Action-state transition is when the facility manager completes an action and the environment is affected by his action, resulting in a state transition after a certain period of time. Since the state of the next step is only affected by the state of the previous step, the state transition of the environment is carried out according to the following probability formula:

[0143] p(S t+1 |S t , a t )

[0144] Set an optimization strategy, that is, all actions should strive to maximize long-term returns. Assume that the environment provides a quantifiable return value at a certain moment, r. This return value mainly refers to the time it takes to achieve the expected results after taking measures. You can take e -time Calculate in the form of t r t .

[0145] This formula is difficult to calculate, so the hit rate is introduced to make long-term returns more meaningful. The formula is corrected to:∑ t＝0 γ r r t Then define the variable - long-term return, separate all the returns after the current state, and multiply them by the corresponding hit rate: Ret t =∑ k＝0 γ k r t+k+1 .

[0146] Since we do not know the future actions when we execute an action, we introduce another definition: strategy value. We consider the impact of each state transition and then solve the expectation of long-term return based on the state transition. Let τ be the sequence obtained by sampling the strategy and state transition, and write the formula definition of value as:

[0147]

[0148] According to the model form of MDP, two types of value functions are established, including state value function and state-action value function.

[0149] Calculate the state value function v π (s): that is, given the current state S, the expected long-term return generated by following a certain strategy. Then, we traverse the value function starting from a certain state and calculate the expected long-term return of these paths based on probability, and we get:

[0150]

[0151] Where τ represents the transition from state s t According to the Markov property, the expanded path is:

[0152]

[0153] According to Bellman formula, we can deduce:

[0154]

[0155] Calculate the state-action value function q π (s, a), reasoning with the current excess state as state S and action a. The expected long-term return generated by acting according to the strategy. When the value function is stable, the value of any state can be obtained from the value of other states. Similar to the above, the state-behavior value function also has a similar formula:

[0156]

[0157] According to the Monte Carlo algorithm, we can get:

[0158]

[0159] Starting from state S and action a, we keep interacting with the environment, collecting samples, and obtaining a reward sequence: Finally, we can get:

[0160]

[0161] The present application also provides a self-feedback passenger flow control device based on deep reinforcement learning, which includes a data acquisition module, a passenger flow feature extraction module, a station passenger flow scoring module, a passenger flow control strategy acquisition module and an update module, wherein:

[0162] The data acquisition module is used to obtain the historical time series data, simulation data and short-term prediction data of the current time point of the station;

[0163] The passenger flow feature extraction module is used to obtain passenger flow features based on the historical time series data, simulation data and short-term prediction data;

[0164] The station passenger flow scoring module is used to obtain the station passenger flow score based on passenger flow characteristics;

[0165] The passenger flow control strategy acquisition module is used to acquire the passenger flow control strategy, which includes multiple control behaviors;

[0166] The updating module is used to update the current station passenger flow control decision plan according to the station passenger flow score and the passenger flow control strategy.

[0167] It should be noted that the above explanations of the method embodiment are also applicable to the device of this embodiment and will not be repeated here.

[0168] The present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, the above-mentioned self-feedback passenger flow control method based on deep reinforcement learning is implemented.

[0169] The present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement the above-mentioned self-feedback passenger flow control method based on deep reinforcement learning.

[0170] Figure 2 It is an exemplary structural diagram of an electronic device that can implement the self-feedback passenger flow control method based on deep reinforcement learning provided according to an embodiment of the present application.

[0171] like Figure 2 As shown, the electronic device includes an input device 501, an input interface 502, a central processor 503, a memory 504, an output interface 505, and an output device 506. The input interface 502, the central processor 503, the memory 504, and the output interface 505 are interconnected through a bus 507, and the input device 501 and the output device 506 are connected to the bus 507 through the input interface 502 and the output interface 505, respectively, and then connected to other components of the electronic device. Specifically, the input device 504 receives input information from the outside, and transmits the input information to the central processor 503 through the input interface 502; the central processor 503 processes the input information based on the computer executable instructions stored in the memory 504 to generate output information, temporarily or permanently stores the output information in the memory 504, and then transmits the output information to the output device 506 through the output interface 505; the output device 506 outputs the output information to the outside of the electronic device for use by the user.

[0172] That is to say, Figure 2 The electronic device shown can also be implemented to include: a memory storing computer-executable instructions; and one or more processors, which can implement the above-mentioned self-feedback passenger flow control method based on deep reinforcement learning when executing the computer-executable instructions.

[0173] In one embodiment, Figure 2 The electronic device shown can be implemented to include: a memory 504, configured to store executable program code; one or more processors 503, configured to run the executable program code stored in the memory 504 to execute the self-feedback passenger flow control method based on deep reinforcement learning in the above-mentioned embodiment.

[0174] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0175] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0176] Computer readable media include permanent and non-permanent, removable and non-removable media, and media can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), data versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0177] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0178] In addition, it is clear that the word "comprising" does not exclude other units or steps. Multiple units, modules or devices stated in the device claim can also be implemented by one unit or the overall device through software or hardware. The words first, second, etc. are used to identify the name, not any specific order.

[0179] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and a module, a program segment or a part of a code includes one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes identified in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or the total flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0180] The processor referred to in this embodiment may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0181] The memory can be used to store computer programs and / or modules. The processor realizes various functions of the device / terminal equipment by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0182] In this embodiment, if the module / unit integrated in the device / terminal equipment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0183] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the above embodiments, a person skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A self-feedback passenger flow control method based on deep reinforcement learning, characterized in that: The self-feedback passenger flow control method based on deep reinforcement learning includes: Obtain the historical time series data, simulation data and short-term forecast data of passenger flow at the current time point of the station; Extracting environmental features based on the passenger flow historical time series data, simulation data, and short-term prediction data; Obtaining a comprehensive evaluation of the capacity load of station facilities based on the environmental characteristics; Acquire a passenger flow control strategy, wherein the passenger flow control strategy includes multiple control behaviors; Update the current station passenger flow control decision plan according to the comprehensive evaluation of the station facility capacity load and the passenger flow control strategy; The acquisition of the historical time series data, simulation data and short-term prediction data of the passenger flow at the current time point of the station includes: Get the historical time series data of passenger flow at the current time point of the station; Acquire passenger flow simulation data of the station at the current time point according to the passenger flow historical time series data; Obtain the passenger flow feedback processing result of the station at the previous time point of the current time point; Acquire short-term passenger flow forecast data at the current time point of the station according to the passenger flow simulation data and the passenger flow feedback processing result; The acquisition of the historical time series data of passenger flow at the current time point of the station includes: Get image preprocessing information; Obtain pedestrian trajectory information based on image preprocessing information; Obtaining historical time series data of passenger flow according to the pedestrian trajectory information analysis and statistics; The step of obtaining the passenger flow simulation data of the station at the current time point according to the passenger flow historical time series data comprises: Generate a station simulation model, which includes: importing the CAD plan of each floor of the station as the model base map in AnyLogic software, setting the scale of the graphics and the scale of the objects, and then drawing the obstacles in the map. The escalator can be directly set as an implanted module. The main setting process of this type of model is: Step 1211: Double-click the rectangular area and draw in the escalator area; Step 1212: drag in the ped Area Descriptor, and select the dragged in rectangular area in Area; Step 1213: If it is an elevator, set Is moving and at speed; if not, just set Throughout; Step 1214: If you need to set the slope, select rectangular area and check slope to set it; Design and generate process logic, which includes: AnyLogic drives the entire simulation through flowcharts. The subway station pedestrian simulation is a discrete event simulation. The various small events in it can be summarized into four sub-steps: generation - destination selection - going - leaving. The specific implementation steps are as follows: Set the process function implementation objects, including target line, target area, walking path, service point, service queue, and set the name and corresponding attributes. When setting the arrival or departure of pedestrians, set it through Target Line. Pedestrians will appear or leave from TargetLine. Target Line is both the entry and exit line of each entrance and exit and the entrance and exit of each escalator. Secondly, set the service queue line where service is needed, and then set the relevant attributes and queue form of the service; Draw a logical flow chart based on process logic; Generate simulation control logic according to the logic flow chart; Set up data collection and statistics based on simulation results; Performing a passenger flow simulation on the station according to the simulation control logic and the historical passenger flow time series data, thereby obtaining passenger flow simulation data at a current time point; The step of obtaining the short-term passenger flow prediction data of the station at the current time point according to the passenger flow simulation data and the feedback processing result includes: Constructing an LSTM model, wherein constructing the LSTM model comprises: constructing an LSTM model including three continuous loop structures, wherein each layer of LSTM includes one loop structure, and only one loop unit needs to be input into the current layer; each loop structure has two outputs, and the number of self-updates of a single loop structure of the LSTM structure is only determined by the dimension of the input data and the number of loops, that is, the total number of parameters of the current layer of LSTM only needs to be calculated for one loop unit, and the total number of multiple continuous units does not need to be calculated, so as to improve the calculation efficiency of the system; Set the input gate, forget gate, output gate and memory cell state for the LSTM cell; Input gate: determines how much historical data input at the current moment needs to be saved in the unit state; Forget gate: determines how much of the simulation result feedback of the previous moment needs to be retained until the current moment and substituted into the loop for recalculation; the forget gate needs to discard some information so that the time span of each batch of data is the same, and monitor the data scale to prevent the explosion of calculation amount, and set the proportion parameter σ of discarded data, whose value range is [0,1]; Output gate: Analyze the short-term prediction results of each loop layer of the LSTM system, and determine how much of the current unit state timing information and prediction information needs to be output to the current result or returned to the hidden layer; at the same time, update the short-term prediction results stored in the memory unit of this layer through this output; Memory unit: ensures that the updated optimized gradient is always passed and converges, controlled by a state function; Output of the feedback processing result of the last moment and the current simulation data input , obtained through the forget gate , the formula is as follows: ， Step 1414: Apply the output of the feedback processing result of the previous moment and the current simulation data input , obtained through the input gate , and obtain the temporary state of the memory unit at the current moment through the unit state ; The formula is as follows: ， ， Apply the memory cell state of the previous cell structure , the forget gate output of this cycle , input gate output And the output of the memory cell state of this cycle , get the current state of the memory unit , the formula is as follows: ， Output of the feedback processing result of the previous moment and the current simulation data input , obtained through the output gate The process, combined with the current cell state and Get the final output , the formula is as follows: ， ； The passenger flow simulation data and the feedback processing result are input into the LSTM model and the output information of the LSTM model is obtained as short-term prediction data.

2. The self-feedback passenger flow control method based on deep reinforcement learning according to claim 1, characterized in that: The obtaining of environmental features according to the passenger flow historical time series data, simulation data and short-term prediction data includes: The deep learning network SAE is used to extract environmental features of heterogeneous data sources formed by historical passenger flow time series data, simulation data and short-term forecast data.

3. The self-feedback passenger flow control method based on deep reinforcement learning as claimed in claim 2, characterized in that: The obtaining of a comprehensive evaluation of the station facility capacity load according to the environmental characteristics comprises: The comprehensive evaluation of station facility capacity load is obtained through the following formula: ,in, v is the evaluation result, the passenger flow characteristic value is z, is the flow rate and velocity standard; M is the penalty value.

4. The self-feedback passenger flow control method based on deep reinforcement learning as claimed in claim 3, characterized in that: The obtaining of the current station passenger flow control decision plan according to the station facility capacity load score and the passenger flow control strategy comprises: Obtaining a state-action transition probability, wherein the state-action transition probability represents a probability of performing a corresponding control action according to the station facility capacity load score at a current time point; Obtaining an action-state transition probability, wherein the action-state transition probability represents a probability that the score changes within a predetermined time after performing a corresponding control action; Generate station passenger flow control decision optimization strategy; The current station passenger flow control decision plan is updated according to the state-action transition probability, the action-state transition probability and the station passenger flow control decision optimization strategy.

5. A self-feedback passenger flow control device based on deep reinforcement learning, characterized in that: The self-feedback passenger flow control device based on deep reinforcement learning includes: A data acquisition module, which is used to acquire historical time series data, simulation data and short-term prediction data of passenger flow at the current time point of the station; An environmental feature extraction module, the environmental feature extraction module is used to obtain environmental features based on the passenger flow historical time series data, simulation data and short-term prediction data; A station facility capacity load evaluation module, the station facility capacity load evaluation module is used to obtain a station facility capacity load evaluation according to the environmental characteristics; A passenger flow control strategy acquisition module, the passenger flow control strategy acquisition module is used to acquire a passenger flow control strategy, the passenger flow control strategy includes a variety of control behaviors; An updating module, the updating module is used to update the current station passenger flow control decision plan according to the station facility capacity load evaluation and the passenger flow control strategy; The acquisition of the historical time series data, simulation data and short-term prediction data of the passenger flow at the current time point of the station includes: Get the historical time series data of passenger flow at the current time point of the station; Acquire passenger flow simulation data of the station at the current time point according to the passenger flow historical time series data; Obtain the passenger flow feedback processing result of the station at the previous time point of the current time point; Acquire short-term passenger flow forecast data at the current time point of the station according to the passenger flow simulation data and the passenger flow feedback processing result; The acquisition of the historical time series data of passenger flow at the current time point of the station includes: Get image preprocessing information; Obtain pedestrian trajectory information based on image preprocessing information; Obtaining historical time series data of passenger flow according to the pedestrian trajectory information analysis and statistics; The step of obtaining the passenger flow simulation data of the station at the current time point according to the passenger flow historical time series data comprises: Generate a station simulation model, which includes: importing the CAD plan of each floor of the station as the model base map in AnyLogic software, setting the scale of the graphics and the scale of the objects, and then drawing the obstacles in the map. The escalator can be directly set as an implanted module. The main setting process of this type of model is: Step 1211: Double-click the rectangular area and draw in the escalator area; Step 1212: drag in the ped Area Descriptor, and select the dragged in rectangular area in Area; Step 1213: If it is an elevator, set Is moving and at speed; if not, just set Throughout; Step 1214: If you need to set the slope, select rectangular area and check slope to set it; Design and generate process logic, which includes: AnyLogic drives the entire simulation through flowcharts. The subway station pedestrian simulation is a discrete event simulation. The various small events in it can be summarized into four sub-steps: generation - destination selection - going - leaving. The specific implementation steps are as follows: Set the process function implementation objects, including target line, target area, walking path, service point, service queue, and set the name and corresponding attributes. When setting the arrival or departure of pedestrians, set it through Target Line. Pedestrians will appear or leave from TargetLine. Target Line is both the entry and exit line of each entrance and exit and the entrance and exit of each escalator. Secondly, set the service queue line where service is needed, and then set the relevant attributes and queue form of the service; Draw a logical flow chart based on process logic; Generate simulation control logic according to the logic flow chart; Set up data collection and statistics based on simulation results; Performing a passenger flow simulation on the station according to the simulation control logic and the historical passenger flow time series data, thereby obtaining passenger flow simulation data at a current time point; The step of obtaining the short-term passenger flow prediction data of the station at the current time point according to the passenger flow simulation data and the feedback processing result includes: Constructing an LSTM model, wherein constructing the LSTM model comprises: constructing an LSTM model including three continuous loop structures, wherein each layer of LSTM includes one loop structure, and only one loop unit needs to be input into the current layer; each loop structure has two outputs, and the number of self-updates of a single loop structure of the LSTM structure is only determined by the dimension of the input data and the number of loops, that is, the total number of parameters of the current layer of LSTM only needs to be calculated for one loop unit, and the total number of multiple continuous units does not need to be calculated, so as to improve the calculation efficiency of the system; Set the input gate, forget gate, output gate and memory cell state for the LSTM cell; Input gate: determines how much historical data input at the current moment needs to be saved in the unit state; Forget gate: determines how much of the simulation result feedback of the previous moment needs to be retained until the current moment and substituted into the loop for recalculation; the forget gate needs to discard some information so that the time span of each batch of data is the same, and monitor the data scale to prevent the explosion of calculation amount, and set the proportion parameter σ of discarded data, whose value range is [0,1]; Output gate: Analyze the short-term prediction results of each loop layer of the LSTM system, and determine how much of the current unit state timing information and prediction information needs to be output to the current result or returned to the hidden layer; at the same time, update the short-term prediction results stored in the memory unit of this layer through this output; Memory unit: ensures that the updated optimized gradient is always passed and converges, controlled by a state function; Output of the feedback processing result of the last moment and the current simulation data input , obtained through the forget gate , the formula is as follows: ， Step 1414: Apply the output of the feedback processing result of the previous moment and the current simulation data input , obtained through the input gate , and obtain the temporary state of the memory unit at the current moment through the unit state ; The formula is as follows: ， ， Apply the memory cell state of the previous cell structure , the forget gate output of this cycle , input gate output And the output of the memory cell state of this cycle , get the current state of the memory unit , the formula is as follows: ， Output of the feedback processing result of the previous moment and the current simulation data input , obtained through the output gate The process, combined with the current cell state and Get the final output , the formula is as follows: ， ； The passenger flow simulation data and the feedback processing result are input into the LSTM model and the output information of the LSTM model is obtained as short-term prediction data.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: When the processor executes the computer program, the self-feedback passenger flow control method based on deep reinforcement learning as described in any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it is capable of implementing the self-feedback passenger flow control method based on deep reinforcement learning as described in any one of claims 1 to 4.

Images