Crowd flow control method and system based on reinforcement learning

Abstract

The embodiment of the invention provides a crowd flow control method and system based on reinforcement learning, and the method comprises the steps: obtaining a target scene feature; determining a target state feature according to the target scene feature; generating a target crowd flow regulation and control strategy according to the target state characteristics; executing target regulation and control operation according to the target crowd flow regulation and control strategy; wherein the target scene feature comprises a target node feature and / or a target path feature. In this way, the matching degree of the crowd flow regulation and control strategy and the specific scene can be improved, and the intelligent level and efficiency of crowd flow management and control are improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of automatic control, and particularly relate to a crowd flow control method and system based on reinforcement learning. BACKGROUND

[0002] Crowd flow regulation is mainly used for managing and optimizing the personnel flow process in a specific scene. At present, in the related art, there are problems such as low matching degree of crowd flow regulation strategy and corresponding scene, poor intelligent level of crowd flow control, and low efficiency.

[0003] Therefore, there is an urgent need for a new technical solution to solve the above technical problems. SUMMARY

[0004] According to embodiments of the present application, a crowd flow control method and system based on reinforcement learning are provided, which can improve the matching degree of crowd flow regulation strategy and specific scene, and improve the intelligent level and efficiency of crowd flow control.

[0005] In a first aspect of the present application, a crowd flow control method based on reinforcement learning is provided, comprising:

[0006] obtaining target scene features;

[0007] determining target state features according to the target scene features;

[0008] generating a target crowd flow regulation strategy according to the target state features;

[0009] performing a target regulation operation according to the target crowd flow regulation strategy;

[0010] The target scene features include target node features and / or target path features.

[0011] In some possible implementations, the flow degree of the target node and / or the target path is greater than or equal to a first preset threshold;

[0012] The crossing degree of the target node and / or the target path is greater than or equal to a second preset threshold;

[0013] The bottleneck degree of the target node and / or the target path is greater than or equal to a third preset threshold;

[0014] And / or, the vulnerability degree of the target node and / or the target path is greater than or equal to a fourth preset threshold.

[0015] In some possible implementations, the target state features include crowd density features, crowd moving speed features, crowd flow direction features, and / or crowd queue length features in a target time period.

[0016] In some possible implementation manners, the generating, according to the target state feature, of the target crowd flow regulation strategy includes:

[0017] generating a target crowd flow dynamic change map according to the target state feature;

[0018] generating a target crowd flow regulation strategy according to the target crowd flow dynamic change map;

[0019] The target crowd flow regulation strategy includes a target node release speed regulation strategy and / or a target passage opening and closing state regulation strategy.

[0020] In some possible implementation manners, the target crowd flow dynamic change map includes a first target layer, a second target layer, a third target layer, and / or a fourth target layer.

[0021] The first target layer corresponds to a target crowd density heat dynamic change map.

[0022] The second target layer corresponds to a target moving speed vector field dynamic change map.

[0023] The third target layer corresponds to a target flow direction vector field dynamic change map.

[0024] The fourth target layer corresponds to a target queue length dynamic change map.

[0025] In some possible implementation manners, the method further includes:

[0026] generating a target abnormal event speculation result according to the target difference in a case where the target difference corresponding to the target crowd flow dynamic change map and the historical crowd flow dynamic change map is greater than a fifth preset threshold;

[0027] generating a target abnormal marker according to the target abnormal event speculation result;

[0028] The target abnormal event speculation result includes a falling event speculation result, a panic running event speculation result, a reverse movement event speculation result, and / or a facility operation abnormal event speculation result.

[0029] correcting the target crowd flow regulation strategy according to the target abnormal marker.

[0030] In some possible implementation manners, the method further includes:

[0031] constructing a target intelligent agent networking according to the target scene feature.

[0032] In some possible implementation manners, the constructing, according to the target scene feature, of the target intelligent agent networking includes:

[0033] According to the crossing degree, the bottleneck degree, and / or the vulnerability degree corresponding to the target node and / or the target path, a target agent running resource allocation parameter is determined;

[0034] According to the target agent running resource allocation parameter, a running resource allocation operation is performed.

[0035] In some possible implementation manners, the above determining the target agent running resource allocation parameter according to the crossing degree, the bottleneck degree, and / or the vulnerability degree corresponding to the target node and / or the target path comprises:

[0036] The computing resource allocation parameter is determined according to the following formula:

[0037]

[0038] Wherein, C PUi is used to represent the computing resource allocation parameter corresponding to the target agent i; Base is used to represent the basic guarantee resource; Δ is used to represent the scalable computing resource; K i is used to represent the comprehensive criticality index corresponding to the target agent i; θ is used to represent the computing resource transition threshold; σ is used to represent the first smoothing coefficient;

[0039] The memory allocation parameter is determined according to the following formula:

[0040]

[0041] Wherein, R AMi is used to represent the memory allocation parameter corresponding to the target agent i; α3 is used to represent the memory allocation base; K i is used to represent the comprehensive criticality index corresponding to the target agent i; β4 is used to represent the comprehensive criticality index K i is used to represent the corresponding amplification factor; Offset is used to represent the basic guarantee memory.

[0042] The communication bandwidth allocation parameter is determined according to the following formula:

[0043]

[0044] Wherein, BW i is used to represent the communication bandwidth allocation parameter corresponding to the target agent i; BW min is used to represent the minimum bandwidth guarantee value; BW max is used to represent the maximum bandwidth upper limit value; K i is used to represent the comprehensive criticality index corresponding to the target agent i; ∈ is used to represent the second smoothing coefficient;

[0045] And / or, a decision frequency allocation parameter is determined according to the following formula:

[0046]

[0047] Wherein, f i is used to represent the decision frequency allocation parameter corresponding to the target agent i; f max is used to represent the maximum decision frequency corresponding to the target agent i; η is used to represent the frequency growth sensitivity; K i is used to represent the comprehensive criticality index corresponding to the target agent i; K0 is used to represent the decision frequency effective threshold;

[0048] Wherein, the comprehensive criticality index K i corresponding to the target agent i is determined according to the following formula:

[0049]

[0050] Wherein, K i is used to represent the comprehensive criticality index corresponding to the target agent i; C i is used to represent the normalized intersection degree; ω c is used to represent the normalized intersection degree C i corresponding to the weight; B i is used to represent the normalized bottleneck degree; ω b is used to represent the normalized bottleneck degree B i corresponding to the weight; γ4 is used to represent the nonlinear adjustment factor; V i is used to represent the normalized vulnerability degree; ω v is used to represent the normalized vulnerability degree V i corresponding to the weight.

[0051] The second aspect of the present application proposes a crowd flow control system based on reinforcement learning, which is suitable for the above method, comprising:

[0052] An acquisition unit is configured to acquire target scene features;

[0053] A determination unit is configured to determine target state features according to the target scene features;

[0054] A generation unit is configured to generate a target crowd flow regulation strategy according to the target state features;

[0055] An execution unit is configured to execute a target regulation operation according to the target crowd flow regulation strategy;

[0056] Wherein, the target scene features include target node features and / or target path features.

[0057] The method provided by the embodiment of the application comprises: obtaining target scene features; determining target state features according to the target scene features; generating a target crowd flow regulation strategy according to the target state features; and performing a target regulation operation according to the target crowd flow regulation strategy. The target scene features comprise target node features and / or target path features. In this way, the matching degree of the crowd flow regulation strategy and a specific scene can be improved, so that the intelligent level and efficiency of crowd flow regulation and control are improved.

[0058] It should be understood that the content described in the summary section is not intended to limit the key or important features of the embodiments of the application, nor to limit the scope of the application. Other features of the application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0059] The above and other features, advantages, and aspects of the embodiments of the present application will become more apparent upon consideration of the following detailed description, taken in conjunction with the accompanying drawings. In the drawings like reference numerals refer to like elements, wherein:

[0060] Figure 1 A flowchart of an intelligent water meter identification method provided by the embodiment of the application;

[0061] Figure 2 A structural schematic diagram of an intelligent water meter identification system provided by the embodiment of the application;

[0062] Figure 3 A structural schematic diagram of an electronic device provided by the embodiment of the application. DETAILED DESCRIPTION

[0063] In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the protection scope of the present disclosure.

[0064] In addition, the term "and / or" herein is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects.

[0065] In a first aspect, the embodiment of the application provides a crowd flow control method based on reinforcement learning.Figure 1 A flowchart of a crowd flow control method 100 based on reinforcement learning provided for an embodiment of the present application is shown in FIG. 1. Figure 1 As shown, the method 100 includes:

[0066] Step S1: Obtain target scene features.

[0067] It should be noted that the above target scene features can include target node features and / or target path features.

[0068] Exemplarily, the above target node features can include entrance and exit node features, intersection node features, and / or bottleneck node features, etc.

[0069] The above entrance and exit node features can include geographic location features, physical structure features, function and control features, and / or state and dynamic features, etc.

[0070] Specifically, the above geographic location features can include longitude and latitude coordinate features corresponding to the entrance and exit nodes, relative position features between multiple entrance and exit nodes, and / or mapping relationship features between multiple entrance and exit nodes and corresponding multiple target paths, etc.

[0071] Specifically, the above physical structure features can include width features, such as actual physical width, effective passing width (e.g., considering obstacles), single or double direction features (e.g., single entrance or double entrance), slope or step features (e.g., steepness, height, etc.), and / or buffer area features (e.g., area of buffer area for crowd queuing or temporary stay, etc.).

[0072] Specifically, the above function and control features can include passing capacity features (e.g., maximum number of people passing through the entrance under certain conditions per unit time), open state features (e.g., open or closed), control mode features (e.g., fixed time release, induction release, manual control, and / or signal control by central system, etc.), and / or release speed or efficiency features (e.g., current or set release frequency or release batch interval time, etc.).

[0073] Specifically, the above state and dynamic features can include real-time or near real-time crowd flow features (e.g., current instantaneous or short-time average rate of passing through the entrance), queue length features (e.g., length of queue waiting to enter or leave in front of the entrance), average waiting time features (e.g., average waiting time of the current queue), and / or population density features (e.g., crowd density in the buffer area or queue area in front of the entrance, etc.).

[0074] The intersection node features can include physical structure features, human flow dynamic and state features, control and rule features, and / or risk features, etc.

[0075] Specifically, the physical structure features can include intersection node type features, such as cross intersection, cross intersection, and / or multi-branch intersection, etc.; connection channel type features, such as connection to main channel, connection to secondary channel, connection to emergency channel, connection to stair channel, and / or connection to escalator channel, etc.; connection channel angle features, such as the angle between the intersection node and the channel connection being a right angle, an acute angle or an obtuse angle, etc.; intersection node corresponding geometric shape features, such as the geometric shape corresponding to the intersection node being circular, square, and / or irregular; area features; intersection node direction passing width features, such as the effective passing width from the intersection to each connection channel; obstacle distribution features, such as the position and size of the column, the position and size of the information kiosk, the position and size of the seat, the position and size of the green plant, and / or the position and size of the security check facility, etc.; and / or visibility and line-of-sight blocking features, such as the mutual visibility of different direction channels in the intersection area, etc.

[0076] Specifically, the human flow dynamic and state features can include real-time or near real-time flow features in each direction, such as the current or short-term flow rate of people passing through the intersection to each connection channel; crowd density in each direction, such as the population density at the intersection core area and the entrance of each direction channel; main and secondary flow direction ratio, such as the relative ratio of the flow in each direction to identify the main flow direction and the secondary direction; flow conflict point distribution and intensity features to identify and quantify the specific location points where conflicts, interweaving and crossing occur between different flow directions, i.e. conflict points, and the intensity of the conflict, such as the number of conflicts, frequency, etc.; average passing speed features, such as the average speed of the crowd passing through the intersection area; average waiting or detention time features, i.e. the average residence time of the crowd at the intersection due to direction selection, waiting for passing right or congestion; and / or turning ratio features, i.e. the proportion of people flowing to each exit channel after entering the intersection from a specific entrance channel, such as X% of people from channel A going to channel B, Y% of people from channel A going to channel C, etc.

[0077] Specifically, the control and rule features can include passing right control mechanism features, such as signal control features, such as passing signal cycle, phase length, direction allowed to pass phase, and / or rule control features, such as specific direction priority; and / or control state features, such as opening or closing, etc.

[0078] In particular, the risk features can include: a conflict point quantity and complexity feature, i.e., a total number of conflict points formed by different paths of people flow intersecting in the intersection and a complexity of spatial distribution thereof; a potential bottleneck strength feature; a historical congestion frequency and duration feature, i.e., a historical record and average duration of congestion occurring in the intersection and node; a historical safety accident record feature, i.e., a record of safety accidents such as pushing, falling, etc. occurring in the intersection and node within a certain duration; an emergency evacuation path impact feature, i.e., a criticality of the intersection and node in an emergency evacuation route; and / or a vulnerability correlation factor feature, i.e., a feature that is likely to cause chaos or a small accident impact, such as: severe visual obstruction, risk of slippery ground, lack of effective physical separation or guidance, possibility of failure of control mechanism, etc.

[0079] In particular, the bottleneck node features can include: a physical constraint feature, a traffic capacity and efficiency feature, a dynamic state and congestion feature, and / or a controllable feature, etc.

[0080] In particular, the physical constraint feature can include: a minimum effective traffic width, i.e., an actual available traffic width of the narrowest point or most restricted area at the bottleneck node; a curvature at a turn, i.e., a curvature corresponding to the bottleneck node in the case that the bottleneck node is at a turn; a slope, i.e., a corresponding slope in the case that the bottleneck node is connected to a slope road; and / or a function, i.e., a function corresponding to the bottleneck node, such as: a security check area, a waiting area, a ticketing area, a commercial area, a luggage pickup area, a viewing area, etc.

[0081] In particular, the traffic capacity and efficiency feature, i.e., a maximum number of safe passages that can be supported by the bottleneck node; a traffic capacity loss rate, i.e., a ratio between a difference between a theoretical traffic capacity and an actual traffic capacity and the theoretical traffic capacity; a saturated flow rate, i.e., an actual average passage rate that can be maintained by the bottleneck node under continuous high demand; and / or a traffic efficiency, i.e., a number of people that can be effectively trafficked per unit of time.

[0082] In particular, the dynamic state and congestion feature can include: real-time or near real-time people flow, i.e., an instantaneous or short-time average people flow rate of the bottleneck node; a people crowd density of the bottleneck node; an average moving speed of the people crowd of the bottleneck node; a queue length of the people crowd of the bottleneck node; an average waiting duration of the bottleneck node; and / or a historical congestion frequency and duration of the bottleneck node.

[0083] In particular, the controllable feature can include: a variable number of channels corresponding to the bottleneck node, and / or a control response sensitivity corresponding to the bottleneck node, such as: a speed of improvement of the bottleneck state.

[0084] Exemplarily, the above-mentioned target passage features can include: physical attribute features, traffic capacity and efficiency features, dynamic state features, connectivity and topology features, environment and facility features, and / or controllable features, etc.

[0085] Specifically, the above-mentioned physical attribute features can include: passage type features, such as passage features, corridor features, and / or stair features, etc.; geometric size features, such as length, width, height, and / or slope, etc.

[0086] Specifically, the above-mentioned traffic capacity and efficiency features can include: theoretical traffic capacity, i.e., the maximum safe traffic number that can pass through the passage per unit time; actual traffic capacity, i.e., the maximum sustainable traffic number; free flow speed, i.e., the average safe walking speed that the passage can reach under low-density state; saturated flow density, i.e., the corresponding crowd density under saturated state; traffic time baseline, i.e., the average time consumption of the whole passage under free flow or low-density state.

[0087] Specifically, the above-mentioned dynamic state features can include: real-time or near real-time crowd flow, i.e., the instantaneous or short-time average crowd flow speed of the passage; passage crowd density distribution, such as the crowd density along the length direction of the passage; average crowd speed in the passage, i.e., the average speed of the current crowd moving in the passage; main flow direction, i.e., the main flow direction of the crowd in the passage; queue position and length, i.e., the specific coordinates and queue length of the queue in the passage; and / or average formation time, i.e., the actual average time consumption of the current state corresponding crowd passing through the passage.

[0088] Specifically, the above-mentioned connectivity and topology features can include: the node corresponding to the starting end of the passage, such as node ID or type; the node corresponding to the terminal end of the passage, such as node ID or type; the intermediate connection node of the passage, such as node ID or type; the alternative passage corresponding to the passage, such as the passage with similar function to the passage, and / or whether the passage is a critical passage, such as a necessary passage or trunk road, etc.

[0089] Specifically, the above-mentioned environment and facility features can include: lighting conditions, ground conditions, and / or visual permeability, etc.

[0090] Specifically, the above-mentioned controllable features can include: flow direction controllability, on-off state controllability, and / or flow speed or flow volume controllability, etc.

[0091] In some possible embodiments, the target node and / or the target path correspond to a flow degree greater than or equal to a first preset threshold, a crossing degree greater than or equal to a second preset threshold, a bottleneck degree greater than or equal to a third preset threshold, and / or a vulnerability degree greater than or equal to a fourth preset threshold.

[0092] Exemplarily, a node with a flow degree greater than or equal to a first preset threshold, a crossing degree greater than or equal to a second preset threshold, a bottleneck degree greater than or equal to a third preset threshold, and / or a vulnerability degree greater than or equal to a fourth preset threshold can be taken as a target node, and / or a path with a flow degree greater than or equal to a first preset threshold, a crossing degree greater than or equal to a second preset threshold, a bottleneck degree greater than or equal to a third preset threshold, and / or a vulnerability degree greater than or equal to a fourth preset threshold can be taken as a target path, for monitoring.

[0093] Specifically, the flow degree corresponding to the target node and / or the target path can be determined according to the number of passing people in a unit time, for example, the number of people passing per minute.

[0094] Specifically, the crossing degree of the target node can be determined according to the following formula:

[0095]

[0096] wherein, Th node1 is used to represent the crossing degree of the target node; β1 is used to represent a safety coefficient; N connect is used to represent the number of paths connected to the target node; A node is used to represent the area of the region corresponding to the target node.

[0097] Specifically, the crossing degree corresponding to the target path can be determined according to the following formula:

[0098]

[0099] wherein, Th path1 is used to represent the crossing degree corresponding to the target path; γ1 is used to represent a redundancy coefficient; L total is used to represent the total length corresponding to the target path; L conflict is used to represent the length of the conflict section corresponding to the target path.

[0100] It should be noted that the length L conflict of the conflict section corresponding to the target path can be determined according to the sum of the length of the entrance corresponding to the target path and the length of the exit corresponding to the target path.

[0101] Specifically, the bottleneck degree of the target node can be determined according to the following formula:

[0102]

[0103] wherein, Th node2 is used to represent the bottleneck degree corresponding to the target node; a1 is used to represent the basic coefficient; A buffer1 is used to represent the buffer area corresponding to the target node; b2 is used to represent the fire-fighting coefficient; R fire fire-fighting correction coefficient

[0104] Specifically, the bottleneck degree of the target path can be determined according to the following formula:

[0105]

[0106] wherein, Th path2 is used to represent the bottleneck degree corresponding to the target path; g2 is used to represent the width coefficient; L min-width is used to represent the length of the narrowest section corresponding to the target path; L total is used to represent the total length of the path corresponding to the target path; d1 is used to represent the weight of the shunt correction coefficient; F divert is used to represent the shunt correction coefficient.

[0107] Specifically, the vulnerability of the target node can be determined according to the following formula:

[0108]

[0109] wherein, Th node3 is used to represent the vulnerability corresponding to the target node; a2 is used to represent the structure risk weight corresponding to the target node; R struct is used to represent the structure safety rating corresponding to the target node; b3 is used to represent the facility dependence weight corresponding to the target node; N dep is used to represent the number of critical facility dependencies corresponding to the target node.

[0110] It should be noted that the structure safety rating R struct is greater than or equal to 0 and less than or equal to 1. Wherein, in the case that the structure safety rating R struct of the target node is equal to 1, the target node is a high-risk node.

[0111] Specifically, the vulnerability of the target path can be determined according to the following formula:

[0112]

[0113] wherein, Th path3 is used to represent the vulnerability corresponding to the target path; g3 is used to represent the risk length weight corresponding to the target path; L riskis used to represent the proportion of high-risk road sections corresponding to the target path, greater than or equal to 0, and less than or equal to 1; δ2 is used to represent the buffer space weight corresponding to the target path; A buffer2 is used to represent the buffer area corresponding to the target path.

[0114] It should be noted that the first preset threshold corresponding to the target node can be less than or equal to the first preset threshold corresponding to the target path; the second preset threshold corresponding to the target node can be less than or equal to the second preset threshold corresponding to the target path; the third preset threshold corresponding to the target node can be less than or equal to the third preset threshold corresponding to the target path; and / or, the fourth preset threshold corresponding to the target node can be less than or equal to the fourth preset threshold corresponding to the target path.

[0115] Therefore, by monitoring the target node and / or the target path with a flow degree greater than or equal to a first preset threshold, a target node and / or a target path with a crossing degree greater than or equal to a second preset threshold, a target node and / or a target path with a bottleneck degree greater than or equal to a third preset threshold, and / or a target node and / or a target path with a vulnerability degree greater than or equal to a fourth preset threshold, the method is advantageous in accurately identifying target nodes and / or target paths with high criticality, improving the efficiency of determining target state characteristics, and thereby improving the generation rate of target crowd flow control strategies, and further improving the execution efficiency of target control operations, to improve the intelligent level and efficiency of crowd flow control.

[0116] Step S2: determining a target state characteristic according to a target scene characteristic.

[0117] For example, the target state characteristic corresponding to the target node characteristic and / or the target path characteristic can be determined according to the target node characteristic and / or the target path characteristic.

[0118] In some possible embodiments, the target state characteristic includes a crowd density characteristic, a crowd moving speed characteristic, a crowd flow direction characteristic, and / or a crowd queue length characteristic in a target period.

[0119] Specifically, the crowd density characteristic, the crowd moving speed characteristic, the crowd flow direction characteristic, and / or the crowd queue length characteristic in a target period corresponding to the target node and / or the target path can be determined according to the target node characteristic and / or the target path characteristic.

[0120] Specifically, the crowd density characteristic in a target period corresponding to the target node can be determined according to the following formula:

[0121]

[0122] wherein, ρnode for representing the crowd density in the target period corresponding to the target node; N node for representing the number of people in the target period corresponding to the target node; A node for representing the area of the region corresponding to the target node.

[0123] Specifically, the crowd density feature in the target period corresponding to the target path can be determined according to the following formula:

[0124]

[0125] wherein, p path,segment for representing the crowd density in the target period corresponding to the target path; N segment for representing the number of people in the target period corresponding to the target path; L segment for representing the target section length corresponding to the target path; W eff for representing the effective width corresponding to the target path.

[0126] It should be noted that the target section length L segment of the target path can correspond to a value range greater than or equal to 5 meters and less than or equal to 10 meters.

[0127] Specifically, the crowd moving speed feature in the target period corresponding to the target node can include the average moving speed of the crowd in the target period corresponding to the target node, which can be determined according to the following formula:

[0128]

[0129] wherein, v node for representing the average moving speed of the crowd in the target period corresponding to the target node; v i for representing the average moving speed of the crowd individual i in the target period corresponding to the target node; N node for representing the total number of crowd individuals in the target period corresponding to the target node; i for representing the index value corresponding to the crowd individual in the target period corresponding to the target node.

[0130] Specifically, the crowd moving speed feature in the target period corresponding to the target path can include the average moving speed of the crowd in the target period corresponding to the target path, which can be determined according to the following formula:

[0131]

[0132] wherein, v path for representing the average moving speed of the crowd in the target period corresponding to the target path; v j for representing the average moving speed of the crowd individual j in the target period corresponding to the target path; Nsegment is used to represent the total number of individuals in the crowd in the target period corresponding to the target path; j is used to represent the index value corresponding to the individual in the crowd in the target period corresponding to the target path.

[0133] Specifically, the crowd flow direction feature in the target period corresponding to the target node, that is, the proportion of people flow corresponding to different directions of the target node, can be determined according to the trajectory data and the number of people entering multiple different paths from the target node.

[0134] Specifically, the crowd queue length feature corresponding to the target node can be determined according to the following formula:

[0135] L queue = N queue × d avg (11);

[0136] wherein L queue may be used to represent the crowd queue length corresponding to the target node; N queue may be used to represent the number of people in line corresponding to the target node; d avg may be used to represent the length per capita occupied by the target node.

[0137] Specifically, the corresponding crowd queue length feature corresponding to the target path can be determined according to the length extended by the bottleneck point corresponding to the target path, for example: the extended queue length is determined by locating the position node corresponding to the sudden increase of crowd density of the target path to determine the corresponding crowd queue length of the target path.

[0138] Therefore, the above method can accurately determine the crowd density feature, crowd moving speed feature, crowd flow direction feature, and / or crowd queue length feature in the target period corresponding to the target node and / or target path according to the target node feature and / or target path feature, improve the matching degree of crowd flow control strategy and specific scene, improve the generation accuracy of target crowd flow control strategy, thereby improving the execution accuracy of target control operation, and improve the intelligent level and efficiency of crowd flow control.

[0139] Step S3: generating a target crowd flow control strategy according to the target state feature.

[0140] Exemplarily, the target crowd flow control strategy corresponding to the target node and / or target path can be generated according to the crowd density feature, crowd moving speed feature, crowd flow direction feature, and / or crowd queue length feature in the target period corresponding to the target node and / or target path.

[0141] In some possible implementation manners, the step S3 of generating the target crowd flow regulation strategy according to the target state feature comprises:

[0142] The step S31 of generating the target crowd flow dynamic change graph according to the target state feature.

[0143] For example, the target crowd flow dynamic change graph corresponding to the target node and / or the target path can be generated according to the crowd density feature, the crowd moving speed feature, the crowd flow direction feature, and / or the crowd queue length feature in the target time period corresponding to the target node and / or the target path.

[0144] In some possible implementation manners, the target crowd flow dynamic change graph comprises a first target layer, a second target layer, a third target layer, and / or a fourth target layer.

[0145] The first target layer corresponds to a target crowd density thermal dynamic change graph.

[0146] For example, the first target layer, that is, the target crowd density thermal dynamic change graph corresponding to the target node and / or the target path can be generated according to the crowd density feature in the target time period corresponding to the target node and / or the target path.

[0147] The second target layer corresponds to a target moving speed vector field dynamic change graph.

[0148] For example, the second target layer, that is, the target moving speed vector field dynamic change graph corresponding to the target node and / or the target path can be generated according to the crowd moving speed feature in the target time period corresponding to the target node and / or the target path.

[0149] The third target layer corresponds to a target flow direction vector field dynamic change graph.

[0150] For example, the third target layer, that is, the target flow direction vector field dynamic change graph corresponding to the target node and / or the target path can be generated according to the crowd moving speed feature and / or the crowd flow direction feature in the target time period corresponding to the target node and / or the target path.

[0151] The fourth target layer corresponds to a target queue length dynamic change graph.

[0152] For example, the fourth target layer, that is, the target queue length dynamic change graph corresponding to the target node and / or the target path can be generated according to the crowd queue length feature in the target time period corresponding to the target node and / or the target path.

[0153] It should be noted that the first target layer, the second target layer, the third target layer, and / or the fourth target layer can also be displayed individually, hierarchically, and / or superimposed according to the actual needs of the target user.

[0154] Therefore, the above method can accurately generate a target crowd density heat dynamic change map, a target moving speed vector field dynamic change map, a target flow direction vector field dynamic change map, and / or a target queue length dynamic change map corresponding to the target node and / or the target path according to the crowd density feature, the crowd moving speed feature, the crowd flow direction feature, and / or the crowd queue length feature in the target period corresponding to the target node and / or the target path, so as to accurately improve the matching degree of the crowd flow control strategy and the specific scene, provide data support for accurately generating the target crowd flow control strategy, and improve the intelligent level and efficiency of crowd flow control.

[0155] Step S32; the target crowd flow dynamic change map generates a target crowd flow control strategy.

[0156] Exemplarily, the release speed control strategy corresponding to the target node and / or the channel opening and closing state control strategy corresponding to the target path can be accurately generated according to the target crowd density heat dynamic change map, the target moving speed vector field dynamic change map, the target flow direction vector field dynamic change map, and / or the target queue length dynamic change map corresponding to the target node and / or the target path.

[0157] Therefore, the above method can accurately generate a target crowd flow dynamic change map according to the target state feature, accurately generate a target node release speed control strategy and / or a target channel opening and closing state control strategy according to the target crowd flow dynamic change map, thereby improving the execution accuracy of the target node release speed control operation and / or the target channel opening and closing state control operation, and improving the intelligent level and efficiency of crowd flow control.

[0158] In some possible implementations, the above method further includes:

[0159] Step S321; in the case that the target difference corresponding to the target crowd flow dynamic change map and the historical crowd flow dynamic change map is greater than a fifth preset threshold, a target abnormal event speculation result is generated according to the target difference; wherein the target abnormal event speculation result includes a fall event speculation result, a panic running event speculation result, a reverse event speculation result, and / or a facility operation abnormal event speculation result.

[0160] Exemplarily, in a case that a target difference exists between the first target layer corresponding to the target crowd flow dynamic change graph, i.e., the target crowd density heat dynamic change graph, and the first target layer corresponding to the historical crowd flow dynamic change graph, i.e., the historical crowd density heat dynamic change graph, and the target difference is greater than a fifth preset threshold, a target abnormal event speculation result can be generated according to the target difference.

[0161] Specifically, the target difference can be identified and determined based on the cosine similarity combined with the local peak value detection algorithm. In a case that the target difference is greater than the fifth preset threshold, for example, in a case of sudden increase of local crowd density, a fall event speculation result can be generated according to the target difference.

[0162] Exemplarily, in a case that a target difference exists between the second target layer corresponding to the target crowd flow dynamic change graph, i.e., the target moving speed vector field dynamic change graph, and the second target layer corresponding to the historical crowd flow dynamic change graph, i.e., the historical moving speed vector field dynamic change graph, and the target difference is greater than a fifth preset threshold, a target abnormal event speculation result can be generated according to the target difference.

[0163] Specifically, the target difference can be identified and determined based on the moving speed direction angle change rate combined with the optical flow analysis algorithm. In a case that the target difference is greater than the fifth preset threshold, for example, in a case of sudden change of speed direction, a panic running event speculation result can be generated according to the target difference.

[0164] Exemplarily, in a case that a target difference exists between the third target layer corresponding to the target crowd flow dynamic change graph, i.e., the target flow direction vector field dynamic change graph, and the third target layer corresponding to the historical crowd flow dynamic change graph, i.e., the historical flow direction vector field dynamic change graph, and the target difference is greater than a fifth preset threshold, a target abnormal event speculation result can be generated according to the target difference.

[0165] Specifically, the target difference can be identified and determined based on the flow direction vector average angle deviation combined with the principal component analysis algorithm. In a case that the target difference is greater than the fifth preset threshold, for example, in a case of existing direction vector, a reverse event speculation result can be generated according to the target difference.

[0166] Exemplarily, in a case that a target difference exists between the fourth target layer corresponding to the target crowd flow dynamic change graph, i.e., the target queue length dynamic change graph, and the fourth target layer corresponding to the historical crowd flow dynamic change graph, i.e., the historical queue length dynamic change graph, and the target difference is greater than a fifth preset threshold, a target abnormal event speculation result can be generated according to the target difference.

[0167] Specifically, the target difference can be determined based on the queue length mean square error combined with the main morphological dilation detection algorithm. In the case where the target difference is greater than the corresponding fifth preset threshold, for example, in the case of abnormal accumulation, the facility operation abnormal event speculation result can be generated according to the target difference.

[0168] It should be noted that the fifth preset threshold corresponding to the target difference between the target crowd density dynamic change map and the historical crowd density dynamic change map, the fifth preset threshold corresponding to the target difference between the target moving speed vector field dynamic change map and the historical moving speed vector field dynamic change map, the fifth preset threshold corresponding to the target difference between the target flow direction vector field dynamic change map and the historical flow direction vector field dynamic change map, and / or the fifth preset threshold corresponding to the target difference between the target queue length dynamic change map and the historical queue length dynamic change map can be different according to actual scene requirements.

[0169] Step S322; generating a target abnormality mark according to the target abnormal event speculation result.

[0170] Exemplarily, the target abnormality mark corresponding to the fall event speculation result, the panic running event speculation result, the reverse event speculation result, and / or the facility operation abnormal event speculation result can be generated.

[0171] Step S323; correcting the target crowd flow control strategy according to the target abnormality mark.

[0172] Exemplarily, the target crowd flow control strategy is corrected according to the target abnormality mark.

[0173] Specifically, in the case where the target abnormality mark corresponds to the fall event speculation result, the target crowd flow control strategy can be corrected to further reduce the release speed of the surrounding target node.

[0174] Specifically, in the case where the target abnormality mark corresponds to the panic running event speculation result, the target crowd flow control strategy can be corrected to further segment the panic running event occurrence area, and simultaneously start the standby evacuation channel.

[0175] Specifically, in the case where the target abnormality mark corresponds to the reverse event speculation result, the target crowd flow control strategy can be corrected to set a one-way channel to further strengthen the guidance of crowd flow direction.

[0176] Specifically, in the case where the target abnormality mark corresponds to the facility operation abnormal event speculation result, the target crowd flow control strategy can be corrected to close the faulty facility, and simultaneously further start the standby channel.

[0177] It should be noted that in the presence of composite event speculation results, the target crowd flow regulation strategy can be modified to start the corresponding regional blockade to isolate the impact of the above composite event.

[0178] Step S4: According to the target crowd flow regulation strategy, the target regulation operation is performed.

[0179] Exemplarily, the release speed regulation strategy corresponding to the target node, and / or the channel opening and closing state regulation strategy corresponding to the target path, controls the target node, and / or the facilities corresponding to the target path, to perform the corresponding target regulation operation.

[0180] Specifically, according to the modified target crowd flow regulation strategy, for example: reducing the release speed of the surrounding target node, segmenting the area where the abnormal event occurs, starting the standby evacuation channel, setting the one-way channel, closing the faulty facility, and / or starting the corresponding regional blockade, the target node, and / or the facilities corresponding to the target path are controlled to perform the target regulation operation.

[0181] Therefore, the above method can accurately and automatically generate the fall event speculation result, the panic running event speculation result, the reverse event speculation result, and / or the facility operation abnormal event speculation result according to the target difference when the target difference corresponding to the target crowd flow dynamic change graph and the historical crowd flow dynamic change graph is greater than the fifth preset threshold; accurately and automatically generate the target abnormality mark according to the fall event speculation result, the panic running event speculation result, the reverse event speculation result, and / or the facility operation abnormal event speculation result; and modify the target crowd flow regulation strategy according to the target abnormality mark, thereby further improving the generation accuracy of the target crowd flow regulation strategy, further improving the execution accuracy of the target regulation operation, and further improving the intelligent level and efficiency of crowd flow control in specific scenarios.

[0182] In some possible implementations, the above method further comprises:

[0183] Step S5: According to the target scene feature, a target agent networking is constructed.

[0184] Exemplarily, the target agent networking can be constructed according to the target node feature and / or the target path feature, so as to generate the target crowd flow regulation strategy corresponding to the target node and / or the target path according to the target state feature corresponding to the target node and / or the target path, and control the target node and / or the target path to automatically perform the target regulation operation according to the target crowd flow regulation strategy corresponding to the target node and / or the target path.

[0185] Therefore, the above method can realize automatic hierarchical management of the multi-level target node and / or target path, improve the execution efficiency of the target node and / or target path corresponding to the target control operation, and improve the intelligent level and efficiency of crowd flow control.

[0186] In some possible implementation manners, the above constructing the target intelligent agent networking according to the target scene characteristics comprises:

[0187] Step S51: deploying the corresponding target intelligent agent at the target node and / or target path.

[0188] For example, the target intelligent agent can be set according to the target state characteristics corresponding to the target node and / or target path.

[0189] Step S52: determining the target intelligent agent running resource allocation parameter according to the crossing degree, bottleneck degree and / or vulnerability degree corresponding to the target node and / or target path.

[0190] For example, the target intelligent agent running resource allocation parameter can be determined according to the crossing degree, bottleneck degree and / or vulnerability degree corresponding to the target node and / or target path.

[0191] Step S53: performing a running resource allocation operation according to the target intelligent agent running resource allocation parameter.

[0192] For example, the target intelligent agent can be performed with the corresponding running resource allocation operation according to the above target intelligent agent running resource allocation parameter.

[0193] Therefore, the above method can accurately determine the target intelligent agent running resource allocation parameter according to the crossing degree, bottleneck degree and / or vulnerability degree corresponding to the target node and / or target path, improve the accuracy of the target intelligent agent networking running resource allocation, improve the execution efficiency of the target node and / or target path corresponding to the target control operation, and further improve the intelligent level and efficiency of crowd flow control in a specific scene.

[0194] In some possible implementation manners, the above determining the target intelligent agent running resource allocation parameter according to the crossing degree, bottleneck degree and / or vulnerability degree corresponding to the target node and / or target path comprises:

[0195] The calculation resource allocation parameter is determined according to the following formula:

[0196]

[0197] C PUiis used to represent the computing resource allocation parameter corresponding to the target agent i; Base is used to represent the basic guarantee resource; Δ is used to represent the scalable computing resource; K i is used to represent the comprehensive criticality index corresponding to the target agent i; θ is used to represent the computing resource transition threshold; σ is used to represent the first smoothing coefficient;

[0198] According to the following formula, the memory allocation parameter is determined:

[0199]

[0200] Wherein, R AMi is used to represent the memory allocation parameter corresponding to the target agent i; α3 is used to represent the memory allocation base; K i is used to represent the comprehensive criticality index corresponding to the target agent i; β4 is used to represent the comprehensive criticality index K i corresponding to the amplification factor; Offset is used to represent the basic guarantee memory;

[0201] According to the following formula, the communication bandwidth allocation parameter is determined:

[0202]

[0203] Wherein, BW i is used to represent the communication bandwidth allocation parameter corresponding to the target agent i; BW min is used to represent the minimum bandwidth guarantee value; BW max is used to represent the maximum bandwidth upper limit value; K i is used to represent the comprehensive criticality index corresponding to the target agent i; ∈ is used to represent the second smoothing coefficient;

[0204] And / or, according to the following formula, the decision frequency allocation parameter is determined:

[0205]

[0206] Wherein, f i is used to represent the decision frequency allocation parameter corresponding to the target agent i; f max is used to represent the maximum decision frequency corresponding to the target agent i; η is used to represent the frequency growth sensitivity; K i is used to represent the comprehensive criticality index corresponding to the target agent i; K0 is used to represent the decision frequency effective threshold;

[0207] Wherein, the comprehensive criticality index K i corresponding to the target agent i is determined according to the following formula:

[0208]

[0209] Wherein, Ki is used to represent the comprehensive criticality index corresponding to the target agent i; C i is used to represent the normalized intersection degree; ω c is used to represent the normalized intersection degree C i corresponding weight; B i is used to represent the normalized bottleneck degree; ω b is used to represent the normalized bottleneck degree B i corresponding weight; γ4is used to represent the nonlinear adjustment factor; V i is used to represent the normalized vulnerability degree; ω v is used to represent the normalized vulnerability degree V i corresponding weight.

[0210] Therefore, the above method can accurately and automatically determine the computing resource allocation parameter, memory allocation parameter, communication bandwidth allocation parameter, and / or decision frequency allocation parameter of the corresponding target agent according to the intersection degree, bottleneck degree, and / or vulnerability degree corresponding to the target node and / or target path, so as to further improve the resource allocation accuracy of the target agent group network operation, thereby further improving the automatic execution efficiency of the target control operation corresponding to the target node and / or target path, and further improving the intelligent level and efficiency of crowd flow control in a specific scene.

[0211] Based on this, the crowd flow control method based on reinforcement learning provided in the present application comprises: acquiring target scene features; determining target state features according to the target scene features; generating a target crowd flow control strategy according to the target state features; and executing a target control operation according to the target crowd flow control strategy; wherein the target scene features comprise target node features and / or target path features. In this way, the matching degree of the crowd flow control strategy and the specific scene can be improved, so as to improve the intelligent level and efficiency of crowd flow control.

[0212] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the action order described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0213] The above is the introduction of the method embodiment, and the scheme described in the present application will be further described through the device embodiment below.

[0214] The second aspect of the embodiment of the present application proposes a crowd flow control system based on reinforcement learning, which is suitable for the method described above. Figure 2A structural schematic diagram of a crowd flow control system 200 based on reinforcement learning is provided for an embodiment of the present application. As shown in the figure Figure 2 The crowd flow control system 200 based on reinforcement learning includes an acquisition unit 210, a determination unit 220, a generation unit 230, and an execution unit 240.

[0215] The acquisition unit 210 is configured to acquire target scene features.

[0216] The determination unit 220 is configured to determine target state features according to the target scene features.

[0217] The generation unit 230 is configured to generate a target crowd flow regulation strategy according to the target state features.

[0218] The execution unit 240 is configured to perform a target regulation operation according to the target crowd flow regulation strategy.

[0219] The target scene features include target node features and / or target path features.

[0220] Figure 3 A structural schematic diagram of an electronic device 300 is provided for an embodiment of the present application. As shown in the figure Figure 3 The electronic device 300 includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 302 or loaded from a storage portion 308 into a random access memory (RAM) 303. Various programs and data required for terminal device or server operation are also stored in the RAM 303. The CPU 301, the ROM 302, and the RAM 303 are connected to each other through a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0221] The following components are connected to the I / O interface 305: an input portion 306 including a keyboard, a mouse, and the like; an output portion 307 including a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a speaker, and the like; a storage portion 308 including a hard disk, and the like; and a communication portion 309 including a network interface card such as a LAN card, a modem, and the like. The communication portion 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as necessary. A removable medium 311 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is mounted on the drive 310 as necessary, so that a computer program read therefrom is installed in the storage portion 308 as necessary.

[0222] In particular, the above method flow steps can be implemented as a computer software program in accordance with embodiments of the present application. For example, embodiments of the present application include a computer program product which includes a computer program tangibly embodied on a machine readable medium, the computer program containing program code for executing the methods illustrated in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 309, and / or installed from the removable media 311. When the computer program is executed by the central processing unit (CPU) 301, the above-described functions defined in the system of the present application are executed.

[0223] Note that the computer readable medium described in the present application can be a computer readable signal medium or a computer readable storage medium or any combination thereof. The computer readable storage medium may, for example, be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present application, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device. In the present application, the computer readable signal medium can include a data signal propagated in a baseband or as a carrier wave in a propagated data signal, in which the computer readable program code is carried. Such a propagated data signal can take many forms, including but not limited to, an electromagnetic signal, an optical signal or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that can send, propagate or transfer the program for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the computer readable medium can be transmitted using any suitable medium, including but not limited to, wireless, wire line, optical fiber cable, RF, etc., or any suitable combination thereof.

[0224] The computer program product of the present application can be a computer program embodied on a non-transitory computer readable medium. Such non-transitory computer readable medium can include, but is not limited to, hard disks, floppy disks, CD-ROMs, DVDs, flash memories, computer memory, or any other suitable medium. The computer program product can be a computer program tangibly embodied on a computer readable medium. The computer program product can be a computer program tangibly embodied on a computer readable medium.

[0225] The units or modules described in the embodiments of the present application can be implemented by software or hardware. The described units or modules can also be arranged in a processor. In some cases, the names of the units or modules do not constitute a limitation on the units or modules themselves.

[0226] The above description is merely illustrative of the principles of the application. Depending upon the implementation, the application can or can not be implemented using the described features. Other arrangements, which are presently deemed better modes of the application, can be devised by those skilled in the art without departing from the essential characteristics of the application. Furthermore, it is to be understood that the application is defined by the appended claims.

Claims

1. A crowd flow control method based on reinforcement learning, characterized in that, The method comprises: acquiring target scene features; determining target state features according to the target scene features; generating a target crowd flow regulation strategy according to the target state features; executing a target regulation operation according to the target crowd flow regulation strategy; wherein the target scene features comprise target node features and / or target path features.

2. The method of claim 1, wherein, The flow degree of the target node and / or the target path is greater than or equal to a first preset threshold value; The crossing degree of the target node and / or the target path is greater than or equal to a second preset threshold value; The bottleneck degree of the target node and / or the target path is greater than or equal to a third preset threshold value; And / or, the vulnerability of the target node and / or the target path is greater than or equal to a fourth preset threshold value.

3. The method of claim 1, wherein, The target state features comprise crowd density features, crowd moving speed features, crowd flow direction features, and / or crowd queue length features in a target period.

4. The method of claim 1, wherein, According to the target state features, a target crowd flow regulation strategy is generated, which comprises: generating a target crowd flow dynamic change graph according to the target state features; The target crowd flow dynamic change graph generates the target crowd flow regulation strategy; wherein the target crowd flow regulation strategy comprises a target node release speed regulation strategy and / or a target channel opening and closing state regulation strategy.

5. The method of claim 4, wherein, The target crowd flow dynamic change graph comprises a first target layer, a second target layer, a third target layer, and / or a fourth target layer; wherein the first target layer corresponds to a target crowd density dynamic change graph; The second target layer corresponds to a target moving speed vector field dynamic change graph; The third target layer corresponds to a target flow direction vector field dynamic change graph; The fourth target layer corresponds to a target queue length dynamic change graph.

6. The method of claim 4, wherein, Further comprising: in the case where the target difference between the target crowd flow dynamic change graph and the historical crowd flow dynamic change graph is greater than a fifth preset threshold value, generating a target abnormal event speculation result according to the target difference; generating a target abnormality label according to the target abnormal event speculation result; wherein the target abnormal event speculation result comprises a fall event speculation result, a panic running event speculation result, a reverse event speculation result, and / or a facility operation abnormal event speculation result; correcting the target crowd flow regulation strategy according to the target abnormality label.

7. The method according to any one of claims 1 to 6, characterized in that, Further comprising: constructing a target intelligent agent networking according to the target scene features.

8. The method of claim 7, wherein, The target intelligent agent networking is constructed according to the target scene features, which comprises: determining corresponding target intelligent agent running resource allocation parameters according to the crossing degree, bottleneck degree, and / or vulnerability of the target node and / or the target path; executing a running resource allocation operation according to the target intelligent agent running resource allocation parameters.

9. The method of claim 8, wherein, The target intelligent agent running resource allocation parameters are determined according to the crossing degree, bottleneck degree, and / or vulnerability of the target node and / or the target path, which comprises: determining the calculation resource allocation parameters according to the following formula: wherein C PUi is used to represent the computing resource allocation parameter corresponding to the target agent i; Base is used to represent the basic guarantee resource; Δ is used to represent the scalable computing resource; K i is used to represent the comprehensive criticality index corresponding to the target agent i; θ is used to represent the computing resource transition threshold; σ is used to represent the first smoothing coefficient; The memory allocation parameter is determined according to the following formula: wherein R AMi is used to represent the memory allocation parameter corresponding to the target agent i; a3 is used to represent the memory allocation base; K i is used to represent the comprehensive criticality index corresponding to the target agent i; b4 is used to represent the comprehensive criticality index K i corresponding to the amplification factor; Offset is used to represent the basic guarantee memory; The communication bandwidth allocation parameter is determined according to the following formula: BW i BW min BW max K i ∈ And / or, the decision frequency allocation parameter is determined according to the following formula: wherein, f i is used to represent the decision frequency allocation parameter corresponding to the target agent i; f max is used to represent the maximum decision frequency corresponding to the target agent i; η is used to represent the frequency growth sensitivity; K i is used to represent the comprehensive criticality index corresponding to the target agent i; K0 is used to represent the decision frequency effectiveness threshold; Wherein, the target intelligent agent i corresponds to the comprehensive key degree index K i , is determined according to the following formula: wherein, K i for representing the integrated criticality indicator corresponding to the target agent i; C i for representing the normalized intersection degree; ω c for representing the normalized intersection degree C i the corresponding weight; B i for representing the normalized bottleneck degree; ω b for representing the normalized bottleneck degree B i the corresponding weight; γ4for representing the non-linear adjustment factor; V i for representing the normalized vulnerability degree; ω v for representing the normalized vulnerability degree V i the corresponding weight.

10. A crowd flow control system based on reinforcement learning, adapted to the method of claim 1, characterized in that, Comprise: An acquisition unit is configured to acquire target scene features; A determination unit is configured to determine target state features according to the target scene features; A generation unit is configured to generate a target crowd flow regulation strategy according to the target state features; An execution unit is configured to perform a target regulation operation according to the target crowd flow regulation strategy; The target scene features include target node features and / or target path features.

Images