Urban traffic control method based on reinforcement learning

Abstract

The invention discloses an urban traffic control method based on reinforcement learning, and relates to the technical field of urban traffic control, and the method comprises the following steps: S1, the urban traffic control method based on reinforcement learning is applied to urban traffic management in rainstorm weather, and the related data of taxies are obtained through an urban traffic information management platform; and processing the acquired relevant data of the taxi, and acquiring a driving track of the taxi based on the processed relevant data of the taxi. The invention provides an urban traffic control method based on reinforcement learning, and the urban traffic control method based on reinforcement learning is applied to urban traffic management in rainstorm weather. According to the method, the passenger flow data and the order average travel speed data of the taxi are obtained by acquiring and processing the GPS data in the operation state of the taxi in the rainstorm weather and the corresponding normal weather before the rainstorm, and the corresponding data of the future time sequence is predicted based on the constructed SARIMA model.

Description

Technical Field

[0001] The present invention relates to the technical field of urban traffic control, and in particular to an urban traffic control method based on reinforcement learning. Background Art

[0002] Urban traffic refers to public travel and passenger and freight transportation between urban road systems. The Chinese patent with application number 202210725869.7 discloses "an intelligent control method for urban traffic based on traffic informatization, comprising the following steps: S1: based on video intelligent analysis and precise positioning of the video, real-time collection of vehicle flow and passenger flow information, and combined with various road monitoring facilities and traffic police command and control system data, combined with video intelligent analysis, intelligent positioning, and intelligent judgment technology, structured intelligent analysis is performed on the vehicle image information taken by the checkpoint, electric police, and some monitoring equipment, and abnormal data is alarmed. The intelligent traffic visualization decision-making platform accesses traffic road condition data to visualize the monitoring and analysis of sudden traffic incident information, and screens and views the monitoring video and traffic resources around the accident. The intelligent control method and system for urban traffic based on traffic informatization can help managers grasp the overall traffic operation situation in real time by setting a monitoring module, effectively maintain road traffic order, and realize intelligent traffic visualization management. "; This existing technology only solves the problem that traffic congestion has become the primary problem affecting the travel of residents in large cities in recent years, and the number of traffic accidents has been on the rise. It does not take into account the resilience assessment of the urban road traffic system under heavy rain weather, and the determination of the criticality of each network node in the urban road traffic system under heavy rain weather, and then determine the key nodes in each network node. By setting up emergency resource facility points to cover the key nodes, the resilience of the key nodes is improved, thereby improving the resilience of the urban road traffic system under heavy rain weather, and facilitating the management of urban road traffic under heavy rain weather. Summary of the invention

[0003] The purpose of the present invention is to provide an urban traffic control method based on reinforcement learning to solve the problems raised in the above background technology.

[0004] To achieve the above object, the present invention provides the following technical solution: an urban traffic control method based on reinforcement learning, comprising the following steps: S1: A reinforcement learning-based urban traffic control method is applied to urban traffic management in rainstorm weather. Taxi-related data is obtained through an urban traffic information management platform, and the obtained taxi-related data is processed, and the taxi driving trajectory is obtained based on the processed taxi-related data; S2: Set the performance evaluation indicators of the urban road traffic system, draw the performance evolution curve of the urban road traffic system under heavy rain weather, and build an urban road traffic system resilience evaluation framework under heavy rain weather, set up corresponding urban road traffic system resilience measurement indicators, and conduct urban road traffic system resilience evaluation under heavy rain weather; S3: Divide the city into regions, process the taxi driving trajectory data, and construct a two-layer network consisting of a travel time layer and a travel flow layer. By measuring the resilience of the two-layer network nodes, the key nodes in the two-layer network are determined; S4: Based on the key node information in the determined two-layer network nodes, emergency resource facility points are set up, and each network node is covered in a differentiated coverage manner, thereby improving the resilience of the urban road traffic system under heavy rain weather.

[0005] Preferably, step S1 also includes acquiring relevant data on the operating status of taxis during heavy rain and the corresponding normal weather before the heavy rain through the urban traffic information management platform, including taxi license plate, data upload time, GPS time, longitude, latitude and taxi status, and preprocessing the acquired GPS data on the operating status of taxis during heavy rain and the corresponding normal weather before the heavy rain, including using Python software to read the GPS data on the operating status of taxis during heavy rain and the corresponding normal weather before the heavy rain and perform data cleaning, and perform coordinate system conversion operations on the GPS data on the operating status of taxis during heavy rain and the corresponding normal weather before the heavy rain, adopting the HMM matching algorithm to match the taxi GPS data with the urban road network information, using the Python third-party library GeoPandas to spatially connect the taxi GPS data with the spatial map data of the urban target area, extracting the taxi GPS data in the urban target area, and obtaining the taxi GPS data in the urban target area during heavy rain and the corresponding normal weather before the heavy rain.

[0006] Preferably, step S1 also includes arranging the GPS data of the same taxi in a time series based on the GPS data of taxis in the target area of ​​the city to form a driving trajectory of the taxi, determining the passenger boarding point time and the corresponding spatial position based on the time point and longitude and latitude information when the vehicle operation state changes from an empty state to a passenger state in the taxi GPS data, determining the passenger alighting point time and the corresponding spatial position when the vehicle operation state changes from a passenger state to an empty state, determining the taxi order travel distance based on the spatial positions of the passenger boarding point and the alighting point, determining the taxi order duration based on the time points of the passenger boarding point and the alighting point, calculating the ratio of the taxi order travel distance to the taxi order duration to obtain the average travel speed of the order, taking the number of taxi orders within one hour as the passenger flow of the taxi, and setting the passenger flow data and the average travel speed data of the taxi orders in rainstorm weather to be lower than the passenger flow data and the average travel speed data of the taxi orders in normal weather before the corresponding rainstorm.

[0007] Preferably, step S2 also includes setting performance evaluation indicators of the urban road traffic system, including the transportation quantity and transportation efficiency of the urban road traffic system, aggregating the performance evaluation indicators of the urban road traffic system with one hour as the minimum time unit, using the passenger flow data of taxis in the urban target area under heavy rain weather and normal weather before the corresponding heavy rain as the transportation quantity indicator of the urban road traffic system under the corresponding weather, using the average travel speed of all taxi orders in the urban target area under heavy rain weather and normal weather before the corresponding heavy rain as the transportation efficiency indicator of the urban road traffic system under the corresponding weather, using the statsmodels module in the Python software to analyze the taxi passenger flow data in the urban target area under heavy rain weather and construct a SARIMA model, and based on the passenger flow data of taxis in the urban target area under clear weather The model training data set and model verification data set are constructed based on the large amount of data. The SARIMA model is trained by the model training data set, and the accuracy of the trained SARIMA model is verified based on the model verification data set. The passenger flow data of taxis in the target area of ​​the city under normal weather before the heavy rain is used as the input of the SARIMA model, and the passenger flow data of taxis in the target area of ​​the city under normal weather at the same time as the corresponding heavy rain is predicted. The predicted result is subtracted from the passenger flow data of taxis in the target area of ​​the city under the corresponding heavy rain, and the disturbance evolution curve of the taxi passenger flow data of the urban road traffic system under heavy rain is drawn based on the difference processing result. The disturbance evolution curve of the average travel speed data of taxi orders in the urban road traffic system under heavy rain is drawn using the same method as above.

[0008] Preferably, step S2 also includes constructing an urban road traffic system resilience assessment framework under heavy rain weather, which is composed of set urban road traffic system resilience measurement indicators, including the urban road traffic system resilience loss, the urban road traffic system disturbance duration, the urban road traffic system damage duration, the urban road traffic system recovery duration and system resilience. Taking the resilience measurement of taxi passenger flow data of the urban road traffic system under heavy rain weather as an example, the calculation formulas of the resilience measurement indicators of each urban road traffic system are as follows: , where It represents the resilience loss of taxi passenger flow data in urban road traffic system under heavy rain weather. It indicates the duration of disturbance of taxi passenger flow data in urban road traffic system under heavy rain. It indicates the duration of the destruction of taxi passenger flow data in the urban road traffic system under heavy rain weather. It indicates the time it takes for taxi passenger flow data in the urban road traffic system to recover under heavy rain. It indicates the resilience of taxi passenger flow data in urban road traffic system under heavy rain. Indicates that the urban road traffic system is Taxi passenger flow data predicted by SARIMA model at each moment, Indicates that the urban road traffic system is Taxi passenger flow data at each moment, It represents the time corresponding to the initial point of the disturbance evolution curve of taxi passenger flow data in the urban road traffic system under heavy rain weather. It represents the time corresponding to the end point of the disturbance evolution curve of taxi passenger flow data in the urban road traffic system under heavy rain weather. It represents the time corresponding to the highest point of the disturbance evolution curve of taxi passenger flow data in the urban road traffic system under heavy rain weather. The same method as above is used to measure the resilience of the average travel speed data of taxi orders in the urban road traffic system under heavy rain weather.

[0009] Preferably, step S3 further includes spatially dividing the urban target area according to the traffic zone division method, taking the divided traffic zones as network nodes, and mapping the taxi trajectory data to different traffic zones through spatial connection, so that the taxi trajectory data is converted into trajectory data of flow between traffic zones, and the duration between the time of first entering the traffic zone and the time of first entering its adjacent traffic zone in the trajectory data is taken as the travel duration between the traffic zones, and a double-layer network consisting of a travel time layer and a travel flow layer is constructed, and when constructing the travel flow layer network, it is necessary to delete the trajectory data whose starting point and end point are both located in the same traffic zone, each network node in the travel flow layer network represents a different traffic zone, and the edge between two network nodes represents the number of passengers between the corresponding traffic zones, and each network node in the constructed travel time layer network represents a different traffic zone, and the edge between two network nodes represents the average order duration of taxis between the corresponding traffic zones.

[0010] Preferably, step S3 further includes measuring the resilience of the double-layer network nodes, determining the key nodes in the double-layer network nodes, and taking the resilience measurement of the travel flow layer network nodes under heavy rain weather as an example, the resilience value of the travel flow layer network nodes under heavy rain weather is calculated by the node resilience measurement algorithm, and the specific calculation formula is as follows: , where Represents the network nodes of the travel flow layer under heavy rain weather The toughness value, Representation Node In heavy rain The number of passengers at a given time, Representation Node Under normal weather conditions at the same time as the corresponding rainstorm The number of passengers at the time, and the number of nodes The toughness loss is calculated and recorded as ,Use the same method as above to calculate the resilience value of the travel time layer network node under heavy rain weather. Based on the calculation results of the resilience value and resilience loss of each network node in the travel flow layer network and the travel time layer network under heavy rain weather, determine the key nodes of the travel flow layer network and the travel time layer network. Specifically, the smaller the network node resilience value, the more critical the network node. When the network node resilience value is the same, the network node with a larger resilience loss is more critical. The criticality of each network node is calculated by the network node key algorithm. The specific calculation formula is as follows: , where Representation Node The criticality of mobility resilience, It represents the maximum value of resilience loss of all network nodes in the travel flow layer under heavy rain weather. Representation Node The criticality of flow toughness, It represents the maximum value of resilience loss of all network nodes in the travel time layer under heavy rain weather. Represents the network nodes of the travel time layer under heavy rain weather The toughness loss, Represents the network nodes of the travel time layer under heavy rain weather The toughness value, Representation Node The criticality of represents the weight of the criticality of travel resilience, represents the weight of the criticality of flow toughness, and , based on the criticality calculation results of each network node, set the top 20 network nodes as critical nodes.

[0011] Preferably, step S4 also includes setting up emergency resource facility points in the target area of ​​the city based on the determined key node information, and each emergency resource facility point adopts a cooperative coverage method to provide the network nodes with the emergency materials needed for heavy rain, and adopts a differentiated coverage method to cover each network node, specifically, multiple coverage methods are adopted for key nodes, and non-key nodes may be allowed to be not fully covered, thereby improving the resilience of the urban road traffic system under heavy rain weather.

[0012] Compared with the prior art, the beneficial effects of the present invention at least include: the present invention proposes an urban traffic control method based on reinforcement learning, and the urban traffic control method based on reinforcement learning is applied to urban traffic management under heavy rain weather, and the passenger flow data and the average travel speed data of the taxi are obtained by obtaining and processing the GPS data of the taxi operation status under heavy rain weather and the corresponding normal weather before the heavy rain, and the corresponding data of the future time series are predicted based on the constructed SARIMA model, and then the disturbance evolution curve of the taxi passenger flow data of the urban road traffic system under heavy rain weather and the disturbance evolution curve of the average travel speed data of the taxi order of the urban road traffic system under heavy rain weather are constructed, based on which the resilience measurement index of the urban road traffic system is set to measure the resilience of the taxi passenger flow data and the average travel speed data of the taxi order of the urban road traffic system under heavy rain weather, respectively, and the key nodes are determined by constructing a two-layer network and measuring the resilience of the network nodes therein, and the resilience of the key nodes is improved by setting emergency resource facility points to cover the key nodes, thereby improving the resilience of the urban road traffic system under heavy rain weather. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1A schematic flow chart of a method for urban traffic control based on reinforcement learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0014] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0015] See also Figure 1 The present invention provides a technical solution: an urban traffic control method based on reinforcement learning, comprising the following steps: S1: A reinforcement learning-based urban traffic control method is applied to urban traffic management in rainstorm weather. Taxi-related data is obtained through an urban traffic information management platform, and the obtained taxi-related data is processed, and the taxi driving trajectory is obtained based on the processed taxi-related data; S2: Set the performance evaluation indicators of the urban road traffic system, draw the performance evolution curve of the urban road traffic system under heavy rain weather, and build an urban road traffic system resilience evaluation framework under heavy rain weather, set up corresponding urban road traffic system resilience measurement indicators, and conduct urban road traffic system resilience evaluation under heavy rain weather; S3: Divide the city into regions, process the taxi driving trajectory data, and construct a two-layer network consisting of a travel time layer and a travel flow layer. By measuring the resilience of the two-layer network nodes, the key nodes in the two-layer network are determined; S4: Based on the key node information in the determined two-layer network nodes, emergency resource facility points are set up, and each network node is covered in a differentiated coverage manner, thereby improving the resilience of the urban road traffic system under heavy rain weather.

[0016] Step S1 also includes acquiring relevant data of taxi operation status in rainstorm weather and normal weather before the rainstorm through the urban traffic information management platform, including taxi license plate, data upload time, GPS time, longitude, latitude and taxi status, and preprocessing the acquired GPS data of taxi operation status in rainstorm weather and normal weather before the rainstorm, including using Python software to read the GPS data of taxi operation status in rainstorm weather and normal weather before the rainstorm and perform data cleaning, and perform coordinate system conversion operation on the GPS data of taxi operation status in rainstorm weather and normal weather before the rainstorm, adopting HMM matching algorithm to match taxi GPS data with urban road network information, using Python third-party library GeoPandas to spatially connect taxi GPS data with spatial map data of urban target area, extracting taxi GPS data in urban target area, and obtaining taxi GPS data in urban target area in rainstorm weather and normal weather before the rainstorm; Step S1 also includes arranging the GPS data of the same taxi in a time series based on the GPS data of taxis in the target area of ​​the city to form a driving trajectory of the taxi, determining the time of the passenger boarding point and the corresponding spatial position based on the time point and longitude and latitude information when the vehicle operation state changes from the empty state to the passenger state in the taxi GPS data, determining the time of the passenger alighting point and the corresponding spatial position when the vehicle operation state changes from the passenger state to the empty state and the longitude and latitude information, determining the taxi order travel distance based on the spatial positions of the passenger boarding point and the alighting point, determining the taxi order duration based on the time points of the passenger boarding point and the alighting point, calculating the ratio of the taxi order travel distance to the taxi order duration to obtain the average travel speed of the order, taking the number of taxi orders within one hour as the passenger flow of the taxi, and setting the passenger flow data of the taxi and the average travel speed data of the taxi order in rainstorm weather to be lower than the passenger flow data of the taxi and the average travel speed data of the taxi order in normal weather before the corresponding rainstorm; Step S2 also includes setting performance evaluation indicators of the urban road traffic system, including the transportation quantity and transportation efficiency of the urban road traffic system, aggregating the performance evaluation indicators of the urban road traffic system with one hour as the minimum time unit, using the passenger flow data of taxis in the urban target area under heavy rain weather and normal weather before the corresponding heavy rain as the transportation quantity indicator of the urban road traffic system under the corresponding weather, using the average travel speed of all taxi orders in the urban target area under heavy rain weather and normal weather before the corresponding heavy rain as the transportation efficiency indicator of the urban road traffic system under the corresponding weather, using the statsmodels module in the Python software to analyze the taxi passenger flow data in the urban target area under heavy rain weather and construct a SARIMA model, and based on the taxi passenger flow data in the urban target area under clear weather Construct a model training data set and a model verification data set, train the SARIMA model through the model training data set, and perform accuracy verification on the trained SARIMA model based on the model verification data set, use the passenger flow data of taxis in the urban target area under normal weather before the rainstorm as the input of the SARIMA model, predict the passenger flow data of taxis in the urban target area under normal weather at the same time as the corresponding rainstorm, perform difference processing on the predicted result and the passenger flow data of taxis in the urban target area under the corresponding rainstorm, draw the disturbance evolution curve of the taxi passenger flow data of the urban road traffic system under rainstorm weather based on the difference processing result, and use the same method as above to draw the disturbance evolution curve of the average travel speed data of taxi orders in the urban road traffic system under rainstorm weather; Step S2 also includes constructing an urban road traffic system resilience assessment framework under heavy rain weather, which is composed of set urban road traffic system resilience measurement indicators, including the urban road traffic system resilience loss, the urban road traffic system disturbance duration, the urban road traffic system damage duration, the urban road traffic system recovery duration and system resilience. Taking the resilience measurement of taxi passenger flow data of the urban road traffic system under heavy rain weather as an example, the calculation formulas of the resilience measurement indicators of each urban road traffic system are as follows: , where It represents the resilience loss of taxi passenger flow data in urban road traffic system under heavy rain weather. It indicates the duration of disturbance of taxi passenger flow data in urban road traffic system under heavy rain. It indicates the duration of the destruction of taxi passenger flow data in the urban road traffic system under heavy rain weather. It indicates the time it takes for taxi passenger flow data in the urban road traffic system to recover under heavy rain. It indicates the resilience of taxi passenger flow data in urban road traffic system under heavy rain. It indicates that the urban road traffic system is Taxi passenger flow data predicted by SARIMA model at each moment, Indicates that the urban road traffic system is Taxi passenger flow data at each moment, It represents the time corresponding to the initial point of the disturbance evolution curve of taxi passenger flow data in the urban road traffic system under heavy rain weather. It represents the time corresponding to the end point of the disturbance evolution curve of taxi passenger flow data in the urban road traffic system under heavy rain weather. It represents the time corresponding to the highest point of the disturbance evolution curve of taxi passenger flow data of the urban road traffic system under heavy rain weather. The same method as above is used to measure the resilience of the average travel speed data of taxi orders in the urban road traffic system under heavy rain weather. Step S3 also includes spatially dividing the urban target area according to the traffic zone division method, taking the divided traffic zones as network nodes, mapping the taxi trajectory data to different traffic zones through spatial connection, so that the taxi trajectory data is converted into trajectory data of flow between traffic zones, taking the time between the time of first entering the traffic zone and the time of first entering its adjacent traffic zone in the trajectory data as the travel time between the traffic zones, constructing a double-layer network consisting of a travel time layer and a travel flow layer, and deleting the trajectory data whose starting point and end point are both located in the same traffic zone when constructing the travel flow layer network, each network node in the travel flow layer network represents a different traffic zone, and the edge between two network nodes represents the number of passengers between the corresponding traffic zones, and each network node in the constructed travel time layer network represents a different traffic zone, and the edge between two network nodes represents the average order time of taxis between the corresponding traffic zones; Step S3 also includes measuring the resilience of the double-layer network nodes, determining the key nodes in the double-layer network nodes, and taking the resilience measurement of the travel flow layer network nodes under heavy rain weather as an example, the resilience value of the travel flow layer network nodes under heavy rain weather is calculated by the node resilience measurement algorithm, and the specific calculation formula is as follows: , where Represents the network nodes of the travel flow layer under heavy rain weather The toughness value, Representation Node In heavy rain The number of passengers at a given time, Representation Node Under normal weather conditions at the same time as the corresponding rainstorm The number of passengers at the time, and the number of nodes The toughness loss is calculated and recorded as ,Use the same method as above to calculate the resilience value of the travel time layer network node under heavy rain weather. Based on the calculation results of the resilience value and resilience loss of each network node in the travel flow layer network and the travel time layer network under heavy rain weather, determine the key nodes of the travel flow layer network and the travel time layer network. Specifically, the smaller the network node resilience value, the more critical the network node. When the network node resilience value is the same, the network node with a larger resilience loss is more critical. The criticality of each network node is calculated by the network node key algorithm. The specific calculation formula is as follows: , where Representation Node The criticality of mobility resilience, It represents the maximum value of resilience loss of all network nodes in the travel flow layer under heavy rain weather. Representation Node The criticality of flow toughness, It represents the maximum value of resilience loss of all network nodes in the travel time layer under heavy rain weather. Represents the network nodes of the travel time layer under heavy rain weather The toughness loss, Represents the network nodes of the travel time layer under heavy rain weather Toughness value, Representation Node The criticality of represents the weight of the criticality of travel resilience, represents the weight of the criticality of flow toughness, and , based on the criticality calculation results of each network node, set the top 20 network nodes as critical nodes; Step S4 also includes setting up emergency resource facility points in the target urban area based on the determined key node information, and each emergency resource facility point adopts a cooperative coverage method to provide network nodes with emergency materials needed for heavy rain, and adopts a differentiated coverage method to cover each network node, specifically, multiple coverage methods are adopted for key nodes, and non-key nodes may be allowed to be not fully covered, thereby improving the resilience of the urban road traffic system under heavy rain weather.

[0017] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0018] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for urban traffic control based on reinforcement learning, Features The following steps are involved: S1: A reinforcement learning-based urban traffic control method is applied to urban traffic management in rainstorm weather. Taxi-related data is obtained through an urban traffic information management platform, and the obtained taxi-related data is processed, and the taxi driving trajectory is obtained based on the processed taxi-related data; S2: Set the performance evaluation indicators of the urban road traffic system, draw the performance evolution curve of the urban road traffic system under heavy rain weather, and build an urban road traffic system resilience evaluation framework under heavy rain weather, set up corresponding urban road traffic system resilience measurement indicators, and conduct urban road traffic system resilience evaluation under heavy rain weather; S3: Divide the city into regions, process the taxi driving trajectory data, and construct a two-layer network consisting of a travel time layer and a travel flow layer. By measuring the resilience of the two-layer network nodes, the key nodes in the two-layer network are determined; S4: Based on the key node information in the determined two-layer network nodes, emergency resource facility points are set up, and each network node is covered in a differentiated coverage manner, thereby improving the resilience of the urban road traffic system under heavy rain weather.

2. According to the urban traffic control method based on reinforcement learning in claim 1, Features: The step S1 also includes acquiring relevant data on the taxi operation status during heavy rain weather and the corresponding normal weather before the heavy rain through the urban traffic information management platform, and preprocessing the acquired GPS data on the taxi operation status during heavy rain weather and the corresponding normal weather before the heavy rain, including using Python software to read the GPS data on the taxi operation status during heavy rain weather and the corresponding normal weather before the heavy rain and perform data cleaning, and perform coordinate system conversion operations on the GPS data on the taxi operation status during heavy rain weather and the corresponding normal weather before the heavy rain, adopting the HMM matching algorithm to match the taxi GPS data with the urban road network information, using the Python third-party library GeoPandas to spatially connect the taxi GPS data with the spatial map data of the urban target area, extracting the taxi GPS data in the urban target area, and obtaining the taxi GPS data in the urban target area during heavy rain weather and the corresponding normal weather before the heavy rain.

3. The urban traffic control method based on reinforcement learning according to claim 1, Features: The step S1 also includes arranging the GPS data of the same taxi in a time series based on the GPS data of taxis in the target area of ​​the city to form a driving trajectory of the taxi, determining the passenger boarding point time and the corresponding spatial position based on the time point and longitude and latitude information when the vehicle operation status changes from an empty state to a passenger state in the taxi GPS data, determining the passenger alighting point time and the corresponding spatial position when the vehicle operation status changes from a passenger state to an empty state and the longitude and latitude information, determining the taxi order travel distance based on the spatial positions of the passenger boarding point and alighting point, determining the taxi order duration based on the time points of the passenger boarding point and alighting point, calculating the ratio of the taxi order travel distance to the taxi order duration to obtain the average travel speed of the order, and taking the number of taxi orders within one hour as the taxi passenger flow.

4. The urban traffic control method based on reinforcement learning according to claim 1, Features: The step S2 also includes setting performance evaluation indicators of the urban road traffic system, including the transportation quantity and transportation efficiency of the urban road traffic system, aggregating the performance evaluation indicators of the urban road traffic system with one hour as the minimum time unit, using the passenger flow data of taxis in the urban target area under heavy rain weather and normal weather before the corresponding heavy rain as the transportation quantity indicator of the urban road traffic system under the corresponding weather, using the average travel speed of all taxi orders in the urban target area under heavy rain weather and normal weather before the corresponding heavy rain as the transportation efficiency indicator of the urban road traffic system under the corresponding weather, and using the statsmodels module in the Python software to analyze the taxi passenger flow data in the urban target area under heavy rain weather and construct a SARIMA model.

5. The urban traffic control method based on reinforcement learning according to claim 1, Features: The step S2 also includes constructing an urban road traffic system resilience assessment framework under heavy rain weather, which is composed of set urban road traffic system resilience measurement indicators, including the amount of resilience loss of the urban road traffic system, the disturbance duration of the urban road traffic system, the damage duration of the urban road traffic system, the recovery duration of the urban road traffic system and the system resilience.

6. The urban traffic control method based on reinforcement learning according to claim 1, Features: The step S3 also includes spatially dividing the urban target area according to the traffic zone division method, taking the divided traffic zones as network nodes, and mapping the taxi trajectory data to different traffic zones through spatial connection, so that the taxi trajectory data is converted into trajectory data of flow between traffic zones, and the time between the first time of entering the traffic zone and the first time of entering its adjacent traffic zone in the trajectory data is taken as the travel time between the traffic zones, and a two-layer network consisting of a travel time layer and a travel flow layer is constructed, and when constructing the travel flow layer network, it is necessary to delete the trajectory data whose starting point and end point are both located in the same traffic zone, each network node in the travel flow layer network represents a different traffic zone, and the edge between two network nodes represents the number of passengers between the corresponding traffic zones, and each network node in the constructed travel time layer network represents a different traffic zone, and the edge between two network nodes represents the average order time of taxis between the corresponding traffic zones.

7. The urban traffic control method based on reinforcement learning according to claim 1, Features: The step S3 also includes measuring the resilience of the double-layer network nodes, determining the key nodes in the double-layer network nodes, taking the resilience measurement of the travel flow layer network nodes under heavy rain weather as an example, calculating the resilience value of the travel flow layer network nodes under heavy rain weather through the node resilience measurement algorithm, and determining the key nodes of the travel flow layer network and the travel time layer network based on the calculation results of the resilience value and resilience loss of each network node in the travel flow layer network and the travel time layer network under heavy rain weather. Specifically, the smaller the network node resilience value, the more critical the network node, and when the network node resilience values ​​are the same, the network node with a larger resilience loss is more critical. The criticality of each network node is calculated through the network node key algorithm, and based on the calculation results of the criticality of each network node, the top 20 network nodes in the calculation results are set as key nodes.

8. The urban traffic control method based on reinforcement learning according to claim 1, Features: The step S4 also includes setting up emergency resource facilities in the target urban area based on the determined key node information, and each emergency resource facility point adopts a cooperative coverage method to provide the network nodes with the emergency materials needed for heavy rain, and adopts a differentiated coverage method to cover each network node, specifically, multiple coverage methods are adopted for key nodes, and non-key nodes may be allowed to be not fully covered, thereby improving the resilience of the urban road traffic system under heavy rain weather.

Images