Intelligent traffic signal optimization method based on key intersection recognition

Abstract

The invention relates to the technical field of data processing, and provides a traffic signal intelligent optimization method based on key intersection identification, and the method comprises the steps: obtaining key intersections in an urban traffic intersection network; constructing a deep reinforcement learning agent corresponding to each key intersection; acquiring traffic state information of each key intersection to obtain a traffic state information set; and determining a traffic signal control scheme corresponding to each key intersection by adopting a deep reinforcement learning agent corresponding to the key intersection and the traffic state information set, so that the traffic signal control scheme can be generated according to the identified key intersection in combination with the deployed deep reinforcement learning agent. And the accuracy of generating the traffic signal control scheme is improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a method for intelligent optimization of traffic signals based on key intersection identification. Background Technology

[0002] Urban traffic congestion has become a core problem commonly faced by modern cities. As a highly coupled and complex network system, congestion at any local intersection in a traffic network can rapidly spread to surrounding areas through chain reaction, leading to a significant decrease in overall traffic efficiency. Therefore, implementing precise and efficient signal control for traffic networks is key to improving urban operational efficiency.

[0003] However, existing methods for identifying node importance still have significant shortcomings: schemes relying on local topology cannot effectively distinguish between true core hubs and general intersections, while methods based on global network indicators suffer from high computational complexity, poor real-time performance, and often ignore the dynamic characteristics of traffic flow. Meanwhile, although deep reinforcement learning provides a new approach to adaptive signal control, mainstream methods typically deploy agents independently at each intersection in the traffic network. In large-scale networks, this "multi-agent network" model leads to an exponential expansion of the state-action space, causing problems such as difficulty in training convergence, high communication load, and high computational resource consumption, severely limiting its scalability in real-world urban scenarios. Summary of the Invention

[0004] This application provides a traffic signal intelligent optimization method based on key intersection identification, which can generate traffic signal control schemes by combining the identified key intersections with the deployed deep reinforcement learning agent, thereby improving the accuracy of traffic signal control scheme generation.

[0005] A first aspect of this application provides a traffic signal intelligent optimization method based on key intersection identification, the method comprising: Identify key intersections in the urban traffic intersection network; Construct a deep reinforcement learning agent corresponding to each key intersection; Obtain traffic status information for each key intersection to obtain a set of traffic status information; By employing a deep reinforcement learning agent corresponding to the key intersection and the set of traffic state information, a traffic signal control scheme is determined for each key intersection.

[0006] In one possible implementation, obtaining key intersections in the urban traffic intersection network includes: Obtain traffic data information from intersections in the urban traffic intersection network; The initial weight values ​​of roads in the urban traffic intersection network are determined based on the traffic data information. The structural hole constraint coefficient for each intersection is determined based on the initial weight of the road. The local importance parameter value corresponding to each intersection is determined based on the structural hole constraint coefficient of each intersection; Obtain the global importance parameter value for each intersection; Based on the local importance parameter value and the corresponding global importance parameter value of each intersection, determine the key intersection identification parameter value for each intersection; The key intersection identification parameter values ​​corresponding to each intersection determine the key intersections in the urban traffic intersection network.

[0007] In one possible implementation, determining the initial weight values ​​of roads in the urban traffic intersection network based on the traffic data information includes: Obtain the free-flow travel time of the road, and obtain the traffic capacity values ​​between the intersections corresponding to the road; Traffic saturation is determined based on the actual traffic flow and capacity of the road. The road blockage travel time is determined based on the traffic saturation, free-flow travel time, congestion coefficient, and congestion index. The utility weight value is obtained by calculating using the attenuation model and the travel time due to road resistance. The initial weight values ​​of roads in the urban traffic intersection network are determined based on the utility weights.

[0008] In one possible implementation, determining the key intersection identification parameter value for each intersection based on its local importance parameter value and its corresponding global importance parameter value includes: The local importance parameter value and the corresponding global importance parameter value of each intersection are normalized to obtain the normalized local importance parameter value and the corresponding normalized global importance parameter value of each intersection. Obtain the balance factor; Identification parameters are calculated based on the balance factor, the normalized local importance parameter value of each intersection, and the corresponding normalized global importance parameter value to obtain the key intersection identification parameter value for each intersection.

[0009] In one possible implementation, the method further includes: Execute the traffic signal control scheme for each key intersection and obtain the execution result information for each key intersection; Construct an empirical tuple for each key intersection based on the execution result information of each key intersection; The deep reinforcement learning agent is updated based on the empirical tuples at each key intersection, resulting in the updated deep reinforcement learning agent for each key intersection.

[0010] A second aspect of this application provides a traffic signal intelligent optimization device based on key intersection identification, the device comprising: The first acquisition unit is used to acquire key intersections in the urban traffic intersection network; Building units are used to construct deep reinforcement learning agents corresponding to each key intersection; The second acquisition unit is used to acquire traffic status information for each key intersection and obtain a set of traffic status information. The determining unit is used to determine the traffic signal control scheme for each key intersection by employing a deep reinforcement learning agent corresponding to the key intersection and the set of traffic state information.

[0011] In one possible implementation, the first acquisition unit is used for: Obtain traffic data information from intersections in the urban traffic intersection network; The initial weight values ​​of roads in the urban traffic intersection network are determined based on the traffic data information. The structural hole constraint coefficient for each intersection is determined based on the initial weight of the road. The local importance parameter value corresponding to each intersection is determined based on the structural hole constraint coefficient of each intersection; Obtain the global importance parameter value for each intersection; Based on the local importance parameter value and the corresponding global importance parameter value of each intersection, determine the key intersection identification parameter value for each intersection; The key intersection identification parameter values ​​corresponding to each intersection determine the key intersections in the urban traffic intersection network.

[0012] In one possible implementation, regarding the determination of the initial weight values ​​of roads in the urban traffic intersection network based on the traffic data information, the first acquisition unit is configured to: Obtain the free-flow travel time of the road, and obtain the traffic capacity values ​​between the intersections corresponding to the road; Traffic saturation is determined based on the actual traffic flow and capacity of the road. The road blockage travel time is determined based on the traffic saturation, free-flow travel time, congestion coefficient, and congestion index. The utility weight value is obtained by calculating using the attenuation model and the travel time due to road resistance. The initial weight values ​​of roads in the urban traffic intersection network are determined based on the utility weights.

[0013] In one possible implementation, regarding the determination of the key intersection identification parameter value corresponding to each intersection based on the local importance parameter value and the corresponding global importance parameter value of each intersection, the first acquisition unit is used to: The local importance parameter value and the corresponding global importance parameter value of each intersection are normalized to obtain the normalized local importance parameter value and the corresponding normalized global importance parameter value of each intersection. Obtain the balance factor; Identification parameters are calculated based on the balance factor, the normalized local importance parameter value of each intersection, and the corresponding normalized global importance parameter value to obtain the key intersection identification parameter value for each intersection.

[0014] In one possible implementation, the device is also used for: Execute the traffic signal control scheme for each key intersection and obtain the execution result information for each key intersection; Construct an empirical tuple for each key intersection based on the execution result information of each key intersection; The deep reinforcement learning agent is updated based on the empirical tuples at each key intersection, resulting in the updated deep reinforcement learning agent for each key intersection.

[0015] A third aspect of this application provides a terminal including a processor, an input device, an output device, and a memory, wherein the processor, input device, output device, and memory are interconnected, wherein the memory is used to store a computer program, the computer program including program instructions, and the processor is configured to invoke the program instructions to execute the step instructions as described in the first aspect of this application.

[0016] A fourth aspect of this application provides a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program causes a computer to perform some or all of the steps described in the first aspect of this application.

[0017] A fifth aspect of this application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in the first aspect of this application. The computer program product may be a software installation package.

[0018] Implementing the embodiments of this application has at least the following beneficial effects: By acquiring key intersections in the urban traffic intersection network, a deep reinforcement learning agent corresponding to each key intersection is constructed. Traffic state information of each key intersection is obtained, resulting in a traffic state information set. Using the deep reinforcement learning agent corresponding to the key intersection and the traffic state information set, a traffic signal control scheme corresponding to each key intersection is determined. Therefore, traffic signal control schemes can be generated based on the identified key intersections and the deployed deep reinforcement learning agent, improving the accuracy of traffic signal control scheme generation. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This application provides a flowchart illustrating a traffic signal intelligent optimization method based on key intersection identification as an embodiment of the present application. Figure 2 This application provides a schematic diagram of a node neighborhood topology. Figure 3 This application provides a schematic diagram of a spatiotemporal attention neural network model structure for a deep reinforcement learning agent. Figure 4 This application provides a training block diagram for a deep reinforcement learning agent. Figure 5 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application; Figure 6 This application provides a schematic diagram of the structure of a traffic signal intelligent optimization device based on key intersection identification. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0023] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0024] Please see Figure 1 , Figure 1 This application provides a flowchart illustrating a traffic signal intelligent optimization method based on key intersection identification, as illustrated in this embodiment. Figure 1 As shown, the method includes: 101. Obtain key intersections in the urban traffic intersection network.

[0025] First, we can construct the node neighborhood topology of the urban traffic intersection network. Specifically, such as... Figure 2 As shown, Figure 2 A schematic diagram of the node neighborhood topology is shown. Figure 2 In, with nodes As the target node for current importance assessment; adjacent to the target The nodes constitute its first-order neighbor set. This represents an intersection that has a direct road connection to the target node; by extending the connection nodes through first-order neighbors, they form a second-order neighbor set. The first node represents an intersection separated from the target node by one intermediate road; the remaining nodes represent other intersection nodes. This topology intuitively reflects the typical grid-like characteristics of urban road networks.

[0026] Specifically, the urban traffic intersection network is modeled as a directed weighted graph structure (node ​​neighborhood topology). ,in It is the set of all signal-controlled intersection nodes in the road network. It is the set of road edges connecting adjacent intersections, and the direction of the edges strictly corresponds to the allowed driving direction of the actual traffic flow. Figure 2 Each directed edge (by node) Pointing to node Taking the connected edges as an example, it has three core elements: actual traffic flow. physical length of road and the number of lanes in one direction This establishes a topological foundation for subsequent calculations that reflects the physical spatial structure and dynamic traffic load.

[0027] A method for acquiring key intersections in an urban traffic intersection network includes: A1. Obtain traffic data information of intersections in the urban traffic intersection network; A2. Determine the initial weight values ​​of roads in the urban traffic intersection network based on the traffic data information; A3. Determine the structural hole constraint coefficient for each intersection based on the initial weight of the road; A4. Determine the local importance parameter value corresponding to each intersection based on the structural hole constraint coefficient of each intersection; A5. Obtain the global importance parameter value for each intersection; A6. Based on the local importance parameter value and the corresponding global importance parameter value of each intersection, determine the key intersection identification parameter value for each intersection; A7. The key intersection identification parameter values ​​corresponding to each intersection determine the key intersections in the urban traffic intersection network.

[0028] In the constructed node neighborhood topology, dynamic traffic flow, static road length and number of lanes data between intersection nodes can be collected, and the road resistance function value representing the road traffic impedance can be calculated. Then, the initial weight value can be calculated based on the road resistance function value.

[0029] A method for determining the initial weight values ​​of roads in an urban traffic intersection network based on the traffic data information, when determining the initial weight values, includes: B1. Obtain the free-flow travel time of the road and the capacity values ​​between the intersections corresponding to the road; B2. Determine the traffic saturation level based on the actual traffic flow and capacity of the road; B3. Determine the road blockage travel time based on the traffic saturation, free-flow travel time, congestion coefficient, and congestion index; B4. The utility weight value is obtained by calculating using the attenuation model and road resistance travel time; B5. Determine the initial weight values ​​of roads in the urban traffic intersection network based on the utility weights.

[0030] Specifically, firstly, based on road length and design free flow velocity Calculate the free-flow travel time under undisturbed conditions. and in combination with the number of lanes Ideal single-lane saturation flow and vehicle correction coefficient Calculate the traffic capacity between intersection nodes ,in This is a vehicle correction factor used to correct for changes in traffic capacity caused by a mix of different vehicle types. and The road is selected based on its grade and relevant urban road design specifications. Furthermore, the saturation level is obtained by using the ratio of actual traffic flow to traffic capacity. Calculate the travel time of road resistance including nonlinear congestion effects. ,in and The parameters of the BPR function reflect the characteristics of road congestion. This represents the congestion coefficient, which determines the percentage increase in travel time relative to free-flow time when traffic volume reaches capacity. The congestion index determines the steepness of the road resistance function curve, i.e., the sensitivity of the congestion effect caused by increased traffic flow. Based on this, a decay model is introduced to calculate the utility weights. ,in This is the attenuation coefficient, reflecting the sensitivity of traffic flow to travel time during path selection. The final node is obtained by combining this coefficient with the traffic volume. For nodes Comprehensive dependency weight We use this as the initial weight of the directed edge (road).

[0031] When determining the constraint coefficients of structural holes, the specific steps are as follows: Combine the weights obtained in step A2 to reconstruct the network constraint coefficients in the structural hole theory. The network constraint coefficients include both direct and indirect inputs. Direct inputs: ; Quantified nodes Nodes during traffic flow dispersal The degree of direct dependence, among which Represents a node The set of first-order neighbors. Indirect input considers the characteristic of urban road networks typically being grid-like, with adjacent intersections lacking first-order common neighbors; therefore, indirect constraints arising from second-order neighbors are introduced: .

[0032] For intersection nodes The structural hole constraint coefficient is calculated as follows: , Used to quantify the degree of structural constraints that each intersection node experiences in its local network.

[0033] The calculation of the local importance parameter value can be specifically as follows: An extended Salton coefficient is introduced to correct the structural hole constraint coefficient in step A3. The Salton coefficient is calculated as follows: ,in , For nodes The extended neighbor set contains first-order and second-order neighbors.

[0034] Furthermore, the correction factor is obtained based on the Salton coefficient: , This is used to measure the impact of a node's higher-order neighbor structure information on its propagation ability. For nodes The size of the expanded neighbor set is then determined. Finally, combining the structural hole constraint coefficients obtained in step A3, the node size is calculated. The local importance evaluation index is: , The smaller the value, the fewer constraints the node is subject to and the higher its local importance.

[0035] When determining the value of the global importance parameter, it can be specifically as follows: The weighted K-Shell decomposition method is adopted. This involves defining segmented weighting degrees and using a weighted interpolation algorithm to calculate improved K-Shell values ​​for nodes. The weighted degree is defined as the inflow and outflow nodes. The sum of the weights of the edges: .

[0036] First, the weighted degree of each node is stratified according to a preset interval, transforming the continuous weighted data into a discrete hierarchical structure. Based on the stratification results, the improved K-Shell value is calculated using a weighted interpolation algorithm. , in For nodes i K-Shell value, For nodes The K-Shell value of the layer below the current layer. For nodes The highest weighted degree among nodes located in the same layer.

[0037] Finally, define the global coefficients. The smaller the value, the more important the node is located in the deeper core area of ​​the road network where traffic is denser and connections are tighter.

[0038] When determining the key intersection identification parameter values, they can specifically be: A method for determining the key intersection identification parameter value for each intersection based on the local importance parameter value and the corresponding global importance parameter value of each intersection, comprising: C1. Normalize the local importance parameter value and the corresponding global importance parameter value of each intersection to obtain the normalized local importance parameter value and the corresponding normalized global importance parameter value of each intersection. C2. Obtain the balance factor; C3. Based on the balance factor, the normalized local importance parameter value of each intersection, and the corresponding normalized global importance parameter value, the identification parameters are calculated to obtain the key intersection identification parameter value for each intersection.

[0039] Normalization can be expressed as: ; ; The total number of nodes at the intersection. To normalize the local importance parameter value, This is the normalized global importance parameter value.

[0040] Furthermore, a weighted linear model is constructed to calculate the hierarchical comprehensive importance coefficient (key intersection identification parameter value): , in This is a balancing factor used to measure the impact of local and global attributes. Finally, based on... Sort all intersections in the network in descending order by value, and select the intersection with the smallest value. The nodes constitute a set of key intersections. , as the target object for subsequent intelligent control.

[0041] 102. Construct a deep reinforcement learning agent corresponding to each key intersection.

[0042] Deploy a deep reinforcement learning agent. Define the core network structure of the deep reinforcement learning agent as a spatiotemporal deep neural network, which integrates spatial attention and temporal attention modules. The input to the spatial attention module is the traffic feature tensor of the target node and its neighbors. For example... Figure 3 As shown, Figure 3 A schematic diagram of the spatiotemporal attention neural network model structure for a deep reinforcement learning agent is shown. Figure 3 The network comprises two parts: a spatial attention module and a temporal attention module. The spatial attention module captures the spatial dependencies between the target intersection and its neighboring nodes, and achieves weighted aggregation of neighboring features by calculating the attention weights between nodes, thereby generating a feature representation that integrates spatial information. The temporal attention module performs multi-head self-attention calculation on the feature sequence output by the spatial module, captures the dynamic changes of traffic status in the time dimension, and generates a decision embedding vector that integrates spatiotemporal information.

[0043] This module utilizes trainable matrices Target node features and neighbor characteristics Mapping to a high-dimensional feature space yields .

[0044] By concatenating features and multiplying them by a trainable weight vector compute nodes and Attention scores between: , The attention coefficients are obtained after Softmax normalization: .

[0045] Subsequently, the neighbor features are weighted and aggregated, and the ELU activation function is used to output a feature vector that incorporates spatial neighborhood information. .

[0046] Finally, the original features and aggregated features are concatenated to form a time step. Module output at time: .

[0047] The temporal attention module receives the feature sequence output by the spatial module: ,in The length of the time series input to the model represents the number of historical time steps traced back.

[0048] Calculate queries separately using a multi-head self-attention mechanism ,key ,value Through calculation and The transposed dot product, scaled, and softmax normalized, generates the attention weight matrix for the time dimension: , Furthermore, on We obtain the time module output by weighting: Finally, the multiple outputs are aggregated and transformed linearly to output a decision embedding vector that deeply integrates spatiotemporal information. This vector captures the spatiotemporal dependence features of traffic states and forms the basis for the traffic signal agent's decision-making.

[0049] After deploying the deep reinforcement learning agent, it still needs to be trained, and the training flowchart is as follows. Figure 4 As shown, the details are as follows: D1: Agent training begins. The agent adopts the DQN (Deep Q Network) framework. First, the network parameters of each agent at key intersections are initialized, including the current Q network for action value evaluation and the target Q network for stabilizing the training process. Then, the experience replay cache of each agent is initialized, and the learning and training process of interacting with the environment is started.

[0050] D2: Traffic State Information Acquisition. At each agent's decision-making time step, real-time traffic state information of the key intersections it controls and their first-order neighboring intersections is acquired under multiple phases. Specific features acquired may include queue lengths, average vehicle speeds, cumulative waiting times, and current phase states for each approach lane. This multi-source heterogeneous data is reconstructed into a high-dimensional temporal tensor. As the state input of the agent's neural network, it provides comprehensive and continuous environmental perception information for the action selection of the policy network.

[0051] D3: Signal-controlled action decision-making. The agent uses its current state tensor... The input is fed into an initialized spatiotemporal deep neural network. The state tensor first aggregates neighborhood information through a spatial attention module, and then extracts time-series features through a temporal attention module. Finally, it is mapped to the Q-value vector of all possible discrete actions in the current state via the output layer. The agent adopts... Strategy for decision-making: When the generated random number is less than If necessary, randomly select an action to explore the environment; otherwise, select the action with the highest Q-value to take advantage of the current policy. The selected action... It is then interpreted into a specific signal control scheme, namely the next signal phase, and then sent out for execution.

[0052] D4: Experience sample storage. At critical intersections, the signal control scheme generated in D3 is executed, and the environment evolves to the next time step. The agent observes the new state. And receive instant rewards The reward function is designed to comprehensively consider both the node's own passage efficiency and the cooperative benefits of neighboring nodes. The state, action, reward, and next state data generated in this interaction are combined into an experience tuple. The data is stored in a dedicated experience replay cache for the agent. When the cache is full, new data will cyclically overwrite the old data, accumulating samples for subsequent model training.

[0053] D5: Policy Network Parameter Update and Model Convergence. The experience replay memory of each agent is periodically checked. When the amount of stored experience data reaches a preset training start threshold, the agent training process is initiated. This process first randomly samples batch data from the experience replay cache. Subsequently, parameterized data is used... The network calculates the current Q-value based on the current Q-value, and simultaneously modulates it according to the parameters. The target network calculates the target Q-value: , in To reinforce learning, a discount factor is used to measure the weight of future rewards on current decisions.

[0054] Policy network parameters Iterative updates are performed using gradient descent to minimize the mean squared error between the current Q-value and the target Q-value, and the policy network parameters are synchronized to the target network every fixed number of steps. To ensure training stability, a gradual decay strategy is employed to control the exploration rate. This allows the agent to explore fully in the early stages of training and gradually converge in the later stages. By repeating this process, the agent's reward curve gradually stabilizes, and the policy network converges to a stable optimal state.

[0055] 103. Obtain traffic status information for each key intersection to obtain a set of traffic status information.

[0056] Traffic status information includes queue length, average speed, cumulative waiting time, and current phase status for each approach lane at key intersections.

[0057] 104. Using a deep reinforcement learning agent corresponding to the key intersection and the set of traffic state information, determine the traffic signal control scheme corresponding to each key intersection.

[0058] The traffic state information set is input into a deep reinforcement learning agent for processing to obtain a traffic signal control scheme. Specifically, the traffic state information of key intersections and the traffic state information of their first-order neighboring intersections under multiple phases can be input into the deep reinforcement learning agent for processing to obtain a traffic signal control scheme.

[0059] For examples consistent with the above embodiments, please refer to... Figure 5 , Figure 5A schematic diagram of a terminal structure provided in an embodiment of this application is shown in the figure. It includes a processor, an input device, an output device, and a memory. The processor, input device, output device, and memory are interconnected. The memory is used to store a computer program, which includes program instructions. The processor is configured to call the program instructions. The program includes instructions for performing the following steps. Identify key intersections in the urban traffic intersection network; Construct a deep reinforcement learning agent corresponding to each key intersection; Obtain traffic status information for each key intersection to obtain a set of traffic status information; By employing a deep reinforcement learning agent corresponding to the key intersection and the set of traffic state information, a traffic signal control scheme is determined for each key intersection.

[0060] The above mainly describes the solutions of the embodiments of this application from the perspective of the method execution process. It is understood that, in order to achieve the above functions, the terminal includes the corresponding hardware structure and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments provided herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0061] This application embodiment can divide the terminal into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0062] For those consistent with the above, please refer to Figure 6 , Figure 6 This application provides a schematic diagram of the structure of a traffic signal intelligent optimization device based on key intersection identification, as an embodiment of the present application. Figure 6 As shown, the device includes: The first acquisition unit 601 is used to acquire key intersections in the urban traffic intersection network; Construction unit 602 is used to construct a deep reinforcement learning agent corresponding to each key intersection; The second acquisition unit 603 is used to acquire traffic status information of each key intersection and obtain a set of traffic status information. The determining unit 604 is used to determine the traffic signal control scheme corresponding to each key intersection by using a deep reinforcement learning agent corresponding to the key intersection and the set of traffic state information.

[0063] In one possible implementation, the first acquisition unit 601 is used for: Obtain traffic data information from intersections in the urban traffic intersection network; The initial weight values ​​of roads in the urban traffic intersection network are determined based on the traffic data information. The structural hole constraint coefficient for each intersection is determined based on the initial weight of the road. The local importance parameter value corresponding to each intersection is determined based on the structural hole constraint coefficient of each intersection; Obtain the global importance parameter value for each intersection; Based on the local importance parameter value and the corresponding global importance parameter value of each intersection, determine the key intersection identification parameter value for each intersection; The key intersection identification parameter values ​​corresponding to each intersection determine the key intersections in the urban traffic intersection network.

[0064] In one possible implementation, regarding the determination of the initial weight values ​​of roads in the urban traffic intersection network based on the traffic data information, the first acquisition unit 601 is used to: Obtain the free-flow travel time of the road, and obtain the traffic capacity values ​​between the intersections corresponding to the road; Traffic saturation is determined based on the actual traffic flow and capacity of the road. The road blockage travel time is determined based on the traffic saturation, free-flow travel time, congestion coefficient, and congestion index. The utility weight value is obtained by calculating using the attenuation model and the travel time due to road resistance. The initial weight values ​​of roads in the urban traffic intersection network are determined based on the utility weights.

[0065] In one possible implementation, regarding the determination of the key intersection identification parameter value corresponding to each intersection based on the local importance parameter value and the corresponding global importance parameter value of each intersection, the first acquisition unit 601 is used to: The local importance parameter value and the corresponding global importance parameter value of each intersection are normalized to obtain the normalized local importance parameter value and the corresponding normalized global importance parameter value of each intersection. Obtain the balance factor; Identification parameters are calculated based on the balance factor, the normalized local importance parameter value of each intersection, and the corresponding normalized global importance parameter value to obtain the key intersection identification parameter value for each intersection.

[0066] In one possible implementation, the device is also used for: Execute the traffic signal control scheme for each key intersection and obtain the execution result information for each key intersection; Construct an empirical tuple for each key intersection based on the execution result information of each key intersection; The deep reinforcement learning agent is updated based on the empirical tuples at each key intersection, resulting in the updated deep reinforcement learning agent for each key intersection.

[0067] This application also provides a computer storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the intelligent traffic signal optimization methods based on key intersection identification as described in the above method embodiments.

[0068] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program that causes a computer to perform some or all of the steps of any of the intelligent traffic signal optimization methods based on key intersection identification as described in the above method embodiments.

[0069] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0070] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0071] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0072] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0073] Furthermore, the functional units in the various embodiments of the application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software program module.

[0074] If the integrated unit is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0075] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage device, which may include: a flash drive, a read-only memory, a random access memory, a magnetic disk, or an optical disk, etc.

[0076] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A traffic signal intelligent optimization method based on key intersection identification, characterized in that, The method includes: Identify key intersections in the urban traffic intersection network; Construct a deep reinforcement learning agent corresponding to each key intersection; Obtain traffic status information for each key intersection to obtain a set of traffic status information; By employing a deep reinforcement learning agent corresponding to the key intersection and the set of traffic state information, a traffic signal control scheme is determined for each key intersection.

2. The intelligent traffic signal optimization method based on key intersection identification according to claim 1, characterized in that, The acquisition of key intersections in the urban traffic intersection network includes: Obtain traffic data information from intersections in the urban traffic intersection network; The initial weight values ​​of roads in the urban traffic intersection network are determined based on the traffic data information. The structural hole constraint coefficient for each intersection is determined based on the initial weight of the road. The local importance parameter value corresponding to each intersection is determined based on the structural hole constraint coefficient of each intersection; Obtain the global importance parameter value for each intersection; Based on the local importance parameter value and the corresponding global importance parameter value of each intersection, determine the key intersection identification parameter value for each intersection; The key intersection identification parameter values ​​corresponding to each intersection determine the key intersections in the urban traffic intersection network.

3. The intelligent traffic signal optimization method based on key intersection identification according to claim 2, characterized in that, Determining the initial weight values ​​of roads in the urban traffic intersection network based on the traffic data information includes: Obtain the free-flow travel time of the road, and obtain the traffic capacity values ​​between the intersections corresponding to the road; Traffic saturation is determined based on the actual traffic flow and capacity of the road. The road blockage travel time is determined based on the traffic saturation, free-flow travel time, congestion coefficient, and congestion index. The utility weight value is obtained by calculating using the attenuation model and the travel time due to road resistance. The initial weight values ​​of roads in the urban traffic intersection network are determined based on the utility weights.

4. The intelligent traffic signal optimization method based on key intersection identification according to any one of claims 1-3, characterized in that, The step of determining the key intersection identification parameter value for each intersection based on its local importance parameter value and corresponding global importance parameter value includes: The local importance parameter value and the corresponding global importance parameter value of each intersection are normalized to obtain the normalized local importance parameter value and the corresponding normalized global importance parameter value of each intersection. Obtain the balance factor; Identification parameters are calculated based on the balance factor, the normalized local importance parameter value of each intersection, and the corresponding normalized global importance parameter value to obtain the key intersection identification parameter value for each intersection.

5. The intelligent traffic signal optimization method based on key intersection identification according to claim 4, characterized in that, The method further includes: Execute the traffic signal control scheme for each key intersection and obtain the execution result information for each key intersection; Construct an empirical tuple for each key intersection based on the execution result information of each key intersection; The deep reinforcement learning agent is updated based on the empirical tuples at each key intersection, resulting in the updated deep reinforcement learning agent for each key intersection.

6. A traffic signal intelligent optimization device based on key intersection identification, characterized in that, The device includes: The first acquisition unit is used to acquire key intersections in the urban traffic intersection network; Building units are used to construct deep reinforcement learning agents corresponding to each key intersection; The second acquisition unit is used to acquire traffic status information for each key intersection and obtain a set of traffic status information. The determining unit is used to determine the traffic signal control scheme for each key intersection by employing a deep reinforcement learning agent corresponding to the key intersection and the set of traffic state information.

7. The intelligent traffic signal optimization device based on key intersection identification according to claim 6, characterized in that, The first acquisition unit is used for: Obtain traffic data information from intersections in the urban traffic intersection network; The initial weight values ​​of roads in the urban traffic intersection network are determined based on the traffic data information. The structural hole constraint coefficient for each intersection is determined based on the initial weight of the road. The local importance parameter value corresponding to each intersection is determined based on the structural hole constraint coefficient of each intersection; Obtain the global importance parameter value for each intersection; Based on the local importance parameter value and the corresponding global importance parameter value of each intersection, determine the key intersection identification parameter value for each intersection; The key intersection identification parameter values ​​corresponding to each intersection determine the key intersections in the urban traffic intersection network.

8. The intelligent traffic signal optimization device based on key intersection identification according to claim 7, characterized in that, In determining the initial weight values ​​of roads in the urban traffic intersection network based on the traffic data information, the first acquisition unit is used to: Obtain the free-flow travel time of the road, and obtain the traffic capacity values ​​between the intersections corresponding to the road; Traffic saturation is determined based on the actual traffic flow and capacity of the road. The road blockage travel time is determined based on the traffic saturation, free-flow travel time, congestion coefficient, and congestion index. The utility weight value is obtained by calculating using the attenuation model and the travel time due to road resistance. The initial weight values ​​of roads in the urban traffic intersection network are determined based on the utility weights.

9. A terminal, characterized in that, The system includes a processor, an input device, an output device, and a memory, which are interconnected. The memory is used to store a computer program, which includes program instructions. The processor is configured to invoke the program instructions to perform the method as described in any one of claims 1-5.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1-5.

Images