A traffic signal intelligent control method and system based on spatio-temporal contrast learning

Abstract

The application provides a traffic signal intelligent control method and system based on space-time contrast learning, and belongs to the field of intelligent transportation. The method comprises the following steps: acquiring multi-source traffic data in real time, constructing a traffic state space-time tensor and a dynamic traffic graph after preprocessing; generating an intersection representation by using a dynamic graph neural network; performing data enhancement on the traffic state space-time tensor to generate an enhanced tensor and an enhanced intersection representation; constructing positive samples based on the intersection representation and the enhanced intersection representation, and constructing negative samples by randomly extracting historical time representation in an experience replay pool; generating an optimized intersection representation through contrast learning, and inputting the optimized intersection representation into a hierarchical reinforcement learning framework; outputting a macroscopic phase selection instruction through an upper policy network, and outputting a microscopic green light duration adjustment amount through a lower policy network, so that a signal timing scheme is finally generated. The application can solve the problems of poor self-adaptability, strong data dependence, insufficient robustness and difficult multi-objective optimization of existing traffic signal control methods.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of intelligent transportation, and particularly relates to a traffic signal intelligent control method and system based on spatio-temporal contrast learning. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute the prior art.

[0003] Urban traffic congestion has become one of the key problems restricting the development of modern cities. As the core means of road network management, the intelligent level of traffic signal control directly affects the road traffic efficiency, traffic safety and energy consumption. Traditional traffic signal control methods mainly include timing control and induction control.

[0004] Timing control systems rely on historical traffic flow statistics to pre-set signal periods, green ratios and phase differences. Although this method is simple in structure and stable in operation, it cannot adapt to the dynamic changes of traffic flow, especially during morning and evening peak hours, sudden events or adverse weather conditions, its control effect is significantly reduced, and unnecessary delays and congestion are easily caused. Induction control systems use sensors such as magnetic coils, radars and video detectors to detect the presence or passage of vehicles near the stop line in real time, and adjust the green light time accordingly. This method has certain real-time response ability compared to timing control, but its sensing range is limited, usually only to the local information at the detector installation location, lacking the ability to predict the spatio-temporal correlation and propagation characteristics of upstream and downstream traffic flow. Therefore, induction control is prone to cause local optimization and poor global performance, making it difficult to achieve regional coordinated control.

[0005] In recent years, reinforcement learning (RL) methods have been introduced into the field of traffic signal control, aiming to learn the optimal control strategy through continuous interaction between the agent and the environment. Deep Q-network (DQN), proximal policy optimization (PPO) and other algorithms have shown potential in this field, capable of handling more complex state spaces and adapting to dynamic environments. However, existing RL methods still have significant limitations: first, their training process is heavily dependent on a large amount of high-quality interaction data, and obtaining sufficient and authentic data in real traffic environments is costly, resulting in low training efficiency and insufficient generalization ability; second, traffic systems have high uncertainty and nonlinearity, with unstable environment state transition probabilities, and the robustness of traditional RL algorithms is often poor when facing sensor noise, data missing or sudden traffic events. SUMMARY

[0006] To overcome the above deficiencies of the prior art, the application provides a traffic signal intelligent control method and system based on spatio-temporal contrast learning, which models the spatio-temporal dependence of traffic flow through a dynamic graph neural network, enhances the robustness of the representation by combining a contrast learning framework, and finally optimizes the signal control strategy by using hierarchical reinforcement learning, thereby significantly improving the data efficiency, environmental adaptability and multi-intersection collaborative control capability of the system.

[0007] To achieve the above object, one or more embodiments of the application provide the following technical solutions:

[0008] The application provides a traffic signal intelligent control method based on spatio-temporal contrast learning in a first aspect.

[0009] The application provides a traffic signal intelligent control method based on spatio-temporal contrast learning, comprising:

[0010] Real-time acquisition of multi-source traffic data and preprocessing, construction of traffic state spatio-temporal tensors and dynamic traffic graphs based on the preprocessed multi-source traffic data.

[0011] Inputting the traffic state spatio-temporal tensors and dynamic traffic graphs into a dynamic graph neural network for feature extraction to generate intersection representations.

[0012] Data augmentation of the traffic state spatio-temporal tensors through spatial masks and temporal perturbations to generate augmented tensors, re-inputting the augmented tensors into the dynamic graph neural network to obtain augmented intersection representations, constructing positive samples based on the intersection representations and the augmented intersection representations, constructing negative samples by randomly extracting historical time representations from an experience replay pool, introducing a loss function for contrast learning optimization to obtain optimized intersection representations.

[0013] Inputting the optimized intersection representations into a hierarchical reinforcement learning framework, jointly optimizing the contrast learning loss and the reinforcement learning reward, outputting macro phase selection instructions through an upper policy network and micro green light duration adjustment amounts through a lower policy network, and finally generating a signal timing scheme.

[0014] As a further technical solution, real-time acquisition of multi-source traffic data and preprocessing comprises:

[0015] Collecting road traffic data using intersection detectors and obtaining road network topology data using a road network database.

[0016] The preprocessing includes data cleaning, missing value filling and normalization processing of the real-time acquired multi-source traffic data.

[0017] As a further technical solution, construction of traffic state spatio-temporal tensors and dynamic traffic graphs based on the preprocessed multi-source traffic data comprises:

[0018] Based on the pretreated road traffic data, a traffic state space-time tensor is constructed in units of lanes according to the queue length and average speed in a continuous time step;

[0019] By using the pretreated road network topology data, an intersection and adjacent road segments are abstracted into a dynamic graph, as follows:

[0020]

[0021] wherein, is a node set; is a time-varying edge set, and the adjacent weight is determined by real-time traffic flow characteristics Dynamic calculation:

[0022]

[0023] wherein, is an adjacent weight; and the corresponding adjacent matrix is denoted as .

[0024] As a further technical solution, the traffic state space-time tensor and the dynamic traffic graph are input into a dynamic graph neural network for feature extraction to generate an intersection representation, including:

[0025] The traffic state space-time tensor is taken as the initial input feature matrix of the first layer of the dynamic graph neural network, carrying the initial space-time traffic features;

[0026] Based on the adjacent matrix of the dynamic traffic graph, the feature matrix output by the previous layer is subjected to spatial feature aggregation by a graph convolution operator to output the feature matrix of the current layer;

[0027] The feature matrix output by the last layer is subjected to a lane dimension mean aggregation operation to obtain the intersection representation.

[0028] As a further technical solution, the traffic state space-time tensor is subjected to data enhancement by spatial masking and time disturbance to generate an enhanced tensor; the enhanced tensor is input again into the dynamic graph neural network to obtain an enhanced intersection representation, including:

[0029] A spatial masking operation is performed on the traffic state space-time tensor to generate a first enhanced tensor; wherein, a part of target lanes is randomly selected from all lanes with a preset probability; all feature data of the selected target lanes in the entire time dimension are set to zero to obtain the first enhanced tensor;

[0030] A time disturbance operation is performed on the traffic state space-time tensor to generate a second enhanced tensor; wherein, independent and identically distributed Gaussian noise is added to the traffic state space-time tensor to simulate sensor measurement error and small random fluctuations of traffic flow, and the second enhanced tensor is obtained;

[0031] The first enhanced tensor and the second enhanced tensor are input into a dynamic graph neural network respectively to obtain a first enhanced intersection representation and a second enhanced intersection representation.

[0032] As a further technical solution, in the process of introducing a loss function for contrastive learning optimization, the contrastive learning loss function is defined as:

[0033]

[0034] Among them, is the contrastive learning loss function; is the expected value; is the temperature coefficient; is the traffic state space-time tensor; is the first enhanced intersection representation; is the second enhanced intersection representation; is the experience replay pool; is the representation of the historical moment; is the cosine similarity.

[0035] As a further technical solution, the macro phase selection instruction is output by the upper policy network, and the micro green light duration adjustment amount is output by the lower policy network, and finally the signal timing scheme is generated, including:

[0036] The upper policy network takes the optimized intersection representation as input and outputs the activation probability distribution of each phase; the upper policy network is composed of two fully connected neural networks, and the specific calculation is as follows:

[0037]

[0038] Among them, represents the probability of each phase being selected, and the final phase is determined by sampling or taking the maximum probability; is a learnable parameter matrix connecting the input layer and the hidden layer; is a learnable parameter matrix connecting the hidden layer and the output layer; is a hidden layer bias vector; is the bias term of the output layer;

[0039] After the upper layer selects the phase, the lower policy network uses a parameterized Gaussian policy to model continuous actions and outputs the mean and standard deviation of the green light duration under the current phase; the mean and standard deviation are used for normal distribution sampling to output the micro green light duration adjustment amount, and finally the signal timing scheme is generated.

[0040] The second aspect of the application provides a traffic signal intelligent control system based on space-time contrastive learning.

[0041] A traffic signal intelligent control system based on space-time contrastive learning comprises:

[0042] a data collection module configured to acquire multi-source traffic data in real time, pre-process the multi-source traffic data, and construct a traffic state spatio-temporal tensor and a dynamic traffic graph based on the pre-processed multi-source traffic data;

[0043] a feature extraction module configured to input the traffic state spatio-temporal tensor and the dynamic traffic graph into a dynamic graph neural network for feature extraction, and generate an intersection representation;

[0044] a contrast learning module configured to perform data augmentation on the traffic state spatio-temporal tensor through a spatial mask and a temporal disturbance to generate an augmented tensor, input the augmented tensor into the dynamic graph neural network again to obtain an augmented intersection representation, construct a positive sample based on the intersection representation and the augmented intersection representation, construct a negative sample by randomly extracting a historical time representation from an experience replay pool, and obtain an optimized intersection representation;

[0045] a signal timing generation module configured to input the optimized intersection representation into a hierarchical reinforcement learning framework, jointly optimize a contrast learning loss and a reinforcement learning reward, output a macroscopic phase selection instruction through an upper policy network, output a microscopic green light duration adjustment amount through a lower policy network, and finally generate a signal timing scheme.

[0046] The third aspect of the present application provides a computer readable storage medium having a program stored thereon, the program being executed by a processor to implement the steps of the traffic signal intelligent control method based on spatio-temporal contrast learning according to the first aspect of the present application.

[0047] The fourth aspect of the present application provides an electronic device comprising a memory, a processor, and a program stored on the memory and executable on the processor, wherein the processor executes the program to implement the steps of the traffic signal intelligent control method based on spatio-temporal contrast learning according to the first aspect of the present application.

[0048] The above one or more technical solutions have the following beneficial effects:

[0049] (1) The present application fuses spatio-temporal contrast learning and hierarchical reinforcement learning framework, in terms of system robustness, the anti-interference representation ability constructed by spatio-temporal contrast learning enables the control system to maintain stable decision performance in complex scenarios such as sensor abnormalities or sudden traffic events; in terms of data utilization efficiency, the innovative contrast learning mechanism greatly reduces the dependence on high-quality labeled data, enabling the system to quickly adapt to new road network environments; in terms of real-time response, the optimized hierarchical decision architecture ensures millisecond-level computing efficiency, providing a feasible basis for city-level road network collaborative control;

[0050] (2) This invention breaks through the limitations of traditional single-objective optimization and achieves comprehensive optimization of traffic efficiency, energy consumption and environmental protection through multi-level strategy collaboration, providing more comprehensive technical support for the actual deployment of intelligent transportation systems. These technological innovations not only solve the key bottlenecks of existing methods, but also provide new technical paths for the future development of intelligent transportation.

[0051] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0052] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0053] Figure 1 This is a flowchart of the method in the first embodiment.

[0054] Figure 2 This is a system structure diagram of the second embodiment. Detailed Implementation

[0055] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0056] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0057] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0058] Example 1

[0059] This embodiment discloses a traffic signal intelligent control method based on spatiotemporal contrast learning;

[0060] like Figure 1 As shown, a traffic signal intelligent control method based on spatiotemporal contrastive learning includes:

[0061] Step S1: Acquire multi-source traffic data in real time and preprocess it. Construct a traffic state spatiotemporal tensor and a dynamic traffic map based on the preprocessed multi-source traffic data.

[0062] Step S2: Input the traffic state spatiotemporal tensor and dynamic traffic map into a dynamic graph neural network for feature extraction to generate intersection representations;

[0063] Step S3, the traffic state space-time tensor is data enhanced by a space mask and a time disturbance to generate an enhanced tensor; the enhanced tensor is input again into a dynamic graph neural network to obtain an enhanced intersection representation;

[0064] Step S4, a positive sample is constructed based on the intersection representation and the enhanced intersection representation, a negative sample is constructed by randomly extracting a historical time representation in an experience replay pool; a loss function is introduced for contrastive learning optimization to obtain an optimized intersection representation;

[0065] Step S5, the optimized intersection representation is input into a hierarchical reinforcement learning framework to jointly optimize the contrastive learning loss and the reinforcement learning reward, a macro phase selection instruction is output by an upper policy network, and a micro green light duration adjustment amount is output by a lower policy network, and finally a signal timing scheme is generated.

[0066] Specifically, the following contents are further included:

[0067] Step S1, real-time multi-source traffic data is acquired and preprocessed, and a traffic state space-time tensor and a dynamic traffic graph are constructed based on the preprocessed multi-source traffic data.

[0068] In step S1, road flow data is collected by an intersection detector, and road network topology data is acquired by a road network database; the real-time multi-source traffic data is subjected to data cleaning, missing value filling and normalization processing, and data of different dimensions and different value ranges are uniformly converted to a specific range.

[0069] Further, based on the preprocessed road flow data, a traffic state space-time tensor is constructed in units of lanes according to the queue length and average speed in a continuous time step; specifically, there are L lanes upstream of an intersection, and the detector uploads data once at a fixed sampling period . The queue length q and average speed v of the last T periods (total time ) are taken as original features to construct a tensor, as follows:

[0070]

[0071] wherein, is a traffic feature tensor; is the value of the traffic feature tensor at time t of a traffic node; the traffic state space-time tensor is obtained by standardizing .

[0072] The pre-processed road network topology data is used to take each vehicle intersection as a node in a graph, the physical connection relationship between vehicle intersections is taken as an edge, an adjacency structure is constructed, and then the space-time feature tensor generated from the queue length and vehicle speed is combined as a node attribute, and finally a traffic state dynamic graph evolving over time is formed, as follows:

[0073]

[0074] wherein, is a node set; is a time-varying edge set. The adjacency weight of the traffic dynamic graph is generated from real-time traffic flow features and is dynamically calculated:

[0075]

[0076] wherein, is an adjacency weight; the corresponding adjacency matrix is denoted as .

[0077] In step S2, the traffic state space-time tensor and the dynamic traffic graph are input into a dynamic graph neural network for feature extraction, and an intersection representation is generated.

[0078] The traffic state space-time tensor is taken as an initial input feature matrix of the first layer of the dynamic graph neural network, which carries initial space-time traffic features, as follows:

[0079]

[0080] wherein, is an input initial feature tensor;

[0081] For the kth layer (k ), based on the adjacency matrix of the dynamic traffic graph , the feature matrix output by the previous layer is subjected to spatial feature aggregation through a graph convolution operator, while a learnable parameter and a bias term are introduced, to calculate an intermediate feature matrix of the layer, which is then subjected to a nonlinear transformation through an activation function to output a feature matrix of the layer , realizing layer-by-layer fusion of traffic flow space-time features and road network spatial correlation features, as follows:

[0082]

[0083] wherein, is a space-time feature representation of the kth layer space-time encoder; is a bias term of the kth layer space-time encoder; is a nonlinear activation function; is a dimension number of the output feature of the kth layer. is the learnable parameter of the m-th convolution kernel of the k-th layer.

[0084] After the K-layer dynamic graph neural network operation is completed, the feature matrix output by the last layer is aggregated by the mean value of the lane dimension, that is, the average value of the feature vectors corresponding to all lanes in the dimension is taken to obtain a lane-level feature vector with a dimension of, which is the lane representation The space-time dynamic characteristics of traffic flow and the spatial correlation characteristics of the road network are fused, which are used for subsequent contrastive learning optimization and hierarchical strategy optimization, as follows:

[0085]

[0086] wherein, is the mean value; is the space-time feature representation of the space-time encoder from the 1st layer to the Kth layer.

[0087] In step S3, the space-time tensor of the traffic state is enhanced by a spatial mask and a temporal disturbance to generate an enhanced tensor; the enhanced tensor is input into the dynamic graph neural network again to obtain an enhanced intersection representation; positive samples are constructed based on the intersection representation and the enhanced intersection representation, negative samples are constructed by randomly extracting historical time representation in the experience replay pool; a loss function is introduced for contrastive learning optimization to narrow the representation distance of the positive samples and push away the representation distance of the negative samples.

[0088] The space-time tensor of the traffic state is enhanced by a spatial mask, which includes: a part of the lanes is randomly selected with a probability , and all features of the lane in the entire time dimension T are set to zero to simulate a local lane detector failure or occlusion scenario, to obtain a first enhanced tensor .

[0089] The space-time tensor of the traffic state is enhanced by a temporal disturbance, which includes: independent and identically distributed Gaussian noise is added to the space-time tensor of the traffic state , that is, , to simulate sensor measurement error and small random fluctuations of traffic flow, to obtain a second enhanced tensor .

[0090] Further, the first enhanced intersection representation and the second enhanced intersection representation corresponding to the current time are extracted by inputting the and ​​​​Construct the positive and negative sample sets required for contrastive learning. Positive sample pairs are semantically similar representations within the same traffic context, including... and Negative samples from the empirical replay pool Randomly selected Representation of a historical moment A negative sample set is created to ensure it is temporally uncorrelated with the current traffic state. The final comparison set contains... One sample (1 positive sample + B - 1 negative sample).

[0091] In addition, a loss function is introduced for contrastive learning optimization, and the contrastive learning loss function is defined as follows:

[0092]

[0093] in, To learn the loss function by comparison; To calculate the expectation; Temperature coefficient; This is the first enhanced intersection representation; This is a secondary enhanced intersection feature; For experience replay pool; As a representation of a historical moment; The cosine similarity is used. This loss function, by bringing positive sample pairs closer together and pushing negative samples further apart, enables the model to learn high-quality traffic state representations that are invariant to spatial occlusion and temporal noise.

[0094] The constructed positive and negative samples are used The loss function is used for comparative learning. Through the above comparative learning process, the model can obtain an optimized intersection representation during the training phase. This representation can maintain stability under conditions of missing local lane information and noise disturbances, and can be used not only for comparing learning loss. The optimization of these features also serves as input characteristics for downstream tasks such as traffic forecasting and signal control.

[0095] Step S4: Generate the optimized intersection representation The input is fed into a hierarchical reinforcement learning framework, where the contrastive learning loss and reinforcement learning reward are jointly optimized. The upper-layer policy network outputs macroscopic phase selection instructions, while the lower-layer policy network outputs microscopic green light duration adjustment amounts, ultimately generating a signal timing scheme.

[0096] Step S41, define the set of optional signal phases. ={EW left, EW right, NS left, NS right} There are 4 phases in total, covering the main traffic flow directions in the east-west and north-south directions.

[0097] upper policy network with optimized intersection representation The activation probability distribution of each phase is output as input. The upper policy network is composed of two fully connected neural networks, and the specific calculation is as follows:

[0098]

[0099] wherein, is the learnable parameter matrix connecting the input layer and the hidden layer; is the learnable parameter matrix connecting the hidden layer and the output layer; is the hidden layer bias vector; is the bias term of the output layer; represents the probability of each phase being selected, and the final phase is determined by sampling or taking the maximum probability. Further, the optimal traffic direction can be dynamically selected according to the real-time traffic flow distribution (such as east-west congestion and north-south free), and macro traffic flow guidance can be realized.

[0100] Step S42, selecting the phase at the upper level After that, the lower policy network finely adjusts the green light duration to adapt to the continuous changes in current traffic demand. A parameterized Gaussian policy is used to model continuous actions:

[0101]

[0102] wherein, is a multi-layer perceptron, and its parameters are independent of the upper policy network; the outputs and respectively represent the mean and standard deviation of the green light duration at the phase

[0103] The green light duration is sampled from the normal distribution:

[0104]

[0105] and the sampled green light duration is clipped to the physically feasible range to ensure that the signal timing meets the actual device constraints. The output is the micro green light duration adjustment, and the final signal timing scheme is generated.

[0106] Step S43, defining the immediate reward function , which is used to evaluate the improvement effect of the current control action on the traffic state:

[0107] ​

[0108] where, is the average travel delay (unit: second) of all vehicles in the current period; is the average queue length of the upstream critical section; is the number of vehicle start-stop times (reflecting driving smoothness); , , is the weight coefficient. The negative sign indicates that the goal is to minimize traffic cost, and the higher the reward (the less negative, the better the control effect.

[0109] Step S44, the robust representation extracted by self-supervised contrastive learning and the decision-making ability of reinforcement learning are used at the same time to define the total loss function as the weighted sum of policy loss and contrastive loss:

[0110]

[0111] where , TrafficCL contrastive loss is defined, which is used for dynamic graph neural network encoder to learn invariance representation. is the reinforcement learning policy loss, which uses the PPO-Clip algorithm, and the expression is:

[0112]

[0113] where, is the probability ratio of the new and old policies; is the generalized advantage estimation, which combines multi-step return and value function estimation; is the clipping threshold to prevent the policy update from being too large and causing unstable training. The joint loss function realizes the end-to-end collaborative training of representation learning and decision optimization, improving the generalization ability and convergence stability of the policy.

[0114] The signal control decision is decomposed into two sub-tasks of upper-level macro phase selection and lower-level green light duration fine-tuning to improve decision efficiency and control precision. The optimized intersection representation is taken as input, and the final signal timing scheme is output.

[0115] Embodiment Two

[0116] The embodiment discloses a traffic signal intelligent control system based on spatiotemporal contrastive learning;

[0117] As Figure 2 shown, a traffic signal intelligent control system based on spatiotemporal contrastive learning comprises:

[0118] The data acquisition module is configured to acquire multi-source traffic data in real time, pre-process the multi-source traffic data, and construct a traffic state space-time tensor and a dynamic traffic graph based on the pre-processed multi-source traffic data.

[0119] The feature extraction module is configured to input the traffic state space-time tensor and the dynamic traffic graph into a dynamic graph neural network for feature extraction, and generate an intersection representation.

[0120] The contrast learning module is configured to perform data augmentation on the traffic state space-time tensor through a spatial mask and a temporal disturbance to generate an augmented tensor, input the augmented tensor into the dynamic graph neural network again to obtain an augmented intersection representation, construct positive samples based on the intersection representation and the augmented intersection representation, construct negative samples by randomly extracting historical time representation in an experience replay pool, introduce a loss function for contrast learning optimization, and obtain an optimized intersection representation.

[0121] The signal timing generation module is configured to input the optimized intersection representation into a hierarchical reinforcement learning framework, jointly optimize a contrast learning loss and a reinforcement learning reward, output a macro phase selection instruction through an upper policy network, output a micro green light duration adjustment amount through a lower policy network, and finally generate a signal timing scheme.

[0122] Embodiment three

[0123] The purpose of this embodiment is to provide a computer-readable storage medium.

[0124] A computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the steps of a traffic signal intelligent control method based on space-time contrast learning according to embodiment 1.

[0125] Embodiment four

[0126] The purpose of this embodiment is to provide an electronic device.

[0127] An electronic device includes a memory, a processor, and a program stored on the memory and executable on the processor, and the processor executes the program to implement the steps of a traffic signal intelligent control method based on space-time contrast learning according to embodiment 1.

[0128] The steps and methods involved in the above embodiments two, three and four correspond to embodiment one, and the specific implementation can be referred to the relevant description part of embodiment one. The term "computer-readable storage medium" should be understood to include a single medium or multiple media of one or more instruction sets; it should also be understood to include any medium that can store, encode or carry instruction sets for execution by a processor and cause the processor to perform any method in the present application.

[0129] Those skilled in the art should understand that the modules or steps of the present application described above can be realized by general computer devices, or alternatively, they can be realized by program codes executable by the computer devices, so that they can be stored in the storage devices and executed by the computer devices, or they can be respectively manufactured into individual integrated circuit modules, or a plurality of modules or steps among them can be manufactured into a single integrated circuit module. The present application is not limited to any specific combination of hardware and software.

[0130] The specific embodiments of the present application described above in conjunction with the accompanying drawings are not intended to limit the protection scope of the present application. Those skilled in the art should understand that various modifications or changes made on the basis of the technical solutions of the present application without creative labor are still within the protection scope of the present application.

Claims

1. A traffic signal intelligent control method based on spatio-temporal contrastive learning, characterized in that, The application relates to a traffic signal timing method based on dynamic graph neural network and reinforcement learning, and belongs to the field of intelligent transportation. Real-time multi-source traffic data is acquired and preprocessed, and a traffic state space-time tensor and a dynamic traffic graph are constructed based on the preprocessed multi-source traffic data; The traffic state space-time tensor and the dynamic traffic graph are input into a dynamic graph neural network for feature extraction to generate intersection representation; The traffic state space-time tensor is subjected to data enhancement through spatial masking and time disturbance to generate an enhanced tensor; The enhanced tensor is input into the dynamic graph neural network again to obtain enhanced intersection representation; specifically, a spatial masking operation is performed on the traffic state space-time tensor to generate a first enhanced tensor; wherein a part of target lanes is randomly selected from all lanes with a preset probability; all feature data of the selected target lanes in the entire time dimension are set to zero to obtain the first enhanced tensor; a time disturbance operation is performed on the traffic state space-time tensor to generate a second enhanced tensor; wherein independent and identically distributed Gaussian noise is added to the traffic state space-time tensor to simulate sensor measurement error and slight random fluctuations of traffic flow, and the second enhanced tensor is obtained; the first enhanced tensor and the second enhanced tensor are input into the dynamic graph neural network to obtain first enhanced intersection representation and second enhanced intersection representation; Based on the intersection representation and the enhanced intersection representation, positive samples are constructed, and negative samples are constructed by randomly extracting historical time representation in an experience replay pool; a loss function is introduced for contrastive learning optimization to obtain optimized intersection representation; the contrastive learning loss function is defined as: wherein, is a contrastive learning loss function; is a computation expectation; is a temperature coefficient; is an intersection representation; is a first enhanced intersection representation; is a second enhanced intersection representation; is an experience replay pool; is a representation of a historical time instant; is a cosine similarity; The optimized intersection representation is input into a hierarchical reinforcement learning framework to jointly optimize the contrastive learning loss and the reinforcement learning reward, and a macro phase selection instruction is output by an upper policy network, and a micro green light duration adjustment amount is output by a lower policy network, so that a signal timing scheme is finally generated, wherein the upper policy network takes the optimized intersection representation as input and outputs the activation probability distribution of each phase; the upper policy network is composed of two layers of full connection neural networks, and the specific calculation is as follows: wherein, denotes the probability of each phase being selected, and the final phase is determined by sampling or taking the maximum probability; is a matrix of learnable parameters connecting the input layer and the hidden layer; is a matrix of learnable parameters connecting the hidden layer and the output layer; is a bias vector for the hidden layer; is a bias term for the output layer; After the upper layer selects a phase, the lower policy network uses a parameterized Gaussian policy to model continuous actions and outputs the mean and standard deviation of the green light duration under the current phase; the mean and standard deviation are used for normal distribution sampling to output the micro green light duration adjustment amount, and finally the signal timing scheme is generated.

2. The traffic signal intelligent control method based on spatio-temporal contrastive learning according to claim 1, characterized in that, Real-time multi-source traffic data is acquired and preprocessed, including: Road traffic data is collected by an intersection detector, and road network topology structure data is acquired by a road network database; The preprocessing includes data cleaning, missing value filling and normalization processing on the real-time acquired multi-source traffic data.

3. The traffic signal intelligent control method based on spatio-temporal contrastive learning of claim 1, wherein, Based on the preprocessed multi-source traffic data, a traffic state space-time tensor and a dynamic traffic graph are constructed, including: Based on the preprocessed road traffic data, a traffic state space-time tensor is constructed in units of lanes according to the queue length and average speed in a continuous time step; The intersection and adjacent road segments are abstracted into a dynamic graph by using the preprocessed road network topology structure data, as follows: wherein, is a set of nodes; is a time-varying edge set, whose adjacency weights are derived from real-time traffic flow characteristics Dynamic computation: wherein is the adjacency weight; the corresponding adjacency matrix is denoted by .

4. The traffic signal intelligent control method based on spatio-temporal contrastive learning of claim 1, wherein, The traffic state space-time tensor and the dynamic traffic graph are input into a dynamic graph neural network for feature extraction to generate intersection representation, including: The traffic state space-time tensor is taken as an initial input feature matrix of a first layer of the dynamic graph neural network, and carries initial space-time traffic features; Based on an adjacency matrix of the dynamic traffic graph, a graph convolution operator is used to aggregate spatial features of the feature matrix output by the previous layer, and a feature matrix of the current layer is output; A mean aggregation operation is performed on the feature matrix output by the last layer in the lane dimension, and an intersection representation is obtained.

5. A traffic signal intelligent control system based on spatio-temporal contrastive learning, characterized in that, The method comprises the following steps: A data acquisition module is configured to acquire multi-source traffic data in real time, pre-process the multi-source traffic data, and construct a traffic state space-time tensor and a dynamic traffic graph based on the pre-processed multi-source traffic data; A feature extraction module is configured to input the traffic state space-time tensor and the dynamic traffic graph into a dynamic graph neural network for feature extraction, and generate an intersection representation; A contrastive learning module is configured to perform data augmentation on the traffic state space-time tensor by using a spatial mask and a temporal disturbance, and generate an augmented tensor; The augmented tensor is input into the dynamic graph neural network again to obtain an augmented intersection representation; specifically, a spatial mask operation is performed on the traffic state space-time tensor to generate a first augmented tensor; wherein a part of target lanes is randomly selected from all lanes with a preset probability; all feature data of the selected target lanes in the entire time dimension are set to zero to obtain the first augmented tensor; a temporal disturbance operation is performed on the traffic state space-time tensor to generate a second augmented tensor; wherein independent and identically distributed Gaussian noise is added to the traffic state space-time tensor to simulate sensor measurement error and small random fluctuations of traffic flow, and the second augmented tensor is obtained; the first augmented tensor and the second augmented tensor are input into the dynamic graph neural network to obtain a first augmented intersection representation and a second augmented intersection representation; Based on the intersection representation and the augmented intersection representation, a positive sample is constructed, a negative sample is constructed by randomly extracting a historical time representation from an experience replay pool, a loss function is introduced for contrastive learning optimization, and an optimized intersection representation is obtained; the contrastive learning loss function is defined as: wherein, is a contrastive learning loss function; is a computation expectation; is a temperature coefficient; is a traffic state spatio-temporal tensor; is a first enhanced intersection representation; is a second enhanced intersection representation; is an experience replay pool; is a representation of a historical time instant; is a cosine similarity; A signal timing generation module is configured to input the optimized intersection representation into a hierarchical reinforcement learning framework, jointly optimize the contrastive learning loss and the reinforcement learning reward, output a macro phase selection instruction through an upper policy network, and output a micro green light duration adjustment amount through a lower policy network, and finally generate a signal timing scheme, wherein the upper policy network takes the optimized intersection representation as input and outputs an activation probability distribution of each phase; the upper policy network is composed of two fully connected neural networks, and the specific calculation is as follows: wherein, denotes the probability of each phase being selected, and the final phase is determined by sampling or taking the maximum probability; is a matrix of learnable parameters connecting the input layer and the hidden layer; is a matrix of learnable parameters connecting the hidden layer and the output layer; is a bias vector for the hidden layer; is a bias term for the output layer; After the upper layer selects a phase, the lower policy network uses a parameterized Gaussian policy to model continuous actions and outputs the mean and standard deviation of the green light duration under the current phase; the mean and standard deviation are used for normal distribution sampling to output a micro green light duration adjustment amount, and finally a signal timing scheme is generated.

6. A computer-readable storage medium having stored thereon a program, characterized in that, The program is executed by the processor to implement the steps of the traffic signal intelligent control method based on space-time contrastive learning in any one of claims 1-4.

7. An electronic device comprising a memory, a processor, and a program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the steps of the traffic signal intelligent control method based on space-time contrastive learning in any one of claims 1-4.

Images