Road traffic signal control method and device

Abstract

The invention discloses a road traffic signal control method and device, and relates to the technical field of traffic management. Collecting image data in a target intersection, and dividing the image data according to a preset grid to construct a vehicle position matrix; combining the vehicle speed and the position matrix to obtain a vehicle speed matrix; substituting the vehicle position matrix and the vehicle speed matrix into a first preset model to obtain a first feature; acquiring a street lamp state during image data, splicing the street lamp state and the first feature, and substituting the spliced street lamp state and the first feature into a full-connection layer to obtain a second feature; and obtaining a traffic signal control scheme set according to a preset rule, and substituting the traffic signal control scheme set and the second feature into a preset network to obtain a target signal lamp control scheme. The spatial distribution and the dynamic speed of the vehicles in the intersection are obtained in real time, then the street lamp state and the traffic flow characteristics are spliced and then input into the preset network together with the traffic signal control scheme set to screen out the optimal signal lamp control scheme, and the road traffic control efficiency is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of traffic management, and in particular relates to a road traffic signal control method and device. Background Art

[0002] In modern urban traffic systems, road traffic signal control is a key link in ensuring road traffic order and improving traffic efficiency. Currently, the commonly used timed control traffic signal control method, however, only presets a fixed signal timing scheme based on historical traffic flow data, and is difficult to adapt to the dynamic changes of traffic flow in real time. When traffic flow fluctuates greatly, it is easy to cause vehicles in some lanes to wait for a long time, while other lanes are empty, resulting in a waste of road resources and reduced traffic efficiency.

[0003] Patent CN113763729A discloses a method for generating traffic control signals for urban roads. In the road grid controlled by the invention, each intersection is provided with two horizontal signal lights and two vertical signal lights arranged opposite to each other. These two horizontal signal lights and two vertical signal lights are controlled by the same traffic signal controller. In the road grid, each intersection is divided into Class A intersections and Class B intersections. It is defined that the four intersections adjacent to each Class A intersection are all Class B intersections; and the four intersections adjacent to each Class B intersection are all Class A intersections. Recommended speed signs are provided in both directions on each road section. The recommended speed signs can dynamically display the recommended driving speed. Through the coordination of Class A and Class B intersections, the present invention ensures that as long as vehicles follow the corresponding recommended driving speeds at different road sections, whether traveling horizontally or longitudinally, they will continue to encounter green lights, achieving the effect of passing through the green wave zone. However, when the traffic volume increases, this solution will make it difficult for the actual vehicle speed to reach the recommended value due to the following constraints between vehicles, and there is a lack of effective control over turning traffic at intersections, resulting in low road traffic control efficiency. Summary of the Invention

[0004] The purpose of the present invention is to solve the problem of low efficiency of road traffic control and to propose a road traffic signal control method and device.

[0005] In a first aspect of the present invention, a road traffic signal control method is first proposed, the method comprising: Collect image data within the target intersection in real time, and divide the image data into preset grids to construct a vehicle position matrix; Collecting the speed of each vehicle entering the intersection in the image data, and combining it with the vehicle position matrix to obtain a vehicle speed matrix; Substituting the vehicle position matrix and the vehicle speed matrix into a first preset model to obtain a first feature; The streetlight status when the image data is acquired is concatenated with the first feature and then fed into a fully connected layer to obtain a second feature; the streetlight status includes a signal light phase and duration; A traffic signal control solution set is obtained according to a preset rule, and the traffic signal control solution set and the second feature are substituted into a preset network to obtain a target traffic light control solution.

[0006] Optionally, substituting the vehicle position matrix and the vehicle speed matrix into a first preset model to obtain a first feature includes: Substituting the vehicle position matrix and the vehicle speed matrix into the first convolution layer to obtain a first convolution feature; Substituting the first convolutional feature into the attention enhancement module to obtain the first attention feature; Substitute the first attention feature into the second convolutional layer to obtain the second convolutional feature; Substitute the second convolution feature into the attention enhancement module to obtain the second attention feature, and flatten the second attention feature to obtain the first feature.

[0007] Optionally, the attention enhancement module includes a first attention enhancement module and a second attention enhancement module: The attention enhancement module obtains input features, substitutes the input features into the first attention enhancement module to perform average pooling operations and maximum pooling operations to obtain first average pooling features and first maximum pooling features; Substituting the first average pooling feature and the first maximum pooling feature into a multilayer perceptron and performing element-wise addition to obtain a first fused pooling feature; Substituting the input features into the second attention enhancement module to perform average pooling operations and maximum pooling operations to obtain second average pooling features and second maximum pooling features; Performing element-wise addition on the second average pooling feature and the second maximum pooling feature to obtain a second fused pooling feature; Convolving the second fused pooling feature with a 7×7 convolution kernel to obtain a third fused pooling feature; The input features, the first fusion pooling features and the third fusion pooling features are fused to obtain the output features of the attention enhancement module.

[0008] Optionally, the preset network includes two deep neural networks with the same structure, namely a main network and a target network; substituting the traffic signal control scheme in combination with the second feature into the preset network to obtain a target traffic light control scheme includes: updating the preset network according to the traffic signal control scheme and the second feature to obtain a traffic light control network; Substituting the traffic signal control scheme set and the second feature into the traffic light control network, and calculating the Q value corresponding to each traffic signal control scheme in the traffic signal control scheme set; The traffic signal control scheme with the highest Q value is obtained as the target traffic signal control scheme.

[0009] Optionally, updating the preset network according to the traffic signal control scheme in combination with the second feature to obtain a traffic light control network includes: Step 1: Initialize the network parameters of the main network and the target network to obtain the main network parameters θ and the target network parameters θ′; the main network parameters and the target network parameters are the same; Step 2: The second feature S t Substitute the action branch and advantage branch in the main network to obtain the corresponding solution Q value Q(S t ,A;θ); Step 3: Select execution plan A from the traffic signal control plan set through the greedy strategy t Execute and get the status feedback S after execution t+1 and reward R t , saving the second feature, the execution plan, the reward, and the state feedback as experience data into an experience replay pool; Step 4: Randomly sample data from the experience replay pool to obtain experience data; based on the current state feature S in the experience data j and signal control scheme A j , evaluate the Q value Q corresponding to the empirical data based on the action branch and advantage branch in the main network eval =Q(S j ,A j ;θ) get the main network Q value Q eval ; Step 5: Feedback the state S t+1 Substitute into the main network to obtain the control solution with the maximum Q value a′=argmax a′ Q(S t+1 ,a′;θ); the target network is calculated according to the target network parameter θ′ in the state feedback S t+1 Next, the Q value Q of action a′ is next =Q′(S t+1 ,a′;θ′); Step 6: Using the formula Q target =R t +γ⋅Q next Get the Q value Q of the target network target , where γ is the discount factor; Step 7: Calculate the Q value Q of the main networkeval The Q value Q of the target network target The mean square error loss between Loss=E[(Q target −Q eval ) 2 ] and use the Adam optimizer to update the parameters θ of the main network; Step 8: Synchronize the parameters θ of the main network to the target network θ′ at every preset number of steps; Step 9: Repeat steps 2 to 8 until the preset conditions are met to terminate the training and obtain the traffic light control network.

[0010] In a second aspect of the present invention, a road traffic signal control device is provided, comprising: A vehicle position matrix construction module is used to collect image data within the target intersection in real time and divide the image data into preset grids to construct a vehicle position matrix; a vehicle speed matrix construction module, configured to collect the speed of each vehicle entering the intersection in the image data and obtain a vehicle speed matrix in combination with the vehicle position matrix; A first feature determination module, configured to substitute the vehicle position matrix and the vehicle speed matrix into a first preset model to obtain a first feature; A second feature determination module is configured to obtain a streetlight status during the image data, and concatenate the streetlight status and the first feature and input the concatenated features into a fully connected layer to obtain a second feature; the streetlight status includes a signal light phase and duration; The target traffic light control scheme determination module is used to obtain a traffic signal control scheme set according to a preset rule, and substitute the traffic signal control scheme set and the second feature into a preset network to obtain a target traffic light control scheme.

[0011] Optionally, the first feature determination module includes: A first convolution feature generation module, configured to substitute the vehicle position matrix and the vehicle speed matrix into a first convolution layer to obtain a first convolution feature; A first attention feature generation module, configured to substitute the first convolution feature into an attention enhancement module to obtain a first attention feature; A second convolutional feature generation module, configured to substitute the first attention feature into a second convolutional layer to obtain a second convolutional feature; The first feature generation module is used to substitute the second convolution feature into the attention enhancement module to obtain the second attention feature, and flatten the second attention feature to obtain the first feature.

[0012] Optionally, the attention enhancement module includes a first attention enhancement module and a second attention enhancement module, and the device further includes: A first attention enhancement pooling module is used for the attention enhancement module to obtain input features, substitute the input features into the first attention enhancement module to perform average pooling operations and maximum pooling operations to obtain first average pooling features and first maximum pooling features; a first fused pooling feature determination module, configured to substitute the first average pooling feature and the first maximum pooling feature into a multilayer perceptron and then perform element-wise addition to obtain a first fused pooling feature; A second attention enhancement pooling module is used to substitute the input features into the second attention enhancement module to perform average pooling operations and maximum pooling operations to obtain second average pooling features and second maximum pooling features; A second fused pooling feature determination module, configured to perform element-wise addition of the second average pooling feature and the second maximum pooling feature to obtain a second fused pooling feature; A third fusion pooling feature determination module is used to convolve the second fusion pooling feature through a 7×7 convolution kernel to obtain a third fusion pooling feature; A feature fusion module is used to fuse the input features, the first fusion pooling features and the third fusion pooling features to obtain the output features of the attention enhancement module.

[0013] Optionally, the preset network includes two deep neural networks with the same structure, namely a main network and a target network; the target traffic light control scheme determination module includes: a preset network updating module, configured to update the preset network according to the traffic signal control scheme and the second feature to obtain a traffic light control network; a control scheme Q value determination module, configured to substitute the traffic signal control scheme set and the second feature into the traffic light control network, and calculate the Q value corresponding to each traffic signal control scheme in the traffic signal control scheme set; The target traffic light control scheme generation module is used to obtain the traffic signal control scheme with the highest Q value as the target traffic light control scheme.

[0014] Optionally, the execution steps of the preset network update module include: Step 1: Initialize the network parameters of the main network and the target network to obtain the main network parameters θ and the target network parameters θ′; the main network parameters and the target network parameters are the same; Step 2: The second feature S t Substitute the action branch and advantage branch in the main network to obtain the corresponding solution Q value Q(S t ,A;θ); Step 3: Select execution plan A from the traffic signal control plan set through the greedy strategy t Execute and get the status feedback S after execution t+1 and reward R t , saving the second feature, the execution plan, the reward, and the state feedback as experience data into an experience replay pool; Step 4: Randomly sample data from the experience replay pool to obtain experience data; based on the current state feature S in the experience data j and signal control scheme A j , evaluate the Q value Q corresponding to the empirical data based on the action branch and advantage branch in the main network eval =Q(S j ,A j ;θ) get the main network Q value Q eval ; Step 5: Feedback the state S t+1 Substitute into the main network to obtain the control solution with the maximum Q value a′=argmax a′ Q(S t+1 ,a′;θ); the target network is calculated according to the target network parameter θ′ in the state feedback S t+1 Next, the Q value Q of action a′ is next =Q′(S t+1 ,a′;θ′); Step 6: Using the formula Q target =R t +γ⋅Q next Get the Q value Q of the target network target , where γ is the discount factor; Step 7: Calculate the Q value Q of the main network eval The Q value Q of the target network target The mean square error loss between Loss=E[(Q target −Q eval ) 2 ] and use the Adam optimizer to update the parameters θ of the main network; Step 8: Synchronize the parameters θ of the main network to the target network θ′ at every preset number of steps; Step 9: Repeat steps 2 to 8 until the preset conditions are met to terminate the training and obtain the traffic light control network.

[0015] Beneficial effects of the present invention: The present invention proposes a road traffic signal control method. The method collects image data from a target intersection in real time, divides the image data into preset grids, and constructs a vehicle position matrix. The method collects the speed of each vehicle entering the intersection in the image data and combines it with the vehicle position matrix to obtain a vehicle speed matrix. The method substitutes the vehicle position matrix and the vehicle speed matrix into a first preset model to obtain a first feature. The method obtains the street light status at the time of the image data, concatenates the street light status and the first feature, and then inserts them into a fully connected layer to obtain a second feature. The method obtains a traffic signal control scheme set according to preset rules, and substitutes the traffic signal control scheme set and the second feature into a preset network to obtain a target signal light control scheme. Through image gridding processing, the spatial distribution and dynamic speed of vehicles in the intersection are obtained in real time, accurately reflecting the current traffic pressure. The method then extracts the spatiotemporal characteristics of vehicle position and speed to determine whether the current signal control effect leads to efficiency loss. The street light status and traffic flow characteristics are then concatenated and input into the preset network together with the traffic signal control scheme set to select the optimal signal light control scheme, thereby improving road traffic control efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The present invention will be further described below with reference to the accompanying drawings.

[0017] Figure 1 A flow chart of a road traffic signal control method provided by an embodiment of the present invention; Figure 2 A flowchart of updating a preset network provided by an embodiment of the present invention; Figure 3 A schematic structural diagram of a road traffic signal control device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0019] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of the present invention.

[0020] The embodiment of the present invention provides a road traffic signal control method. Figure 1 , Figure 1 A flow chart of a road traffic signal control method provided by an embodiment of the present invention. The method comprises the following steps: S101, collecting image data within the target intersection in real time, dividing the image data into preset grids to construct a vehicle position matrix; S102, collecting the speed of each vehicle entering the intersection in the image data, and combining it with the vehicle position matrix to obtain a vehicle speed matrix; S103, substituting the vehicle position matrix and the vehicle speed matrix into a first preset model to obtain a first feature; S104, obtaining the streetlight status during the image data, concatenating the streetlight status and the first feature, and then inputting the concatenated features into a fully connected layer to obtain a second feature; S105 , obtaining a traffic signal control solution set according to a preset rule, and substituting the traffic signal control solution set and the second feature into a preset network to obtain a target traffic light control solution.

[0021] The street light status includes the phase and duration of the signal light.

[0022] A road traffic signal control method provided by an embodiment of the present invention uses image gridding processing to obtain the spatial distribution and dynamic speed of vehicles at an intersection in real time, accurately reflecting the current traffic pressure. The method then extracts the spatiotemporal characteristics of vehicle position and speed to determine whether the current signal control effect leads to efficiency loss. The street light status and traffic flow characteristics are then spliced together and input into a preset network together with a set of traffic signal control schemes to screen out the optimal signal light control scheme, thereby improving road traffic control efficiency.

[0023] In one implementation, the vehicle position matrix is constructed by dividing the image data into preset grids, which can locate the vehicle position in the intersection. The specific operation of dividing the image data into preset grids to construct the vehicle position matrix is as follows: for the image data, the road and vehicle position in the image are identified by the target recognition model, and the lanes entering the intersection are divided into cells of the same size to represent vehicle information. The size of each cell (preset grid) is determined by the technician (usually 5.3 to 6 meters in length) to ensure that each cell can accommodate at most one vehicle. If a cell is occupied by a vehicle, the value of the cell is set to 1; otherwise, it is set to 0; thus, the intersection entry is obtained. The initial vehicle position matrix corresponding to the intersection is obtained, and the initial vehicle position matrices corresponding to all positions of the intersection (the four initial vehicle position matrices from west to east, from east to west, from north to south, and from south to north) are put into a set as the vehicle position matrix. Similarly, after determining the vehicle position matrix, the speed of each vehicle will be measured. The speed in each initial vehicle position matrix will be normalized through minimum and maximum normalization, and the normalized speed will be mapped to the initial vehicle position matrix to replace the original number 1, thereby obtaining the initial speed matrix. The four initial vehicle position matrices are put into a set as the vehicle speed matrix.

[0024] In one implementation, a traffic signal control scheme set is obtained according to preset rules, including formulating an action execution space according to the preset rules, which includes eight categories, namely, northbound open traffic, southbound open traffic, westbound open traffic, eastbound open traffic, going straight and turning right in the north-south direction, turning left in the north-south direction, going straight and turning right in the east-west direction, and turning left in the east-west direction; each category corresponds to a code; and the execution time range of each category is (5 to 80 seconds).

[0025] In one implementation, real-time image data from the target intersection is collected and divided into pre-set grids to construct a vehicle position matrix. This matrix format intuitively and clearly displays the distribution of vehicles in each area of the intersection, providing precise spatial information for subsequent traffic analysis. Vehicle speeds are collected for each direction of approach to the intersection and combined with the vehicle position matrix to generate a vehicle speed matrix. This speed matrix reflects the speed of vehicles at different locations, further refining traffic status information.

[0026] In one implementation, the streetlight status (including signal phase and duration) at the time the image data is acquired is concatenated with the first feature and then fed into a fully connected layer to generate the second feature. Streetlight status is a crucial factor in traffic signal control, as signal phase and duration directly impact vehicle traffic flow. Fusion of streetlight status with features derived from vehicle position and speed allows the second feature to contain richer traffic information and more comprehensively reflect traffic dynamics at the intersection.

[0027] In one implementation, a pre-defined network uses a deep learning algorithm to learn and train a large amount of traffic data and solutions. It can then select the optimal target traffic light control solution from a set of solutions based on the current traffic conditions (the second characteristic). This approach combines the reliability of pre-defined rules with the intelligence of deep learning, improving the scientific nature and accuracy of traffic signal control.

[0028] In one embodiment, substituting the vehicle position matrix and the vehicle velocity matrix into the first preset model to obtain the first feature includes: Substitute the vehicle position matrix and vehicle speed matrix into the first convolution layer to obtain the first convolution feature; Substitute the first convolution feature into the attention enhancement module to obtain the first attention feature; Substitute the first attention feature into the second convolutional layer to obtain the second convolutional feature; The second convolution feature is substituted into the attention enhancement module to obtain the second attention feature, and the second attention feature is flattened to obtain the first feature.

[0029] In one implementation, the convolution kernels of the first and second convolution layers are both 3x3. The convolution layer performs sliding calculations on the vehicle position matrix and the vehicle velocity matrix using the convolution kernel to capture feature information within the local area. The convolution layer uses a parameter sharing mechanism, where the same convolution kernel uses the same parameters when performing sliding calculations on the entire matrix. This not only reduces the number of model parameters and computational complexity, but also makes the model have a certain degree of translation invariance.

[0030] In one implementation, the attention enhancement module can automatically learn the importance weights of different features and focus more attention on key features closely related to traffic signal control, thereby more accurately grasping the core information of traffic status and improving the effectiveness and representativeness of features.

[0031] In one implementation, the second convolutional layer performs further convolution operations based on the first attention features, which can extract higher-level and more abstract features. The flattening operation not only realizes the fusion of features and integrates features of different dimensions and positions, but also plays a role in dimensionality reduction, reducing the dimension of the features and reducing the complexity of subsequent network processing.

[0032] In one embodiment, the attention enhancement module includes a first attention enhancement module and a second attention enhancement module: The attention enhancement module obtains input features, substitutes the input features into the first attention enhancement module to perform average pooling operations and maximum pooling operations to obtain the first average pooling features and the first maximum pooling features; Substitute the first average pooling feature and the first maximum pooling feature into the multilayer perceptron and add the elements to obtain the first fused pooling feature; Substitute the input features into the second attention enhancement module for average pooling and maximum pooling operations to obtain the second average pooling features and the second maximum pooling features; Perform element-wise addition on the second average pooling feature and the second maximum pooling feature to obtain a second fused pooling feature; The third fusion pooling feature is obtained by convolving the second fusion pooling feature with a 7×7 convolution kernel; The input features, the first fusion pooling features and the third fusion pooling features are fused to obtain the output features of the attention enhancement module.

[0033] In one implementation, average pooling can obtain the global average information of the input features, reflecting the overall distribution of the features; while maximum pooling can extract significant local maxima within the features, highlighting the key information in the features. By performing average pooling and maximum pooling operations simultaneously, the first average pooling feature and the first maximum pooling feature are obtained, followed by the second average pooling feature and the second maximum pooling feature. This can comprehensively capture the information of the input features from different angles, avoiding the information loss that may be caused by a single pooling method.

[0034] In one implementation, after the first average pooling feature and the first maximum pooling feature are substituted into the multilayer perceptron, the multilayer perceptron can perform a nonlinear transformation on the feature through its internal nonlinear activation function, thereby generating a more expressive feature.

[0035] In one implementation, the multilayer perceptron can adapt and adjust the dimensions of the input features so that different pooling features can be better fused in terms of dimensions; the first average pooling feature and the first maximum pooling feature processed by the multilayer perceptron are element-wise added to obtain the first fused pooling feature, which does not introduce excessive computational complexity and can improve the richness and expressiveness of the features.

[0036] In one implementation, the third fused pooled feature is obtained by convolving the second fused pooled feature with a 7×7 convolution kernel. The convolution operation can further extract local patterns and structural information from the second fused pooled feature. The 7×7 convolution kernel can capture these local patterns within a larger receptive field, providing more valuable feature information for subsequent feature fusion.

[0037] In one implementation, the input features, the first fused pooled features, and the third fused pooled features are fused to obtain an average.

[0038] In one embodiment, the preset network includes two deep neural networks with the same structure, namely a main network and a target network; the traffic signal control solution combined with the second feature is substituted into the preset network to obtain the target traffic light control solution, which includes: According to the traffic signal control scheme and the second feature, the preset network is updated to obtain a traffic light control network; Substitute the traffic signal control scheme combined with the second feature into the traffic light control network, and calculate the Q value corresponding to each traffic signal control scheme in the traffic signal control scheme set; The traffic signal control scheme with the highest Q value is obtained as the target traffic signal control scheme.

[0039] In one implementation, the pre-set network consists of two identical deep neural networks: a main network and a target network. The main network is used for real-time updates and learning to adapt to changing traffic conditions; the target network is relatively stable, with less frequent parameter updates. This dual-network structure effectively avoids training instability caused by frequent updates to the main network parameters. In traffic signal control, where traffic conditions are complex and ever-changing, this dual-network structure enables the model to maintain relatively stable performance in dynamic environments, improving decision-making reliability.

[0040] In one implementation, the traffic signal control problem is modeled as a reinforcement learning problem, where the optimal control strategy is learned through continuous interaction with the environment (i.e., traffic conditions). The primary network updates its parameters based on the current set of traffic signal control solutions and the secondary feature to optimize the decision-making strategy. The target network provides stable Q-value estimates to help the primary network learn better.

[0041] In one implementation, traffic signal control schemes, combined with the second characteristic, are substituted into the signal control network. The Q-value corresponding to each traffic signal control scheme is calculated. The Q-value is a quantitative metric used to assess the long-term cumulative reward of adopting a control scheme under given conditions. By calculating the Q-value, different traffic signal control schemes can be objectively evaluated and compared.

[0042] In one embodiment, see Figure 2 , Figure 2 A flow chart for updating a preset network is provided, which includes: S201: Initialize the network parameters of the main network and the target network to obtain the main network parameters θ and the target network parameters θ′; S202: The second feature S t Substitute the action branch and advantage branch in the main network to obtain the corresponding solution Q value Q(S t ,A;θ); S203: Selecting execution plan A from the traffic signal control plan set through a greedy strategy t Execute and get the status feedback S after execution t+1 and reward R t , save the second feature, execution plan, reward and state feedback as experience data into the experience replay pool; S204: Randomly sample data from the experience replay pool to obtain experience data; based on the current state feature S in the experience data j and signal control scheme A j , evaluate the Q value Q corresponding to the empirical data based on the action branch and advantage branch in the main network eval =Q(S j ,A j;θ) get the main network Q value Q eval ; S205: Feedback the status S t+1 Substitute into the main network to obtain the control solution with the maximum Q value a′=argmax a′ Q(S t+1 ,a′;θ); the target network is calculated based on the target network parameter θ′ in the state feedback S t+1 Next, the Q value Q of action a′ is next =Q′(S t+1 ,a′;θ′); S206: Through Formula Q target =R t +γ⋅Q next Get the Q value Q of the target network target ; S207: Calculate the Q value Q of the main network eval and the Q value Q of the target network target The mean square error loss between Loss=E[(Q target −Q eval ) 2 ] and use the Adam optimizer to update the parameters θ of the main network; S208: Synchronize the parameters θ of the main network to the target network θ′ at every preset number of steps; S209: Repeat steps S202 to S208 until a preset condition is met and the training is terminated to obtain a traffic light control network; Among them, the main network parameters and the target network parameters are the same; and γ is the discount factor.

[0043] In one implementation, the action branch outputs the Q-value for each possible action (i.e., traffic signal control solution); the advantage branch outputs the advantage of each action relative to other actions, measuring the relative value of the actions. Initializing the parameters of the main and target networks to the same value provides a consistent starting point for subsequent learning. This dual-network architecture (main and target) helps improve learning stability. The main network is responsible for real-time learning and updating to adapt to changing traffic conditions; the target network provides relatively stable Q-value estimates, avoiding training instability caused by frequent updates to the main network parameters, enabling more reliable model learning in dynamic traffic environments.

[0044] In one implementation, the main network is configured with an action branch and an advantage branch. The action branch is used to output the Q value under different traffic signal control schemes, and the advantage branch can further refine the evaluation of the advantages of different actions.

[0045] In one implementation, the state feedback, rewards, and other information after executing the traffic signal control scheme are saved as experience data in an experience replay pool, and data is randomly sampled from it for learning. This breaks the temporal correlation between the data, making the data used in each training more diverse and preventing the model from falling into a local optimal solution due to continuous reception of related data.

[0046] In one implementation, a greedy strategy is used to select execution plans. While ensuring a certain degree of exploratory potential, it also leverages the currently learned optimal strategy for decision-making, thus achieving a balance between exploration and exploitation. As the number of execution plans increases, a rich pool of data accumulates in the experience replay pool, providing ample information for model training and helping the model learn more comprehensive traffic patterns. The Adam optimizer is used to update the parameters θ of the main network to bring the Q value of the main network closer to that of the target network. The preset number of steps is determined by technical personnel (default is 20). The preset condition can be to increase the threshold by 0.5%. If the reduction in mean squared error loss does not exceed 0.5% in 10 consecutive iterations, training is terminated. The preset condition can also be a preset number of cycles, which is determined by technical personnel.

[0047] In one implementation, the main network and the target network are used to evaluate the Q value of the control scheme in different states. The main network evaluates the Q value of the current state (Q eval ), the target network evaluates the Q value after executing the optimal action in the next state (Q next ), this multi-stage Q-value evaluation method can more comprehensively consider the long-term impact of decisions.

[0048] In one implementation, γ is a discount factor, and its value is (0, 1). The specific value is determined by technical personnel; the closer γ is to 1, the more important the future reward is; the closer γ is to 0, the less important the future reward is.

[0049] Based on the same inventive concept, the present invention also provides a road traffic signal control device. Figure 3 , Figure 3 A schematic structural diagram of a road traffic signal control device provided in an embodiment of the present invention includes: The vehicle position matrix construction module is used to collect image data within the target intersection in real time and divide the image data into preset grids to construct the vehicle position matrix; The vehicle speed matrix construction module is used to collect the vehicle speed of each vehicle entering the intersection in the image data and combine it with the vehicle position matrix to obtain the vehicle speed matrix; A first feature determination module, configured to substitute the vehicle position matrix and the vehicle speed matrix into a first preset model to obtain a first feature; The second feature determination module is used to obtain the street light status during image data, and concatenate the street light status and the first feature and input them into the fully connected layer to obtain the second feature; the street light status includes the phase and duration of the signal light; The target traffic light control scheme determination module is used to obtain a traffic signal control scheme set according to a preset rule, and substitute the traffic signal control scheme set and the second feature into a preset network to obtain the target traffic light control scheme.

[0050] A road traffic signal control device provided by an embodiment of the present invention uses image gridding processing to obtain the spatial distribution and dynamic speed of vehicles in an intersection in real time, accurately reflecting the current traffic pressure. It then extracts the spatiotemporal characteristics of vehicle positions and speeds to determine whether the current signal control effect leads to efficiency loss. The street light status and traffic flow characteristics are then spliced together and input into a preset network together with a set of traffic signal control schemes to screen out the optimal signal light control scheme, thereby improving road traffic control efficiency.

[0051] In one embodiment, the first feature determination module includes: A first convolution feature generation module, configured to substitute the vehicle position matrix and the vehicle speed matrix into the first convolution layer to obtain a first convolution feature; A first attention feature generation module is used to substitute the first convolution feature into the attention enhancement module to obtain the first attention feature; A second convolutional feature generation module is used to substitute the first attention feature into the second convolutional layer to obtain a second convolutional feature; The first feature generation module is used to substitute the second convolution feature into the attention enhancement module to obtain the second attention feature, and flatten the second attention feature to obtain the first feature.

[0052] In one embodiment, the attention enhancement module includes a first attention enhancement module and a second attention enhancement module, and the apparatus further includes: The first attention enhancement pooling module is used for the attention enhancement module to obtain input features, substitute the input features into the first attention enhancement module for average pooling operations and maximum pooling operations to obtain the first average pooling features and the first maximum pooling features; A first fused pooling feature determination module is used to substitute the first average pooling feature and the first maximum pooling feature into a multilayer perceptron and then perform element-wise addition to obtain a first fused pooling feature; The second attention enhancement pooling module is used to substitute the input features into the second attention enhancement module for average pooling operations and maximum pooling operations to obtain second average pooling features and second maximum pooling features; A second fused pooling feature determination module, configured to perform element-wise addition of the second average pooling feature and the second maximum pooling feature to obtain a second fused pooling feature; A third fusion pooling feature determination module is used to convolve the second fusion pooling feature through a 7×7 convolution kernel to obtain a third fusion pooling feature; The feature fusion module is used to fuse the input features, the first fusion pooling features and the third fusion pooling features to obtain the output features of the attention enhancement module.

[0053] In one embodiment, the preset network includes two deep neural networks with the same structure, namely a main network and a target network; the target traffic light control solution determination module includes: A preset network updating module, configured to update the preset network according to the traffic signal control scheme and the second feature to obtain a signal light control network; a control scheme Q value determination module, configured to substitute the traffic signal control scheme in combination with the second feature into the signal light control network and calculate the Q value corresponding to each traffic signal control scheme in the traffic signal control scheme set; The target traffic light control scheme generation module is used to obtain the traffic signal control scheme with the highest Q value as the target traffic light control scheme.

[0054] In one embodiment, the execution steps of the preset network update module include: Step 1: Initialize the network parameters of the main network and the target network to obtain the main network parameters θ and the target network parameters θ′; the main network parameters and the target network parameters are the same; Step 2: The second feature S t Substitute the action branch and advantage branch in the main network to obtain the corresponding solution Q value Q(S t ,A;θ); Step 3: Select execution plan A from the traffic signal control plan set through the greedy strategy t Execute and get the status feedback S after execution t+1 and reward R t , save the second feature, execution plan, reward and state feedback as experience data into the experience replay pool; Step 4: Randomly sample data from the experience replay pool to obtain experience data; based on the current state feature S in the experience data j and signal control scheme A j , evaluate the Q value Q corresponding to the empirical data based on the action branch and advantage branch in the main network eval =Q(S j ,A j ;θ) get the main network Q value Q eval ; Step 5: Feedback the state S t+1 Substitute into the main network to obtain the control solution with the maximum Q value a′=argmax a′ Q(St+1 ,a′;θ); the target network is calculated based on the target network parameter θ′ in the state feedback S t+1 Next, the Q value Q of action a′ is next =Q′(S t+1 ,a′;θ′); Step 6: Using the formula Q target =R t +γ⋅Q next Get the Q value Q of the target network target , where γ is the discount factor; Step 7: Calculate the Q value Q of the main network eval and the Q value Q of the target network target The mean square error loss between Loss=E[(Q target −Q eval ) 2 ] and use the Adam optimizer to update the parameters θ of the main network; Step 8: Synchronize the parameters θ of the main network to the target network θ′ at every preset number of steps; Step 9: Repeat steps 2 to 8 until the preset conditions are met to terminate the training and obtain the traffic light control network.

[0055] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and should not be considered to limit the scope of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A road traffic signal control method, characterized in that: The method comprises: Collect image data within the target intersection in real time, and divide the image data into preset grids to construct a vehicle position matrix; Collecting the speed of each vehicle entering the intersection in the image data, and combining it with the vehicle position matrix to obtain a vehicle speed matrix; Substituting the vehicle position matrix and the vehicle speed matrix into a first preset model to obtain a first feature; The streetlight status when the image data is acquired is concatenated with the first feature and then fed into a fully connected layer to obtain a second feature; the streetlight status includes a signal light phase and duration; A traffic signal control solution set is obtained according to a preset rule, and the traffic signal control solution set and the second feature are substituted into a preset network to obtain a target traffic light control solution.

2. A road traffic signal control method according to claim 1, characterized in that: Substituting the vehicle position matrix and the vehicle speed matrix into a first preset model to obtain a first feature includes: Substituting the vehicle position matrix and the vehicle speed matrix into the first convolution layer to obtain a first convolution feature; Substituting the first convolutional feature into the attention enhancement module to obtain the first attention feature; Substitute the first attention feature into the second convolutional layer to obtain the second convolutional feature; Substitute the second convolution feature into the attention enhancement module to obtain the second attention feature, and flatten the second attention feature to obtain the first feature.

3. A road traffic signal control method according to claim 2, characterized in that: The attention enhancement module includes a first attention enhancement module and a second attention enhancement module: The attention enhancement module obtains input features, substitutes the input features into the first attention enhancement module to perform average pooling operations and maximum pooling operations to obtain first average pooling features and first maximum pooling features; Substituting the first average pooling feature and the first maximum pooling feature into a multilayer perceptron and performing element-wise addition to obtain a first fused pooling feature; Substituting the input features into the second attention enhancement module to perform average pooling operations and maximum pooling operations to obtain second average pooling features and second maximum pooling features; Performing element-wise addition on the second average pooling feature and the second maximum pooling feature to obtain a second fused pooling feature; Convolving the second fused pooling feature with a 7×7 convolution kernel to obtain a third fused pooling feature; The input features, the first fusion pooling features and the third fusion pooling features are fused to obtain the output features of the attention enhancement module.

4. A road traffic signal control method according to claim 1, characterized in that: The preset network includes two deep neural networks with the same structure, namely a main network and a target network; Substituting the traffic signal control solution set and the second feature into a preset network to obtain a target traffic light control solution includes: updating the preset network according to the traffic signal control scheme and the second feature to obtain a traffic light control network; Substituting the traffic signal control scheme set and the second feature into the traffic light control network, and calculating the Q value corresponding to each traffic signal control scheme in the traffic signal control scheme set; The traffic signal control scheme with the highest Q value is obtained as the target traffic signal control scheme.

5. A road traffic signal control method according to claim 4, characterized in that: Updating the preset network according to the traffic signal control scheme and the second feature to obtain a traffic light control network includes: Step 1: Initialize the network parameters of the main network and the target network to obtain the main network parameters θ and the target network parameters θ′; the main network parameters and the target network parameters are the same; Step 2: The second feature S t Substitute the action branch and advantage branch in the main network to obtain the corresponding solution Q value Q(S t ,A;θ); Step 3: Select execution plan A from the traffic signal control plan set through the greedy strategy t Execute and get the status feedback S after execution t+1 and reward R t , saving the second feature, the execution plan, the reward, and the state feedback as experience data into an experience replay pool; Step 4: Randomly sample data from the experience replay pool to obtain experience data; based on the current state feature S in the experience data j and signal control scheme A j , evaluate the Q value Q corresponding to the empirical data based on the action branch and advantage branch in the main network eval =Q(S j ,A j ;θ) get the main network Q value Q eval ; Step 5: Feedback the state S t+1 Substitute into the main network to obtain the control solution with the maximum Q value a′=argmax a′ Q(S t+1 ,a′;θ); the target network is calculated according to the target network parameter θ′ in the state feedback S t+1 Next, the Q value Q of action a′ is next =Q′(S t+1 ,a′;θ′); Step 6: Using the formula Q target =R t +γ⋅Q next Get the Q value Q of the target network target , where γ is the discount factor; Step 7: Calculate the Q value Q of the main network eval The Q value Q of the target network target The mean square error loss between Loss=E[(Q target −Q eval ) 2 ] and use the Adam optimizer to update the parameters θ of the main network; Step 8: Synchronize the parameters θ of the main network to the target network θ′ at every preset number of steps; Step 9: Repeat steps 2 to 8 until the preset conditions are met to terminate the training and obtain the traffic light control network.

6. A road traffic signal control device, characterized in that: The device comprises: A vehicle position matrix construction module is used to collect image data within the target intersection in real time and divide the image data into preset grids to construct a vehicle position matrix; a vehicle speed matrix construction module, configured to collect the speed of each vehicle entering the intersection in the image data and obtain a vehicle speed matrix in combination with the vehicle position matrix; A first feature determination module, configured to substitute the vehicle position matrix and the vehicle speed matrix into a first preset model to obtain a first feature; A second feature determination module is configured to obtain a streetlight status during the image data, and concatenate the streetlight status and the first feature and input the concatenated features into a fully connected layer to obtain a second feature; the streetlight status includes a signal light phase and duration; The target traffic light control scheme determination module is used to obtain a traffic signal control scheme set according to a preset rule, and substitute the traffic signal control scheme set and the second feature into a preset network to obtain a target traffic light control scheme.

7. A road traffic signal control device according to claim 6, characterized in that: The first feature determination module includes: A first convolution feature generation module, configured to substitute the vehicle position matrix and the vehicle speed matrix into a first convolution layer to obtain a first convolution feature; A first attention feature generation module, configured to substitute the first convolution feature into an attention enhancement module to obtain a first attention feature; A second convolutional feature generation module, configured to substitute the first attention feature into a second convolutional layer to obtain a second convolutional feature; The first feature generation module is used to substitute the second convolution feature into the attention enhancement module to obtain the second attention feature, and flatten the second attention feature to obtain the first feature.

8. A road traffic signal control device according to claim 7, characterized in that: The attention enhancement module includes a first attention enhancement module and a second attention enhancement module, and the device further includes: A first attention enhancement pooling module is used for the attention enhancement module to obtain input features, substitute the input features into the first attention enhancement module to perform average pooling operations and maximum pooling operations to obtain first average pooling features and first maximum pooling features; a first fused pooling feature determination module, configured to substitute the first average pooling feature and the first maximum pooling feature into a multilayer perceptron and then perform element-wise addition to obtain a first fused pooling feature; A second attention enhancement pooling module is used to substitute the input features into the second attention enhancement module to perform average pooling operations and maximum pooling operations to obtain second average pooling features and second maximum pooling features; A second fused pooling feature determination module, configured to perform element-wise addition of the second average pooling feature and the second maximum pooling feature to obtain a second fused pooling feature; A third fusion pooling feature determination module is used to convolve the second fusion pooling feature through a 7×7 convolution kernel to obtain a third fusion pooling feature; A feature fusion module is used to fuse the input features, the first fusion pooling features and the third fusion pooling features to obtain the output features of the attention enhancement module.

9. A road traffic signal control device according to claim 6, characterized in that: The preset network includes two deep neural networks with the same structure, namely a main network and a target network; the target traffic light control scheme determination module includes: a preset network updating module, configured to update the preset network according to the traffic signal control scheme and the second feature to obtain a traffic light control network; a control scheme Q value determination module, configured to substitute the traffic signal control scheme set and the second feature into the traffic light control network, and calculate the Q value corresponding to each traffic signal control scheme in the traffic signal control scheme set; The target traffic light control scheme generation module is used to obtain the traffic signal control scheme with the highest Q value as the target traffic light control scheme.

10. A road traffic signal control device according to claim 9, characterized in that: The execution steps of the preset network update module include: Step 1: Initialize the network parameters of the main network and the target network to obtain the main network parameters θ and the target network parameters θ′; the main network parameters and the target network parameters are the same; Step 2: The second feature S t Substitute the action branch and advantage branch in the main network to obtain the corresponding solution Q value Q(S t ,A;θ); Step 3: Select execution plan A from the traffic signal control plan set through the greedy strategy t Execute and get the status feedback S after execution t+1 and reward R t , saving the second feature, the execution plan, the reward, and the state feedback as experience data into an experience replay pool; Step 4: Randomly sample data from the experience replay pool to obtain experience data; based on the current state feature S in the experience data j and signal control scheme A j , evaluate the Q value Q corresponding to the empirical data based on the action branch and advantage branch in the main network eval =Q(S j ,A j ;θ) get the main network Q value Q eval ; Step 5: Feedback the state S t+1 Substitute into the main network to obtain the control solution with the maximum Q value a′=argmax a′ Q(S t+1 ,a′;θ); the target network is calculated according to the target network parameter θ′ in the state feedback S t+1 Next, the Q value Q of action a′ is next =Q′(S t+1 ,a′;θ′); Step 6: Using the formula Q target =R t +γ⋅Q next Get the Q value Q of the target network target , where γ is the discount factor; Step 7: Calculate the Q value Q of the main network eval The Q value Q of the target network target The mean square error loss between Loss=E[(Q target −Q eval ) 2 ] and use the Adam optimizer to update the parameters θ of the main network; Step 8: Synchronize the parameters θ of the main network to the target network θ′ at every preset number of steps; Step 9: Repeat steps 2 to 8 until the preset conditions are met to terminate the training and obtain the traffic light control network.

Images