Edge computing signal light control method and system based on traffic participant behavior analysis

Abstract

The application provides an edge computing signal lamp control method and system based on traffic participant behavior analysis, acquires video stream data of a target intersection, performs frame-by-frame behavior state analysis on the video stream data through a space-time feature coding network, extracts behavior state features of each traffic participant, inputs the behavior state features into a preset behavior matching model, generates a behavior trigger identifier of the traffic participant within a current time window, generates a signal lamp control instruction of an edge computing node according to the time sequence correlation between the behavior trigger identifier and a phase state of a current signal lamp, and finally adjusts a signal lamp phase switching time sequence of the target intersection based on the signal lamp control instruction, so that a traffic participant that meets a traffic rule conflict condition obtains a priority in the signal lamp phase switching time sequence. The application reduces the risk of traffic accidents, and optimizes the overall traffic efficiency of the intersection by dynamically adjusting the minimum response period and the phase duration parameters.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data processing, in particular to an edge computing signal lamp control method and system based on traffic participant behavior analysis. BACKGROUND

[0002] With the rapid growth of urban traffic flow, intelligent traffic signal control technology has become a key means to improve road traffic efficiency and safety. Traditional signal lamp control methods mainly rely on fixed timing scheme triggered phase switching mechanism, through preset period or simple vehicle queue length detection to realize signal timing adjustment. However, such methods are difficult to cope with complex traffic participant behavior dynamic change scenarios, and the fixed timing mechanism cannot perceive sudden behaviors such as pedestrian illegal crossing and vehicle abnormal lane changing, resulting in risk response lag, difficulty in distinguishing normal traffic from potential conflict behaviors, and easy misjudgment. Although multi-source data fusion scheme can improve the judgment accuracy, it is limited by the cloud centralized computing architecture, and there are data transmission delay and computing resource bottleneck, which cannot meet the millisecond level response demand of real-time signal control. The above defects make it difficult for the existing technology to realize the adaptive optimization of dynamic traffic flow under the premise of ensuring traffic safety. SUMMARY

[0003] The present application provides an edge computing signal lamp control method and system based on traffic participant behavior analysis.

[0004] In a first aspect, the present application embodiment provides an edge computing signal lamp control method based on traffic participant behavior analysis, the method comprising: acquiring video stream data of a target intersection, the video stream data containing motion trajectories of at least one traffic participant and phase state of a current signal lamp; performing frame-by-frame behavior state analysis on the video stream data through a space-time feature coding network, and extracting behavior state features of each traffic participant, wherein the behavior state features include motion direction vector, speed change sequence and spatial position offset; inputting the behavior state features into a preset behavior matching model to generate behavior trigger identifiers of the traffic participants within a current time window, the behavior trigger identifiers being used to indicate whether the traffic participants meet preset traffic rule conflict conditions; generating signal lamp control instructions of an edge computing node according to the time sequence correlation between the behavior trigger identifiers and the phase state of the current signal lamp; and adjusting the signal lamp phase switching timing of the target intersection based on the signal lamp control instructions, so that the traffic participants meeting the traffic rule conflict conditions obtain traffic priority under the signal lamp phase switching timing.

[0005] In a second aspect, the present application embodiment provides a signal lamp control system, comprising: a memory, the memory storing a computer program; and a processor, configured to load the computer program to realize the above edge computing signal lamp control method based on traffic participant behavior analysis.

[0006] The application provides an edge computing signal lamp control method based on traffic participant behavior analysis, which significantly improves the control accuracy and response real-time performance in complex traffic scenes by modeling the time sequence correlation of multi-dimensional dynamic behavior characteristics and signal lamp phase states. Compared with the traditional fixed time sequence control method, the method uses the cooperative analysis of motion direction vector, speed change sequence and spatial position offset to dynamically capture the abnormal behavior trend of traffic participants, and combines the remaining time of the signal lamp to evaluate the conflict priority, effectively avoiding the phase switching lag or redundancy problem caused by feature misjudgment. Through the end-to-end closed-loop processing mechanism of the edge computing node, the millisecond-level localized decision-making from behavior feature extraction to control instruction generation is realized, breaking through the delay bottleneck caused by cloud computing dependence. At the same time, based on the risk-driven phase switching strategy, on the premise of ensuring the priority of the conflict direction, through the reverse phase conflict pre-check and dynamic emptying time calculation, the safety of the opposite traffic flow is considered, forming a self-adaptive control effect that unifies local optimization and global coordination. The method reduces the risk of traffic accidents, optimizes the overall traffic efficiency of the intersection by dynamically adjusting the minimum response period and phase duration parameters, and realizes high-robustness intelligent signal control without relying on multi-source sensor fusion and artificial rule preset. BRIEF DESCRIPTION OF DRAWINGS

[0007] Figure 1 is a flowchart of an edge computing signal lamp control method based on traffic participant behavior analysis provided by an embodiment of the application.

[0008] Figure 2 is a composition schematic diagram of a signal lamp control system provided by an embodiment of the application. DETAILED DESCRIPTION

[0009] Please refer to Figure 1 , Figure 1 is a flowchart of an edge computing signal lamp control method based on traffic participant behavior analysis provided by an embodiment of the application. The edge computing signal lamp control method based on traffic participant behavior analysis can be executed by a signal lamp control system. The edge computing signal lamp control method based on traffic participant behavior analysis can include the following steps:

[0010] Step S100: Obtain video stream data of a target intersection, wherein the video stream data contains the motion trajectory of at least one traffic participant and the phase state of the current signal lamp.

[0011] In the embodiments of the present application, the target intersection is a specific traffic intersection that needs to be controlled by traffic lights, such as crossroads, T-shaped intersections, and other types of traffic nodes with traffic light control in cities. Video stream data is a continuous sequence of image data continuously collected by camera devices installed around the target intersection. Traffic participants include pedestrians, vehicles (such as cars, motorcycles, bicycles, etc.), and motion trajectories refer to their movement paths in the intersection space within a certain period of time, which can be obtained by tracking and recording the positions of traffic participants in each frame of video stream data. The phase state of the current traffic light represents the display state of the traffic light at a certain time, such as red light, green light, yellow light, and the remaining time of each light turning on.

[0012] Step S200: Perform frame-by-frame behavior state analysis on the video stream data through the spatio-temporal feature encoding network to extract the behavior state features of each traffic participant, wherein the behavior state features include the motion direction vector, the speed change sequence, and the spatial position offset.

[0013] The spatio-temporal feature encoding network is a deep learning network for processing data with time and space dimensions, which can extract meaningful spatio-temporal features from video stream data. Frame-by-frame behavior state analysis refers to analyzing each frame of video stream data to determine the behavior state of traffic participants in that frame. Behavior state features are key information describing the behavior of traffic participants. The motion direction vector represents the motion direction of a traffic participant at a certain time, which can be obtained by analyzing its motion trajectory. The speed change sequence records the speed changes of a traffic participant over a period of time, reflecting its dynamic characteristics. The spatial position offset represents the position change between adjacent frames of a traffic participant.

[0014] In practical applications, the spatio-temporal feature encoding network can adopt an architecture combining convolutional neural network (CNN) and recurrent neural network (RNN). First, CNN is used to extract spatial features in video frames, and RNN is used to process time series information, thereby realizing spatio-temporal feature encoding of video stream data. For example, for a car driving in an intersection, frame-by-frame analysis of video stream data through the spatio-temporal feature encoding network can obtain the motion direction vector, speed change sequence, and spatial position offset of the car in each frame. Specifically, the motion direction vector can be obtained by calculating the direction of position change between adjacent frames; the speed change sequence can be obtained by differentiating the position change of the car in consecutive frames; and the spatial position offset can be determined by comparing the specific coordinate positions of the car in adjacent frames. Through these behavior state features, the behavior patterns of traffic participants can be accurately described, providing a basis for subsequent traffic rule conflict detection and traffic light control.

[0015] As an implementation, in step S200, the spatio-temporal feature coding network is used to analyze the behavior state of each traffic participant in the video stream data, and the behavior state features of each traffic participant are extracted, which can include the following steps S210-S250:

[0016] In step S210, a three-dimensional spatio-temporal tensor of the video stream data is constructed, which includes a time dimension, a spatial position dimension, and a pixel feature dimension.

[0017] The three-dimensional spatio-temporal tensor is a multi-dimensional data structure used to represent the video stream data, which integrates the time, space and pixel feature information of the video stream data. The time dimension represents the frame sequence of the video stream data, reflecting the change of the traffic scene over time; the spatial position dimension represents the spatial coordinates of each pixel point in the video frame, determining the position of the traffic participant in the image; and the pixel feature dimension represents the feature information of each pixel point, such as color, brightness, etc.

[0018] The specific implementation of constructing the three-dimensional spatio-temporal tensor can be implemented as follows: first, the video stream data is divided by frame to obtain a series of video frames. Then, for each frame of video, the pixel information is extracted and coded to form a two-dimensional pixel feature matrix. Then, these two-dimensional pixel feature matrices are stacked in time order to form a three-dimensional spatio-temporal tensor. For example, for a video stream data containing 100 frames, each frame of video has a resolution of 640x480 pixels, and each pixel has 3 color channels (RGB), then the three-dimensional spatio-temporal tensor constructed has dimensions of 100 (time dimension) x 640 (spatial position dimension - width) x 480 (spatial position dimension - height) x 3 (pixel feature dimension). By constructing the three-dimensional spatio-temporal tensor, the video stream data can be converted into a format suitable for processing by the spatio-temporal feature coding network, facilitating the extraction of spatio-temporal features therein.

[0019] In step S220, a convolutional long short-term memory network is used to extract spatio-temporal features from the three-dimensional spatio-temporal tensor, and a motion feature map of each traffic participant is obtained.

[0020] Convolutional Long Short-Term Memory Network (ConvLSTM) is a deep learning model that combines the characteristics of Convolutional Neural Network (CNN) and Long Short-Term Memory Network (LSTM). It can effectively process data with spatio-temporal structure. In the embodiments of the present application, ConvLSTM is used to extract spatio-temporal features from three-dimensional spatio-temporal tensors to capture the motion information of traffic participants in time and space. The motion feature map is a two-dimensional feature map that reflects the motion patterns and features of traffic participants in video frames. For example, for a car turning at an intersection, the motion feature map of the car can be obtained by processing its corresponding three-dimensional spatio-temporal tensor with ConvLSTM. Different channels in the motion feature map may respectively represent the activation intensity of the motion patterns such as straight driving, turning, accelerating, and decelerating of the car. By analyzing the numerical values of these channels, the motion state of the car can be accurately identified.

[0021] Specifically, step S220 can include the following steps S221-S229:

[0022] Step S221: divide the three-dimensional spatio-temporal tensor into consecutive time step slices in the time dimension, each time step slice containing the spatial position dimension and the pixel feature dimension of the current frame.

[0023] The time step slice is a two-dimensional data block obtained by dividing the three-dimensional spatio-temporal tensor in the time dimension, and each time step slice corresponds to a frame in the video stream data. By dividing the three-dimensional spatio-temporal tensor into time step slices, the processing of spatio-temporal data can be converted into the processing of a series of two-dimensional data, facilitating subsequent convolution operations and feature extraction.

[0024] In specific implementation, the three-dimensional spatio-temporal tensor can be divided at fixed time intervals. For example, for a three-dimensional spatio-temporal tensor containing 100 frames, it can be divided by frame to obtain 100 time step slices. The dimension of each time step slice is the spatial position dimension (such as 640x480) and the pixel feature dimension (such as 3 color channels). In this way, the three-dimensional spatio-temporal tensor can be converted into a series of two-dimensional data blocks suitable for convolution operations, laying the foundation for subsequent spatio-temporal feature extraction.

[0025] Step S222: perform three-dimensional convolution operation on each time step slice to generate spatio-temporal convolution feature map of the current time step, and the convolution kernel of the three-dimensional convolution operation spans adjacent frames in the time dimension to capture motion continuity.

[0026] The three-dimensional convolution operation is a convolution operation in three-dimensional space, and its convolution kernel not only convolves the input data in the spatial dimension, but also spans adjacent frames in the time dimension, so as to capture the change information of data in time and space. The spatio-temporal convolution feature map is the output result of the three-dimensional convolution operation, which reflects the spatio-temporal features of the current time step.

[0027] When performing a three-dimensional convolution operation, the convolution kernel slides in both the time and spatial dimensions, performing convolution calculations on a portion of the region of each time step slice and its adjacent frames. For example, assuming the size of the convolution kernel is 3x3x3 (time dimension x spatial width x spatial height), it will span 3 adjacent frames in the time dimension and perform convolution calculations on a 3x3 region in the spatial dimension. In this way, the motion continuity of the traffic participants between adjacent frames can be captured. For each time step slice, after the three-dimensional convolution operation, a spatio-temporal convolution feature map is generated, the dimension of which depends on the number of convolution kernels and the dimension of the input data. By generating the spatio-temporal convolution feature map, the spatio-temporal features in the video stream data can be extracted, providing a basis for subsequent feature processing and analysis.

[0028] Step S223: inputting the spatio-temporal convolution feature map into the forget gate structure of the convolution long short-term memory network, generating a forget gate output through element-wise multiplication and activation function processing to control the retention ratio of the previous hidden state.

[0029] The forget gate structure is an important component in the convolution long short-term memory network (ConvLSTM), which is used to control the retention ratio of the previous hidden state, thereby realizing selective forgetting of historical information. The spatio-temporal convolution feature map is the output result of the previous three-dimensional convolution operation, containing the spatio-temporal feature information of the current time step. In the forget gate structure, the spatio-temporal convolution feature map is first concatenated with the previous hidden state, and then processed through a convolution layer and a Sigmoid activation function. The output value of the Sigmoid activation function ranges from 0 to 1, which represents the retention ratio of each element. By element-wise multiplication of the previous hidden state and the forget gate output, selective retention of the previous hidden state can be realized. For example, if the value of an element of the forget gate output is 0.8, it means that the previous hidden state information corresponding to the element will be retained at a ratio of 80%. In this way, the forget gate structure can dynamically adjust the retention ratio of the previous hidden state according to the current input information, avoiding the network from falling into overfitting or forgetting important historical information, thereby improving the network's processing capability for spatio-temporal sequence data.

[0030] Step S224: inputting the spatio-temporal convolution feature map and the forget gate output into the input gate structure after concatenation, generating a candidate cell state of the current time step through convolution operation and activation function processing.

[0031] The input gate structure is also a key part of the ConvLSTM, which is used to determine how much of the current input information should be added to the cell state. In the embodiment of the present application, the spatio-temporal convolution feature map and the forget gate output are concatenated to obtain a feature vector containing the current input information and historical information.

[0032] The specific operation is as follows: first, the spatio-temporal convolution feature map and the forget gate output are spliced in the channel dimension to form a new feature matrix. Then, the feature matrix is input into the convolution layer of the input gate structure, and the features are extracted through convolution operation. Then, the convolution output is processed through the Sigmoid activation function and the hyperbolic tangent activation function respectively. The output of the Sigmoid activation function is used to control whether to allow the input information to enter the cell state, and the output of the hyperbolic tangent activation function is the preliminary estimate of the candidate cell state. Finally, the two outputs are multiplied element by element to obtain the candidate cell state of the current time step. For example, for a vehicle driving at an intersection, through the input gate structure, it can be dynamically decided whether to add the new motion information of the vehicle to the cell state according to the current spatio-temporal features and historical information, so as to more accurately track the motion trajectory of the vehicle.

[0033] Step S225: The candidate cell state is weighted and superimposed with the previous cell state adjusted by the forget gate to generate the updated current cell state.

[0034] The cell state is a data structure used to store and transmit historical information in ConvLSTM. In the embodiment of the application, the candidate cell state generated in the previous step is weighted and superimposed with the previous cell state adjusted by the forget gate to update the current cell state.

[0035] The specific process is as follows: first, the previous cell state has been processed by the forget gate, and part of the important historical information is retained. Then, the candidate cell state is added to the previous cell state adjusted by the forget gate element by element to obtain the updated current cell state. This weighted superposition method can combine historical information and current input information, so that the cell state can be continuously updated and evolved, thereby better adapting to the motion changes of traffic participants. For example, for a vehicle that turns at an intersection multiple times, the historical information of previous turns can be combined with the current motion information through this method to accurately reflect the overall motion state of the vehicle.

[0036] Step S226: The current cell state is input into the output gate structure, processed through convolution operation and hyperbolic tangent activation function to generate the hidden state of the current time step.

[0037] The output gate structure is used to determine how much information in the current cell state should be output as the hidden state. The hidden state is the output result of ConvLSTM, which contains important information of the current time step and can be used for subsequent feature fusion and analysis.

[0038] The specific operation is as follows: the current cell state is input into the convolution layer of the output gate structure, and the features are extracted through convolution operation. Then, the convolution output is processed through the Sigmoid activation function to obtain a control vector, and each element of the vector represents the output proportion of the corresponding element in the cell state. At the same time, the current cell state is processed through the hyperbolic tangent activation function, and the value is mapped to the range of -1 to 1. Finally, the two outputs are multiplied element by element to obtain the hidden state of the current time step. For example, for a vehicle that suddenly accelerates at an intersection, the output gate structure can accurately output the current motion state information of the vehicle according to the information in the cell state, providing a basis for subsequent motion feature analysis.

[0039] Step S227: Traverse all time step slices, and perform cross-layer feature fusion on the hidden state of each time step along the channel dimension to generate a multi-scale motion feature map.

[0040] Cross-layer feature fusion is a method of integrating feature information at different levels, which can fully utilize the hidden state information of different time steps to extract more rich and comprehensive motion features. The multi-scale motion feature map is the result of cross-layer feature fusion, which contains motion feature information at different scales and levels.

[0041] In specific implementation, all time step slices are traversed, and the hidden state of each time step is spliced along the channel dimension. Then, a convolution layer is used to fuse the spliced features to generate a multi-scale motion feature map. For example, for a video stream data containing 100 time step slices, the hidden state of each time step is spliced in the channel dimension to obtain a feature matrix with larger dimensions. Then, the feature matrix is reduced and fused through convolution operation to generate a feature map containing different scale motion features. Through cross-layer feature fusion, the motion information of different time steps can be effectively integrated, and the expression ability of motion features can be improved to provide more rich feature information for subsequent local motion pattern feature extraction.

[0042] Step S228: Perform spatial pyramid pooling on the multi-scale motion feature map to extract local motion pattern features of different field of view ranges.

[0043] Spatial pyramid pooling (SPP) is a method for extracting features at different scales. It can extract local features at different field of view ranges without changing the size of the input feature map. In the embodiments of the present application, spatial pyramid pooling is performed on the multi-scale motion feature map to extract local motion pattern features at different field of view ranges. The specific operation is as follows: the multi-scale motion feature map is input into the spatial pyramid pooling layer, which divides the feature map into regions of different sizes and then performs pooling operation on each region. Possible pooling operations include max pooling and average pooling. For example, the feature map can be divided into regions of different sizes such as 1x1, 2x2, 4x4, and then max pooling operation is performed on each region to obtain local feature vectors at different scales. Finally, these local feature vectors are spliced to obtain a feature vector containing local motion pattern features at different field of view ranges. Through spatial pyramid pooling, local motion pattern features at different field of view ranges in the multi-scale motion feature map can be extracted, which can more comprehensively describe the motion patterns of traffic participants and provide more abundant information for subsequent motion feature analysis and behavior recognition.

[0044] Step S229: Channel attention weighted fusion of local motion pattern features and hidden state at the last time step to generate motion feature maps of each traffic participant, each channel of the motion feature map corresponding to the activation strength of the preset motion pattern.

[0045] Channel attention weighted fusion is a method for weighted combination of different features, which can dynamically adjust the weight of the features according to the importance of different channels. In the embodiments of the present application, channel attention weighted fusion is performed on the local motion pattern features and the hidden state at the last time step to generate motion feature maps of each traffic participant.

[0046] For example, the following operations can be implemented: first, calculate the channel attention weight of the local motion pattern features and the hidden state at the last time step. The channel attention weight can be calculated by an attention mechanism, for example, using a fully connected layer and a Sigmoid activation function to map the input features to a weight vector between 0 and 1. Then, multiply the local motion pattern features and the hidden state at the last time step by the corresponding channel attention weight element by element to obtain the weighted features. Finally, add the weighted local motion pattern features and the hidden state at the last time step to generate motion feature maps of each traffic participant. Each channel of the motion feature map corresponds to the activation strength of the preset motion pattern, and by analyzing the values of these channels, the motion pattern of the traffic participant can be accurately identified. For example, for a vehicle driving at an intersection, a certain channel of the motion feature map may correspond to the acceleration motion pattern of the vehicle, and the higher the activation strength of this channel, the greater the possibility of vehicle acceleration.

[0047] Step S230: Key point trajectory tracking is performed on the motion feature map to generate a motion direction vector and a spatial position offset.

[0048] Key point trajectory tracking is a method for tracking the change of key point positions of traffic participants in video frames. By performing key point trajectory tracking on the motion feature map, the motion trajectory information of the traffic participants can be obtained. The motion direction vector represents the motion direction of the traffic participant at a certain time, and the spatial position offset represents the position change amount of the traffic participant between adjacent frames. For example, for a pedestrian walking at an intersection, the motion direction vector and the spatial position offset of the pedestrian can be obtained through key point trajectory tracking. The motion direction vector can indicate the walking direction of the pedestrian, and the spatial position offset can reflect the moving distance and direction of the pedestrian between adjacent frames.

[0049] Specifically, step S230 can include the following steps S231-S236:

[0050] Step S231: Extract a candidate key point set of each traffic participant from the motion feature map, the candidate key point set being obtained by response value threshold screening, wherein each candidate key point contains spatial coordinates and motion intensity features.

[0051] The candidate key point set is a set of key points with high response values selected from the motion feature map, which can represent important features and motion information of the traffic participants. The response value threshold is a pre-set value used to screen key points with response values higher than the threshold. The spatial coordinates represent the position of the key point in the motion feature map, and the motion intensity features represent the motion intensity corresponding to the key point.

[0052] Specific extraction, for example, can be implemented as follows: first, traverse the motion feature map to calculate the response value of each point. The response value can be calculated by calculating the value of a certain channel or multiple channels of the motion feature map, for example, using methods such as maximum value, average value, etc. Then, points with response values higher than the response value threshold are selected as candidate key points, and their spatial coordinates and motion intensity features are recorded. For example, for a motion feature map representing vehicle motion, by setting an appropriate response value threshold, the key points such as the front and rear of the vehicle can be screened, and the spatial coordinates and motion intensity features of these key points can be used for subsequent trajectory tracking and behavior analysis.

[0053] Step S232: Bidirectional matching is performed on the candidate key point sets of adjacent two frames, a matching cost matrix is calculated based on the motion intensity feature similarity and the spatial coordinate offset distance, and the first K pairs of key points with the smallest matching cost in the cost matrix are retained as initial trajectory segments, K≥1.

[0054] Bidirectional matching is a method for matching key points between two adjacent frames, which can improve the accuracy and reliability of matching. The matching cost matrix is calculated according to the motion intensity feature similarity and the spatial coordinate offset distance, which is used to measure the matching degree between key points in two adjacent frames. The initial trajectory segment is obtained by retaining the top K pairs of key points with the smallest matching cost in the matching cost matrix, and these key point pairs can constitute the preliminary motion trajectory of the road user.

[0055] The calculation of the matching cost matrix can be implemented as follows: for the candidate key point set of the two adjacent frames, the motion intensity feature similarity and the spatial coordinate offset distance between each pair of key points are calculated. The motion intensity feature similarity can be obtained by calculating the cosine similarity between the motion intensity feature vectors of the two key points, and the spatial coordinate offset distance can be obtained by calculating the Euclidean distance between the spatial coordinates of the two key points. Then, the motion intensity feature similarity and the spatial coordinate offset distance are combined by weighting to obtain the matching cost between each pair of key points. The matching costs of all key point pairs form a matrix, i.e. the matching cost matrix. Finally, the top K pairs of key points with the smallest matching cost are selected from the matching cost matrix, which are used as the initial trajectory segment. For example, for the key point set representing a vehicle in two adjacent frames, the corresponding key points of the vehicle in the adjacent frames can be accurately found through bidirectional matching and matching cost calculation, thereby constructing the initial motion trajectory of the vehicle.

[0056] Step S233: input the initial trajectory segment into the trajectory continuity verification unit, detect whether there is a sequence of trajectory points with continuous spatial position changes within three consecutive frames, eliminate abnormal trajectory segments with a jump amplitude exceeding a preset threshold, and generate an optimized trajectory set.

[0057] The trajectory continuity verification unit is a module for verifying the continuity of the trajectory, which can detect whether there is a sequence of trajectory points with continuous spatial position changes within three consecutive frames. Abnormal trajectory segments refer to trajectory segments with a jump amplitude exceeding a preset threshold, which may be caused by false matching due to noise, occlusion, etc. and need to be eliminated. The optimized trajectory set is the trajectory set obtained after trajectory continuity verification, which contains more accurate and reliable trajectory information.

[0058] The specific verification can be implemented as follows: input the initial trajectory segment into the trajectory continuity verification unit, and for each trajectory segment, check its spatial position change within three consecutive frames. Calculate the spatial coordinate offset distance of the trajectory point between two adjacent frames. If the offset distance change within three consecutive frames exceeds a preset threshold, it is considered that there is a jump in the trajectory segment, and it is eliminated as an abnormal trajectory segment. For example, for a trajectory segment representing pedestrian movement, if the position of the pedestrian suddenly jumps greatly within three consecutive frames, and the jump amplitude exceeds the preset threshold, the trajectory segment will be considered an abnormal trajectory segment and will be eliminated. Through trajectory continuity verification, the accuracy and reliability of trajectory tracking can be improved, and more accurate trajectory information can be provided for subsequent calculations of motion direction vectors and spatial position offsets.

[0059] Step S234: Calculate the direction angle of each trajectory in the optimized trajectory set, generate an initial direction vector based on the spatial coordinate difference between the starting point and the end point of the trajectory, and fit the direction change trend of N consecutive trajectory points through a sliding window to generate a smoothed motion direction vector.

[0060] Direction angle calculation is used to determine the direction angle of a trajectory. The initial direction vector is generated based on the spatial coordinate difference between the trajectory's starting and ending points, representing the approximate direction of the trajectory. Sliding window fitting is used to smooth changes in trajectory direction. By fitting the directional trends of N consecutive trajectory points, a smoother and more accurate motion direction vector can be generated.

[0061] The specific calculation can be implemented as follows: for each trajectory in the optimized trajectory set, the spatial coordinate difference between its starting point and end point is calculated, and the direction angle is calculated based on the difference to generate an initial direction vector. Then, a sliding window of fixed length is used to slide on the trajectory, and for each consecutive N trajectory points in the window, the direction change trend is fitted by the least squares method and other methods to obtain a smoothed direction vector. These smoothed direction vectors are combined in sequence to obtain a smoothed motion direction vector. For example, for a trajectory representing vehicle travel, the initial direction vector is obtained by calculating the coordinate difference between the starting point and the end point of the trajectory, and then the sliding window is used to fit the direction change trend of 10 consecutive trajectory points to generate a smoothed motion direction vector, which can more accurately reflect the vehicle's travel direction.

[0062] Step S235: Calculate the cumulative displacement in the X-axis and Y-axis directions based on the spatial coordinate difference between the starting point and the end point of the trajectory corresponding to the smoothed motion direction vector, and generate the instantaneous spatial position offset by combining the position difference of the trajectory point between the current frame and the previous frame.

[0063] The accumulated displacement amount refers to the position change amount of the trajectory start point and the end point in the X-axis and Y-axis directions, which reflects the overall moving distance of the traffic participant in a period of time. The instantaneous spatial position offset refers to the position difference of the trajectory points between the current frame and the previous frame, which reflects the instantaneous moving distance of the traffic participant between adjacent frames.

[0064] The specific calculation can be implemented, for example, as follows: for the trajectory corresponding to the smoothed motion direction vector, the coordinate difference between the start point and the end point in the X-axis and Y-axis directions is calculated to obtain the accumulated displacement amount. At the same time, for the trajectory points of the current frame and the previous frame, the coordinate difference between them in the X-axis and Y-axis directions is calculated to obtain the instantaneous spatial position offset. For example, for a vehicle driving at an intersection, the accumulated displacement amount of the vehicle in the X-axis and Y-axis directions can be obtained by calculating the coordinate difference between the start point and the end point of its trajectory, and the instantaneous spatial position offset of the vehicle can be obtained by calculating the coordinate difference between the trajectory points of the current frame and the previous frame. These displacement amount information can be used to analyze the motion speed and motion trend of the vehicle.

[0065] Step S236: The accumulated displacement amount and the instantaneous spatial position offset are weighted and fused to generate a final spatial position offset, and the weight coefficient of the weighted fusion is dynamically adjusted according to the trajectory length and the stability of the motion direction vector; the motion direction vector and the final spatial position offset are stored in groups according to the traffic participant identifier to form a trajectory tracking result sequence aligned with the video stream data timestamp.

[0066] By weighting and fusing the accumulated displacement amount and the instantaneous spatial position offset, a more accurate and reliable final spatial position offset can be obtained. The weight coefficient can be pre-set and dynamically adjusted according to the trajectory length and the stability of the motion direction vector. The longer the trajectory length and the more stable the motion direction vector, the greater the weight of the accumulated displacement amount; otherwise, the greater the weight of the instantaneous spatial position offset. The trajectory tracking result sequence is a sequence obtained by storing the motion direction vector and the final spatial position offset in groups according to the traffic participant identifier and aligning it with the video stream data timestamp, which contains the motion trajectory information of each traffic participant.

[0067] For example, the weight coefficients of the accumulated displacement amount and the instantaneous spatial position offset are calculated according to the trajectory length and the stability of the motion direction vector. Then, the accumulated displacement amount and the instantaneous spatial position offset are multiplied by the corresponding weight coefficients respectively, and the results are added to obtain the final spatial position offset. Finally, the motion direction vector and the final spatial position offset are stored in groups according to the traffic participant identifier and aligned with the timestamp of the video stream data to form a trajectory tracking result sequence. For example, for a traffic scene containing multiple vehicles and pedestrians, the motion trajectory information of each vehicle and pedestrian can be obtained through weighted fusion and group storage, which can be used for subsequent traffic rule conflict detection and signal light control.

[0068] Step S240: difference calculation on the motion feature map of the continuous frames by sliding window to generate a speed change sequence.

[0069] The sliding window is a method for processing time series data, which can analyze and calculate data within a continuous time window. Difference calculation refers to calculating the difference between adjacent data. By difference calculation on the motion feature map of the continuous frames, the speed change information of the road users within different time windows can be obtained. The speed change sequence is composed of the speed change values within each time window, which reflects the speed change trend of the road users.

[0070] For example, for a vehicle driving at an intersection, by difference calculation on the motion feature map of its continuous frames by sliding window, the speed change sequence of the vehicle within different time windows can be obtained. The larger the element value in the speed change sequence, the more violent the speed change of the vehicle within the time window. By analyzing the speed change sequence, the acceleration, deceleration and other motion states of the vehicle can be accurately understood.

[0071] Specifically, step S240 can include the following steps S241-S247:

[0072] Step S241: set a sliding window of fixed length on the time dimension of the motion feature map, and the sliding window covers the motion feature map of continuous T frames.

[0073] The sliding window is a time window of fixed length, which slides on the time dimension of the motion feature map for processing the continuous T frames of motion feature map. The purpose of setting the sliding window is to analyze the motion change of the road users within a predetermined time range.

[0074] Specific setting can be implemented as follows: according to actual demand and analysis accuracy, the length T of the sliding window is determined. For example, T can be set to 5 frames, i.e. the sliding window covers the motion feature map of continuous 5 frames. Then, the sliding window is slid in time sequence from the first frame of the motion feature map until it covers all frames of the motion feature map. By setting the sliding window, the continuous motion feature map can be divided into multiple time windows of fixed length, which facilitates subsequent difference calculation and speed change analysis.

[0075] Step S242: perform channel-level difference operation on the motion feature map of adjacent two frames within the sliding window to generate a channel difference feature map of each pair of adjacent frames within the window.

[0076] The channel-level difference operation refers to calculating the difference between two adjacent frames in each channel of the motion feature map. Through the channel-level difference operation, the channel difference feature map of each pair of adjacent frames in the window can be obtained. The channel difference feature map reflects the motion change of each channel between the two adjacent frames.

[0077] The specific calculation can be implemented as follows: for the motion feature map of each pair of adjacent frames in the sliding window, perform element-by-element subtraction operation on each channel to obtain the channel difference feature map. For example, for a motion feature map containing 3 channels, perform subtraction operation on each channel of the two adjacent frames to obtain 3 channel difference feature maps. Through the channel-level difference operation, the motion change information between adjacent frames can be extracted, providing a basis for subsequent cumulative motion change feature map generation and speed change analysis.

[0078] Step S243: Accumulate and sum the channel difference feature map along the time dimension to generate a cumulative motion change feature map within the window. The cumulative motion change feature map reflects the sum of motion changes of each channel within the sliding window.

[0079] The accumulation and summation refers to adding the channel difference feature map in the time dimension. Through the accumulation and summation, the cumulative motion change feature map within the window can be obtained. The cumulative motion change feature map reflects the sum of motion changes of each channel within the sliding window, which can more comprehensively describe the motion change of the traffic participant within the time window.

[0080] The specific calculation can be implemented as follows: add all channel difference feature maps within the sliding window in the time dimension to obtain the cumulative motion change feature map. For example, for a sliding window containing 5 frames of motion feature maps, add the channel difference feature maps of each pair of adjacent frames in the time dimension to obtain the cumulative motion change feature map. Through the generation of the cumulative motion change feature map, the motion change information between adjacent frames can be integrated, providing more accurate information for subsequent speed change index extraction.

[0081] Step S244: Divide the cumulative motion change feature map into spatial regions, and extract the cumulative change amount of each traffic participant corresponding region as the initial speed change index.

[0082] The spatial region division refers to dividing the cumulative motion change feature map into different regions, and each region corresponds to a traffic participant. The initial speed change indicator refers to the speed change information extracted from the cumulative change amount of the region corresponding to each traffic participant, which can reflect the speed change of the traffic participant in the time window. The division and extraction can be implemented as follows: according to the position information of the traffic participant in the motion feature map, the cumulative motion change feature map is divided into different regions, and each region corresponds to a traffic participant. Then, the cumulative change amount in each region is calculated by summation or averaging, and the cumulative change amount of the region is obtained as the initial speed change indicator. For example, for a traffic scene containing multiple vehicles and pedestrians, according to the positions of the vehicles and pedestrians in the motion feature map, the cumulative motion change feature map is divided into multiple regions, and the cumulative change amount of each region is calculated to obtain the initial speed change indicator of each vehicle and pedestrian. Through spatial region division and initial speed change indicator extraction, the speed change information of each traffic participant can be accurately obtained, providing a basis for subsequent speed change trend analysis and normalization processing.

[0083] Step S245: comparing the initial speed change indicator of the current sliding window with the initial speed change indicator of the previous sliding window to generate a window-to-window speed change trend vector.

[0084] The trend comparison refers to comparing the change trend of the initial speed change indicator of the current sliding window with the initial speed change indicator of the previous sliding window, and through the trend comparison, a window-to-window speed change trend vector can be generated. The window-to-window speed change trend vector reflects the speed change trend of the traffic participant in the adjacent two time windows. The specific comparison and generation can be implemented as follows: for each traffic participant, the initial speed change indicator of the current sliding window is subtracted or divided by the initial speed change indicator of the previous sliding window to obtain the window-to-window speed change trend vector. For example, if the initial speed change indicator of the current sliding window is [10, 20, 30], and the initial speed change indicator of the previous sliding window is [5, 15, 25], then the window-to-window speed change trend vector is [5, 5, 5]. Through the generation of the window-to-window speed change trend vector, the speed change trend of the traffic participant can be analyzed, providing a basis for subsequent speed change indicator adjustment.

[0085] Step S246: adjusting the initial speed change indicator of the current sliding window according to the window-to-window speed change trend vector to generate a normalized speed change vector.

[0086] The initial speed change indicator of the current sliding window is adjusted to make the speed change indicator more accurately reflect the actual speed change of the traffic participant. The normalized speed change vector is the adjusted speed change vector, and its value range is usually between 0 and 1, which is convenient for subsequent analysis and comparison.

[0087] The specific adjustment can be implemented as follows: the initial speed change indicator of the current sliding window is adjusted according to the inter-window speed change trend vector. For example, if the inter-window speed change trend vector shows that the speed is increasing, the initial speed change indicator of the current sliding window can be appropriately increased; if the inter-window speed change trend vector shows that the speed is decreasing, the initial speed change indicator of the current sliding window can be appropriately decreased. Then, the adjusted speed change indicator is normalized to make its value range between 0 and 1, and a normalized speed change vector is obtained. For example, the maximum-minimum normalization method can be used to subtract the minimum value from the adjusted speed change indicator, and then divide by the difference between the maximum value and the minimum value to obtain the normalized speed change vector. Through adjustment and normalization, the speed change indicator is more comparable and accurate, and reliable data is provided for subsequent speed change sequence generation.

[0088] Step S247: sliding the window along the time axis until covering all frames of the motion feature map, splicing the normalized speed change vectors of each window in time sequence to form a speed change sequence, each element of the speed change sequence corresponding to the speed change intensity of the traffic participant in the preset time window.

[0089] Sliding the window along the time axis means sliding the window from the first frame of the motion feature map in time sequence until covering all frames of the motion feature map. Splicing means connecting the normalized speed change vectors of each window in time sequence to form a continuous sequence. The speed change sequence reflects the speed change of the traffic participant in the entire video stream data, and each element corresponds to the speed change intensity of the traffic participant in the preset time window.

[0090] The specific splicing can be implemented as follows: starting from the first frame of the motion feature map, a sliding window is set, difference calculation, cumulative summation, region division, trend comparison, index adjustment and normalization processing are performed, and the normalized speed change vector of the first window is obtained. Then, the sliding window is slid back by one frame, and the above process is repeated to obtain the normalized speed change vector of the second window. In this way, until the sliding window covers the entire frame of the motion feature map. Finally, the normalized speed change vectors of each window are spliced in time sequence to form a speed change sequence. For example, for a video stream data containing 100 frames of motion feature maps, a sliding window length of 5 frames is set, and after the above processing, 20 normalized speed change vectors of the window are obtained. Splicing these vectors in time sequence forms a speed change sequence containing 20 elements. By generating the speed change sequence, the speed change trend of the traffic participant can be comprehensively understood, and important information is provided for subsequent traffic rule conflict detection and signal light control.

[0091] Step S250: Normalizing and splicing the motion direction vector, the speed change sequence and the spatial position offset to form a behavior state feature matrix.

[0092] Normalization refers to converting data of different ranges and scales into data of the same range and scale, so as to facilitate subsequent processing and analysis. Splicing refers to connecting the motion direction vector, the speed change sequence and the spatial position offset in sequence to form a matrix. The behavior state feature matrix is a matrix composed of the normalized motion direction vector, the speed change sequence and the spatial position offset, which can comprehensively describe the behavior state of the traffic participant.

[0093] The specific normalization and splicing can be implemented as follows: first, the motion direction vector, the speed change sequence and the spatial position offset are normalized respectively. For the motion direction vector, each element can be divided by the length of the vector to make its length 1; for the speed change sequence and the spatial position offset, the maximum-minimum normalization method can be used to convert the value range to 0 to 1. Then, the normalized motion direction vector, the speed change sequence and the spatial position offset are spliced in the form of columns or rows to form a behavior state feature matrix. For example, assuming that the dimension of the motion direction vector is 3, the length of the speed change sequence is 10, and the dimension of the spatial position offset is 2, the dimension of the spliced behavior state feature matrix can be (3+10+2) rows 1 column or 1 row (3+10+2) columns. Through normalization and splicing, the behavior state feature matrix formed can be more effectively used for subsequent behavior matching and traffic rule conflict detection.

[0094] As an implementation manner, the training process of the space-time feature coding network includes the following steps S201-S206:

[0095] Step S201: Collect sample video stream data of historical traffic scenes and perform behavior annotation on traffic participants in the sample video stream data to generate behavior label sequences.

[0096] The sample video stream data of historical traffic scenes refers to video stream data collected at the target intersection or other similar intersections in the past period of time, which contains rich behavior information of traffic participants. Behavior annotation refers to classifying and labeling the behavior of traffic participants in the sample video stream data, for example, labeling the behavior of pedestrians as normal walking, running a red light, etc., and labeling the behavior of vehicles as normal driving, illegal turning, etc. The behavior label sequence is a sequence composed of the behavior labels of each traffic participant, which can be used as supervision information for training the spatio-temporal feature encoding network.

[0097] Specific collection and annotation can be implemented as follows: collect sample video stream data of historical traffic scenes through camera devices installed at intersections. Then, manually or using automated annotation tools, perform behavior annotation on traffic participants in the sample video stream data. For each traffic participant, assign a corresponding behavior label according to its behavior performance in the video stream data. Arrange the behavior labels of all traffic participants in chronological order to form a behavior label sequence. For example, for a sample video stream data containing 100 traffic participants, after annotation, a behavior label sequence containing 100 behavior labels can be obtained. By collecting sample video stream data and generating behavior label sequences, supervised learning data can be provided for the training of the spatio-temporal feature encoding network, improving the training effect and accuracy of the network.

[0098] Step S202: Construct a hybrid architecture of three-dimensional convolutional neural network and recurrent neural network, which includes a spatio-temporal feature extraction branch and a behavior classification branch.

[0099] The three-dimensional convolutional neural network (3D CNN) is a convolutional neural network for processing data with three-dimensional structure, which can effectively extract spatio-temporal features in video stream data. The recurrent neural network (RNN) is a neural network for processing sequence data, which can capture the temporal dependence in the data. The hybrid architecture combines 3D CNN and RNN, which can take full advantage of both and better handle video stream data. The spatio-temporal feature extraction branch is used to extract spatio-temporal features from video stream data, and the behavior classification branch is used to classify the behavior of traffic participants according to the extracted spatio-temporal features.

[0100] The specific construction can be implemented as follows: first, the 3D CNN part is constructed, and the input video stream data can be processed using multiple 3D convolution layers and pooling layers to extract spatio-temporal features. Then, the output of the 3D CNN is input to the RNN part, and the recurrent unit such as LSTM or GRU can be used to process the spatio-temporal features to capture the time dependence. The spatio-temporal feature extraction branch is composed of 3D CNN and RNN, and its output is a spatio-temporal feature vector. The behavior classification branch is a fully connected layer or a multi-layer perceptron, which takes the spatio-temporal feature vector as input and outputs the behavior classification result of the traffic participant. For example, for an input video stream data, after processing by 3D CNN and RNN, a spatio-temporal feature vector is obtained, and then the vector is input to the behavior classification branch, and the behavior label of the traffic participant is output, such as normal driving, illegal turning, etc. By constructing the hybrid architecture, the spatio-temporal features in the video stream data can be effectively extracted, and the behavior of the traffic participant can be accurately classified.

[0101] Step S203: A multi-scale feature fusion module is set in the spatio-temporal feature extraction branch, and the multi-scale feature fusion module is used to integrate the motion features under different receptive fields.

[0102] The multi-scale feature fusion module is a module for integrating different scale and hierarchical feature information, which can fully utilize the motion features under different receptive fields and improve the expression ability of the features. The receptive field refers to the size of the input region that each neuron in the convolutional neural network can perceive, and different convolutional layers have different receptive fields, so different scale features can be extracted.

[0103] The specific setting can be implemented as follows: in the spatio-temporal feature extraction branch, the output features of different convolutional layers are input into the multi-scale feature fusion module. The multi-scale feature fusion module can use convolution operation, pooling operation or attention mechanism to fuse features of different scales. For example, a 1x1 convolutional layer can be used to reduce the dimension of features of different scales, then the reduced features are spliced, and a convolutional layer is used for feature fusion. By setting the multi-scale feature fusion module, the motion features under different receptive fields can be integrated, and more rich and comprehensive spatio-temporal features can be extracted, providing more accurate feature information for subsequent behavior classification.

[0104] Step S204: The feature discrimination of the spatio-temporal feature extraction branch is optimized by the contrastive learning loss function, which promotes the feature vectors of the same behavior samples to gather in the embedding space.

[0105] The contrastive learning loss function is a loss function for optimizing feature discrimination, which promotes the aggregation of feature vectors of the same behavior samples in the embedding space and the separation of feature vectors of different behavior samples in the embedding space by comparing the feature similarity between different samples. The embedding space refers to a low-dimensional space to which the feature vectors are mapped, in which the relationship between the feature vectors can be more intuitively observed and analyzed.

[0106] The specific optimization can be implemented as follows: first, randomly selecting a positive sample pair and a negative sample pair from the sample video stream data, the positive sample pair containing different instances of the same behavior category, and the negative sample pair containing instances of different behavior categories. Then, generating transformed views of the positive sample pair by data augmentation, the transformed views including operations such as spatial cropping, time slicing, and color jittering to increase the diversity of the samples. Calculating the cosine similarity between the feature vectors of the positive sample pair and constructing a positive sample similarity distribution matrix; calculating the cosine similarity between the feature vectors of the negative sample pair and constructing a negative sample similarity distribution matrix. The temperature scaling cross-entropy loss function is used to maximize the separation degree of the positive sample similarity distribution matrix and the negative sample similarity distribution matrix. Finally, the convolution kernel parameters of the spatio-temporal feature extraction branch and the gating weights of the recurrent neural network are adjusted through back propagation to optimize the feature discrimination. For example, for the two behaviors of normal driving and illegal turning, the contrastive learning loss function is optimized to make the sample feature vectors of the normal driving behavior aggregate together in the embedding space, the sample feature vectors of the illegal turning behavior aggregate together in the embedding space, and the distance between the two types of feature vectors is as large as possible. By optimizing the feature discrimination of the spatio-temporal feature extraction branch, the accuracy and reliability of behavior classification can be improved.

[0107] As an implementation, the optimization process of the contrastive learning loss function includes the following steps S2041-S2046:

[0108] Step S2041: Randomly selecting a positive sample pair and a negative sample pair from the sample video stream data, the positive sample pair containing different instances of the same behavior category, and the negative sample pair containing instances of different behavior categories.

[0109] The positive sample pair refers to a sample pair composed of different instances of the same behavior category, for example, a sample pair composed of two normally driving vehicle instances. The negative sample pair refers to a sample pair composed of instances of different behavior categories, for example, a sample pair composed of a normally driving vehicle instance and an illegal turning vehicle instance. Randomly selecting a positive sample pair and a negative sample pair is to ensure the diversity and randomness of the samples and improve the effect of contrastive learning.

[0110] Specifically, for example, the following can be implemented: first, classify the behaviors of traffic participants in the sample video stream data into different behavior categories. Then, randomly select different instances from each behavior category to form positive sample pairs. At the same time, randomly select instances from different behavior categories to form negative sample pairs. For example, for the vehicle behavior in the sample video stream data, the behavior categories are divided into normal driving, illegal turning, and red light running, etc. Two different vehicle instances are randomly selected from the normal driving category to form a positive sample pair, and one vehicle instance is selected from the normal driving category and the illegal turning category to form a negative sample pair. By randomly selecting positive sample pairs and negative sample pairs, diverse samples can be provided for contrastive learning, which helps to optimize the feature discrimination of the spatio-temporal feature extraction branch.

[0111] Step S2042: generating transformed views of the positive sample pairs through data augmentation, the transformed views including spatial cropping, temporal slicing, and color jittering.

[0112] Data augmentation is a method for increasing the number and diversity of samples. By augmenting the positive sample pairs, more samples can be generated, improving the generalization ability of the model. Transformed views refer to the sample views obtained after data augmentation, including spatial cropping, temporal slicing, and color jittering operations.

[0113] Specifically, for example, the following can be implemented: for each sample in the positive sample pair, perform spatial cropping operation to randomly crop a part of the sample area to change the spatial features of the sample. Perform temporal slicing operation to randomly select a part of the time segment of the sample to change the temporal features of the sample. Perform color jittering operation to randomly adjust the color of the sample, such as changing brightness, contrast, saturation, etc., to change the color features of the sample. Through these operations, the transformed views of the positive sample pairs are generated. For example, for a sample video containing vehicle driving, spatial cropping can crop part of the background around the vehicle, temporal slicing can select a certain time segment during the vehicle driving, and color jittering can change the color tone of the video. By generating transformed views, the diversity of positive sample pairs can be increased, and the effect of contrastive learning can be improved.

[0114] Step S2043: calculating the cosine similarity between the feature vectors of the positive sample pairs, and constructing a positive sample similarity distribution matrix.

[0115] Cosine similarity is an index for measuring the similarity between two vectors, which represents the similarity by calculating the cosine of the angle between the two vectors. The positive sample similarity distribution matrix is a matrix composed of the cosine similarity between the feature vectors of the positive sample pairs, which can reflect the similarity distribution between the positive sample pairs.

[0116] The specific calculation and construction can be implemented as follows: for each positive sample pair, input it into the spatio-temporal feature extraction branch to obtain a feature vector. Then, the cosine similarity between the feature vectors of each positive sample pair is calculated. The cosine similarities of all positive sample pairs are arranged in order to construct a positive sample similarity distribution matrix. For example, for a sample set containing 100 positive sample pairs, the cosine similarity between the feature vectors of each positive sample pair is calculated to obtain 100 cosine similarity values, which are arranged in order to form a 100x1 positive sample similarity distribution matrix. By constructing the positive sample similarity distribution matrix, the similarity distribution between the positive sample pairs can be observed intuitively, providing a basis for subsequent calculation of the contrastive learning loss function.

[0117] Step S2044: Calculate the cosine similarity between the feature vectors of the negative sample pairs, and construct a negative sample similarity distribution matrix.

[0118] Similar to step S2043, the cosine similarity between the feature vectors of the negative sample pairs is calculated, and a negative sample similarity distribution matrix is constructed. The negative sample similarity distribution matrix reflects the similarity distribution between the negative sample pairs.

[0119] The specific calculation and construction can be implemented as follows: for each negative sample pair, input it into the spatio-temporal feature extraction branch to obtain a feature vector. Then, the cosine similarity between the feature vectors of each negative sample pair is calculated. The cosine similarities of all negative sample pairs are arranged in order to construct a negative sample similarity distribution matrix. For example, for a sample set containing 100 negative sample pairs, the cosine similarity between the feature vectors of each negative sample pair is calculated to obtain 100 cosine similarity values, which are arranged in order to form a 100x1 negative sample similarity distribution matrix. By constructing the negative sample similarity distribution matrix, the similarity distribution between the negative sample pairs can be observed intuitively, providing a basis for subsequent calculation of the contrastive learning loss function.

[0120] Step S2045: Maximize the separation degree of the positive sample similarity distribution matrix and the negative sample similarity distribution matrix by the temperature scaling cross-entropy loss function.

[0121] The temperature scaling cross-entropy loss function is a loss function used to optimize classification problems, which adjusts the separation degree between the positive sample similarity distribution matrix and the negative sample similarity distribution matrix to promote the aggregation of feature vectors of samples with the same behavior in the embedding space and the separation of feature vectors of samples with different behaviors in the embedding space.

[0122] The specific optimization can be implemented, for example, by inputting the positive sample similarity distribution matrix and the negative sample similarity distribution matrix into a temperature scaling cross-entropy loss function. The temperature scaling cross-entropy loss function calculates the loss value according to the similarity distribution of the positive sample and the negative sample. Through the back propagation algorithm, the convolution kernel parameters of the spatio-temporal feature extraction branch and the gating weights of the recurrent neural network are adjusted to minimize the loss value, thereby maximizing the separation degree of the positive sample similarity distribution matrix and the negative sample similarity distribution matrix. For example, by adjusting the parameters, the values in the positive sample similarity distribution matrix are as large as possible, and the values in the negative sample similarity distribution matrix are as small as possible, thereby realizing the aggregation of feature vectors of the same type of behavior samples and the separation of feature vectors of different types of behavior samples. By using the temperature scaling cross-entropy loss function, the feature discrimination of the spatio-temporal feature extraction branch can be effectively optimized, and the accuracy of behavior classification can be improved.

[0123] Step S2046: adjusting the convolution kernel parameters of the spatio-temporal feature extraction branch and the gating weights of the recurrent neural network through back propagation.

[0124] In the embodiments of the present application, the convolution kernel parameters of the spatio-temporal feature extraction branch and the gating weights of the recurrent neural network are adjusted through the back propagation algorithm to optimize the performance of the network.

[0125] The specific adjustment can be implemented, for example, by calculating the gradient of the loss function with respect to the convolution kernel parameters of the spatio-temporal feature extraction branch and the gating weights of the recurrent neural network according to the calculation result of the temperature scaling cross-entropy loss function. Then, the convolution kernel parameters and the gating weights are updated according to the gradient descent algorithm. For example, for the convolution kernel parameters, optimization algorithms such as stochastic gradient descent (SGD), Adam, etc. can be used for updating. By continuously performing back propagation and parameter updating, the spatio-temporal feature extraction branch can better extract features with discrimination, and the accuracy of behavior classification can be improved.

[0126] Step S205: using a dynamic focal loss function in the behavior classification branch, the dynamic focal loss function automatically adjusts the classification weights according to the class distribution of the samples.

[0127] The dynamic focal loss function is a loss function used to solve the problem of class imbalance. It can automatically adjust the classification weights according to the class distribution of the samples, so that the model pays more attention to the samples of minority classes. In the behavior classification branch, due to the large difference in the number of samples of different behavior classes, the use of the dynamic focal loss function can improve the classification performance of the model for minority class behaviors.

[0128] Specifically, the predicted result and the true label are input into a dynamic focal loss function at the output layer of the behavior classification branch, for example. The dynamic focal loss function automatically adjusts the classification weight of each class according to the class distribution of the sample. For the sample of the minority class, the classification weight is increased so that the model pays more attention to these samples; for the sample of the majority class, the classification weight is reduced to avoid the model being too biased towards the majority class. For example, in the traffic participant behavior classification, the number of samples of the illegal behavior may be small, and the number of samples of the normal behavior may be large. Using the dynamic focal loss function can increase the classification weight of the illegal behavior samples and improve the classification accuracy of the model for illegal behavior. By using the dynamic focal loss function, the class imbalance problem can be effectively solved, and the overall performance of the behavior classification can be improved.

[0129] Step S206: The output of the optimized spatiotemporal feature extraction branch is taken as the input feature of the preset behavior matching model.

[0130] The preset behavior matching model is a model for judging whether the behavior of the traffic participant meets the preset traffic rule conflict condition. It takes the output of the optimized spatiotemporal feature extraction branch as the input feature to perform behavior matching and conflict detection. Specifically, after training and optimization in the above steps, the spatiotemporal feature extraction branch can extract spatiotemporal features with high discrimination. Taking these features as the input of the preset behavior matching model, the preset behavior matching model can judge whether the behavior of the traffic participant meets the preset traffic rule conflict condition according to these features. For example, the preset behavior matching model can judge whether a vehicle runs a red light or a pedestrian violates the rule of crossing the road according to the input spatiotemporal features. By taking the output of the optimized spatiotemporal feature extraction branch as the input feature of the preset behavior matching model, the accuracy and reliability of behavior matching and conflict detection can be improved.

[0131] Step S300: The behavior state feature is input into the preset behavior matching model to generate a behavior trigger identifier of the traffic participant in the current time window. The behavior trigger identifier is used to indicate whether the traffic participant meets the preset traffic rule conflict condition.

[0132] The behavior state feature is extracted through steps S200-S250 and contains information such as the motion direction vector, the speed change sequence, and the spatial position offset of the traffic participant. The preset behavior matching model is a trained and optimized model that can judge whether the behavior of the traffic participant meets the preset traffic rule conflict condition according to the input behavior state feature. The behavior trigger identifier is an identifier used to indicate whether the behavior of the traffic participant triggers the traffic rule conflict. It can be a binary value (such as 0 indicating no trigger and 1 indicating trigger) or an identifier containing the conflict type and trigger timestamp.

[0133] Specifically, the preset behavior matching model can be constructed based on a machine learning or deep learning algorithm, such as a decision tree model, a neural network model, etc. In the embodiment, a decision tree model combined with rule judgment is adopted to realize behavior matching.

[0134] As an implementation, in step S300, inputting the behavior state features into the preset behavior matching model to generate the behavior trigger identifier of the traffic participant in the current time window can include the following steps S310-S390:

[0135] Step S310: Calculate the direction intersection angle between the motion direction vector in the behavior state features and the set of allowed passing directions corresponding to the phase state of the current signal light to generate a sequence of direction deviation angles.

[0136] The motion direction vector represents the motion direction of the traffic participant at a certain moment, and the set of allowed passing directions corresponding to the phase state of the current signal light is the legal passing direction determined according to the display state of the signal light. The direction intersection angle calculation is to calculate the intersection angle between two vectors through the dot product and the module length of the vector. The sequence of direction deviation angles is a sequence composed of the intersection angle between the motion direction vector and the allowed passing direction in each time window.

[0137] Specific calculation can be implemented as follows: for the motion direction vector in the behavior state features, perform dot product operation with each allowed passing direction vector in the set of allowed passing directions corresponding to the phase state of the current signal light, and then divide by the product of the module lengths of the two vectors to obtain the cosine value of the intersection angle. Calculate the intersection angle value through the inverse cosine function, and take the intersection angle value as the direction deviation angle in the time window. Calculate each time window in turn to obtain the sequence of direction deviation angles. For example, in a crossroad, the current signal light displays a straight green light, and the allowed passing direction is the positive east direction. The motion direction vector of a vehicle points to the northeast direction. By calculating the intersection angle of the two vectors, the direction deviation angle of the vehicle in the time window is obtained. By generating the sequence of direction deviation angles, the deviation between the motion direction of the traffic participant and the allowed passing direction can be intuitively understood, which provides a basis for subsequent direction conflict detection.

[0138] Step S320: Compare the value of each time window in the speed change sequence with the upper and lower bounds of the speed threshold interval associated with the phase state to generate a speed out-of-bound marker sequence.

[0139] The speed change sequence is generated by step S240, which reflects the speed change of the traffic participant in different time windows. The phase state associated speed threshold interval is the legal speed range determined according to the phase state of the current traffic light and the traffic rules, for example, the speed of the vehicle should be 0 during the red light, and the speed of the vehicle has an upper limit and a lower limit during the green light. The speed out-of-boundary marker sequence is a sequence composed of markers of whether the speed is out of boundary in each time window, and the marker can be a binary value (such as 0 for no out-of-boundary and 1 for out-of-boundary).

[0140] The specific comparison can be implemented, for example, for the numerical value of each time window in the speed change sequence, compare it with the upper and lower limits of the phase state associated speed threshold interval. If the numerical value is less than the lower limit or greater than the upper limit, it is marked as out-of-boundary, and the marker value is 1; otherwise, it is marked as no out-of-boundary, and the marker value is 0. Compare each time window in turn to obtain the speed out-of-boundary marker sequence. For example, in the green straight phase state, the speed threshold interval is [20, 60] km / h, and the speed of a vehicle in a certain time window is 70 km / h, then the marker value of the time window is 1. By generating the speed out-of-boundary marker sequence, it can be quickly judged whether the speed of the traffic participant meets the requirements of the current phase state, and provide a basis for subsequent speed anomaly detection.

[0141] Step S330: sliding window accumulation in time dimension is performed on the direction deviation angle sequence, and a direction conflict identifier is generated when the cumulative angles of three consecutive windows exceed a preset deviation threshold.

[0142] The sliding window accumulation in time dimension means that a sliding window of a fixed length is set on the direction deviation angle sequence, and the direction deviation angles in the window are accumulated and summed. The preset deviation threshold is a pre-set angle value, and when the cumulative angles of three consecutive windows exceed the threshold, it is considered that the movement direction of the traffic participant has a conflict, and a direction conflict identifier is generated.

[0143] The specific accumulation and judgment can be implemented, for example, a sliding window of length 3 is set on the direction deviation angle sequence, starting from the first element of the sequence, the cumulative value of the direction deviation angle in each window is calculated in turn. When the cumulative angle of a certain window exceeds the preset deviation threshold, a direction conflict identifier is generated. For example, the preset deviation threshold is 60°, and in a certain three consecutive time windows, the direction deviation angles are 20°, 30° and 25°, and the cumulative angle is 75°, which exceeds the preset deviation threshold, so a direction conflict identifier is generated. By performing sliding window accumulation and judgment on the direction deviation angle sequence, the direction conflict behavior of the traffic participant can be effectively detected.

[0144] Step S340: Perform continuous positive event detection on the speed boundary crossing label sequence, and generate a speed anomaly label when detecting that the speed breaks the same direction threshold twice in a row.

[0145] Continuous positive event detection refers to detecting consecutive positive labels (i.e., boundary crossing label value is 1) in the speed boundary crossing label sequence. When detecting that the speed breaks the same direction threshold twice in a row, it is considered that the speed of the traffic participant is abnormal, and a speed anomaly label is generated. The same direction threshold refers to the speed exceeding the upper limit or falling below the lower limit.

[0146] Specific detection can be implemented, for example, as follows: traverse the speed boundary crossing label sequence, when encountering a positive label, check whether the next label is also a positive label, and the directions of the two boundary crossings are the same (i.e., both exceeding the upper limit or both falling below the lower limit). If the conditions are met, a speed anomaly label is generated. For example, in the speed boundary crossing label sequence, the 3rd and 4th label values are both 1, and both are caused by the speed exceeding the upper limit, so a speed anomaly label is generated. By performing continuous positive event detection on the speed boundary crossing label sequence, the speed anomaly behavior of the traffic participant can be accurately detected.

[0147] Step S350: Input the direction conflict label and the speed anomaly label into the phase state associated decision tree model, and calculate the priority weight of the conflict behavior according to the remaining duration of the phase state.

[0148] The phase state associated decision tree model is a model that makes decisions according to the phase state of the current signal light and the conflict behavior information. It can calculate the priority weight of the conflict behavior according to different phase states and conflict types. The remaining duration of the phase state refers to the time remaining for the current signal light phase state.

[0149] Specific calculation can be implemented, for example, as follows: input the direction conflict label and the speed anomaly label as inputs into the phase state associated decision tree model. The decision tree model makes node judgments and branch selections according to the phase state of the current signal light and the conflict behavior information. For each conflict behavior, its priority weight is calculated according to the remaining duration of the phase state. For example, in the red light phase state, if the remaining time is short and there are direction conflict and speed anomaly behaviors, the decision tree model may consider that the priority of the direction conflict is higher, because the direction conflict may cause more serious traffic conflicts. By calculating the priority weight of the conflict behavior, different types of conflict behaviors can be sorted, providing a basis for subsequent dynamic fusion.

[0150] Step S360: Perform dynamic fusion on the direction conflict label and the speed anomaly label based on the priority weight, and generate an initial behavior trigger probability.

[0151] Dynamic fusion involves combining direction conflict indicators and speed anomaly indicators based on their priority to generate an initial maneuver trigger probability. This initial maneuver trigger probability represents the likelihood that a traffic participant will trigger a traffic rule conflict within the current time window.

[0152] Specifically, fusion can be implemented by taking a weighted sum of the direction conflict flag and the speed anomaly flag based on the priority weights calculated in step S350. For example, if the direction conflict flag has a priority weight of 0.6, the speed anomaly flag has a priority weight of 0.4, the direction conflict flag has a value of 1, and the speed anomaly flag has a value of 0, then the initial behavior trigger probability is 0.6 × 1 + 0.4 × 0 = 0.6. Through dynamic fusion, both direction conflict and speed anomaly conflict behaviors can be comprehensively considered to generate a more accurate initial behavior trigger probability.

[0153] Step S370: aligning the initial behavior trigger probability with the trigger probability sequence of the historical time window, and generating a corrected behavior trigger probability through an exponential smoothing algorithm.

[0154] Time series alignment involves aligning the initial behavior trigger probability with the trigger probability sequence of the historical time window in the temporal dimension to ensure data consistency. Exponential smoothing is an algorithm used to smooth time series data. It can generate smoother and more accurate forecasts based on historical and current data.

[0155] The specific processing can be implemented as follows: the initial behavior trigger probability and the trigger probability sequence of the historical time window are arranged in chronological order to align the time series. Then, the exponential smoothing algorithm is used to process the initial behavior trigger probability and the historical trigger probability sequence. The formula of the exponential smoothing algorithm is: S t =αy t +(1-α)S t-1 , where S t is the corrected behavior trigger probability, y t is the initial behavior triggering probability, S t-1 is the corrected trigger probability for the previous time window, and α is the smoothing coefficient, ranging from [0 to 1]. For example, if the smoothing coefficient α is 0.3, the corrected trigger probability for the previous time window is 0.5, and the initial behavior trigger probability is 0.6, then the corrected behavior trigger probability is 0.3 × 0.6 + (1 - 0.3) × 0.5 = 0.53. Time series alignment and exponential smoothing can reduce the impact of noise and fluctuations, generating a more stable and accurate corrected behavior trigger probability.

[0156] Step S380: When the corrected behavior trigger probability exceeds a dynamic trigger threshold that is negatively correlated with the current intersection traffic density, a valid behavior trigger flag is generated.

[0157] The dynamic triggering threshold is a threshold dynamically adjusted according to the current intersection traffic density, which is negatively related to the traffic density, that is, the greater the traffic density, the smaller the dynamic triggering threshold. When the corrected behavior triggering probability exceeds the dynamic triggering threshold, it is considered that the behavior of the traffic participant triggers the traffic rule conflict, and an effective behavior triggering identifier is generated. The specific judgment can be implemented, for example: real-time monitoring of the traffic density of the current intersection, calculating the dynamic triggering threshold according to the traffic density. For example, the traffic density information can be obtained through the vehicle detector installed at the intersection or video analysis technology, and then the dynamic triggering threshold is calculated according to the pre-set function relationship. Compare the corrected behavior triggering probability with the dynamic triggering threshold, if the corrected behavior triggering probability exceeds the dynamic triggering threshold, an effective behavior triggering identifier is generated. By using the dynamic triggering threshold, the actual traffic situation of the intersection can be used to flexibly judge whether the behavior of the traffic participant triggers the conflict, and the accuracy of conflict detection is improved.

[0158] Step S390: logically constrain and verify the effective behavior triggering identifier and the remaining time of the phase state, eliminate the triggering event with the remaining time less than the safety response threshold, and generate a final behavior triggering identifier set, each element in the final behavior triggering identifier set containing a conflict type and a triggering timestamp.

[0159] The logical constraint verification refers to the logical judgment of the effective behavior triggering identifier and the remaining time of the phase state, to ensure that the triggering event has actual significance and processability. The safety response threshold is a pre-set time value, when the remaining time of the phase state is less than the threshold, it is considered that even if the conflict event is triggered, it cannot be processed in time, so the triggering event is eliminated. The final behavior triggering identifier set is the triggering event set obtained after logical constraint verification, which contains the conflict type and triggering timestamp of each triggering event.

[0160] The specific verification and generation can be implemented, for example: for each effective behavior triggering identifier, compare it with the remaining time of the phase state. If the remaining time of the phase state is less than the safety response threshold, the triggering event is eliminated; otherwise, the conflict type and triggering timestamp of the triggering event are added to the final behavior triggering identifier set. For example, the safety response threshold is 5 seconds, and the remaining time of the phase state corresponding to an effective behavior triggering identifier is 3 seconds, then the triggering event is eliminated; if the remaining time is 8 seconds, the conflict type (such as direction conflict, speed anomaly) and triggering timestamp of the triggering event are added to the final behavior triggering identifier set. Through logical constraint verification and generation of the final behavior triggering identifier set, it can be ensured that the triggering event has actual processing value, and accurate information is provided for subsequent signal light control.

[0161] Step S400: generating a signal light control instruction of the edge computing node according to the time sequence correlation between the behavior trigger identification and the phase state of the current signal light.

[0162] The behavior trigger identification contains the behavior conflict information of the traffic participant, and the phase state of the current signal light represents the display state and the remaining time of the signal light at a certain moment. The time sequence correlation refers to the time relationship between the trigger time of the behavior trigger identification and the phase state of the current signal light. The edge computing node refers to a computing device located at the edge of the network, which can perform real-time processing and decision-making on local data. The signal light control instruction is an instruction for controlling the phase switching of the signal light, which contains information such as phase switching direction and duration.

[0163] As an implementation, step S400, generating a signal light control instruction of the edge computing node according to the time sequence correlation between the behavior trigger identification and the phase state of the current signal light, can include the following steps S410-S4100:

[0164] Step S410: mapping and matching each behavior trigger identification in the final behavior trigger identification set with the remaining time of the phase state to generate a priority parameter set containing conflict type, trigger timestamp and remaining time.

[0165] Mapping and matching refers to associating each behavior trigger identification in the final behavior trigger identification set with the remaining time of the current signal light phase state to generate a priority parameter set containing conflict type, trigger timestamp and remaining time. The priority parameter set is used for subsequent phase adjustment rule activation and priority calculation.

[0166] Specific mapping and matching can be implemented, for example, for each behavior trigger identification in the final behavior trigger identification set, obtaining its conflict type and trigger timestamp. At the same time, obtain the remaining time of the current signal light phase state. Combine the conflict type, trigger timestamp and remaining time into a priority parameter, and arrange all priority parameters in order to generate a priority parameter set. For example, there is a direction conflict trigger identification in the final behavior trigger identification set, the trigger timestamp is 10:00:00, and the remaining time of the current signal light phase state is 30 seconds, then the generated priority parameter is (direction conflict, 10:00:00, 30 seconds). Through mapping and matching and generating a priority parameter set, the behavior trigger identification and the time information of the phase state can be integrated to provide more comprehensive information for subsequent signal light control decision-making.

[0167] Step S420: activating the corresponding phase adjustment rule according to the conflict type in the priority parameter set, the phase adjustment rule containing the direction conflict priority strategy and the speed anomaly degradation strategy.

[0168] The phase adjustment rule is a signal light phase adjustment strategy formulated according to different conflict types, and includes a direction conflict priority strategy and a speed anomaly degradation strategy. The direction conflict priority strategy refers to, when there is a direction conflict, preferentially adjusting the phase of the signal light to solve the direction conflict problem. The speed anomaly degradation strategy refers to, when there is a speed anomaly, appropriately reducing the priority of processing the conflict.

[0169] Specific activation may be implemented as: traversing the priority parameter set, and activating the corresponding phase adjustment rule according to the conflict type therein. If the conflict type is a direction conflict, the direction conflict priority strategy is activated; if the conflict type is a speed anomaly, the speed anomaly degradation strategy is activated. For example, the priority parameter set has a direction conflict priority parameter and a speed anomaly priority parameter, for the direction conflict priority parameter, the direction conflict priority strategy is activated; for the speed anomaly priority parameter, the speed anomaly degradation strategy is activated. By activating the corresponding phase adjustment rule according to the conflict type, different processing strategies can be adopted for different conflict situations, improving the effectiveness of signal light control.

[0170] Step S430: Spatial orientation aggregation is performed on the multiple direction conflict identifiers in the same time window through the direction conflict priority strategy, to generate a phase switching direction candidate set.

[0171] The direction conflict priority strategy is a strategy that is preferentially considered when handling direction conflicts, and the spatial orientation aggregation refers to integrating the spatial orientation information of the multiple direction conflict identifiers in the same time window. The phase switching direction candidate set is a set composed of possible phase switching directions obtained after spatial orientation aggregation.

[0172] Specific aggregation and generation may be implemented as: for the multiple direction conflict identifiers in the same time window, spatial orientation information such as the driving direction of the vehicle, the lane where the vehicle is located, etc. is extracted. The spatial orientation information is classified and merged, and direction conflict identifiers with the same or similar spatial orientation are found. According to the aggregated spatial orientation information, possible phase switching directions are determined, and a phase switching direction candidate set is generated. For example, at a crossroads, there are multiple vehicle direction conflict identifiers in the same time window, and after spatial orientation aggregation, it is found that most of the conflict vehicles come from the east-west direction, so the phase switching direction candidate set may include a green light phase switching from the current phase to the east-west direction. Through spatial orientation aggregation and generation of the phase switching direction candidate set, the direction that needs to be phase switched can be more accurately determined, improving the pertinence of signal light control.

[0173] Step S440: The duration of the direction conflict is calculated based on the remaining time of the phase state and the trigger timestamp, and a phase adjustment parameter is generated in combination with the occurrence frequency of the speed anomaly identifier in the speed anomaly degradation strategy.

[0174] The direction conflict duration refers to a length of time from a direction conflict triggering time to a current time or a phase state ending time, which can be calculated by a remaining time of the phase state and the triggering time stamp. The frequency of the speed anomaly identification in the speed anomaly degradation strategy refers to a number of times of the speed anomaly identification appearing in a limited time range. The phase adjustment parameter is a parameter for adjusting the signal light phase, which comprehensively considers the direction conflict duration and the frequency of the speed anomaly identification.

[0175] Specific calculation and generation can be implemented as follows: for each direction conflict identification, the direction conflict duration is calculated according to the remaining time of the phase state and the triggering time stamp. For example, the triggering time stamp is 10:00:00, the current time is 10:00:10, and the phase state remaining time is 20 seconds, and then the direction conflict duration is 30 seconds. At the same time, the frequency of the speed anomaly identification is counted. According to the direction conflict duration and the frequency of the speed anomaly identification, the phase adjustment parameter is generated. For example, a weighted summation method can be used, the weight of the direction conflict duration is 0.7, the weight of the frequency of the speed anomaly identification is 0.3, the direction conflict duration is 30 seconds, and the frequency of the speed anomaly identification is 2 times, and then the phase adjustment parameter is 0.7*30+0.3*2=21.6. By calculating the direction conflict duration and combining the frequency of the speed anomaly identification to generate the phase adjustment parameter, the influence of different types of conflicts can be more comprehensively considered, and a more reasonable basis for signal light phase adjustment can be provided.

[0176] Step S450: inputting the phase switching direction candidate set and the phase adjustment parameter into a time sequence optimizer to generate a phase switching time sequence scheme meeting minimum phase switching interval and maximum traffic efficiency constraints.

[0177] The time sequence optimizer is a model for optimizing the signal light phase switching time sequence, which can generate a phase switching time sequence scheme meeting minimum phase switching interval and maximum traffic efficiency constraints according to the phase switching direction candidate set and the phase adjustment parameter. The minimum phase switching interval refers to a minimum time interval between adjacent two phase switching, which is used to ensure the stability and safety of traffic; the maximum traffic efficiency refers to a phase switching scheme in which the traffic flow is the largest under the premise of meeting the minimum phase switching interval.

[0178] The specific optimization can be implemented as follows: inputting the phase switching direction candidate set and the phase adjustment parameter into a time series optimizer. The time series optimizer can use optimization algorithms such as genetic algorithm, simulated annealing algorithm, etc. to search and optimize the phase switching time sequence. In the optimization process, the constraints of minimum phase switching interval and maximum traffic efficiency are considered. For example, through genetic algorithm, a group of phase switching time sequence schemes are randomly generated, and the traffic efficiency of each scheme and whether it meets the minimum phase switching interval requirement are calculated. Then, through selection, crossover and mutation operations, better schemes are constantly evolved until the optimal phase switching time sequence scheme that meets the constraint conditions is found. By generating the phase switching time sequence scheme through the time series optimizer, the traffic efficiency can be improved while ensuring traffic stability and safety.

[0179] Step S460: generating phase switching direction encoding according to the difference between the target phase direction in the phase switching time sequence scheme and the current phase state.

[0180] The phase switching direction encoding is an encoding method for representing the phase switching direction, which can be generated according to the difference between the target phase direction in the phase switching time sequence scheme and the current phase state. The target phase direction refers to the phase direction that needs to be switched to in the phase switching time sequence scheme, and the current phase state refers to the current display state of the traffic light.

[0181] The specific generation can be implemented as follows: comparing the target phase direction in the phase switching time sequence scheme with the current phase state, and calculating the difference between them. According to the difference, different encoding methods are used to generate the phase switching direction encoding. For example, in a crossroad, the current phase state is south-north direction green light, and the target phase direction is east-west direction green light, then the difference is from south-north direction to east-west direction. Binary encoding can be used to represent south-north direction as 00 and east-west direction as 01, then the phase switching direction encoding is 01. By generating the phase switching direction encoding, the phase switching direction information can be converted into digital encoding, which is convenient for subsequent traffic light control instruction generation and transmission.

[0182] Step S470: binary bit splicing the phase switching direction encoding and the duration in the phase adjustment parameter to form a traffic light control instruction containing direction identification bits and duration identification bits.

[0183] Binary bit splicing refers to converting the phase switching direction encoding and the duration in the phase adjustment parameter into binary bits and splicing them together to form a complete traffic light control instruction. The direction identification bits are used to represent the direction of phase switching, and the duration identification bits are used to represent the duration after phase switching.

[0184] The specific splicing can be implemented as follows: the phase switching direction code is converted into a binary bit, for example, the phase switching direction code is 01, and the binary bit is 00000001. The duration in the phase adjustment parameter is also converted into a binary bit, for example, the duration is 30 seconds, and the binary bit is 00011110. Then, the direction identification bit and the duration identification bit are spliced to form the signal lamp control instruction. For example, the direction identification bit 00000001 and the duration identification bit 00011110 are spliced together to obtain the signal lamp control instruction 0000000100011110. Through the binary bit splicing, the signal lamp control instruction formed can accurately convey the phase switching direction and duration information, facilitating the edge computing node to control the signal lamp.

[0185] Step S480: Before generating the signal lamp control instruction, a conflict pre-check is performed to verify whether the phase switching direction code has a spatial overlap risk with the track of the unfinished passing vehicle of the opposite lane.

[0186] The conflict pre-check refers to checking the phase switching direction code and the track of the unfinished passing vehicle of the opposite lane before generating the signal lamp control instruction to verify whether there is a spatial overlap risk. The spatial overlap risk refers to the possibility that the unfinished passing vehicle of the opposite lane may collide or conflict with the vehicle in the new phase direction after the phase switching. The specific pre-check can be implemented as follows: the track information of the unfinished passing vehicle of the opposite lane is obtained through the video stream data and the track tracking result. The phase switching direction corresponding to the phase switching direction code is spatially analyzed with the track of the unfinished passing vehicle to determine whether there is a possibility of spatial overlap. For example, the intersection of the vehicle track and the lane line in the new phase direction is calculated to determine whether a vehicle will enter the lane in the new phase direction after the phase switching. If there is a spatial overlap risk, the phase switching scheme needs to be adjusted or other measures need to be taken to avoid the conflict. By performing the conflict pre-check, potential conflict risks can be discovered in advance, and the safety of the signal lamp control can be improved.

[0187] Step S490: When the spatial overlap risk is detected, a forced waiting period identifier is embedded in the signal lamp control instruction, and the forced waiting period identifier prolongs the duration of the current phase state until the risk is resolved.

[0188] The forced waiting cycle identifier is an identifier for extending the duration of the current phase state, which is embedded into the traffic light control instruction when the spatial overlap risk is detected. By extending the duration of the current phase state, the unfinished traffic vehicles in the opposite lane have enough time to pass through the intersection, avoiding conflicts with vehicles in the new phase direction. The specific embedding is implemented, for example, as follows: when the spatial overlap risk is detected, the forced waiting cycle identifier is added to the traffic light control instruction. The forced waiting cycle identifier can be a binary code, such as 1111. At the same time, according to the trajectory and speed of the unfinished traffic vehicles in the opposite lane, the time that needs to be extended is calculated. The extended time is added to the duration identifier bit of the traffic light control instruction. For example, the original traffic light control instruction is 0000000100011110, after detecting the spatial overlap risk, the forced waiting cycle identifier 1111 is embedded, and the extended time (such as 10 seconds, converted to binary bit as 00001010) is added to the duration identifier bit, obtaining the new traffic light control instruction 0000000111110001111000001010. By embedding the forced waiting cycle identifier, the spatial overlap risk can be effectively avoided, and the safety and smoothness of traffic can be ensured.

[0189] Step S4100: encapsulate the verified traffic light control instruction into an instruction frame structure executable by the edge computing node, which contains a timestamp synchronization field and an instruction effective countdown field.

[0190] The instruction frame structure is a structure for encapsulating the traffic light control instruction, which contains a timestamp synchronization field and an instruction effective countdown field. The timestamp synchronization field is used to ensure the time synchronization between the edge computing node and the traffic light device, and the instruction effective countdown field is used to indicate the effective time of the traffic light control instruction.

[0191] The specific encapsulation can be implemented, for example, as follows: add the verified traffic light control instruction to the instruction frame structure. Add the timestamp synchronization field in the instruction frame structure, which can be the current timestamp, such as 10:00:00. At the same time, add the instruction effective countdown field, which can be calculated according to the duration in the traffic light control instruction and the current time. For example, the duration in the traffic light control instruction is 30 seconds, and the current time is 10:00:00, then the instruction effective countdown field is 10:00:30. Combine these fields into a complete instruction frame structure, such as [timestamp synchronization field: 10:00:00, traffic light control instruction: 0000000111110001111000001010, instruction effective countdown field: 10:00:30]. By encapsulating the traffic light control instruction into the instruction frame structure, the communication between the edge computing node and the traffic light device can be ensured to be accurate and effective, and the correct execution of the traffic light control instruction can be ensured.

[0192] Step S500: Adjust the signal light phase switching timing of the target intersection based on the signal light control instruction, so that the traffic participants meeting the traffic rule conflict condition obtain the priority at the signal light phase switching timing.

[0193] The signal light control instruction contains information such as phase switching direction and duration, and by executing the instruction, the signal light phase switching timing of the target intersection can be adjusted. The traffic participants meeting the traffic rule conflict condition are those traffic participants marked as triggering traffic rule conflicts in the behavior trigger identifier. By adjusting the signal light phase switching timing, these traffic participants obtain the priority at the appropriate time, thereby reducing traffic conflicts and congestion.

[0194] As an implementation, step S500, based on the signal light control instruction to adjust the signal light phase switching timing of the target intersection, can include the following steps S510-S560:

[0195] Step S510: Analyze the phase switching direction identifier and phase duration parameter in the signal light control instruction.

[0196] Analysis refers to the conversion of binary coded information in the signal light control instruction into specific phase switching direction and phase duration parameters. The phase switching direction identifier indicates the phase direction to which the signal light needs to switch, and the phase duration parameter indicates the duration of the phase state.

[0197] Specific analysis can be implemented, for example, according to the coding rules of the signal light control instruction, the binary bits in the instruction are segmented, and the phase switching direction identifier and phase duration parameter are extracted. For example, the signal light control instruction is 0000000111110001111000001010, according to the coding rule, the first 8 bits 00000001 represent the phase switching direction identifier, which is converted to the specific phase direction as east-west direction; the binary bits 11110001111000001010 behind represent the phase duration parameter, which is converted to decimal number as 40 seconds. By analyzing the signal light control instruction, the accurate phase switching direction and duration information can be obtained, which provides the basis for subsequent phase switching operation.

[0198] Step S520: Determine whether the next phase state is a reverse phase sequence according to the phase switching direction identifier.

[0199] Reverse phase sequence refers to a phase sequence opposite or conflicting with the current phase state, for example, the current phase state is south-north direction green light, and the next phase state is east-west direction green light, then the east-west direction green light phase sequence is a reverse phase sequence relative to the south-north direction green light phase sequence.

[0200] The specific determination can be implemented as follows: comparing the phase switching direction identifier with the current signal phase state, and determining whether the next phase state is the reverse phase sequence. For example, the current phase state is a green light in the south-north direction, and the phase switching direction identifier is a green light in the east-west direction. It is determined that the next phase state is the reverse phase sequence. By determining whether the next phase state is the reverse phase sequence, corresponding measures can be taken in advance to avoid phase conflicts and traffic confusion.

[0201] Step S530: When the reverse phase sequence is detected, start the phase conflict detection mechanism to verify whether there is an unfinished passing vehicle that conflicts with the reverse phase sequence.

[0202] The phase conflict detection mechanism is a mechanism for detecting whether the phase switching will cause a conflict. When the reverse phase sequence is detected, the mechanism is started to verify whether there is an unfinished passing vehicle that conflicts with the reverse phase sequence. The unfinished passing vehicle refers to a vehicle that has not passed through the intersection under the current phase state.

[0203] The specific detection can be implemented as follows: obtaining vehicle trajectory information of the current intersection through video stream data and trajectory tracking results. The passing direction of the reverse phase sequence is compared with the trajectory of the unfinished passing vehicle to determine whether there is a conflict. For example, the reverse phase sequence is a green light in the east-west direction, and it is detected whether there is an unfinished passing vehicle in the south-north direction that will conflict with the vehicle in the east-west direction when the green light in the east-west direction is on. If there is a conflict, it is marked as a phase conflict risk. By starting the phase conflict detection mechanism, potential phase conflict risks can be discovered in advance to provide a basis for subsequent processing.

[0204] Step S540: Calculate the expected emptying time of the conflict vehicle and compare the expected emptying time with the phase duration parameter.

[0205] The conflict vehicle refers to an unfinished passing vehicle that conflicts with the reverse phase sequence, and the expected emptying time refers to the time required for all conflict vehicles to pass through the intersection. Comparing the expected emptying time with the phase duration parameter can determine whether there is enough time for the conflict vehicle to pass through the intersection to avoid traffic conflicts caused by phase switching.

[0206] As an implementation manner, step S540, calculating the expected emptying time of the conflict vehicle and comparing the expected emptying time with the phase duration parameter, can include the following steps S541-S5411:

[0207] Step S541: Extract the position coordinate sequence in the historical trajectory segment of the conflict vehicle, and the position coordinate sequence contains vehicle center point coordinates of past consecutive N frames, N>1.

[0208] The historical trajectory segment refers to the motion trajectory of the conflict vehicle in the past period of time, and the position coordinate sequence is the coordinate sequence of the vehicle center point extracted from the historical trajectory segment, which contains the vehicle center point coordinates of the past continuous N frames. By extracting the position coordinate sequence, the historical motion information of the conflict vehicle can be obtained, providing a basis for subsequent motion trend prediction.

[0209] The specific extraction can be implemented, for example, as follows: from the video stream data and the trajectory tracking result, the historical trajectory segment of the conflict vehicle is obtained. Then, the vehicle center point coordinates of the past continuous N frames are extracted from the historical trajectory segment to form the position coordinate sequence. For example, N is 10, and the vehicle center point coordinates of the past 10 frames are extracted from the historical trajectory segment of the conflict vehicle to obtain the position coordinate sequence [(x1, y1), (x2, y2), …, (x10, y10)]. By extracting the position coordinate sequence, the historical position information of the conflict vehicle can be accurately recorded, providing data support for subsequent motion analysis.

[0210] Step S542: inputting the position coordinate sequence into the space-time coupled encoder to extract the vehicle motion trend feature through the cyclic convolution in the time dimension, and capturing the relative position relationship between the vehicle and the adjacent lane through the attention mechanism in the space dimension.

[0211] The space-time coupled encoder is an encoder for processing space-time data, which can extract features in both time and space dimensions. The cyclic convolution in the time dimension is a convolution operation for processing time series data, which can capture the time dependence and trend of vehicle motion. The attention mechanism in the space dimension is a mechanism for focusing on important parts of data, which can capture the relative position relationship between the vehicle and the adjacent lane.

[0212] The specific processing can be implemented, for example, as follows: the position coordinate sequence is input into the space-time coupled encoder. In the time dimension, the position coordinate sequence is processed using a cyclic convolution layer to extract the vehicle motion trend feature. For example, the position coordinate sequence is processed using a recurrent neural network such as long short-term memory (LSTM) or gated recurrent unit (GRU) to obtain the motion trend feature vector of the vehicle. In the space dimension, the relative position relationship between the vehicle and the adjacent lane is captured using the attention mechanism. The attention mechanism can calculate the attention weight of each position according to the position coordinate of the vehicle and the position information of the adjacent lane, and then perform weighted summation on the position information of the vehicle according to the attention weight to obtain the relative position relationship feature vector between the vehicle and the adjacent lane. Through the processing of the space-time coupled encoder, the motion trend feature of the vehicle and the relative position relationship feature with the adjacent lane can be extracted simultaneously, providing more comprehensive information for subsequent trajectory prediction.

[0213] Step S543: Feature cross fusion of the vehicle motion trend feature and the relative position relationship to generate a spatio-temporal coupling feature vector.

[0214] Feature cross fusion refers to combining and fusing the vehicle motion trend feature and the relative position relationship feature to generate a spatio-temporal coupling feature vector with better expressiveness. Through feature cross fusion, the motion trend of the vehicle and the relative position relationship information with the adjacent lane can be fully utilized to improve the accuracy of trajectory prediction.

[0215] Specific fusion can be implemented, for example, by splicing or weighted summation of the vehicle motion trend feature vector and the relative position relationship feature vector to generate a spatio-temporal coupling feature vector. For example, the vehicle motion trend feature vector and the relative position relationship feature vector are spliced in the channel dimension to obtain a feature vector with a larger dimension. Then, the spliced feature vector is processed through a fully connected layer to further fuse the feature information and generate a spatio-temporal coupling feature vector. Through feature cross fusion, different types of feature information can be integrated to provide richer feature representation for subsequent trajectory prediction.

[0216] Step S544: Input the spatio-temporal coupling feature vector into the trajectory prediction decoder to generate a prediction position coordinate sequence of M future frames through deconvolution operation, M≥1.

[0217] The trajectory prediction decoder is a decoder for predicting the future trajectory of the vehicle according to the input feature vector, and the deconvolution operation is an operation for recovering spatial information from the feature vector. Through deconvolution operation, a prediction position coordinate sequence of M future frames can be generated frame by frame.

[0218] Specific prediction can be implemented, for example, by inputting the spatio-temporal coupling feature vector into the trajectory prediction decoder. The trajectory prediction decoder can adopt a model such as deconvolution neural network (DeconvNet) or generative adversarial network (GAN). In the decoder, the spatio-temporal coupling feature vector is processed through deconvolution operation to generate a prediction position coordinate sequence of M future frames frame by frame. For example, M is 5, and through deconvolution operation, prediction position coordinates [(x1', y1'), (x2', y2'),..., (x5', y5')] of 5 future frames are generated in turn. Through the processing of the trajectory prediction decoder, the future motion trajectory of the conflict vehicle can be predicted to provide a basis for subsequent estimated clearance time calculation.

[0219] Step S545: Calculate the longitudinal distance between each frame position in the prediction position coordinate sequence and the target lane stop line to generate a distance change curve.

[0220] The target lane stop line refers to the stop line position of the lane that the conflict vehicle needs to reach, and the longitudinal distance refers to the distance between each frame position in the predicted position coordinate sequence and the target lane stop line in the longitudinal direction. The distance change curve is a curve composed of the longitudinal distances between each frame position and the target lane stop line, which can reflect the change of the distance between the conflict vehicle and the target lane stop line over time.

[0221] Specific calculations can be implemented, for example, for each predicted position coordinate in the predicted position coordinate sequence, the longitudinal distance between it and the target lane stop line is calculated. For example, the longitudinal coordinate of the target lane stop line is y0, and the predicted position coordinate is (xi, yi), then the longitudinal distance is |yi-y0|. The longitudinal distances of each frame are arranged in chronological order to generate the distance change curve. By calculating the distance change curve, the trend of the distance change between the conflict vehicle and the target lane stop line can be observed intuitively, providing a basis for subsequent instantaneous speed feature extraction and predicted clearance time calculation.

[0222] Step S546: Perform first-order derivative operation on the distance change curve to extract the instantaneous speed feature of the vehicle at each predicted frame.

[0223] The first-order derivative operation refers to the derivative of the distance change curve to obtain the slope of the curve. Through the first-order derivative operation, the instantaneous speed feature of the vehicle at each predicted frame can be extracted. The instantaneous speed feature represents the instantaneous motion speed of the vehicle at each predicted frame.

[0224] Specific operations can be implemented, for example, by performing discrete processing on the distance change curve to convert it into a set of discrete data points. Then, using numerical differentiation methods such as forward difference, backward difference, or central difference, etc., the first-order derivative operation is performed on the discrete data points. For example, using the central difference method, for the ith data point, its derivative is approximately (yi+1-yi-1) / (2Δt), where yi+1 and yi-1 are adjacent data points, and Δt is the time interval. The derivative of each data point is taken as the instantaneous speed feature of the vehicle at that predicted frame. Through the first-order derivative operation, the instantaneous speed feature of the vehicle at each predicted frame can be accurately extracted, providing key information for subsequent predicted clearance time calculation.

[0225] Step S547: Calculate the remaining time sequence required for the vehicle to reach the stop line according to the instantaneous speed feature and the current frame position coordinate.

[0226] The remaining time sequence refers to the time sequence required for the vehicle to reach the target lane stop line from the current frame position, which can be calculated according to the instantaneous speed feature and the current frame position coordinate.

[0227] The specific calculation can be implemented as follows: for the instantaneous speed feature and the current frame position coordinate of each prediction frame, the remaining distance of the vehicle to the target lane stop line is calculated. Then, the remaining distance is divided by the instantaneous speed feature to obtain the remaining time required for the vehicle to reach the stop line. The remaining time of each prediction frame is arranged in order to generate a remaining time sequence. For example, the current frame position coordinate is (xi, yi), the longitudinal coordinate of the target lane stop line is y0, and the instantaneous speed feature is vi. The remaining distance is |yi-y0|, and the remaining time is |yi-y0| / vi. By calculating the remaining time sequence, the time for the vehicle to reach the stop line can be predicted, which provides a basis for subsequent predicted clearance time calculation.

[0228] Step S548: Compare the minimum value in the remaining time sequence with the phase duration parameter, and mark that there is a risk of traffic conflict when the minimum value is less than the phase duration parameter.

[0229] The minimum value in the remaining time sequence represents the time required for the vehicle to reach the stop line the fastest. Comparing it with the phase duration parameter can determine whether there is enough time for the vehicle to pass through the intersection. When the minimum value is less than the phase duration parameter, it means that the vehicle may not be able to pass through the intersection within the current phase duration, and there is a risk of traffic conflict.

[0230] The specific comparison and marking can be implemented as follows: find the minimum value in the remaining time sequence. Compare the minimum value with the phase duration parameter. If the minimum value is less than the phase duration parameter, mark that there is a risk of traffic conflict; otherwise, mark that there is no risk of traffic conflict. For example, the remaining time sequence is [5, 6, 7, 8, 9], the minimum value is 5 seconds, and the phase duration parameter is 4 seconds. Mark that there is a risk of traffic conflict. Through comparison and marking, potential traffic conflict risks can be discovered in time, which provides a basis for subsequent risk level indication parameter generation and predicted clearance time adjustment.

[0231] Step S549: Associate map the traffic conflict risk mark with the switch direction encoding of the reverse phase sequence to generate a risk level indication parameter.

[0232] The association mapping means that the traffic conflict risk mark is corresponded with the switch direction encoding of the reverse phase sequence to generate a risk level indication parameter. The risk level indication parameter is used to represent the risk level of traffic conflict under different switch directions.

[0233] The specific mapping and generation may be implemented as follows: according to the traffic conflict risk label and the switching direction code of the reverse phase sequence, a mapping relationship is established. For example, the traffic conflict risk label is 1, indicating that there is a traffic conflict risk, and 0, indicating that there is no traffic conflict risk; the switching direction code of the reverse phase sequence is 01, indicating switching from the north-south direction to the east-west direction. The traffic conflict risk label and the switching direction code are combined into a risk level indication parameter, such as (01, 1), indicating that there is a traffic conflict risk when switching from the north-south direction to the east-west direction. By mapping and generating the risk level indication parameter, the traffic conflict risk in different switching directions can be clearly represented, providing a basis for subsequent calculation of the reference adjustment of the predicted clearance time.

[0234] Step S5410: Adjusting the calculation reference of the predicted clearance time based on the risk level indication parameter, and recalculating the remaining time sequence using the intermediate value of the predicted position coordinate sequence when the risk level exceeds the preset threshold.

[0235] The calculation reference of the predicted clearance time refers to the basic data and method used to calculate the predicted clearance time, and the risk level indication parameter represents the level of traffic conflict risk. When the risk level exceeds the preset threshold, it indicates that the traffic conflict risk is high, and the calculation reference of the predicted clearance time needs to be adjusted to more accurately calculate the predicted clearance time.

[0236] Specific adjustment may be implemented as follows: according to the risk level indication parameter, it is judged whether the risk level exceeds the preset threshold. If it exceeds the preset threshold, the remaining time sequence is recalculated using the intermediate value of the predicted position coordinate sequence. For example, the predicted position coordinate sequence is [(x1, y1), (x2, y2),..., (x5, y5)], and the intermediate value is (x3, y3). Based on the intermediate value, the remaining distance and the remaining time sequence of the vehicle to the stop line are recalculated. By adjusting the calculation reference of the predicted clearance time, the predicted clearance time can be more accurately calculated in the case of high traffic conflict risk, avoiding traffic conflicts caused by phase switching.

[0237] Step S5411: Updating the minimum value in the recalculated remaining time sequence as the final predicted clearance time, and writing the final predicted clearance time into the phase conflict detection result.

[0238] The final predicted clearance time refers to the time required for all conflict vehicles to pass through the intersection after adjustment and calculation, which is written into the phase conflict detection result to provide a basis for subsequent phase switching decision.

[0239] The specific updating and writing can be implemented as follows: finding the minimum value in the recalculated residual time sequence as the final estimated emptying time. The final estimated emptying time is added to the phase conflict detection result, which can be a record containing conflict vehicle information, risk level indication parameters, and the final estimated emptying time. For example, the phase conflict detection result is [(vehicle ID: 123, risk level indication parameter: (01, 1), final estimated emptying time: 8 seconds)]. By updating the final estimated emptying time and writing the phase conflict detection result, accurate time information can be provided for subsequent phase switching operations, ensuring the safety and smoothness of traffic.

[0240] Step S550: When the estimated emptying time is less than the phase duration parameter, perform the switching operation of the reverse phase sequence.

[0241] When the estimated emptying time is less than the phase duration parameter, it means that the conflict vehicle has enough time to pass through the intersection within the current phase duration, and at this time the switching operation of the reverse phase sequence can be performed to adjust the phase state of the signal light to meet the traffic demand of the traffic participants.

[0242] The specific switching operation can be implemented as follows: according to the phase switching direction identifier and the phase duration parameter in the signal light control instruction, control the signal light to perform phase switching. For example, the phase switching direction identifier is to switch from the north-south direction to the east-west direction, and the phase duration parameter is 30 seconds, then switch the signal light from the north-south direction green light to the east-west direction green light, and set the green light duration to 30 seconds. By performing the switching operation of the reverse phase sequence, the phase state of the signal light can be reasonably adjusted to improve the traffic efficiency.

[0243] Step S560: When the estimated emptying time is greater than or equal to the phase duration parameter, maintain the current phase state until the preset safe switching condition is met.

[0244] When the estimated emptying time is greater than or equal to the phase duration parameter, it means that the conflict vehicle cannot pass through the intersection within the current phase duration, and at this time the current phase state needs to be maintained until the preset safe switching condition is met. The preset safe switching condition can be that the conflict vehicle has passed through the intersection, the estimated emptying time is less than the phase duration parameter, etc.

[0245] The specific maintenance and waiting, for example, can be implemented as: continuing to maintain the current signal light phase state, while continuously monitoring the movement of the conflict vehicle and the estimated emptying time. When the preset safety switching condition is met, the phase switching operation is performed again. For example, the current phase state is the green light in the north-south direction, the estimated emptying time is 40 seconds, and the phase duration parameter is 30 seconds. The north-south direction green light state is maintained until all conflict vehicles pass through the intersection or the estimated emptying time is less than the new phase duration parameter. By maintaining the current phase state until the safety switching condition is met, traffic conflicts caused by phase switching can be avoided, and the safety and order of traffic can be ensured.

[0246] In summary, the edge computing signal light control method based on traffic participant behavior analysis comprehensively analyzes and accurately judges the behavior state of the traffic participant, generates reasonable signal light control instructions in combination with the phase state of the signal light, realizes intelligent adjustment of the signal light phase switching timing, and can effectively reduce traffic conflicts and congestion, and improve traffic efficiency and safety.

[0247] It should be noted that those skilled in the art can implement without obstacles on the basis of their own technical knowledge in the art during reading of the above-mentioned embodiments of the application, for example, when different dimensional variables are calculated, the general normalization or standardization means can be used to eliminate the dimensional difference and then perform subsequent operation. For example, for the scene not involved, the technical means disclosed in the application can be used to continue to adaptively extend, for example, in step S530, when the reverse phase sequence is detected, if the estimated emptying time of the conflict vehicle (step S540) is delayed compared with the phase duration parameter, a dynamic priority adjustment mechanism of real-time trajectory prediction can be introduced, in combination with the low delay characteristic of edge computing, the feedback cycle of prediction and decision is shortened. In step S350, the priority weight of the direction conflict identifier and the speed anomaly identifier only depends on the remaining time, a multi-factor weight allocation model can be introduced to dynamically adjust the priority by comprehensively considering the conflict type, traffic flow, historical accident data, etc. For the use of the model, adaptive selection can also be made according to the specific use scene, for example, the key point bidirectional matching in step S231 in the occlusion or dense scene, in order to overcome the trajectory jump (such as the effective data lost by step S232 of removing abnormal segments), the key point correlation can be enhanced by combining target detection (such as YOLO) and optical flow method, the graph neural network is introduced to optimize the trajectory continuity, etc.

[0248] Please refer to Figure 2 , Figure 2A structural diagram of a signal lamp control system provided by an embodiment of the present application is provided, for example, an edge computer system, which at least includes a processor 101, a communication interface 102 and a memory 103. Wherein the processor 101, the communication interface 102 and the memory 103 can be connected through a bus or other means. Wherein the processor 101 (or called Central Processing Unit, CPU) is the computing core and control core of the signal lamp control system, which can parse various instructions in the signal lamp control system and process various data of the signal lamp control system. The communication interface 102 can optionally include a standard wired interface, a wireless interface (such as WI-FI, a mobile communication interface, etc.), which can be used for transmitting and receiving data under the control of the processor 101; the communication interface 102 can also be used for internal data transmission and interaction of the signal lamp control system. The memory 103 is a memory device in the signal lamp control system, used for storing programs and data. It can be understood that the memory 103 here can include the built-in memory of the signal lamp control system, and of course can also include the expansion memory supported by the signal lamp control system. The memory 103 provides a storage space, which stores the operating system of the signal lamp control system, and the present application does not limit this.

[0249] In an embodiment, the processor 101 executes the edge computing signal lamp control method based on traffic participant behavior analysis provided by the above embodiment of the present application by running the computer program in the memory 103.

Claims

1. An edge computing signal light control method based on traffic participant behavior analysis, characterized in that, The method comprises: acquiring video stream data of a target intersection, the video stream data containing motion trajectories of at least one traffic participant and a phase state of a current signal light; performing frame-by-frame behavior state analysis on the video stream data by a space-time feature coding network to extract behavior state features of each traffic participant, wherein the behavior state features include a motion direction vector, a speed change sequence and a spatial position offset; inputting the behavior state features into a preset behavior matching model to generate a behavior trigger identifier of the traffic participant within a current time window, the behavior trigger identifier being used to indicate whether the traffic participant satisfies a preset traffic rule conflict condition; specifically comprising: performing direction angle calculation on the motion direction vector in the behavior state features and a set of allowed directions of travel corresponding to the phase state of the current signal light to generate a direction deviation angle sequence; comparing a value of each time window in the speed change sequence with a speed threshold interval associated with the phase state to generate a speed out-of-bound marker sequence; performing sliding window accumulation in the time dimension on the direction deviation angle sequence, and generating a direction conflict identifier when accumulated angles of three continuous windows exceed a preset deviation threshold; performing continuous positive event detection on the speed out-of-bound marker sequence, and generating a speed anomaly identifier when it is detected that the speed breaks through the same direction threshold twice in succession; inputting the direction conflict identifier and the speed anomaly identifier into a decision tree model associated with the phase state, calculating a priority weight of a conflict behavior according to a remaining duration of the phase state; dynamically fusing the direction conflict identifier and the speed anomaly identifier based on the priority weight to generate an initial behavior trigger probability; performing time sequence alignment on the initial behavior trigger probability and a trigger probability sequence of a historical time window to generate a corrected behavior trigger probability by an exponential smoothing algorithm; generating an effective behavior trigger identifier when the corrected behavior trigger probability exceeds a dynamic trigger threshold negatively related to a traffic flow density of a current intersection; performing logical constraint verification on the effective behavior trigger identifier and a remaining time of the phase state to eliminate trigger events with a remaining time less than a safety response threshold to generate a final behavior trigger identifier set, each element in the final behavior trigger identifier set containing a conflict type and a trigger timestamp; generating a signal light control instruction of an edge computing node according to a time sequence correlation between the behavior trigger identifier and the phase state of the current signal light; adjusting a signal light phase switching time sequence of the target intersection based on the signal light control instruction so that a traffic participant satisfying the traffic rule conflict condition obtains a priority in travel under the signal light phase switching time sequence. 2.The edge computing signal light control method based on traffic participant behavior analysis according to claim 1, wherein, The behavior state analysis on the video stream data by the space-time feature coding network to extract the behavior state features of each traffic participant comprises: constructing a three-dimensional space-time tensor of the video stream data, the three-dimensional space-time tensor including a time dimension, a spatial position dimension and a pixel feature dimension; performing space-time feature extraction on the three-dimensional space-time tensor by a convolutional long short-term memory network to obtain a motion feature map of each traffic participant; Tracking key points of the motion feature map to generate the motion direction vector and the spatial position offset; Differencing the motion feature map of the continuous frames through a sliding window to generate the speed change sequence; Normalizing and splicing the motion direction vector, the speed change sequence, and the spatial position offset to form the behavior state feature matrix. 3.The edge computing signal light control method based on traffic actor behavior analysis according to claim 2, wherein, The spatio-temporal feature extraction of the three-dimensional spatio-temporal tensor through the convolutional long short-term memory network to obtain the motion feature map of each traffic participant includes: Dividing the three-dimensional spatio-temporal tensor into continuous time step slices according to the time dimension, each time step slice containing the spatial position dimension and the pixel feature dimension of the current frame; Performing a three-dimensional convolution operation on each time step slice to generate a spatio-temporal convolution feature map of the current time step, the convolution kernel of the three-dimensional convolution operation spanning adjacent frames in the time dimension to capture motion continuity; Inputting the spatio-temporal convolution feature map into the forget gate structure of the convolutional long short-term memory network, generating a forget gate output through element-wise multiplication and activation function processing to control the retention ratio of the previous hidden state; Splicing the spatio-temporal convolution feature map and the forget gate output and inputting the result into the input gate structure, generating a candidate cell state of the current time step through convolution operation and activation function processing; Weighted superimposing the candidate cell state and the previous cell state adjusted by the forget gate to generate an updated current cell state; Inputting the current cell state into the output gate structure, generating a hidden state of the current time step through convolution operation and hyperbolic tangent activation function processing; Traversing all time step slices, performing cross-layer feature fusion on the hidden state of each time step along the channel dimension to generate a multi-scale motion feature map; Performing spatial pyramid pooling on the multi-scale motion feature map to extract local motion pattern features of different field of view ranges; Channel attention weighted fusion of the local motion pattern features and the hidden state of the last time step to generate the motion feature map of each traffic participant, each channel of the motion feature map corresponding to the activation strength of a preset motion pattern. 4.The edge computing signal light control method based on traffic participant behavior analysis according to claim 1, wherein, The signal light control instruction of the edge computing node is generated according to the time sequence correlation between the behavior trigger identifier and the phase state of the current signal light, including: Mapping and matching each behavior trigger identifier in the final behavior trigger identifier set with the remaining time of the phase state to generate a priority parameter set containing conflict types, trigger time stamps, and remaining times; According to the conflict type in the priority parameter set, activating the corresponding phase adjustment rule, the phase adjustment rule containing a direction conflict priority strategy and a speed anomaly degradation strategy; Spatial orientation aggregation of multiple direction conflict identifiers within the same time window through the direction conflict priority strategy to generate a phase switching direction candidate set; Based on the remaining time of the phase state and the trigger time stamp, calculating the duration of the direction conflict, and combining the occurrence frequency of the speed anomaly identifier in the speed anomaly degradation strategy to generate a phase adjustment parameter; inputting the phase switching direction candidate set and the phase adjustment parameter time sequence into a time sequence optimizer to generate a phase switching timing scheme satisfying minimum phase switching interval and maximum traffic efficiency constraints; generating a phase switching direction code according to a difference between a target phase direction in the phase switching timing scheme and a current phase state; splicing the phase switching direction code and a duration in the phase adjustment parameter by binary bits to form a signal lamp control instruction containing direction identification bits and duration identification bits; performing conflict pre-check before generating the signal lamp control instruction to verify whether the phase switching direction code has a spatial overlap risk with an unfinished traffic vehicle trajectory of an opposite lane; when detecting the spatial overlap risk, embedding a forced waiting period identifier in the signal lamp control instruction, the forced waiting period identifier prolonging a duration of the current phase state until the risk is resolved; packaging the verified signal lamp control instruction into an instruction frame structure executable by an edge computing node, the instruction frame structure containing a time stamp synchronization field and an instruction validity countdown field. 5.The edge computing signal light control method based on traffic actor behavior analysis according to claim 1, wherein, the signal lamp control instruction adjusting the signal lamp phase switching timing of the target intersection includes: parsing phase switching direction identification and phase duration parameters in the signal lamp control instruction; determining whether a next phase state is a reverse phase sequence according to the phase switching direction identification; when detecting the reverse phase sequence, starting a phase conflict detection mechanism to verify whether there is an unfinished traffic vehicle conflicting with the reverse phase sequence; calculating a predicted emptying time of the conflicting vehicle and comparing the predicted emptying time with the phase duration parameter; when the predicted emptying time is less than the phase duration parameter, performing a switching operation of the reverse phase sequence; when the predicted emptying time is greater than or equal to the phase duration parameter, maintaining the current phase state until a preset safe switching condition is met. 6.The edge computing signal light control method based on traffic actor behavior analysis according to claim 1, wherein, the training method of the space-time feature coding network includes: collecting sample video stream data of historical traffic scenes and behavior labeling traffic participants in the sample video stream data to generate behavior label sequences; building a hybrid architecture of a three-dimensional convolutional neural network and a recurrent neural network, the hybrid architecture containing a space-time feature extraction branch and a behavior classification branch; setting a multi-scale feature fusion module in the space-time feature extraction branch, the multi-scale feature fusion module being used for integrating motion features under different receptive fields; optimizing feature distinguishability of the space-time feature extraction branch by a contrastive learning loss function, the contrastive learning loss function promoting feature vectors of same behavior samples to gather in an embedding space; adopting a dynamic focal loss function in the behavior classification branch, the dynamic focal loss function automatically adjusting classification weights according to a class distribution of samples; taking an output of the optimized space-time feature extraction branch as an input feature of the preset behavior matching model.

7. The edge computing signal light control method based on traffic actor behavior analysis according to claim 6, characterized in that, the optimization process of the contrastive learning loss function includes: randomly selecting positive sample pairs and negative sample pairs from the sample video stream data, the positive sample pairs containing different instances of the same behavior category, and the negative sample pairs containing instances of different behavior categories; generating transformed views of the positive sample pairs through data augmentation, the transformed views including spatial cropping, temporal slicing, and color jittering; calculating cosine similarity between feature vectors of the positive sample pairs, and constructing a positive sample similarity distribution matrix; calculating cosine similarity between feature vectors of the negative sample pairs, and constructing a negative sample similarity distribution matrix; maximizing separation of the positive sample similarity distribution matrix and the negative sample similarity distribution matrix through temperature scaling cross-entropy loss function; adjusting convolution kernel parameters of the spatio-temporal feature extraction branch and gating weights of the recurrent neural network through back propagation. 8.The edge computing signal light control method based on traffic actor behavior analysis of claim 5, wherein, The calculating the estimated clearance time of the conflict vehicle and comparing the estimated clearance time with the phase duration parameter includes: extracting a sequence of position coordinates in a historical trajectory segment of the conflict vehicle, the sequence of position coordinates containing vehicle center point coordinates of past consecutive N frames, N>1; inputting the sequence of position coordinates into a spatio-temporal coupling encoder, extracting vehicle motion trend features through recurrent convolution in the time dimension, and capturing relative position relationships between the vehicle and adjacent lanes through attention mechanisms in the spatial dimension; cross-fusing the vehicle motion trend features and the relative position relationships to generate a spatio-temporal coupling feature vector; inputting the spatio-temporal coupling feature vector into a trajectory prediction decoder, and generating a predicted sequence of position coordinates of future M frames through deconvolution operation, M≥1; calculating longitudinal distances between each frame position in the predicted sequence of position coordinates and a target lane stop line to generate a distance change curve; performing first-order derivative operation on the distance change curve to extract instantaneous speed features of the vehicle in each predicted frame; calculating a sequence of remaining times required for the vehicle to reach the stop line based on the instantaneous speed features and the current frame position coordinates; comparing a minimum value in the sequence of remaining times with the phase duration parameter, and marking as existing risk of traffic conflict when the minimum value is less than the phase duration parameter; associating a traffic conflict risk mark with a switching direction code of the reverse phase sequence to generate a risk level indication parameter; adjusting a calculation benchmark of the estimated clearance time based on the risk level indication parameter, and recalculating the sequence of remaining times using a middle value of the predicted sequence of position coordinates when the risk level exceeds a preset threshold; updating the minimum value in the recalculated sequence of remaining times as a final estimated clearance time, and writing the final estimated clearance time into a phase conflict detection result.

9. A signal light control system, characterized in that: The method comprises: a memory in which a computer program is stored; a processor configured to load the computer program to implement the edge computing signal lamp control method based on behavior analysis of traffic participants according to any one of claims 1-8.

Images