Traffic early warning method and device based on space-time dynamic network

Abstract

The embodiment of the invention provides a traffic early warning method and device based on a space-time dynamic network. The method comprises the following steps: acquiring multi-source traffic data; registration of time dimension and space dimension is carried out on the multi-source traffic data, and feature coding is carried out to obtain coded data; performing normalization processing on the encoded data to obtain normalized visual data, point cloud data and communication data, and creating an explicit physical diagram according to the normalized visual data, point cloud data and communication data; according to the normalized visual data, the normalized point cloud data and the normalized communication data, potential information is mined, and an implicit semantic graph is created; according to the explicit physical graph and the implicit semantic graph, time sequence dependence features are extracted through space graph convolution, time convolution and causal expansion convolution, semantic features are obtained through space-time position coding and sparse self-attention mechanism optimization, feature fusion is carried out, and fusion features are obtained; and determining traffic early warning information according to the fusion features. The prediction precision can be improved through acquisition and processing of various data.

Description

Technical Field

[0001] This application relates to the technical field of traffic control systems, and in particular, to a traffic warning method and device based on a spatio-temporal dynamic network. Background Art

[0002] Currently, C-V2X (Cellular Vehicle-to-Everything), as a vehicle wireless communication technology, can achieve all-round communication between vehicles and the surrounding environment (including other vehicles, technical facilities, pedestrians, and networks), and has a wide range of applications in the field of vehicle communication technology. At the same time, the GNN (Graph Neural Network) based on the vehicle networking system can obtain vehicle and surrounding information through C-V2X, so as to predict traffic flow. However, in the actual use process, due to the interference of factors such as environmental interference and vehicle movement, the prediction accuracy of the vehicle networking system is often low. Summary of the Invention

[0003] The purpose of the embodiments of this application is to provide a traffic warning method and device based on a spatio-temporal dynamic network to solve the problem of low prediction accuracy in the vehicle networking system. The specific technical solutions are as follows: In the first aspect of the embodiments of this application, a traffic warning method based on a spatio-temporal dynamic network is first provided. The method includes: Obtain multi-source traffic data, where the multi-source traffic data includes vehicle-side data, roadside data, and network data; Register the multi-source traffic data in the time dimension and the space dimension to obtain registered traffic data; Perform feature encoding on the registered traffic data to obtain encoded data, where the encoded data includes visual data, point cloud data, and communication data; Perform normalization processing on the encoded data to obtain normalized visual data, point cloud data, and communication data; According to the normalized visual data, point cloud data, and communication data, create an explicit physical graph, where the explicit physical graph is used to represent communication quality and physical distance; according to the normalized visual data, point cloud data, and communication data, mine potential information, and create an implicit semantic graph, where the potential information includes at least one of potential vehicle direction change, potential braking, and potential collision information; According to the explicit physical graph and the implicit semantic graph, extract temporal dependence features through spatial graph convolution, temporal convolution, and causal dilation convolution, and optimize through spatio-temporal position encoding and sparse self-attention mechanism to obtain semantic features; Perform feature fusion on the temporal dependence features and the semantic features to obtain fused features; Determine traffic warning information according to the fused features, where the traffic warning information includes whether a collision occurs and / or whether the traffic flow is greater than a preset threshold.

[0004] In a possible implementation manner, the extracting of the temporal dependence features according to the explicit physical graph and the implicit semantic graph through spatial graph convolution, temporal convolution, and causal dilated convolution includes: According to the explicit physical graph and the implicit semantic graph, perform extraction and aggregation of image spatial features through spatial graph convolution to obtain spatial graph features; Perform temporal convolution on the extracted spatial graph features to obtain graph features that satisfy the temporal order; Perform causal dilated convolution on the obtained graph features that satisfy the temporal order to obtain the temporal dependence features that satisfy the causality law.

[0005] In a possible implementation manner, the obtaining of the semantic features through spatio-temporal position encoding and sparse self-attention mechanism optimization includes: According to the explicit physical graph and the implicit semantic graph, perform coordinate normalization to obtain coordinate-normalized features; Perform sine time encoding on the coordinate-normalized features to obtain sine time-encoded features; Divide the sine time-encoded features into multiple neighborhood grids; for each node in the grid, perform feature calculation according to the node and its multiple adjacent local nodes to obtain local window features; according to the calculated local window features, perform extraction and fusion of multi-granularity features through multi-level memory nodes to obtain the semantic features.

[0006] In a possible implementation manner, the extracting of the temporal dependence features according to the explicit physical graph and the implicit semantic graph through spatial graph convolution, temporal convolution, and causal dilated convolution, and the obtaining of the semantic features through spatio-temporal position encoding and sparse self-attention mechanism optimization includes: According to the explicit physical graph and the implicit semantic graph, extract the temporal dependence features through spatial graph convolution, temporal convolution, and causal dilated convolution, and obtain the semantic features through spatio-temporal position encoding and sparse self-attention mechanism optimization; Extract a first image feature through a local spatio-temporal model and the explicit physical graph and the implicit semantic graph; extract a second image feature through a global semantic model and the explicit physical graph and the implicit semantic graph; Through cross-attention, using the local spatio-temporal model, calculate the temporal dependence features according to the first image feature and the second image feature; through cross-attention, using the global semantic model, calculate the semantic features according to the first image feature and the second image feature.

[0007] In a possible implementation manner, after determining the traffic warning information according to the fusion feature, the traffic warning method based on the spatio-temporal dynamic network further includes: According to the traffic warning information, calculate the contribution degrees of the attention heads in the global semantic model; According to the calculated contribution degrees, prune the corresponding convolutional channels of the attention heads whose corresponding contribution degrees are lower than the preset threshold.

[0008] In a second aspect of the embodiments of the present application, a traffic warning device based on a spatio-temporal dynamic network is provided. The device includes: A data acquisition module, configured to acquire multi-source traffic data, where the multi-source traffic data includes vehicle-side data, roadside data, and network data; A data registration module, configured to perform registration on the multi-source traffic data in the time dimension and the space dimension to obtain registered traffic data; A data encoding module, configured to perform feature encoding on the registered traffic data to obtain encoded data, where the encoded data includes visual data, point cloud data, and communication data; A normalization module, configured to perform normalization processing on the encoded data to obtain normalized visual data, point cloud data, and communication data; A graph creation module, configured to create an explicit physical graph according to the normalized visual data, point cloud data, and communication data, where the explicit physical graph is used to represent communication quality and physical distance; mine potential information according to the normalized visual data, point cloud data, and communication data, and create an implicit semantic graph, where the potential information includes at least one of potential vehicle direction change, or potential braking and potential collision information; A feature extraction module, configured to extract temporal dependence features according to the explicit physical graph and the implicit semantic graph through spatial graph convolution, temporal convolution, and causal dilated convolution, and optimize through spatio-temporal position encoding and sparse self-attention mechanism to obtain semantic features; A feature fusion module, configured to perform feature fusion on the temporal dependence features and the semantic features to obtain a fusion feature; A warning determination module, configured to determine traffic warning information according to the fusion feature, where the traffic warning information includes whether a collision occurs, and / or whether the traffic flow is greater than a preset threshold.

[0009] In a possible implementation, the feature extraction module includes: A graph feature acquisition sub-module, configured to extract and aggregate image spatial features through spatial graph convolution based on the explicit physical graph and the implicit semantic graph, so as to obtain spatial graph features; A temporal convolution sub-module, configured to perform temporal convolution based on the extracted spatial graph features to obtain graph features that satisfy the temporal order; A causal dilated convolution sub-module, configured to perform causal dilated convolution based on the obtained graph features that satisfy the temporal order to obtain the temporal dependence features that satisfy the causality law.

[0010] In a possible implementation, the feature extraction module includes: A coordinate normalization sub-module, configured to perform coordinate normalization based on the explicit physical graph and the implicit semantic graph to obtain coordinate-normalized features; A temporal encoding sub-module, configured to perform sinusoidal temporal encoding on the coordinate-normalized features to obtain sinusoidally temporally encoded features; A feature extraction sub-module, configured to divide the sinusoidally temporally encoded features into multiple neighborhood grids; for each node in the grid, perform feature calculation based on the node and its multiple adjacent local nodes to obtain local window features; based on the calculated local window features, perform extraction and fusion of features with multiple granularities through multi-level memory nodes to obtain the semantic features.

[0011] In a possible implementation, the feature extraction module is specifically configured to extract temporal dependence features based on the explicit physical graph and the implicit semantic graph through spatial graph convolution, temporal convolution, and causal dilated convolution, and optimize through spatio-temporal position encoding and sparse self-attention mechanism to obtain semantic features; extract first image features through a local spatio-temporal model and the explicit physical graph and the implicit semantic graph; extract second image features through a global semantic model and the explicit physical graph and the implicit semantic graph; calculate the temporal dependence features through cross-attention, using the local spatio-temporal model, based on the first image features and the second image features; calculate the semantic features through cross-attention, using the global semantic model, based on the first image features and the second image features.

[0012] In a possible implementation, the traffic warning device based on the spatio-temporal dynamic network further includes: A pruning module, configured to calculate the contribution degree of each attention head in the global semantic model according to the traffic warning information; according to the calculated contribution degree, prune the corresponding convolution channels of the attention heads whose corresponding contribution degrees are lower than a preset threshold.

[0013] In another aspect of the embodiments of the present application, there is also provided an electronic device, including: a memory for storing a computer program; a processor, configured to implement the traffic warning method based on the spatio-temporal dynamic network according to any one of the above when executing the program stored on the memory.

[0014] In another aspect of the embodiments of the present application, there is also provided a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the traffic warning method based on the spatio-temporal dynamic network according to any one of the above is implemented.

[0015] In another aspect of the embodiments of the present application, there is also provided a computer program product containing instructions, which when running on a computer, causes the computer to execute the traffic warning method based on the spatio-temporal dynamic network according to any one of the above.

[0016] Advantageous effects of the embodiments of the present application: A traffic warning method and device based on a spatio-temporal dynamic network provided by an embodiment of the present application, the method includes: obtaining multi-source traffic data, where the multi-source traffic data includes vehicle-side data, roadside data, and network data; registering the multi-source traffic data in the time dimension and the space dimension to obtain registered traffic data; performing feature encoding on the registered traffic data to obtain encoded data, where the encoded data includes visual data, point cloud data, and communication data; performing normalization processing on the encoded data to obtain normalized visual data, point cloud data, and communication data; creating an explicit physical map according to the normalized visual data, point cloud data, and communication data, where the explicit physical map is used to characterize communication quality and physical distance; mining potential information according to the normalized visual data, point cloud data, and communication data, and creating an implicit semantic map, where the potential information includes at least one of potential vehicle direction change, or potential braking and potential collision information; extracting temporal dependence features according to the explicit physical map and the implicit semantic map through spatial graph convolution, temporal convolution, and causal dilated convolution, and optimizing through spatio-temporal position encoding and sparse self-attention mechanism to obtain semantic features; performing feature fusion on the temporal dependence features and the semantic features to obtain fused features; determining traffic warning information according to the fused features, where the traffic warning information includes whether a collision occurs, and / or whether the traffic flow is greater than a preset threshold. Through the solution of the embodiment of the present application, by obtaining multi-source traffic data and creating a dynamic spatio-temporal map including an explicit physical map and an implicit semantic map according to the multi-source data, the temporal dependence features and semantic features can be extracted and fused according to the dynamic spatio-temporal map of the explicit physical map and the implicit semantic map to obtain fused features, and then traffic warning information can be determined according to the fused features, so as to improve the prediction accuracy by obtaining and processing multiple types of data.

[0017] Of course, it is not necessary for any product or method implementing the present application to achieve all the above-mentioned advantages simultaneously. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application, and those of ordinary skill in the art can also obtain other embodiments based on these drawings.

[0019] Figure 1 It is a schematic flowchart of a traffic warning method based on a spatio-temporal dynamic network provided by an embodiment of the present application; Figure 2a It is another schematic flowchart of a traffic warning method based on a spatio-temporal dynamic network provided by an embodiment of the present application; Figure 2b It is a schematic flowchart of a process for creating a dynamic representation modeling flowchart provided by an embodiment of the present application; Figure 2c It is a schematic flowchart of a process for dynamic representation modeling provided by an embodiment of the present application; Figure 3 It is a schematic flowchart of a process for obtaining temporal dependence features provided by an embodiment of the present application; Figure 4 It is a schematic flowchart of a process for obtaining semantic features provided by an embodiment of the present application; Figure 5 It is a schematic structural diagram of a traffic warning device based on a spatio-temporal dynamic network provided by an embodiment of the present application; Figure 6 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0020] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art based on the present application belong to the scope of protection of the present application.

[0021] In the first aspect of the embodiments of the present application, first, a traffic warning method based on a spatio-temporal dynamic network is provided. Refer to Figure 1 , Figure 1 It is a schematic flowchart of a traffic warning method based on a spatio-temporal dynamic network provided by an embodiment of the present application. The method includes: Step S11, obtaining multi-source traffic data, where the multi-source traffic data includes vehicle-side data, road-side data, and network data; Step S12, registering the multi-source traffic data in the time dimension and the space dimension to obtain registered traffic data; Step S13, performing feature encoding on the registered traffic data to obtain encoded data, where the encoded data includes visual data, point cloud data, and communication data; Step S14, performing normalization processing on the encoded data to obtain normalized visual data, point cloud data, and communication data; Step S15: Create an explicit physical map based on the normalized visual data, point cloud data, and communication data, where the explicit physical map is used to characterize communication quality and physical distance; mine potential information based on the normalized visual data, point cloud data, and communication data, and create an implicit semantic map, where the potential information includes at least one of potential vehicle direction change, potential braking, and potential collision information. Step S16: Extract temporal dependence features based on the explicit physical map and the implicit semantic map through spatial graph convolution, temporal convolution, and causal dilated convolution, and optimize them through spatio-temporal position encoding and sparse self-attention mechanism to obtain semantic features. Step S17: Perform feature fusion on the temporal dependence features and the semantic features to obtain fused features. Step S18: Determine traffic warning information based on the fused features, where the traffic warning information includes whether a collision occurs and / or whether the traffic flow is greater than a preset threshold.

[0022] Corresponding to the above step S11, where the multi-source traffic data includes vehicle-side data, roadside data, and network data. When obtaining the multi-source traffic data, refer to Figure 2a , the multi-source traffic data may include: obtaining on-vehicle sensors, roadside units, and cellular network data. Specifically, the vehicle-side data may include information such as the vehicle's speed, lane, position, as well as the vehicle's visual data and point cloud data; the roadside data may include information such as the road speed limit, road surface conditions, such as whether it is slippery, etc.; the network data may include information such as cellular network, mobile network, C-V2X (Cellular Vehicle-to-Everything, vehicle networking), etc.

[0023] It should be noted that the method of the embodiment of the present application can be implemented through a network model, and this network model can be deployed in an intelligent terminal device on the roadside. Through this intelligent terminal device, the road traffic conditions and warnings can be carried out.

[0024] Corresponding to the above step S12, when registering the multi-source traffic data in the time dimension and the space dimension, it is mainly to perform spatio-temporal alignment on the multi-source traffic data, that is, time synchronization and space registration. Since the time of data from different sources may vary, through time synchronization, the data from different sources can be unified to the same time dimension, eliminating the time deviation between multiple devices / sensors, and ensuring that data collection or operations are based on a unified time benchmark. When performing space registration, the data from different sources can be unified to the same space coordinate system, solving the problem of coordinate deviation of different sensors, and ensuring that the data corresponds precisely in the unified space coordinate system, thus facilitating subsequent processing. Specifically, calculations can be performed through methods such as cubic spline interpolation.

[0025] Corresponding to the above step S13, wherein the encoded data includes visual data, point cloud data, and communication data. Feature encoding of the registered traffic data can achieve classification of different data types, and the registered traffic data can be classified into visual data, point cloud data, and communication data. Specifically, it can be encoded and processed by methods such as label encoding, one-hot encoding, frequency encoding, and target encoding.

[0026] Corresponding to the above step S14, perform normalization processing on the encoded data. Through normalization processing, the dimensional difference between different features can be eliminated, and the model training efficiency and effect can be improved. Specifically, through normalization, all features can be in a similar numerical range (such as [0,1] or [-1,1]), ensuring the balanced contribution of features.

[0027] Corresponding to the above step S15, this step mainly constructs a dynamic graph, which includes an explicit physical graph and an implicit semantic graph. According to the normalized visual data, point cloud data, and communication data, an explicit physical graph and an implicit semantic graph can be created. Among them, the explicit physical graph is used to represent communication quality and physical distance. Among them, the potential information includes at least one of the potential direction change of the vehicle, or potential braking and potential collision information. The explicit graph generates an adjacency matrix and edge weights based on communication quality and physical distance. Specifically, the communication quality and the physical distance between vehicles can be determined through visual data, point cloud data, and communication data, and an explicit physical graph can be created according to the preset edge weights. According to the normalized visual data, point cloud data, and communication data, potential information can be mined. Potential information can be predicted by LSTM (Long Short-Term Memory, a special recurrent neural network) and potential relationships can be mined by GAT (Graph Attention Network). The potential information includes at least one of the potential direction change of the vehicle, or potential braking and potential collision information, such as whether the vehicle will change lanes, whether the vehicle will brake, and whether there is a possibility of collision. In the actual use process, a dual-graph coupling process with spatio-temporal consistency constraints can also be performed on the dynamic graph including the explicit physical graph and the implicit semantic graph. In this application, aiming at the high dynamicity of vehicles, roadside devices, and communication links in the vehicle network, the limitations of traditional static graph models are broken through. Through the dynamic spatio-temporal graph generation and adaptive update mechanism, network topology changes (such as vehicle movement, communication link fluctuation) are captured in real time, and the joint representation ability of traffic state and communication quality is improved. See Figure 2b, the construction process of the dynamic spatio-temporal graph may include: after obtaining multi-source data, data preprocessing is performed. The specific preprocessing process may include: spatio-temporal alignment, then interpolation synchronization, followed by multi-modal feature encoding. Then, for the encoded features, visual features: CNN (Convolutional Neural Networks), point cloud features: 3D (three-dimensional) convolution, and communication features: GNN (Graph Neural Network) are used to obtain features, and then feature normalization is performed. Finally, the dynamic spatio-temporal graph is constructed based on the normalized features. The specific construction process includes: display graph construction, dynamic adjacency matrix generation, edge weight calculation, real-time topology update to obtain the processing result of the first branch, and through implicit semantic graph construction, vehicle intention prediction, semantic association modeling to perform potential relationship extraction to obtain the processing result of the second path. Then, double-graph coupling and spatio-temporal consistency constraint are performed based on the processing results of the two paths. Finally, feature extraction and fusion are carried out.

[0028] Corresponding to the above step S16, when performing spatial graph convolution, temporal convolution, and causal dilated convolution based on the explicit physical graph and the implicit semantic graph, spatial convolution can update the node representation by aggregating the information of nodes and their neighbors, thereby capturing the spatial topological relationship; causal dilated convolution can combine the composite operation of causal constraints and dilated convolution, aiming to efficiently model long-term temporal dependencies. Specifically, the methods of spatial graph convolution, temporal convolution, and causal dilated convolution can refer to the prior art. Through spatio-temporal position encoding and sparse self-attention mechanism optimization, when obtaining semantic features, spatio-temporal position encoding is used to explicitly model the spatial topological relationship and the position information of the time series in the model, and sparse self-attention improves the model efficiency and alleviates the memory bottleneck in long sequence modeling by reducing the scope and complexity of attention calculation. In one example, the method of this step can be implemented through the STGNN (Spatio-Temporal Graph Neural Network)-Transformer (a deep learning model architecture) collaborative framework. Among them, the STGNN branch (local spatio-temporal model) performs spatial graph convolution. Specifically, it can aggregate the features of neighboring nodes by improving GATv2 (Graph Attention Networks v2, an improved network of graph attention networks), introduce edge attributes such as relative position and communication delay for spatial convolution; then perform temporal convolution through a temporal convolution network. Specifically, it can perform temporal convolution through TCN (Temporal Convolutional Network); finally, perform causal dilated convolution. Specifically, it can process the historical 5 frames of data through Dilation=2 (a dilated convolution network with a dilation factor of 2) to extract temporal dependence features. The semantic features can be extracted through the Transformer branch, that is, the global semantic model. Specifically, spatio-temporal position encoding can be performed, such as UTM (a coordinate system including longitude zone, latitude zone, distance east, and distance north) coordinate normalization and sine time encoding, and then sparse self-attention optimization can be performed, such as combining the local window (8-neighborhood) with the global memory node. Among them, spatio-temporal position encoding can encode the position information of time and space into the model when processing spatio-temporal data. The sparse self-attention mechanism optimization can reduce the computational amount in the attention mechanism, for example, by restricting the number of key-value pairs that each query focuses on, thereby reducing the complexity of the model and the consumption of computing resources. Through the deep fusion of STGNN and Transformer, the present invention proposes spatio-temporal cross-attention fusion, dynamically fuses the spatial adjacency matrix and the temporal dependence weight, realizes the unified representation of multi-source network states, solves the core bottleneck of traditional models in spatio-temporal dependence modeling, and provides a theoretical breakthrough and technical tool for multi-source fusion in the vehicle network.Moreover, the hybrid architecture of STGNN and Transformer can achieve multi-granularity modeling of local spatio-temporal patterns (such as vehicle following and sudden lane changes) and global long-term dependencies (such as traffic flow periodicity and communication interference propagation), overcoming the defect of insufficient spatio-temporal correlation modeling of a single model.

[0029] For the STGNN-Transformer network, a cross-attention module can be designed, such as a gate control algorithm, so that features of different modalities can complement and enhance each other. Image features and point cloud features can interact through the attention mechanism to capture their spatio-temporal correlations. The acquisition of features between the two branches is realized. In a possible implementation manner, according to the explicit physical graph and the implicit semantic graph, temporal dependence features are extracted through spatial graph convolution, temporal convolution, and causal dilated convolution, and are optimized through spatio-temporal position encoding and a sparse self-attention mechanism to obtain semantic features, including: according to the explicit physical graph and the implicit semantic graph, temporal dependence features are extracted through spatial graph convolution, temporal convolution, and causal dilated convolution, and are optimized through spatio-temporal position encoding and a sparse self-attention mechanism to obtain semantic features; first image features are extracted through a local spatio-temporal model and the explicit physical graph and the implicit semantic graph; second image features are extracted through a global semantic model and the explicit physical graph and the implicit semantic graph; through cross-attention, using the local spatio-temporal model, the temporal dependence features are calculated according to the first image features and the second image features; through cross-attention, using the global semantic model, the semantic features are calculated according to the first image features and the second image features.

[0030] See Figure 2c , the dynamic representation modeling process may include: multi-source data input, preprocessing and feature extraction, and then dynamic graph construction; then local spatio-temporal features are obtained through the STGNN branch, and global friendship features are obtained through the Transformer branch; then through spatio-temporal cross-attention, dynamic representation fusion is performed to achieve spatio-temporal cross-attention fusion, and finally fused feature output is performed.

[0031] Corresponding to the above step S17, for the temporal dependence features and the semantic features, feature fusion can be performed through feature fusion methods such as weighted summation. Specifically, reference can also be made to the prior art for feature fusion through other feature fusion methods. In this application, heterogeneous data such as vehicle sensor data (position, speed), C-V2X communication status (channel quality, delay), and environmental information (road conditions, weather) are uniformly encoded as spatio-temporal graph node and edge features, and cross-modal feature alignment and semantic enhancement are realized through an adaptive graph attention mechanism, which can improve the perception robustness of complex scenarios.

[0032] Corresponding to the above step S18, traffic warning information is determined according to the fusion features, where the traffic warning information includes whether a collision occurs and / or whether the traffic flow is greater than a preset threshold. Specifically, the traffic flow can be predicted through the fusion features. When the traffic flow is greater than the preset threshold, a warning message is sent, or the driving trajectory of the vehicle is predicted, so that when it is determined that a collision may occur, a warning message is sent.

[0033] In one example, see Figure 2a , including: obtaining in-vehicle sensors, roadside units, and cellular network data to obtain multi-source heterogeneous data as data input; then, after spatio-temporal alignment, data cleaning, and feature encoding processing, it is input into the edge and processing layer for processing; then, according to the processing results, a dynamic graph is constructed. Through node definition: vehicle / roadside unit, edge definition: spatio-temporal relationship, dynamic update mechanism, and real-time topology adjustment, a dynamic graph is constructed; then, according to the constructed dynamic graph, feature extraction is performed through the STGNN module and the Transformer module. Then, the extracted features are fused through the feature fusion core, and after the fusion result is compressed by the model, it is processed by the edge large model. An adaptive learning and dynamic topology update mechanism are also introduced during the processing. Then, the processing result is input into the edge-cloud collaborative optimization for model update and resource scheduling; and through the results of traffic flow prediction, collision warning, and path planning, it is output through intelligent decision-making.

[0034] It can be seen that through the solution of the embodiment of the present application, multi-source traffic data can be obtained, and a dynamic spatio-temporal graph including an explicit physical graph and an implicit semantic graph can be created according to the multi-source data. Thus, temporal dependence features and semantic features are extracted and fused according to the dynamic spatio-temporal graph of the explicit physical graph and the implicit semantic graph to obtain fusion features, so as to determine traffic warning information according to the fusion features, and achieve improving the prediction accuracy through the acquisition and processing of various data.

[0035] In a possible implementation manner, see Figure 3 , the extraction of temporal dependence features according to the explicit physical graph and the implicit semantic graph through spatial graph convolution, temporal convolution, and causal dilated convolution includes: Step 31, according to the explicit physical graph and the implicit semantic graph, through spatial graph convolution, extract and aggregate image spatial features to obtain spatial graph features; Step S32, according to the extracted spatial graph features, perform temporal convolution to obtain graph features that satisfy the time order; Step S33, according to the obtained graph features that satisfy the time order, perform causal dilated convolution to obtain the temporal dependence features that satisfy the causality law.

[0036] Among them, spatial graph convolution can model spatial relationships (such as traffic road network nodes, sensor network topologies) through graph structures, define the interaction weights between nodes using the adjacency matrix, and extract spatial features by combining graph neural networks. Specifically, it can include graph construction based on topology, such as mapping physical connections (such as roads, power lines) to the edges of the graph, and aggregating neighborhood information through graph convolutional layers; then performing multi-modal feature fusion, such as integrating node attributes (such as traffic flow, meteorological data) and topological relationships, and dynamically adjusting neighborhood weights using the attention mechanism. Temporal convolution can slide a one-dimensional convolutional kernel along the time axis to extract dynamic patterns in the sequence, support parallel computing, and avoid the problem of gradient disappearance. Specifically, it can include causal constraints, such as ensuring that the output at the current moment only depends on historical inputs through zero-padding to avoid leakage of future information, and dilated convolution, such as inserting holes between the elements of the convolutional kernel to exponentially expand the receptive field. Finally, residual connections are made, such as introducing a residual structure when stacking multiple layers of dilated convolution to alleviate the problem of deep network degradation. Causal dilated convolution can combine causal constraints and dilated convolution to efficiently model long-range dependencies while ensuring temporal causality. Specifically, it can be achieved through a staged dilation strategy, such as exponentially increasing the Dilation value layer by layer (such as 1, 2, 4, 8) to cover features at different time scales, and then performing dynamic kernel selection, such as adaptively adjusting the convolutional kernel weights through a gating mechanism to optimize the ability to capture different time patterns.

[0037] In a possible implementation, referring to Figure 4 , the semantic features obtained by optimizing through spatio-temporal position encoding and sparse self-attention mechanism include: Step 41: Perform coordinate normalization on the explicit physical graph and the implicit semantic graph to obtain normalized coordinate features; Step 42: Perform sinusoidal time encoding on the normalized coordinate features to obtain sinusoidal time-encoded features; Step 43: Divide the sinusoidal time-encoded features into multiple neighborhood grids; for each node in the grid, calculate features based on the node and its multiple adjacent local nodes to obtain local window features; based on the calculated local window features, extract and fuse features of multiple granularities through multi-level memory nodes to obtain the semantic features.

[0038] Among them, the spatio-temporal position encoding can include coordinate normalization and sinusoidal time encoding. Coordinate normalization can unify the features into the same coordinate system. Specifically, UTM coordinate normalization can be performed, which specifically includes: regional scaling, which can divide geographical regions and independently perform zoom normalization on the coordinates within each region to eliminate cross-regional scale differences, and dynamic parameter update, which periodically updates the zoom values for real-time data streams to adapt to changes in the coordinate range. The sinusoidal time encoding can include frequency function embedding, which maps the time step t to a multi-dimensional vector and captures the periodic features of the time series through a combination of sine / cosine functions with different frequencies, and time scale expansion, which generates a composite encoding by combining multiple time granularities such as hours, days, and weeks to enhance the periodic modeling ability.

[0039] The optimization of the sparse self-attention mechanism can include probabilistic sparse screening, structured sparse design, and dynamic sparse patterns. Among them, probabilistic sparse screening can reduce the computational complexity by only retaining the key-value pairs that have the greatest impact on the current query through probabilistic methods (such as maximum entropy screening). The structured sparse design can limit the global attention range to a local sliding window (such as a fixed neighborhood or block), only calculate the similarity of the elements within the window, and then divide the long sequence into sub-blocks, perform inter-block interaction at the coarse-grained layer, and perform intra-block refinement at the fine-grained layer to balance local and global dependencies. That is, the features of the sinusoidal time encoding are divided into multiple neighborhood grids; for each node in the grid, feature calculations are performed based on the node and its multiple adjacent local nodes to obtain local window features. The dynamic sparse pattern can dynamically generate a sparse pattern (such as locality-sensitive hashing) based on the similarity between the query and the key, adaptively focus on the highly relevant regions, and then combine the global sparse head (capturing key features) with the local dense head (retaining details) to balance efficiency and accuracy. Thus, not only can the computational complexity be reduced and the computational efficiency be improved, but also the computational accuracy can be taken into account, and multi-granularity feature extraction and fusion are performed through multi-level memory nodes to obtain the semantic features.

[0040] In a possible implementation manner, after determining the traffic warning information according to the fusion features, refer to Figure 4 , the traffic warning method based on the spatio-temporal dynamic network further includes: Step S41, calculating the contribution degree of each attention head in the global semantic model according to the traffic warning information; Step S42, pruning the corresponding convolutional channels for the attention heads whose calculated contribution degree is lower than the preset threshold.

[0041] By calculating the contribution degrees of the attention heads in the global semantic model, for the attention heads with corresponding contribution degrees lower than the preset threshold, pruning is performed on the corresponding convolutional channels. Specifically, the attention heads with low contribution degrees in the Transformer in the above embodiments can be removed, and redundant convolutional channels can be trimmed. Specifically, the determination and pruning of attention heads and channels can be performed through channel importance ranking and a greedy algorithm. In the actual use process, hardware adaptation and acceleration, dynamic offloading strategies, and protocol adaptation can also be performed to achieve a closed-loop of "training - compression - deployment - monitoring". At the same time, incremental learning is introduced to regularly update model parameters for new scenario data (such as road construction, sudden weather). Through the embodiments of the present application, lightweight deployment and optimization can be achieved. Through model compression and hardware adaptation optimization, the model is marginalized to adapt to the architecture of the edge server and meet the real-time requirements.

[0042] The inventors' research found that through the solution of the embodiments of the present application, by combining dynamic spatio-temporal graph modeling, spatio-temporal cross-attention mechanism, and lightweight edge deployment, the problems of "spatio-temporal fragmentation, modal conflict, and resource inefficiency" in dynamic network representation are solved, providing a feasible technical paradigm for the deep integration of intelligent transportation systems and communications. The core technical effects are specifically as follows: 1. Improvement in dynamic spatio-temporal modeling ability. Dynamic graph adaptive update: By adjusting the graph topology (node connection relationship and edge weights) in real time, the vehicle movement trajectory prediction error is reduced by 30% (compared with the traditional solution), especially showing significant advantages in high-speed (>80 km / h) and high-density (>100 vehicles / km²) scenarios. Spatio-temporal coupled feature extraction: By combining local spatio-temporal convolution and the global attention mechanism of the Transformer, the traffic flow prediction accuracy is improved by 25%, and the communication link quality prediction error ≤ 5%.

[0043] 2. Optimization of multi-modal data fusion. Cross-modal feature alignment: Based on the graph attention mechanism, the semantic alignment efficiency of multi-source heterogeneous data (sensors, communications, environment) is improved by 40%, reducing feature conflicts caused by sampling frequency differences. Enhanced robustness: In complex environments such as rain, snow, and tunnel occlusion, the accuracy fluctuation range of collaborative perception tasks is reduced from ±15% in traditional methods to ±5%.

[0044] 3. Real-time performance and resource efficiency. Low-latency inference: Through model lightweighting (pruning + quantization), the edge-side inference latency ≤ 10 ms, supporting high-throughput processing of 1000 nodes / second. Communication resource savings: The dynamic channel allocation strategy reduces redundant resource occupancy, the spectrum utilization rate is increased by 20%, and the base station energy consumption is reduced by 15%.

[0045] In the second aspect of the embodiments of the present application, a traffic warning device based on a spatio-temporal dynamic network is provided. See Figure 5 , the device includes: The data acquisition module 501 is used to acquire multi-source traffic data, where the multi-source traffic data includes vehicle-side data, roadside data, and network data; The data registration module 502 is used to register the multi-source traffic data in the time dimension and the space dimension to obtain registered traffic data; The data encoding module 503 is used to perform feature encoding on the registered traffic data to obtain encoded data, where the encoded data includes visual data, point cloud data, and communication data; The normalization module 504 is used to perform normalization processing on the encoded data to obtain normalized visual data, point cloud data, and communication data; The graph creation module 505 is used to create an explicit physical graph based on the normalized visual data, point cloud data, and communication data, where the explicit physical graph is used to characterize communication quality and physical distance; mine potential information based on the normalized visual data, point cloud data, and communication data, and create an implicit semantic graph, where the potential information includes at least one of potential vehicle direction change, or potential braking and potential collision information; The feature extraction module 506 is used to extract temporal dependence features based on the explicit physical graph and the implicit semantic graph through spatial graph convolution, temporal convolution, and causal dilated convolution, and optimize them through spatio-temporal position encoding and sparse self-attention mechanism to obtain semantic features; The feature fusion module 507 is used to perform feature fusion on the temporal dependence features and the semantic features to obtain fused features; The warning determination module 508 is used to determine traffic warning information based on the fused features, where the traffic warning information includes whether a collision occurs, and / or whether the traffic flow is greater than a preset threshold.

[0046] In a possible implementation manner, the feature extraction module includes: The graph feature acquisition sub-module is used to extract and aggregate image spatial features through spatial graph convolution based on the explicit physical graph and the implicit semantic graph to obtain spatial graph features; The temporal convolution sub-module is used to perform temporal convolution on the extracted spatial graph features to obtain graph features that satisfy the time order; The causal dilated convolution sub-module is used to perform causal dilated convolution on the obtained graph features that satisfy the time order to obtain the temporal dependence features that satisfy the causality law.

[0047] In a possible implementation manner, the feature extraction module includes: The coordinate normalization sub-module is used to perform coordinate normalization based on the explicit physical graph and the implicit semantic graph to obtain coordinate-normalized features; A time encoding sub-module for performing sine time encoding on the coordinates-normalized features to obtain sine time-encoded features; A feature extraction sub-module for dividing the sine time-encoded features into multiple neighborhood grids; for each node in the grid, calculating features based on the node and its multiple adjacent local nodes to obtain local window features; and based on the calculated local window features, extracting and fusing features of multiple granularities through multi-level memory nodes to obtain the semantic features.

[0048] In a possible implementation manner, the feature extraction module is specifically configured to extract temporal dependence features according to the explicit physical graph and the implicit semantic graph through spatial graph convolution, temporal convolution, and causal dilated convolution, and optimize them through spatio-temporal position encoding and sparse self-attention mechanism to obtain semantic features; extract first image features through a local spatio-temporal model, the explicit physical graph, and the implicit semantic graph; extract second image features through a global semantic model, the explicit physical graph, and the implicit semantic graph; calculate the temporal dependence features through cross-attention using the local spatio-temporal model based on the first image features and the second image features; and calculate the semantic features through cross-attention using the global semantic model based on the first image features and the second image features.

[0049] In a possible implementation manner, the traffic warning device based on a spatio-temporal dynamic network further includes: A pruning module for calculating the contribution degrees of the attention heads in the global semantic model according to the traffic warning information; and pruning the corresponding convolution channels of the attention heads with contribution degrees lower than a preset threshold according to the calculated contribution degrees.

[0050] It can be seen that through the solution of the embodiments of the present application, multi-source traffic data can be obtained, and a dynamic spatio-temporal graph including an explicit physical graph and an implicit semantic graph can be created according to the multi-source data, so as to extract and fuse temporal dependence features and semantic features according to the dynamic spatio-temporal graph of the explicit physical graph and the implicit semantic graph to obtain fused features, and thus determine traffic warning information according to the fused features, realizing the improvement of prediction accuracy through the acquisition and processing of various data.

[0051] Embodiments of the present application further provide an electronic device, as Figure 6 shown, including: A memory 601 for storing a computer program; A processor 602 for, when executing the program stored in the memory 601, implementing the following steps: Obtain multi-source traffic data, where the multi-source traffic data includes vehicle-side data, roadside data, and network data; Perform registration on the multi-source traffic data in the time dimension and the space dimension to obtain registered traffic data; Perform feature encoding on the registered traffic data to obtain encoded data, where the encoded data includes visual data, point cloud data, and communication data; Perform normalization processing on the encoded data to obtain normalized visual data, point cloud data, and communication data; According to the normalized visual data, point cloud data, and communication data, create an explicit physical map, where the explicit physical map is used to characterize communication quality and physical distance; according to the normalized visual data, point cloud data, and communication data, mine potential information, and create an implicit semantic map, where the potential information includes at least one of potential vehicle direction change, or potential braking and potential collision information; According to the explicit physical map and the implicit semantic map, extract temporal dependence features through spatial graph convolution, temporal convolution, and causal dilated convolution, and optimize through spatio-temporal position encoding and sparse self-attention mechanism to obtain semantic features; Perform feature fusion on the temporal dependence features and the semantic features to obtain fused features; Determine traffic warning information according to the fused features, where the traffic warning information includes whether a collision occurs, and / or whether the traffic flow is greater than a preset threshold.

[0052] The communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity, only a thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus.

[0053] The communication interface is used for communication between the above electronic device and other devices.

[0054] The memory may include a Random Access Memory (RAM), and may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0055] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0056] In another embodiment provided by the present application, a computer-readable storage medium is further provided. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the above traffic warning methods based on the spatio-temporal dynamic network are implemented.

[0057] In another embodiment provided by the present application, a computer program product containing instructions is further provided. When it runs on a computer, the computer is caused to execute any one of the traffic warning methods based on the spatio-temporal dynamic network in the above embodiments.

[0058] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a Solid State Disk (SSD), etc.

[0059] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the said element.

[0060] Each embodiment in this specification is described in a related manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the embodiments of the device, electronic device, and storage medium, since they are basically similar to the method embodiments, the description is relatively simple, and reference can be made to the corresponding parts of the method embodiments for the relevant content.

[0061] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are all included in the protection scope of the present application.

Claims

1. A traffic warning method based on spatiotemporal dynamic network, characterized in that: The method comprises: Acquiring multi-source traffic data, wherein the multi-source traffic data includes vehicle-side data, road test data, and network data; Performing registration of the multi-source traffic data in terms of time dimension and space dimension to obtain registered traffic data; Performing feature encoding on the registered traffic data to obtain encoded data, wherein the encoded data includes visual data, point cloud data and communication data; Normalizing the encoded data to obtain normalized visual data, point cloud data, and communication data; Creating an explicit physical graph based on the normalized visual data, point cloud data, and communication data, wherein the explicit physical graph is used to characterize communication quality and physical distance; mining potential information based on the normalized visual data, point cloud data, and communication data, and creating an implicit semantic graph, wherein the potential information includes at least one of potential direction change, potential braking, and potential collision information of the vehicle; According to the explicit physical graph and the implicit semantic graph, temporal dependency features are extracted through spatial graph convolution, temporal convolution and causal dilation convolution, and semantic features are obtained through spatiotemporal position encoding and sparse self-attention mechanism optimization; Performing feature fusion on the temporal dependency feature and the semantic feature to obtain a fused feature; Traffic warning information is determined according to the fusion feature, wherein the traffic warning information includes whether a collision occurs and / or whether the traffic flow is greater than a preset threshold.

2. The traffic warning method based on spatiotemporal dynamic network according to claim 1 is characterized in that: The extracting of temporal dependency features according to the explicit physical graph and the implicit semantic graph through spatial graph convolution, temporal convolution and causal dilation convolution includes: According to the explicit physical graph and the implicit semantic graph, image spatial features are extracted and aggregated through spatial graph convolution to obtain spatial graph features; According to the extracted spatial graph features, temporal convolution is performed to obtain graph features that satisfy the time order; According to the acquired graph features that satisfy the time sequence, causal dilation convolution is performed to obtain the temporal dependency features that satisfy the causal law.

3. The traffic warning method based on spatiotemporal dynamic network according to claim 1 is characterized in that: The semantic features obtained by optimizing the spatiotemporal position encoding and sparse self-attention mechanism include: Performing coordinate normalization according to the explicit physical graph and the implicit semantic graph to obtain coordinate normalized features; Performing sinusoidal time coding on the coordinate-normalized features to obtain sinusoidal time-coded features; The features of the sinusoidal time coding are divided into multiple neighborhood grids; for each node in the grid, feature calculation is performed based on the node and multiple adjacent local nodes of the node to obtain local window features; based on the calculated local window features, multiple granularity features are extracted and fused through multi-level memory nodes to obtain the semantic features.

4. The traffic warning method based on spatiotemporal dynamic network according to claim 1 is characterized in that: The method extracts temporal dependency features according to the explicit physical graph and the implicit semantic graph through spatial graph convolution, temporal convolution and causal dilation convolution, and obtains semantic features through spatiotemporal position encoding and sparse self-attention mechanism optimization, including: According to the explicit physical graph and the implicit semantic graph, temporal dependency features are extracted through spatial graph convolution, temporal convolution and causal dilation convolution, and semantic features are obtained through spatiotemporal position encoding and sparse self-attention mechanism optimization; Extracting a first image feature through a local spatiotemporal model, the explicit physical map, and the implicit semantic map; extracting a second image feature through a global semantic model, the explicit physical map, and the implicit semantic map; Through cross attention, the local spatiotemporal model is used to calculate the temporal dependency feature based on the first image feature and the second image feature; through cross attention, the global semantic model is used to calculate the semantic feature based on the first image feature and the second image feature.

5. The traffic warning method based on spatiotemporal dynamic network according to claim 4 is characterized in that: After determining the traffic warning information according to the fusion feature, the traffic warning method based on spatiotemporal dynamic network further includes: Calculating the contribution of each attention head in the global semantic model according to the traffic warning information; According to the calculated contribution, the corresponding convolution channels of the attention heads whose contributions are lower than the preset threshold are pruned.

6. A traffic warning device based on a spatiotemporal dynamic network, characterized in that: The device comprises: A data acquisition module, used to acquire multi-source traffic data, wherein the multi-source traffic data includes vehicle-side data, road test data and network data; A data registration module, used to register the multi-source traffic data in terms of time dimension and space dimension to obtain registered traffic data; A data encoding module, used for performing feature encoding on the registered traffic data to obtain encoded data, wherein the encoded data includes visual data, point cloud data and communication data; A normalization module, used for normalizing the encoded data to obtain normalized visual data, point cloud data and communication data; A graph creation module is used to create an explicit physical graph based on the normalized visual data, point cloud data and communication data, wherein the explicit physical graph is used to characterize communication quality and physical distance; and to mine potential information based on the normalized visual data, point cloud data and communication data, and to create an implicit semantic graph, wherein the potential information includes at least one of potential direction change, potential braking and potential collision information of the vehicle; A feature extraction module, for extracting temporal dependency features according to the explicit physical graph and the implicit semantic graph through spatial graph convolution, temporal convolution and causal dilation convolution, and obtaining semantic features through spatiotemporal position encoding and sparse self-attention mechanism optimization; A feature fusion module, used for fusing the temporal dependency feature and the semantic feature to obtain a fused feature; The warning determination module is used to determine traffic warning information according to the fusion feature, wherein the traffic warning information includes whether a collision occurs and / or whether the traffic flow is greater than a preset threshold.

7. The traffic warning device based on spatiotemporal dynamic network according to claim 6 is characterized in that: The feature extraction module comprises: A graph feature acquisition submodule, used to extract and aggregate image spatial features through spatial graph convolution according to the explicit physical graph and the implicit semantic graph, so as to obtain spatial graph features; The temporal convolution submodule is used to perform temporal convolution based on the extracted spatial graph features to obtain graph features that satisfy the temporal order; The causal expansion convolution submodule is used to perform causal expansion convolution according to the acquired graph features that satisfy the time sequence, so as to obtain the temporal dependency features that satisfy the causal law.

8. The traffic warning device based on spatiotemporal dynamic network according to claim 6 is characterized in that: The feature extraction module comprises: A coordinate normalization submodule, used to perform coordinate normalization according to the explicit physical graph and the implicit semantic graph to obtain coordinate normalized features; A time coding submodule, used for performing sinusoidal time coding on the coordinate normalized features to obtain sinusoidal time coded features; The feature extraction submodule is used to divide the features of the sinusoidal time coding into multiple neighborhood grids; for each node in the grid, feature calculation is performed based on the node and multiple adjacent local nodes of the node to obtain local window features; based on the calculated local window features, multiple granularity features are extracted and integrated through multi-level memory nodes to obtain the semantic features.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; The processor is used to implement the traffic warning method based on spatiotemporal dynamic network described in any one of claims 1-5 when executing the program stored in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the traffic warning method based on a spatiotemporal dynamic network described in any one of claims 1-5 is implemented.

Images