Abnormal traffic situation generation method based on large language model and diffusion model

Abstract

The application discloses an abnormal traffic situation generation method based on a large language model and a diffusion model, relates to the field of computer machine learning, and comprises the following steps: acquiring a traffic event text description and a historical traffic state, aligning timestamps of the text description and the historical traffic state, and dividing a training set, a verification set and a test set; acquiring a traffic road network structure, making the traffic road network structure into an adjacency matrix, and using the adjacency matrix for subsequent spatial relationship modeling of a model; constructing an abnormal traffic situation generation model, training the abnormal traffic situation generation model by using the training set, adjusting parameters by using the verification set, and evaluating generation effect by using the test set; and finally, optimizing the model by using a weighted mixed objective function combining a noise prediction loss and a latent reconstruction loss. The application can effectively generate an abnormal traffic situation with semantic consistency and spatial rationality, and provides support for traffic prediction, emergency management and intelligent traffic scheduling.

Description

Technical Field

[0001] This invention relates to the field of computer machine learning, specifically to a method for generating abnormal traffic situations based on large language models and diffusion models. Background Technology

[0002] With the development of intelligent transportation and urban computing, traffic flow prediction and situation generation have gradually become important components of intelligent transportation systems. Existing research can be broadly divided into two categories: traffic prediction based on historical data and traffic prediction based on text data. The current development status is as follows:

[0003] (1) Traffic prediction based on historical data

[0004] Early traffic flow prediction research primarily relied on statistical modeling and early machine learning methods. Seasonal and periodic patterns were typically captured using the Seasonal Autoregressive Integral Moving Average (SARIMA) model (Williams and Hoel 2003), while the Support Vector Regression (SVR) framework (Wu, Ho, and Lee 2004) combined historical data with real-time sensor data to improve the accuracy of short-term predictions. To address the issue of missing data, researchers introduced low-rank representation methods with adaptive spatiotemporal constraints (Du et al. 2016) and explored toll ticket data analysis as an alternative for traffic volume estimation (Zheng et al. 2019). Although these methods are computationally efficient and easy to interpret, their reliance on linear assumptions often limits their ability to model the nonlinear and dynamic characteristics of urban traffic systems.

[0005] The rise of deep learning has provided new capabilities for modeling complex temporal dependencies and capturing rich spatial interactions. Sequential modeling architectures, such as Long Short-Term Memory (LSTM) networks (Ma et al. 2015) and Gated Recurrent Units (GRUs) (Chung et al. 2014), have been extended to multidimensional traffic sequences through fully connected LSTMs (Zhao and Zhu 2021) and convolutional LSTMs (Shi et al. 2015). Further developments have replaced the fully connected layers of GRUs with convolutional operations (Zhao and Zhu 2021) to enhance spatial feature extraction, and employed three-dimensional convolutional networks (Guo et al. 2019) to jointly learn spatiotemporal dependencies. Graph-based frameworks, including Spatiotemporal Graph Convolutional Networks (STGCN) (Yu, Yin, and Zhu 2018), Multi-Task Graph Neural Networks (MTGNN) (Wu et al. 2020), Spatiotemporal Synchronous Graph Convolutional Networks (STSGCN) (Song et al. 2020), and Context-Conscious Graph Design (Huo et al. 2023), provide more natural representations of irregular road network structures. However, most of these methods rely on predefined static adjacency matrices, limiting their adaptability to dynamic adjustments in road network connectivity and congestion propagation.

[0006] (2) Traffic flow prediction based on text data

[0007] Recent studies have shown that textual information from social media and online platforms can effectively supplement traditional traffic datasets. Research indicates that integrating filtered social network information (Abidin, Kolberg, and Hussain 2014) can improve the accuracy of public transport demand forecasting, while combining microblog content with historical passenger flow data (Ni, He, and Gao 2017) helps improve subway traffic forecasting during emergencies. By integrating real-time traffic reports, weather observations, and social media posts into deep sequence models (Essien et al. 2021), joint modeling of temporal dynamics and external influences can be achieved. Other studies have explored large-scale Twitter stream data to discover patterns in daily activities (Yao and Qian 2021) and combined calendar, weather, and traffic records with relevant textual content (Shoaeinaeini, Ozturk, and Gupta 2022) to improve forecast accuracy. These research findings fully demonstrate the value of textual information in detecting abnormal traffic conditions and providing contextual cues beyond numerical measurements. However, most existing methods still rely on extracting structured indicators from raw text, often ignoring the rich semantic information that can further enhance traffic flow modeling.

[0008] Despite some progress in generating abnormal traffic situations, existing technologies still face the following challenges:

[0009] (1) Limited input signals: Existing methods usually rely on limited sensor data or coarse-grained event attributes (such as event type, location and duration), which are difficult to fully reflect the sudden, local and rapidly evolving characteristics of traffic anomalies.

[0010] (2) Insufficient semantic representation ability: Most methods treat event information as static features and cannot effectively extract or utilize the potential causal semantics of events, such as the propagation pattern of congestion in the road network and the impact of event severity on traffic evolution.

[0011] (3) Rigid generation mechanism: Existing models mostly adopt static conditional input methods and lack the ability to adaptively introduce high-level semantic information and fine-grained spatiotemporal features in the time dimension. These limitations prevent traffic event information from being fully utilized in the generation of abnormal traffic situations, thereby reducing the authenticity and interpretability of the generated results.

[0012] In recent years, the application of diffusion models in traffic situation generation has offered new possibilities for addressing the problems of limited input signals, insufficient semantic representation, and rigid generation mechanisms in existing technologies. Diffusion models generate conditional samples through a progressive denoising process, and their iterative generation characteristics can well match the dynamic response of traffic under abnormal events; that is, traffic flow typically evolves from initial random fluctuations to a stable congestion diffusion pattern. This inherent consistency makes diffusion models particularly suitable for capturing the complex spatiotemporal patterns of traffic evolution. Therefore, how to effectively integrate event semantic information, guide the macroscopic structure and microscopic details of traffic situations during the generation process, and maintain the spatiotemporal consistency of the generated results has become a key issue in improving the quality of abnormal traffic situation generation. Summary of the Invention

[0013] The purpose of this invention is to provide a method for generating abnormal traffic situations based on a large language model and a diffusion model, so as to solve the problems mentioned in the background art.

[0014] To achieve the above objectives, the present invention provides the following technical solution:

[0015] A method for generating abnormal traffic situations based on large language models and diffusion models includes the following steps:

[0016] Obtain text descriptions of traffic events and historical traffic status, align the timestamps of the text descriptions and historical traffic status, and divide the data into training, validation, and test sets.

[0017] Obtain the traffic network structure and create an adjacency matrix for subsequent spatial relationship modeling.

[0018] A non-normal traffic situation generation model is constructed, trained using a training set, tuned using a validation set, and evaluated using a test set.

[0019] The model is optimized using a weighted hybrid objective function that combines noise prediction loss and potential reconstruction loss. The noise prediction loss is used to fit the real noise in the diffusion process, while the potential reconstruction loss is used to improve the accuracy of the reconstruction of the potential representation.

[0020] The abnormal traffic situation generation model includes: a structured text description generation module, a latent space encoding module, a forward diffusion module, a graph structure spatial relationship enhancement module, and a conditionally guided reverse denoising module. The denoising network used in the reverse denoising module is a U-Net conditional noise prediction network based on a time-aware semantic injection mechanism.

[0021] Structured text description generation module: Guided by prompts that integrate road connectivity knowledge, congestion diffusion theory and natural language instructions, it uses a large language model to transform traffic event text into structured semantic descriptions;

[0022] Latent spatial encoding module: compresses historical traffic states into a latent spatial representation through a variational autoencoder (VAE) to reduce computational complexity and preserve key spatiotemporal semantics;

[0023] Forward diffusion module: Constructs a Markov process that progressively injects Gaussian noise into the latent representation to simulate the evolution of traffic conditions from order to chaos, and generates a noisy latent representation as the starting point for reverse denoising;

[0024] Graph structure spatial relationship enhancement module: Based on the road network adjacency matrix and graph convolutional network, the road topology is embedded into the latent representation to enhance the spatial consistency of the generated results;

[0025] Conditionally guided reverse denoising module: Guided by structured semantic description, it uses conditional Markov chains to gradually denoise the latent representation, so that the generation process satisfies the semantic constraints of traffic events while maintaining the authenticity and diversity of traffic situations.

[0026] U-Net Conditional Noise Prediction Network Based on Time-Aware Semantic Injection Mechanism: It adopts the U-Net structure and introduces a time-aware semantic injection mechanism to dynamically balance global trends and local details in different denoising stages. It achieves multi-scale alignment of semantics and latent representation through cross-attention mechanism, and finally obtains traffic situation that conforms to event semantics and road network topology.

[0027] The construction of a structured text description generation module includes the following steps:

[0028] We first construct a set of prompts incorporating road connectivity knowledge, congestion diffusion theory, and natural language instructions. Road connectivity knowledge is derived from a road network adjacency matrix, where a value of 1 indicates a direct connection between two roads. Non-zero values ​​are extracted and converted into textual road pairings, such as "Road A is connected to Road B," to enhance the large language model's understanding of road network topology. Congestion diffusion theory describes the propagation mechanism of congestion between affected road segments and their adjacent segments, constrained by topological distance, directionality, and event severity. This knowledge is integrated into the prompts to improve the large language model's reasoning ability in traffic scenarios. Natural language instructions specify task objectives, guiding the large language model to extract structured information from the original event text. These tasks include key event attribute identification (timestamp, location, type, severity), related event clustering, and prediction of potential congestion propagation range.

[0029] In summary, the final prompt words are divided into two parts: system prompt words and user prompt words, as follows:

[0030] (1) System prompt:

[0031] You are a traffic flow prediction assistant, integrating structured domain knowledge with real-world traffic theory. You will analyze timestamped traffic reports to predict how congestion will propagate through the road network.

[0032] You will receive:

[0033] 1. Key principle of congestion diffusion theory: Congestion tends to spread from congested roads to adjacent road segments over time, influenced by topology and flow direction.

[0034] 2. Detailed knowledge of road connections (road segments and their connections): For example, road A is connected to road B and road C.

[0035] Based on these factors, reason spatially and temporally to infer which roads are likely to become congested in the near future. Output a concise summary containing major roads and their congestion levels. You must output the results in a compact, consistent, single-line format suitable for input into a machine learning model.

[0036] (2) User prompt words:

[0037] Please extract structured and insightful information from the following original report.

[0038] illustrate:

[0039] 1. Start outputting timestamps in the format yyyy-mm-dd hh:mm.

[0040] 2. Write a brief prediction and overall summary of today's traffic conditions (maximum 30 words), focusing on how these events might affect traffic flow, such as potential congestion, reduced speed, delays, or route detours.

[0041] 3. Automatically group traffic events by event type. Common types include: accidents, road closures, construction, and traffic control, but other types may also appear—don't limit yourself.

[0042] 4. For each event type, list the affected road segments and their severity in parentheses. Valid severity levels are: High, Medium, Low. Separate roads using commas.

[0043] 5. Strictly adhere to the provided road connectivity information and congestion diffusion theory. Predicting potential congestion spread Output the predicted congested roads and their severity in the same format.

[0044] 6. All output should be in lowercase. Do not use newlines. Use semicolons to separate different categories. Road names must be separated by spaces, not underscores.

[0045] 7. The final output must be a compact line that follows this pattern:

[0046] Timestamp: [Time]; Note: [Prediction Summary]; Accidents: [loc1 (Severity), loc2 (Severity", ...]; Road Closures: [loc3 (Severity), loc4 (Severity", ...]; Construction: [loc5 (Severity), loc6 (Severity, ...]; Traffic Control: [loc7 (Severity), loc8 (Severity, ...]; ...; Predicted Congestion: [loc9 (Severity), loc10 (Severity, ...]; "

[0047] Subsequently, we used the designed prompt words in the structured text description generation unit to transform the input event text R and generate a structured text description S, defined as:

[0048] (1)

[0049] in, This indicates a prompt to construct the function. This represents the response function of the large language model; here, DeepSeek-R1-8B is used.

[0050] To facilitate the use of structured text in the generation process of the diffusion model, we pass the structured text description S through a text encoder. Encoded as semantic embedding vector :

[0051] (2)

[0052] The embedding vector This will be used as a semantic condition to guide the downstream traffic situation generation process.

[0053] Constructing the latent space coding module includes the following steps:

[0054] We use the BJTT dataset, where each historical traffic state is represented as a tensor. Where D represents the feature dimension, This represents the total number of road segments arranged in a two-dimensional grid. Each spatial location corresponds to a specific road segment. This spatial organization effectively maps the traffic network into a "pseudo-image," allowing each frame to serve as a three-channel input to the potential diffusion model.

[0055] To improve generation efficiency while preserving traffic semantics, the diffusion process is performed in the compressed latent space. Each traffic state is first mapped to a latent representation using a variational autoencoder (VAE). :

[0056] (3)

[0057] (4)

[0058] in, The encoder module represents the high-dimensional traffic tensor y that transforms it into a low-dimensional latent representation. ; It's a decoder, responsible for... Reconstruct the original traffic image.

[0059] Furthermore, compared to directly diffusing in the original traffic tensor space, the latent space has lower dimensions and higher semantic density, which reduces computational overhead while preserving key spatiotemporal traffic patterns, thus better meeting the modeling needs of traffic scenarios.

[0060] Constructing the forward diffusion module includes the following steps:

[0061] To facilitate diffusion modeling, the latent tensor is... Flattened into a node sequence ,in This represents the total number of spatial locations. The forward diffusion process originates from the reshaped latent representation. To begin, in order to simulate the gradual disruption of spatial traffic conditions, we defined a forward Markov process, which, within T time steps... Gaussian noise is added gradually. This process is defined as follows:

[0062] (5)

[0063] in The cumulative product representing noise scheduling, where the actual noise injection operation at time step t can be implemented through reparameterization:

[0064] (6)

[0065] in Is with The unit covariance matrix for dimension matching. It is injected noise. The potential noise representation at the final time step T. The samples were also obtained through the same forward diffusion process and used as the starting point for subsequent denoising. This stepwise denoising process semantically simulates the evolution of traffic conditions from order to chaos.

[0066] Constructing a graph structure spatial relationship enhancement module includes the following steps:

[0067] To enhance spatial consistency during the generation process, we utilize Graph Convolutional Networks (GCNs) to introduce the road network topology into the latent space. The road graph is composed of an adjacency matrix. The formula for calculating its normalized adjacency matrix is ​​as follows:

[0068] (7)

[0069] in For degree matrix, I It is the identity matrix. Subsequently, it is represented by a noisy latent representation. As input node features, we apply two GCN layers sequentially:

[0070] (8)

[0071] in , The weights are learnable, and ReLU and Sigmoid are the activation functions. Finally, the enhanced features are fused with the original latent representation via residual connections.

[0072] (9)

[0073] Received It will be used in the subsequent denoising process.

[0074] Constructing a conditionally guided inverse denoising module includes the following steps:

[0075] The backdiffusion process is the process of inputting spatial latent representations. Stepwise reconstruction of the initial latent representation And subject to external semantic conditions Guided by a conditional Markov chain, at each time step t, the model samples the previous state from a Gaussian distribution. :

[0076] (10)

[0077] in, This represents the mean term predicted by the conditional noise prediction network at time step t, used to guide the gradual reconstruction of the latent representation from the noisy state. The calculation is as follows:

[0078] (11)

[0079] in It is a U-Net conditional noise prediction network based on a time-aware semantic injection mechanism, used to predict noise added at time step t. The variance term learned by the model is used to control the noise intensity at each sampling step and is determined by noise scheduling.

[0080] (12)

[0081] This scheduling strategy controls the amount of noise retained at each step to balance the stability and fidelity of the generation process.

[0082] Subsequently, the denoised latent representation Obtained through the following formula:

[0083] (13)

[0084] in, Standard Gaussian noise is used to introduce random perturbations during backdiffusion. This random term prevents the model from getting stuck in a single solution, thus generating diverse traffic situation results under semantic constraints.

[0085] As the time step t decreases from T to 0, this iterative process progressively optimizes the latent representation, ultimately generating a latent representation that integrates spatial topology and semantic intent. Finally, this potential representation By pre-trained decoder Decode to generate the final traffic situation.

[0086] U-Net Conditional Noise Prediction Network Based on Time-Aware Semantic Injection Mechanism It includes the following steps:

[0087] Conditional noise prediction network The U-Net architecture, combined with a time-aware semantic injection mechanism, is based on the following idea: semantic conditions. The semantic role of the input changes at different stages of denoising: in the early stages, Gaussian noise dominates, and the model relies on global semantics to outline the overall traffic situation; in the later stages, noise gradually weakens, and the model needs more fine-grained local semantics to refine spatial details (such as congestion boundaries and road-level differences). Specifically, the input... They are projected into two distinct vectors:

[0088] (14)

[0089] (15)

[0090] in, and It consists of two parallel multilayer perceptrons, which extract global semantic trends and local structural details, respectively. and These represent two types of semantic embeddings, respectively. Simultaneously, the current back-diffusion time step t is embedded into a time vector. After processing by a one-dimensional convolution and feedforward network, dynamic weights are generated to control the balance between global and local semantics.

[0091] (16)

[0092] Where W is the learnable parameter matrix, It is a one-dimensional convolution that extracts time-related features. The final modulation vector is calculated as follows:

[0093] (17)

[0094] This is then injected into the intermediate hidden state h of the U-Net:

[0095] (18)

[0096] Where h represents the current hidden state of the U-Net. It is a lightweight convolutional network used to adjust semantic vectors to match the dimensions of hidden states.

[0097] The U-Net backbone adopts an "encoder-bottleneck-decoder" structure: input latent representation The semantic modulation vector is obtained by progressive downsampling to extract multi-scale spatial features, followed by upsampling to reconstruct the denoised latent representation. Selected modules are repeatedly injected into the encoder and decoder to achieve semantic control across resolutions.

[0098] In addition to direct modulation, semantic conditions are also injected through a cross-attention mechanism in the decoder layer to achieve content-aware alignment. The attention calculation is as follows:

[0099] (19)

[0100] Among the queries From the decoder's current hidden state, key Sum Then, based on the conditional input Export. Learnable matrix. , and Features are projected into a shared latent space for matching;

[0101] The objective loss function of the non-normal traffic situation generation method based on large language models and diffusion models is a weighted mixed loss:

[0102] (20)

[0103] in, and The loss weighting coefficient represents the noise prediction loss. Used to measure the conditional noise prediction network and the actual injected noise in the forward process. Mean square error between:

[0104] (twenty one)

[0105] in, This represents the prediction result of the conditional noise prediction network at time step t. This is the time-step weighting coefficient. This term is used to constrain the model to accurately fit the noise in the diffusion process.

[0106] Potential Reconstruction Loss It is necessary to back-calculate the estimate of the original latent representation based on the model's predicted noise. :

[0107] (twenty two)

[0108] in, For the noisy latent representation at time step t, The noise scheduling parameter is then defined. The L1 error is subsequently defined as the potential reconstruction loss:

[0109] (twenty three)

[0110] This section emphasizes the accuracy of the reconstruction, ensuring that the model output is consistent with the true latent space.

[0111] The final training loss strikes a balance between the stability of noise predictions and the accuracy of the underlying representation. The model is trained by minimizing the above objective function, thereby improving generation accuracy and preventing overfitting, resulting in better generalization ability across different traffic scenarios.

[0112] Compared with existing technologies, the advantages of this invention are as follows: By combining a large language model with a latent diffusion model, this invention can expand concise traffic event descriptions into structured, semantically rich representations, thereby accurately capturing the generation, propagation, and interaction patterns of congestion with road network topology, and realistically reproducing abnormal traffic situations. Simultaneously, the time-aware semantic injection mechanism proposed in this invention introduces global semantics and local spatiotemporal information in stages during the generation process, making macroscopic traffic patterns reasonable and microscopic details precise, resulting in more coherent and interpretable spatiotemporal evolution of the generated results. Furthermore, this invention can accurately model complex abnormal traffic events by combining road topology, historical event patterns, and congestion diffusion patterns, providing a reference for traffic management, simulation, and emergency response. Extensive experiments show that in tests on the first large-scale public text-traffic dataset—the Beijing Text-Traffic Dataset (BJTT)—the abnormal traffic situations generated by this invention significantly outperform existing technologies in terms of realism, spatiotemporal consistency, and semantic alignment, thus laying a technical foundation for the intelligent generation and practical application of abnormal traffic scenarios. Attached Figure Description

[0113] Figure 1 This invention presents a method for generating abnormal traffic situations based on a large language model and a diffusion model. Detailed Implementation

[0114] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0115] A method for generating abnormal traffic situations based on large language models and diffusion models includes the following steps:

[0116] Obtain text descriptions of traffic events and historical traffic status, align the timestamps of the text descriptions and historical traffic status, and divide the data into training, validation, and test sets.

[0117] Obtain the traffic network structure and create an adjacency matrix for subsequent spatial relationship modeling.

[0118] A non-normal traffic situation generation model is constructed, trained using a training set, tuned using a validation set, and evaluated using a test set.

[0119] The model is optimized using a weighted hybrid objective function that combines noise prediction loss and potential reconstruction loss. The noise prediction loss is used to fit the real noise in the diffusion process, while the potential reconstruction loss is used to improve the accuracy of the reconstruction of the potential representation.

[0120] The abnormal traffic situation generation model includes: a structured text description generation module, a latent space encoding module, a forward diffusion module, a graph structure spatial relationship enhancement module, and a conditionally guided reverse denoising module. The denoising network used in the reverse denoising module is a U-Net conditional noise prediction network based on a time-aware semantic injection mechanism.

[0121] Structured text description generation module: Guided by prompts that integrate road connectivity knowledge, congestion diffusion theory and natural language instructions, it uses a large language model to transform traffic event text into structured semantic descriptions;

[0122] Latent spatial encoding module: compresses historical traffic states into a latent spatial representation through a variational autoencoder (VAE) to reduce computational complexity and preserve key spatiotemporal semantics;

[0123] Forward diffusion module: Constructs a Markov process that progressively injects Gaussian noise into the latent representation to simulate the evolution of traffic conditions from order to chaos, and generates a noisy latent representation as the starting point for reverse denoising;

[0124] Graph structure spatial relationship enhancement module: Based on the road network adjacency matrix and graph convolutional network, the road topology is embedded into the latent representation to enhance the spatial consistency of the generated results;

[0125] Conditionally guided reverse denoising module: Guided by structured semantic description, it uses conditional Markov chains to gradually denoise the latent representation, so that the generation process satisfies the semantic constraints of traffic events while maintaining the authenticity and diversity of traffic situations.

[0126] U-Net Conditional Noise Prediction Network Based on Time-Aware Semantic Injection Mechanism: It adopts the U-Net structure and introduces a time-aware semantic injection mechanism to dynamically balance global trends and local details in different denoising stages. It achieves multi-scale alignment of semantics and latent representation through cross-attention mechanism, and finally obtains traffic situation that conforms to event semantics and road network topology.

[0127] The construction of a structured text description generation module includes the following steps:

[0128] We first construct a set of prompts incorporating road connectivity knowledge, congestion diffusion theory, and natural language instructions. Road connectivity knowledge is derived from a road network adjacency matrix, where a value of 1 indicates a direct connection between two roads. Non-zero values ​​are extracted and converted into textual road pairings, such as "Road A is connected to Road B," to enhance the large language model's understanding of road network topology. Congestion diffusion theory describes the propagation mechanism of congestion between affected road segments and their adjacent segments, constrained by topological distance, directionality, and event severity. This knowledge is integrated into the prompts to improve the large language model's reasoning ability in traffic scenarios. Natural language instructions specify task objectives, guiding the large language model to extract structured information from the original event text. These tasks include key event attribute identification (timestamp, location, type, severity), related event clustering, and prediction of potential congestion propagation range.

[0129] In summary, the final prompt words are divided into two parts: system prompt words and user prompt words, as follows:

[0130] (1) System prompt:

[0131] You are a traffic flow prediction assistant, integrating structured domain knowledge with real-world traffic theory. You will analyze timestamped traffic reports to predict how congestion will propagate through the road network.

[0132] You will receive:

[0133] 1. Key principle of congestion diffusion theory: Congestion tends to spread from congested roads to adjacent road segments over time, influenced by topology and flow direction.

[0134] 2. Detailed knowledge of road connections (road segments and their connections): For example, road A is connected to road B and road C.

[0135] Based on these factors, reason spatially and temporally to infer which roads are likely to become congested in the near future. Output a concise summary containing major roads and their congestion levels. You must output the results in a compact, consistent, single-line format suitable for input into a machine learning model.

[0136] (2) User prompt words:

[0137] Please extract structured and insightful information from the following original report.

[0138] illustrate:

[0139] 1. Start outputting timestamps in the format yyyy-mm-dd hh:mm.

[0140] 2. Write a brief prediction and overall summary of today's traffic conditions (maximum 30 words), focusing on how these events might affect traffic flow, such as potential congestion, reduced speed, delays, or route detours.

[0141] 3. Automatically group traffic events by event type. Common types include: accidents, road closures, construction, and traffic control, but other types may also appear—don't limit yourself.

[0142] 4. For each event type, list the affected road segments and their severity in parentheses. Valid severity levels are: High, Medium, Low. Separate roads using commas.

[0143] 5. Strictly adhere to the provided road connectivity information and congestion diffusion theory. Predicting potential congestion spread Output the predicted congested roads and their severity in the same format.

[0144] 6. All output should be in lowercase. Do not use newlines. Use semicolons to separate different categories. Road names must be separated by spaces, not underscores.

[0145] 7. The final output must be a compact line that follows this pattern:

[0146] Timestamp: [Time]; Note: [Prediction Summary]; Accidents: [loc1 (Severity), loc2 (Severity", ...]; Road Closures: [loc3 (Severity), loc4 (Severity", ...]; Construction: [loc5 (Severity), loc6 (Severity, ...]; Traffic Control: [loc7 (Severity), loc8 (Severity, ...]; ...; Predicted Congestion: [loc9 (Severity), loc10 (Severity, ...]; "

[0147] Subsequently, we used the designed prompt words in the structured text description generation unit to transform the input event text R and generate a structured text description S, defined as:

[0148] (1)

[0149] in, This indicates a prompt to construct the function. This represents the response function of the large language model; here, DeepSeek-R1-8B is used.

[0150] To facilitate the use of structured text in the generation process of the diffusion model, we pass the structured text description S through a text encoder. Encoded as semantic embedding vector :

[0151] (2)

[0152] The embedding vector This will be used as a semantic condition to guide the downstream traffic situation generation process.

[0153] Constructing the latent space coding module includes the following steps:

[0154] We use the BJTT dataset, where each historical traffic state is represented as a tensor. Where D represents the feature dimension, This represents the total number of road segments arranged in a two-dimensional grid. Each spatial location corresponds to a specific road segment. This spatial organization effectively maps the traffic network into a "pseudo-image," allowing each frame to serve as a three-channel input to the potential diffusion model.

[0155] To improve generation efficiency while preserving traffic semantics, the diffusion process is performed in the compressed latent space. Each traffic state is first mapped to a latent representation using a variational autoencoder (VAE). :

[0156] (3)

[0157] (4)

[0158] in, The encoder module represents the high-dimensional traffic tensor y that transforms it into a low-dimensional latent representation. ; It's a decoder, responsible for... Reconstruct the original traffic image.

[0159] Furthermore, compared to directly diffusing in the original traffic tensor space, the latent space has lower dimensions and higher semantic density, which reduces computational overhead while preserving key spatiotemporal traffic patterns, thus better meeting the modeling needs of traffic scenarios.

[0160] Constructing the forward diffusion module includes the following steps:

[0161] To facilitate diffusion modeling, the latent tensor is... Flattened into a node sequence ,in This represents the total number of spatial locations. The forward diffusion process originates from the reshaped latent representation. To begin, in order to simulate the gradual disruption of spatial traffic conditions, we defined a forward Markov process, which, within T time steps... Gaussian noise is added gradually. This process is defined as follows:

[0162] (5)

[0163] in The cumulative product representing noise scheduling, where the actual noise injection operation at time step t can be implemented through reparameterization:

[0164] (6)

[0165] in Is with The unit covariance matrix for dimension matching. It is injected noise. The potential noise representation at the final time step T. The samples were also obtained through the same forward diffusion process and used as the starting point for subsequent denoising. This stepwise denoising process semantically simulates the evolution of traffic conditions from order to chaos.

[0166] Constructing a graph structure spatial relationship enhancement module includes the following steps:

[0167] To enhance spatial consistency during the generation process, we utilize Graph Convolutional Networks (GCNs) to introduce the road network topology into the latent space. The road graph is composed of an adjacency matrix. The formula for calculating its normalized adjacency matrix is ​​as follows:

[0168] (7)

[0169] in For degree matrix, I It is the identity matrix. Subsequently, it is represented by a noisy latent representation. As input node features, we apply two GCN layers sequentially:

[0170] (8)

[0171] in , The weights are learnable, and ReLU and Sigmoid are the activation functions. Finally, the enhanced features are fused with the original latent representation via residual connections.

[0172] (9)

[0173] Received It will be used in the subsequent denoising process.

[0174] Constructing a conditionally guided inverse denoising module includes the following steps:

[0175] The backdiffusion process is the process of inputting spatial latent representations. Stepwise reconstruction of the initial latent representation And subject to external semantic conditions Guided by a conditional Markov chain, at each time step t, the model samples the previous state from a Gaussian distribution. :

[0176] (10)

[0177] in, This represents the mean term predicted by the conditional noise prediction network at time step t, used to guide the gradual reconstruction of the latent representation from the noisy state. The calculation is as follows:

[0178] (11)

[0179] in It is a U-Net conditional noise prediction network based on a time-aware semantic injection mechanism, used to predict noise added at time step t. The variance term learned by the model is used to control the noise intensity at each sampling step and is determined by noise scheduling.

[0180] (12)

[0181] This scheduling strategy controls the amount of noise retained at each step to balance the stability and fidelity of the generation process.

[0182] Subsequently, the denoised latent representation Obtained through the following formula:

[0183] (13)

[0184] in, Standard Gaussian noise is used to introduce random perturbations during backdiffusion. This random term prevents the model from getting stuck in a single solution, thus generating diverse traffic situation results under semantic constraints.

[0185] As the time step t decreases from T to 0, this iterative process progressively optimizes the latent representation, ultimately generating a latent representation that integrates spatial topology and semantic intent. Finally, this potential representation By pre-trained decoder Decode to generate the final traffic situation.

[0186] U-Net Conditional Noise Prediction Network Based on Time-Aware Semantic Injection Mechanism It includes the following steps:

[0187] Conditional noise prediction network The U-Net architecture, combined with a time-aware semantic injection mechanism, is based on the following idea: semantic conditions. The semantic role of the input changes at different stages of denoising: in the early stages, Gaussian noise dominates, and the model relies on global semantics to outline the overall traffic situation; in the later stages, noise gradually weakens, and the model needs more fine-grained local semantics to refine spatial details (such as congestion boundaries and road-level differences). Specifically, the input... They are projected into two distinct vectors:

[0188] (14)

[0189] (15)

[0190] in, and It consists of two parallel multilayer perceptrons, which extract global semantic trends and local structural details, respectively. and These represent two types of semantic embeddings, respectively. Simultaneously, the current back-diffusion time step t is embedded into a time vector. After processing by a one-dimensional convolution and feedforward network, dynamic weights are generated to control the balance between global and local semantics.

[0191] (16)

[0192] Where W is the learnable parameter matrix, It is a one-dimensional convolution that extracts time-related features. The final modulation vector is calculated as follows:

[0193] (17)

[0194] This is then injected into the intermediate hidden state h of the U-Net.

[0195] (18)

[0196] Where h represents the current hidden state of the U-Net. It is a lightweight convolutional network used to adjust semantic vectors to match the dimensions of hidden states.

[0197] The U-Net backbone adopts an "encoder-bottleneck-decoder" structure: input latent representation The semantic modulation vector is obtained by progressive downsampling to extract multi-scale spatial features, followed by upsampling to reconstruct the denoised latent representation. Selected modules are repeatedly injected into the encoder and decoder to achieve semantic control across resolutions.

[0198] In addition to direct modulation, semantic conditions are also injected through a cross-attention mechanism in the decoder layer to achieve content-aware alignment. The attention calculation is as follows:

[0199] (19)

[0200] Among the queries From the decoder's current hidden state, key Sum Then, based on the conditional input Export. Learnable matrix. , and Features are projected into a shared latent space for matching;

[0201] The objective loss function of the non-normal traffic situation generation method based on large language models and diffusion models is a weighted mixed loss:

[0202] (20)

[0203] in, and The loss weighting coefficient is the loss weighting coefficient, and the noise prediction loss is... Used to measure the conditional noise prediction network and the actual injected noise in the forward process. Mean square error between:

[0204] (twenty one)

[0205] in, This represents the prediction result of the conditional noise prediction network at time step t. This is the time-step weighting coefficient. This term is used to constrain the model to accurately fit the noise in the diffusion process.

[0206] Potential Reconstruction Loss It is necessary to back-calculate the estimate of the original latent representation based on the model's predicted noise. :

[0207] (twenty two)

[0208] in, For the noisy latent representation at time step t, The noise scheduling parameter is then defined. The L1 error is subsequently defined as the potential reconstruction loss:

[0209] (twenty three)

[0210] This section emphasizes the accuracy of the reconstruction, ensuring that the model output is consistent with the true latent space.

[0211] The final training loss strikes a balance between the stability of noise predictions and the accuracy of the underlying representation. The model is trained by minimizing the above objective function, thereby improving generation accuracy and preventing overfitting, resulting in better generalization ability across different traffic scenarios.

[0212] The present invention has experimentally verified the above method and achieved significant results, as described below:

[0213] 1. Dataset

[0214] The Beijing Text-Traffic (BjTT) dataset is a large-scale, multimodal data resource for urban traffic situation analysis, encompassing numerical sensor data and textual descriptions of emergencies. This dataset covers 1,260 major roads within the Fifth Ring Road of Beijing, spanning from January to March 2022, collected at 4-minute intervals, and contains over 32,000 time-series records. Each record includes traffic indicators such as average speed and congestion level, along with natural language descriptions related to emergencies, such as "On January 2, 2022, at 17:28, a minor traffic accident occurred on the S50 East Fifth Ring Road, Wufang Bridge was closed, traffic control was implemented on Lianshi West Road, and there was construction and a minor traffic accident on Jingliang Road...". To better capture abnormal traffic situations, we constructed a peak-hour subset based on this dataset, filtering records from 7:00–9:00 and 17:00–20:00 on weekdays based on the time information in the text descriptions, corresponding to typical morning and evening rush hours. This subset more prominently reflects the exacerbation of traffic congestion and the significance of event impacts, making it more suitable for evaluating the effectiveness of methods for generating abnormal traffic situations. The dataset was ultimately divided into a 70% training set, 10% validation set, and 20% test set.

[0215] 2. Evaluation Indicators

[0216] In terms of model evaluation, this invention employs two metrics: Mean Absolute Error (MAE) and Root Mean Square Error (RMSE). MAE measures the average deviation between the generated results and the actual values, directly reflecting the accuracy of overall traffic situation generation. RMSE, on the other hand, imposes a higher penalty on larger errors, thus being more sensitive to local peaks or sudden changes in traffic flow. By combining these two metrics, the model's performance in terms of overall accuracy and ability to detect local anomalies can be comprehensively evaluated. For all metrics, lower scores indicate better predictive performance.

[0217] 3. Implementation details

[0218] The key hyperparameter settings for the proposed abnormal traffic situation generation model, LIST-Diffusion, during training are as follows: a base learning rate of 0.00001; linear noise scheduling for the diffusion process with an initial value of 0.00085 and an ending value of 0.012, for a total of 1000 diffusion steps; a batch size of 4; and a UNet-based backbone structure containing 256 basic channels, 8 attention heads, and with the spatial Transformer module enabled. All experiments were conducted using a single NVIDIA RTX4090 graphics card (24GB VRAM) and a 16-core Intel Xeon Gold 6430 processor.

[0219] 4. Comparison Methods

[0220] DCRNN: This method uses diffusing convolution combined with recurrent neural networks for modeling, and can simultaneously capture the spatial dependence and temporal dynamic features in traffic networks.

[0221] GWN: This method is based on graph wavelet convolution to build a model and improves the ability to extract spatial features through multi-scale graph signal processing.

[0222] GMAN: This method uses a multi-attention mechanism to encode and decode the input sequence, thereby effectively modeling dynamic spatiotemporal dependencies.

[0223] DGCRN: This method combines dynamic graph structures with recurrent neural networks, enabling it to adapt to the spatial relationships of traffic networks that change over time and capture non-stationary dynamics.

[0224] AGCRN: This method automatically learns potential spatial adjacency relationships through an adaptive graph convolution mechanism, avoiding reliance on a fixed road network topology.

[0225] GCNGPT: This method combines graph convolution with a pre-trained language model to enhance the expressive power of spatiotemporal features by leveraging prior language knowledge.

[0226] GATGPT: This method integrates graph attention mechanism with GPT architecture, and has strong modeling ability in capturing complex spatiotemporal correlations.

[0227] STGCN: This method is based on spatiotemporal convolution operations, which directly processes spatial topology and time series features on the graph structure at the same time.

[0228] STSGCN: This method improves the accuracy of predictions by modeling local spatiotemporal dependencies through a spatiotemporal synchronization graph convolution module.

[0229] ASTGCN: This method introduces a spatiotemporal attention mechanism, which can selectively focus on key spatial locations and temporal segments in the input sequence.

[0230] MTGNN: This method utilizes a multivariate time series graph neural network structure to enhance the ability to capture long-distance spatiotemporal dependencies.

[0231] STG-NCDE: This method introduces neural network regular differential equations into the framework of graph neural networks to model complex spatiotemporal dynamic processes in the continuous time domain.

[0232]

[0233] Table 1: Experimental Results of the BjTT Dataset

[0234]

[0235] Table 2: Results of the Peak Period Subset Experiment

[0236] All data were evaluated. We highlighted the best results in bold, as shown in Tables 1 and 2.

[0237] As shown in Tables 1 and 2, existing technologies suffer from error accumulation as the number of generation steps T increases, making it difficult to guarantee the stability and reliability of the results. In contrast, the LIST-Diffusion model proposed in this invention achieves optimal results on both the complete dataset and peak-period subsets. By employing a progressive generation mechanism based on potential diffusion and combining it with a time-aware semantic injection module to guide the dynamic fusion of global and local traffic features, this invention maintains higher accuracy and consistency under complex and ever-changing abnormal traffic conditions, effectively overcoming the problem of existing methods failing over long periods and in multiple scenarios.

[0238] In summary, this invention innovatively proposes an abnormal traffic situation generation method (LIST-Diffusion) based on a large language model and a diffusion model, constructing a novel multimodal framework capable of generating realistic and interpretable traffic event situations. This invention utilizes world knowledge and historical event information from the large language model to expand concise traffic event descriptions into structured and semantically rich representations, capturing the formation, propagation, and interaction patterns of congestion with road network topology. Furthermore, these knowledge-enhanced descriptions are injected into a time-aware latent diffusion model. Through a time-aware semantic injection module, the fusion of global semantics and local spatiotemporal features is adaptively guided during the generation process, shaping the macroscopic traffic structure in the early stages of generation and accurately reconstructing microscopic details in the later stages.

[0239] This invention also designs a structured prompting strategy, integrating road topology, domain knowledge, and congestion diffusion patterns into the text description, thereby enhancing the semantic richness and interpretability of the generated results. Extensive experiments show that this invention outperforms existing methods on the BjTT dataset and peak-hour subsets, maintaining high spatiotemporal consistency and semantic alignment, thus achieving high-quality generation of abnormal traffic situations. This fully demonstrates the technological innovation and practical value of this invention.

[0240] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0241] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for generating abnormal traffic situations based on a large language model and a diffusion model, characterized in that, Includes the following steps: Obtain text descriptions of traffic events and historical traffic status, align the timestamps of the text descriptions and historical traffic status, and divide the data into training, validation, and test sets. Obtain the traffic network structure and create an adjacency matrix from it; A non-normal traffic situation generation model is constructed, trained using a training set, tuned using a validation set, and evaluated using a test set. The abnormal traffic situation generation model is optimized by using a weighted hybrid objective function that combines noise prediction loss and potential reconstruction loss. The noise prediction loss is used to fit the real noise in the diffusion process, and the potential reconstruction loss is used to improve the reconstruction accuracy of the potential representation. The abnormal traffic situation generation model includes: a structured text description generation module, a latent space encoding module, a forward diffusion module, a graph structure spatial relationship enhancement module, and a conditionally guided reverse denoising module. The denoising network in the reverse denoising module is a U-Net conditional noise prediction network based on a time-aware semantic injection mechanism. Structured text description generation module: Guided by prompts that integrate road connectivity knowledge, congestion diffusion theory and natural language instructions, it uses a large language model to transform traffic event text into structured semantic descriptions; Latent spatial coding module: compresses historical traffic states into a latent spatial representation using a variational autoencoder (VAE); Forward diffusion module: Constructs a Markov process that progressively injects Gaussian noise into the latent representation to simulate the evolution of traffic conditions from order to chaos, and generates a noisy latent representation as the starting point for reverse denoising; Graph structure spatial relationship enhancement module: Based on the road network adjacency matrix and graph convolutional network, the road topology is embedded into the latent representation to enhance the spatial consistency of the generated results; Conditionally guided reverse denoising module: Guided by structured semantic description, it uses conditional Markov chains to gradually denoise the latent representation, so that the generation process satisfies the semantic constraints of traffic events while maintaining the authenticity and diversity of traffic situations. U-Net Conditional Noise Prediction Network Based on Time-Aware Semantic Injection Mechanism: It adopts the U-Net structure and introduces a time-aware semantic injection mechanism to dynamically balance global trends and local details in different denoising stages. It achieves multi-scale alignment of semantics and latent representation through cross-attention mechanism, and finally obtains traffic situation that conforms to event semantics and road network topology.

2. The method for generating abnormal traffic situations based on a large language model and a diffusion model according to claim 1, characterized in that, The construction of a structured text description generation module includes the following steps: A set of prompt words incorporating road connectivity knowledge, congestion diffusion theory, and natural language instructions is constructed. Road connectivity knowledge is derived from a road network adjacency matrix, where a value of 1 indicates a direct connection between two roads. Non-zero values ​​are extracted and converted into textual road pairings. Congestion diffusion theory describes the propagation mechanism of congestion between affected road segments and their adjacent segments, constrained by topological distance, directionality, and event severity. Road connectivity knowledge is integrated into the prompt words. Natural language instructions specify the task objectives, guiding the large language model to extract structured information from the original event text. These tasks include key event attribute identification, related event clustering, and prediction of potential congestion propagation range. The designed prompt words are used in the structured text description generation unit to transform the input event text R and generate a structured text description S, defined as: （1） in, This indicates a prompt to construct the function. Represents the response function of a large language model; The structured text description S is passed through a text encoder. Encoded as semantic embedding vector : （2） Embedded vector This will be used as a semantic condition to guide the downstream traffic situation generation process.

3. The method for generating abnormal traffic situations based on a large language model and a diffusion model according to claim 2, characterized in that, Constructing the latent space coding module includes the following steps: Each historical traffic state in the dataset is represented as a tensor. Where D represents the feature dimension, This represents the total number of road segments arranged in a two-dimensional grid, with each spatial location corresponding to a specific road segment. This spatial organization maps the traffic network into a "pseudo-image," enabling each frame to serve as a three-channel input for the potential diffusion model. The diffusion process takes place in the compressed latent space, where each traffic state is first mapped to a latent representation via a variational autoencoder. : （3） （4） in, The encoder module represents the high-dimensional traffic tensor y that transforms it into a low-dimensional latent representation. ; It's a decoder, responsible for... Reconstruct the original traffic image, where This represents the total number of spatial locations.

4. The method for generating abnormal traffic situations based on a large language model and a diffusion model according to claim 3, characterized in that, Constructing the forward diffusion module includes the following steps: potential tensor Flattened into a node sequence The forward diffusion process from the reshaped latent representation Begin; Define a forward Markov process to simulate the gradual disruption of spatial traffic conditions, within T time steps. Gradually add Gaussian noise: （5） in The cumulative product representing the noise scheduling is implemented through reparameterization at time step t, where the actual noise injection operation is achieved. （6） in Is with The unit covariance matrix for dimension matching. The injected noise, and its potential representation at time step T. It is also sampled through the same forward diffusion process and used as the starting point for subsequent denoising processes.

5. The method for generating abnormal traffic situations based on a large language model and a diffusion model according to claim 4, characterized in that, Constructing a graph structure spatial relationship enhancement module includes the following steps: The road network topology is introduced into the latent space using a graph convolutional network (GCN). The road graph is composed of an adjacency matrix. The formula for calculating its normalized adjacency matrix is ​​as follows: （7） in For degree matrix, I The identity matrix is ​​then used as a noisy latent representation. As input node features, two GCN layers are applied sequentially: （8） in , For learnable weights, ReLU and Sigmoid are used as activation functions. Finally, the enhanced features are fused with the original latent representation through residual connections. （9） Received It will be used in the subsequent denoising process.

6. The method for generating abnormal traffic situations based on a large language model and a diffusion model according to claim 5, characterized in that, Constructing a conditionally guided inverse denoising module includes the following steps: The backdiffusion process starts from the spatial latent representation input. Stepwise reconstruction of the initial latent representation And subject to external semantic conditions Guided by a conditional Markov chain, at each time step t, the previous state is sampled from a Gaussian distribution. : （10） in, This represents the mean term predicted by the conditional noise prediction network at time step t, used to guide the gradual reconstruction of the latent representation from the noisy state. The calculation is as follows: （11） in This is a U-Net conditional noise prediction network based on a time-aware semantic injection mechanism, used to predict noise added at time step t. The variance term obtained through learning is used to control the noise intensity at each sampling step and is determined by noise scheduling. （12） Subsequently, the denoised latent representation Obtained through the following formula: （13） in, Standard Gaussian noise is used to introduce random perturbations during the back diffusion process; As the time step t decreases from T to 0, the iterative process progressively optimizes the latent representation, ultimately generating a latent representation that integrates spatial topology and semantic intent. Finally, potential representation By pre-trained decoder Decode to generate the final traffic situation.

7. The method for generating abnormal traffic situations based on a large language model and a diffusion model according to claim 6, characterized in that, U-Net Conditional Noise Prediction Network Based on Time-Aware Semantic Injection Mechanism It includes the following steps: Conditional noise prediction network It adopts the U-Net architecture and combines it with a time-aware semantic injection mechanism to input... They are projected into two distinct vectors: （14） （15） in, and It consists of two parallel multilayer perceptrons, which extract global semantic trends and local structural details, respectively. and These represent two types of semantic embeddings, and the current back-diffusion time step t is embedded into a time vector. After processing by a one-dimensional convolution and feedforward network, dynamic weights are generated to control the balance between global and local semantics. （16） Where W is the learnable parameter matrix, It is a one-dimensional convolution that extracts time-related features, and the final modulation vector is calculated as follows: （17） This is then injected into the intermediate hidden state h of the U-Net. （18） Where h represents the current hidden state of the U-Net. This is a lightweight convolutional network used to adjust semantic vectors to match the dimensions of the hidden states, and to semantically modulate the vectors. Selected modules are repeatedly injected into the encoder and decoder to achieve semantic control across resolutions; Semantic conditions are also injected through a cross-attention mechanism in the decoder layer to achieve content-aware alignment. The attention calculation is as follows: （19） Among the queries From the decoder's current hidden state, key Sum Then, based on the conditional input Export, learnable matrix , and Features are projected into a shared latent space for matching; The objective loss function of the non-normal traffic situation generation method based on large language models and diffusion models is a weighted mixed loss: （20） in, and The loss weighting coefficient represents the noise prediction loss. Used to measure the conditional noise prediction network and the actual injected noise in the forward process. Mean square error between: （21） in, This represents the prediction result of the conditional noise prediction network at time step t. These are the weighting coefficients for each time step. Potential Reconstruction Loss It is necessary to back-calculate the estimate of the original latent representation based on the model's predicted noise. : （22） in, For the noisy latent representation at time step t, The noise scheduling parameters are then defined, and the L1 error is defined as the potential reconstruction loss: （23） The final training loss is a trade-off between the two, aiming to balance the stability of the noise prediction with the accuracy of the underlying representation.

Images