Traffic accident detection method and system based on combined scene video generation and decoupled representation learning

Abstract

The application relates to a traffic accident detection method and system based on combined scene video generation and decoupling representation learning. First, real monitoring accident samples are combinedly modeled according to vehicle categories, collision topologies, road geometries and environmental conditions, and high-value rare accident scenes are selected through risk perception sub-module coverage optimization. Then, structured scene analysis, scene compatibility evaluation, camera view alignment script generation and image condition video generation are performed based on real monitoring scene images, a closed-loop regeneration mechanism is constructed by means of semantic consistency verification, and verified synthetic accident videos are obtained. Finally, a decoupling accident detection model containing a vehicle representation branch and an accident dynamics representation branch is trained by using real videos and synthetic videos, and robust identification of long-tail rare traffic accidents is realized. The application can improve the high-risk rare combination coverage under limited generation budget, reduce generation noise, and enhance the generalization to unseen vehicle-accident combinations.

Description

Technical Field

[0001] This invention belongs to the field of intelligent transportation technology, specifically relating to a traffic accident detection method and system based on combined scene video generation and decoupled representation learning. Background Technology

[0002] Automatic traffic accident detection is one of the key foundational capabilities of intelligent transportation systems. Its goal is to quickly identify accident events from large-scale roadside surveillance videos and issue alerts to management departments. Existing technologies typically employ 3D convolutional networks, spatiotemporal Transformers, video self-supervised pre-trained backbone networks, or anomaly detection methods to model surveillance videos, achieving certain results on common accident types and common vehicle combinations.

[0003] However, real-world traffic accidents exhibit a significant long-tail characteristic in the joint distribution of vehicle type, collision topology, road geometry, and environmental conditions. Accidents involving special vehicles such as school buses, prison vans, car carriers, tank trucks, and ambulances often involve high risk, high attention, and high handling requirements, but the samples in existing training data are extremely scarce, or even completely absent. Due to insufficient training samples, existing detection models are prone to misinterpreting vehicle appearance or background texture as accident discrimination clues, resulting in poor generalization ability for unseen vehicle-accident combinations.

[0004] To alleviate the problem of insufficient long-tail samples, existing solutions have attempted to expand the training set using general data augmentation, simulator-generated data, or random text-driven video generation. However, simply increasing the number of synthetic samples cannot guarantee effective coverage of key scenarios: on the one hand, without systematic planning for rare combinations of targets, the generation budget may be wasted on low-value or highly repetitive scenarios; on the other hand, if the generation process is not constrained by real monitoring perspectives, road topology, and vehicle driving constraints, the generated videos are prone to semantic noise such as "no accident occurred," "incompatible vehicle type," "incompatible collision topology," and "unreasonable spatiotemporal relationships," which in turn weakens the performance of downstream detection models.

[0005] Furthermore, existing detection models often encode vehicle appearance information in combination with collision dynamics patterns, which can easily lead to appearance shortcut learning, especially in long-tail scenarios involving special vehicles. For example, the model may directly bind appearance features such as the yellow body of a school bus or the multi-layered contours of a car carrier to accident labels, rather than truly learning collision patterns such as rear-end collisions, T-shaped collisions, head-on collisions, side impacts, and rollovers. This results in a significant drop in detection performance once the vehicle's appearance distribution changes.

[0006] Therefore, there is an urgent need for a traffic accident detection scheme that can prioritize the supplementation of high-value rare accident combinations under a limited generation budget, generate highly semantically consistent synthetic accident videos based on real monitoring scenarios under constraints, and further enhance the generalization ability of unseen combinations through decoupled representation learning. Summary of the Invention

[0007] The purpose of this invention is to provide a traffic accident detection method and system based on combined scene video generation and decoupled representation learning, so as to solve the problems of insufficient long-tail rare accident combination samples, large semantic noise in synthetic videos, and insufficient generalization ability of accident detection models for unseen vehicle-accident combinations in the existing technology.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: a traffic accident detection method based on combined scene video generation and decoupled representation learning, comprising:

[0009] S1. Obtain real traffic monitoring videos and scene images corresponding to the real traffic monitoring videos. Perform combined annotations on accident samples according to vehicle type, collision topology, road geometry and environmental conditions to construct an accident scene space.

[0010] S2. Based on the sample frequency and risk score of each scenario in the accident scenario space, perform sub-model coverage optimization for risk perception to obtain a set of target rare accident scenarios to be generated.

[0011] S3. Perform structured scene analysis on the scene image to obtain road topology, lane direction, traffic control facilities, camera view, vehicle driving path and conflict area, and evaluate the scene compatibility of the target rare accident scene set based on the analysis results.

[0012] S4. For the image-scene pair that passes the scene compatibility assessment, generate an accident script text that is aligned with its camera viewpoint, and input the scene image and the accident script text into the image conditional video generation model to obtain candidate accident videos.

[0013] S5. Perform semantic consistency verification on the candidate accident videos, and execute script correction and closed-loop regeneration on the candidate accident videos that do not reach the consistency threshold to obtain a verified synthetic accident video set.

[0014] S6. The real traffic monitoring video and the verified synthetic accident video set are jointly input into the accident detection model for training. The accident detection model includes a shared spatiotemporal feature backbone network, a vehicle representation branch and an accident dynamics representation branch. The classification result is output by the accident dynamics representation branch.

[0015] S7. Input the surveillance video to be detected into the trained accident detection model, and output the accident probability and / or accident alarm results.

[0016] A traffic accident detection system includes: a data acquisition and combination annotation module for acquiring real traffic monitoring videos and scene images, and performing combination annotation on accident samples according to vehicle category, collision topology, road geometry, and environmental conditions; a target scene selection module for optimizing and selecting a set of target rare accident scenes based on scene frequency, risk score, and risk perception sub-module coverage; a structured scene parsing module for performing structured scene parsing on the monitoring scene images and outputting road topology, lane direction, traffic control facilities, camera view, vehicle drivable paths, and conflict areas; and a compatibility evaluation module for assessing the geometric compatibility between the target accident scene and the monitoring scene. The system performs joint evaluations of feasibility, vehicle adaptability, and contextual consistency; a script generation module generates accident script text aligned with the camera's viewpoint; a video generation module generates candidate accident videos based on monitoring scene images and accident script text; a semantic verification module scores the semantic consistency of candidate accident videos and triggers script correction and closed-loop regeneration for candidate accident videos that do not meet the threshold requirements; a model training module trains a decoupled accident detection model based on real monitoring videos and validated synthetic accident videos; and an online inference module outputs the accident probability or accident alarm result for the monitoring video to be detected based on the trained decoupled accident detection model.

[0017] Compared with the prior art, the present invention has at least the following beneficial effects:

[0018] (1) By constructing a combined accident scenario space and introducing risk perception sub-model coverage optimization, this invention can prioritize the coverage of rare accident combinations that are high-risk, low-frequency, and more sensitive to actual traffic safety under a fixed generation budget, thus avoiding budget waste caused by simple random generation.

[0019] (2) By performing structured scene parsing, compatibility assessment and camera view alignment script generation on real surveillance scene images, the present invention makes the generation process subject to road topology, lane direction, vehicle path and conflict area constraints at the same time, which improves the geometric consistency and visual credibility between the synthesized accident video and the real surveillance scene.

[0020] (3) By introducing video semantic verification and closed-loop regeneration mechanism, the present invention can automatically remove semantic error samples and effectively reduce the negative impact of generation noise such as “the accident did not occur”, “the vehicle type does not match”, and “the collision topology does not match” on downstream training.

[0021] (4) By constructing vehicle representation branches and accident dynamics representation branches, and combining contrast constraints, discorrelation regularization and appearance perception hard sample mining, this invention can significantly reduce the interference of vehicle appearance on accident judgment and improve the robustness and transferability of detection of unseen vehicle-accident combinations.

[0022] (5) The present invention achieves an AUC of 78.6% on the rare accident test set, which is a significant improvement over the baseline method with an AUC of 59.5% trained only with real videos. It also reduces the AUC gap between common accidents and rare accidents from 28.1 percentage points to 11.7 percentage points, thus verifying the effectiveness of the present invention in long-tail traffic accident detection tasks. Attached Figure Description

[0023] Figure 1 This is a flowchart of the overall method of the present invention.

[0024] Figure 2 Flowchart for selecting risk perception sub-module coverage.

[0025] Figure 3 This is a system deployment structure diagram of the present invention.

[0026] Figure 4 This is a diagram demonstrating the effect of an example. Detailed Implementation

[0027] This invention proposes a traffic accident detection method based on combined scene video generation and decoupled representation learning, employing a phased technical approach: First, a combined accident scene space composed of vehicle category, collision topology, road geometry, and environmental conditions is constructed from real-world monitored accident samples. Risk perception sub-modal coverage is optimized based on scene frequency and risk score, selecting the most valuable rare accident combination to generate. Second, for each real-world monitored scene image, a multimodal large model is used to generate a structured scene description. Based on this, geometric feasibility, vehicle adaptability, and contextual consistency evaluations are performed on candidate rare accident combinations. Only compatible image-scene pairs generate accidents aligned with the camera's viewpoint. The script text is then used as input to an image-conditional video generation model to generate candidate accident videos. A video semantic verification model is then used to extract whether an accident occurred, the types of vehicles involved, the collision topology, and the collision timing, and to calculate semantic consistency scores. A closed-loop regeneration mechanism is used to retain high-quality synthetic accident videos. Finally, a decoupled accident detection model is jointly trained using real surveillance videos and verified synthetic accident videos. This model includes a shared spatiotemporal feature backbone network, a vehicle representation branch, and an accident dynamics representation branch. Through contrastive learning, uncorrelation constraints, and hard sample mining based on appearance perception, appearance shortcut learning is suppressed. During the inference phase, only the accident dynamics representation is used to output the accident probability.

[0028] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the following specific embodiments are used to explain the technical principles and implementation paths of the present invention, and are not intended to limit the scope of protection of the present invention; equivalent substitutions or modifications made by those skilled in the art to the order of steps, model implementation forms, parameter settings and module divisions without departing from the spirit and substance of the present invention should all fall within the scope of protection of the present invention.

[0029] To facilitate understanding of this invention, some terms and symbols are explained uniformly below: Represents a set of vehicle categories; Represents the set of collision topologies; Represents the geometric set of roads; Represents a set of environmental conditions; Represents the space of the accident scene; This represents a specific accident scenario; Representing a scene Sample frequency in the training corpus; Indicates the rare threshold; Representing a scene Risk score; This represents the set of rare accident scenarios for the target. Indicates the budget for the target scenario combination; Represents images of a monitored scene; This represents the results of structured scene parsing; This represents the accident script text; Indicates candidate accident videos; The semantic consistency score represents the candidate accident video. This represents a backbone network with shared spatiotemporal features; Indicates vehicle characteristics; Represents the dynamic characterization of the accident; Indicates the probability of an accident occurring; This represents the total training loss.

[0030] Involving vehicles In areas where vehicles are in the same combination and counted repeatedly due to different orders, it is preferable to use an ascending order of vehicle category numbers. For example, if a collision between a car and a school bus is recorded as (sedan, school bus), then it will no longer be recorded separately as (school bus, sedan).

[0031] When it comes to "collision topology", categories such as rear-end collision, T-collision, oncoming collision, side collision, and rollover are preferred, but not limited to. When it comes to "road geometry", categories such as urban intersection, highway straight road, highway ramp, and tunnel entrance are preferred, but not limited to. When it comes to "environmental conditions", categories such as sunny daytime, nighttime, rainy day, and foggy day are preferred, but not limited to.

[0032] like Figure 1 As shown, the method of the present invention unfolds in accordance with the process of scene planning, constrained synthetic video generation, and decoupling model training and online inference.

[0033] (I) Construction of Combined Accident Scene Space and Planning of Target Scene

[0034] This section mainly explains how to abstract a combinatorial accident space that can be used for scenario planning from existing real-world accident monitoring corpora, and further select the most worthwhile target rare accident scenarios to generate.

[0035] Step A1: Define the composite accident scene space. Any traffic accident scene is characterized by four discrete factors: vehicle category, collision topology, road geometry, and environmental conditions. Preferably, the accident scene space satisfies the following equation:

[0036] ;

[0037] in, Represents the space of the accident scene; Represents a set of vehicle categories; Represents the set of collision topologies; Represents the geometric set of roads; Represents the set of environmental conditions; any scenario configuration is denoted as ,in These represent the first and second participating vehicle categories, respectively. Represents the collision topology. Represents road geometry. Indicates environmental conditions.

[0038] Step A2: Count the frequency of scenarios and identify rare scenarios. Perform combined annotation on each accident sample in the real-world surveillance accident corpus, and count the configuration of each scenario. Sample frequency And identify rare accident scenarios according to the following formula:

[0039] ;

[0040] in, This represents a set of rare accident scenarios; Representing a scene The number of times it appears in the training corpus; This represents the rarity threshold. Preferably, It can be set to 5, but is not limited to that value.

[0041] Step A3: Calculate the risk score. To prioritize the completion of more valuable rare scenarios within a limited generation budget, this implementation introduces a risk score composed of vehicle combination severity, collision topology hazard, and road-environment vulnerability:

[0042] ;

[0043] in, Representing a scene Risk score; The vehicle combination severity function reflects the severity of the potential consequences that different vehicle combinations may cause. This represents the collision topology hazard function, reflecting the inherent hazard level of different collision topologies; This represents the road-environment vulnerability function, reflecting the amplification effect of road geometry and environmental conditions on accident consequences; These represent the weighting coefficients for the three sub-items mentioned above. Preferably, Severity, Hazard, and Vulnerability can be quantified into ratings of 1 to 3 levels, or a more granular continuous rating can be used.

[0044] Step A4: Perform risk perception sub-module coverage optimization. For example... Figure 2 As shown, by constructing a coverage reward function and using a greedy strategy to iteratively select candidate rare accident scenarios with the largest marginal gain, a set of target rare accident scenarios is obtained. To avoid excessive scenario duplication due to simple sorting by risk score, this implementation further defines the rare scenario coverage reward function:

[0045]

[0046] in, This represents the set of rare incident scenarios that have been selected. Represents a set Collection of rare accident scenarios Coverage benefits; This represents any rare incident scenario that needs to be covered. Representing a scene Risk score; Representing a scene With scene The degree of overlap between them.

[0047] Preferably, the scene overlap kernel It can be represented as:

[0048]

[0049] in, [·] is an indicator function; it takes the value 1 if the two scenes have the same value in the corresponding dimension, and 0 otherwise. Representing a scene In dimensions The value of ; The larger the value, the more similar the two scenarios are in more dimensions.

[0050] For the currently selected set Add candidate scenarios to it The marginal gain can be expressed as:

[0051]

[0052] in, Indicates the selected set Add candidate scenarios under certain conditions The resulting additional coverage benefits. In practice, this can be seen from... The marginal gain is calculated for each scenario, and a greedy strategy is used to iteratively select the candidate scenario with the largest marginal gain until the preset budget is reached. .

[0053] Step A5: Output the target scenario combination. After execution, a set of target rare accident scenarios is obtained. It takes into account both the level of risk and the diversity of scenarios, thus providing higher-value generation targets for the subsequent constrained video generation stage.

[0054] (ii) Generation of constrained accident videos based on real monitoring scenarios

[0055] This section mainly explains how to generate semantically consistent accident videos under the geometric and viewpoint constraints of real surveillance scene images, and improve the quality of synthetic samples through semantic verification closed loop.

[0056] Step B1: Perform structured scene parsing on the real surveillance scene images. For each surveillance scene image Ic, use a multimodal large model to extract road topology, lane direction, traffic control facilities, camera view, vehicle drivable paths, and candidate conflict regions. The parsing process can be represented as follows:

[0057]

[0058] in, This represents the surveillance scene image to be parsed; Represents the analytical operators for multimodal large models; This indicates a parsing prompt template; Representing an image The corresponding structured scene description. The structured scene description includes at least the scene type, road topology, number and direction of lanes, traffic control facilities, camera pitch and orientation, set of drivable vehicle paths, and set of candidate conflict areas; preferably, It can be organized as JSON, key-value pairs or other machine-readable structures, and its fields include at least scene_type, lane_topology, lane_direction, traffic_control, camera_pose, drivable_paths and conflict_zones.

[0059] Step B2: Perform a compatibility assessment between the target accident scene and the current monitoring scene. To ensure that the generated video meets the scene geometry constraints and vehicle appearance scene constraints, the following compatibility score can be used:

[0060]

[0061] in, Represents images of monitored scenes With the target accident scenario Compatibility score between them; The geometric feasibility score measures the degree of consistency between the collision topology and the road topology, the drivable path of the vehicle, and the conflict area. For example, a T-shaped collision should correspond to an intersecting path, and a rear-end collision should correspond to a path in the same direction. The vehicle suitability score measures the degree of consistency between the participating vehicle category and road grade, lane width, and scene attributes, such as whether large vehicles are suitable for the current road grade and lane width. The context consistency score measures the degree of consistency between the road geometry and environmental semantics in the target accident scenario and the monitoring scenario context. For example, whether the road geometry in the target accident scenario is consistent with the current scenario parsing results. This represents the weighting coefficient of the three scores, preferably satisfying... .

[0062] Preferably, when the compatibility score satisfies the following formula, the corresponding scenario is considered a generateable scenario:

[0063]

[0064] in, In the scene image The following is a set of target scenarios that have passed compatibility assessment; This indicates the compatibility threshold.

[0065] Step B3: Generate accident script text aligned with the camera's viewpoint. For each compatible image-scene combination... The accident script text is generated using a large language model.

[0066]

[0067] in, Represents the accident script text aligned with the camera's viewpoint; LLM(·) represents the large language model generation operator; This represents a structured scene description; Indicates the target accident scenario; This indicates the script generation prompt template. The accident script text describes the vehicle's entry direction from the monitoring screen, its driving trajectory, collision location, collision posture, collision sequence, and changes in the vehicle's state after the collision. It also uses the monitoring screen reference frame to express spatial relationships. For example, it can describe that "a yellow school bus entered the intersection from the upper right of the screen along the main road, and a truck entered the conflict area from the left side of the screen. A T-shaped collision occurred in the center of the intersection, and the cargo on the upper layer of the truck shook significantly after the collision."

[0068] Step B4: Perform image-conditional video generation. This involves generating images of the monitored scene. With accident script text A conditional video generation model based on common input images generates candidate accident videos.

[0069] ;

[0070] in, Represents the scene image With accident script text Generated candidate accident videos; (·) represents the image-conditional video generation model. Preferably, for each image-scene combination, a video generation model is generated. Several candidate accident videos were selected to increase the probability of obtaining high-quality samples. This indicates the number of candidate videos for each combination.

[0071] Step B5: Perform video semantic verification. Because the generative model may have issues such as vehicle category bias, collision topology errors, or accidents that did not actually occur, semantic verification is performed on each candidate accident video. Perform semantic extraction:

[0072]

[0073] in, Indicates the first One candidate accident video, This represents a video semantic verification model. This indicates that the video semantic verification model is used to select candidate accident videos. The semantic results extracted include at least whether an accident occurred, the types of vehicles involved, the collision topology, and the time of the collision.

[0074] Calculate semantic consistency scores for candidate accident videos:

[0075]

[0076] in, Indicates candidate accident video The semantic consistency score; The consistency index indicates whether the accident actually occurred. It can be set to 1 if the accident has actually occurred in the candidate video, and 0 otherwise. The vehicle consistency index, which indicates whether the participating vehicle categories match the target accident scenario, can be calculated based on the degree of matching between the identified participating vehicle categories and the target scenario vehicle categories. A topology consistency index indicating whether the collision topology is consistent with the target accident scenario; This indicates the time deviation between the collision time identified by the verification model and the collision time set in the target script. This represents the time deviation normalization parameter; This represents the weighting coefficient of each consistency indicator.

[0077] Step B6: Perform closed-loop regeneration. When At that time, Retained as verified synthetic accident video; when At that time, based on the semantic verification results The discrepancies between the accident script text and the target scene are analyzed, and the content related to vehicle descriptions, entry directions, collision locations, or timing information is corrected. The image-conditional video generation model is then invoked again to generate new video candidates. This closed loop can be repeated until a threshold is reached. Or reach the maximum number of iterations ,in This represents the semantic consistency threshold. This represents the maximum number of closed-loop iterations.

[0078] (III) Decoupling Accident Detection Model Training and Online Inference

[0079] This section mainly explains how to jointly train a traffic accident detection model using real videos and verified synthetic videos, and how to improve the generalization ability to unseen combinations through decoupled representation learning.

[0080] Step C1: Construct a shared spatiotemporal feature backbone network and a dual-branch representation structure. For the input surveillance video... First, intermediate spatiotemporal features are extracted from the shared spatiotemporal feature backbone network, and then corresponding representations are extracted from the vehicle branch and the accident dynamics branch respectively, as follows:

[0081] , , , ;

[0082] in, This represents a backbone network with shared spatiotemporal features; This represents the intermediate spatiotemporal features extracted by the shared spatiotemporal feature backbone network; Indicates the vehicle representation branch; This represents the branch of accident dynamics characterization; Represents the vehicle representation vector; Represents the accident dynamics characterization vector; and These represent the weight parameters and bias parameters of the classification head, respectively; Represents the Sigmoid function; This represents the probability of an accident occurring. Preferably, during the inference phase, only the shared spatiotemporal feature backbone network, the accident dynamics branch, and the classification head are retained, thereby suppressing the direct involvement of vehicle appearance information in the judgment.

[0083] Step C2: Construct a bi-branch contrast constraint. For the accident dynamics branch, it is desirable for samples with the same collision topology but different vehicle combinations to be close to each other in the dynamics representation space; for the vehicle branch, it is desirable for samples with the same vehicle combination but different collision topology to be close to each other in the vehicle representation space. The following contrast loss is used:

[0084] ,

[0085] ;

[0086] in, The accident dynamics branch contrast loss is used to constrain samples with the same collision topology but different vehicle combinations to approach each other in the accident dynamics representation space. This represents the vehicle branch contrast loss, which is used to constrain samples with the same vehicle combination but different collision topologies to move closer to each other in the vehicle representation space. Represents the similarity function; and These represent the temperature parameters for the dynamics branch and the vehicle branch, respectively; superscript Indicates anchor sample, This represents a sample that forms a positive sample pair with the anchor sample. This represents the index of the candidate sample in the denominator.

[0087] Step C3: Execute branch solution-related constraints. To prevent vehicle appearance information from being leaked into the accident dynamics branch, the following solution-related regularization term is preferably introduced:

[0088] ;

[0089] in, and These represent the vehicle characterization matrix and the accident dynamics characterization matrix in the current batch, respectively. Indicates the transpose operation; This represents the Frobenius norm. The smaller this term is, the weaker the statistical correlation between the two branches.

[0090] Step C4: Construct hard sample mining based on vehicle appearance distance. Preferably, first use a pre-trained visual encoder to calculate the average appearance embedding for each vehicle category, and define the vehicle category-level appearance distance:

[0091] ;

[0092] in, Indicates vehicle category and Category-level appearance distance between them; and These represent the average appearance embeddings of vehicle categories extracted by the pre-trained visual encoder; when A larger difference indicates a significant visual difference between the two types of vehicles.

[0093] Furthermore, for the two video samples and Its video-level appearance distance can be expressed as:

[0094] ;

[0095] in, Indicates video samples and The video-level visual distance between them; and These represent video samples respectively. The first and second participating vehicle categories; and These represent video samples respectively. The first and second participating vehicle categories.

[0096] In the dynamics branch, positive samples can be preferentially selected from those with significant appearance differences but the same topology, while negative samples with similar appearances but different collision topologies are given greater weight, resulting in the following hard sample loss:

[0097]

[0098]

[0099] in, This represents the accident dynamics branch loss after introducing hard sample mining; Indicates the anchor sample Dynamic characterization of positive samples with the same collision topology but significant appearance differences; Indicates the first The weights of each negative sample; Represents the hard sample weight adjustment coefficient; when When the weight is small, it indicates that the negative sample and the anchor sample are similar in appearance, and they should be given greater weight to suppress appearance shortcut learning. The vehicle branch can adopt a symmetrical hard sample construction strategy.

[0100] Step C5: Construct the total training loss and perform training. Preferably, the total training loss can be expressed as:

[0101] ,

[0102] ;

[0103] in, Indicates the total training loss; Indicates the loss from accident detection; This represents the hard sample loss for vehicle branches; Indicates the loss associated with branch solutions; These represent the weighting coefficients for each loss term; Represents the true label of the surveillance video, when the accident sample is taken. Non-accident samples ; This represents the probability of an accident occurring. During training, real videos can be primarily used for accident detection and supervision, while validated synthetic videos are mainly used to construct a balanced composite comparison sample. During inference, the accident probability and alarm results are output only based on the accident dynamics branch.

[0104] (iv) System Implementation and Deployment Methods

[0105] At the system implementation level, this invention can be deployed as a central server, edge devices, or a cloud-edge collaborative system. For example... Figure 3 As shown, the system of the present invention includes a data acquisition and combination annotation module, a target scene selection module, a structured scene parsing module, a compatibility evaluation module, a script generation module, a video generation module, a semantic verification module, a model training module, and an online inference module.

[0106] The system comprises several modules: a data acquisition and annotation module, a target scene selection module, and a model training module. The data acquisition and annotation module is responsible for acquiring monitoring videos and scene images from traffic monitoring systems, historical accident databases, or open video sources, and completing combined annotations of vehicle categories, collision topologies, road geometry, and environmental conditions. The target scene selection module is responsible for performing risk perception sub-model coverage optimization and outputting a set of rare accident scenes with high generation priority. The structured scene parsing module and compatibility evaluation module are responsible for ensuring that the subsequent generation process meets the constraints of real monitoring perspective and road geometry. The script generation module, video generation module, and semantic verification module together constitute a closed-loop generation subsystem. The model training module is responsible for completing the training of the decoupled accident detection model. The online inference module receives real-time monitoring videos and outputs accident probabilities or alarm results.

[0107] In edge deployment scenarios, only the online inference module and the necessary scene parsing results can be cached and deployed on the roadside device to reduce latency; in central server deployment scenarios, computationally intensive steps such as target scene selection, batch video generation, and model training can be performed on the server side.

[0108] Example

[0109] The following embodiments illustrate a feasible implementation of the present invention in specific data configurations, parameter configurations, and experimental environments. It should be understood that the specific model names, parameter values, data scales, and evaluation metrics in the embodiments are only used to verify the effectiveness of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0110] Example 1: Instantiation of datasets and combinatorial spaces;

[0111] In this embodiment, the real accident and non-accident videos are derived from a large-scale roadside monitoring accident dataset. Preferably, the real dataset contains 3,556 accident videos and 4,102 non-accident videos, covering types such as single-vehicle accidents, vehicle-to-vehicle accidents, non-motorized vehicle accidents, and pedestrian accidents; the training set contains 2,489 accident videos and 2,870 non-accident videos, the validation set contains 356 accident videos and 410 non-accident videos, and the test set contains 711 accident videos and 822 non-accident videos.

[0112] To evaluate the performance of this invention in truly rare scenarios, a RareCrash-Test test set was further constructed. Preferably, this test set contains 250 videos, of which 96 are positive samples of rare accidents and 154 are negative samples of non-accidents; the positive samples of rare accidents include events that rarely or never occur in the training set, such as ambulance collisions, heavy truck-bus collisions, tanker truck collisions, school bus accidents, and car carrier accidents.

[0113] In this embodiment, the vehicle category set V is instantiated into 11 vehicle types: sedans, SUVs, trucks, buses, motorcycles, ambulances, tank trucks, water trucks, school buses, prison vans, and car carriers; the collision topology set A is instantiated into 5 types: rear-end collisions, T-collisions, head-on collisions, side impacts, and rollovers; the road geometry set C is instantiated into 4 types: urban intersections, highway straight sections, highway ramps, and tunnel entrances; and the environmental conditions set E is instantiated into 4 types: sunny daytime, nighttime, rainy day, and foggy day.

[0114] Under the condition of normalized vehicle pair ranking, this embodiment can generate C(11+1, 2)×5×4×4=5280 unique accident combination configurations, where C(11+1, 2) represents the number of unordered vehicle pairs after considering similar and dissimilar combinations from the 11 vehicle categories. Statistical analysis shows that the proportion of combinations with sample coverage in existing real monitoring data is less than 5% of all combinations, indicating that the accident combination space has significant sparsity.

[0115] Example 2: Target scene selection, video generation, and training configuration;

[0116] In this embodiment, the rare threshold is... The risk score is set to 5, meaning that any scenario configuration that appears less than 5 times in the training set is considered a rare scenario; the weighting coefficient in the risk score is... and All are set to 1 / 3 to ensure that the three dimensions of vehicle combination severity, collision topology hazard and road-environment vulnerability have the same weight in the initial implementation; the target scenario combination budget K is set to 65.

[0117] In terms of scene image collection, preferably, 120 real CCTV scene images are collected, covering typical roadside monitoring scenes such as intersections, highways, ramps and tunnel entrances. First, structured scene parsing is performed on each scene image to obtain scene descriptions for subsequent compatibility assessment and script generation.

[0118] Regarding the configuration generation, preferably, it is generated for each image-scene combination. One candidate accident video, scene compatibility threshold. Set to 0.6, semantic consistency threshold Set the value to 0.7; trigger script correction and loop closure regeneration for samples with insufficient semantic consistency. Ultimately, in this embodiment, approximately... When rare incident synthetic videos that pass semantic verification are averaged across 65 target combinations, each combination corresponds to approximately 23 verified synthetic videos.

[0119] For model training, preferably, a pre-trained video backbone network is used as the shared spatiotemporal feature backbone network, and both the vehicle branch and the accident dynamics branch use two-layer perceptron projection heads; the AdamW optimizer is used with a learning rate of 1×10^-4, a batch size of 16, and weighting coefficients in the total training loss. The hard sample weight adjustment coefficients were set to 0.5, 0.3, and 0.1 respectively. Set to 2.0, input video is uniformly sampled at 16 frames and resolution 224×224.

[0120] Example 3: Comparison of main results and verification of validity;

[0121] Using the data and parameter configurations from Examples 1 and 2, the present invention was trained and tested, and compared with a baseline method trained only with real videos and a randomly generated augmentation method. Preferably, AUC, AP, and F1 were used as evaluation metrics. For ease of explanation, typical results are summarized in Table 1.

[0122] Table 1 Comparison of Main Results

[0123] method Common accidents AUC (%) Rare incidents AUC(%) Rare incident AP (%) Rare accidents F1 (%) Common-Rare AUC Difference VideoMAE (Real Data Only) 87.6 59.5 52.7 51.0 28.1 VideoMAE + Random Generation Enhancement 88.7 65.4 58.8 56.5 23.3 Method of the present invention 90.3 78.6 72.3 69.8 11.7

[0124] As shown in Table 1, the method of this invention achieves an AUC of 90.3% on the common accident test set and 78.6% on the rare accident test set, significantly outperforming the baseline method (59.5%) trained only on real data, with a 19.1 percentage point improvement in AUC for rare accidents. Figure 4 As shown, the "common-rare AUC difference" reflects the performance gap between the model in common and rare accident scenarios. The smaller the difference, the closer the model's ability to identify long-tailed rare accidents is to that of common accidents, thus exhibiting better distribution balance. The method of this invention significantly reduces this difference, indicating that it can effectively alleviate the performance degradation caused by long-tailed distribution and improve the model's generalization ability.

[0125] Compared with random generation enhancement methods, this invention does not simply rely on the number of generated samples, but improves the quality of synthesized samples through target scene planning, real scene constraints and semantic verification. Therefore, it can further improve the AUC of rare accidents by 13.2 percentage points.

[0126] Example 4: Ablation experiments, combined generalization and additional verification;

[0127] To verify the contribution of each component module of this invention, ablation experiments were conducted on combinatorial planning, scene parsing, compatibility evaluation, semantic verification, decoupling learning, and hard sample mining. The results are shown in Table 2.

[0128] Table 2 Ablation Experiment Results

[0129] Configuration Rare incidents AUC(%) Rare incident AP (%) AUC change relative to the complete model Remove combinatorial programming 70.1 63.4 -8.5 Remove scene parsing 72.8 66.3 -5.8 Rule-based compatibility replaces large model compatibility 76.1 69.8 -2.5 Remove semantic validation 69.7 63.0 -8.9 Decoupling learning 70.8 64.3 -7.8 Hard sample removal mining 75.8 69.5 -2.8 Complete model 78.6 72.3 —

[0130] As shown in Table 2, the semantic verification module and the combinatorial planning module have the greatest impact on performance, contributing 8.9 percentage points and 8.5 percentage points to the AUC, respectively. This indicates that "what to generate" and "how to ensure correct generation" are the two key aspects of the success of this invention. The decoupling learning module contributes 7.8 percentage points to the AUC, indicating that further suppressing appearance shortcut learning after expanding the samples is equally important.

[0131] In asymptotic rarity analysis, when all real vehicle accident samples are retained, the present invention still has an AUC gain of 2.4 percentage points compared to the model trained only with real data; when the proportion of real samples decreases to 10%, the gain increases to 13.3 percentage points; when the proportion of real samples decreases to 1%, the gain reaches 17.9 percentage points; under the extreme condition of zero samples, when trained only with the synthetic data generated by the present invention, an AUC of 72.0% can still be obtained, which is approximately equivalent to the baseline model trained with only about 17% real samples.

[0132] In leave-one out-of-combination tests, the average AUC of the standard classifier was 52.6%, which increased to 59.2% after adding the two-branch structure. It was further improved to 72.4% after adopting the complete decoupled learning and hard sample mining of this invention, which is 19.8 percentage points higher than the baseline. The improvement was more significant in combinations with significant visual differences, such as school bus-car transport vehicle T-collision and car transport vehicle-tanker truck overturning.

[0133] In the cross-scene generalization experiment, the AUC of the present invention was 90.3% in the training scene and 86.5% in 40 unmonitored scenes, with a scene transfer difference of only 3.8 percentage points, while the baseline model trained with only real data had a difference of 8.5 percentage points. In the subjective evaluation by humans, the overall accuracy rate of distinguishing between real and fake videos was 58.0%, the accident type recognition rate was 79.8%, and the reasonableness score reached 3.79 / 5, indicating that the videos generated by the present invention achieved a good balance between visual realism and semantic consistency.

Claims

1. A traffic accident detection method based on combinatorial scene video generation and decoupled representation learning, characterized in that, include: S1. Obtain real traffic monitoring videos and scene images corresponding to the real traffic monitoring videos. Perform combined annotations on accident samples according to vehicle type, collision topology, road geometry and environmental conditions to construct an accident scene space. S2. Based on the sample frequency and risk score of each scenario in the accident scenario space, perform sub-model coverage optimization for risk perception to obtain a set of target rare accident scenarios to be generated. S3. Perform structured scene analysis on the scene image to obtain road topology, lane direction, traffic control facilities, camera view, vehicle driving path and conflict area, and evaluate the scene compatibility of the target rare accident scene set based on the analysis results. S4. For the image-scene pair that passes the scene compatibility assessment, generate an accident script text that is aligned with its camera viewpoint, and input the scene image and the accident script text into the image conditional video generation model to obtain candidate accident videos. S5. Perform semantic consistency verification on the candidate accident videos, and execute script correction and closed-loop regeneration on the candidate accident videos that do not reach the consistency threshold to obtain a verified synthetic accident video set. S6. The real traffic monitoring video and the verified synthetic accident video set are jointly input into the accident detection model for training. The accident detection model includes a shared spatiotemporal feature backbone network, a vehicle representation branch and an accident dynamics representation branch. The classification result is output by the accident dynamics representation branch. S7. Input the surveillance video to be detected into the trained accident detection model, and output the accident probability and / or accident alarm results.

2. The method according to claim 1, characterized in that, In step S1, the accident scene space is represented as follows: ， in, Represents the space of the accident scene; Represents a set of vehicle categories; Represents the set of collision topologies; Represents the geometric set of roads; Represents the set of environmental conditions; any scenario configuration is denoted as ,in and These represent the first and second participating vehicle categories, respectively. Represents the collision topology. Represents road geometry. Indicates environmental conditions; uses normalized sorting for vehicles. To eliminate sequential redundancy, Indicate vehicle category number; and count scene frequency based on training corpus. , will satisfy The scenario configuration was determined to be a rare accident scenario, in which This represents the rare threshold.

3. The method according to claim 2, characterized in that, In step S2, for any rare accident scenario Calculate risk score The risk score satisfies: ， in, The vehicle combination severity function is used to characterize the severity of potential accident consequences caused by a combination of vehicle types. The collision topology hazard function is used to characterize the inherent hazard of different collision topologies. This represents the road-environment vulnerability function, used to characterize the amplification effect of road geometry and environmental conditions on accident consequences; These represent the weighting coefficients for vehicle combination severity, collision topology hazard, and road-environment vulnerability, respectively.

4. The method according to claim 3, characterized in that, In step S2, a set of target rare accident scenarios to be generated is selected from the rare accident scenario set using a weighted submodular coverage objective function. The weighted submodule covering objective function satisfies: ， in, Represents the set of rare accident scenarios for the target. Collection of rare accident scenarios Coverage benefits; This represents any rare incident scenario that needs to be covered. Representing a scene Risk score; Representing a scene With the selected scene The scene overlap kernel between them; the scene overlap kernel satisfies: ， in, [·] indicates an indicator function, which takes the value 1 when the corresponding dimension values ​​are the same, and 0 otherwise; Representing a scene The value taken in dimension d; and through marginal gain. Iteratively select the candidate scene with the largest marginal gain using a greedy approach until the preset generation budget is reached. ,in This indicates the number of possible combinations of target scenes that can be generated.

5. The method according to claim 1, characterized in that, In step S3, the structured scene parsing satisfies: ， in, This represents the monitoring scene image to be parsed; MLLM(·) represents the multimodal large model parsing operator; This represents a scenario parsing prompt template used to constrain output format and parse fields; Representing an image The corresponding structured scene description includes scene type, road topology, number and direction of lanes, traffic control facilities, camera pitch and orientation, set of drivable vehicle paths, and set of candidate conflict areas. The scenario compatibility assessment meets the following requirements: ， in, Representing scene images With the target accident scenario Compatibility score between them; The geometric feasibility score measures the degree of consistency between the collision topology and the road topology, the drivable vehicle path, and the conflict zone. The vehicle compatibility score measures the degree of consistency between the participating vehicle category and road grade, lane width, and scenario attributes. The context consistency score measures the degree of consistency between the road geometry and environmental semantics in the target accident scenario and the monitoring scenario context. This represents the weighting coefficient for each scoring item, and ;when The scene is preserved at that time, where This represents the compatibility threshold; and yields the set of generateable scenes corresponding to the current scene image. .

6. The method according to claim 1, characterized in that, In step S4, the accident script text satisfies: ， in, Represents the accident script text aligned with the camera's viewpoint; LLM(·) represents the large language model generation operator; This represents a structured scene description; Indicates the target accident scenario; The script generates a prompt template; the accident script text describes the vehicle's direction of entry, driving trajectory, collision location, collision posture, collision sequence, and changes in vehicle state after the collision, and uses a monitoring screen reference system to express spatial relationships; The image conditional video generation satisfies: ， in, Represents the scene image With accident script text Generated candidate accident videos; (·) represents the image-conditional video generation model; it generates a video for each image-scene combination. Among the candidate accident videos, This indicates the number of candidate videos for each combination.

7. The method according to claim 6, characterized in that, In step S5, the semantic consistency check satisfies: ， ， in, Indicates the first One candidate accident video; This represents a video semantic verification model; Indicates from candidate accident videos The semantic results extracted from it; Indicates candidate accident video The semantic consistency score; A consistency indicator indicating whether an accident actually occurred; A vehicle consistency index indicating whether the category of participating vehicles matches the target accident scenario; A topology consistency index indicating whether the collision topology is consistent with the target accident scenario; This indicates the time deviation between the collision time identified by the verification model and the collision time set in the target script. This represents the time deviation normalization parameter; This represents the weighting coefficient of each consistency index, and ;when At that time, candidate accident videos Retained as verified synthetic accident video; when At that time, according to The accident script text is corrected based on the differences between the target accident scenario and the original text, and then regenerated until the threshold is reached. Or reach the maximum number of iterations .

8. The method according to claim 1, characterized in that, In step S6, the accident detection model satisfies: ， ， ， ， in, This indicates that the input is a surveillance video; This represents a backbone network with shared spatiotemporal features; This represents the intermediate spatiotemporal features extracted by the shared spatiotemporal feature backbone network; Indicates the vehicle representation branch; This represents the branch of accident dynamics characterization; Represents the vehicle representation vector; Represents the accident dynamics characterization vector; and These represent the weight parameters and bias parameters of the classification head, respectively; Represents the Sigmoid function; Indicates the probability of an accident occurring; The decoupled representation learning employs a contrastive loss and a branch solution correlation constraint, wherein the contrastive loss and branch solution correlation constraint satisfy: ， ， ， in, The accident dynamics branch contrast loss is used to constrain samples with the same collision topology but different vehicle combinations to approach each other in the accident dynamics representation space. This represents the vehicle branch contrast loss, which is used to constrain samples with the same vehicle combination but different collision topologies to move closer to each other in the vehicle representation space. Represents the similarity function; and These represent the temperature parameters for the dynamics branch and the vehicle branch, respectively; superscript Indicates anchor sample, This represents a sample that forms a positive sample pair with the anchor sample. Indicates the index of the candidate sample in the denominator; and These represent the vehicle characterization matrix and the accident dynamics characterization matrix in the current batch, respectively. Indicates the transpose operation; Denotes the Frobenius norm; A hard sample mining strategy based on vehicle appearance distance is adopted, wherein the vehicle appearance distance satisfies: ， ， ， ， in, Indicates vehicle category and Category-level appearance distance between them; and These represent the average appearance embeddings of vehicle categories extracted by the pre-trained visual encoder; Indicates video samples and The video-level visual distance between them and These represent video samples respectively. The first and second participating vehicle categories; and These represent video samples respectively. The first and second participating vehicle categories; Indicates the anchor sample Accident dynamics characterization of positive samples with the same collision topology but different appearances; This represents the accident dynamics branch loss after introducing hard sample mining; Indicates the first The weights of each negative sample; This represents the hard sample weight adjustment coefficient; the vehicle branch employs a symmetrical hard sample mining strategy.

9. The method according to claim 8, characterized in that, In step S6, the total training loss satisfies: ， ， in, Indicates the total training loss; Indicates the loss from accident detection; This represents the vehicle branch loss after introducing hard sample mining; Indicates the loss associated with branch solutions; These represent the weighting coefficients for each loss term; Represents the true label of the surveillance video, when the accident sample is taken. Non-accident samples ; This indicates the probability of an accident occurring.

10. A traffic accident detection system based on combined scene video generation and decoupled representation learning, used to implement the method described in any one of claims 1 to 9, characterized in that, The system includes: The data acquisition and combination annotation module is used to acquire real traffic monitoring videos and scene images, and to perform combination annotation on accident samples according to vehicle category, collision topology, road geometry and environmental conditions; The target scenario selection module is used to optimize the selection of a set of rare accident scenarios based on scenario frequency, risk score, and risk perception sub-module coverage. The structured scene parsing module is used to perform structured scene parsing on the surveillance scene images and output road topology, lane direction, traffic control facilities, camera view, vehicle driving path and conflict area; The compatibility assessment module is used to jointly assess the geometric feasibility, vehicle compatibility, and contextual consistency between the target accident scenario and the monitoring scenario. The script generation module is used to generate accident script text aligned with the camera's viewpoint; The video generation module is used to generate candidate accident videos based on monitoring scene images and accident script text; The semantic verification module is used to score the semantic consistency of candidate accident videos and trigger script correction and closed-loop regeneration for candidate accident videos that do not meet the threshold requirements. The model training module is used to train a decoupled accident detection model based on real surveillance videos and validated synthetic accident videos. The online inference module is used to output the probability of accidents and / or the alarm results of accidents based on the trained decoupled accident detection model of the monitoring video to be detected.

Images