All-weather traffic incident recognition method based on joint decision-making by multiple ai large models

By using multiple AI models for joint decision-making, and leveraging road surveillance cameras and large language models to denoise and analyze traffic images, the accuracy and real-time performance of traffic incident identification have been improved. This enables precise traffic incident identification and efficient early warning around the clock, enhancing the level of intelligent traffic management.

WO2026237715A1PCT designated stage Publication Date: 2026-11-19SOUTH CHINA UNIV OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2026/093427
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-15
Filing Date
2026-04-28
Publication Date
2026-11-19

AI Technical Summary

Technical Problem

Existing technologies are insufficient in terms of accuracy, understanding ability, and real-time performance in traffic incident identification, making it difficult to achieve accurate traffic incident identification and early warning in complex scenarios.

Method used

A multi-AI model joint decision-making approach is adopted. All-weather traffic images are collected through road monitoring cameras, noise reduction is performed using a diffusion model, traffic event information is extracted by combining a large language model, and the autonomous learning and real-time interaction capabilities of AI-Agent are combined to determine the severity of traffic accidents and issue warnings.

Benefits of technology

It enables accurate identification and efficient early warning of traffic incidents under all-weather conditions, improving the intelligence level and emergency response efficiency of traffic management departments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2026093427_19112026_PF_FP_ABST
    Figure CN2026093427_19112026_PF_FP_ABST
Patent Text Reader

Abstract

The present invention provides an all-weather traffic incident recognition method based on joint decision-making by multiple AI large models, mainly comprising the following steps: first, acquiring all-weather traffic images in real time by means of a road monitoring camera, and preprocessing the all-weather traffic images to improve image clarity; subsequently, using a large language model to perform semantic analysis on the traffic images, and implementing structured data storage; using a locally deployed large language model to infer the type of a traffic incident; determining the severity of a traffic accident by means of a large language model; and implementing system maintenance by means of a large language model. The present invention overcomes the limitations of conventional image recognition methods in complex traffic environments by integrating multiple large models to construct an AITI-Agent, and achieves more accurate and real-time all-weather traffic incident recognition and early-warning by integrating different large language models.
Need to check novelty before this filing date? Find Prior Art

Description

A method for all-weather traffic incident identification based on joint decision-making of multiple AI models Technical Field

[0001] This invention relates to the fields of computer vision and intelligent transportation technology, and in particular to an all-weather traffic event recognition method based on joint decision-making of multiple AI models. Background Technology

[0002] In the field of intelligent transportation, automatic identification and early warning of traffic incidents is an important research direction for improving road safety and emergency response efficiency. Traditional methods mainly rely on object detection technology, using deep learning models to detect and classify accident images. However, these methods have limited generalization ability to complex scenes and struggle to accurately understand the background and semantic information of accidents from multiple perspectives. Compared to traditional methods, large language models and AI-Agents demonstrate powerful knowledge reasoning and autonomous decision-making capabilities. Specifically, large language models can combine multimodal information to deeply understand the background, related factors, and potential risks of accidents, while AI-Agents possess autonomous learning and real-time interaction capabilities, adjusting identification and early warning strategies according to the dynamic environment, greatly improving the accuracy and intelligence level of event identification.

[0003] Based on currently available technologies, such as the traffic accident recognition method and device based on a multimodal large language model (CN202480000606.7), the method mainly includes: firstly, acquiring multimodal data such as video data, audio data, and sensor data under the driving state of the target vehicle; inputting the acquired multimodal data into an accident recognition model trained based on a multimodal large language model for multimodal information mining; determining whether the target vehicle has been involved in a traffic accident based on the mined multimodal information; and if the target vehicle has been involved in a traffic accident, issuing an alarm message. No publicly available all-weather traffic event image recognition method based on joint decision-making using multiple AI large language models has been disclosed. Summary of the Invention

[0004] In view of the limitations of existing technologies, the purpose of this invention is to provide an all-weather traffic incident identification method based on joint decision-making of multiple AI models, so as to solve the shortcomings of existing technologies in terms of incident identification accuracy, understanding ability, and real-time performance, and improve the intelligence level and efficiency of traffic incident early warning.

[0005] The present invention is achieved by at least one of the following technical solutions.

[0006] A method for all-weather traffic incident identification based on joint decision-making of multiple AI models includes the following steps:

[0007] S1. Real-time collection of images of road traffic operation status around the clock using road traffic monitoring camera equipment to provide data support for subsequent analysis;

[0008] S2. Use a diffusion model to denoise the acquired traffic images;

[0009] S3. Use a large language model to analyze the denoised traffic images, extract vehicle information, road conditions, personnel information, weather conditions, whether there is congestion, and whether an accident has occurred, which are required for traffic event analysis, and store them in the database in a structured manner.

[0010] S4. Use locally deployed large language models to reason and judge different traffic events;

[0011] S5. Based on the damage to vehicles, the number of vehicles involved in the accident, and the injuries sustained by personnel, the severity of traffic accidents is determined as minor accidents, general accidents, serious accidents, and major accidents.

[0012] S6. Send real-time warning information to traffic management departments regarding event types and the severity of traffic accidents;

[0013] S7. Through user feedback, analysis of historical early warning effects, and combination with a large language model, the system for traffic incident image recognition and early warning is maintained.

[0014] Furthermore, in step S1, images of various traffic events are collected using surveillance cameras. The image data includes various types of traffic roads at different times and under different weather conditions.

[0015] The categories for different time periods include: morning peak hours (7:00-9:00), midday hours (11:00-13:00), evening peak hours (17:00-19:00), nighttime hours (19:00-6:00), and off-peak hours (6:00-7:00, 9:00-11:00, 13:00-17:00). The categories for different weather conditions include: sunny, cloudy, rainy, foggy, and snowy.

[0016] Traffic road categories include: highways, urban expressways, rural roads, roundabouts, tunnels, bridges, ramps, service areas, and construction sections, to ensure that the system can identify and warn of traffic accidents in different road environments.

[0017] Furthermore, in step S2, the diffusion model performs noise reduction processing on the acquired traffic images, including the following steps:

[0018] S2-1. Using a traffic accident image dataset containing different weather conditions, time periods, road sections, and viewpoints, construct training samples and refine the original clear images. Perform normalization processing;

[0019] S2-2, Based on the forward diffusion process, the original clear image Gaussian noise is added gradually to simulate different degrees of image distortion. The noise addition process follows the formula below:

[0020]

[0021] In the formula, express The probability distribution, This represents the noisy image at time step t. This represents the proportion of cumulative noise. It is the identity matrix. It follows a Gaussian distribution;

[0022] S2-3. Train the denoising model to learn how to denoise images. Predicted noise and the clear image after noise reduction The prediction process follows the formula below:

[0023]

[0024] In the formula, This represents the original image estimate obtained after denoising and reverse engineering. Indicates noise;

[0025] S2-4. Based on the reverse denoising process, a clear image is generated through progressive denoising. The denoising calculation for each time step t follows the formula below:

[0026]

[0027] In the formula, Indicates that the current image is known. In the case of generating the image from the previous moment. The probability distribution, This represents the denoised mean. Indicates the denoised variance;

[0028] S2-5. The mean squared error (MSE) loss function L is used to calculate the error between the predicted noise and the actual noise, and the neural network parameters are optimized. The loss function is defined as follows:

[0029]

[0030] In the formula, The training loss function of the model depends on the network parameters. , This represents true Gaussian noise. This represents the noise prediction value output by the neural network, with the input being... and time step The output is The estimate, Indicates the original image Time step and noise The expectation is calculated from the joint distribution.

[0031] Furthermore, step S3 includes the following steps:

[0032] S3-1. The traffic event information extracted from the image is divided into four categories: traffic accidents, traffic congestion, road construction, and debris spills.

[0033] S3-2. For the vehicle information extracted from the image, extract its features, including the number of vehicles. Number of vehicles involved in the accident Vehicle types, including:

[0034] ;

[0035] ;

[0036] in, Represents a non-negative integer;

[0037] The road conditions extracted from the image include road condition and road segment types. Number of lanes in a road section Number of lanes where accidents occurred Are there any traffic jams on the road section? And whether there are any risk factors that affect road traffic. The definition is as follows:

[0038]

[0039]

[0040]

[0041] S3-3. Regarding the personnel information extracted from the image, including determining whether there are pedestrians on the road. Number of pedestrians Are there any casualties? The constraints are defined as follows:

[0042]

[0043] when hour,

[0044]

[0045]

[0046] S3-4. Weather conditions extracted from the image. including daytime ,at night ,snow ,rain ,sunny Dense fog The definition is as follows:

[0047]

[0048] S3-5, Regarding whether an accident occurred, the image was extracted to determine this. The definition is as follows:

[0049]

[0050] S3-6. Store traffic incident information, vehicle information, road conditions, personnel information, weather conditions, and whether an accident has occurred in a structured manner in the database.

[0051] Further, in step S4, the traffic event type is inferred and determined by combining the data stored in step S3 and utilizing the locally deployed large language model, including:

[0052] Analyze the vehicle collision locations in traffic accidents to determine whether a rear-end collision, side collision, oncoming collision, side impact, or rollover accident occurred.

[0053] Analyze the number of vehicles involved in the accident to determine whether it is a single-vehicle accident, a two-vehicle accident, or a multi-vehicle chain collision, and infer the cause of the accident by combining the vehicle's driving direction and the collision sequence.

[0054] For traffic congestion, determine whether traffic congestion has occurred on the road surface; for road construction, determine the number of lanes occupied by road construction; for debris spillage, determine the volume and type of debris, and infer the degree of impact of debris on road traffic; based on the obtained road condition data, determine whether the traffic accident was caused by extreme weather factors; based on the obtained weather condition data, determine whether the traffic accident was caused by abnormal road factors such as road collapse, rockfall, water accumulation, or snow accumulation.

[0055] Furthermore, in step S5, the severity of the traffic accident is determined by combining the large language model, and the accident is classified into minor accidents, general accidents, serious accidents, and major accidents. The severity of the accident follows the formula below:

[0056]

[0057] in,

[0058]

[0059] Accident severity types are defined based on the D-value. Follow the formula below:

[0060]

[0061] In the formula, Indicates the type of accident severity, where D indicates the severity of the accident. Indicates the casualty situation. This indicates the total number of lanes in the image segment. This indicates the number of lanes affected by an accident in the image segment. This represents the impact coefficient of different types of vehicles involved in accidents. This indicates the number of accident vehicles corresponding to different vehicle types. This indicates the road congestion situation in the image segment. This indicates whether a traffic accident has occurred in the road segment of the image, where ω1, ω2, ω3, and ω4 are the weight coefficients of each influencing factor.

[0062] Furthermore, in step S7, the accident identification and early warning system is adaptively optimized by combining user feedback and historical early warning effects. The system first collects user feedback data on early warning information, including false alarms and missed alarms, and establishes an early warning dataset to compare the differences between the model's early warning results and the actual accident situation. Combining the self-supervised learning ability of the large language model, the system continuously optimizes the model's performance in tasks such as semantic information extraction from traffic accident images, image feature association reasoning, and accident category determination. Through multiple rounds of iterative training, the system enhances its ability to identify and warn of traffic accidents under complex environments such as different environments, weather conditions, and lighting conditions.

[0063] Furthermore, the AITI-Agent architecture is adopted, and through task division, the information extracted in step S3, the reasoning and judgment of different traffic events in step S4, the determination of the severity of traffic accidents in step S5, and the maintenance task in step S7 are assigned to the corresponding large language models for processing.

[0064] The system for implementing the all-weather traffic incident identification method based on multi-AI large model joint decision-making includes:

[0065] The image data acquisition module collects traffic image data in real time, around the clock, through front-end camera equipment, and transmits the raw image data to the image enhancement module;

[0066] The image enhancement module is used to perform image sharpening preprocessing on the acquired images to improve image quality and recognizability, and then transmits the enhanced image data to the deep semantic feature extraction module.

[0067] The deep semantic feature extraction module is used to perform deep semantic analysis and feature extraction on the acquired traffic images, generate semantic feature information containing vehicles, pedestrians, traffic facilities and traffic events, and store the acquired information in the form of structured data. The semantic feature information is then transmitted to the traffic event intelligent reasoning module.

[0068] The traffic incident intelligent reasoning module is used to intelligently determine whether a traffic incident exists in the current scene and send the identification results to the accident severity determination module;

[0069] The accident severity determination module is used to determine the severity of an incident based on the identified traffic incident type, scope of impact, number of vehicles involved, and the reasoning ability of a large language model, and to provide a reference for subsequent emergency response.

[0070] The accident early warning module provides real-time early warning information to relevant traffic management departments based on the accident assessment results, thereby improving traffic safety response efficiency.

[0071] The system maintenance module is used to monitor the operating status of each module of the system, record logs, update models regularly, and optimize strategies to ensure the stability, accuracy, and scalability of the system.

[0072] The modules mentioned above exchange data and control signals through a high-speed data bus and a unified information interaction interface to ensure the high efficiency and real-time performance of the system.

[0073] The present invention provides a computer device comprising: a memory and a processor, and a computer program stored in the memory. When the computer program is executed on the processor, it implements the aforementioned all-weather traffic event identification method based on joint decision-making of multiple AI models.

[0074] Compared with existing technologies, the beneficial effects of the present invention are as follows:

[0075] This invention, based on an AI-Agent architecture, integrates multiple large language models for collaborative reasoning to intelligently analyze traffic incident images. By reasoning from deep semantic information extracted from images, it extracts data such as accident type and severity in real time, achieving accurate traffic accident detection and early warning. Simultaneously, an adaptive optimization mechanism based on historical data and user feedback is designed to continuously improve the accuracy of event judgment and the effectiveness of early warning. This invention can provide traffic management departments with more intelligent and efficient traffic incident monitoring and dispatch support, improving road safety management and possessing broad application value. Attached Figure Description

[0076] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following description is provided with accompanying drawings of the relevant technical solutions in the embodiments of the present invention or the prior art. It should be understood that the accompanying drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0077] Figure 1 is a flowchart illustrating an all-weather traffic incident identification method based on joint decision-making of multiple AI large models in an embodiment.

[0078] Figure 2 is a schematic diagram of the image enhancement model training in the embodiment;

[0079] Figure 3 shows a traffic accident case diagram used in the embodiments;

[0080] Figure 4 is a schematic diagram of the structured database for extracting semantic information from traffic event images based on the integration of Gemini2.0 and GPT-4o in the embodiment;

[0081] Figure 5 is a schematic diagram of the interface for determining the severity of a traffic accident in an embodiment;

[0082] Figure 6. Schematic diagram of the database and front-end structure in an embodiment of the present invention. Embodiments of the present invention

[0083] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0084] Example 1

[0085] As shown in Figures 1-6, this embodiment presents an all-weather traffic incident identification method based on joint decision-making of multiple AI models. By constructing an AITI-Agent (Artificial Intelligent Traffic Incident-Agent) architecture, different large language models are integrated and task-oriented, allowing each model to perform specific tasks based on its own strengths. Through interaction with a traffic data platform, real-time traffic incident identification and early warning are achieved, improving the efficiency of road safety management. The specific steps include:

[0086] Step 1: Real-time acquisition of road traffic operation images around the clock using road traffic monitoring camera equipment to provide data support for subsequent analysis, including:

[0087] The system utilizes surveillance cameras to collect images of various traffic incidents, covering a wide range of road types and times under different weather conditions.

[0088] As one example, the classification of different time periods includes: morning peak hours (7:00-9:00), midday hours (11:00-13:00), evening peak hours (17:00-19:00), nighttime hours (19:00-6:00), and off-peak hours (6:00-7:00, 9:00-11:00, 13:00-17:00). Classification of different weather conditions includes: sunny, cloudy, rainy, foggy, and snowy. Classification of traffic roads includes: highways, urban expressways, rural roads, roundabouts, tunnels, bridges, ramps, service areas, and construction zones, ensuring the system has the ability to identify and issue warnings for traffic events under different road conditions.

[0089] Step 2: Use a diffusion model to denoise the acquired traffic images and construct an image enhancement module suitable for traffic images.

[0090] As shown in Figure 2, the image enhancement module based on the diffusion model is constructed. The process of denoising the acquired traffic images by training the diffusion model includes the following steps:

[0091] First, training samples are constructed using a traffic event image dataset containing different weather conditions, time periods, road sections, and viewpoints. Then, the original clear images are processed. Normalization was performed; and the sharp image was processed based on the forward diffusion process. Gaussian noise is added gradually to simulate different degrees of image distortion. The noise addition process follows the formula below:

[0092]

[0093] In the formula, express The probability distribution, This represents the noisy image at time step t. This represents the proportion of cumulative noise. It is the identity matrix. It follows a Gaussian distribution;

[0094] Next, the denoising model is trained to learn how to denoise images. Predicted noise and the clear image after noise reduction The prediction process follows the formula below:

[0095]

[0096] In the formula, This represents the original image estimate obtained after denoising and reverse engineering. This represents the noise predicted by a neural network (Ho J, Jain A, Abbeel P. Denoising diffusion probabilistic models[J]. Advances in neural information processing systems, 2020, 33: 6840-6851.).

[0097] Based on the inverse denoising process, a clear image is generated by progressively denoising. The denoising calculation for each time step t follows the formula below:

[0098]

[0099] In the formula, Indicates that the current image is known. In the case of generating the image from the previous moment. The probability distribution, This represents the denoised mean. Indicates the denoised variance;

[0100] Finally, the mean squared error (MSE) loss function L is used to calculate the error between the predicted noise and the actual noise, and the neural network parameters are optimized. The loss function is defined as follows:

[0101]

[0102] In the formula, The training loss function of the model depends on the network parameters. , This represents true Gaussian noise. This represents the noise prediction value output by the neural network, with the input being... and time step The output is The estimate, Indicates the original image Time step and noise The expectation is calculated from the joint distribution;

[0103] After the model is trained according to the above training process, it can perform noise reduction processing on the acquired traffic images, remove environmental noise, increase image details, restore image clarity under complex weather conditions, and improve image quality.

[0104] Step 3: Use large language models (such as Gemini 2.0 and GPT-4o) to make joint decisions and perform deep analysis on the obtained high-definition traffic operation images to further extract structured data information such as traffic events, vehicle information, road conditions, personnel information, weather conditions, shooting location, and whether an accident has occurred.

[0105] Traffic incidents are categorized into four types: traffic accidents, traffic congestion, road construction, and debris spills. For vehicle information extracted from images, their physical features are extracted, including the number of vehicles. Number of vehicles involved in the accident Vehicle types, including:

[0106] ;

[0107] ;

[0108] in, Represents a non-negative integer.

[0109] The road conditions extracted from the image include road condition and road segment types. Number of lanes in a road section Number of lanes where accidents occurred Are there any traffic jams on the road section? And whether there are any risk factors that affect road traffic. The definition is as follows:

[0110]

[0111]

[0112]

[0113] The information about people extracted from the image includes determining whether there are pedestrians on the road. Number of pedestrians Are there any casualties? The constraints are defined as follows:

[0114]

[0115] when hour,

[0116]

[0117]

[0118] Weather conditions extracted from the image including daytime ,at night ,snow ,rain ,sunny Dense fog The definition is as follows:

[0119]

[0120] Whether an accident occurred was extracted from the image. The definition is as follows:

[0121]

[0122] Finally, information such as traffic incidents, vehicle information, road conditions, personnel information, weather conditions, shooting locations, and whether an accident has occurred are stored in the database in a structured manner. Among them, the structured data of traffic accident semantic information extracted based on Gemini2.0 is shown in Table 1.

[0123] Table 1. Structured data extracted from traffic accident semantic information based on Gemini 2.0

[0124]

[0125] The defined database is built on the SQLAlchemy framework to implement structured storage of traffic scene data. The system connects to a local MySQL database and creates a data table named "Traffic Structured Data". The table contains several fields, including: an auto-incrementing "serial number" to uniquely identify each record; an "image" field to store the traffic image path; descriptions of "vehicle information", "road conditions", "personnel information", and "weather conditions" detected in the image; annotations of the image's shooting time and location; a Boolean field recording whether the scene involves a traffic accident; and an automatically generated record timestamp field. This database supports subsequent structured analysis, querying, and visualization processing of traffic image data, and has good scalability and versatility. The database definition code segment is shown in Table 2.

[0126] Table 2 defines the database model code segment.

[0127]

[0128] Table 3 shows the code segment that calls Gemini 2.0 to parse images and store them in the database.

[0129]

[0130] This document describes the process of automatically semantically parsing traffic images using the Gemini 2.0 model and writing the analysis results into a database. The process comprises two main steps: First, a traffic image analysis function is constructed, taking the image as input and sending prompts to the Gemini 2.0 model. The model is instructed to act as a traffic analysis expert, performing structured parsing of the image content from multiple dimensions, including vehicle information, road conditions, pedestrian information, weather conditions, shooting location, and whether an accident has occurred. The model returns results in JSON format, which are then parsed and extracted. Next, the system calls a database writing function to encapsulate the structured information and image path into a single record and store it in the defined "Traffic Structured Data" table. This process achieves a closed-loop operation of automatic image understanding and structured database storage, providing data support for subsequent intelligent retrieval and traffic risk assessment. The code segment for calling Gemini 2.0 to parse images and store them in the database is shown in Table 3.

[0131] Figure 4 shows the database visualization interface, which includes four buttons: "Upload Image", "Extract Information Based on AI Large Language Model", "Save to Database", and "Accident Severity Judgment". Clicking the "Upload Image" button allows you to upload video surveillance traffic images. Clicking "Extract Information Based on AI Large Language Model" allows you to analyze the uploaded image using a large language model and save it to the database. The left box in the middle contains traffic information extracted by the large language model from the uploaded image, and the right box contains information output by the large language model according to the specified fields. The table at the bottom represents the structured data containing different specified fields.

[0132] Step 4: The traffic event types extracted in Step S3 are shown in Table 4. Among them, in traffic accidents, the images and corresponding data identified as "accidents have occurred" are used to infer and determine the traffic accident type according to the collision method using the locally deployed llava:34b-v1.6-fp16 large language model; in traffic congestion, traffic flow characteristics are combined to determine whether traffic congestion has occurred in the image; in road construction, the cause of road construction is determined to infer whether it is temporary or long-term construction; in the event of spilled material, the volume and type of spilled material are determined to predict whether danger may occur.

[0133] Table 4 Types of Traffic Incidents

[0134]

[0135] As one example, for the identified accident images, the llava:34b-v1.6-fp16 model infers the traffic accident type based on the vehicle collision method, specifically including:

[0136] 1. Rear-end collision: Determine whether the vehicles involved in the accident were involved in a longitudinal collision, especially whether they struck the rear of the vehicle in front.

[0137] 2. Side collision: Analyze whether the vehicle involved in the accident collided with the side of the vehicle, such as contact with vehicles in adjacent lanes.

[0138] 3. Oncoming collision: Determine whether the accident occurred in the oncoming lane, including head-on collisions.

[0139] 4. Side collision: Analyze whether the vehicle was damaged due to a side impact with an obstacle (such as a guardrail or barrier).

[0140] 5. Rollover accident: Inspect whether the accident vehicle has overturned, rolled, or otherwise caused damage.

[0141] The llava:34b-v1.6-fp16 model categorizes accidents into single-vehicle accidents, two-vehicle accidents, and multi-vehicle chain collisions based on the number of vehicles involved. It also determines whether extreme weather conditions caused the traffic accident based on road condition data. Furthermore, it assesses whether road collapses, rockfalls, flooding, snow accumulation, or other abnormal road conditions caused the accident. The traffic types and causes are shown in Table 5.

[0142] Table 5 Types and Causes of Traffic Accidents

[0143]

[0144] Step 5: Combine the DeepSeek large model to determine the severity of the accident. Based on the damage to the vehicles, the number of vehicles involved, and the injuries to the personnel, the accident is classified into minor accidents, general accidents, serious accidents, and major accidents.

[0145] The severity of an accident is determined by the following formula:

[0146]

[0147] Based on the accident information, has an accident occurred within this section of the road? Defined as follows:

[0148]

[0149] Based on traffic flow information, is there any congestion within the road segment? Defined as follows:

[0150]

[0151] Depending on the type of vehicle involved in the accident Defined as follows:

[0152]

[0153] Based on the above formula, the accident severity type T is defined according to the accident severity value D. D Follow the formula below:

[0154]

[0155] In the formula, Indicates the type of accident severity, where D indicates the severity of the accident. Indicates the casualty situation. This indicates the total number of lanes in the image segment. This indicates the number of lanes affected by an accident in the image segment. This represents the impact coefficient of different types of vehicles involved in accidents. This indicates the number of accident vehicles corresponding to different vehicle types. This indicates the road congestion situation in the image segment. This indicates whether a traffic accident has occurred in the road segment of the image, where ω1, ω2, ω3, and ω4 are the weight coefficients of each influencing factor.

[0156] The weighting coefficients ω1, ω2, ω3, and ω4 are 0.4, 0.2, 0.3, and 0.1, respectively. Based on the data obtained in step 3, the results of the accident severity determination are shown in Table 6.

[0157] Table 6. Results of Accident Severity Assessment

[0158]

[0159] Figure 5 shows the accident severity determination interface. When the “Accident Severity Determination” button in Figure 4 is clicked, the interface will automatically redirect to the accident severity determination interface. The table includes the parameters for accident severity determination, and the calculated accident severity type is displayed at the bottom of the interface.

[0160] Step 6: Push real-time early warning information on accident type and severity to traffic management departments. By pushing key information such as collected accident images and data to traffic management departments in real time, relevant departments can quickly take response measures and handle emergencies.

[0161] Step 7: Through user feedback and analysis of historical warning effects, combined with large language models, the traffic incident image recognition and warning system is adaptively optimized. The capabilities of each large language model in different aspects such as image semantic information extraction, image information reasoning and judgment are continuously strengthened to improve the accuracy of traffic incident recognition and warning.

[0162] Based on user feedback and historical early warning results, the traffic incident identification and early warning system is adaptively optimized, specifically including the following steps:

[0163] 1. Data Collection and Processing

[0164] The system collects user feedback data on early warning information through an interactive interface. The feedback data includes false alarm information and missed alarm information. At the same time, the system collects historical early warning effect data and establishes an early warning dataset for analyzing the difference between model early warning results and actual accident situations.

[0165] 2. System maintenance and optimization

[0166] The system was iteratively optimized using code analysis capabilities combined with Claude's three major models, gradually improving its performance under complex environmental conditions. Specific optimization tasks included:

[0167] (1) Optimization of semantic information extraction capability from traffic accident images;

[0168] (2) Optimization of image feature association reasoning ability;

[0169] (3) Optimization of the accuracy of accident category determination;

[0170] 3. Performance Evaluation and Feedback

[0171] The system periodically evaluates the performance of the optimized model, combines the evaluation results with user feedback data to form a closed-loop optimization mechanism, thereby achieving continuous adaptive optimization of the system. Figure 6 summarizes the database and front-end structure flow of this embodiment.

[0172] This embodiment combines one or more large language models for joint decision-making, constructing an intelligent architecture called AITI-Agent. By integrating multiple large language models, it fully leverages the advantages of each model to achieve collaborative operation, thereby improving the performance of the traffic event image recognition and early warning system in complex environments.

[0173] This embodiment implements a system for all-weather traffic incident recognition based on multi-AI large model joint decision-making. The system includes an image data acquisition module, an image enhancement module, a deep semantic feature extraction module, a traffic incident intelligent reasoning module, an accident severity determination module, an accident early warning module, and a system maintenance module. The above modules transmit data and exchange control signals through a high-speed data bus and a unified information interaction interface to ensure the system's high efficiency and real-time operation.

[0174] The image data acquisition module collects traffic image data around the clock and in real time through the front-end camera equipment, and transmits the raw image data to the image enhancement module;

[0175] The image enhancement module is used to perform image sharpening preprocessing on the acquired images to improve image quality and recognizability, and then transmits the enhanced image data to the deep semantic feature extraction module.

[0176] The deep semantic feature extraction module is used to perform deep semantic analysis and feature extraction on the acquired traffic images, generate semantic feature information containing vehicles, pedestrians, traffic facilities and traffic events, and store the acquired information in the form of structured data. The semantic feature information is then transmitted to the traffic event intelligent reasoning module.

[0177] The traffic incident intelligent reasoning module is used to intelligently determine whether a traffic incident exists in the current scene and send the identification results to the accident severity determination module;

[0178] The accident severity determination module is used to determine the severity of an incident based on the identified traffic incident type, scope of impact, number of vehicles involved, and the reasoning ability of a large language model, and to provide a reference for subsequent emergency response.

[0179] Based on the accident assessment results, the accident early warning module provides real-time early warning information to relevant traffic management departments, thereby improving traffic safety response efficiency.

[0180] The system maintenance module is used to monitor the operating status of each module of the system, record logs, update models regularly, and optimize strategies to ensure the stability, accuracy, and scalability of the system.

[0181] In summary, this invention adopts the AITI-Agent (Artificial Intelligent Traffic Incident-Agent) architecture, building a collaborative framework composed of multiple AI large language models. Through task division, sub-tasks such as deep semantic feature extraction (step S3), intelligent traffic incident reasoning (step S4), accident severity determination (step S5), and system maintenance (step S7) are assigned to the most suitable large language model for processing. AITI-Agent interacts with the traffic data platform in real time, receiving high-quality traffic images and related data processed by the image enhancement module, coordinating the execution of each sub-task and integrating the processing results, thereby achieving efficient identification and early warning of traffic incidents and improving the efficiency of road safety management.

[0182] Specifically, the system first uses a diffusion model to sharpen the acquired images, then extracts structured information about traffic image semantics based on large language models (e.g., Gemini 2.0 and GPT-4o); it then uses a locally deployed large language model (e.g., llava:34b-v1.6-fp16) to determine the type of traffic incident; it then uses a large language model (e.g., Deepseek) to determine the severity of traffic accidents; and it uses a large language model (e.g., Claude 3) to maintain the traffic incident image recognition and early warning system. Finally, by interacting with the traffic data platform, it achieves real-time traffic accident recognition and early warning, thereby improving the efficiency of road safety management.

[0183] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, enabling those skilled in the art to better understand and utilize the invention.

Claims

1. An all-weather traffic incident identification method based on multi-AI large model joint decision, characterized in that, Includes the following steps: S1. Real-time collection of images of road traffic operation status around the clock using road traffic monitoring camera equipment to provide data support for subsequent analysis; S2. Use a diffusion model to denoise the acquired traffic images; S3. Use a large language model to analyze the denoised traffic images, extract vehicle information, road conditions, personnel information, weather conditions, whether there is congestion, and whether an accident has occurred, which are required for traffic event analysis, and store them in the database in a structured manner. S4. Use locally deployed large language models to reason and judge different traffic events; S5. Based on the damage to vehicles, the number of vehicles involved in the accident, and the injuries sustained by personnel, the severity of traffic accidents is determined as minor accidents, general accidents, serious accidents, and major accidents. S6. Send real-time warning information to traffic management departments regarding event types and the severity of traffic accidents; S7. Through user feedback, analysis of historical early warning effects, and combination with a large language model, the system for traffic incident image recognition and early warning is maintained.

2. The all-weather traffic incident identification method based on multi-AI large model joint decision according to claim 1, characterized in that, In step S1, images of various traffic events are collected using surveillance cameras. The image data includes various types of traffic roads at different times and under different weather conditions. The categories for different time periods include: morning peak hours (7:00-9:00), midday hours (11:00-13:00), evening peak hours (17:00-19:00), nighttime hours (19:00-6:00), and off-peak hours (6:00-7:00, 9:00-11:00, 13:00-17:00). The categories for different weather conditions include: sunny, cloudy, rainy, foggy, and snowy. Traffic road categories include: highways, urban expressways, rural roads, roundabouts, tunnels, bridges, ramps, service areas, and construction sections, to ensure that the system can identify and warn of traffic accidents in different road environments.

3. The all-weather traffic incident identification method based on multi-AI large model joint decision according to claim 1, characterized in that, In step S2, the diffusion model performs noise reduction processing on the acquired traffic images, including the following steps: S2-1, using a traffic accident image dataset containing different weather, time period, road section, view angle, constructing training samples, and performing original clear image Perform normalization processing; S2-2, based on a forward diffusion process on the original sharp image Gaussian noise is added gradually to simulate different degrees of image distortion. The noise addition process follows the formula below: In the formulae, denotes a probability distribution of the number of times the user has accessed the content, denotes a noisy image at time step t, a proportion representing the accumulated noise, is the identity matrix, It follows a Gaussian distribution; S2-3, training the denoising model to learn to transform a noisy image Predictive noise and a clear image after denoising The prediction process follows the formula below: In the formulae, denotes the original image estimate of the de-noising back- propagation, Indicates noise; S2-4, based on a reverse process, generate a clear image by step-by-step denoising The denoising calculation for each time step t follows the formula below: In the formulae, representing in the known current image in the case where the preceding time image is generated a probability distribution of the number of times the user has accessed the content, denotes the denoised mean, Indicates the denoised variance; S2-5, the mean square error (MSE) loss function L is used to calculate the error between the predicted noise and the real noise, and the neural network parameters are optimized The loss function is defined as follows: In the formulae, a training loss function representing the model, dependent on the network parameters , denotes real Gaussian noise, a noise prediction value representing a neural network output, input is and time steps , the output is a pair of estimate of the number of representing the original image , time step and noise The expectation is calculated from the joint distribution.

4. The all-weather traffic incident identification method based on multi-AI large model joint decision according to claim 1, characterized in that, Step S3 includes the following steps: S3-1. The traffic event information extracted from the image is divided into four categories: traffic accidents, traffic congestion, road construction, and debris spills. S3-2, for the vehicle information extracted in the image, extract its features, including the number of vehicles , number of accident vehicles Vehicle types, including: ; ; wherein Represents a non-negative integer; For the road conditions extracted in the image, including road condition road segment types , number of lanes of a road segment , number of lanes where the accident occurred whether congestion occurs in the link and whether there are risk factors that affect road traffic The definition is as follows: S3-3, for the situation of the person extracted in the image, including determining whether there is a pedestrian on the road surface , number of pedestrians • presence or absence of human casualties The constraints are defined as follows: When hour, S3-4, for weather conditions extracted in the image , including daytime , night snowing , it is raining , sunny heavy fog The definition is as follows: S3-5, for whether an accident occurs in the image extracted The definition is as follows: S3-6. Store traffic incident information, vehicle information, road conditions, personnel information, weather conditions, and whether an accident has occurred in a structured manner in the database.

5. The all-weather traffic incident identification method based on multi-AI large model joint decision according to claim 1, characterized in that, In step S4, the traffic event type is inferred and determined by combining the data stored in step S3 and utilizing the locally deployed large language model, including: Analyze the vehicle collision locations in traffic accidents to determine whether a rear-end collision, side collision, oncoming collision, side impact, or rollover accident occurred. Analyze the number of vehicles involved in the accident to determine whether it is a single-vehicle accident, a two-vehicle accident, or a multi-vehicle chain collision, and infer the cause of the accident by combining the vehicle's driving direction and the collision sequence. For traffic congestion, determine whether traffic congestion has occurred on the road surface; for road construction, determine the number of lanes occupied by road construction; for debris spillage, determine the volume and type of debris, and infer the degree of impact of debris on road traffic; based on the obtained road condition data, determine whether the traffic accident was caused by extreme weather factors; based on the obtained weather condition data, determine whether the traffic accident was caused by abnormal road factors such as road collapse, rockfall, water accumulation, or snow accumulation.

6. The all-weather traffic incident identification method based on multi-AI large model joint decision according to claim 1, characterized in that, In step S5, the severity of the traffic accident is determined using a large language model, and the accident is classified into four levels: minor accident, general accident, serious accident, and major accident. The severity of the accident follows the formula below: in, Defining the accident severity type according to the accident severity D value Follow the formula below: In the formulae, D represents the severity of the accident, representing the number of casualties, represents the total number of lanes in the image segment, a number of lanes representing the impact of an incident in an image segment, a coefficient of influence representing different types of accident vehicles, represents the number of accident vehicles corresponding to different vehicle types, representing road congestion in an image segment, This indicates whether a traffic accident has occurred in the road segment of the image, where ω1, ω2, ω3, and ω4 are the weight coefficients of each influencing factor.

7. The all-weather traffic incident identification method based on multi-AI large model joint decision according to claim 1, characterized in that, In step S7, the accident identification and early warning system is adaptively optimized by combining user feedback and historical early warning effects. The system first collects user feedback data on early warning information, including false alarms and missed alarms, and establishes an early warning dataset to compare the differences between the model's early warning results and the actual accident situation. Combining the self-supervised learning ability of the large language model, the system continuously optimizes the model's performance in tasks such as semantic information extraction from traffic accident images, image feature association reasoning, and accident category determination. Through multiple rounds of iterative training, the system enhances its ability to identify and warn of traffic accidents under complex environments such as different environments, weather conditions, and lighting conditions. 8.The all-weather traffic incident identification method based on multi-AI large model joint decision according to any one of claims 1-7, characterized in that, The AITI-Agent architecture is adopted. Through task division, the information extracted in step S3, the reasoning and judgment of different traffic events in step S4, the determination of the severity of traffic accidents in step S5, and the maintenance task in step S7 are assigned to the corresponding large language models for processing.

9. The system for implementing the all-weather traffic incident identification method based on multi-AI large model joint decision of claim 1, characterized in that, include: The image data acquisition module collects traffic image data in real time, around the clock, through front-end camera equipment, and transmits the raw image data to the image enhancement module; The image enhancement module is used to perform image sharpening preprocessing on the acquired images to improve image quality and recognizability, and then transmits the enhanced image data to the deep semantic feature extraction module. The deep semantic feature extraction module is used to perform deep semantic analysis and feature extraction on the acquired traffic images, generate semantic feature information containing vehicles, pedestrians, traffic facilities and traffic events, and store the acquired information in the form of structured data. The semantic feature information is then transmitted to the traffic event intelligent reasoning module. The traffic incident intelligent reasoning module is used to intelligently determine whether a traffic incident exists in the current scene and send the identification results to the accident severity determination module; The accident severity determination module is used to determine the severity of an incident based on the identified traffic incident type, scope of impact, number of vehicles involved, and the reasoning ability of a large language model, and to provide a reference for subsequent emergency response. The accident early warning module provides real-time early warning information to relevant traffic management departments based on the accident assessment results, thereby improving traffic safety response efficiency. The system maintenance module is used to monitor the operating status of each module of the system, record logs, update models regularly, and optimize strategies to ensure the stability, accuracy, and scalability of the system. The modules mentioned above exchange data and control signals through a high-speed data bus and a unified information interaction interface to ensure the high efficiency and real-time performance of the system.

10. A computer device, comprising: It includes: a memory and a processor, and a computer program stored in the memory, which, when executed on the processor, implements the all-weather traffic event identification method based on joint decision-making of multiple AI large models as described in claim 8.