Traffic event self-correction iteration training method and system based on visual language large model

Abstract

This invention relates to a self-correcting iterative training method and system for traffic events based on a large visual language model. The method includes: acquiring traffic monitoring video data and inputting it into a pre-trained traffic event detection model to obtain traffic event detection results; the training process of the traffic event detection model includes: acquiring traffic monitoring video training samples to train a base model; repeatedly inputting the same traffic monitoring video training sample into the base model to obtain multiple candidate output results; matching a correct candidate output result with each erroneous candidate output result to obtain error-correcting training samples; constructing discriminative training samples based on multiple candidate output results and labeled reference information; training the base model based on an iterative training set until convergence to obtain an updated model; continuously updating the current training model until a preset stopping condition is reached to obtain a trained traffic event detection model. Compared with existing technologies, this invention has the advantage of strong detection robustness.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and intelligent transportation systems, and in particular to a method and system for self-correction iterative training of traffic events based on a large visual language model. Background Technology

[0002] Traffic surveillance video has become a common information collection method in urban roads, highways, and intersections. Identifying and handling incidents such as abnormal parking, traffic congestion, construction encroachment, traffic violations, and traffic accidents using surveillance video has become a crucial technical requirement in traffic management. Because traffic incidents are typically sudden, widespread, and time-sensitive, failure to detect and respond promptly can easily lead to decreased traffic efficiency and, in severe cases, may even trigger secondary accidents. Therefore, how to achieve automatic detection of traffic incidents based on traffic surveillance video has been a continuously relevant technical issue in the field of intelligent transportation.

[0003] Existing traffic incident detection methods can be broadly categorized into two types: those based on traffic flow parameters and those based on video image analysis. The former typically relies on devices such as inductive loop detectors, radar, or microwave detectors to identify traffic anomalies using statistical parameters like vehicle speed, traffic flow, and occupancy. While these methods have been used for a longer period and have relatively mature implementation paths, they often struggle to directly identify specific incident types or reflect the specific process of an incident. The latter, on the other hand, directly utilizes information about the road environment, traffic participants, and their behaviors from surveillance videos for incident identification. Common technical approaches include target detection, behavior recognition, spatiotemporal feature extraction, and video classification. While these methods improve traffic incident detection capabilities to some extent, they are still prone to misjudgments, missed judgments, and unclear judgment criteria in complex traffic scenarios.

[0004] With the development of large-scale visual language models, methods based on these models to understand video content and output event type determinations and descriptions offer a new technical approach for traffic event detection. Compared to traditional methods that only output category labels, large-scale visual language models can provide corresponding semantic descriptions simultaneously with event recognition, thereby enhancing the comprehensibility of the output. However, current training methods typically rely heavily on traffic monitoring video training samples and their corresponding labeled reference information for basic supervised learning. There is a lack of further mechanisms to utilize multiple candidate outputs generated under the same video input conditions. In particular, existing technologies often fail to convert erroneous candidate outputs into error correction supervision signals, nor do they convert the differences in quality between multiple candidate outputs into discriminative supervision signals. Consequently, in complex scenarios, problems such as insufficient output stability, inadequate utilization of candidate results, and limited detection performance persist.

[0005] The invention disclosed in CN119229388A, a traffic accident recognition method based on unsupervised pre-training and a multimodal large model, includes the following steps: S1, collecting and organizing video data from traffic monitoring scenarios, selecting video segments without traffic accidents as data samples, extracting keyframes from the videos, performing unsupervised pre-training of a masked autoencoder, and reconstructing randomly occluded video segments; S2, calculating the reconstruction error between the original video and the reconstructed video to evaluate the reconstruction quality, and filtering out keyframe video segments with poor reconstruction quality. This invention, through unsupervised training of a masked autoencoder, can filter out a large number of segments without traffic accidents, even with limited labeled data, and quickly identify video segments that may contain traffic anomalies. By constructing multimodal traffic data for fine-tuning training of a multimodal large model, it can improve the accuracy and efficiency of traffic accident recognition in complex scenarios, providing real-time alerts and detailed analysis, and enabling timely response measures. However, this scheme only provides a basic unsupervised training method and fails to convert erroneous candidate output results into error correction supervision signals, lacking stability in detection performance under complex scenarios.

[0006] Based on this, it is necessary to propose a new self-correction iterative training method for traffic events based on a large visual language model. This method verifies and judges multiple candidate output results generated by the same traffic monitoring video training sample under the same video input conditions, and constructs error correction training samples and discrimination training samples based on the verification results. These samples are then jointly iteratively trained with the basic training samples to improve the accuracy, stability, and candidate discrimination ability of the traffic event detection model. Summary of the Invention

[0007] This invention aims to address the problems of existing traffic incident detection models based on large visual language models, such as reliance on basic supervised samples during training, lack of effective utilization of multiple candidate outputs generated under the same video input conditions, difficulty in converting erroneous candidate outputs into error correction supervised signals, and difficulty in converting discriminative information among multiple candidate outputs into discriminative supervised signals. To this end, this invention provides a self-correcting iterative training method and system for traffic incident detection based on large visual language models. This method verifies and judges multiple candidate outputs generated from the same traffic monitoring video training samples, constructs error correction training samples and discriminative training samples, and jointly performs multiple rounds of iterative training with basic training samples, thereby improving the accuracy, stability, and practical applicability of traffic incident detection models based on large visual language models.

[0008] The objective of this invention can be achieved through the following technical solutions: A self-correction iterative training method for traffic events based on a large visual language model includes: Acquire traffic monitoring video data and preprocess it; input the preprocessed data into a pre-trained traffic incident detection model to obtain traffic incident detection results; the training process of the traffic incident detection model includes: Step 1: Obtain multiple traffic monitoring video training samples with labeled reference information, and perform basic training on the traffic incident detection model until the training converges to obtain the basic model; Step 2: Repeatedly input the same traffic monitoring video training sample into the base model to obtain multiple candidate output results; based on the labeled reference information, determine whether the multiple candidate output results are correct or incorrect candidate output results; Step 3: Based on the preset pairing rules, match each erroneous candidate output with a correct candidate output to obtain an error correction training sample; construct a discrimination training sample based on the multiple candidate outputs and labeled reference information; construct an iterative training set based on the traffic monitoring video training sample, the error correction training sample, and the discrimination training sample; train the basic model based on the iterative training set until convergence to obtain an updated model. Step 4: Based on the updated model, repeat steps 2 and 3 to update the updated model; continuously update the current training model until the preset stopping condition is reached to obtain the trained traffic event detection model.

[0009] Furthermore, the same traffic monitoring video training sample is repeatedly input into the updated model, and steps two and three are repeated to generate a new iterative training set and update the updated model. The current training model is initially the base model, and the current training model in subsequent training rounds is the model updated in the previous training round.

[0010] Furthermore, the candidate output results include the traffic incident type determination result and the incident description content; The annotation reference information includes traffic event category labels and event description text corresponding to the traffic monitoring video training samples.

[0011] Furthermore, the traffic incident detection model is further fine-tuned using the traffic monitoring video training samples and the labeled reference information to obtain the base model.

[0012] Furthermore, based on the labeled reference information, the plurality of candidate output results are determined to be either correct or incorrect candidate output results. The corresponding determination process includes: The category consistency determination result is determined based on the correspondence between the traffic event type determination result and the traffic event category label in the candidate output results; The event description content, event description text, and preset verification prompt information in the candidate output results are input into the current training model to output semantic similarity judgment results; based on the category consistency judgment results and semantic similarity judgment results, the multiple candidate output results are judged to be correct candidate output results or incorrect candidate output results.

[0013] Furthermore, when the category consistency determination result is consistent and the semantic similarity determination result is similar, the candidate output result is determined as the correct candidate output result; when the category consistency determination result is inconsistent or the semantic similarity determination result is dissimilar, the candidate output result is determined as the incorrect candidate output result.

[0014] Furthermore, the error correction training samples include traffic monitoring video training samples, an erroneous candidate output result, and a correct candidate output result paired with it; the error correction training samples are used to train the current training model to correct the erroneous candidate output result based on the same video input.

[0015] Furthermore, the discrimination training samples include traffic monitoring video training samples, multiple candidate output results corresponding to the same traffic monitoring video training sample, and labeled reference information; The discrimination training samples are used to train the current training model to identify the target candidate output result among the multiple candidate output results; the target candidate output result is determined by the candidate output result that is verified as correct according to a preset determination rule.

[0016] Furthermore, the preset stopping conditions include: the number of training rounds reaches the preset number of training rounds, the traffic incident detection performance reaches the preset threshold, or the number of newly determined correct candidate output results no longer increases in multiple consecutive training rounds.

[0017] The present invention also provides a system for a traffic event self-correction iterative training method based on a large visual language model, comprising a memory and a processor, wherein the memory stores a computer program, and the processor calls the computer program to execute the steps of any of the methods described above.

[0018] Compared with the prior art, the present invention has the following advantages: (1) This invention constructs an iterative training set including error correction training samples and discrimination training samples, enabling the model to continuously generate new high-quality supervision signals from limited manually labeled data in each round of training and to continuously evolve through active learning. The error correction training samples explicitly pair the erroneous candidate outputs and correct candidate outputs corresponding to the same video, forming a training signal in the form of a mistake notebook, enabling the model to accurately compare the differences between the correct and incorrect outputs and to specifically correct the discrimination bias of easily confused events and ambiguous scenes. The discrimination training samples utilize multiple candidate outputs and labeled reference information to enable the training model to learn to evaluate, filter, and select the best output. This systematically improves the accuracy, stability, and engineering application effect of the traffic incident detection model. When facing complex, dynamic, and diverse traffic scenes in actual surveillance videos, it not only has high-precision discrimination capabilities but also outputs the most reliable results through internal self-verification and selection mechanisms, significantly reducing the false negative rate and false positive rate, and truly achieving a leap from being able to detect to being able to detect accurately, being able to detect stably, and being able to detect intelligently.

[0019] (2) This invention obtains multiple candidate output results by repeatedly inputting the same traffic monitoring video training sample into the basic model, and further constructs error correction training samples and discrimination training samples, so that the visual language large model can further absorb the additional supervision information formed by its own generated results in addition to traditional basic supervised learning, thereby realizing the self-proliferation and closed-loop optimization of supervision information. Under the condition of a limited number of initial labeled samples, this invention can significantly reduce the demand for massive accurate labeled data through a self-iteration mechanism, thereby saving a lot of expensive manual labeling costs and time. It is especially suitable for vertical application fields such as traffic incident detection, which are complex, have diverse event types, and have scarce professional labelers and high labeling costs. This makes the method still very feasible and practical in actual projects with limited data resources.

[0020] (3) This invention adopts a multi-round iterative training strategy, that is, it uses the updated model obtained from each training to re-execute the complete process of candidate output result generation, verification and judgment, construction of error correction training samples and discrimination training samples, and update of the current training model. This makes the upper limit of the model's performance far exceed the level that can be achieved by ordinary single training or simple fine-tuning. In the process of repeated self-comparison and correction, the model learns how to self-check and correct itself. As a result, when facing newly collected surveillance video data, newly emerging traffic event types, or scenarios that are different from the distribution of the training set, it exhibits stronger generalization recognition ability and adaptive adjustment ability, effectively alleviating the problem of the sharp decline in performance of traditional deep learning models on out-of-domain data. Attached Figure Description

[0021] Figure 1This is a flowchart of the self-correction iterative training method for traffic events based on a large visual language model provided in this embodiment of the invention. Figure 2 This is an overall flowchart of the traffic event self-correction iterative training method based on a large visual language model provided in this embodiment of the invention; Figure 3 This is a schematic diagram illustrating the verification of candidate output results and the division of correct and incorrect results for the traffic event self-correction iterative training method based on a large visual language model provided in this embodiment of the invention. Figure 4 This is a schematic diagram illustrating the construction of error correction training samples and discrimination training samples for the traffic event self-correction iterative training method based on a large visual language model provided in this embodiment of the invention. Figure 5 This is a schematic diagram illustrating the iterative training set construction and model update of the traffic event self-correction iterative training method based on a large visual language model provided in this embodiment of the invention. Figure 6 This is a comparison chart of the classification performance of the traffic event self-correction iterative training method based on a large visual language model provided in this embodiment of the invention. Figure 7 This is an evaluation graph of the semantic consistency of event description text in the traffic event self-correction iterative training method based on a large visual language model provided in this embodiment of the invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0023] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0024] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0025] Example 1 like Figure 1As shown, this embodiment provides a self-correction iterative training method for traffic events based on a large visual language model. The method includes the following steps: S1: Acquire traffic monitoring video data and preprocess it; input the preprocessed data into the pre-trained traffic incident detection model to obtain the traffic incident detection results; Multiple traffic monitoring video training samples were obtained, and each traffic monitoring video training sample has corresponding labeled reference information. Preferred, The annotation reference information includes traffic event category labels and event description text corresponding to the training samples of traffic monitoring videos.

[0026] S2: The training process of the traffic incident detection model includes: acquiring multiple traffic monitoring video training samples with labeled reference information, and performing basic training on the traffic incident detection model until the training converges to obtain the basic model. The initial visual language model was trained using traffic monitoring video training samples and labeled reference information to obtain the basic model. Preferred, Basic training includes: supervising and fine-tuning the initial large visual language model using traffic monitoring video training samples and labeled reference information to obtain the basic model.

[0027] S3: Repeatedly input the same traffic monitoring video training sample into the base model to obtain multiple candidate output results; based on the labeled reference information, determine whether the multiple candidate output results are correct or incorrect candidate output results; S301: The same traffic monitoring video training sample is repeatedly input into the current training model, and multiple generation is performed under the same video input conditions to obtain multiple candidate output results. The current training model in the initial round is the base model, and the current training model in subsequent rounds is the updated model obtained from the previous round of training. Preferred, The candidate output results include the traffic incident type determination result and the incident description.

[0028] S302: Based on the corresponding annotation reference information, verify and judge multiple candidate output results to determine the correct candidate output results and the incorrect candidate output results; Preferred, Verification criteria include: (1) Determine the category consistency judgment result based on the correspondence between the traffic event type judgment result and the traffic event category label in the candidate output results; (2) Input the event description content, event description text and preset verification prompt information from the candidate output results into the current training model to output the semantic similarity judgment result or the semantic dissimilarity judgment result.

[0029] Preferred, When the category consistency determination result is consistent and the current training model outputs a semantic similarity determination result, the candidate output result is determined as the correct candidate output result; when the category consistency determination result is inconsistent, or the current training model outputs a semantic dissimilarity determination result, the candidate output result is determined as the incorrect candidate output result.

[0030] S4: Based on the preset pairing rules, match a correct candidate output result for each erroneous candidate output result to obtain an error correction training sample; construct a discrimination training sample based on multiple candidate output results and labeled reference information; construct an iterative training set based on the traffic monitoring video training sample, error correction training sample, and discrimination training sample; train the basic model based on the iterative training set until convergence to obtain an updated model; S401: For each erroneous candidate output, a correct candidate output is selected from the correct candidate outputs corresponding to the same traffic monitoring video training sample according to a preset pairing rule and paired with it to construct an error correction training sample. Preferred, The error correction training sample consists of a traffic monitoring video training sample, an erroneous candidate output result, and a correct candidate output result paired with it. The erroneous candidate output result and the correct candidate output result are derived from multiple candidate output results corresponding to the same traffic monitoring video training sample. The error correction training sample is used to train the current training model to correct the erroneous candidate output result based on the same video input.

[0031] S402: For the same traffic monitoring video training sample, construct a discrimination training sample based on multiple candidate output results and annotation reference information corresponding to the traffic monitoring video training sample, and determine the target candidate output result from the candidate output results that are verified and judged to be correct according to the preset determination rules. The discrimination training samples consist of traffic monitoring video training samples, multiple candidate output results corresponding to the same traffic monitoring video training sample, and labeled reference information; the target candidate output result is determined by the candidate output result that has been verified as correct according to a preset determination rule; the discrimination training samples are used to train the current training model to identify the target candidate output result among multiple candidate output results.

[0032] S403: The traffic monitoring video training samples and their corresponding labeled reference information are used to form the basic training samples. The basic training samples, error correction training samples and discrimination training samples are jointly organized into an iterative training set. The iterative training set is used to train the current training model to obtain an updated model. Preferred, The iterative training set includes a basic training sample set, an error correction training sample set, and a discrimination training sample set. The basic training sample set consists of traffic monitoring video training samples and their corresponding labeled reference information. The error correction training sample set consists of error correction training samples, and the discrimination training sample set consists of discrimination training samples.

[0033] S5: Based on the updated model, repeat steps three and four to update the updated model; continuously update the current training model until the preset stopping condition is reached to obtain the trained traffic event detection model.

[0034] S501: Re-execute S3 to S4 using the updated model to update the error correction training samples, discrimination training samples, iterative training set, and current training model; S502: When the preset stopping conditions are met, output the traffic event detection model that has been trained.

[0035] Preferred, The preset stopping condition is any of the following: the preset number of training rounds is reached; the traffic event detection performance on the validation set reaches a preset threshold; or the number of newly identified correct candidate output results no longer increases in multiple consecutive training rounds.

[0036] Example 2 This embodiment provides a specific application example of a traffic event self-correction iterative training method based on a large visual language model. Using the same traffic monitoring video training samples, it sequentially performs the following steps: basic training sample construction, basic training of the large visual language model, candidate output result generation, candidate output result verification and judgment, error correction training sample construction, discriminant training sample construction, iterative training set construction, and model update. After meeting a preset stopping condition, it outputs the trained traffic event detection model. The output of the previous stage serves as the input for the next stage, thus forming a closed-loop iterative training process. The overall process is as follows: Figure 2 As shown.

[0037] I. Construction of Basic Training Samples The input for this stage is the raw traffic monitoring video data.

[0038] First, acquire traffic monitoring video data. Traffic monitoring video data can come from traffic monitoring scenarios such as urban roads, highways, bridges, ramps, tunnels, toll stations, and intersections, and can cover different collection conditions such as daytime, nighttime, sunny days, and rainy days.

[0039] Subsequently, the original traffic monitoring video data is segmented to form traffic monitoring video training samples. Specifically, continuous monitoring videos can be segmented according to preset duration, event trigger windows, or sliding windows, so that each video segment corresponds to an independent traffic monitoring video training sample.

[0040] After obtaining traffic monitoring video training samples, each sample is labeled to generate corresponding annotation reference information. This reference information includes traffic event category labels and event description text. The traffic event category labels represent the event type of the corresponding training video sample; the event description text represents the event subject, main actions, and outcome status within the training video sample. The event description text can be written according to a standardized template to improve the consistency of annotation format across different samples.

[0041] Furthermore, each traffic surveillance video training sample is associated with its corresponding traffic event category label and event description text to form a basic training sample. The basic training sample consists of the traffic surveillance video training samples and their corresponding annotation reference information.

[0042] Therefore, the output of this stage is the basic training sample set. The basic training sample set serves as the input for the next stage of basic model training, and also as a reference for subsequent candidate output result verification, error correction training sample construction, and discriminant training sample construction.

[0043] In this embodiment, to illustrate the method of the present invention, a traffic incident detection dataset is constructed based on real traffic monitoring videos. This dataset contains 2050 video clips, covering five scenarios: two-wheeled vehicles entering the road illegally, abnormal parking, congestion, road construction, and normal traffic. The training set contains 1638 video clips, and the test set contains 412 video clips. Specific examples are shown in Table 1. It should be noted that the above data sources, sample size, and category settings are only for illustrating the present invention and do not constitute a limitation on the applicable scenarios of the present invention.

[0044] Table 1. Example of the composition of the traffic incident video dataset in this embodiment. II. Basic Training and Candidate Output Generation of the Visual Language Large Model The input for this stage is the basic training sample set formed in the previous stage and the initial visual language model.

[0045] First, a basic training model is obtained by using a basic training sample set to train the initial visual language model. Specifically, traffic monitoring video training samples and their corresponding labeled reference information are input into the initial visual language model, and supervised fine-tuning is performed on it. This enables the visual language model to establish the correspondence between traffic monitoring videos and traffic event category labels, and to learn the correspondence between traffic monitoring videos and event description text.

[0046] After basic training is completed, the base model is output. The base model serves as the current training model in the initial round.

[0047] Subsequently, the same traffic monitoring video training sample is repeatedly input into the current training model, and multiple generation operations are performed under the same video input conditions to obtain multiple candidate output results for that sample. The same video input conditions include the same traffic monitoring video training sample, the same input prompt template, and the same output task requirements. Based on this, output differences can be introduced by repeatedly generating and calling the model, sampling multiple times, setting different random seeds, setting different decoding parameters, or combinations thereof.

[0048] In this embodiment, each traffic monitoring video training sample is generated four times to obtain four candidate output results. Each candidate output result includes a traffic event type determination result and an event description. The traffic event type determination result represents the current training model's judgment of the event category of the input video sample, and the event description represents the current training model's textual expression of the event subject, main behavior, and outcome status in the input video sample based on the video understanding results.

[0049] Furthermore, multiple candidate output results corresponding to the same traffic monitoring video training sample are merged to form a set of candidate output results corresponding to that sample.

[0050] Therefore, the output of this stage includes the base model and a set of multiple candidate output results. The set of candidate output results serves as the input for the verification and judgment of candidate output results in the next stage; in subsequent iterations, the current trained model is replaced by the updated model obtained from the previous training round.

[0051] III. Verification and Judgment of Candidate Output Results and Classification of Correctness and Error The input for this stage is the set of candidate output results formed in the previous stage and the corresponding labeled reference information in the basic training samples.

[0052] First, a category consistency determination is performed for each candidate output. Specifically, the traffic event type determination result in the candidate output is read and compared with the traffic event category label of the corresponding traffic monitoring video training sample. When the two are consistent, the category consistency determination result is determined to be consistent; when the two are inconsistent, the category consistency determination result is determined to be inconsistent.

[0053] Subsequently, a semantic similarity determination is performed on each candidate output result. Specifically, the event description content in the candidate output result, the event description text in the corresponding labeled reference information, and the preset verification prompt information are input into the determination module, which then outputs a semantic similarity determination result or a semantic dissimilarity determination result. The preset verification prompt information is used to require the determination module to determine whether the event description content in the candidate output result covers the event subject, main behavior, and result status.

[0054] The judgment module can be the current large visual language model, an independently set semantic judgment module, or an external semantic evaluation model. In this embodiment, the current large visual language model is used as the judgment module. In this case, the candidate output generation task and the descriptive semantic similarity judgment task belong to different task types: the former takes traffic monitoring video as input and performs video understanding and text generation; the latter takes the event description content in the candidate output results, the event description text in the annotated reference information, and the preset verification prompt information as input and performs semantic comparison and judgment. Therefore, the descriptive semantic similarity judgment result is still constrained by the annotated reference information, rather than simply being an unconstrained evaluation of the model's own generated results.

[0055] After obtaining the category consistency determination result and the description semantic similarity determination result, the candidate output results are classified as correct or incorrect. When the category consistency determination result is consistent and the description semantic similarity determination result is semantically similar, the candidate output result is determined as the correct candidate output result; when the category consistency determination result is inconsistent or the description semantic similarity determination result is semantically dissimilar, the candidate output result is determined as the incorrect candidate output result.

[0056] Therefore, the output of this stage is the verification result, the set of correct candidate outputs, and the set of incorrect candidate outputs. This output serves as the input for the next stage of constructing error-correction training samples and discrimination training samples, and is used for subsequent iterative optimization of the large-scale visual language model. The process of verifying candidate outputs and classifying them as correct or incorrect is as follows: Figure 3 As shown.

[0057] IV. Construction of Error Correction Training Samples The input for this stage is the set of correct candidate output results, the set of incorrect candidate output results, and the corresponding traffic monitoring video training samples obtained in the previous stage.

[0058] First, for each erroneous candidate output, a correct candidate output is selected from the correct candidate outputs corresponding to the same traffic monitoring video training sample and paired with it according to a preset pairing rule. The preset pairing rule may include a generation order priority rule, a description completeness priority rule, or other pre-defined rules. Description completeness characterizes the degree to which the event description content covers the event subject, main behaviors, and outcome state.

[0059] Subsequently, error correction training samples are constructed based on the pairing results. Specifically, a traffic monitoring video training sample, an erroneous candidate output, and its paired correct candidate output are combined to form an error correction training sample. The error correction training sample is used to train the current training model to correct erroneous candidate outputs under the same video input conditions, enabling the visual language large model to learn the correction relationship from erroneous output to correct output.

[0060] Furthermore, when all candidate outputs corresponding to a traffic monitoring video training sample are classified as incorrect candidate outputs, it indicates that there are no correct candidate outputs available for pairing with that sample. In this case, the event description text in the annotation reference information corresponding to the sample is used as a substitute for the correct candidate output and paired with the corresponding incorrect candidate outputs to construct an error correction training sample. Through this substitution method, samples where all candidate outputs are incorrect can still participate in error correction training.

[0061] Therefore, the output of this stage is a set of error-correcting training samples. This set of error-correcting training samples serves as one of the inputs for constructing the training set in subsequent iterations, providing error correction supervision signals to the current training model to enhance the self-correction capability of the large visual language model.

[0062] V. Discriminant Training Sample Construction The input for this stage consists of multiple candidate output results corresponding to the same traffic monitoring video training sample, the verification and judgment results formed in the previous stage, and the corresponding annotation reference information.

[0063] First, for the same traffic monitoring video training sample, its multiple candidate output results and corresponding annotation reference information are jointly organized to form a processing unit for discrimination training.

[0064] Subsequently, from the candidate outputs that are judged as correct, a target candidate output is determined according to a preset determination rule. This preset determination rule may include a generation order priority rule, a description completeness priority rule, or other pre-defined rules. In this way, a target candidate output is uniquely determined within the same discrimination training samples.

[0065] After the target candidate output is determined, a discriminative training sample is constructed. Specifically, the traffic monitoring video training sample, multiple candidate outputs corresponding to the same sample, and corresponding labeled reference information are combined to form the discriminative training sample, and the target candidate output is used as the target supervision result in this discriminative training sample. The discriminative training sample is used to train the current training model to identify, rank, or filter the target candidate output from multiple candidate outputs, thereby enhancing the visual language large model's ability to discriminate candidate outputs and its output stability.

[0066] When all candidate outputs corresponding to a training sample of a traffic monitoring video are classified as incorrect candidate outputs, there is no directly identifiable correct candidate output. In this case, the event description text in the annotation reference information corresponding to the sample is used as a replacement for the target candidate output to continue constructing discriminative training samples.

[0067] When all the candidate outputs corresponding to a traffic monitoring video training sample are classified as correct candidate outputs, since there are no incorrect candidate outputs that need to be distinguished, this sample does not need to be used to construct a discrimination training sample in this round of training.

[0068] Therefore, the output of this stage is the discriminative training sample set. This discriminative training sample set serves as one of the inputs for constructing the training set in subsequent iterations, providing candidate discrimination supervision signals to the current training model to improve the candidate result selection ability of the large-scale visual language model. The process of constructing error correction training samples and discriminative training samples is as follows: Figure 4 As shown.

[0069] VI. Iterative Training Set Construction and Model Update The inputs for this stage are the basic training sample set, the error correction training sample set, and the discrimination training sample set.

[0070] First, the basic training sample set, the error correction training sample set, and the discrimination training sample set are organized according to a unified field structure to form an iterative training set. The unified field structure may include a video sample identifier field, a video content field or a video segment index field, a labeling reference information field, a candidate output result field, a verification judgment result field, and a sample type field. This unified field structure allows training samples from different sources to be jointly used within the same training framework.

[0071] Subsequently, the current training model is trained using an iterative training set to obtain an updated model, thereby achieving continuous iterative optimization of the large visual language model. Specifically, during the same training round, basic training samples, error correction training samples, and discrimination training samples are mixed and input into the current training model, and parameter updates are performed uniformly. This allows the current training model to simultaneously obtain basic supervised learning signals, error correction learning signals, and candidate discrimination learning signals in the same training round, thereby improving the large visual language model's ability to understand, describe, and distinguish traffic events.

[0072] After one round of training, the updated model is used as the current training model for the next round, and the processes of generating candidate output results, verifying and judging candidate output results, constructing error-correcting training samples, constructing discriminative training samples, and iteratively updating the training set are repeated. In other words, after each round of updates, the model regenerates candidate output results based on the new parameter state, and converts the new correct and incorrect candidate output results into supervision signals again, thus forming a closed-loop iterative training process of "generation—verification—sample construction—model update". The iterative training set construction and model update process is as follows: Figure 5 As shown.

[0073] Therefore, the output of this stage is the updated model and the next round of training input regenerated based on the updated model, where the updated model is the large visual language model trained through self-correction iteration.

[0074] VII. Stopping Condition Judgment and Model Output The input for this stage is the updated model and corresponding training state information obtained in the previous stage.

[0075] First, determine whether the preset stopping conditions are met. The preset stopping conditions may include any of the following: reaching a preset number of training rounds; the traffic event detection performance on the validation set reaching a preset threshold; or the number of newly identified correct candidate output results no longer increasing in multiple consecutive training rounds.

[0076] If the preset stopping condition is not met, the updated model will be used as the new current training model, and the process will return to the candidate output generation stage to continue the subsequent closed-loop training process.

[0077] When the preset stopping condition is met, the iterative training ends, and the trained traffic incident detection model is output. This traffic incident detection model is a large visual language model obtained through self-correction iterative training. The output traffic incident detection model can output traffic incident type determination results and incident descriptions based on traffic monitoring videos. Furthermore, because it further incorporates error correction and discrimination supervision signals transformed from the results generated by the large visual language model itself during training, it has higher accuracy, stability, and engineering application applicability compared to models trained only with basic supervision.

[0078] VIII. Experimental Verification Results To further illustrate the effectiveness of the method of this invention, in this embodiment, based on the same traffic event video dataset, a general visual language model, a visual language model obtained through basic supervised fine-tuning, and a visual language model obtained using the iterative training method of this invention are compared and verified. The verification includes two parts: traffic event classification performance and semantic consistency of event description text.

[0079] (I) Explanation of Verification Indicators 1. Classification performance indicators Classification performance reflects the model's ability to identify traffic event categories. Considering the differences in sample size across different traffic event categories, this embodiment uses a weighted average method to evaluate classification performance. The metrics used include accuracy, weighted average precision, weighted average recall, and weighted average F1 score.

[0080] Among them, precision represents the proportion of correctly classified samples out of the total sample size; weighted average precision represents the overall result of weighting the precision of each category according to the sample size; weighted average recall represents the overall result of weighting the recall of each category according to the sample size; and weighted average F1 score comprehensively reflects the precision and recall performance of each category, and is weighted by the sample size. These indicators allow for a comprehensive evaluation of the recognition performance of the method of this invention in traffic incident classification tasks.

[0081] 2. Event description text semantic consistency index Event description text semantic consistency reflects the degree of semantic matching between the event description content generated by the model and the reference event description text. This embodiment uses the BERTScore evaluation method for measurement. This method obtains the contextual semantic representation of each token in the text based on a pre-trained language model, and calculates the text semantic precision score, text semantic recall score, and text semantic comprehensive score based on the semantic similarity between the generated text and the reference text.

[0082] Among them, the text semantic precision score is used to reflect the degree of semantic matching between the generated text and the reference text; the text semantic recall score is used to reflect the degree to which the semantic content in the reference text is covered by the generated text; and the text semantic comprehensive score is used to comprehensively characterize the overall semantic consistency between the generated text and the reference text.

[0083] The above indicators are used together to illustrate the effectiveness of the method of the present invention in two aspects: traffic incident category identification and incident description text generation.

[0084] (II) Verification Results In this embodiment, based on the same traffic incident video dataset, the general visual language large model, the basic supervised fine-tuning model, and the model obtained using the iterative training method of this invention were tested respectively. The verification results are shown in Tables 2 and 3.

[0085] Table 2. Examples of verification results of the method of the present invention and the comparative training method. Table 3. Examples of verification results of the method of the present invention and the comparative training method. As shown in Tables 2 and 3, compared to the general visual language model and the visual language model obtained only through basic supervision fine-tuning, the visual language model obtained using the iterative training method of this invention exhibits superior results in both traffic event classification performance and semantic consistency of event description text. Specifically, in terms of classification performance, the weighted average F1 score of the iteratively trained model of this invention reaches 44.11%, higher than both the general visual language model and the basic supervision fine-tuning model; and in terms of text semantic consistency, the comprehensive text semantic score of the iteratively trained model of this invention reaches 73.07%, also higher than the comparative models.

[0086] The above results demonstrate that the method of this invention not only improves the visual language large model's ability to identify traffic event categories, but also enhances the semantic consistency between the generated event descriptions and the reference event description text. This is because, in addition to basic supervised training, this invention further transforms multiple candidate output results corresponding to the same traffic monitoring video training sample into error correction training samples and discrimination training samples. Through multiple rounds of iterative training, newly added supervisory signals are continuously fed back to the current training model, thereby further improving the visual language large model's output accuracy, output stability, and the ability to distinguish candidate output results.

[0087] Figure 6 The comparison of classification performance under different training methods is shown. Figure 7 The results of semantic consistency evaluation of event description text under different training methods are shown. Figure 6 and Figure 7 It can be further explained that the method of the present invention has good application effect in traffic incident detection tasks based on large visual language models.

[0088] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A self-correcting iterative training method for traffic events based on a large visual language model, characterized in that, include: Acquire traffic surveillance video data and perform preprocessing; The preprocessed data is input into the pre-trained traffic incident detection model to obtain the traffic incident detection results; The training process of the traffic incident detection model includes: Step 1: Obtain multiple traffic monitoring video training samples with labeled reference information, and perform basic training on the traffic incident detection model until the training converges to obtain the basic model; Step 2: Repeatedly input the same traffic monitoring video training sample into the base model to obtain multiple candidate output results; based on the labeled reference information, determine whether the multiple candidate output results are correct or incorrect candidate output results; Step 3: Based on the preset pairing rules, match each erroneous candidate output with a correct candidate output to obtain an error correction training sample; construct a discrimination training sample based on the multiple candidate outputs and labeled reference information; construct an iterative training set based on the traffic monitoring video training sample, the error correction training sample, and the discrimination training sample; train the basic model based on the iterative training set until convergence to obtain an updated model. Step 4: Based on the updated model, repeat steps 2 and 3 to update the updated model; continuously update the current training model until the preset stopping condition is reached to obtain the trained traffic event detection model.

2. The traffic event self-correction iterative training method based on a large visual language model according to claim 1, characterized in that, The same traffic monitoring video training sample is repeatedly input into the updated model, and steps two and three are repeated to generate a new iterative training set and update the updated model. The current training model is initially the base model, and the current training model in subsequent training rounds is the model updated in the previous training round.

3. The traffic event self-correction iterative training method based on a large visual language model according to claim 1, characterized in that, The candidate output results include the traffic incident type determination result and the incident description content; The annotation reference information includes traffic event category labels and event description text corresponding to the traffic monitoring video training samples.

4. The self-correction iterative training method for traffic events based on a large visual language model according to claim 1, characterized in that, The traffic incident detection model is fine-tuned under supervision using the traffic monitoring video training samples and the labeled reference information to obtain the base model.

5. The traffic event self-correction iterative training method based on a large visual language model according to claim 1, characterized in that, Based on the labeled reference information, the multiple candidate output results are determined to be either correct or incorrect candidate output results. The corresponding determination process includes: The category consistency determination result is determined based on the correspondence between the traffic event type determination result and the traffic event category label in the candidate output results; The event description content, event description text, and preset verification prompt information in the candidate output results are input into the current training model to output semantic similarity judgment results; based on the category consistency judgment results and semantic similarity judgment results, the multiple candidate output results are judged to be correct candidate output results or incorrect candidate output results.

6. The traffic event self-correction iterative training method based on a large visual language model according to claim 5, characterized in that, When the category consistency determination result is consistent and the semantic similarity determination result is similar, the candidate output result is determined as the correct candidate output result; When the category consistency determination result is inconsistent, or the semantic similarity determination result is dissimilar, the candidate output result is determined as an incorrect candidate output result.

7. The traffic event self-correction iterative training method based on a large visual language model according to claim 1, characterized in that, The error correction training samples include traffic monitoring video training samples, an erroneous candidate output result, and a correct candidate output result paired with it; the error correction training samples are used to train the current training model to correct the erroneous candidate output result based on the same video input.

8. The traffic event self-correction iterative training method based on a large visual language model according to claim 1, characterized in that, The discrimination training samples include traffic monitoring video training samples, multiple candidate output results corresponding to the same traffic monitoring video training sample, and annotation reference information; The discriminative training samples are used to train the current training model to identify the target candidate output result among the multiple candidate output results; The target candidate output result is determined by the candidate output result that is verified as correct according to a preset determination rule.

9. The self-correction iterative training method for traffic events based on a large visual language model according to claim 1, characterized in that, The preset stopping conditions include: the number of training rounds reaches a preset number of training rounds, the traffic incident detection performance reaches a preset threshold, or the number of newly determined correct candidate output results no longer increases in multiple consecutive training rounds.

10. A system for a traffic event self-correction iterative training method based on a large visual language model, characterized in that, It includes a memory and a processor, the memory storing a computer program, the processor invoking the computer program to perform the steps of the method as described in any one of claims 1 to 9.

Images