A traffic accident detection method based on video segment saliency guidance

Abstract

The application discloses a traffic accident detection method based on video segment saliency guidance, and has the characteristics that the method comprises the following steps: collecting urban road traffic data sets, and performing segmentation of training sets and test sets on the urban road traffic data sets; performing similarity saliency preprocessing on the video segments; designing an attention model guided by the saliency; designing a loss function regularization method based on a dynamic time warping algorithm; and fully training the designed network, and the like. Compared with the prior art, the application has the advantages that a more robust video motion aggregation feature is provided for a multi-instance learning algorithm, the traffic anomaly detection capability is enhanced, the anomaly detection performance is improved, the data sparsity and data label missing problems in the anomaly detection task are effectively solved, and therefore, better detection effect than a baseline algorithm is achieved for a weakly supervised video traffic accident detection task based on a MIL framework, and the problem that a conventional weakly supervised video detection algorithm is difficult to be applied to a traffic scene is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video analysis, in particular to a traffic accident detection method based on video segment saliency guidance. BACKGROUND

[0002] With the increase of urban population density, the per capita car ownership is rising year by year, and the incidence of road traffic accidents is also growing. Therefore, timely reporting of traffic accidents is of great significance to protect public safety and reduce property and economic losses. In order to reduce the response time of road traffic accidents, the traffic department sets up a large number of monitoring cameras at important traffic locations (such as crossroads, bridge locations, etc.) and feeds back the real-time traffic conditions of the road in a timely manner through manual observation. With the generation of massive video data, the cost of manual monitoring has increased significantly.

[0003] In recent years, with the development of automation technology, intelligent traffic born through the joint role of computers, Internet of Things and Internet technology is replacing manpower and improving the quality of urban traffic in an automatic and efficient manner. Among them, video-based urban road traffic accident detection (VURTAD) has become an important and challenging task in the field of intelligent transportation. This task obtains urban road traffic monitoring video information continuously, sends these data into a computer vision processing model, calculates abnormal signals belonging to traffic accidents and reports the accurate time of occurrence, and then feeds back the information to rescue and accident handling personnel. Through the realization of VURTAD, the response speed of accidents can be improved while saving manpower costs. Therefore, it is of great significance to research a robust and complete VURTAD algorithm.

[0004] Under weakly supervised training, the mainstream approach for video-based traffic accident detection is the Multiple Instance Learning (MIL) framework, originally proposed by Sultani et al. in 2018. The core idea of ​​this framework is to first extract features from video clips, then represent each video as a bag of multiple single instances. Finally, representative features are selected from the entire group of instances (bags) and backpropagated through a deep model under the constraints of a loss function. This algorithmic framework has been used to model pedestrian anomaly detection tasks, achieving relatively good results. However, in the field of traffic accident detection, this baseline algorithm performs poorly. The core of the MIL task is that the initialized binary classification network must have a certain prior classification capability to ensure that it can select the correct sample instances within the class. However, compared to anomaly detection in pedestrian videos, the feature differences between accidents and normal behavior in traffic videos are less clear. In pedestrian videos, the intrusion of non-pedestrian objects (such as bicycles and cars) and abnormal pedestrian movements (running, bumping) both have significant feature differences from normal walking movements. In traffic accident video data, accidents typically occur during interactions between vehicles or between vehicles and people. The characteristic differences between accident actions and normal traffic behavior are much more subtle than in pedestrian data. These subtle characteristic differences can cause the model to make errors when extracting representative features for groups of instances, accumulating training errors and ultimately limiting traffic accident detection performance. Therefore, in the field of traffic accident detection, it is necessary to explore new weakly supervised training strategies to improve model performance.

[0005] Existing technologies for traffic accident detection in surveillance videos have low accuracy. Subtle differences in motion features in traffic videos are difficult to distinguish, making them prone to misidentification as accidents. However, feature preprocessing can identify key locations in the time series. Therefore, introducing an attention mechanism based on feature similarity differences into this baseline model is a promising approach. This allows the model to focus more on significant time segments, further improving detection performance. Summary of the Invention

[0006] The purpose of the present application is to provide a traffic accident detection method based on video segment saliency guidance to overcome the shortcomings of the prior art. The similarity saliency-guided attention (SSGA) algorithm is used to provide more robust video motion aggregation features for the multi-instance learning algorithm to enhance the detection ability of traffic anomalies. By introducing the similarity saliency attention mechanism, the representative features of the instance group are more accurate. The contrast loss objective function is used to increase the spatial distance between the feature representations of the positive and negative instance groups, which helps to improve the anomaly detection performance. At the same time, by training a binary classifier for each video segment, the weak label supervision information can be fully utilized to further improve the detection accuracy. The overall loss function combined with the DTW regularization loss can effectively deal with the data sparsity and data label missing problems in the anomaly detection task, thereby achieving better detection effect than the baseline algorithm for the weakly supervised video traffic accident detection task based on the MIL framework, improving the stability and robustness of the model in the training process, realizing the detection of abnormal traffic behavior, and solving the problem that the conventional weakly supervised video detection algorithm is difficult to apply to the traffic scene.

[0007] The specific technical solution of the present application is: a traffic accident detection method based on video segment saliency guidance, characterized by using a similarity saliency-guided attention algorithm (SSGA) to provide more robust video motion aggregation features for the multi-instance learning algorithm to enhance the detection ability of traffic anomalies. The method comprises: collecting a city road traffic data set and dividing it into a training set and a test set; similarity saliency preprocessing of video segments; designing a saliency-guided attention model; designing a loss function regularization method based on a dynamic time warping algorithm; and fully training the designed network, the specific steps being as follows:

[0008] Step S1: collecting a city road traffic data set.

[0009] Step S2: dividing the data set into a training set and a test set.

[0010] Step S3: similarity saliency preprocessing of the segments in the video segment package, which introduces an external attention signal for the MIL aggregation model by preprocessing the similarity of the video segments and calculating their saliency.

[0011] The similarity saliency-guided attention algorithm is designed as follows:

[0012] For any video = {f1, f2, …, f n}, where f i(i=1,2,…,n) is the action feature extracted based on the action classification pre-training model, and its similarity significance is defined as S={s1,s2,…,s n}, where s i is the similarity significance of the i-th segment defined by the following formula (a):

[0013]

[0014] Where d(-,-) is the cosine similarity between feature vectors expressed by the following formula (a-1):

[0015] d(f a ,f b )=Cosine(f a ,fb) (a-1).

[0016] By normalizing the video similarity saliency S to 0-1, the following formula (b) can be obtained to represent the standardized video saliency signal:

[0017]

[0018] in, and s∈[0,1] is the similarity significance of the i-th segment; R T is the set of real numbers.

[0019] In the MIL paradigm, for the video clip package B a ,B n , and the corresponding significance signals are calculated by the above methods respectively Subsequently, these saliency signals are fed into the module designed in the next step as external attention signals to guide the attention mechanism model to perform calculations.

[0020] Step S4: Design an attention model guided by this saliency.

[0021] The saliency signal obtained in step S3 Combine the instance group feature F with the saliency signal At the same time, we input Q=f into the model, V = f. After performing linear transformations on Q, K, and V, the attention score A of the instance group feature on the salient feature is calculated using the following formula (c):

[0022] A=(QW 9 )(KW K ) T (c).

[0023] Among them, W 9 and W Kare the transition matrices of Q and K, i.e., the linear transformation weights; Q, K, and V are the Query vector, Key vector, and Value vector, respectively.

[0024] Perform Softmax normalization on the attention score A to obtain the normalized attention score W expressed by the following formula (d):

[0025] W = Softmax(A) (d).

[0026] The original features and the normalized attention scores are processed with residuals to obtain the output feature H of the aggregation model represented by the following formula (e):

[0027] H=W(VW V )+F (e).

[0028] Among them, W V is the transition matrix of V.

[0029] Step S5: Design a loss function regularization method based on the dynamic time warping algorithm. Combining the research results of Tian et al., a targeted design was conducted on the urban road traffic dataset and accident detection task.

[0030] This paper introduces the Dynamic Time Warping (DTW) algorithm to regularize the model training process. Dynamic Time Warping uses a weaker constraint than MSE to limit the scoring ability of the aggregate model to the instance, making it closer to the distribution of the feature similarity significance difference signal while ensuring the learning ability of the aggregate model. As a set of pseudo labels belonging to abnormal packet signals, the dynamic time warping loss is defined as By introducing the DTW loss, the model will tend to select segments with significantly different action features as representative instance groups when selecting segment instances within a video package, and use them in the loss function calculation. This makes the model training process more stable.

[0031] In the actual training process, since the DTW solution is discrete, it cannot be directly applied as a loss function to the gradient derivation process of the neural network. In order to make DTW a part of the loss function, the present invention uses the Soft Dynamic Time Warping (SDTW) algorithm. First, the contrast target loss function is defined, which helps to increase the feature representation space distance between the positive and negative instance groups, and obtains the regularized loss function represented by the following formula (f):

[0032]

[0033] y a =1,y n =0, the others are 0.

[0034] in, is the video action aggregate feature, the subscript n represents the feature representation aggregated from normal videos, and a represents the aggregate feature representation from abnormal videos; m is the margin parameter used to limit the anomaly score interval for comparing positive and negative instance groups; d θ,k (-) is the difference in feature strength between a group of positive and negative instance groups calculated by the aggregation model θ; k is the number of feature strength values ​​selected from the beginning after sorting in descending order as the representative of the instance group; y i and y j They represent frame-level annotations, which are 0 for normal segments and 1 for abnormal segments.

[0035] In addition, a binary classifier is trained for each video clip (packet) to fully utilize the weak label supervision information of the packet. The present invention selects the video clip action aggregation representation g with the largest k feature strength values φ (H) is used as logits and trained using the following binary cross entropy function to obtain the contrast target loss function expressed by the following formula (g):

[0036]

[0037] in, The aggregated representation of the video clip action representing the largest k feature strength values ​​is used. This loss function can maximize the correct prediction of whether the video clip is abnormal.

[0038] Combining the above two loss functions, we get the overall loss function expressed by the following formula (h):

[0039] Among them, λ1 and λ2 represent the binary cross entropy loss weight and DTW regularization loss weight, respectively.

[0040] Step S6: Based on the saliency-guided attention model designed in step S4 and derived from step S3, the designed network is fully trained using the loss function designed in step S5 and the dataset prepared in step S2.

[0041] Step S7: After the model is fully trained, the parameters are fixed and the detection results of the test set are output.

[0042] Step S8: Use AUC as a metric to evaluate anomaly detection performance and test the detection effect of the model.

[0043] Compared with the prior art, the present invention has the following beneficial technical effects and significant improvements:

[0044] 1) By introducing the similarity saliency attention mechanism, the representative features of the instance group are extracted more accurately, and the contrast loss objective function is used to increase the spatial distance between the feature representations of the positive and negative instance groups, which helps to improve the anomaly detection performance.

[0045] 2) By training a binary classifier for each video clip, weakly labeled supervision information can be fully utilized to further improve detection accuracy. The overall loss function combined with the DTW regularization loss can effectively address the data sparsity and missing data labels in anomaly detection tasks, thus achieving better detection results than the baseline algorithm for the weakly supervised video traffic accident detection task based on the MIL framework.

[0046] 3) The stability and robustness of the model during training are improved, the detection of abnormal traffic behavior is realized, and the problem that conventional weakly supervised video detection algorithms are difficult to apply to traffic scenarios is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 Schematic diagram of the saliency-guided attention mechanism (SSGA) model of the present invention architecture. DETAILED DESCRIPTION

[0048] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0049] Example 1

[0050] See Figure 1 , an attention algorithm based on video clip saliency guidance for traffic accident detection. The specific implementation steps and details include:

[0051] Step S1: Collect urban road traffic data set

[0052] The simulated dataset CTAD and the real datasets TAD, CADP and UTD traffic accident video data are used as the training and test datasets of the present invention. The statistics of the datasets are shown in Table 1 below:

[0053] Table 1 Traffic accident dataset statistics

[0054]

[0055] The above real traffic video dataset is briefly introduced as follows:

[0056] (1) TAD dataset, collected from YouTube by Hui Lv et al., includes real-world traffic surveillance videos of accidents, traffic violations, and normal events. TAD provides frame-level annotations. This paper uses 250 normal traffic videos and 104 accident videos for experiments.

[0057] (2) Car Accident Detection and Prediction (CADP) dataset, collected from YouTube. In the dataset, all video clips contain accidents. This paper uses CADP data to test the effectiveness of the model trained on the CTAD simulation dataset, so all data is used as the test set.

[0058] (3) Urban Traffic Dataset (UTD). This example collects a real-world urban traffic dataset (UTD), which includes 83 traffic surveillance videos with an average length of 15 minutes. Each video is divided into two parts: normal and accident segments, and the accident segments are manually labeled at the frame level.

[0059] Step S2: Split the dataset into a training set and a test set. Both the training set and the test set contain normal videos V n and abnormal video V a Among them, all videos in the training set only contain normal and abnormal labels for the entire video. In order to test the performance of anomaly detection in the test set, it is necessary to configure the abnormal video with the frame-level label where the anomaly occurs.

[0060] Step S3: Preprocessing the similarity saliency of video clips. By preprocessing the similarity of video clips and calculating their saliency, an external attention signal is introduced to provide the MIL aggregation model. The algorithm is designed as follows: for any video V = {f1, f2, ..., f n}. Among them, f i (i=1,2,…,n) is the action feature extracted based on the action classification pre-training model, and its similarity significance is defined as S={s1,s2,…,s n}, where s i The similarity significance of the first segment is defined by the following formula (a):

[0061]

[0062] Where d(-,-) is the cosine similarity between feature vectors defined by the following formula (a-1):

[0063] d(f a ,fb )=Cosine(f a ,f b ) (a-1).

[0064] By normalizing the video similarity saliency S to 0-1, we can obtain the following formula (b) a standardized video saliency signal:

[0065]

[0066] Where, Where s∈[0,12.

[0067] In the MIL paradigm, for the video clip package B a ,B n The present invention calculates the corresponding significance signals by the above methods. Subsequently, the present invention sends these saliency signals as external attention signals to the module designed in the next step to guide the attention mechanism model to perform calculations.

[0068] Step S4: Design an attention model guided by this saliency.

[0069] The saliency signal obtained in step S3 Combine the instance group feature F with the saliency signal At the same time, we input Q=f into the model, V = f. After performing linear transformations on Q, K, and V, the attention score A of the instance group feature on the salient feature is calculated using the following formula (c):

[0070] A=(QW Q )(KW K ) T (c.)

[0071] Among them, W 9 ,W K and W V are the transition matrices of Q, K, and V, i.e., the linear transformation weights. Then, the attention score A is subjected to Softmax normalization to obtain the normalized attention score W represented by the following formula (d):

[0072] W = Softmax(A) (d).

[0073] The original features and the normalized attention scores are processed with residuals to obtain the output feature H of the aggregation model represented by the following formula (e):

[0074] H=W(VW V )+F (e).

[0075] Step S5: Design a loss function regularization method based on the dynamic time warping algorithm. This invention combines the research results of Tian et al. and conducts targeted design on urban road traffic data sets and accident detection tasks. According to the research conclusions of Tian et al., the aggregated features obtained in the previous step are The value of the 2-norm calculation for each instance h has the following rules in the action visual feature space of the video clip: a In the feature space, instances with larger feature strength values ​​are more densely distributed, while instances with the largest feature strength values ​​in negative sample (normal video) packets are more sparsely distributed in the feature space. Therefore, in their research, for the calculation of a set of positive and negative packets, only the k instances with the largest feature strengths were selected from each packet for comparative learning. During the experiments, it was observed that when selecting instances from positive packets, the model still inevitably selected instances with negative categories, resulting in errors during training. This phenomenon is caused by the feature representations of some negative examples being close to those of positive examples, making them impossible to separate from positive examples during the feature strength calculation process.

[0076] In order to avoid the impact of this phenomenon on the model, the present invention introduces the Dynamic Time Warping (DTW) algorithm to regularize the model training process. Dynamic Time Warping uses a weaker constraint than MSE to limit the scoring ability of the aggregate model to the instance, making it closer to the distribution of the feature similarity significance difference signal while ensuring the learning ability of the aggregate model. As a set of pseudo labels belonging to abnormal packet signals, the dynamic time warping loss is defined as By introducing the DTW loss, the model will tend to select segments with significantly different action features as representative instance groups when selecting segment instances within a video package, and use them in the loss function calculation. This makes the model training process more stable.

[0077] It is worth noting that in the actual training process, since the DTW solution is discrete, it cannot be directly applied as a loss function to the gradient derivation process of the neural network. In order to make DTW a part of the loss function, the present invention uses the Soft Dynamic Time Warping (SDTW) algorithm. First, the contrast target loss function is defined, which helps to increase the feature representation space distance between the positive and negative instance groups, and obtains the regularized loss function represented by the following formula (f):

[0078]

[0079] y a =1,y n=0, the others are 0.

[0080] in, is the video action aggregate feature calculated in the previous step. The subscript n represents the feature representation aggregated from normal videos, and a represents the aggregate feature representation from abnormal videos. m represents the margin parameter, which is used to limit the anomaly score interval between positive and negative instance groups. θ,k (-) is the difference in feature strength between a set of positive and negative instance groups calculated by the aggregation model, where θ is the aggregation model and k is the number of feature strength values ​​selected from the beginning after sorting in descending order as the representative instance group. i and y j They represent frame-level annotations, which are 0 for normal segments and 1 for abnormal segments.

[0081] The present invention trains a binary classifier for each video clip package to fully utilize the weak label supervision information of the package. The video clip action aggregation representation g with the largest k feature strength values ​​is selected. φ (H) is used as logits and trained using the following binary cross entropy function to obtain the contrast target loss function expressed by the following formula (g):

[0082]

[0083] in, The aggregated representation of the video clip action representing the largest k feature strength values. This loss function can maximize the correct prediction of whether a video clip is abnormal.

[0084] Combining the above two loss functions, we get the overall loss function expressed by the following formula (h):

[0085] Among them, λ1 and λ2 represent the binary cross entropy loss weight and DTW regularization loss weight, respectively.

[0086] Step S6: Based on the saliency-guided attention model designed in step S4 and derived from step S3, the designed network is fully trained using the loss function designed in step S5 and the dataset prepared in step S2.

[0087] The present application uses a NVIDIA GeForce GTX 3090 graphics card under the operating system of Ubuntu 20.04 to perform 15,000 rounds of iterative training on each data set. In each iteration of training, the present application sets the batch to 64, each video sampling segment T to 32, and each video segment to consist of 16 frames. For each video segment, the present application extracts action features using an I3D model and performs data augmentation using a 10-crop method, so that the video action feature representation of each segment contains 10x2048 floating-point data. When calculating the saliency signal, the average pooling method is used to reduce the dimension of the action features.

[0088] Step S7: After sufficient training of the model, the parameters are fixed, and the detection results of the test set are output.

[0089] Step S8: The AUC is used as a measure of the performance of the anomaly detection, and the detection effect of the model is tested, and the results are shown in Table 2 as follows:

[0090] Table 2 AUC performance comparison of weakly supervised traffic accident detection

[0091]

[0092]

[0093] In the case of adding saliency attention mechanism, compared with the baseline method, the detection performance of the present application on different data sets is obviously improved. Among them, on the simulated data set CTAD, the AUC index is improved from 0.8133 of the baseline method and 0.8201 of the AR-Net method to 0.8257. In the real data set, on the TAD data set, the algorithm of the present application is improved from 0.6089 and 0.6083 of the above two representative weakly supervised algorithms to 0.6132; on the CADP data set, from 0.5724 and 0.5772 to 0.5879; on the UTD data set, from 0.5316 and 0.5378 to 0.5449.

[0094] From the above experimental results, it can be seen that the similarity saliency attention mechanism introduced in the present application has obvious performance improvement in traffic accident detection. In the past, such representative researches have not added prior statistical data of action features to the aggregation process of the MIL model due to the difference in experimental object data. The design of the present application fully utilizes the difference between traffic anomalies and normal events in traffic video and performs feature enhancement in the training process, thus playing a certain auxiliary role for the task.

[0095] This paper conducted ablation experiments on the DTW loss, adding it to different backbone network models and comparing them with the training results of the original loss function. The resulting AUCs were generally better than those without DTW. This phenomenon may be due to the fact that the model trained with the DTW loss function can maintain a certain correlation with the time series of the motion features represented in the original video, reducing the randomness of the model's selection of representatives from the bag during multi-instance learning, and maximizing the accuracy of the positive and negative example pairs obtained in each round, thereby reducing the model's accumulated error. This experiment shows that DTW loss can enhance model performance to a certain extent.

[0096] The present invention proposes an attention mechanism algorithm based on video saliency guidance, which provides a more robust video action aggregation feature for the multi-instance learning algorithm to enhance the detection ability of traffic anomalies. By introducing the similarity saliency attention mechanism, it is more accurate in extracting the representative features of the instance group, and the contrast loss objective function is used to increase the spatial distance of the feature representation between the positive and negative instance groups, which helps to improve the anomaly detection performance. At the same time, by training a binary classifier for each video clip, the weak label supervision information can be fully utilized to further improve the detection accuracy. Finally, the overall loss function combined with the DTW regularization loss can effectively deal with the data sparsity and data label missing problems in the anomaly detection task, thereby achieving better detection results than the baseline algorithm for the weakly supervised video traffic accident detection task based on the MIL framework, improving the stability and robustness of the model during training, realizing the detection of abnormal traffic behavior, and solving the problem that conventional weakly supervised video detection algorithms are difficult to apply to traffic scenarios.

[0097] The above description is only a preferred embodiment of the present invention and does not limit the implementation mode and protection scope of the present invention. For those skilled in the art, it should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. A traffic accident detection method based on video clip saliency guidance, characterized in that: The traffic accident detection specifically includes the following steps: Step S1: Collect urban road traffic data set; Step S2: Split the collected data set into a training set and a test set; Step S3: Perform similarity saliency processing on the segments in the video segment package and introduce them into the multi-instance learning aggregation model as an external attention signal to obtain a saliency-guided attention model; Step S4: Combine instance group features F and saliency signal At the same time, the above-built attention model is input to calculate the attention score A of the instance group feature F on the salient feature, which is then normalized by Softmax. Then, the residual of the original feature and the normalized attention score A is processed to obtain the output feature H of the aggregation model θ. Step S5: Using the soft dynamic time warping algorithm to regularize the loss function of the training process of the aggregation model θ, and defining a comparative target loss function, training a binary classifier for each video clip package, and combining the regularized loss function and the comparative target loss function to obtain an overall loss function; Step S6: Based on the aggregation model θ designed in step S4, the overall loss function obtained in step S5 is used, and the training set and test set are introduced to train the designed aggregation model θ network; Step S7: Fix the trained parameters and output the test results of the test set; Step S8: Using AUC as a metric to evaluate the anomaly detection performance of the above detection results, the aggregation model is tested to test the detection effect of traffic accidents.

2. The traffic accident detection method based on video clip saliency guidance according to claim 1, characterized in that: The data set segmentation process in step S2 is to use CADP data for the test set and the remaining data for the training set, and the division ratio of the training set to the test set is 7:

3.

3. The traffic accident detection method based on video clip saliency guidance according to claim 1, characterized in that: The similarity significance processing in step S3 specifically includes: S3-1: For any video V={f1,f2,…,f n }, where f1,f2,…,f n Action features extracted based on the action classification pre-training model; S3-2: Similarity significance S = {s1, s2, ..., s n }, where s i The similarity significance of the i-th segment is defined by the following formula (a): Among them, d(-,-) is the cosine similarity between feature vectors: S3-3: Perform 0-1 normalization on the similarity significance S to obtain the standardized significance signal represented by the following formula (b): in, Head s∈[0,1] is the similarity significance of the i-th segment; R T is the set of real numbers; S3-4: Signaling salience It is fed into the aggregation model θ designed in step S4 as an external attention signal to guide the attention mechanism model to perform calculations.

4. The traffic accident detection method based on video clip saliency guidance according to claim 1, characterized in that: The step S4 specifically includes: S4-1: Combine instance group feature F with saliency signal At the same time, input the aggregation model θ and get Q = F, V = F. After performing linear transformation on Q, K, and V, the attention score A of the instance group feature F on the salient feature is calculated by the following formula (c): A=(QW Q )(KW K ) T (c); Among them, W Q and W K are the transition matrices of Q and K, i.e., the linear transformation weights; Q, K, and V are the Query vector, Key vector, and Value vector, respectively; S4-2: Perform Softmax normalization on the attention score A to obtain the normalized attention score W represented by the following formula (d): W = Softmax(A) (d); S4-3: Perform residual processing on the original features and the normalized attention score A to obtain the output feature H of the aggregation model θ represented by the following formula (e); H=W(VW V )+F (e); Among them, W V is the transition matrix of V.

5. The traffic accident detection method based on video clip saliency guidance according to claim 1, characterized in that: The soft dynamic time warping algorithm in step S5 uses the significance signal S θ As a set of pseudo labels belonging to abnormal packet signals, they specifically include: S5-1: Define the dynamic time warping loss as and contrast target loss function; S5-2: By calculating the feature representation space distance between the positive and negative instance groups, we obtain the regularized loss function represented by the following formula (f): in, is the video action aggregation feature, subscript represents the feature representation aggregated from normal videos, and a represents the aggregated feature representation from abnormal videos; is a marginal parameter used to limit the anomaly score interval for comparing positive and negative instance groups; d θ，k (-) is the difference in feature strength between a group of positive and negative instance groups calculated by the aggregation model θ; k is the number of feature strength values ​​selected from the beginning after sorting in descending order as the representative of the instance group; y i and y j Represents frame-level annotations, which are 0 for normal segments and 1 for abnormal segments; S5-3: Train a binary classifier for each video clip package and select the video clip action aggregate representation g with the largest k feature strength values φ (H) is used as logits, and the binary cross entropy function expressed by the following formula (g) is used for training to obtain the contrast target loss function expressed by the following formula (g) in, Aggregate representation of video clip actions representing the largest k feature strength values; S5-4: Combining the above two loss functions, we get the overall loss function represented by the following formula (h): Among them, λ1 and λ2 are the binary cross entropy loss weight and DTW regularization loss weight, respectively.

6. The traffic accident detection method based on video clip saliency guidance according to claim 1, characterized in that: The step S6 trains the designed aggregation model θ network on the Ubuntu 20.04 operating system using the NVIDIA GeForce GTX 3090 graphics card for 15,000 rounds of iterative training for each data set. In each training iteration, the batch is set to 64, each video sampling segment T is 32, each video segment consists of 16 frames, and the I3D model is used to extract action features for each video segment. The data is enhanced using the 10-crop method so that the video action feature representation of each segment contains 10×2048 floating-point data.

7. The traffic accident detection method based on video clip saliency guidance according to claim 1 or claim 3, characterized in that: The significance signal During calculation, the average pooling method is used to reduce the dimension of the action features.

Images