Traffic accident detection method based on VAE model

Abstract

The invention discloses a traffic accident detection method based on a VAE model, and the method comprises the following steps: obtaining original data, and carrying out the primary processing of the obtained data; dividing a data set; converting a data format into a tensor format; performing cross validation on the data; pre-training the VAE model, adjusting and training a classifier, calculating a sample reconstruction error, KL divergence, a potential spatial distance and an output probability of the classifier, and calculating a mixed score through a mixed scoring formula; according to the method, the accuracy of a traffic accident detection model is improved, the accuracy of accident detection in an unbalanced data scene is effectively improved by fusing the characterization learning ability of the VAE and the discrimination ability of the classifier, and a mixed scoring mechanism is further introduced, so that the accuracy of the traffic accident detection in the unbalanced data scene is improved. Information of multiple dimensions such as VAE reconstruction error, KL divergence, potential spatial distance and classifier output probability is integrated, and the robustness of the model is enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to a data processing method, in particular to a traffic accident detection method. BACKGROUND

[0002] In the process of modern urbanization, intelligent transportation systems have become a key component to improve traffic efficiency, ensure travel safety, alleviate traffic congestion, and reduce environmental pollution. Among them, accurate and real-time judgment of traffic flow state, especially rapid detection and identification of traffic accidents, is the core prerequisite for the effective operation of intelligent transportation systems. This directly affects traffic signal control optimization, path induction strategy development, public travel information service quality, and even emergency response speed and efficiency.

[0003] However, traditional traffic accident detection methods based on supervised learning face significant challenges in practical applications. Such methods usually rely on a large amount of training data with accurate labels, i.e., the occurrence and type of accidents need to be clearly defined in advance. However, in complex real traffic environments, obtaining large-scale, high-quality labeled accident data is often costly, time-consuming, and may introduce errors due to the subjectivity and lag of manual labeling. More importantly, real traffic accident data is usually severely imbalanced in quantity compared to normal traffic flow data, i.e., accident samples are much less than non-accident samples. This imbalance in data quantity makes traditional supervised learning models prone to bias towards non-accident data with larger data quantity during training, resulting in poor detection performance for a small number of accident data, manifested as high false negative rate.

[0004] In addition, traditional single detection models or models relying only on a few features often struggle to fully capture the complex patterns and subtle features of accident occurrence. Variational autoencoder (VAE) as a powerful deep generative model can learn effective low-dimensional latent representations of data from unlabeled or large normal data, and is often used in anomaly detection tasks. The basic idea is that the reconstruction error of normal samples is usually lower than that of abnormal samples. However, relying solely on VAE's reconstruction error and other single indicators for classification or detection may not fully utilize all available information in the data, and its discrimination ability is limited, especially in distinguishing real accidents from atypical normal behavior in complex backgrounds, which may result in false positives or false negatives. SUMMARY

[0005] To address the above data labeling difficulties, imbalance between accident data and non-accident data, and insufficient discrimination ability of single models, the present application proposes a traffic accident detection method based on VAE, aiming to fully utilize a large amount of easily accessible non-accident data and a small amount of labeled accident data to build a model that can more accurately and robustly identify different traffic states and traffic accidents.

[0006] The technical scheme adopted by the present application to solve its technical problems is: A traffic accident detection method based on a VAE model, comprising the following steps: Step 1: obtaining original data, the original data including an accident record table derived from a police mobile information system and a traffic flow data table collected from a road sensor network; Step 2: preliminarily processing the data obtained in step 1; inputting the accident record table and the traffic flow data table into a data processing module predefined in MATLAB software, using the data processing module to perform data cleaning and feature extraction on the accident record table and the traffic flow data table, generating an accident data table and a non-accident data table with several dimensional feature data, the label value in the accident data table being set to 1, indicating accident data, the label value in the non-accident data table being set to 0, indicating non-accident data; Step 3: dividing data sets; dividing the data in the accident data table into an accident data training set and an accident data test set in a 50:50 ratio, and dividing the data in the non-accident data table into a normal data training set and a normal data test set in a 70:30 ratio; according to the above data sets, constructing a VAE pre-training set, a classifier training set and an initial test set; wherein the VAE pre-training set is composed of the normal data training set, the classifier training set is composed of the accident data training set and the normal data training set, and the initial test set is composed of the accident data test set and the normal data test set; Step 4: converting the data format of the VAE pre-training set, the classifier training set and the initial test set in step 3 into a tensor format to support subsequent neural network training and reasoning; Step 5: cross-validation of data; performing stratified K-fold cross-validation on the classifier training set to ensure the reliability of model performance evaluation, in the Kth cycle of stratified K-fold cross-validation, dividing the classifier training set into a current fold training subset and a current fold test subset according to the stratification index; performing standardization on the data within the fold, selecting non-accident data with a label value of 0 in the current fold training subset, and calculating the mean and standard deviation of the samples in the current fold training subset and initializing a standardizer, and then using the standardizer to perform standardization processing on the current fold training subset, the current fold test subset and the VAE pre-training set; Step 6: configuring an Adam optimizer for the VAE model and the classifier, the Adam optimizer updating the parameters of the classifier and the parameters of the encoder of the VAE model; Step 7: pre-training the VAE model, using the standardized VAE pre-training set in step 5 to input the VAE model for pre-training, so that the weighted sum of the reconstruction loss and the KL divergence loss of the VAE model is minimized; Step 8: Classifier adjustment training, after the VAE model pre-training is completed, the features of the current fold training subset after step 5 standardization are extracted using the VAE model, the sample data of the current fold training subset is input into the classifier for training, and the parameters of the VAE model are adjusted; Step 9: After the training in step 7 is completed, the current fold test subset is input into the VAE model, and the reconstruction error, KL divergence, latent space distance and classifier output probability of the samples in the current fold test subset are calculated, the hybrid score is calculated by a hybrid score formula, and the specific formula of the hybrid score formula is:

[0007] Wherein is the hybrid weight, is the reconstruction error, is the KL divergence, is the latent space distance, is the classifier output probability, is the hybrid score; The value set of the hybrid weight a is {0, 0.25, 0.5, 0.75, 1.0}, the decision threshold t is uniformly sampled in the interval of the minimum and maximum values of the hybrid score, and the value set of the decision threshold t is {t_1,...,t_20}; Substitute all values of the hybrid weight a into the hybrid score formula to calculate the hybrid score, then enumerate the value set of the decision threshold t for each value of the hybrid weight a, traverse all (a, t) combinations, calculate the F1 score, and obtain the optimal hybrid weight a * and the optimal decision threshold t * ; Step 10: Apply the optimal hybrid weight a * and the optimal decision threshold t * to the current fold test subset, recalculate the hybrid score of the samples in the current fold test subset, and set the threshold to the optimal decision threshold t * to generate a predicted label, calculate the F1 score, AUC and accuracy of all samples in the current fold test subset, write the F1 score, AUC and accuracy into a two-dimensional list, and finally calculate the average value of the F1 score, AUC and accuracy on K folds, and output the average value as the detection result.

[0008] The data processing module specifically includes a spatiotemporal matching module for matching accidents with data, an accident assessment module for defining the severity of accidents, an accident data extraction module for extracting key traffic flow parameters during accident periods and non-accident periods, a non-accident data extraction module, an assessment module for scoring and evaluating data quality and relevance, and a data cleaning and screening module.

[0009] The filtering module scores the completeness of three features—vehicle flow, speed, and occupancy—in the accident record table and traffic flow data table. If all three features are present, 100 points are awarded; if any one feature is missing, 60 points are awarded; if any two features are missing, 30 points are awarded; and if none of the three features are present, 0 points are awarded. Subsequently, the filtering module writes the accident data and non-accident data with a completeness score of 100 into the accident data table and the non-accident data table, respectively.

[0010] The loss function used for the classifier is the binary cross-entropy loss function, and the specific formula for the binary cross-entropy loss function is as follows:

[0011] in The label value of the sample. To predict probabilities.

[0012] The reconstruction loss adopts the mean squared error loss function, and the formula of the mean squared error loss function is as follows:

[0013] in, It is the sample size. It is the true value of the i-th sample. It is the predicted value of the i-th sample.

[0014] The VAE model includes an encoder and a decoder. The encoder has the following structure: it includes two fully connected layers. The first layer receives 30-dimensional input features and performs a linear mapping from 30 to 64 dimensions. The second layer performs a linear mapping from 64 to 32 dimensions. Then it splits into two branches: one branch outputs the latent mean μ through a linear mapping from 32 to 16 dimensions, and the other branch outputs the latent log-variance log(σ²) through a linear mapping from 32 to 16 dimensions, which is used for reparameterization sampling.

[0015] The decoder comprises three fully connected layers: the first layer performs a linear mapping from 16 to 32 dimensions, the second layer performs a linear mapping from 32 to 64 dimensions, and the third layer performs a linear mapping from 64 to 30 dimensions to restore the original input dimension.

[0016] The classifier receives the 16-dimensional latent mean mu of the encoder output of the VAE model as input, first passes through a 16 to 32-dimensional fully connected layer, then passes through a 32 to 1-dimensional fully connected layer to generate a single logit value, and finally obtains the prediction probability through the Sigmoid function activation.

[0017] The activation function in the encoder and the decoder is a LeakyReLU function for non-linear transformation of the input of the encoder and the decoder, and the activation function in the classifier is also a LeakyReLU function.

[0018] The beneficial effects of the present application are: the present application improves the accuracy of the traffic accident detection model by fusing the representation learning ability of VAE and the discrimination ability of the classifier, effectively improves the accuracy of accident detection in the unbalanced data scene, and introduces a hybrid scoring mechanism, which comprehensively considers multiple dimensions of information such as VAE reconstruction error, KL divergence, latent space distance and classifier output probability, and enhances the robustness of the model; and the VAE model in the present application only uses a large amount of non-accident data for unsupervised pre-training, which can deeply learn the internal structure and distribution boundary of normal traffic patterns, and uses a binary cross-entropy loss function with class weights during classifier training, which effectively improves the attention to a small number of accident samples; the present application uses a large amount of easily accessible non-accident data and a small amount of labeled accident data, which reduces the dependence on large-scale artificial labeled data. BRIEF DESCRIPTION OF DRAWINGS

[0019] The present application will be further described below in conjunction with the drawings and embodiments.

[0020] Fig. 1 is the principle flowchart of the present application.

[0021] Fig. 2 is the structure schematic diagram of the VAE model.

[0022] Fig. 3 is the structure schematic diagram of the classifier.

[0023] Fig. 4 is the flowchart of the hybrid scoring mechanism. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical scheme and advantages of the present application more clear and obvious, the present application will be further described in detail below in conjunction with specific embodiments and with reference to the drawings. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0025] It is understood that these descriptions are only exemplary and are not intended to limit the scope of the present application.

[0026] REFERENCE Figs. 1 to 4A traffic accident detection method based on a VAE model, comprising the following steps: Step 1: obtaining original data, wherein the original data comprises an accident record table derived from a police mobile information system and a traffic flow data table collected from a road sensor network; Step 2: performing preliminary processing on the data obtained in step 1; inputting the accident record table and the traffic flow data table into a data processing module predefined in MATLAB software, performing data cleaning and feature extraction on the accident record table and the traffic flow data table by using the data processing module, generating an accident data table and a non-accident data table with several dimensional feature data, setting a label value in the accident data table as 1 to represent accident data, and setting a label value in the non-accident data table as 0 to represent non-accident data; The data processing module predefined in the MATLAB software specifically comprises the following modules and corresponding functions: a main program module responsible for overall process scheduling; a space-time matching module aligning accident events and the closest sensor data according to timestamps and geographic location information to obtain the most representative traffic flow information at the time of the accident; an accident evaluation module loading or defining a severity dictionary of various accidents for subsequent scoring and analysis; an accident data extraction module calculating the occupancy rate, average speed and traffic flow of corresponding sensor points in a predetermined time window before and after the accident for the matched accident record; a non-accident data extraction module selecting a reference period corresponding to the accident period and performing the same occupancy rate, average speed and flow calculation on the same group of sensor data to obtain normal traffic flow characteristics; an evaluation module scoring the quality of each accident data and non-accident data according to the accident severity, time period attribute and data integrity; and a random sampling module randomly sampling a plurality of sample points from the cleaned data for subsequent statistical analysis or balanced sample distribution.

[0027] The present application reduces the dependence on large-scale artificial labeled data by using a large amount of easily accessible non-accident data and a small amount of labeled accident data. The VAE model of the present application is only used for unsupervised pre-training with a large amount of non-accident data, and can deeply learn the internal structure and distribution boundary of normal traffic patterns.

[0028] Step 3: Divide the data set; divide the data in the accident data table into an accident data training set and an accident data test set in a 50:50 ratio, and divide the data in the non-accident data table into a normal data training set and a normal data test set in a 70:30 ratio; according to the above data set, construct a VAE pre-training set, a classifier training set and an initial test set; wherein the VAE pre-training set is composed of the normal data training set, the classifier training set is composed of the accident data training set and the normal data training set, and the initial test set is composed of the accident data test set and the normal data test set; Step 4: Convert the data format of the VAE pre-training set, the classifier training set and the initial test set in step 3 to a tensor format to support subsequent neural network training and inference; The VAE pre-training set, the classifier training set and the initial test set generated in step 3 are loaded as PandasDataFrame or NumPy arrays, and preprocessed according to actual needs on the Python side. For the classifier training set, it can be directly divided into intra-fold training set and intra-fold validation set by stratified sampling, or left for intra-fold cross-validation. All data used for VAE model pre-training, classifier fine-tuning and subsequent verification or testing will finally be converted to torch.tensor format and moved to the specified device (such as CUDA GPU or CPU) to support subsequent neural network training and inference.

[0029] Step 5: Cross-validation of data; stratified K-fold cross-validation is performed on the classifier training set to ensure the reliability of model performance evaluation. In the Kth cycle of stratified K-fold cross-validation, the classifier training set is divided into a current fold training subset and a current fold test subset according to the stratified index; standardization is performed on the intra-fold data, the non-accident data with a label value of 0 is selected from the current fold training subset, and the mean and standard deviation of the samples in the current fold training subset are calculated and a standardizer is initialized, and then the current fold training subset, the current fold test subset and the VAE pre-training set are standardized using the standardizer; Step 6: Configure the Adam optimizer for the VAE model and the classifier, which updates the parameters of the classifier and the parameters of the encoder of the VAE model; Step 7: Pre-training of the VAE model; the VAE pre-training set standardized in step 5 is input into the VAE model for pre-training, so that the weighted sum of the reconstruction loss and the KL divergence loss of the VAE model is minimized; Step 8: Classifier adjustment training, after the VAE model pre-training is completed, the features of the current fold training subset after step 5 standardization are extracted by using the VAE model, the sample data of the current fold training subset are input into the classifier for training, and the parameters of the VAE model are adjusted; Step 9: After the training in step 7 is completed, the current fold test subset is input into the VAE model, and the reconstruction error, KL divergence, latent space distance and output probability of the classifier are calculated for the samples in the current fold test subset, the mixed score is calculated by using a mixed score formula, and the specific formula of the mixed score formula is:

[0030] Wherein is a mixed weight, is a reconstruction error, is a KL divergence, is a latent space distance, is a classifier output probability, is a mixed score; The value set of the mixed weight alpha is {0, 0.25, 0.5, 0.75, 1.0}, and the decision threshold t is uniformly sampled in the interval of the minimum value and the maximum value of the mixed score, and the value set {t_1,..., t_20} of the decision threshold t is obtained; The present application performs grid search on the value range of the preset mixed weight alpha (usually 0, 0.25, 0.5, 0.75, 1.0) and the value range of the decision threshold t (20 uniformly sampled points between the minimum value and the maximum value of all sample fusion scores). Specifically, the mixed score Score is calculated according to each candidate alpha value: the reconstruction error, KL divergence and latent space distance of the current sample are weighted by a proportion (0.1 times KL, 0.5 times latent distance) and multiplied by the mixed weight alpha, and then (1-alpha) times the classifier output probability is added to obtain the final fusion score of the current sample under the mixed weight alpha.

[0031] All values of the mixed weight alpha are substituted into the mixed score formula to calculate the mixed score, and then the value set of the decision threshold t is enumerated for each value of the mixed weight alpha, all (alpha, t) combinations are traversed, the F1 score is calculated, and the optimal mixed weight alpha * and the optimal decision threshold t * are obtained, which make the F1 score maximum. Step 10: The optimal mixed weight alpha * and the optimal decision threshold t *Applying to the current fold test subset, recalculating the mixed score of the samples in the current fold test subset, and setting the threshold value as the optimal decision threshold t * Generate a predicted label, calculate the F1 score, AUC and accuracy of all samples in the current fold test subset, write the F1 score, AUC and accuracy into a two-dimensional list, and finally calculate the average value of the above F1 score, AUC and accuracy on K folds, and output the average value as the detection result; In this embodiment, K is preferably 3, that is, 3-fold cross validation is performed.

[0032] The application improves the accuracy of the traffic accident detection model by fusing the representation learning ability of VAE and the discrimination ability of the classifier, effectively improves the accuracy of accident detection in an unbalanced data scene, and introduces a mixed scoring mechanism, which comprehensively considers VAE reconstruction error, KL divergence, latent space distance and classifier output probability and other multiple dimensions of information, and enhances the robustness of the model.

[0033] The data processing module specifically includes a space-time matching module for matching accidents and data, an accident evaluation module for defining the severity of accidents, an accident data extraction module and a non-accident data extraction module for extracting key traffic flow parameters of accident periods and non-accident periods, an evaluation module for scoring and evaluating data quality and correlation, and a screening module for data cleaning.

[0034] The screening module scores the completeness of the three features of traffic flow, speed and occupancy in the accident record table and the traffic flow data table; if all three are present, 100 points are assigned, if any one is missing, 60 points are assigned, if any two are missing, 30 points are assigned, and if all three are missing, 0 points are assigned; then the screening module writes the accident data and non-accident data with a completeness of 100 points into the accident data table and the non-accident data table, respectively.

[0035] The loss function of the classifier selects a binary cross entropy loss function, and the specific function formula of the binary cross entropy loss function is:

[0036] Among them, is the label value of the sample, is the predicted probability.

[0037] In the application, a binary cross entropy loss function with class weight is used in the training of the classifier, which effectively improves the attention to the minority accident samples.

[0038] The reconstruction loss adopts a mean square error loss function, and the formula of the mean square error loss function is specifically:

[0039] wherein, is the number of samples, is the true value of the i-th sample, is the predicted value of the i-th sample.

[0040] The specific formula for calculating the KL divergence loss is:

[0041] wherein is the sample in the data set, is the probability of the sample in the probability distribution P and Q.

[0042] The reconstruction error is defined as the sum of the squared differences of each feature dimension between the reconstruction result of the current sample and the original input divided by the number of feature dimensions; the KL divergence is defined as the sum of the KL differences of each dimension between the latent distribution of the sample and the standard normal distribution, which specifically includes the superposition of "1 + log variance - square of mean - log variance index" and takes the negative half; the latent space distance is defined as the Euclidean distance between the latent mean extracted by the sample and the average value of the latent mean vector of the normal subset in the fold; the classifier output probability is defined as the logit of Classifier(μ_te) mapped to 0 to 1 by the Sigmoid function. The above four measurement indexes (reconstruction error, KL divergence, latent space distance and classifier probability) are regarded as four different signals participating in subsequent fusion.

[0043] The VAE model includes an encoder and a decoder; the structure of the encoder is specifically: including two fully connected layers, wherein the first layer receives 30-dimensional input features and performs linear mapping from 30 dimensions to 64 dimensions; the second layer is linear mapping from 64 dimensions to 32 dimensions; then it is divided into two branches: one branch outputs the latent mean μ through linear mapping from 32 dimensions to 16 dimensions, and the other branch outputs the latent log variance log(σ²) through linear mapping from 32 dimensions to 16 dimensions, for reparameterization sampling.

[0044] The decoder includes three fully connected layers: the first layer performs linear mapping from 16 dimensions to 32 dimensions, the second layer performs linear mapping from 32 dimensions to 64 dimensions, and the third layer performs linear mapping from 64 dimensions to 30 dimensions to recover to the original input dimension.

[0045] The classifier receives the 16-dimensional latent mean μ output by the encoder of the VAE model as input, first passes through a 16-to-32-dimensional fully connected layer, then passes through a 32-to-1-dimensional fully connected layer to generate a single logit value, and finally obtains the predicted probability through the Sigmoid function activation.

[0046] The activation function in the encoder and the decoder adopts a LeakyReLU function to perform nonlinear transformation on the input of the encoder and the decoder, and the activation function in the classifier also adopts a LeakyReLU function.

[0047] The application adopts a two-stage learning framework: in the first stage, a VAE model is used to learn effective feature representation on unlabeled normal data; in the second stage, a classifier is trained in combination with a small amount of labeled data, and a mixed scoring mechanism is introduced to fuse the multi-dimensional output of the VAE model and the discrimination result of the classifier, so as to realize more accurate detection. The core goal is to significantly improve the accuracy and F1 score of traffic accident detection, especially in the actual scene of limited labeled data, to provide more reliable data support and decision basis for intelligent transportation systems.

[0048] The method of the application performs excellently in key performance indicators. The average experimental results obtained by strict 3-fold cross-validation show that the F1 score reaches about 0.9142, which indicates that the model has achieved an excellent balance between precision (i.e. how many true accidents are predicted as accidents) and recall (i.e. how many true accidents are successfully detected), and is accurate and comprehensive in identifying accidents. The AUC (area under the ROC curve) reaches about 0.9823, which is very close to 1, and powerfully proves that the model can efficiently distinguish between accident samples and non-accident samples under various decision thresholds, has a strong overall discrimination ability and a low probability of misjudgment. At the same time, the average accuracy reaches about 0.9506, which means that the model can correctly classify most samples. These comprehensive high performance indicators fully confirm the effectiveness and advancement of the method of the application in the task of traffic accident detection.

[0049] The inherent class imbalance characteristic of traffic accident data is the main challenge faced by traditional detection methods. The application effectively alleviates this problem through a double mechanism: in the first stage, the VAE model is trained only using a large amount of non-accident data. This enables the VAE model to deeply learn the internal structure and distribution boundary of normal traffic patterns, laying a solid foundation for subsequent differentiation of abnormal data, rather than being biased by a small number of accident samples. In the second stage of classifier fine-tuning training, a binary cross-entropy loss function with specific weights is used. This weighting mechanism significantly improves the model's attention to minority class data, avoiding the model's tendency to simply predict all samples as the majority class to obtain a superficial high accuracy, thereby effectively improving the recall rate of true accidents.

[0050] Through its multi-layer nonlinear transformation, the VAE model encoder can learn more informative and robust latent representations from the original input features that may contain redundancy and noise. These low-dimensional features can better capture the essential differences in the data.

[0051] In the classifier training phase, not only the parameters of the classifier are trained, but also part of the fully connected layers of the VAE model encoder are fine-tuned. This means that the VAE model can be optimized and adjusted according to the final classification task target, so that the extracted latent features are more discriminative, thereby improving the performance of the classifier. This end-to-end learning idea makes the feature extraction and classification tasks better promote each other.

[0052] Traditional VAE-based anomaly detection often only relies on reconstruction error. The application innovatively designs a hybrid scoring mechanism that integrates four different dimension information sources: VAE reconstruction error: reflects the degree of conformity of the input sample to the normal mode learned by the VAE, and the accident sample usually has a higher reconstruction error.

[0053] VAE KL divergence: measures the difference between the sample representation in the latent space and the standard normal prior distribution, and the abnormal latent representation may deviate from the prior.

[0054] Latent space distance: calculates the Euclidean distance between the latent representation of the sample and the average latent representation of all normal training samples, which intuitively measures the deviation from the "normal cluster".

[0055] Classifier output probability: provides direct discriminative information based on supervised learning.

[0056] These information sources are dynamically balanced by a learnable weight factor α, and the three indicators related to the VAE model are further weighted by fixed weights 0.1 and 0.5, combined with the optimized decision threshold t for final judgment. This multi-angle, multi-level information fusion significantly enhances the robustness of the decision, better identifies different types of accident features, and effectively reduces the one-sidedness and misjudgment risk that may be caused by a single indicator.

[0057] The application not only proposes a model architecture and training process, but also contains a systematic parameter optimization method, especially for the weight factor α and the decision threshold t in the hybrid scoring mechanism. Through two-dimensional grid search in each fold of cross-validation, the goal is to maximize the F1 score, ensuring that the model can achieve the best performance under a specific evaluation standard.

[0058] The training of the model is divided into VAE pre-training and classifier fine-tuning. From the training time of the experimental output, the training of each cross-validation fold can be completed within a reasonable time. This shows that the training efficiency of the model is relatively high. At the same time, the structure of the model is mainly based on fully connected layers, which has fast inference speed and is suitable for traffic accident detection application scenarios that have certain real-time requirements.

[0059] In summary, the application shows high performance, high robustness and good practicability in the traffic accident detection task through the carefully designed model structure, the phased training strategy, the effective data imbalance processing mechanism, the innovative multi-source information fusion scoring method and the system parameter optimization process.

[0060] In the present application, the term "a plurality of" refers to two or more, unless otherwise expressly specified. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items. The terms "mounting", "connected", "connecting", "fixed", and the like should be interpreted broadly, for example, "connected" can be fixed connection, can also be detachable connection, or integrally connected; "connected" can be directly connected, or indirectly connected through an intermediate medium. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0061] It should be noted that when an element is referred to as "assembled", "mounted", "fixed" or "disposed" on another element, it can be directly on another element or there can be a middle element. When an element is considered to be "connected" to another element, it can be directly connected to another element or there can be a middle element. The terms "vertical", "horizontal", "up", "down", "left", "right" and similar expressions used herein are for illustrative purposes only and are not the only implementation.

[0062] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "a specific embodiment" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms does not necessarily mean the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0063] Although embodiments of the present application have been shown and described, those skilled in the art can understand that various changes, modifications, replacements and variations can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the claims and their equivalents.

Claims

1. A traffic accident detection method based on a VAE model, characterized in that... Includes the following steps: Step 1: Obtain raw data, which includes accident record tables exported from the police mobile information system and traffic flow data tables collected from the road sensor network; Step 2: Perform preliminary processing on the data obtained in Step 1; input the accident record table and traffic flow data table into the predefined data processing module in MATLAB software, and use the data processing module to perform data cleaning and feature extraction on the accident record table and traffic flow data table to generate an accident data table and a non-accident data table with several dimensions of feature data. The label value in the accident data table is set to 1 to indicate accident data, and the label value in the non-accident data table is set to 0 to indicate non-accident data. Step 3: Split the dataset; The data in the accident data table is divided into an accident data training set and an accident data test set in a 50:50 ratio, and the data in the non-accident data table is divided into a normal data training set and a normal data test set in a 70:30 ratio. Based on the above datasets, a VAE pre-training set, a classifier training set, and an initial test set are constructed. The VAE pre-training set consists of the normal data training set, the classifier training set consists of the accident data training set and the normal data training set, and the initial test set consists of the accident data test set and the normal data test set. Step 4: Convert the data format of the VAE pre-training set, classifier training set, and initial test set from Step 3 into tensor format to support subsequent neural network training and inference; Step 5: Perform cross-validation on the data; perform hierarchical K-fold cross-validation on the classifier training set to ensure the reliability of model performance evaluation. In the Kth fold of the hierarchical K-fold cross-validation loop, the classifier training set is divided into the current fold training subset and the current fold test subset according to the hierarchical index. The data within the current fold is standardized. Non-accident data with a label value of 0 are selected from the current fold training subset. The mean and standard deviation of the samples in the current fold training subset are calculated and the standardizer is initialized. Then, the standardizer is used to standardize the current fold training subset, the current fold test subset and the VAE pre-training set. Step 6: Configure the Adam optimizer for the VAE model and classifier, and the Adam optimizer updates the classifier parameters and the parameters of the VAE model encoder; Step 7: Pre-train the VAE model. Use the standardized VAE pre-training set from Step 5 to input the VAE model for pre-training, so as to minimize the weighted sum of the reconstruction loss and KL divergence loss of the VAE model. Step 8: Classifier training adjustment. After the VAE model pre-training is completed, the features of the current training subset after standardization in step 5 are extracted using the VAE model. The sample data of the current training subset is input into the classifier for training, and the parameters of the VAE model are adjusted. Step 9: After training in Step 7 is completed, the current test subset is input into the VAE model, and the reconstruction error, KL divergence, latent space distance, and output probability of the classifier are calculated for the samples in the current test subset. A mixed score is then calculated using the mixed scoring formula, the specific formula of which is: in For mixed weights, For reconstruction error, Let KL divergence be the KL divergence. For potential spatial distance, Output probabilities for the classifier. For mixed scores; The set of values ​​for the mixed weight α is {0, 0.25, 0.5, 0.75, 1.0}, and the decision threshold t is obtained by uniformly sampling the minimum and maximum values ​​of the mixed score into 20 equal parts, resulting in the set of values ​​for the decision threshold t as {t_1, ..., t_20}. Substitute all possible values ​​of the mixing weight α into the mixing scoring formula to calculate the mixing score. Then, enumerate the set of possible values ​​of the decision threshold t for each value of the mixing weight α, traverse all (α, t) combinations, calculate the F1 score, and obtain the optimal mixing weight α that maximizes the F1 score. * and the optimal decision threshold t * ; Step 10: Calculate the optimal mixed weight α that maximizes the F1 score from step 9. * and the optimal decision threshold t * Applied to the current folded test subset, the mixture score of the samples in the current folded test subset is recalculated, and the threshold is set as the optimal decision threshold t. * Generate predicted labels, calculate the F1 score, AUC and accuracy of all samples in the current fold test subset, write the F1 score, AUC and accuracy into a two-dimensional list, and finally calculate the average of the above F1 score, AUC and accuracy over K folds, and output the average as the detection result.

2. The traffic accident detection method based on VAE model according to claim 1, characterized in that... The data processing module specifically includes a spatiotemporal matching module for matching accidents with data, an accident assessment module for defining the severity of accidents, an accident data extraction module for extracting key traffic flow parameters during accident periods and non-accident periods, a non-accident data extraction module, an assessment module for scoring and evaluating data quality and relevance, and a data cleaning and screening module.

3. The traffic accident detection method based on the VAE model according to claim 2, characterized in that... The filtering module scores the completeness of three features—vehicle flow, speed, and occupancy—in the accident record table and traffic flow data table. If all three features are present, 100 points are awarded; if any one feature is missing, 60 points are awarded; if any two features are missing, 30 points are awarded; and if none of the three features are present, 0 points are awarded. Subsequently, the filtering module writes the accident data and non-accident data with a completeness score of 100 into the accident data table and the non-accident data table, respectively.

4. The traffic accident detection method based on the VAE model according to claim 3, characterized in that... The loss function used for the classifier is the binary cross-entropy loss function, and the specific formula for the binary cross-entropy loss function is as follows: in The label value of the sample. To predict probabilities.

5. The traffic accident detection method based on the VAE model according to claim 4, characterized in that... The reconstruction loss adopts the mean squared error loss function, and the formula of the mean squared error loss function is as follows: in, It is the sample size. It is the true value of the i-th sample. The predicted value of the i-th sample.

6. The traffic accident detection method based on a VAE model according to claim 5, characterized in that... The VAE model includes an encoder and a decoder. The encoder has the following structure: it includes two fully connected layers. The first layer receives 30-dimensional input features and performs a linear mapping from 30 to 64 dimensions. The second layer performs a linear mapping from 64 to 32 dimensions. Then it splits into two branches: one branch outputs the latent mean μ through a linear mapping from 32 to 16 dimensions, and the other branch outputs the latent log-variance log(σ²) through a linear mapping from 32 to 16 dimensions, which is used for reparameterization sampling.

7. The traffic accident detection method based on a VAE model according to claim 6, characterized in that... The decoder comprises three fully connected layers: the first layer performs a linear mapping from 16 to 32 dimensions, the second layer performs a linear mapping from 32 to 64 dimensions, and the third layer performs a linear mapping from 64 to 30 dimensions to restore the original input dimension.

8. The traffic accident detection method based on a VAE model according to claim 7, characterized in that... The classifier receives the 16-dimensional latent mean μ output by the encoder of the VAE model as input, first passes through a 16- to 32-dimensional fully connected layer, then through a 32- to 1-dimensional fully connected layer to generate a single logit value, and finally obtains the predicted probability by activating the Sigmoid function.

9. The traffic accident detection method based on a VAE model according to claim 8, characterized in that... The activation functions in both the encoder and decoder employ the LeakyReLU function to perform a nonlinear transformation on the inputs of the encoder and decoder, and the activation function in the classifier also employs the LeakyReLU function.

Images