Road traffic accident duration prediction method based on CTGAN-Transform

Abstract

The invention provides a road traffic accident duration prediction method based on CTGAN-Transform, and the method comprises the steps: obtaining a historical road traffic accident data set, and constructing a road traffic accident duration prediction model, the road traffic accident duration prediction model comprises a characteristic factor extraction module, a generative adversarial network CTGAN, a wavelet noise reduction module and a Transform time prediction module which are connected in sequence; extracting a characteristic factor matrix related to the accident duration in the historical road traffic accident data set by using a characteristic factor extraction module; inputting the characteristic factor matrix into a generative adversarial network CTGAN for data enhancement to obtain new accident data; performing noise reduction processing on the new accident data by using a wavelet noise reduction module; and inputting the new accident data subjected to noise reduction processing into a Transformer time prediction module for calculation, and accurately predicting the duration of the accident. According to the method, the training process of the model is accelerated while the historical data is enhanced, and meanwhile, the prediction precision is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of road traffic accident duration prediction and analysis, and in particular to a road traffic accident duration prediction method based on CTGAN-Transformer. Background Art

[0002] Under the background of rapid development of motor traffic, traffic accidents have become an important factor affecting the quality of life and life safety of residents. Compared with other forms of traffic accidents, road traffic accidents cause the most losses to personal safety, property safety, and article safety to a certain extent. Secondary accidents or large vehicle accidents will cause paralysis and congestion to the entire road network. An important factor that is closely related to this is the duration of the accident. Accurate prediction of the duration of the accident is the premise for realizing modern road network safety management and intelligent transportation, and it is also one of the important factors affecting the safety and accessibility of the entire road network. The duration of traffic accidents is random and variable, which increases the difficulty of accident management for traffic management departments. In order to reduce the impact of traffic accidents on road safety and accessibility, it is necessary to explore the evolution law of the duration of traffic accidents. Accurate and scientific prediction of the duration of road traffic accidents is conducive to controlling the overall time of the accident, which can effectively shorten the time between the arrival of the response team and the resumption of normal traffic, and has a great impact on the treatment of emergency departments; at the same time, it can adjust the traffic flow in time and take corresponding measures to avoid congestion and secondary occurrence of traffic accidents.

[0003] Among the existing research methods, most studies use traditional statistical analysis techniques, machine learning or deep learning techniques to conduct multidimensional analysis of historical accident data. These techniques require a large amount of historical road traffic accident data to better capture the coupling relationship between data; traditional statistical analysis techniques require a large amount of historical data to support them, and there is a certain degree of subjectivity. Machine learning and deep learning also require a large amount of historical data, but due to the lack of or abnormal records of accident data recorded by sensors on the road, the amount of historical accident data is small, and it is impossible to make an accurate prediction of the accident duration. At the same time, the accuracy of accident duration prediction has not been widely carried out, and the model training time is long. Most accident duration studies only use historical accident data for analysis, and do not perform data enhancement or expansion on insufficient data.

[0004] The invention patent with application number CN2022109465070 discloses a method for predicting the duration of highway accidents. The steps include data collection, data cleaning, variable classification and assignment, using the LASSO algorithm to screen variables related to the accident duration, using chain equation multiple interpolation MICE to fill in missing variables, and building an accident duration prediction model based on a complete sample set: first, a feedforward neural network including an input layer, two hidden layers and an output layer is constructed to map the data set, and then a COX semi-parametric model is constructed for predicting the duration of highway accidents. The COX model based on deep learning in the above invention has a certain tolerance for missing data, and the data enhancement strategy of deep learning can make the model more resilient to common data noise and data missing in the data set. However, it does not take into account the problems of missing or abnormal historical accident data and the efficient operation of the model. Summary of the invention

[0005] In view of the technical problems that the existing technology does not take into account the missing or abnormal historical accident data and the efficient operation of the model, the present invention proposes a road traffic accident duration prediction method based on CTGAN-Transformer, which constructs a historical road traffic accident data enhancement model by using conditional tabular data synthesis using generative adversarial networks (CTGAN) to effectively expand the accident data, and then uses wavelet denoising technology to perform denoising on the expanded accident data, and then uses the parallel processing capability of the Transformer time prediction model to accurately predict the accident duration, thereby enhancing historical data while achieving accurate prediction of the accident duration.

[0006] In order to achieve the above object, the technical solution of the present invention is achieved as follows:

[0007] A road traffic accident duration prediction method based on CTGAN-Transformer, comprising the steps of:

[0008] S0: Obtain a historical road traffic accident data set and construct a road traffic accident duration prediction model, wherein the road traffic accident duration prediction model includes a feature factor extraction module, a generative adversarial network CTGAN, a wavelet denoising module and a Transformer time prediction module connected in sequence;

[0009] S1: Extract the characteristic factor matrix related to accident duration from the historical road traffic accident dataset using the characteristic factor extraction module;

[0010] S2: Input the feature factor matrix into the generative adversarial network (CTGAN) for data enhancement to obtain new accident data;

[0011] S3: Use the wavelet denoising module to perform denoising on the new accident data;

[0012] S4: The new accident data after noise reduction is input into the Transformer time prediction module for calculation to accurately predict the accident duration.

[0013] The generative adversarial network CTGAN described in step S0 includes a conditional classifier, a generator and a discriminator connected in sequence; the Transformer time prediction module described in step S0 is composed of an encoder and a decoder connected in sequence, the encoder includes multiple encoder layers, the decoder includes multiple decoder layers, each encoder layer is composed of a multi-head attention layer and a fully connected feedforward layer, and each decoder layer is composed of a masked multi-head attention layer, a multi-head attention layer and a fully connected feedforward layer connected in sequence.

[0014] The method for extracting the characteristic factor matrix related to the accident duration in the historical road traffic accident data set described in step S1 is:

[0015] X=[x ij ]

[0016] Where X represents the characteristic factor matrix extracted from historical road traffic accident data; x ij represents the jth characteristic factor of the i-th accident; i∈0~M, M represents the total number of accidents; j∈0~N, N represents the total number of characteristic factors for each accident;

[0017] The method of inputting the characteristic factor matrix into the generative adversarial network CTGAN for data enhancement described in step S2 is: sampling a random noise vector from the noise distribution in the conditional classifier, obtaining a conditional vector according to the characteristic factor matrix, obtaining the random noise vector and the conditional vector from the conditional classifier, inputting them into the generator and outputting generated data simulating real data, then inputting the generated data and the characteristic factor matrix representing the real data into the discriminator for discrimination, the discriminator discriminates the generated data and the characteristic factor matrix representing the real data to update the discriminator, and then feeding back the discrimination result of the generated data to the generator to update the generator, minimizing the discriminator loss and maximizing the generator loss according to the objective function of the generative adversarial network CTGAN, and finally generating new accident data x that the discriminator cannot judge fake .

[0018] The probability density function of the noise distribution is:

[0019]

[0020] Where z represents the random noise vector; μ is the mean; σ is the standard deviation;

[0021] The method to obtain the conditional vector based on the characteristic factor matrix is:

[0022]

[0023] Among them, K means that there are K discrete features in the M*N feature factors in the feature factor matrix, and C k Indicates that each discrete feature has C k categories;

[0024] The method for updating the discriminator is: calculating the discriminator loss according to the discriminator loss function, performing back propagation and parameter update; the discriminator loss function is:

[0025] V(D)=logD(x)+log(1-D(G(z,x cond )))

[0026] Among them, x represents the real data sample vector obtained from the feature factor matrix X, D() represents the discriminant function of the discriminator, and G() represents the generator generation function;

[0027] The method for updating the generator is: calculating the generator loss according to the generator loss function, performing back propagation and parameter update: Generator loss function

[0028] V(G)=log(1-D(G(z,x cond )))

[0029] The objective function of the generative adversarial network CTGAN is:

[0030]

[0031] in, represents the expected value of the real data sample vector x obtained from the feature factor matrix X, E z～p(z) Represents the expected value of a random noise vector z sampled from the noise distribution p(z).

[0032] The method of using the wavelet denoising module to denoise the new accident data in step S3 is: fake The noisy signal is decomposed into wavelet coefficients of different frequencies through discrete wavelet transform. By setting a threshold, the wavelet coefficient is compared with the set threshold. According to the comparison result, the wavelet coefficient is set to zero or shrunk to zero by a threshold amount. Then, the wavelet coefficient processed by the threshold is reconstructed into the reply signal through inverse discrete wavelet transform.

[0033] The new accident data x fakeThe noisy signal is expressed as:

[0034] s(t)=y(t)+n(t)

[0035] Among them, y(t) is the new accident data x generated by the CTGAN model fake The corresponding original signal; n(t) is a high-frequency noise signal; the method of decomposing into wavelet coefficients of different frequencies by discrete wavelet transform is:

[0036]

[0037] Among them, j represents the scale of decomposition, l represents the position index at this scale, and Ψ j, l(t) represents the scaled and translated version of the continuous wavelet function at a specific scale j and position l; secondly, the wavelet coefficient is compared with the threshold by setting a threshold, and the wavelet coefficient is set to zero or adjusted according to the comparison result; the threshold processing process is as follows:

[0038]

[0039] When the wavelet coefficients after threshold processing When the absolute value of is less than the threshold λ, the wavelet coefficient is set to zero; however, when the wavelet coefficient after threshold processing When the absolute value of is greater than or equal to the threshold, it is shrunk toward zero by a threshold amount; where sgn(d j, l) is the wavelet coefficient The symbolic function of

[0040] Finally, the processed wavelet coefficients Perform wavelet reconstruction to restore the denoised signal The process of wavelet reconstruction is as follows:

[0041]

[0042] The method for inputting the Transformer time prediction module for calculation described in step S4 is: the output signal after the inverse discrete wavelet transform is used as the Transformer time prediction module to undergo embedding operation and position encoding in sequence to obtain the encoder embedding vector E, the encoder embedding vector E is then subjected to encoder calculation and then enters the decoder for decoder calculation, and the output result of the decoder is passed through the linear transformation layer and the softmax layer to obtain the predicted probability of the accident duration data.

[0043] The method for performing encoder calculation is as follows: the input feature sequence of the first encoder layer is the encoder embedding vector E, and the input feature sequence of each subsequent encoder layer is the output feature sequence of the previous encoder layer. The input feature sequence of each encoder layer is linearly transformed to generate corresponding queries, key values, and values. The queries, key values, and values ​​are calculated by multi-head attention and then sequentially subjected to Add&Norm operations, Feed Forward operations, and Add&Norm operations to obtain the output feature sequence of each encoder layer. After multiple encoder layer calculations, the last encoder layer obtains the output feature sequence Y.

[0044] The decoder calculation method is as follows: the output feature sequence of the decoder in the previous time step is subjected to embedding operation and position encoding to obtain the input embedding vector of the decoder, the input feature sequence of the first decoder layer is the input embedding vector of the decoder, and then the input feature sequence of each decoder layer is the output feature sequence of the previous decoder layer, and the input feature sequence of each decoder layer is subjected to linear transformation to obtain the corresponding query, key value and value, and then the masked multi-head attention calculation and Add&Norm operation are performed to obtain the masked multi-head attention output feature sequence, and then the query Q corresponding to the masked multi-head attention output feature sequence of each decoder layer is obtained through linear transformation. m , the output feature sequence Y of the last encoder layer undergoes linear transformation to obtain the key value K1 and value V1, the key value K1, value V1 and the query Q corresponding to the output feature sequence of each decoder layer mask multi-head attention m After multi-head attention calculation, the output feature sequence of each decoder layer is obtained by Add&Norm operation, Feed Forward operation and Add&Norm operation in sequence. After the decoder calculation, the last decoder layer obtains the output feature sequence Z of the decoder at the current time step t , the output feature sequence of each time step passes through the linear transformation layer and the softmax layer in turn to obtain the predicted probability of the accident duration data.

[0045] The multi-head attention calculation process is:

[0046] MultiHead(Q b ,K b ,K b )=Concat(head1,…,head h ,…,head H )W O

[0047] head h =Attention(Q (h) ,K (h) ,V (h)), for h=1,…,H.

[0048]

[0049] Among them, Q b ,K b ,K b Represents the query, key and value input to each multi-head attention layer, Q (h) ,K (h) ,V (h) The query, key, and value corresponding to each head are obtained after linear transformation. d represents the dimension of the input vector of each multi-head attention layer, H represents the number of heads, and head h represents the hth head, and softmax is the activation function;

[0050] The calculation process of the masked multi-head attention layer is:

[0051] maskedMultiHeadAttention(Q m ,K1,V1,M)=Concat(Head1,…,Head h …,Head H )W O

[0052]

[0053] Among them, Q m ,K1,V1 represent the query, key and value input to each masked multi-head attention layer, Q′ m ,K′1,V′1 represents the query,key and value corresponding to each head obtained after linear transformation, d k Represents the dimension of the input vector of the masked multi-head attention layer;

[0054] The calculation process of the Feed Forward operation is:

[0055] FFN(XIN)=max(0,XINW a +b a )W b +b b

[0056] Among them, XIN represents the input feature of the Feed Forward operation, W a , W b is the weight, b a 、b b is the bias, max is the ReLu activation function;

[0057] The output feature sequence of each time step passes through the linear transformation layer and the softmax layer in turn to obtain the calculation process of predicting the accident duration:

[0058] p=W c Z t +b d

[0059]

[0060] Among them, W c is the weight, b d is the bias, p is the log probability vector of the output, p i represents the value of the i-th element in the log-probability vector p, ks represents the size of the output vocabulary, and σ() represents the normalization operation.

[0061] Beneficial effects of the present invention:

[0062] The present invention extracts relevant feature factor matrices from historical road traffic accident data and uses them as the input of a generative adversarial network (CTGAN) to conduct adversarial work between the generator and the discriminator to generate new accident data, effectively solving the problem of inaccurate prediction caused by insufficient historical accident data.

[0063] The present invention utilizes wavelet denoising technology to perform denoising on the expanded data, and combines the set threshold to reduce or set the wavelet coefficient to zero to achieve the denoising process, effectively solving the problem of large prediction errors caused by more noise in the data; uses the Transformer time prediction model to process the denoised data in parallel, accelerates the training of the model to effectively predict the accident duration, and enhances the historical accident data before making an accurate prediction of the accident duration.

[0064] The present invention combines historical accident data to achieve effective prediction of the duration of road traffic accidents. While enhancing historical data, it accelerates the training process of the model and improves prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0066] Figure 1 It is a flow chart of the present invention.

[0067] Figure 2It is a structural diagram of the CTGAN model of the present invention.

[0068] Figure 3 It is a structural diagram of the Transformer time prediction model of the present invention. DETAILED DESCRIPTION

[0069] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0070] A road traffic accident duration prediction method based on CTGAN-Transformer, such as Figure 1 As shown, the following steps are included:

[0071] S0: Obtain a historical road traffic accident data set and construct a road traffic accident duration prediction model, wherein the road traffic accident duration prediction model includes a feature factor extraction module, a generative adversarial network CTGAN, a wavelet denoising module and a Transformer time prediction module connected in sequence.

[0072] The generative adversarial network CTGAN includes a conditional classifier, a generator and a discriminator connected in sequence; the Transformer time prediction module consists of an encoder and a decoder connected in sequence, the encoder includes multiple encoder layers, the decoder includes multiple decoder layers, each encoder layer consists of a multi-head attention layer and a fully connected feedforward layer, and each decoder layer consists of a masked multi-head attention layer, a multi-head attention layer and a fully connected feedforward layer connected in sequence.

[0073] S1: Extract the characteristic factor matrix related to accident duration from the historical road traffic accident dataset using the characteristic factor extraction module;

[0074] The extraction process of the characteristic factor matrix is ​​as follows:

[0075] X=[x ij ]

[0076] Where X represents the characteristic factor matrix extracted from historical road traffic accident data; x ij represents the jth characteristic factor of the i-th accident; i∈0~M, M represents the total number of accidents; j∈0~N, N represents the total number of characteristic factors for each accident.

[0077] This experiment uses a foreign road traffic accident data set as research data. After deleting null values ​​and outliers, the final screened road traffic accident characteristic factors are shown in Table 1 below.

[0078] Table 1. Factors affecting road traffic accidents

[0079]

[0080] According to the above table, the characteristic factor matrix related to the accident duration is extracted as follows:

[0081]

[0082] S2: Input the feature factor matrix into the generative adversarial network (CTGAN) for data enhancement to obtain new accident data.

[0083] like Figure 2 As shown, the CTGAN model consists of a generator, a discriminator, and a conditional classifier. It is a GAN architecture designed specifically for tabular data generation and is designed to process heterogeneous data containing both numerical and categorical features.

[0084] The specific processing process is as follows: a random noise vector is sampled from the noise distribution in the conditional classifier, a conditional vector is obtained according to the characteristic factor matrix, the random noise vector and the conditional vector are obtained from the conditional classifier, and the generated data simulating the real data is output after being input into the generator, and then the generated data and the characteristic factor matrix representing the real data are input into the discriminator for discrimination, the discriminator discriminates the generated data and the characteristic factor matrix representing the real data to update the discriminator, and then the discrimination result of the generated data is fed back to the generator to update the generator, and the discriminator loss is minimized and the generator loss is maximized according to the objective function of the generative adversarial network CTGAN, and finally new accident data x that the discriminator cannot judge is generated. fake .

[0085] The function of the generator is to input a random noise vector and a conditional vector. The goal is to learn the distribution of real accident data and generate false sample data similar to the real accident data.

[0086] The method for updating the generator is: calculating the generator loss according to the generator loss function, performing back propagation and parameter update. The generator loss function is as follows:

[0087] V(G)=log(1-D(G(z,x cond )))

[0088] Where G represents the generator, G() represents the generator generation function; D represents the discriminator; D() represents the discriminant function of the discriminator; z represents the random noise vector, which can be expressed as z=[z1,z2,…,zd ] T , d represents the dimension of the input feature factor matrix X; x cond Represents the conditional vector; the real accident data, i.e., the feature factor matrix X, contains M samples (i.e., accident data), each sample has N features, and the M*N accident data features include K discrete features. For each discrete feature f k (where k∈{1,2,…,K}), assuming that C k possible categories, then the conditional vector x cond Represents the sum of the total number of all discrete feature categories, that is, G(z,x cond ) represents the generated fake sample data. The goal of the generator is to maximize the generator loss to increase the discriminator’s misclassification rate of the generated samples.

[0089] The role of the discriminator is to input a real accident data and the false sample data generated by the generator, and the output is the probability that the sample is real accident data.

[0090] The method for updating the discriminator is: calculating the discriminator loss according to the discriminator loss function, performing back propagation and parameter update; the discriminator loss function is as follows:

[0091] V(D)=logD(x)+log(1-D(G(z,x cond )))

[0092] Where x represents the real data sample vector obtained from the feature factor matrix X, logD(x) represents the discrimination result of the real data input into the discriminator; the goal of the discriminator is to minimize the discriminator loss to reduce the discriminator's misjudgment rate of real accident data and the misjudgment rate of false sample data.

[0093] The objective function of the generative adversarial network model CTGAN is:

[0094]

[0095] Among them, V(D,G) represents the objective function, which represents the sum of the losses of the generator G and the discriminator D; Represents the distribution of real data p data The expected value of the real data sample vector x sampled in (x), the characteristic factor matrix X is expressed as the real data distribution p data (x); E z～p(z) represents the expected value of the random noise vector z sampled from the noise distribution p(z)); wherein the noise distribution p(z) is randomly generated, and its distribution type is preset. In this embodiment, the noise distribution is represented by a Gaussian distribution, and its probability density function is shown in the following formula (5):

[0096]

[0097] Where z represents a random noise vector; μ is the mean; σ is the standard deviation. In the generative adversarial network (CTGAN), a specific mean and standard deviation are usually selected to initialize the noise distribution, such as μ = 0, σ = 1, so that the noise will fluctuate around the value of 0 and have unit variance. During the training process, the generator eventually generates data that the discriminator cannot judge, which can be used as the output result, that is, the new accident data: x fake =G(z,x cond ).

[0098] S3: Use wavelet denoising technology to denoise new accident data.

[0099] The method of using wavelet denoising technology to perform denoising on the new accident data in step S2 is: using wavelet denoising technology to decompose the new accident data into wavelet coefficients of different frequency bands, where the wavelet coefficients are the results obtained after wavelet transformation, representing the decomposition information of the original signal at different scales and frequencies. The high-frequency noise is eliminated or reduced by setting a threshold, thereby achieving the purpose of eliminating the redundant noise in the new accident data.

[0100] The process of wavelet denoising is as follows: decomposing the noisy signal in the expanded new accident data into wavelet coefficients of different frequencies, and by setting an appropriate threshold, setting the noise coefficient to zero or reducing its value to achieve denoising.

[0101] The steps are as follows: New accident data x fake The noisy signal is expressed as:

[0102] s(t)=y(t)+n(t)

[0103] Among them, y(t) is the new accident data x generated by the CTGAN model fake The corresponding original signal; n(t) is the high-frequency noise signal, which refers to the noise component with high-frequency characteristics superimposed on the original signal, usually manifested as fluctuations or jitters that change rapidly in a short period of time; s(t) is the noisy signal; the noisy signal s(t) is decomposed into a series of wavelet coefficients d j, l, the decomposition process is obtained by the following formula:

[0104]

[0105] Where j represents the number of decomposition levels or scales; l represents the time or location index at that scale; Ψ j, l(t) represents the scaled and translated version of the continuous wavelet function at a specific scale j and position l. The above formula represents the wavelet coefficient d of each layer. j,l is the signal s(t) combined with the scaled and translated wavelet function Ψ j, l(t) is obtained by convolution or inner product.

[0106] By setting a threshold, the wavelet coefficients are compared with the threshold, and the wavelet coefficients are retained, set to zero, or adjusted according to the comparison results. The threshold processing process is as follows:

[0107]

[0108] When the wavelet coefficients after the threshold is exceeded When the absolute value of is less than the threshold λ, the wavelet coefficient is set to zero; but when the wavelet coefficient after the threshold is exceeded When the absolute value of is greater than or equal to the threshold, it is shrunk toward zero by a threshold amount; where sgn(d j, l) is the wavelet coefficient d j, The sign function of l.

[0109] Finally, the processed wavelet coefficients Perform wavelet reconstruction to restore the denoised signal The process of wavelet reconstruction is as follows:

[0110]

[0111] S4: The new accident data after noise reduction is input into the Transformer time prediction module for calculation to accurately predict the accident duration.

[0112] like Figure 3 As shown, the output signal after the above inverse discrete wavelet transform is As the input sequence is input into the encoder of the Transformer time prediction model, the input sequence is first Each element in is converted into a vector of fixed dimension through the embedding layer. This vector is called the embedding vector and is represented by e i Indicates that i∈1~n, n is the number of elements; at the same time, each element needs to be positionally encoded to obtain the position information of the element in the sequence. The position encoding formula is as follows:

[0113]

[0114] Among them, PE (pos,2i) and PE (pos,2i+1) They represent the position encoding values ​​at dimensions 2i and 2i+1 respectively; pos represents the position index of the data in the sequence; i represents the index of each element in the vector; d represents the dimension of the position encoding vector, which is also the dimension of the model encoder input vector.

[0115] The embedding vector of each element is added to the position encoding vector to obtain the final representation vector of the element; for all elements in the input sequence, the above process is repeated, that is, the embedding vector of each element is added to the position encoding vector to obtain the final representation vector of each element. These final representation vectors are arranged in sequence order into a matrix, namely the encoder embedding vector matrix E.

[0116] In the first layer of the decoder, the encoder embedding vector matrix E is transformed by three different linear transformations to obtain the query Q, key value K and value V corresponding to the encoder embedding vector matrix E. The formula of linear transformation is as follows:

[0117] Q=E*w Q

[0118] K=E*w K

[0119] V=E*w V

[0120] Among them, w Q , w K , w V They represent the weight matrices of query and key value respectively. In actual calculation, since the three weights corresponding to each head in the multi-head attention calculation are not the same, the query, key value and value corresponding to each head are also different. This is just a global representation.

[0121] Next, the query Q, key value K and value V are input into the Transformer encoder, which is composed of multiple encoder layers stacked together. Each encoder layer contains two main sublayers: multi-head self-attention mechanism (MultiHead) and fully connected feed-forward layer (Feed Forward). Each sublayer is followed by residual connection (Add) and layer normalization (Norm). The calculation formula of the multi-head attention mechanism is as follows:

[0122] MultiHeadAttention(E)=MultiHead(Q,K,V)=Concat(head1,…,head h ,…,head H )W O

[0123] head h =Attention(Q (h) ,K (h) ,V (h) ), for h=1,…,H.

[0124]

[0125] Among them, WO Represents a linear transformation matrix, which is used to map the concatenated output of multiple heads back to the desired dimension, Q (h) ,K (h) ,V (h) represents the query, key value, and value corresponding to each head obtained after linear transformation; H represents the number of heads, head h represents the hth head; Concat is used to concatenate different heads calculated by multiple head attention mechanisms to improve the representation ability of the model; d represents the dimension of the input feature sequence, which is used to scale the dot product attention to avoid the gradient vanishing or exploding problem; softmax is an activation function used to normalize the attention weights and convert them into probability distributions so that the sum of each weight is one, thereby more intuitively expressing the importance of each position; Attention(Q,K,V) means that the similarity between the calculated value Q and the key K is converted into a weight head through the softmax function h , reflecting the importance of different positions to the current position, so as to better capture the dependencies in long sequences; this set of weights is then used to perform a weighted sum operation on the value V, and finally a weighted representation for the query Q is obtained.

[0126] After being encoded by the multi-head attention mechanism, it is input into the Add&Norm layer, which consists of two parts: the residual connection layer (Add) and the normalization layer (Norm). The calculation formulas are as follows:

[0127] Z=LayerNorm(E+MultiHeadAttention(E))

[0128] Among them, E represents the embedding vector matrix of the multi-head attention mechanism input; LayerNorm represents normalization; the Add&Norm layer is used to add the input and output of multiple sub-layers and normalize the result of the addition, which retains the original characteristics of the data while accelerating the convergence process of the model.

[0129] The output feature sequence Z of the Add&Norm layer is used as the input of the Feed Forward fully connected layer (FFN). The fully connected feedforward layer (Feed Forward) is used to further process the output of the multi-head attention mechanism; the fully connected feedforward layer consists of three layers, the first and third layers are linear layers, and the second layer is the activation function ReLu, as shown in the following formula:

[0130] FFN(Z)=max(0,ZW1+b1)W2+b2

[0131] Among them, W1 and W2 are the weights of the two layers respectively; b1 and b2 are the bias items of the two layers respectively; max is the ReLu activation function, which is used to nonlinearly transform the high-dimensional space vector output by the linear layer of the first layer, so that the linear layer of the third layer can map the output after this nonlinear transformation back to the original dimension; the output result of the fully connected layer is the feature vector after nonlinear transformation. Then, the output result FFN(Z) of the fully connected layer is residually connected and normalized to obtain the output feature sequence of the first encoder layer:

[0132] Y0=LayerNorm(Z+FFN(Z))

[0133] After that, the input feature sequence of each encoder layer is the output feature sequence of the previous encoder layer. The input feature sequence of each encoder layer is linearly transformed to generate the corresponding query, key value and value. The query, key value and value are calculated by multi-head attention and then sequentially subjected to Add&Norm operation, Feed Forward operation and Add&Norm operation to obtain the output feature sequence of each encoder layer. After multiple layers of encoder layer calculation, the last encoder layer obtains the output feature sequence Y.

[0134] like Figure 3 As shown, in the decoder, there are multiple decoder layers. In the first decoder layer, in order for the model to understand the order of each element in the input sequence, the output feature sequence predicted by the decoder at the previous time step is embedded and positionally encoded to obtain the decoder input embedding vector Z t-1 ; At this time, the input embedding vector Z of the decoder t-1 After linear transformation, we get the new query Q'1, key value K'1, value V'1. The linear transformation formula is as follows:

[0135] Q'1=Z t-1 *w Q

[0136] K'1=Z t-1 *w Q

[0137] V'1=Z t-1 *w Q

[0138] Among them, w Q , w K , w V Represent the weight matrices of query, key value, and value respectively.

[0139] The mask part of the masked multi-head attention layer introduces the mask matrix M into the masked multi-head self-attention mechanism in the decoder. The formula is as follows:

[0140] maskedMultiHeadAttention(Z t-1 )=Concat(Head1,…,Head h …,Head H )W O

[0141]

[0142] Among them, the mask matrix M and Q'1K'1 T The dimensions are the same. Q'1', K'1', and V'1' represent the query, key, and value corresponding to each head obtained after linear transformation. Its elements are set to 0 or negative infinity according to the needs of the mask. When an element in the mask matrix M is negative infinity, after applying the softmax function, the attention weight corresponding to this position will be close to zero, thereby achieving a masking effect.

[0143] The input feature sequence Z t-1 Add it to the output after the masked multi-head attention transformation and normalize it as the output of the Add&Norm layer. The formula is as follows:

[0144] Z'=LayerNorm(Z t-1 +maskedMultiHeadAttention(Z t-1 ))

[0145] At this time, the sequence feature Z' is the output of the attention mechanism after adding the mask matrix;

[0146] Next, the sequence feature Z' output by the masked multi-head attention layer is linearly transformed to obtain the query Q2 and the key value K1 obtained by linearly transforming the output feature sequence Y of the last encoder layer, and the value V1 is input into the multi-head attention layer of the decoder layer, so that the decoder can pay attention to all positions of the input sequence when generating each output.

[0147] The multi-head attention layer processing process of the decoder layer includes splitting multiple heads, scaling dot product attention and merging the output of the head. The formula of the specific process is as follows:

[0148] MultiHeadAttention(Z')=Concat(head1,…,head H )W O

[0149]

[0150] Q'2, K'1, V1' represent the query, key value and value corresponding to each head obtained after linear transformation. The multi-head attention mechanism of the decoder layer in the above formula is consistent with the working principle of the multi-head attention mechanism in the encoder layer.

[0151] Then, the output feature sequence after the multi-head attention layer transformation of the decoder layer is input into the Add&Norm layer, and the formula is as follows:

[0153] Z”=LayerNorm(Z’+MultiHeadAttention(Z’))

[0154] The feature sequence after the residual connection and normalization of the Add&Norm layer is used as the input of the fully connected layer for nonlinear transformation. The transformation process is shown in the following formula:

[0155] FFN(Z”)=max(0,Z”W'1+b'1)W'2+b'2

[0156] Among them, W'1 and W'2 are the weights of the two layers respectively; b'1 and b'2 are the biases of the two layers respectively; the ReLu activation function is max, and its function is the same as above.

[0157] After the nonlinear transformation of the fully connected layer, it is input into Add&Norm for processing. The formula is as follows:

[0158] Z”'=LayerNorm(Z”+FFN(Z”))

[0159] After that, the input feature sequence of each decoder layer is the output feature sequence of the previous decoder layer. The input feature sequence of each decoder layer is linearly transformed to obtain the corresponding query, key value and value, and then the masked multi-head attention calculation and Add&Norm operation are performed to obtain the masked multi-head attention output feature sequence, and then the linear transformation is performed to obtain the query corresponding to the masked multi-head attention output feature sequence of each decoder layer. The output feature sequence of the last encoder layer is linearly changed to obtain the key value K1 and value V1. The key value K1, value V1 and the query corresponding to the masked multi-head attention output feature sequence Y of each decoder layer are calculated by multi-head attention and then successively subjected to Add&Norm operation, Feed Forward operation and Add&Norm operation to obtain the output feature sequence of each decoder layer. After calculation of multiple decoder layers, the last decoder layer obtains the output feature sequence Z of the decoder at the current time step. t

[0160] Then the output is sent to the last layer of the decoder, which consists of a linear transformation layer and a softmax layer, which together convert the output of the decoder into the probability distribution of the predicted data. The formula of the linear transformation layer is as follows:

[0161] p=W3Z t +b3

[0162] Among them, p is the output log probability vector; W3 is the weight matrix; b3 is the bias term.

[0163] Finally, the output logarithmic probability vector is input into the softmax layer to achieve the predicted probability of accident duration. Its mathematical expression is as follows:

[0164]

[0165] Among them, p i represents the value of the i-th element in the vector p; used for normalization; ks represents the size of the corresponding output vocabulary; σ(p i ) represents the normalized result, that is, the predicted probability of accident duration data.

[0166] Evaluation indicators are introduced to evaluate and analyze the prediction results, and their calculation formulas are as follows:

[0167] Mean Absolute Error (MAE):

[0168]

[0169] Where n is the number of predicted samples; y i is the actual value of the accident duration; is the predicted value of accident duration, that is, the above σ(p i ); The range of MAE is [0, +∞). When the predicted value is exactly the same as the true value, it is equal to 0, that is, a perfect model; the larger the error, the larger the value.

[0170] Root Mean Square Error (RMSE):

[0171]

[0172] Where n is the number of prediction samples; y i is the actual value of the accident duration; is the predicted value of accident duration; the value range of RMSE is between [0, +∞), 0 means a completely accurate prediction, and the larger the RMSE, the less accurate the prediction.

[0173] Mean Absolute Percent Error (MAPE):

[0174]

[0175] Where n is the number of predicted samples; y i is the actual value of the duration; is the duration prediction value; the MAPE value range is between [0, +∞), 0 represents a completely accurate prediction, and the larger the MAPE, the less accurate the prediction.

[0176] The present invention is applied to the accident data set to predict the accident duration, and the evaluation index comparison results are shown in Table 2 below.

[0177] Table 2 Comparison of evaluation indicators of prediction results

[0178]

[0179] From the results in Table 2, it can be seen that with the increase in the number of training times, the error evaluation indicators gradually decrease, the mean absolute error MAE value is controlled below 0.54, the root mean square error RMSE value is controlled below 0.72, and the mean absolute percentage error MAPE value is controlled below 0.76, indicating that the error index of the present invention can be controlled below 1.0, which is an effective method for predicting the duration of road traffic accidents.

[0180] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A road traffic accident duration prediction method based on CTGAN-Transformer, characterized in that: Includes steps: S0: Obtain a historical road traffic accident data set and construct a road traffic accident duration prediction model, wherein the road traffic accident duration prediction model includes a feature factor extraction module, a generative adversarial network CTGAN, a wavelet denoising module and a Transformer time prediction module connected in sequence; S1: Extract the characteristic factor matrix related to accident duration from the historical road traffic accident dataset using the characteristic factor extraction module; S2: Input the feature factor matrix into the generative adversarial network (CTGAN) for data enhancement to obtain new accident data; S3: Use the wavelet denoising module to perform denoising on the new accident data; S4: The new accident data after noise reduction is input into the Transformer time prediction module for calculation to accurately predict the accident duration.

2. The road traffic accident duration prediction method based on CTGAN-Transformer according to claim 1 is characterized in that: The generative adversarial network CTGAN described in step S0 includes a conditional classifier, a generator and a discriminator connected in sequence; the Transformer time prediction module described in step S0 is composed of an encoder and a decoder connected in sequence, the encoder includes multiple encoder layers, the decoder includes multiple decoder layers, each encoder layer is composed of a multi-head attention layer and a fully connected feedforward layer, and each decoder layer is composed of a masked multi-head attention layer, a multi-head attention layer and a fully connected feedforward layer connected in sequence.

3. The road traffic accident duration prediction method based on CTGAN-Transformer according to claim 2 is characterized in that: The method for extracting the characteristic factor matrix related to the accident duration in the historical road traffic accident data set described in step S1 is: X=[x ij ] Where X represents the characteristic factor matrix extracted from historical road traffic accident data; x ij represents the jth characteristic factor of the ith accident; i∈0~M, M represents the total number of accidents; j∈0~N, N represents the total number of characteristic factors of each accident; the method of inputting the characteristic factor matrix into the generative adversarial network CTGAN for data enhancement as described in step S2 is: sampling a random noise vector from the noise distribution in the conditional classifier, obtaining a conditional vector according to the characteristic factor matrix, obtaining the random noise vector and the conditional vector from the conditional classifier, inputting them into the generator and outputting generated data simulating real data, then inputting the generated data and the characteristic factor matrix representing the real data into the discriminator for discrimination, the discriminator discriminates the generated data and the characteristic factor matrix representing the real data and updates the discriminator, and then feeds back the discrimination result of the generated data to the generator to update the generator, and according to the objective function of the generative adversarial network CTGAN, minimizes the discriminator loss and maximizes the generator loss, and finally generates new accident data x that the discriminator cannot judge fake .

4. The road traffic accident duration prediction method based on CTGAN-Transformer according to claim 3 is characterized in that: The probability density function of the noise distribution is: Where z represents the random noise vector; μ is the mean; σ is the standard deviation; The method to obtain the conditional vector based on the characteristic factor matrix is: Among them, K means that there are K discrete features in the M*N feature factors in the feature factor matrix, and C k Indicates that each discrete feature has C k categories; The method for updating the discriminator is: calculating the discriminator loss according to the discriminator loss function, performing back propagation and parameter update; the discriminator loss function is: V(D)=logD(x)+log(1-D(G(z,x cond ))) Among them, x represents the real data sample vector obtained from the feature factor matrix X, D() represents the discriminant function of the discriminator, G() represents the generator generation function; The method for updating the generator is: calculating the generator loss according to the generator loss function, performing back propagation and parameter update: Generator loss function V(G)=log(1-D(G(z,x cond ))) The objective function of the generative adversarial network CTGAN is: in, represents the expected value of the real data sample vector x obtained from the feature factor matrix X, E z～p(z) Represents the expected value of a random noise vector z sampled from the noise distribution p(z).

5. The road traffic accident duration prediction method based on CTGAN-Transformer according to claim 3 or 4, It is characterized in that The method of using the wavelet noise reduction module to perform noise reduction processing on the new accident data in step S3 is: fake The noisy signal is decomposed into wavelet coefficients of different frequencies through discrete wavelet transform. By setting a threshold, the wavelet coefficient is compared with the set threshold. According to the comparison result, the wavelet coefficient is set to zero or shrunk to zero by a threshold amount. Then, the wavelet coefficient processed by the threshold is reconstructed into the reply signal through inverse discrete wavelet transform.

6. The road traffic accident duration prediction method based on CTGAN-Transformer according to claim 4 is characterized in that: The new accident data x fake The noisy signal is expressed as: s(t)=y(t)+n(t) Among them, y(t) is the new accident data x generated by the CTGAN model fake The corresponding original signal; n(t) is the high-frequency noise signal; The method of decomposing into wavelet coefficients of different frequencies by discrete wavelet transform is: Among them, j represents the scale of decomposition, l represents the position index at this scale, and Ψ j, l(t) represents the scaled and translated version of the continuous wavelet function at a specific scale j and position l; secondly, the wavelet coefficient is compared with the threshold by setting a threshold, and the wavelet coefficient is set to zero or adjusted according to the comparison result; the threshold processing process is as follows: When the wavelet coefficients after threshold processing When the absolute value of is less than the threshold λ, the wavelet coefficient is set to zero; however, when the wavelet coefficient after threshold processing When the absolute value of is greater than or equal to the threshold, it is shrunk toward zero by a threshold amount; where sgn(d j, l) is the wavelet coefficient The symbolic function of Finally, the processed wavelet coefficients Perform wavelet reconstruction to restore the denoised signal The process of wavelet reconstruction is as follows:

7. The road traffic accident duration prediction method based on CTGAN-Transformer according to claim 6 is characterized in that: The method for inputting the Transformer time prediction module for calculation described in step S4 is: the output signal after the inverse discrete wavelet transform is used as the Transformer time prediction module to undergo embedding operation and position encoding in sequence to obtain the encoder embedding vector E, the encoder embedding vector E is then subjected to encoder calculation and then enters the decoder for decoder calculation, and the output result of the decoder is passed through the linear transformation layer and the softmax layer to obtain the predicted probability of the accident duration data.

8. The road traffic accident duration prediction method based on CTGAN-Transformer according to claim 7 is characterized in that: The method for performing encoder calculation is as follows: the input feature sequence of the first encoder layer is the encoder embedding vector E, and the input feature sequence of each subsequent encoder layer is the output feature sequence of the previous encoder layer. The input feature sequence of each encoder layer is linearly transformed to generate corresponding queries, key values, and values. The queries, key values, and values ​​are calculated by multi-head attention and then sequentially subjected to Add&Norm operations, Feed Forward operations, and Add&Norm operations to obtain the output feature sequence of each encoder layer. After multiple encoder layer calculations, the last encoder layer obtains the output feature sequence Y.

9. The method for predicting the duration of a road traffic accident based on CTGAN-Transformer according to claim 8, characterized in that: The decoder calculation method is as follows: the output feature sequence of the decoder in the previous time step is subjected to embedding operation and position encoding to obtain the input embedding vector of the decoder, the input feature sequence of the first decoder layer is the input embedding vector of the decoder, and then the input feature sequence of each decoder layer is the output feature sequence of the previous decoder layer, and the input feature sequence of each decoder layer is subjected to linear transformation to obtain the corresponding query, key value and value, and then the masked multi-head attention calculation and Add&Norm operation are performed to obtain the masked multi-head attention output feature sequence, and then the query Q corresponding to the masked multi-head attention output feature sequence of each decoder layer is obtained through linear transformation. m , the output feature sequence Y of the last encoder layer undergoes linear transformation to obtain the key value K1 and value V1, the key value K1, value V1 and the query Q corresponding to the output feature sequence of each decoder layer mask multi-head attention m After multi-head attention calculation, the output feature sequence of each decoder layer is obtained by Add&Norm operation, FeedForward operation and Add&Norm operation in sequence. After the decoder calculation, the last decoder layer obtains the output feature sequence Z of the decoder at the current time step t , the output feature sequence of each time step passes through the linear transformation layer and the softmax layer in turn to obtain the predicted probability of the accident duration data.

10. The road traffic accident duration prediction method based on CTGAN-Transformer according to claim 9 is characterized in that: The multi-head attention calculation process is: MultiHead(Q b ,K b ,K b )=Concat(head1,…,head h ,…,head H )W O head h =Attention(Q (h) ,K (h) ,V (h) ),for h=1,…,H Among them, Q b ,K b ,K b Represents the query, key and value input to each multi-head attention layer, Q (h) ,K (h) ,V (h) The query, key, and value corresponding to each head are obtained after linear transformation. d represents the dimension of the input vector of each multi-head attention layer, H represents the number of heads, and head h represents the hth head, and softmax is the activation function; The calculation process of the masked multi-head attention layer is: maskedMultiHeadAttention(Q m ,K1,V1,M)=Concat(Head1,…,Head h …,Head H )W O Among them, Q m ,K1,V1 represent the query, key and value input to each masked multi-head attention layer, Q′ m ,K1′,V1′ represents the query,key and value corresponding to each head obtained after linear transformation, d k Represents the dimension of the input vector of the masked multi-head attention layer; The calculation process of the Feed Forward operation is: FFN(XIN)=max(0,XINW a +b a )W b +b b Among them, XIN represents the input feature of the Feed Forward operation, W a , W b is the weight, b a , b b is the bias, max is the ReLu activation function; The output feature sequence of each time step passes through the linear transformation layer and the softmax layer in turn to obtain the calculation process of predicting the accident duration: p=W c Z t +b d Among them, W c is the weight, b d is the bias, p is the log probability vector of the output, p i represents the value of the i-th element in the log-probability vector p, ks represents the size of the output vocabulary, and σ() represents the normalization operation.

Images