Traffic flow prediction method based on combination of channel pruning and network quantification

Abstract

The invention discloses a traffic flow prediction method based on combination of channel pruning and network quantification. The method comprises the following steps: collecting data; preprocessing the data; constructing a traffic flow prediction model; optimizing the model; channel pruning and network quantification are combined to form an efficient deep learning traffic flow prediction model, the possibility of overfitting is reduced by simplifying the model, reducing the depth of the network or the number of neurons in each layer and reducing the degree of freedom of the model, and the complexity is reduced by simplifying the model on the premise of ensuring the high precision of the model. The generalization ability of the model is improved; the traffic flow can be accurately predicted, and effective support is provided for traffic management and planning.

Description

Technical Field

[0001] The present invention relates to a traffic flow prediction method, and particularly to a traffic flow prediction method based on the combination of channel pruning and network quantization. Background Art

[0002] A traffic flow prediction model is a mathematical model or algorithm used to predict traffic flow within a specific time period. By analyzing factors such as historical traffic data, weather conditions, holidays, and road conditions, it predicts future traffic flow changes for traffic management, planning, optimization, and early warning. Traffic flow prediction helps reduce traffic congestion, improve road capacity, optimize traffic signals, and reasonably arrange trips. The main problem that a traffic flow prediction model may face during deep learning modeling is overfitting. How to avoid overfitting in the deep learning modeling of a traffic flow prediction model and ensure its good generalization ability.

[0003] A traffic flow prediction model establishment method, device, and equipment disclosed in Application No. 202211175626.7, which relates to the field of artificial intelligence technology. The traffic flow prediction model establishment method includes: obtaining traffic flow in different time periods and first data, where the first data is data related to vehicle operation; obtaining traffic flow in a second time period associated with the traffic flow in the first time period according to the traffic flow in different time periods; obtaining target data associated with the traffic flow according to the traffic flow and the first data; and obtaining a traffic flow prediction model according to the traffic flow in the first time period, the traffic flow in the second time period, the target data, and the ASTGCN model. In the above technical solution, during deep learning modeling, if the training data is insufficient or the model is too complex, overfitting may occur, that is, the model performs well on the training set but has poor generalization ability in actual applications. Summary of the Invention

[0004] Object of the Invention: The object of the present invention is to provide a traffic flow prediction method based on the combination of channel pruning and network quantization that reduces overfitting and improves generalization ability.

[0005] Technical Solution: A traffic flow prediction method based on the combination of channel pruning and network quantization according to the present invention includes the following steps:

[0006] (1) Collect data;

[0007] (2) Data preprocessing: Fill in missing values in the traffic flow data with the average value of adjacent time point data, correct data outliers, and number the data information.

[0008] (3) Construct a traffic flow prediction model: Use the preprocessed data as input, and combine a convolutional neural network and a recurrent neural network to extract spatio-temporal features, and perform channel pruning and network quantization operations on the model;

[0009] (4) Model optimization: Fine-tune the pruned model, perform quantization training synchronously, and then perform layer-by-layer quantization and pruning after training to enable accurate prediction of traffic flow.

[0010] In step (2), the average value of adjacent time point data is used to fill the missing values in the traffic flow data, the abnormal data deviating from the normal range is corrected, and the weather data, holiday information, and road construction information affecting the traffic flow are numbered.

[0011] Step (3) specifically includes the following steps:

[0012] (31) Input the preprocessed data, and the input data dimension is [number of samples, time series length, feature dimension];

[0013] (32) Use a convolutional neural network and a recurrent neural network to extract spatio-temporal features. Among them, the convolutional neural network is used to extract spatial features, and the recurrent neural network is used to capture the dependencies of time series;

[0014] (33) Perform channel pruning: Measure the weight size of the channel by calculating the L1 norm and the L2 norm, and then determine the importance of pruning by calculating the gradient information;

[0015] (34) Perform network quantization: Convert floating-point values to low-precision integers, including weight quantization and activation value quantization;

[0016] The weight quantization formula is:

[0017]

[0018] Where: W is the floating-point weight, min(W) is the minimum value of the weight, S is the quantization step size, calculated as Where n is the number of bits after quantization, and round() represents rounding to the nearest integer;

[0019] The activation value quantization formula is:

[0020]

[0021] Where, A is the activation value, min(A) is the minimum value of the activation value, S A is the quantization step size of activation;

[0022] In step (4), the pruned model is fine-tuned, and the fine-tuning formula is:

[0023]

[0024] Among them, W new is the fine-tuned weight, W pruned is the pruned weight, and α is the learning rate.

[0025] During the fine-tuning process, calculate the gradient of the loss function with respect to the pruned weight, and update the weight according to the gradient direction and the learning rate.

[0026] The quantization training in step (4) is quantization-aware training, and the formula is:

[0027]

[0028] Among them, W ft is the fine-tuned weight; in quantization-aware training, quantization error is introduced, and the formula for the quantization-aware training error is:

[0029] Lquantized = Loriginal + λ·||W quantized - W|| 2 ,

[0030] Among them, Loriginal is the original loss function, Lquantized is the loss function after adding quantization error, λ is the regularization coefficient, and ||W quantized - W|| 2 is the sum of the squares of the quantization errors.

[0031] The layer-by-layer quantization and pruning in step (4) specifically prunes each layer of channels according to the channel pruning method, and then quantizes the weights and activation values of each layer to determine appropriate quantization parameters.

[0032] After the layer-by-layer quantization and pruning in step (4), final optimization is performed. By evaluating the performance of the model on the validation set, the parameters of the model are adjusted according to the performance metrics. If the performance of the model on the validation set decreases, the learning rate is reduced or the regularization coefficient is increased, and the model is continuously trained until the performance of the model on the validation set is stable and meets the requirements; finally, the optimized model is used to predict the test set, and the performance of the model on the test set is evaluated.

[0033] A computer device includes one or more processors, a memory, and one or more programs, where the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the program is executed by the processor, it implements the steps of the traffic flow prediction method based on the combination of channel pruning and network quantization.

[0034] A computer-readable storage medium stores a computer program thereon. When the computer program is executed by a processor, the steps of the traffic flow prediction method based on the combination of channel pruning and network quantization are implemented.

[0035] Beneficial effects: Compared with the prior art, the present invention has the following remarkable advantages: (1) Channel pruning reduces the computational amount and storage requirements in the network by removing redundant channels. In a deep neural network, some channels may contribute less to the final prediction of the model. Therefore, pruning these unimportant channels can reduce the consumption of computing resources without significantly sacrificing accuracy. Network quantization further reduces the storage occupancy and memory bandwidth requirements, and reduces the complexity of hardware computing by mapping the floating-point weights and activation values of the model to low-precision integers. This combination not only improves the inference speed of the model but also meets the requirements of low-power devices and edge computing. (2) The pruning technique reduces the number of parameters in the network, enabling the network to run in a smaller storage space. Quantization reduces the bit width of the data, lowering the requirements for hardware computing and storage capabilities. It can reduce the hardware cost and improve the processing ability of the device, enabling the model to be deployed on more types of hardware platforms, thus accelerating its application in the industrial and consumer electronics fields. (3) The combination of channel pruning and network quantization can simplify the model while maintaining high accuracy. Pruning effectively reduces the complexity of the model by removing unnecessary channels or parameters, and quantization further reduces the redundant information in model calculations. Although these optimizations will cause a slight decrease in accuracy, through a carefully designed fine-tuning strategy, part of the accuracy can be restored after pruning and quantization to ensure that the performance of the model after simplification is not overly damaged. Therefore, this technical solution can reduce the consumption of computing resources without significantly sacrificing accuracy, providing a balanced solution for tasks with high requirements for efficiency and accuracy in practical applications. Description of the Drawings

[0036] Figure 1 It is a flowchart of channel pruning and network quantization for the model of the present invention. Detailed Embodiments

[0037] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0038] A traffic flow prediction method based on the combination of channel pruning and network quantization includes the following steps:

[0039] (1) Collect data;

[0040] (2) Data preprocessing: Fill in the missing values in the traffic flow data with the average values of adjacent time points, correct the data outliers, and number the data information.

[0041] (3) Build a traffic flow prediction model: Use the preprocessed data as input, and combine a convolutional neural network and a recurrent neural network to extract spatio-temporal features, and perform channel pruning and network quantization operations on the model.

[0042] (4) Model optimization: Fine-tune the pruned model, perform quantization training synchronously, and then perform layer-by-layer quantization and pruning after training to enable accurate prediction of traffic flow.

[0043] The following specifically explains each step of the traffic flow prediction method based on the combination of channel pruning and network quantization.

[0044] (1) Collect data;

[0045] Obtain the traffic flow data of the urban area in the past year from the traffic management department. The time interval of the data record is 15 minutes, including traffic volume, vehicle speed, and corresponding timestamps. At the same time, collect the weather data, holiday information, and road construction conditions in the same time period of this area.

[0046] (2) Data preprocessing;

[0047] Fill in the missing values in the traffic flow data with the average value of adjacent time point data. For the missing traffic volume data within a 15-minute interval, calculate the average value of the traffic volume at the two adjacent time points as the filling value.

[0048] Correct the outliers, and reasonably correct the data that significantly deviates from the normal range according to the distribution of historical data.

[0049] Convert the timestamp information into time features that can be used as model input, and encode the weather data, holiday information, and road construction conditions. Sunny days are encoded as 0, rainy days are encoded as 1; holidays are classified into different categories such as weekdays, weekend holidays, and statutory holidays for encoding; road construction conditions are encoded as under construction and not under construction.

[0050] (3) Build a traffic flow prediction model;

[0051] Design a traffic flow prediction model that combines deep learning and spatio-temporal feature analysis.

[0052] Use the preprocessed traffic flow data, time features, weather encoding, holiday encoding, road construction encoding and other multi-source data as input. The dimension of the input data is [number of samples, time series length, feature dimension].

[0053] The spatio-temporal features are extracted by combining a Convolutional Neural Network (CNN) and a Recurrent Neural Network (RNN). The CNN is used to extract spatial features, and a convolutional layer with a kernel size of 3*3 is used to model the relationship between different features at the same time point; the RNN is used to capture the dependencies in the time series, and the number of hidden units of the LSTM is set to 64.

[0054] After the spatio-temporal feature extraction layer, a channel pruning operation is performed.

[0055] Pruning based on the L1 norm: Calculate the L1 norm of each channel weight, and the formula is

[0056]

[0057] where W is the convolutional kernel weight matrix, c is the channel index, h is the height of the convolutional kernel, w is the width, and d is the depth. After calculating the L1 norms of all channels, sort them in ascending order of the norm, and select the 30% channels with smaller norms for pruning.

[0058] Pruning based on the L2 norm: The L2 norm calculation formula is

[0059]

[0060] Similarly, sort the channels according to the L2 norm, and select the 25% channels with smaller L2 norms for pruning.

[0061] Pruning based on gradient information: According to the formula

[0062]

[0063] where L is the loss function, W i,c is the weight of the i-th sample on channel c, and n is the number of samples. Calculate the gradient information of each channel. The smaller the gradient of a channel, the smaller its impact on the loss function. Select the 20% channels with the smallest gradients for pruning. During the training process, calculate the gradient of each channel through backpropagation.

[0064] Quantize the pruned model, converting floating-point values to low-precision integers.

[0065] Use the linear quantization formula:

[0066]

[0067] where W is the floating-point weight, W min is the minimum value of the weight, s is the quantization step, and b is the number of bits after quantization.

[0068] The activation quantization formula is

[0069]

[0070] Where A is the activation value, and A min is the minimum value of the activation value, and s a is the step size of activation quantization. In actual calculations, A min and s a are determined according to the distribution of activation values. At the same time, in quantization-aware training, quantization error is introduced, and the quantization-aware training error calculation formula is:

[0071]

[0072] Where L org is the original loss function, λ is the regularization coefficient, m is the number of weights, and n is the number of activation values. By adjusting the value of λ, the influence of the original loss and quantization error is balanced.

[0073] (4) Model optimization;

[0074] After pruning, fine-tuning is performed, and the fine-tuning optimization formula is:

[0075]

[0076] Where W new is the fine-tuned weight, W pruned is the pruned weight, and α is the learning rate. During the fine-tuning process, the gradient of the loss function with respect to the pruned weight is calculated, and the weight is updated according to the gradient direction and learning rate.

[0077] After quantization, quantization-aware training (QAT) is performed, and the formula is:

[0078]

[0079] Where W ft is the fine-tuned weight. The model is optimized by simulating low-precision calculations to make it adapt to low-precision inference. After QAT training, quantization is performed again to reduce the performance loss caused by quantization.

[0080] Then, layer-by-layer quantization and pruning are performed, and quantization and pruning are applied separately to each convolutional layer. First, the channels of each layer are pruned according to the above channel pruning method, and then the weights and activation values of each layer are quantized, and appropriate quantization parameters are determined according to the specific situation of each layer.

[0081] After completing the pruning and quantization operations, the model is fine-tuned and optimized again. By evaluating the performance of the model on the validation set, the parameters of the model are adjusted according to the performance metrics. If it is found that the performance of the model on the validation set decreases, the learning rate can be appropriately reduced or the regularization coefficient can be increased, and then the model is continued to be trained until the performance of the model on the validation set is stable and meets the requirements. Finally, the optimized model is used to make predictions on the test set, and the performance of the model on the test set is evaluated.

Claims

1. A traffic flow prediction method based on channel pruning and network quantization, characterized in that: The method comprises the following steps: (1) Collect data; (2) Data preprocessing: Use the average value of data from adjacent time points to fill in missing values ​​in traffic flow data, correct data outliers, and number the data information; (3) Constructing a traffic flow prediction model: Taking the preprocessed data as input, the convolutional neural network and the recurrent neural network are combined to extract spatiotemporal features, and channel pruning and network quantization operations are performed on the model; (4) Model optimization: Fine-tune the pruned model and perform quantization training simultaneously. After the training is completed, perform layer-by-layer quantization and pruning to accurately predict traffic flow.

2. According to claim 1, a traffic flow prediction method based on channel pruning and network quantization is characterized in that: The step (2) uses the average value of the data at adjacent time points to fill in the missing values ​​in the traffic flow data, corrects the abnormal data that deviates from the normal range, and numbers the weather data, holiday information and road construction information that affect the traffic flow.

3. The traffic flow prediction method based on channel pruning and network quantization according to claim 1 is characterized in that: The step (3) specifically comprises the following steps: (31) Input preprocessed data, the input data dimension is [sample number, time series length, feature dimension]; (32) Convolutional neural network and recurrent neural network are used to extract spatiotemporal features, where convolutional neural network is used to extract spatial features and recurrent neural network is used to capture the dependency of time series; (33) Perform channel pruning: Measure the weight of the channel by calculating the L1 norm and L2 norm, and then determine the importance of pruning by calculating the gradient information; (34) Perform network quantization: convert floating-point values ​​into low-precision integers, including weight quantization and activation value quantization; The weight quantization formula is: Where: W is a floating point weight, min(W) is the minimum value of the weight, S is the quantization step size, and is calculated as Where n is the number of digits after quantization, and round() means rounding to the nearest integer; The activation value quantization formula is: Among them, A is the activation value, min(A) is the minimum activation value, S A is the step size of activation quantization.

4. The traffic flow prediction method based on channel pruning and network quantization according to claim 1 is characterized in that: The step (4) fine-tunes the pruned model, and the fine-tuning formula is: Among them, W new is the weight after fine-tuning, W pruned is the weight after pruning, and α is the learning rate.

5. The traffic flow prediction method based on channel pruning and network quantization according to claim 4 is characterized in that: During the fine-tuning process, the gradient of the loss function to the pruned weights is calculated, and the weights are updated according to the gradient direction and the learning rate.

6. The traffic flow prediction method based on channel pruning and network quantization according to claim 1 is characterized in that: The quantization training in step (4) is quantization-aware training, and the formula is: Among them, W ft is the weight after fine-tuning; in quantization-aware training, quantization error is introduced, and the calculation formula of quantization-aware training error is: Lquantized=Loriginal+λ·||W quantized -W|| 2 , Among them, Loriginal is the original loss function, Lquantized is the loss function after adding quantization error, λ is the regularization coefficient, ||W quantized -W|| 2 is the sum of squares of quantization errors.

7. The traffic flow prediction method based on channel pruning and network quantization according to claim 1 is characterized in that: The step (4) of layer-by-layer quantization and pruning specifically involves pruning the channels of each layer according to the channel pruning method, and then quantizing the weights and activation values ​​of each layer to determine appropriate quantization parameters.

8. The traffic flow prediction method based on channel pruning and network quantization according to claim 1 is characterized in that: After layer-by-layer quantization and pruning, the step (4) performs a final optimization process by evaluating the performance of the model on the validation set and adjusting the parameters of the model according to the performance indicators. If the performance of the model on the validation set decreases, the learning rate is reduced or the regularization coefficient is increased, and the model is continued to be trained until the performance of the model on the validation set is stable and meets the requirements; finally, the optimized model is used to predict the test set and the performance of the model on the test set is evaluated.

9. A computer device, characterized in that: The method comprises one or more processors, a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of a traffic flow prediction method based on the combination of channel pruning and network quantization as described in any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a traffic flow prediction method based on the combination of channel pruning and network quantization as described in any one of claims 1 to 8 are implemented.

Images