Characteristic difference learning network traffic flow data prediction processing method and system

Abstract

The invention provides a characteristic difference learning network traffic flow data prediction processing method and system, relates to the field of data processing, and can solve the problem of low accuracy of traffic time sequence prediction. The method comprises the steps that data preprocessing is carried out on input original traffic time sequence data, a feature difference learning sample is obtained, and the original traffic time sequence data comprises traffic data of a plurality of target road nodes in a duration range; thirdly, constructing a deep learning model DXGAN by integrating a sample positioning encoder, a feature difference generator and a discriminator; and then, inputting the feature difference learning sample into a deep learning model DXGAN to obtain a prediction result. Spatial and temporal feature difference distinguishing is performed through trending feature difference learning, and the accuracy of traffic time sequence prediction is improved by using the high accuracy of the DXGAN model.

Description

Technical Field

[0001] The present invention relates to data processing, and in particular to a method and system for predicting and processing network traffic flow data through feature difference learning. Background Art

[0002] With the development of intelligent transportation systems, predictive techniques are being used for urban computing, planning, and planning of urban development including population migration.

[0003] Usually, people use statistical algorithms for the above planning, such as autoregressive integrated moving average (ARIMA) and support vector machine (SVM), which can consider the time correlation between road nodes. However, due to the nonlinear characteristics, such models are often limited by the model and cannot provide accurate predictions. With the development of deep learning, recursive neural network (RNN) models and convolutional neural network (CNN) models can learn rich spatiotemporal dependencies in data due to their rich nonlinear characteristics, gradually replacing the traditional models in prediction problems, and more and more excellent models have been proposed.

[0004] However, due to the existence of fine-grained data, the above models tend to learn consistent features in the historical window and fail to effectively distinguish the heterogeneity of the data, which may lead to large differences between the prediction results and the actual situation. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention provides a method and system for predicting and processing network traffic flow data by using feature difference learning.

[0006] The present invention is achieved through the following technical solutions: The present invention provides a method for predicting and processing network traffic flow data by feature difference learning, comprising: The input original traffic time series data is preprocessed to obtain feature difference learning samples, wherein the original traffic time series data includes traffic data of multiple target road nodes within a time range, and the feature difference learning samples at least include: By integrating the sample localization encoder, feature difference generator and discriminator, a deep learning model dynamic exchange generative adversarial network DXGAN is constructed; The feature difference learning sample is input into the deep learning model DXGAN to obtain a prediction result.

[0007] Furthermore, the data preprocessing of the input original traffic time series data to obtain feature difference learning samples includes: Record the change characteristics of the original traffic time series data in time and space dimensions through the spatial sample position difference observation matrix and the time sample information difference matrix; The spatial sample position difference observation matrix is: When detected by the model, , the value of the matrix remains , on the contrary, when the position information of the sample hour, ,in represents the spatial retrieval state matrix, where Represents the coordinate position of the sensor node in the spatial dimension.

[0008] The representation of the time sample information difference matrix is ​​more inclined to be in time The change of the spatial sample position difference observation matrix can be expressed as: ; In the formula, Indicates the data in the sensor at time The data at the moment, represents the time sample information difference matrix. It means that the model did not record the spatial and temporal changes of the original traffic time series data at the previous moment.

[0009] Furthermore, the data preprocessing is performed on the input original traffic time series data to obtain feature difference learning samples, including: The original traffic time series data are respectively , obtain data embedding of two different channel dimensions , It means that the original traffic time series data is mapped through the fully connected layer to obtain the embedded features in the spatial dimension. It means that the original traffic time series data is mapped through the fully connected layer to obtain the embedded features in the time dimension; According to the time sample information difference matrix , construct features , obtain the time sample information difference characteristics ; Embedding features in spatial and temporal dimensions And the original traffic time series data and trainable parameters Through the convolution operation , get the features on the convolution ,in, Represents the convolution operation function; according to And the location information features obtained using encoding technology Perform sum operation to obtain comprehensive information features Among them, the location information features obtained by using the encoding technology include: First location information features: ; Second location information features: ; Third location information features: ; Fourth location information feature: ; in, Indicates the position of the element index. represents the spatial distance parameter matrix, represents the transpose of the spatial distance parameter matrix; according to And the location information features obtained using encoding technology The sum of the two determines the comprehensive characteristics ; According to the location information characteristics of the target road node trend difference change and time-delay characteristics The sum of the two determines the comprehensive characteristics , Through normalization operation Determine the comprehensive information characteristics of the node , represents the normalization operation, Represents trainable weights.

[0010] Furthermore, the deep learning model DXGAN includes: a sample positioning encoder, a feature difference generator and a discriminator, wherein: The feature difference generator includes: a difference sample generation layer, a difference generator representation layer, and a difference generator output layer; The difference sample generation layer includes recording the feature differences at different times. The representation method is to convert the comprehensive information features of the nodes obtained above into Compare the differences and use It can be expressed as follows ,in Indicates at time The comprehensive feature representation of the input nodes.

[0011] The difference generator representation layer represents the difference information generated by the difference sample generation layer by receiving the difference information generated by the difference sample generation layer. , and the input at different times To learn the rich information representation in time series data, the representation layer consists of a GRU network and the representation is: , where the output state is , Denoted as the difference generator representation layer The hidden state of the layer is represented by Represent the update gate state and candidate hidden state respectively. The update gate state is represented as , Represented as the difference information feature generated by the difference sample generation layer, Represents the comprehensive information characteristics of the node, Denoted as the difference generator representation layer The hidden state of the layer is represented by, and the reset gate state is represented by , the candidate hidden state is represented as , concat represents the serial effect between states, , as well as , , Represents the network learnable parameters; The output layer of the difference generator is fitted with data through a fully connected layer. , and get the final output prediction sequence , where FC represents the fully connected layer, represents learnable parameters.

[0012] Furthermore, the feature difference discriminator is , determine the overall loss ,in, is the MAE loss, is the discriminator loss, Expressed as a hyperparameter; The discriminator loss is expressed as: , represents the discriminator symbol, represents the generator symbol, means that the expectation is calculated based on samples drawn from the distribution P, Representation sample Samples drawn from distribution P (representing the true data distribution); Furthermore, the step of inputting the feature difference learning sample into the deep learning model DXGAN to obtain a prediction result includes: The prediction results are evaluated by the mean absolute percentage error MAPE, root mean square error RMSE and mean absolute error MAE, where: The formula for mean absolute percentage error MAPE is: ; The formula for root mean square error RMSE is: ; The formula for mean absolute error MAE is: ; In the formula, is the total number of samples, is the true value, is the predicted value.

[0013] The present invention also provides a feature difference learning network traffic flow data prediction and processing system, comprising: The data preprocessing module is used to perform data preprocessing on the input original traffic time series data to obtain feature difference learning samples, wherein the original traffic time series data includes traffic data of multiple target road nodes within a time range, and the feature difference learning samples at least include: DXGAN neural network model building and training module, which is used to build the deep learning model DXGAN by integrating the sample location encoder, feature difference generator and discriminator; The prediction evaluation module is used to input the feature difference learning sample into the deep learning model DXGAN to obtain the prediction result.

[0014] Furthermore, the data preprocessing module records the variation characteristics of the original traffic time series data in time and space dimensions through a spatial sample position difference observation matrix and a temporal sample information difference matrix; The spatial sample position difference observation matrix is: When detected by the model, , the value of the matrix remains , on the contrary, when the position information of the sample hour, ,in represents the spatial retrieval state matrix, where Represents the coordinate position of the sensor node in the spatial dimension.

[0015] The representation of the time sample information difference matrix is ​​more inclined to be in time The change of the spatial sample position difference observation matrix can be expressed as: ; In the formula, Indicates the data in the sensor at time The data at the moment, represents the time sample information difference matrix. It means that the model did not record the spatial and temporal changes of the original traffic time series data at the previous moment.

[0016] Furthermore, the feature difference learning traffic flow data prediction and processing method is characterized in that the data preprocessing of the input original traffic time series data to obtain feature difference learning samples includes: The original traffic time series data are respectively , obtain data embedding of two different channel dimensions , It means that the original traffic time series data is mapped through the fully connected layer to obtain the embedded features in the spatial dimension. It means that the original traffic time series data is mapped through the fully connected layer to obtain the embedded features in the time dimension; According to the time sample information difference matrix , construct features , obtain the time sample information difference characteristics ; Embedding features in spatial and temporal dimensions And the original traffic time series data and trainable parameters Through the convolution operation , get the features on the convolution ,in, Represents the convolution operation function; according to And the location information features obtained using encoding technology Perform sum operation to obtain comprehensive information features Among them, the location information features obtained by using the encoding technology include: First location information features: ; Second location information features: ; Third location information features: ; Fourth location information feature: ; in, Indicates the position of the element index. represents the spatial distance parameter matrix, represents the transpose of the spatial distance parameter matrix; according to And the location information features obtained using encoding technology The sum of the two determines the comprehensive characteristics ; According to the location information characteristics of the target road node trend difference change and time-delay characteristics The sum of the two determines the comprehensive characteristics , Furthermore, the deep learning model DXGAN includes: a bit sample positioning encoder, a feature difference generator and a discriminator, wherein: The feature difference generator includes: a difference sample generation layer, a difference generator representation layer, and a difference generator output layer; The difference sample generation layer includes recording the feature differences at different times to convert the comprehensive information features of the nodes obtained above into Compare the differences and use It can be expressed as follows ,in Indicates at time The comprehensive feature representation of the input nodes.

[0017] The difference generator representation layer represents the difference information generated by the difference sample generation layer by receiving the difference information generated by the difference sample generation layer. , and the input at different times To learn the rich information representation in time series data, the representation layer consists of a GRU network and the representation is: , where the output state is , Denoted as the difference generator representation layer The hidden state of the layer is represented by Represent the update gate state and candidate hidden state respectively. The update gate state is represented as , Represented as the difference information feature generated by the difference sample generation layer, Represents the comprehensive information characteristics of the node, Denoted as the difference generator representation layer The hidden state of the layer is represented by, and the reset gate state is represented by , the candidate hidden state is represented as , concat represents the serial effect between states, , as well as , , Represents the network learnable parameters; The output layer of the difference generator is fitted with data through a fully connected layer. , and get the final output prediction sequence , where FC represents the fully connected layer, represents learnable parameters.

[0018] Compared with the prior art, the present invention has the following beneficial technical effects: The beneficial effects of the present invention are as follows: In view of the problems existing in the prior art, the present invention combines the generative adversarial model and the feature difference learning strategy, and for the first time proposes a feature difference learning network traffic flow data prediction processing strategy based on robustness analysis, analyzes the autocorrelation between nodes through the sample positioning encoder, and uses its spatiotemporal entanglement information to accurately locate the feature position, providing a balanced and rich learning sample for the feature difference generator. In addition, in order to improve the robustness and generalization ability of the model, we adopt the method of trend comparison between nodes, which not only maintains the consistency characteristics of the data, but also restores the heterogeneous characteristics of the original sample to the greatest extent. At the same time, we also analyze the characteristics of the scale-free network in the road network system, and combine the influence of individuals on neighboring individuals in complex systems, and distinguish the spatiotemporal feature differences between individuals through trended difference feature difference learning, which further enhances the robustness and generalization of the network. Finally, to solve the problem of data out-of-distribution, we adopt an adversarial feature difference discriminator to process spatiotemporal information in an adversarial decoupling manner, thereby improving the explanatory power and robustness of the model.

[0019] Compared with the existing technology, the present invention further improves the accuracy and applicability of traffic time series prediction, greatly enhances the prediction ability of existing models, and provides an explainable solution from the perspective of robustness: the high accuracy of the DXGAN model is used to improve the accuracy of traffic time series prediction, thereby providing a more accurate and personalized prediction solution for smart transportation.

[0020] The present invention provides a method and system for predicting and processing traffic flow data using a feature difference learning network. The method obtains feature difference learning samples by performing data preprocessing on the input original traffic time series data, wherein the original traffic time series data includes traffic data of multiple target road nodes within a time range; then, a deep learning model DXGAN is constructed by integrating a sample positioning encoder, a feature difference generator, and a discriminator; then, the feature difference learning samples are input into the deep learning model DXGAN to obtain a prediction result. The temporal and spatial feature differences between individuals are distinguished by trended difference learning, and the high accuracy of the DXGAN model is used to improve the accuracy of traffic time series prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flowchart of a method for predicting and processing network traffic flow data based on feature difference learning according to an embodiment of the present invention; Figure 2 is a flow chart of the preprocessing stage of an embodiment of the present invention; Figure 3 is a schematic diagram of a DXGAN neural network model in one embodiment of the present invention; Figure 4It is a structural diagram of a characteristic difference learning network traffic flow data prediction and processing system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The present invention is further described in detail below in conjunction with specific embodiments, which are intended to explain the present invention rather than to limit it.

[0023] The innovation of the present invention is that it proposes a method and system for predicting and processing traffic flow data based on feature difference learning network based on robustness analysis. The dynamic exchange generative adversarial network (DXGAN) further improves the prediction ability of the model from the perspective of robustness by encoding the spatiotemporal information of the original data and constructing the generator and the discriminator. The experiment was conducted on four data sets of PEMS03 / 04 / 07 / 08. The experimental results show that the strategy has good performance indicators in terms of overall performance and can achieve the best performance with fewer parameters, which provides a new time series prediction method with low spatiotemporal complexity under the condition of performance resource limitations.

[0024] In order to solve the problem of prediction errors caused by existing models ignoring local heterogeneity in spatiotemporal data, a feature difference learning network traffic flow data prediction processing method and system based on robust analysis was proposed. The coded spatiotemporal feature information and the idea of ​​difference learning were used to maximize the authenticity of the model for the long-term spatiotemporal heterogeneity of the original spatiotemporal data. Generator and discriminator networks were constructed to analyze and judge the comparison samples generated by the model, guiding the model to correct and improve the prediction results from a robust perspective, effectively expanding the performance and robustness problems of existing prediction models.

[0025] In response to the problems of long calculation time and poor effect of existing prediction algorithms, a feature difference learning network based on robustness analysis is proposed, which greatly improves the operation efficiency of the model while ensuring the excellent performance of the model itself.

[0026] In order to solve the problem of prediction errors caused by existing models ignoring local heterogeneity in spatiotemporal data, this paper proposes an innovative deep learning model - "Dynamic exchange Generative Adversarial Network" (DXGAN). The model consists of a sample positioning encoder, a feature difference generator and a discriminator. From the perspective of the robustness of the network system itself, the feature difference learning idea is used to design a model for prediction tasks.

[0027] In the test on the PEMS03 / 04 / 07 / 08 dataset, DXGAN shows excellent interpolation performance. The present invention conducts prediction experiments on the PEMS03 / 04 / 07 / 0 dataset.

[0028] Experimental results show that the mean absolute error (MAE) reaches 12.74 / 12.76 / 11.34 / 13.40 / 2.95 / 1.59, the mean absolute percentage error (MAPE) reaches 12.76 / 11.34 / 7.73 / 8.57 / 8.10 / 3.60, and the root mean square error (RMSE) reaches 1.44 / 28.03 / 29.53 / 20.98 / 8.10 / 3.60 respectively. The model effect is better than all the existing models proposed.

[0029] Due to its unique model architecture, DXGAN of the present invention is more applicable to intelligent transportation systems and a wide range of scenario applications, and also highlights the important role of technological innovation in improving the quality of life of urban residents.

[0030] Figure 1 FIG. 1 is a flow chart of a method for predicting network traffic flow data by learning characteristic differences according to an embodiment of the present invention; Figure 1 As shown, the feature difference learning network traffic flow data prediction processing method includes the following steps: Step S101: preprocess the input original traffic time series data to obtain feature difference learning samples.

[0031] In this embodiment, the original traffic time series data includes traffic data of multiple target road nodes within a time range; In the preprocessing stage, the PEMS04 dataset is extremely widely used in the transportation field. This dataset contains a total of 307 target road nodes. The dataset records a total of 16,992 timestamp traffic features from January to January 2018. This dataset has been proven to be effective and is used as a typical dataset in the field of traffic prediction.

[0032] like Figure 2 As shown, in order to facilitate the observation of the data change characteristics of the time series data in the time and space dimensions in the original features, the change characteristics of the original traffic time series data in the time and space dimensions are recorded through the spatial sample position difference observation matrix and the time sample information difference matrix; The spatial sample position difference observation matrix is: When detected by the model, , the value of the matrix remains , on the contrary, when the position information of the sample hour, ,in represents the spatial retrieval state matrix, where Represents the coordinate position of the sensor node in the spatial dimension.

[0033] The representation of the time sample information difference matrix is ​​more inclined to be in time The change of the spatial sample position difference observation matrix can be expressed as: ; In the formula, Indicates the data in the sensor at time The data at the moment, represents the time sample information difference matrix. It means that the model did not record the spatial and temporal changes of the original traffic time series data at the previous moment.

[0034] When learning task-related features, nonlinear neural networks often also capture features that are not relevant to the task. This makes it difficult for the network to maintain good generalization performance when facing data distribution shifts. At the same time, the complex heterogeneity of spatiotemporal data in road network systems often prevents neural networks from effectively learning high-quality features. To address these problems and enhance the learning of task-related temporal and spatial features, we propose a new method to process temporal and spatial information in raw time series data by encoding position information. This method helps improve the generalization ability of the model and the quality of feature learning.

[0035] For the original traffic time series data After being mapped by a fully connected layer and obtaining data embeddings of two different channel dimensions, the form can be expressed as: ; ; In the formula, It means that the original traffic time series data is mapped through the fully connected layer to obtain the embedded features in the spatial dimension. It means that the original traffic time series data is mapped through the fully connected layer to obtain the embedded features in the time dimension; according to the time sample information difference matrix , construct features , obtain the time sample information difference characteristics ; Embedding features in spatial and temporal dimensions And the original traffic time series data and trainable parameters Through the convolution operation , get the features on the convolution ,in, Represents the convolution operation function; according to And the location information features obtained using encoding technology Perform sum operation to obtain comprehensive information features Among them, the location information features obtained by using the encoding technology include: First location information features: ; Second location information features: ; Third location information features: ; Fourth location information feature: ; in, Indicates the position of the element index. represents the spatial distance parameter matrix, represents the transpose of the spatial distance parameter matrix; according to And the location information features obtained using encoding technology The sum of the two determines the comprehensive characteristics ; According to the location information characteristics of the target road node trend difference change and time-delay characteristics The sum of the two determines the comprehensive characteristics , Through normalization operation Determine the comprehensive information characteristics of the node , represents the normalization operation, Represents trainable weights.

[0036] After systematic comprehensive verification, the present invention confirms that the constructed sample positioning encoder can provide the model with sampling data that is more evenly distributed and can retain the original spatiotemporal characteristics to the greatest extent for the model to learn.

[0037] Step S102, constructing a deep learning model DXGAN by integrating a sample positioning encoder, a feature difference generator and a discriminator; like Figure 3 As shown, the core technology of building a deep learning model DXGAN in the present invention includes: the present invention proposes a feature difference learning network traffic flow data prediction processing method based on robustness analysis, which is used to solve the prediction task problem. The DXGAN neural network model mainly includes a data preprocessing module, a DXGAN neural network model construction and training module, and a prediction evaluation module.

[0038] The data preprocessing module is used to locate and analyze the initial time series information and to receive and transmit it.

[0039] The DXGAN neural network model construction and training module includes: a sample location encoder, a feature difference generator and a discriminator. The DXGAN neural network model is trained. In the process of multiple rounds of iterations, the DXGAN neural network model construction and training module has the ability to distinguish between real data and comparison samples. The DXGAN neural network model construction and training module will continuously update parameters during the training process to improve the model prediction performance.

[0040] The prediction evaluation module is used to judge the performance advancement of the model by combining the obtained prediction results with the performance evaluation indicators and save them.

[0041] For the sample localization encoder, this component can be used to provide the model with more uniform data samples to solve the local heterogeneity problem in the data samples.

[0042] Through the feature difference generator and discriminator, spatiotemporal features are captured from different potential perspectives, thereby enhancing the robustness and generalization of the system.

[0043] The feature difference generator includes: a difference sample generation layer, a difference generator representation layer, and a difference generator output layer; The difference sample generation layer includes recording the feature differences at different times to convert the comprehensive information features of the nodes obtained above into Compare the differences and use It can be expressed as follows ,in Indicates at time The comprehensive feature representation of the input nodes.

[0044] The difference generator representation layer represents the difference information generated by the difference sample generation layer by accepting the difference , and the input at different times To learn rich information representations in time series data to support the learning of model training strategies. Specifically, first, we use the GRU network as the unit for processing learning trends in our neural network framework, and the representation is: ; ; ; ; In the formula, Represents the update door status, Represents resetting the door state. represents the candidate hidden state, where the output state is , Represented as the difference information feature generated by the difference sample generation layer, Represents the comprehensive information characteristics of the node, Denoted as the difference generator representation layer The hidden state of the layer represents the concatenation between states. , as well as , , Represents the network learnable parameters; The output layer of the difference generator is fitted to the data through a fully connected layer , and get the final output prediction sequence , where FC represents the fully connected layer, represents learnable parameters.

[0045] The structure of the feature difference discriminator is similar to that of the generator, both of which have GRU units and output layers. constitute.

[0046] ; In the formula, Represents the overall loss, whose structure is the MAE loss in the general form And the discriminator loss constituted of which Represents a hyperparameter.

[0047] The discriminator loss The representation of can be expanded into: ; In the formula represents the discriminator symbol, means that the expectation is calculated based on samples drawn from the distribution P, Representation sample Extracted from distribution P (representing the real data distribution), that is is a sample that conforms to the actual data distribution P. represents the discriminator symbol, represents the generator symbol, means that the expectation is calculated based on samples drawn from the distribution P, Representation sample Samples drawn from distribution P (representing the true data distribution); In step S102, the DXGAN deep learning model includes: a sample positioning encoder module composed of a fully connected layer and encoding technology, a generator module Generator composed of GRU units, and a discriminator module Discriminator, so that the data processed by each module is output in parallel; the encoder module Encoder is used to realize the embedded representation of the road signal and convert it into more uniform sampling data that preserves the original spatiotemporal features to the greatest extent. The generator module Generator uses the GRU network for feature learning and constructs feature difference learning data. The discriminator module is used to maximize the accuracy of the real data while minimizing the accuracy of the comparison data generated by the generator module.

[0048] Step S103 inputs the feature difference learning sample into the deep learning model DXGAN to obtain a prediction result.

[0049] Specifically, this embodiment includes: Step 1: Input the original road node sensor information , the maximum number of iterations is , the training sample positioning encoder and the feature difference generator generate feature difference learning samples for model learning; Step 2: Input the feature difference learning information generated by the generator in step 1 into the feature difference discriminator, and train the feature difference discriminator so that the total loss The value is the smallest; Step 3: Alternate the feature difference generator and discriminator to make the model loss converge to the correct imitation direction, iterate times, until the maximum number of iterations N; Step 4: Obtain the road network dataset through the output layer of the discriminator , and calculate the loss value.

[0050] In step S103, the deep learning model DXGAN constructed by inputting the feature difference learning sample data is analyzed, and the model prediction performance is analyzed, including: the prediction performance is evaluated, and the model operation efficiency, time complexity, actual memory usage and actual performance are evaluated.

[0051] In step S103, the deep learning model DXGAN constructed by inputting the feature difference learning sample data is analyzed, and its indexes are judged and saved, including: in order to fully evaluate the performance advantages of the model, the mean absolute percentage error MAPE, the root mean square error RMSE, and the mean absolute error MAE are used, and the calculation formula is expressed as: ; ; ; In the formula, is the total number of samples, is the true value, is the predicted value; It can be seen from the above embodiments that the expected benefits and commercial value of the technical solution of the present invention after transformation are as follows: a. Improve the efficiency and accuracy of traffic network system management: The high accuracy and sensitivity of the DXGAN model in predicting traffic flow at road network nodes indicate that it can significantly improve the efficiency and accuracy of road network system node prediction in real-world applications. This is of great significance for improving the quality of life of urban residents and reducing road travel delays.

[0052] b. Reducing the burden on intelligent transportation systems: Due to the efficiency and reliability of DXGAN, it can reduce false predictions, thereby reducing transportation costs and system burdens.

[0053] c. Commercialization potential: Due to its high performance in traffic system prediction, the DXGAN model has strong commercialization potential and may attract the attention of urban road system builders and residents' travel developers.

[0054] d. Technology promotion and application: This model can be integrated into a variety of smart transportation systems, such as wearable devices or smartphone applications, to provide users with real-time traffic travel predictions and road congestion warning systems.

[0055] The technical solution of the present invention fills the technical gap in the industry: a. Innovative model design: DXGAN provides a feature difference learning network traffic flow data prediction processing method based on robustness analysis. This unique structure is rare in the existing technology.

[0056] b. Efficient resource utilization: Due to its lightweight design, DXGAN is able to maintain high performance in environments with limited computing resources, which is a significant advancement in existing traffic flow prediction techniques.

[0057] c. Accuracy of data processing and prediction: Compared with existing technologies, DXGAN demonstrates higher accuracy and reliability in data processing and traffic system node prediction.

[0058] d. Applicability and efficiency: The design of the DXGAN model takes into account the generalization of different data sets, which enables it to be applicable to road network data in different scenarios and provide generally applicable solutions. In addition, due to its lightweight design, the DXGAN model can achieve higher prediction accuracy with fewer parameters, greatly improving computational efficiency.

[0059] The technical solution of the present invention overcomes technical prejudice: a. Overcoming the reliance on traditional methods: Previously, many road network flow prediction methods relied on traditional statistics or simple machine learning techniques. DXGAN introduced advanced deep learning technology to break the reliance on traditional methods.

[0060] b. Solve the limitations of data processing: Traditional road traffic flow data prediction and processing methods often ignore the problem of reduced prediction accuracy caused by the local heterogeneity of the signal itself. DXGAN solves this problem from the perspective of system robustness by integrating position sample positioning encoders and feature difference generators and discriminators, providing more comprehensive data analysis.

[0061] c. Challenges across dataset diversity: Faced with differences in the distribution of different datasets, many methods find it difficult to adapt. DXGAN successfully overcomes this technical bias through its design flexibility and provides a personalized and adaptable solution.

[0062] Example 2, as another implementation of the present invention, the feature difference learning network traffic flow data prediction processing method based on robustness analysis and the system operation process provided in the embodiment of the present invention include: Specifically, the following steps are included: I. System initialization and configuration, import libraries and tools, including 'numpy', 'pandas', 'scipy', 'torch', 'matplotlib', etc.

[0063] Set global parameters, batch size: 64; learning rate: 0.001; number of training rounds: 100; initialize CUDA environment.

[0064] II. Data Processing and Feature Extraction,(1) Traffic data import,Use 'numpy.load' to load the data file in .npy format from the specified path.

[0065] Data preprocessing, including: In order to facilitate the observation of the data change characteristics of the time series data in the original features in the time and space dimensions, the change characteristics of the original traffic time series data in the time and space dimensions are recorded by designing the spatial sample position difference observation matrix and the time sample information difference matrix; The spatial sample position difference observation matrix is: When detected by the model, , the value of the matrix remains , on the contrary, when the position information of the sample hour, ,in Represents the coordinate position of the sensor node in the spatial dimension.

[0066] The representation of the time sample information difference matrix is ​​more inclined to be in time The change of the spatial sample position difference observation matrix can be expressed as: ; In the formula, Indicates the data in the sensor at time The data at the moment. , it means that the model did not record the spatial and temporal changes of the original traffic time series data at the previous moment.

[0067] III. Build and train a neural network model, including: (1) DXGAN neural network model design; (2) Dataset preparation: For PEMS03 / 04 / 07 / 08, the dataset was divided according to a ratio of 6:2:2, and the data were randomly shuffled and divided.

[0068] (3) Model training, including: using 'torch.optim.Adam' as the optimizer; using 5-fold cross validation; loss function: ' ';Record the loss, MAE, MAPE, RMSE and other indicators of training and testing.

[0069] IV. Performance evaluation and result recording, including: (1) Evaluation metric calculation: MSE, RMSE, MAE; (2) Best model selection: based on test results , number of training parameters, and training iteration time; (3) Result visualization: prediction effect diagram and synthetic prediction and true distribution; (4) Result saving: save the training and test results to CSV files.

[0070] V. System optimization and adjustment, including: Parameter tuning: Adjust the network structure and training parameters based on the performance results.

[0071] Feature selection: Analyze the impact of different features on model performance.

[0072] Code optimization: Optimize code to improve efficiency.

[0073] VI. Practical applications and testing include: Practical application test: Evaluate the performance of DXGAN neural network model on datasets such as PEMS04; System deployment: Gradually expand the DXGAN neural network model into existing public data sets to improve the performance of data sets.

[0074] It can be seen from the above embodiments that DXGAN consists of a sample positioning encoder, a feature difference generator and a discriminator, which effectively solves the local heterogeneity problem and data inhomogeneity of traffic flow prediction in intelligent transportation systems, and significantly improves the robustness and generalization ability of the prediction model.

[0075] Solutions to data inhomogeneity and collection limitations: Due to the layout and functional limitations of sensors, the acquired traffic data often has the problem of uneven distribution, which will affect the accuracy and robustness of the prediction model. By introducing sample location encoders, feature difference generators and discriminators, DXGAN can analyze the spatiotemporal characteristics of traffic flow from multiple perspectives, enhancing the model's ability to handle uneven data and prediction accuracy.

[0076] Solution to the local heterogeneity problem: Traditional traffic flow prediction models often have difficulties in dealing with fine-grained sensor data with spatial and temporal heterogeneity. DXGAN's sample localization encoder solves the local heterogeneity problem in data samples by providing more uniform data samples, helping the model to more effectively learn and predict traffic nodes with complex dependencies.

[0077] Good robustness: Test results on multiple datasets such as PEMS04 show that DXGAN improves the model's adaptability and generalization capabilities when facing complex traffic conditions in the real world through the application of feature difference learning and deep neural network technology, thereby enabling the model to show superior prediction performance on multiple real-world datasets.

[0078] Embodiment 3, as Figure 3 As shown, the feature difference learning network traffic flow data prediction and processing system based on robustness analysis provided by the embodiment of the present invention includes: The data preprocessing module is used to locate and analyze the initial time series information and receive and transmit it.

[0079] A DXGAN neural network model construction and training module is used to train the DXGAN neural network model by integrating a sample location encoder, a feature difference generator and a discriminator, The prediction evaluation module is used to judge the performance advancement of the model by combining the obtained prediction results with the performance evaluation indicators and save them.

[0080] It can be seen from the above embodiments that the present invention effectively solves the local heterogeneity problem and data inhomogeneity of traffic flow prediction in intelligent transportation systems by developing a new type of spatiotemporal feature difference learning neural network, and significantly improves the robustness and generalization ability of the prediction model.

[0081] Secondly, the present invention provides a new network system that optimizes the model prediction ability from the perspective of the robustness of the model system itself, using the feature difference learning method to capture and analyze spatiotemporal features from different potential perspectives. This not only helps the model take into account the trend and consistency of traffic flow when predicting, but also enhances the robustness and generalization ability of the model in the face of data imbalance and collection restrictions.

[0082] Figure 4 FIG. 1 is a schematic diagram of the structure of a feature difference learning network traffic flow data prediction and processing system according to an embodiment of the present invention. Figure 4 As shown, the feature difference learning network traffic flow data prediction and processing system includes: a data preprocessing module 41, a DXGAN neural network model construction and training module 42 and a prediction evaluation module 43, wherein: The data preprocessing module 41 is used to perform data preprocessing on the input original traffic time series data to obtain feature difference learning samples, wherein the original traffic time series data includes traffic data of multiple target road nodes within a time range, and the feature difference learning samples at least include: A DXGAN neural network model building and training module 42, for building a deep learning model DXGAN by integrating a sample positioning encoder, a feature difference generator, and a discriminator; The prediction evaluation module 43 is used to input the feature difference learning sample into the deep learning model DXGAN to obtain a prediction result.

[0083] On the basis of the above embodiment, the data preprocessing module 41 is used to record the variation characteristics of the original traffic time series data in time and space dimensions through the spatial sample position difference observation matrix and the time sample information difference matrix; The spatial sample position difference observation matrix is: When detected by the model, , the value of the matrix remains , on the contrary, when the position information of the sample hour, ,in Represents the spatial retrieval state matrix, which takes values ​​of 0 or 1. Represents the coordinate position of the sensor node in the spatial dimension.

[0084] The representation of the time sample information difference matrix is ​​more inclined to be in time The change of the spatial sample position difference observation matrix can be expressed as: ; In the formula, Indicates the data in the sensor at time The data at the moment. , it means that the model did not record the spatial and temporal changes of the original traffic time series data at the previous moment.

[0085] Furthermore, the original traffic time series data are respectively , obtain data embedding of two different channel dimensions , It means that the original traffic time series data is mapped through the fully connected layer to obtain the embedded features in the spatial dimension. It means that the original traffic time series data is mapped through the fully connected layer to obtain the embedded features in the time dimension; According to the time sample information difference matrix , construct features , obtain the time sample information difference characteristics ; Embedding features in spatial and temporal dimensions And the original traffic time series data and trainable parameters Through the convolution operation , get the features on the convolution ,in, Represents the convolution operation function; according to And the location information features obtained using encoding technology Perform sum operation to obtain comprehensive information features Among them, the location information features obtained by using the encoding technology include: First location information features: ; Second location information features: ; Third location information features: ; Fourth location information feature: ; in, Indicates the position of the element index. represents the spatial distance parameter matrix, represents the transpose of the spatial distance parameter matrix; according to And the location information features obtained using encoding technology The sum of the two determines the comprehensive characteristics ; According to the location information characteristics of the target road node trend difference change and time-delay characteristics The sum of the two determines the comprehensive characteristics , Through normalization operation Determine the comprehensive information characteristics of the node , represents the normalization operation, Represents trainable weights.

[0086] Based on the above embodiment, the deep learning model DXGAN includes: a sample positioning encoder, a feature difference generator and a discriminator, wherein: The feature difference generator includes: a difference sample generation layer, a difference generator representation layer, and a difference generator output layer; The difference sample generation layer includes recording the feature differences at different times to convert the comprehensive information features of the nodes obtained above into Compare the differences and use It can be expressed as follows ,in Indicates at time The comprehensive feature representation of the input nodes.

[0087] The difference generator representation layer represents the difference information generated by the difference sample generation layer by receiving the difference information generated by the difference sample generation layer. , and the input at different times To learn the rich information representation in time series data, the representation layer consists of a GRU network and the representation is: , where the output state is , Denoted as the difference generator representation layer The hidden state of the layer is represented by Represent the update gate state and candidate hidden state respectively. The update gate state is represented as , Represented as the difference information feature generated by the difference sample generation layer, Represents the comprehensive information characteristics of the node, Denoted as the difference generator representation layer The hidden state of the layer is represented by, and the reset gate state is represented by , the candidate hidden state is represented as , concat represents the serial effect between states, , as well as , , Represents the network learnable parameters; The output layer of the difference generator is fitted with data through a fully connected layer. , and get the final output prediction sequence , where FC represents the fully connected layer, represents learnable parameters.

[0088] The implementation principle and technical effect of this embodiment are Figure 1-Figure 4 The implementation principles and technical effects shown are similar and will not be repeated here.

[0089] An embodiment of the present invention further provides a computer device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps in any of the above-mentioned method embodiments when executing the computer program.

[0090] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0091] An embodiment of the present invention further provides an information data processing terminal, which, when executed on an electronic device, provides a user input interface to implement the steps in the above method embodiments. The information data processing terminal is not limited to mobile phones, computers, and switches.

[0092] An embodiment of the present invention further provides a server, which is used to provide a user input interface to implement the steps in the above method embodiments when executed on an electronic device.

[0093] An embodiment of the present invention provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0094] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera device / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0095] To further illustrate the effects of the embodiments of the present invention, the following experiment is conducted: During the training process, the DXGAN neural network first preprocesses the original data using the sample positioning encoder, then alternately trains the feature difference generator and the discriminator, and uses The function is constrained to achieve the best prediction effect. In each iteration, the feature difference generator and the discriminator are trained the same number of times. By comprehensively comparing the outputs of the feature difference generator and the discriminator, the neural network model can achieve the best prediction effect under the given constraints. The model is trained and converged, and the Adam optimizer is used to optimize the neural network model parameters. After multiple rounds of iterations, the neural network model will tend to converge and reach a stable area.

[0096] As shown in Table 1, experiments on four datasets show that our model DXGAN outperforms the existing state-of-the-art models in all evaluation indicators. The performance data of these baseline models are derived from widely cited original literature, ensuring the fairness of the comparison. The experimental results show that the performance of the network model proposed in this paper is better than other model indicators.

[0097] Table 1 Performance test of DXGAN model on PEMS03 / 04 / 07 / 08 dataset

[0098] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for predicting and processing network traffic flow data by feature difference learning, characterized in that: include: Performing data preprocessing on the input original traffic time series data to obtain feature difference learning samples, wherein the original traffic time series data includes traffic data of multiple target road nodes within a time range; Through the sample positioning encoder, feature difference generator and discriminator, a deep learning model dynamic exchange generative adversarial network DXGAN is constructed; The feature difference learning sample is input into the deep learning model DXGAN to obtain a prediction result.

2. The feature difference learning network traffic flow data prediction and processing method according to claim 1 is characterized in that: The data preprocessing of the input original traffic time series data to obtain feature difference learning samples includes: Record the variation characteristics of the original traffic time series data in time and space dimensions through the spatial sample position difference observation matrix and the temporal sample information difference matrix; The spatial sample position difference observation matrix is: When detected by the model, determine When , the value of the matrix remains , on the contrary, when the position information of the sample hour, ,in represents the spatial retrieval state matrix, Represents the coordinate position of the sensor node in the spatial dimension; The representation of the time sample information difference matrix is ​​more inclined to be in time The change of the observation matrix of the spatial sample position difference is expressed as: ； In the formula, Indicates the data in the sensor at time The data at the moment, represents the time sample information difference matrix, when It means that the model did not record the spatial and temporal changes of the original traffic time series data at the previous moment.

3. The method for predicting and processing traffic flow data using feature difference learning network according to claim 1 is characterized in that: The data preprocessing of the input original traffic time series data to obtain feature difference learning samples includes: The original traffic time series data are respectively , obtain data embedding of two different channel dimensions , It means that the original traffic time series data is mapped through the fully connected layer to obtain the embedded features in the spatial dimension. It means that the original traffic time series data is mapped through the fully connected layer to obtain the embedded features in the time dimension; According to the time sample information difference matrix , construct features , obtain the time sample information difference characteristics ; Embedding features in spatial and temporal dimensions And the original traffic time series data and trainable parameters Through the convolution operation , get the features on the convolution ,in, Represents the convolution operation function; according to And the location information features obtained using encoding technology Perform sum operation to obtain comprehensive information features , where the location information features obtained using the encoding technology include: First location information features: ; Second location information features: ; Third location information features: ; Fourth location information feature: ; in, Indicates the position of the element index. represents the spatial distance parameter matrix, represents the transpose of the spatial distance parameter matrix; according to And the location information features obtained using encoding technology The sum of the two determines the comprehensive characteristics ; According to the location information characteristics of the target road node trend difference change and time-delay characteristics The sum of the two determines the comprehensive characteristics , Through normalization operation Determine the comprehensive information characteristics of the node , represents the normalization operation, Represents trainable weights.

4. The method for predicting and processing traffic flow data using feature difference learning network according to claim 3 is characterized in that: The deep learning model DXGAN includes: a sample positioning encoder, a feature difference generator and a discriminator, wherein: The feature difference generator includes: a difference sample generation layer, a difference generator representation layer, and a difference generator output layer; The difference sample generation layer includes recording the feature differences at different times to convert the comprehensive information features of the nodes obtained above into Compare the differences and use It can be expressed as follows ,in Indicates at time Comprehensive feature representation of input nodes; The difference generator representation layer represents the difference information generated by the difference sample generation layer by receiving the difference information generated by the difference sample generation layer. , and the input at different times To learn the rich information representation in time series data, the representation layer consists of a GRU network and the representation is: , where the output state is , Denoted as the difference generator representation layer The hidden state of the layer is represented by Represent the update gate state and candidate hidden state respectively; the update gate state is represented as , Represented as the difference information feature generated by the difference sample generation layer, Represents the comprehensive information characteristics of the node, Denoted as the difference generator representation layer The hidden state of the layer is represented by, and the reset gate state is represented by , the candidate hidden state is represented as , concat represents the serial effect between states, , as well as , , Represents the network learnable parameters; The output layer of the difference generator is fitted with data through a fully connected layer. , and get the final output prediction sequence , where FC represents the fully connected layer, represents learnable parameters.

5. The method for predicting and processing traffic flow data using feature difference learning network according to claim 4 is characterized in that: The feature difference discriminator is , determine the overall loss ,in, is the MAE loss, is the discriminator loss, Expressed as a hyperparameter; The discriminator loss is expressed as: , represents the discriminator symbol, represents the generator symbol, means that the expectation is calculated based on samples drawn from the distribution P, Representation sample Samples drawn from distribution P (representing the true data distribution).

6. The method for predicting and processing traffic flow data using feature difference learning network according to claim 5 is characterized in that: The step of inputting the feature difference learning sample into the deep learning model DXGAN to obtain a prediction result includes: The prediction results are evaluated by the mean absolute percentage error MAPE, root mean square error RMSE and mean absolute error MAE, where: The formula for mean absolute percentage error MAPE is: ; The formula for root mean square error RMSE is: ; The formula for mean absolute error MAE is: ; In the formula, is the total number of samples, is the true value, is the predicted value.

7. A feature difference learning network traffic flow data prediction and processing system, characterized in that: include: The data preprocessing module is used to perform data preprocessing on the input original traffic time series data to obtain feature difference learning samples, wherein the original traffic time series data includes traffic data of multiple target road nodes within a time range, and the feature difference learning samples at least include: Dynamic exchange generative adversarial network DXGAN neural network model construction and training module, which is used to build the deep learning model DXGAN by integrating sample positioning encoder, feature difference generator and discriminator; The prediction evaluation module is used to input the feature difference learning sample into the deep learning model DXGAN to obtain a prediction result.

8. The feature difference learning network traffic flow data prediction and processing system according to claim 7 is characterized in that: The data preprocessing module is used to record the variation characteristics of the original traffic time series data in time and space dimensions through a spatial sample position difference observation matrix and a temporal sample information difference matrix; The spatial sample position difference observation matrix is: When detected by the model, determine When , the value of the matrix remains , on the contrary, when the position information of the sample hour, ,in represents the spatial retrieval state matrix, Represents the coordinate position of the sensor node in the spatial dimension; The representation of the time sample information difference matrix is ​​more inclined to be in time The change of the spatial sample position difference observation matrix can be expressed as: ； In the formula, Indicates the data in the sensor at time The data at the moment, represents the time sample information difference matrix, when It means that the model did not record the spatial and temporal changes of the original traffic time series data at the previous moment.

9. The feature difference learning network traffic flow data prediction and processing system according to claim 8 is characterized in that: The data preprocessing module is also used to process the original traffic time series data respectively , obtain data embedding of two different channel dimensions , It means that the original traffic time series data is mapped through the fully connected layer to obtain the embedded features in the spatial dimension. It means that the original traffic time series data is mapped through the fully connected layer to obtain the embedded features in the time dimension; According to the time sample information difference matrix , construct features , obtain the time sample information difference characteristics ; Embedding features in spatial and temporal dimensions And the original traffic time series data and trainable parameters Through the convolution operation , get the features on the convolution ,in, Represents the convolution operation function; according to And the location information features obtained using encoding technology Perform sum operation to obtain comprehensive information features , where the location information features obtained using the encoding technology include: First location information features: ; Second location information features: ; Third location information features: ; Fourth location information feature: ; in, Indicates the position of the element index. represents the spatial distance parameter matrix, represents the transpose of the spatial distance parameter matrix; according to And the location information features obtained using encoding technology The sum of the two determines the comprehensive characteristics ; According to the location information characteristics of the target road node trend difference change and time-delay characteristics The sum of the two determines the comprehensive characteristics , Through normalization operation Determine the comprehensive information characteristics of the node , represents the normalization operation, Represents trainable weights.

10. The feature difference learning network traffic flow data prediction and processing system according to claim 9 is characterized in that: The deep learning model DXGAN includes: a sample positioning encoder, a feature difference generator and a discriminator, wherein: The feature difference generator includes: a difference sample generation layer, a difference generator representation layer, and a difference generator output layer; The difference sample generation layer includes recording the feature differences at different times to convert the comprehensive information features of the nodes obtained above into Compare the differences and use It can be expressed as follows ,in Indicates at time Comprehensive feature representation of input nodes; The difference generator representation layer represents the difference information generated by the difference sample generation layer by receiving the difference information generated by the difference sample generation layer. , and the input at different times To learn the rich information representation in time series data, the representation layer consists of a GRU network and the representation is: , where the output state is , Denoted as the difference generator representation layer The hidden state of the layer is represented by Represent the update gate state and candidate hidden state respectively, where the update gate state is represented as , Represented as the difference information feature generated by the difference sample generation layer, Represents the comprehensive information characteristics of the node, Denoted as the difference generator representation layer The hidden state of the layer is represented by, and the reset gate state is represented by , the candidate hidden state is represented as , concat represents the serial effect between states, , as well as , , Represents the network learnable parameters; The output layer of the difference generator is fitted with data through a fully connected layer. , and get the final output prediction sequence , where FC represents the fully connected layer, represents learnable parameters.

Images