Traffic flow data processing method and feature selection method, device and storage medium

Abstract

This application relates to the field of intelligent transportation and power system load forecasting technology, and provides a method, device, and storage medium for processing traffic flow data and selecting features. The method includes: constructing a generative model based on temporal generative adversarial networks and long short-term memory neural networks to generate simulation sequences for data augmentation; performing quality enhancement processing on the original traffic flow data and simulation sequences based on a general temporal difference feature construction method to construct an enhanced dataset containing original features and newly added statistical features; reconstructing trend components and residual components into fluctuation components based on a local weighted smoothing method; evaluating the feature importance of periodic components and fluctuation components respectively based on a limit gradient boosting algorithm to obtain feature importance evaluation indices; and quantitatively sorting and filtering all features based on the feature importance evaluation indices to eliminate redundant features and extract a subset of core features.

Description

Technical Field

[0001] This application relates to the field of intelligent transportation and power system load forecasting technology, and in particular to a method for processing traffic flow data, a method for feature selection, an apparatus, and a storage medium. Background Technology

[0002] With the increasing popularity of electric vehicles and the development of intelligent urban transportation, accurate prediction of electric vehicle traffic flow has become a key prerequisite for optimizing traffic control and achieving coordinated scheduling of power grids and charging facilities. Accurate prediction relies on high-quality spatiotemporal traffic flow data acquired from multi-source heterogeneous sensors (e.g., inductive loops, cameras, GPS, etc.). However, this raw data often suffers from inconsistent quality, feature redundancy, and difficulty in directly revealing its inherent variation patterns, posing a significant challenge to the performance of downstream prediction models.

[0003] Currently, to improve the quality of input data for predictive models, the industry typically employs various data processing and feature engineering methods. For example, missing traffic flow data can be filled in using simple interpolation or statistical methods; traditional filtering algorithms can be used to smooth the data and suppress noise; or domain knowledge can be relied upon to manually construct statistical features and directly input them into the predictive model. These methods alleviate some data quality problems to a certain extent.

[0004] However, existing technical solutions still have the following significant limitations: 1) For problems such as sparse data or missing extreme scenarios, simple interpolation or statistical methods are insufficient to generate simulation data that conforms to the spatiotemporal distribution of real traffic flow, resulting in insufficient and undiversified training samples for the model, affecting generalization ability; 2) Traditional manual feature engineering methods often fail to deeply explore the complex temporal dependencies and multi-scale periodic characteristics in traffic flow data, resulting in limited discriminative power of the constructed features and a tendency to introduce redundancy or noise; 3) There is a lack of a mechanism that can automatically and quantitatively evaluate the contribution of various feature components (e.g., long-term trends, periodic patterns, random fluctuations) in high-dimensional spatiotemporal features to the prediction target, leading to a feature selection process that relies on experience, is inefficient, and makes it difficult to obtain the optimal feature subset. These problems collectively restrict further improvements in the accuracy and reliability of traffic flow prediction models. Summary of the Invention

[0005] This application provides a method for processing traffic flow data, a method, apparatus, and storage medium for feature selection, which can automatically generate a high-quality subset of core features for traffic flow prediction models.

[0006] On the one hand, this application provides a method for processing traffic flow data and a feature selection method, the method comprising: Step S1: Construct a generative model based on temporal generative adversarial network and long short-term memory neural network. Use the autoencoder component in the generative model to encode and decode the input traffic flow spatiotemporal data, and use the generative adversarial component to generate and identify the data. Learn the data distribution pattern in a joint training manner to generate simulation sequences for data augmentation. Step S2: Based on the general temporal difference feature construction method, perform quality enhancement processing on the original traffic flow data and the simulation sequence to construct an enhanced dataset containing the original features and newly added statistical features. The quality enhancement processing on the original traffic flow data and the simulation sequence includes calculating the difference components of the temporal features, grouping the features according to the difference components, and calculating the statistics of each group. Step S3: Based on the local weighted smoothing method, the traffic flow data in the enhanced dataset is decomposed into trend components, periodic components and residual components through an internal iterative process that includes detrending and deperiodicization. The trend components and the residual components are then reconstructed into fluctuation components that characterize non-periodic changes, so that the periodic components and the fluctuation components can be feature-analyzed and evaluated separately. Step S4: Based on the extreme gradient boosting algorithm and using the feature splitting criterion based on the "gain" mechanism, the feature importance of the periodic composition and the fluctuation composition obtained in step S3 are evaluated respectively. By calculating the sum of the squared error risk estimation improvement brought by each feature when splitting at all boosting decision tree nodes, the feature importance evaluation index directly related to the performance improvement of the prediction model is obtained. Step S5: Based on the feature importance evaluation index, all features constituting the periodic and volatile components are mixed and sorted, and filtered based on the dynamic correlation threshold to remove redundant features and extract the core feature subset.

[0007] On the other hand, this application provides a method for processing traffic flow data and a feature selection device, the device comprising: The training module is used to construct a generative model based on temporal generative adversarial network and long short-term memory neural network. It uses the autoencoder component in the generative model to encode and decode the input traffic flow spatiotemporal data, and uses the generative adversarial component to generate and identify the data. It learns the data distribution pattern in a joint training manner and generates simulation sequences for data augmentation. The construction module is used to perform quality enhancement processing on the original traffic flow data and the simulation sequence based on the general time difference feature construction method, so as to construct an enhanced dataset containing the original features and the newly added statistical features. The quality enhancement processing of the original traffic flow data and the simulation sequence includes calculating the difference components of the time series features, grouping the features according to the difference components, and calculating the statistics of each group. The reconstruction module is used to decompose the traffic flow data in the enhanced dataset into trend components, periodic components, and residual components through an internal iterative process that includes detrending and deperiodicization based on the local weighted smoothing method. The trend components and the residual components are then reconstructed into fluctuation components that characterize non-periodic changes, so that the periodic components and the fluctuation components can be feature-analyzed and evaluated separately. The evaluation module is used to evaluate the feature importance of the periodic component and the fluctuation component obtained in step S3 based on the limit gradient boosting algorithm and the feature splitting criterion based on the "gain" mechanism. By calculating the sum of the squared error risk estimation improvement brought by each feature when splitting at all boosting decision tree nodes, the feature importance evaluation index directly related to the performance improvement of the prediction model is obtained. The extraction module is used to perform mixed sorting of all features constituting the periodic and volatile components based on the feature importance evaluation index, and to filter based on the dynamic correlation threshold to remove redundant features and extract the core feature subset.

[0008] Thirdly, this application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the technical solutions of the traffic flow data processing method and feature selection method described above.

[0009] Fourthly, this application provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the technical solutions of the traffic flow data processing method and feature selection method described above.

[0010] As can be seen from the technical solutions provided in this application, on the one hand, by constructing a generative model based on temporal generative adversarial networks and long short-term memory neural networks, and jointly training the original data, the generated simulation sequences can not only effectively fill the missing scenarios in the original data, but also have high fidelity because they follow the distribution of real data. This significantly expands the high-quality and diverse sample library required for downstream model training, laying a solid data foundation for improving the generalization ability and robustness of the model. On the other hand, by using a general temporal difference feature construction method for quality enhancement, not only are the original features preserved, but also new statistical features reflecting the trend of data change are innovatively constructed from the perspective of temporal difference components. This results in an enhanced dataset with richer information and stronger discriminative power, enabling subsequent models to capture the evolution pattern of traffic flow from more dimensions. On the other hand, by decomposing the enhanced traffic flow data into trend, periodic and residual components based on the local weighted smoothing method, and reconstructing the trend and residual into fluctuation components, the mixed original signals are separated into components with different physical meanings. This decomposition allows subsequent analysis to target the relatively stable and regular periodic components and Characterizing short-term fluctuations and anomalous events helps the model learn traffic flow patterns at different scales more clearly and effectively, avoiding interference between different patterns. Thirdly, by using a limit gradient boosting algorithm and a feature splitting criterion based on a "gain" mechanism, features in both the periodic and fluctuating components are evaluated. The sum of the squared error risk estimation improvements brought by each feature when splitting at all decision tree nodes is accurately calculated. This metric provides a quantifiable, data-driven importance score for each feature from the perspective of its contribution to model performance improvement, overcoming the blindness and subjectivity of manual experience-based selection and making the feature selection process more data-driven. Fourthly, based on the above quantitative evaluation metrics, all features are quantitatively ranked and filtered, automatically and efficiently eliminating redundant features with low contribution to the prediction target. The final extracted core feature subset consists of high-impact feature factors selected through filtering. Inputting this concise and powerful feature subset into the downstream prediction model effectively reduces model complexity, reduces overfitting risk, accelerates model training, and ultimately significantly improves the accuracy and reliability of traffic flow prediction or charging load prediction. In summary, the technical solution of this application provides a complete process from data generation, enhancement, decomposition to evaluation and screening, which can automatically generate high-quality core feature subsets for traffic flow prediction models. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of the traffic flow data processing method and feature selection method provided in the embodiments of this application; Figure 2 This is a schematic diagram of the traffic flow data processing method and feature selection device provided in the embodiments of this application; Figure 3 This is a schematic diagram of the device provided in the embodiments of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] In this specification, adjectives such as "first" and "second" are used only to distinguish one element or action from another, without necessarily requiring or implying any actual such relationship or order. Where circumstances permit, reference to an element or component or step (etc.) should not be construed as being limited to only one of the elements, components, or steps, but may be one or more of the elements, components, or steps, etc.

[0015] For ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn to actual scale.

[0016] Currently, to improve the quality of input data for prediction models, the industry typically employs various data processing and feature engineering methods. For example, simple interpolation or statistical methods are used to fill in missing traffic flow data; traditional filtering algorithms are used to smooth the data and suppress noise; or domain knowledge is relied upon to manually construct statistical features (e.g., mean, variance) and directly input them into the prediction model. These methods alleviate some data quality issues to a certain extent. However, existing technical solutions still have the following significant limitations: 1) For problems of sparse data or missing data in extreme scenarios, simple interpolation or statistical methods struggle to generate simulation data that conforms to the spatiotemporal distribution of real traffic flow, resulting in insufficient and diverse training samples for the model, affecting generalization ability; 2) Traditional manual feature engineering methods often fail to deeply explore the complex temporal dependencies and multi-scale periodic characteristics in traffic flow data, resulting in limited discriminative power of the constructed features and a tendency to introduce redundancy or noise; 3) There is a lack of a mechanism that can automatically and quantitatively evaluate the contribution of various feature components (e.g., long-term trends, periodic patterns, random fluctuations) in high-dimensional spatiotemporal features to the prediction target, leading to a feature selection process that relies on experience, is inefficient, and struggles to obtain the optimal feature subset. These issues collectively hinder further improvements in the accuracy and reliability of traffic flow prediction models.

[0017] To address the aforementioned problems in the prior art, this application proposes a method for processing traffic flow data and a method for feature selection, the flowchart of which is attached. Figure 1 As shown, it mainly includes steps S1 to S5, which are detailed below: Step S1: Construct a generative model based on temporal generative adversarial network and long short-term memory neural network. Use the autoencoder component in the generative model to encode and decode the input traffic flow spatiotemporal data, and use the generative adversarial component to generate and identify the data. Learn the data distribution pattern through joint training to generate simulation sequences for data augmentation.

[0018] In the field of traffic flow data modeling, a long-standing challenge is data sparsity and insufficient sample coverage. Traditional methods, such as linear interpolation, fill in data based only on adjacent known data points, failing to learn and reproduce the complex distribution patterns of traffic flow over longer time series and across spatial dimensions (e.g., peak hour patterns, holiday patterns). The generated filler data is often too smooth, losing true fluctuation characteristics. Traditional time series generative models (e.g., ARIMA, state-space models) are usually based on strong linear or stationarity assumptions, making it difficult to characterize the complex nonlinear and non-stationary dynamics of traffic flow under the interplay of multiple factors such as weather and events, resulting in weak generalization ability. In view of this, the technical solution adopted in this application is to construct a generative model based on temporal generative adversarial networks and long short-term memory neural networks. The autoencoder component in the generative model encodes and decodes the input spatiotemporal traffic flow data, and the generative adversarial component generates and identifies the data. Through joint training, the data distribution patterns are learned, generating simulation sequences for data augmentation. The aforementioned technical solution constructs a generative model by combining temporal generative adversarial networks (GANs) and long short-term memory (LSTM) neural networks. This model can learn the temporal dependencies and spatial distribution relationships of traffic flow from limited real-world data through joint training. The resulting simulation sequences not only match the original data numerically, but more importantly, their data distribution patterns are highly similar to those of real-world data. This achieves high-fidelity, physically consistent creative filling of missing scenarios, expands the high-quality training samples, and provides underlying support for solving data sparsity issues.

[0019] In this embodiment, traffic flow spatiotemporal data refers to a multidimensional data sequence collected from multiple monitoring points (or traffic stations) in an urban road network and arranged in chronological order. It not only includes traffic flow data for each station at different times, such as vehicle volume, speed, and occupancy rate, but its spatiotemporal characteristics are further reflected in the time-dependent relationships between the data. For example, the current traffic flow is influenced by the previous flow, and the data from different stations are interconnected due to road network connectivity. The core objective of the generative model is to learn the joint probability distribution of this real data, thereby enabling the sampling of new data samples that are similar to but not identical to the real data—i.e., simulation sequences.

[0020] To achieve this goal, the model constructed in this application is a temporal model that deeply integrates the ideas of autoencoders and generative adversarial networks (GANs). The temporal GAN ​​framework ensures the overall authenticity of the generated data distribution through adversarial game between the generator and discriminator. The introduction of a Long Short-Term Memory (LSTM) neural network (LSN) endows the model with powerful temporal dependency modeling capabilities, ensuring that the generated sequences are reasonable and coherent in temporal dynamics. The autoencoder component consists of an embedding function (encoder) and a recovery function (decoder), whose role is to learn meaningful latent representations of the data. The GAN component includes a generator and a discriminator, responsible for generating data from random noise and judging the authenticity of the data. Joint training is key to the efficient learning of the model in this application. It means that the reconstruction objective of the autoencoder and the game objective of the GAN are simultaneously optimized, enabling the model to achieve the best balance between accurately reconstructing details and matching the overall distribution.

[0021] The training of the generative model in the above embodiments is not an isolated optimization of autoencoders or generative adversarial systems, but a carefully designed joint optimization process. Specifically, the joint training of the generative model can be achieved through steps S1a to S1d, as detailed below: Step S1a: Encode and decode the input traffic flow spatiotemporal data using the embedding and recovery functions in the autoencoding component, and calculate the recovery loss between the original data and the decoded data.

[0022] Step S1a ensures that the generative model has the ability to accurately reconstruct the input data, which is the foundation for learning the essential features of the data. The recovery loss measures the difference between the original input and the output after encoding and decoding. In one embodiment of this application, calculating the recovery loss between the original data and the decoded data includes: calculating the sum of the Euclidean distances between the original input data sequence and the data sequence decoded by the recovery function at all time steps through expectation operations. Its mathematical expression can be formalized as: for a data sequence from the true distribution... p Traffic flow sequence samples After embedding function e Encoded as a sequence of latent variables Then through the recovery function r Decoded as Then recover the loss. for .in, E Expressing expectations, express Norm (Euclidean distance) T The sequence length is given.

[0023] Step S1b: The generator and discriminator in the generative adversarial component play a game. The generator receives random noise and generates simulated data. The discriminator distinguishes whether the input data is real data or generated data and calculates the discrimination loss.

[0024] Step S1b is the core of the generative adversarial network. Generator G Receive random noise sampled from a prior distribution (e.g., a standard normal distribution). z It outputs a simulated traffic flow sequence. Discriminator D It acts as a binary classifier, attempting to distinguish whether the input sequence comes from the real data distribution. p of Or does it come from the generator distribution? of The discrimination loss is used to train the discriminator to make accurate judgments and simultaneously train the generator to "deceive" the discriminator. In one embodiment of this application, calculating the discrimination loss includes: calculating the expected log probability of the discriminator classifying data from the real data distribution as real data and the expected log probability of the discriminator classifying data from the generator data distribution as real data through expectation operations, and taking the negative of the sum of the two as the discrimination loss. This is typically expressed in the form of cross-entropy, which can be expressed as follows: When training the generator, the goal is to minimize... Or equivalently maximize .

[0025] Step S1c: Calculate the intermediate state loss between the intermediate states generated by the autoencoder component and the intermediate states generated by the generator.

[0026] The intermediate state loss introduced in step S1c is a key innovation of the generative model, connecting the autoencoder component and the generative adversarial component. The autoencoder component generates a sequence of latent variables during the encoding process. This is considered a meaningful representation of intermediate states in the data. To ensure that the generator's internal dynamics approximate the patterns of intermediate states in real data during data generation, the generator is constrained to ensure that the statistical distribution of one of its intermediate layer outputs (or its latent variables) is as close as possible to the corresponding intermediate states generated by the autoencoder. This can be achieved by calculating the distributional distance between the two (e.g., maximum mean difference (MMD) or adversarial loss). This loss function forces the generator not only to generate data that looks realistic but also to resemble real data in its data formation process or representation, thereby significantly improving the temporal consistency and structural realism of the generated data.

[0027] Step S1d: Iteratively update the network parameters of the generative model by jointly optimizing the recovery loss, discrimination loss, and intermediate state loss until the model converges.

[0028] Step S1d involves weighted summation of the three losses to form the total loss function: ,in, , and For hyperparameters, The intermediate state loss is used. Gradient descent algorithms (e.g., Adam) are used to simultaneously optimize all parameters of the generative model (including the embedding function, recovery function, generator, and discriminator parameters), iteratively updating until the total loss converges. This joint optimization mechanism enables the model to collaboratively learn accurate reconstruction of data, overall distribution matching, and intermediate representation alignment. The ultimately trained generator can overcome noise... z This generates high-quality simulated traffic flow sequences.

[0029] In step S1a, the encoding and decoding process is implemented through a long short-term memory neural network unit. Specifically, at each time step, the traffic flow spatiotemporal data of the current moment is concatenated with the hidden state of the previous time step, and the flow and filtering of information are controlled through an input gate, a forget gate, and an output gate to capture the temporal dependencies of the traffic flow data.

[0030] LSTM units are ideal for handling time-dependent problems. The embedding and recovery functions are essentially recurrent neural networks composed of multiple stacked LSTM layers. For the encoding process (embedding function), at time step... t The LSTM unit receives the current input. The hidden state of the previous time step and memory cell state Input gate Control how much new information (from the current input) is generated. and The transformation is stored in the memory unit; the forget gate Control the state of the previous memory cell How much has been forgotten; Output Gate Control the current memory cell state How many were output to the current hidden state? Through the coordinated operation of these three gating mechanisms, LSTM can selectively remember long-term important patterns (e.g., daily peak periods) and forget short-term irrelevant fluctuations, thereby effectively capturing the temporal dependencies of traffic flow data. The encoder ultimately outputs a sequence of latent variables containing information about the entire sequence. Or a summarized context vector. The decoder (reconstruction function), conditioned on the latent variables, generates the reconstructed sequence in reverse through another LSTM network. In this way, the model can gain a deep understanding of the causal relationships and long-term trends when processing traffic flow data, which is the basis for generating realistic simulation sequences.

[0031] Step S2: Based on the general temporal difference feature construction method, the original traffic flow data and simulation sequence are subjected to quality enhancement processing to construct an enhanced dataset containing the original features and newly added statistical features. The quality enhancement processing of the original traffic flow data and simulation sequence includes calculating the difference components of the time series features, grouping the features according to the difference components, and calculating the statistics of each group.

[0032] After step S1, the original data and its simulation extensions are obtained. However, the features directly provided by either the original or simulated data (e.g., raw flow values) may not fully reveal the dynamic change patterns within the data. To further enhance the data's performance for downstream prediction tasks, feature enhancement is needed.

[0033] In this embodiment, the original traffic flow data refers to the initial dataset collected from the real world. The general temporal difference feature construction method is a method designed to automatically extract more discriminative features from raw time-series data. Its core idea is that the instantaneous rate of change of traffic flow (i.e., the difference between adjacent time points) often reflects key events such as sudden changes in traffic conditions, the formation and dissipation of congestion, more effectively than absolute flow values. The difference component is precisely the quantification of this instantaneous change. Then, features with similar change patterns (i.e., similar difference component magnitudes) are grouped, and statistics are calculated within each group, thereby aggregating the instantaneous, high-dimensional difference information into stable, low-dimensional statistical features. These statistical features, together with the original features, constitute a more informative and expressive enhanced dataset.

[0034] Calculating the difference component of the time-series features is the first step of this method. In one embodiment of this application, calculating the difference component of the time-series features includes: for any feature, calculating the difference between its feature value at the current time and its feature value at the adjacent previous time, to characterize the instantaneous change trend of the feature. Taking a specific traffic flow feature as an example... For example, among which, j Indicates the first j Features (e.g., traffic to a specific site). t Indicates time, d Indicates the first d The corresponding feature difference components. The calculation formula is: This simple operation can transform a stable flow sequence into a changing sequence that reflects acceleration or deceleration, which is crucial for predicting future flow trends.

[0035] After calculating the differences of all features at all time points, a difference matrix is ​​obtained. Then, based on the feature difference matrix... Grouping features. An effective grouping strategy is based on feature differences. The time points are divided into ranges based on their size. For example, several threshold intervals can be set, and feature-time point pairs whose feature differences fall within the same interval can be grouped together. In this way, the features within each group have similar intensity of change at a specific time (e.g., they all increase slightly, decrease significantly, or remain basically unchanged). The purpose of grouping is to extract more statistically significant macroscopic patterns from a large number of fine-grained difference points.

[0036] After grouping, the statistics for each group are calculated. These statistics aim to summarize the distribution characteristics of the differences within each group and use them as new features. Step S2 involves calculating the statistics for each group, including: Step S2a: Calculate the mean of the features within the same difference group, and use it as the group mean feature.

[0037] The group mean characteristic reflects the overall average level of a specific pattern of change intensity represented by that group, and its calculation formula is: ,in, To follow the characteristic difference components The original feature data is grouped according to its size, resulting in differential grouped data.

[0038] Step S2b: Calculate the standard deviation of the feature within the same difference group, and use it as the group standard deviation feature.

[0039] Grouped standard deviation characteristics The standard deviation measures the consistency of variation within a group. A small standard deviation indicates that the variation amplitudes of all characteristics within the group are very similar, possibly corresponding to a systematic variation pattern; a large standard deviation indicates large differences in variation amplitudes and an unstable pattern. The formula for calculating the grouped standard deviation characteristic can be expressed as follows: .

[0040] Step S2c: Calculate the maximum value of the feature within the same difference group, and use it as the maximum value feature of the group.

[0041] Group maximum value feature .

[0042] Step S2d: Calculate the minimum value of the feature within the same difference group, and use it as the minimum feature of the group.

[0043] Group minimum value feature .

[0044] The grouped maximum / minimum features capture extreme cases within this variation pattern, which is invaluable for identifying anomalous fluctuations or peak events.

[0045] Step S2e: Combine the group mean feature, group standard deviation feature, group maximum value feature, and group minimum value feature to form a new statistical feature.

[0046] For each time point t Each feature is grouped based on its differences, and the four statistics for each group are calculated. Therefore, for a given feature... J From the original dataset with 1000 original features at each time point, this step will add 4*K new features (K is the number of groups). Ultimately, the augmented dataset will consist of the original dataset with 10000 features at each time point. J The 4K-dimensional feature vector is concatenated with the newly added 4K-dimensional statistical feature vector, and its dimension is 1. J +4K. This method not only significantly expands the feature space, but more importantly, it transforms implicit, high-dimensional temporal dynamic information into explicit, stable, and easily understood static features by machine learning models through differencing and statistical aggregation, thereby achieving a deep enhancement of data quality. This enhanced dataset provides a more informative foundation for subsequent in-depth analysis.

[0047] Step S3: Based on the local weighted smoothing method, the traffic flow data in the enhanced dataset is decomposed into trend components, periodic components, and residual components through an internal iterative process that includes detrending and deperiodicization. The trend components and residual components are then reconstructed into fluctuation components that characterize non-periodic changes, so that the periodic components and fluctuation components can be feature-analyzed and evaluated separately.

[0048] After enhancement in step S2, a more informative traffic flow dataset is obtained. However, traffic flow data is not simply random fluctuation; it contains multiple time-scale patterns: long-term growth or decline trends, fixed periodic patterns (e.g., daily or weekly cycles), and unpredictable random disturbances. Separating these mixed patterns helps the model learn the patterns of different components more effectively, thereby improving prediction accuracy. Traditional decomposition methods, such as classic seasonal-trend decomposition, usually require pre-setting the period length and are poorly adapted to non-stationary and non-linear traffic flow sequences. The decomposed components may not be pure enough, and there may still be residual elements between them.

[0049] To address the aforementioned issues, this application employs the following technical solution: Based on a local weighted smoothing method, the traffic flow data in the enhanced dataset is decomposed into trend components, periodic components, and residual components through an internal iterative process that includes detrending and deperiodicization. The trend components and residual components are then reconstructed into fluctuation components representing non-periodic changes, enabling the periodic and fluctuation components to be analyzed and evaluated separately. This technical solution utilizes an iterative decomposition process based on a local weighted smoothing method, which adaptively and accurately separates traffic flow data into trend, periodic, and residual components. Reconstructing the trend and residual components into fluctuation components means merging long-term slow changes with unpredictable random disturbances to jointly represent non-periodic fluctuations, thus creating a stark contrast with the regular periodic components. This decomposition and reconstruction mechanism allows the periodic and fluctuation components to be analyzed and evaluated separately, significantly reducing confusion between different patterns and laying a structural foundation for improving model accuracy and interpretability.

[0050] It should be noted that in the above embodiments, the local weighted smoothing method is a non-parametric regression method. It obtains a smooth curve by performing weighted least squares fitting on the points in the neighborhood of each data point. Its advantage is that it does not require a preset function form and can adaptively capture local changes in the data. Trend components refer to long-term, slowly changing components in traffic flow data, such as the slow annual increase in traffic volume due to urban development, or long-term traffic suppression caused by major construction. Periodic components refer to components with fixed repetitive patterns, most typically daily variation patterns with a 24-hour cycle (e.g., morning and evening peak hours, off-peak hours, and nighttime troughs) and weekly variation patterns with a 7-day cycle (differences between weekdays and weekends). Residual components are the seemingly irregular fluctuations remaining after removing trends and periods, usually including random noise, the impact of sudden events (e.g., accidents, sudden weather changes, etc.). An innovative operation of this application is to reconstruct the trend components and residual components into fluctuation components. This is because, in traffic flow forecasting, the slowly changing long-term trends and unpredictable random disturbances constitute non-periodic fluctuations or background noise that need to be processed for short-term forecasting models. Combining them into a fluctuation component can create a clearer contrast with the highly regular periodic component, making it easier to perform feature mining separately in the future.

[0051] To achieve high-precision decomposition, this application employs a sophisticated method incorporating internal iteration and external processes. Specifically, based on local weighted smoothing, the traffic flow data in the enhanced dataset is decomposed into trend components, periodic components, and residual components through an internal iterative process that includes detrending and deperiodicization. The internal iterative process includes the following steps S3a to S3e as an example: Step S3a: Initialize trend component data.

[0052] Before the iteration begins, the trend components need to be initialized. Typically, the trend components can be initialized to a zero sequence, i.e. The superscript indicates the iteration number. This means that in the first iteration, the traffic flow data in the enhanced dataset is considered to consist primarily of cycles and residuals.

[0053] Step S3b: Perform detrending processing and smooth the detrended subsequence using local weighted smoothing to obtain the initial periodic subsequence.

[0054] Subtract the currently estimated trend from the traffic flow data to be decomposed. To obtain detrended subsequences Then, locally weighted smoothing is applied to this detrended subsequence. The purpose of smoothing is to filter out high-frequency random noise and anomalous spikes while preserving the general periodic shape, thereby initially extracting the periodic contour. The smoothed sequence is the initial periodic subsequence for the current iteration. .

[0055] Step S3c: Perform low-pass filtering and local weighted smoothing on the initial periodic subsequence to obtain the trend subsequence, and subtract the trend subsequence from the initial periodic subsequence to obtain the periodic composition of the current iteration.

[0056] Initial periodic subsequence There may still be some very slow variations remaining (e.g., slow variations in the period amplitude itself). To obtain a purer periodic signal, it is necessary to further extract its trend. Here, a low-pass filter (e.g., moving average, or again using a wider-bandwidth locally weighted smoothing) is applied to the initial periodic subsequence. Processing is performed to obtain its trend subsequence. Then, this trend is subtracted from the initial periodic subsequence to obtain the refined periodic composition. This step ensures the final obtained periodic composition. It is a purely periodic signal that fluctuates around the value of zero.

[0057] Step S3d: Perform deperiodicization and smooth the deperiodic data using local weighted smoothing to obtain the updated trend composition.

[0058] Subtract the current cycle composition from the traffic flow data to be decomposed. To obtain the deperiodic subsequence This subsequence primarily contains the trend and residuals. Then, for... The application of locally weighted smoothing aims to extract its long-term trend, and the smoothed result is the updated trend composition. .

[0059] Step S3e: Repeat steps S3b to S3d until the convergence condition is met, and output the final trend composition and cycle composition.

[0060] The steps S3b to S3d described above constitute one complete iteration. Each iteration updates the periodic composition. Composition of trends The estimation is performed. Iteration continues until the change in trend and periodic components between two consecutive iterations is less than a preset threshold, or the maximum number of iterations is reached. At this point, the decomposition is considered converged, and the final trend components (i.e., the long-term trend) are output. and periodic composition .

[0061] After the internal iteration converges, the residuals need to be calculated through an external process. Following step S3, the process also includes: Step S3f: Extracting the final trend component and final cycle component output by the internal iteration process from the traffic flow data in the enhanced dataset, and calculating the residual components.

[0062] This step is the final step in the decomposition process. According to the additive model, the original data equals the sum of the trend, period, and residuals: x = t + s + r Therefore, the residual composition r It is possible This is calculated directly. The residual consists of... r It contains all the information that is not explained by trend and cycle models, mainly the effects of random noise and sudden events.

[0063] After obtaining the three components of trend, cycle, and residual, this application performs a key reconstruction operation, namely, reconstructing the trend component and residual component into a fluctuation component can be achieved by aggregating and adding the final trend component and residual component to generate the fluctuation component.

[0064] That is, the composition of fluctuations The deeper significance of this restructuring lies in the fact that for many machine learning models, especially those focused on short-term predictions, long-term trends... Changes are slow within a short window and can be considered a bias or baseline, while the residuals constitute... r This represents the instantaneous fluctuations around the baseline. Combining the two into a single fluctuation component... This forms a comprehensive signal characterizing non-periodic changes. Thus, the original complex traffic flow sequence has been clearly decoupled into two more model-friendly signals: a highly regular and predictable periodic component and a fluctuating component containing trends and random disturbances. This decomposition and reconstruction provides a more reliable data foundation for subsequent refined feature evaluation and selection.

[0065] Step S4: Based on the extreme gradient boosting algorithm and using the feature splitting criterion based on the "gain" mechanism, the feature importance of the periodic and volatile components obtained in Step S3 is evaluated. By calculating the sum of the squared error risk estimation improvement brought by each feature when splitting at all boosting decision tree nodes, the feature importance evaluation index directly related to the performance improvement of the prediction model is obtained.

[0066] After step S3, two sets of data are obtained: the periodic composition and the fluctuation composition. Each set of data contains a large number of features (original features and newly added statistical features obtained in step S2). However, not all features are equally important to the prediction target (e.g., the flow rate at the next moment). Many features may be redundant, irrelevant, or even noisy. Traditional feature selection methods, such as filtering (based on correlation coefficients) or wrapping (based on the performance iteration of a specific model), are problematic. The former cannot consider the interaction effects between features, while the latter has extremely high computational costs and the results depend on the selected base model.

[0067] To efficiently and accurately extract core variables from high-dimensional spatiotemporal features, this application introduces an advanced machine learning algorithm: based on the extreme gradient boosting algorithm and employing a feature splitting criterion based on the "gain" mechanism, it evaluates the feature importance of the periodic and volatile components obtained in step S3. By calculating the sum of the squared error risk estimates improved by each feature during splitting at all decision tree nodes, a feature importance evaluation index directly related to the performance improvement of the prediction model is obtained. This scheme, by applying the extreme gradient boosting algorithm and explicitly adopting a feature splitting criterion based on the "gain" mechanism, achieves a feature evaluation method intrinsically coupled with high-performance prediction models. The "gain" value essentially measures the total reduction in squared error loss for the entire model due to using a certain feature for node splitting. This index has the following advantages: 1) Objective data-driven: entirely determined by the model's performance on training data, requiring no prior knowledge; 2) Captures complex relationships: the ensemble tree model can automatically evaluate the nonlinear and interaction effects of features; 3) Targeted and precise: its evaluation goal is to directly improve prediction accuracy, highly consistent with the final goal. Therefore, step S4 provides a reliable quantitative tool for uncovering deep physical correlations from the composition of cycles and fluctuations, and serves as the scientific basis for subsequent precise feature selection.

[0068] In this embodiment, the Limit Gradient Boosting algorithm is an efficient implementation of gradient boosting decision trees. It constructs multiple decision trees sequentially, with each new tree learning to correct the prediction residuals of the previous tree set, thus forming a powerful ensemble model. It inherently possesses the ability to evaluate feature importance. The feature splitting criterion based on the "gain" mechanism is the standard used in algorithms such as XGBoost to determine how each node in the decision tree selects splitting features and splitting points. Gain intuitively represents how much reduction in the overall model's prediction error (usually expressed as squared error) can be achieved by using a certain feature and its specific splitting point to split a node. This reduction is the improvement in squared error risk estimation. By accumulating the improvement brought by a feature across all trees and all splitting nodes, a global, quantitative feature importance evaluation index can be obtained, which directly measures the contribution of the feature to improving the model's prediction accuracy.

[0069] The evaluation process of the feature importance evaluation index is a systematic calculation process. Specifically, as an embodiment of this application, the feature importance evaluation index directly related to the performance improvement of the prediction model is obtained by calculating the sum of the squared error risk estimation improvement brought by each feature when splitting at all decision tree nodes. This can be achieved through steps S4a to S4ac, as detailed below: Step S4a: For the input periodic or fluctuation data, construct multiple boosting decision trees, each consisting of multiple internal nodes.

[0070] Step S4a trains an XGBoost model independently for both the periodic composition dataset and the fluctuation composition dataset. Each XGBoost model consists of dozens to hundreds of boosting decision trees. Each tree starts from the root node and splits by continuously selecting features, growing into multiple internal nodes (non-leaf nodes) and leaf nodes. The internal nodes are where feature judgment and path selection are performed, and it is the splitting behavior of these nodes that contains information about the importance of features.

[0071] Step S4b: When constructing any single boosting decision tree, when determining the optimal split for each internal node, traverse all candidate features and candidate split points, and select the feature and split point that maximizes the improvement in squared error risk estimation for node splitting.

[0072] This is one of the core steps in training the XGBoost model. When growing to a node in the tree, the algorithm "traverses all candidate features and candidate split points." For each candidate (feature, split point) pair, the algorithm calculates how much the split would reduce the overall loss function (e.g., squared error) of the model by dividing the current node's data into two child nodes. This reduction is the "improvement in squared error risk estimation," and its calculation typically involves the change in the label variance (or first- or second-order gradient statistics) of samples within the node before and after the split. The algorithm selects the feature and split point that maximizes this improvement as the final split for that node. This means that the selected feature provides the largest reduction in impurity at the current node, i.e., contributes the most to distinguishing samples and approximating the target value.

[0073] Step S4c: For any feature, accumulate the improvement in squared error risk estimation brought by it at each internal node of all boosting decision trees, and obtain the total cumulative improvement. The total cumulative improvement is the feature importance evaluation index of that feature.

[0074] After the model training is complete, the importance of each feature is statistically analyzed. For each feature... i This is done by scanning every internal node of the boosting decision tree. If a node uses a feature... i If a split is performed, the improvement resulting from this split is recorded. Then, sum all such improvements. This cumulative sum is the feature. i The global importance score is calculated. If a feature is frequently used for splitting, and each split brings a significant reduction in error, its cumulative improvement will be high, indicating that it is a strongly correlated core feature. In this way, an objective and comparable feature importance evaluation index is calculated for each feature in the periodic and oscillatory components. This index is rooted in the improvement of model performance and has a clear and convincing physical meaning.

[0075] Step S5: Based on the feature importance evaluation index, all features constituting the periodic and volatile components are mixed and sorted, and then filtered based on the dynamic correlation threshold to remove redundant features and extract the core feature subset.

[0076] As an embodiment of this application, based on the feature importance evaluation index, all features constituting the periodic and fluctuating components are mixed and sorted, and then filtered based on a dynamic correlation threshold to remove redundant features and extract the core feature subset. This can be achieved through steps S5a to S5c, as detailed below: Step S5a: Based on the feature importance evaluation index, perform a mixed ranking of all features that constitute the periodic and volatile components.

[0077] The importance scores of all features (including original features and statistically enhanced features from step S2) in the periodic and variability components are combined and ranked. The ranking is from highest to lowest, with the feature having the highest importance score listed first. This ranking provides a panoramic view of the relative contribution of all features to predicting traffic flow (or its periodic / variability components).

[0078] Step S5b: Set the dynamic association threshold.

[0079] To select core features from the ranking list, a screening threshold needs to be set. This application uses a "dynamic association threshold" instead of a fixed value. A common dynamic setting method is to select the cumulative contribution curve of importance scores (i.e., the proportion of the sum of the importance of the top N features to the total importance), and the position corresponding to when the cumulative contribution rate reaches a preset value (e.g., 95%) is the threshold. Another method is based on the distribution of importance scores, selecting the position where a clear inflection point (a sharp drop in importance) appears in the ranking. The dynamic threshold can adapt to the feature importance distribution of different datasets.

[0080] Step S5c: Select features whose ranking is higher than the dynamic association threshold as the core feature subset.

[0081] Based on the dynamic threshold set in step S5b, all features with importance scores higher than this threshold are retained. These features are considered core variables significantly correlated with traffic flow changes. Features below the threshold are considered redundant or low-correlation features and are removed. Ultimately, a core feature subset with significantly reduced dimensionality but extremely high information density and predictive relevance is obtained. This core feature subset integrates the essential information retained after data augmentation, pattern decomposition, and rigorous evaluation, providing optimal input data for any downstream traffic flow modeling task requiring high accuracy and efficiency (e.g., short-term prediction, congestion identification).

[0082] Through steps S1 to S5, a high-quality, low-redundancy core feature subset was successfully extracted from the original multi-source, high-dimensional, and noisy traffic flow data. This subset is the core output of the method in this application, and its value is ultimately reflected in empowering downstream intelligent transportation and energy management tasks. Traditional prediction models usually use raw or simply processed data directly, resulting in a large feature space containing a lot of noise, leading to low model training efficiency, poor generalization ability, and difficulty in directly converting prediction results into actionable decisions.

[0083] To address the aforementioned issues, this application constructs a complete technological closed loop from data preprocessing to intelligent decision-making. Following step S5, Figure 1 The example method may also include inputting a subset of core features into a downstream traffic flow prediction model or an electric vehicle charging load prediction model to generate traffic control instructions or charging scheduling strategies. Here, the downstream traffic flow prediction model can be any machine learning or deep learning model suitable for time-series forecasting, such as a specialized prediction network built based on the LSTM or Transformer architecture used in step S1. The electric vehicle charging load prediction model, based on traffic flow prediction, further incorporates charging behavior models, battery characteristics, charging pile information, etc., to predict the future charging load of a specific area. The subset of core features, as input to these models, has undergone data augmentation, pattern decomposition, and rigorous screening, enabling downstream models to focus more on learning true causal relationships, thereby significantly improving prediction accuracy and stability. Based on the high-precision prediction results, the system can automatically generate traffic control instructions (e.g., optimizing traffic light timing) or charging scheduling strategies (e.g., guiding vehicles to idle charging stations, adjusting charging power), ultimately achieving coordinated optimization of the traffic system and the power system.

[0084] Before delving into any of the processing steps, a crucial prerequisite is obtaining high-quality, comprehensive raw data. Prior to step S1, this includes acquiring multi-source fused data containing historical traffic flow sequences from multiple transportation stations, geographic location information, time information, and weather index data, as the raw traffic flow data.

[0085] Multi-source fusion data is the foundation of the method in this application, and its diversity and completeness directly determine the quality of the final feature subset. Traditional methods may rely on only a single data source (such as loop detector traffic flow), ignoring the profound impact of the external environment on traffic flow. This application systematically integrates the following key data sources: 1. Historical traffic flow sequences from multiple transportation hubs: Time-series data continuously collected from detection devices (e.g., inductive loops, cameras, radar, etc.) at key nodes (e.g., intersections, road segments) of the urban road network, including traffic volume, average vehicle speed, and time occupancy. This data constitutes the core observation of traffic conditions.

[0086] 2. Geographic location information: This includes the latitude and longitude coordinates of transportation stations, the road class (e.g., expressway, arterial road, secondary arterial road), the number of lanes, and upstream and downstream topological connections. Spatial information is fundamental to understanding traffic flow propagation and correlation.

[0087] 3. Time Information: This includes not only the timestamps of the recorded data, but also rich temporal characteristics derived from them, such as "time of day" (the minute of the day, used to capture intraday cycles), "day of the week" (used to capture weekly cycles), and whether it is a holiday or a special event day. These are key bases for decomposing the components of the cycle.

[0088] 4. Weather data: including temperature, precipitation, wind speed, visibility, etc. Weather conditions have a significant impact on driving behavior and road capacity, and are an important external factor causing non-periodic fluctuations in traffic flow.

[0089] After aligning these heterogeneous data sources (time synchronization, spatial correlation) and performing preliminary cleaning (handling missing values ​​and obvious errors), the original traffic flow data of this application is formed. This comprehensive and high-quality data foundation ensures that all subsequent advanced processing steps can uncover deep and robust traffic flow patterns.

[0090] In the field of intelligent transportation systems, high-precision short-term traffic flow prediction is the core of implementing proactive traffic management. In the above embodiment, the generation of traffic control instructions can be as follows: inputting a subset of core features into a spatiotemporal prediction model to predict the electric vehicle traffic flow at a specific time period and station in the future, and optimizing the timing scheme of traffic lights in the corresponding area or adjusting the display content of variable message signs based on the predicted traffic flow. Specifically, the spatiotemporal prediction model (e.g., a model combining a graph neural network and LSTM) uses a subset of core features as input. This subset already contains key spatiotemporal information and external factors that best reflect the evolution of traffic flow. The model learns the complex mapping relationship between these features and traffic flow over a future time period (e.g., the next 15 minutes, 30 minutes, or 1 hour) to output predicted values ​​for the future traffic flow at each station in the road network. Compared to using all features, the prediction model trained using a subset of core features has faster inference speed, lower process risk, and higher prediction accuracy.

[0091] Once reliable traffic flow forecasts are obtained, they can be translated into specific control actions. In the above embodiments, the generation of traffic control instructions also includes: optimizing the timing scheme of traffic lights in the corresponding area or adjusting the display content of variable message signs based on the predicted traffic flow. This is the final manifestation of the closed loop of data value. For example: optimizing traffic light timing: when a forecast indicates that congestion is about to occur downstream of a certain intersection, the traffic control center can dynamically adjust the green light duration and phase difference of the traffic lights at that intersection and even a series of upstream intersections to balance and guide traffic flow in advance, preventing congestion from occurring or worsening. "Optimizing the timing scheme of traffic lights in the corresponding area" is a specific and executable traffic control instruction generated based on the forecast results. Adjusting variable message signs: when an accident risk or severe congestion is predicted on the road ahead, real-time traffic information, suggested detour routes, or safety tips can be released through the "variable message signs" on the roadside to guide drivers to choose routes in advance and balance the road network load. "Adjusting the display content of variable message signs" is another important information-guided control instruction. By following the chain of "core feature subset → high-precision prediction → real-time control command", this application closely integrates the pre-processing technology with actual traffic management, and realizes accurate perception, advanced prediction and proactive intervention of urban traffic flow.

[0092] Another important application of this application is supporting the coordinated interaction between electric vehicles (EVs) and the power grid. EV charging load exhibits significant spatiotemporal randomness and aggregation effects, making accurate prediction crucial for the safe and economical operation of the power grid. The application of the core feature subset also includes using it for training or as input to predict the spatiotemporal distribution of EV charging load, and generating charging load guidance strategies based on the prediction results to optimize charging facility operation or grid coordinated scheduling. In the embodiments of this application, predicting the spatiotemporal distribution of EV charging load refers to predicting the total charging power demand of EV clusters in different time periods and different areas of a city in the future. Achieving this prediction typically requires a two-stage model: the first stage uses the core feature subset (especially features reflecting traffic flow, destination characteristics, and dwell time) to predict the travel trajectory and parking charging demand of EVs; the second stage combines a charging behavior model to convert the charging demand into a load curve on the grid side. Since the core feature subset already contains key factors affecting travel and charging, it can significantly improve the accuracy of load prediction.

[0093] Based on high-precision spatiotemporal distribution prediction of charging load, various optimization strategies can be generated, including optimizing the operation of charging facilities and grid coordinated scheduling, as detailed below: 1) Optimize the operation of charging facilities: guide vehicles to charging stations with high vacancy rates to avoid some stations being overloaded while others are idle; during peak grid load periods, dynamically adjust the output power of charging piles or implement flexible charging to achieve peak shaving and valley filling.

[0094] 2) Grid Coordinated Dispatch: Grid dispatching departments can treat predicted charging loads as important dispatchable resources and incorporate them into the power system's short-term or ultra-short-term dispatching plans. When renewable energy (such as wind power and photovoltaic) output is high, charging is encouraged to absorb green electricity; when system reserves are insufficient, charging loads can be appropriately reduced through price signals or direct control strategies to provide ancillary services to the grid.

[0095] Generating charging load guidance strategies for optimizing charging facility operation or grid-based coordinated dispatch is the value proposition of this application in the energy sector. It seamlessly integrates the results of intelligent transportation data processing into the operation and planning of new power systems, providing robust data-driven decision support for promoting the deep integration and coordinated development of "vehicle-road-grid".

[0096] From the above Figure 1The example of traffic flow data processing and feature selection methods demonstrates that, on the one hand, by constructing a generative model based on temporal generative adversarial networks and long short-term memory neural networks, and jointly training the original data, the generated simulation sequences not only effectively fill in the missing scenarios in the original data but also possess high fidelity due to their adherence to the real data distribution. This significantly expands the high-quality and diverse sample library required for downstream model training, laying a solid data foundation for improving the model's generalization ability and robustness. Furthermore, by employing a general temporal difference feature construction method for quality enhancement, not only are the original features preserved but also new statistical features reflecting data change trends are innovatively constructed from the perspective of temporal difference components. This results in an enhanced dataset with richer information and stronger discriminative power, enabling subsequent models to capture traffic flow evolution patterns from more dimensions. On the other hand, by decomposing the enhanced traffic flow data into trend, periodic, and residual components based on local weighted smoothing, and reconstructing the trend and residual into fluctuation components, the mixed original signals are separated into components with different physical meanings. This decomposition allows subsequent analysis to target relatively stable and regular periods separately. The first aspect involves evaluating the components and characteristics of short-term fluctuations and anomalous events. This helps the model learn the different scale patterns of traffic flow more clearly and attentively, avoiding mutual interference between different patterns. Secondly, by using a limit gradient boosting algorithm and a feature splitting criterion based on a "gain" mechanism, features in both the periodic and fluctuating components are evaluated. The sum of the squared error risk estimation improvements brought by each feature when splitting at all decision tree nodes is accurately calculated. This metric provides a quantifiable, data-driven importance score for each feature from the perspective of its contribution to model performance improvement, thus overcoming the blindness and subjectivity of manual experience-based selection and making the feature selection process more data-driven. Thirdly, by quantitatively ranking and filtering all features based on the above quantitative evaluation metrics, redundant features with low contribution to the prediction target can be automatically and efficiently eliminated. The final extracted core feature subset consists of high-impact feature factors selected through screening. Inputting this concise and powerful feature subset into the downstream prediction model can effectively reduce model complexity, reduce overfitting risk, accelerate model training, and ultimately significantly improve the accuracy and reliability of traffic flow prediction or charging load prediction. In summary, the technical solution of this application provides a complete process from data generation, enhancement, decomposition to evaluation and screening, which can automatically generate high-quality core feature subsets for traffic flow prediction models.

[0097] Please see the appendix Figure 2 This application provides a method for processing traffic flow data and a feature selection device. The device includes a training module 201, a construction module 202, a reconstruction module 203, an evaluation module 204, and an extraction module 205, which are described in detail below: Training module 201 is used to build a generative model based on temporal generative adversarial network and long short-term memory neural network. It uses the autoencoder component in the generative model to encode and decode the input traffic flow spatiotemporal data, and uses the generative adversarial component to generate and identify the data. It learns the data distribution pattern in a joint training manner and generates simulation sequences for data augmentation. The construction module 202 is used to perform quality enhancement processing on the original traffic flow data and simulation sequence based on the general time difference feature construction method, so as to construct an enhanced dataset containing the original features and the newly added statistical features. The quality enhancement processing of the original traffic flow data and the simulation sequence includes calculating the difference components of the time series features, grouping the features according to the difference components, and calculating the statistics of each group. The reconstruction module 203 is used to decompose the traffic flow data in the enhanced dataset into trend components, periodic components and residual components through an internal iterative process that includes detrending and deperiodicization based on the local weighted smoothing method. The trend components and residual components are then reconstructed into fluctuation components that characterize non-periodic changes, so that the periodic components and fluctuation components can be feature-analyzed and evaluated separately. Evaluation module 204 is used to evaluate the feature importance of periodic components and fluctuation components based on the limit gradient boosting algorithm and the feature splitting criterion based on the "gain" mechanism. By calculating the sum of the squared error risk estimation improvement brought by each feature when splitting at all boosting decision tree nodes, the feature importance evaluation index directly related to the performance improvement of the prediction model is obtained. The extraction module 205 is used to perform mixed sorting of all features constituting the periodic and volatile components based on the feature importance evaluation index, and to filter based on the dynamic correlation threshold in order to remove redundant features and extract the core feature subset.

[0098] From the above Figure 2As can be seen from the example of traffic flow data processing methods and feature selection devices, on the one hand, by constructing a generative model based on temporal generative adversarial networks and long short-term memory neural networks, and jointly training the original data, the generated simulation sequences can not only effectively fill the missing scenarios in the original data, but also have high fidelity because they follow the distribution of real data. This significantly expands the high-quality and diverse sample library required for downstream model training, laying a solid data foundation for improving the generalization ability and robustness of the model. Furthermore, by using a general temporal difference feature construction method for quality enhancement, not only are the original features preserved, but new statistical features reflecting the data change trend are also innovatively constructed from the perspective of temporal difference components. This results in an enhanced dataset with richer information and stronger discriminative power, enabling subsequent models to capture the evolution pattern of traffic flow from more dimensions. On the other hand, by decomposing the enhanced traffic flow data into trend, periodic, and residual components based on the local weighted smoothing method, and reconstructing the trend and residual into fluctuation components, the mixed original signals are separated into components with different physical meanings. This decomposition allows subsequent analysis to target relatively stable and regular periods separately. The first aspect involves evaluating the components and characteristics of short-term fluctuations and anomalous events. This helps the model learn the different scale patterns of traffic flow more clearly and attentively, avoiding mutual interference between different patterns. Secondly, by using a limit gradient boosting algorithm and a feature splitting criterion based on a "gain" mechanism, features in both the periodic and fluctuating components are evaluated. The sum of the squared error risk estimation improvements brought by each feature when splitting at all decision tree nodes is accurately calculated. This metric provides a quantifiable, data-driven importance score for each feature from the perspective of its contribution to model performance improvement, thus overcoming the blindness and subjectivity of manual experience-based selection and making the feature selection process more data-driven. Thirdly, by quantitatively ranking and filtering all features based on the above quantitative evaluation metrics, redundant features with low contribution to the prediction target can be automatically and efficiently eliminated. The final extracted core feature subset consists of high-impact feature factors selected through screening. Inputting this concise and powerful feature subset into the downstream prediction model can effectively reduce model complexity, reduce overfitting risk, accelerate model training, and ultimately significantly improve the accuracy and reliability of traffic flow prediction or charging load prediction. In summary, the technical solution of this application provides a complete process from data generation, enhancement, decomposition to evaluation and screening, which can automatically generate high-quality core feature subsets for traffic flow prediction models.

[0099] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. For example... Figure 3As shown, the electronic device 3 in this embodiment mainly includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30, such as a program for a traffic flow data processing method and a feature selection method. When the processor 30 executes the computer program 32, it implements the steps in the above-described traffic flow data processing method and feature selection method embodiments, for example... Figure 1 Steps S1 to S5 are shown. Alternatively, when processor 30 executes computer program 32, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 2 The functions of the training module 201, construction module 202, reconstruction module 203, evaluation module 204, and extraction module 205 are shown.

[0100] For example, the computer program 32 for the traffic flow data processing method and feature selection method mainly includes: constructing a generative model based on temporal generative adversarial networks and long short-term memory neural networks; using the autoencoding component in the generative model to encode and decode the input traffic flow spatiotemporal data; generating and identifying data through generative adversarial components; learning the data distribution pattern through joint training; and generating simulation sequences for data augmentation; performing quality enhancement processing on the original traffic flow data and simulation sequences based on a general temporal difference feature construction method to construct an enhanced dataset containing original features and newly added statistical features. The quality enhancement processing of the original traffic flow data and simulation sequences includes calculating the difference components of the temporal features, grouping the features according to the difference components, and calculating the statistics of each group; and using a local weighted smoothing method to process the enhanced dataset... Traffic flow data is decomposed into trend components, periodic components, and residual components through an internal iterative process that includes detrending and deperiodicization. The trend components and residual components are then reconstructed into fluctuation components that represent non-periodic changes, allowing the periodic and fluctuation components to be analyzed and evaluated separately. Based on the extreme gradient boosting algorithm and using a feature splitting criterion based on a "gain" mechanism, the feature importance of the periodic and fluctuation components is evaluated separately. By calculating the sum of the squared error risk estimation improvement brought by each feature when splitting at all boosting decision tree nodes, a feature importance evaluation index directly related to the performance improvement of the prediction model is obtained. Based on the feature importance evaluation index, all features constituting the periodic and fluctuation components are mixed and sorted, and filtered based on a dynamic association threshold to remove redundant features and extract a subset of core features. The computer program 32 can be divided into one or more modules / units, which are stored in memory 31 and executed by processor 30 to complete this application. One or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 32 in the electronic device 3.For example, computer program 32 can be divided into the functions of training module 201, construction module 202, reconstruction module 203, evaluation module 204, and extraction module 205 (a module in the virtual device). The specific functions of each module are as follows: Training module 201 is used to construct a generative model based on temporal generative adversarial network and long short-term memory neural network. It uses the autoencoder component in the generative model to encode and decode the input traffic flow spatiotemporal data, and uses the generative adversarial component to generate and identify data. It learns the data distribution pattern in a joint training manner and generates simulation sequences for data augmentation. Construction module 202 is used to perform quality enhancement processing on the original traffic flow data and simulation sequences based on the general temporal difference feature construction method to construct an enhanced dataset containing original features and newly added statistical features. The quality enhancement processing of the original traffic flow data and the simulation sequences includes calculating the difference components of temporal features, grouping the features according to the difference components, and calculating the statistics of each group. The reconstruction module 203 is used to decompose the traffic flow data in the enhanced dataset into trend components, periodic components, and residual components through an internal iterative process that includes de-trending and de-periodicization, based on the local weighted smoothing method. The trend components and residual components are then reconstructed into fluctuation components that represent non-periodic changes, so that the periodic components and fluctuation components can be feature-analyzed and evaluated separately. The evaluation module 204 is used to evaluate the feature importance of the periodic components and fluctuation components based on the limit gradient boosting algorithm and the feature splitting criterion based on the "gain" mechanism. By calculating the sum of the squared error risk estimation improvement brought by each feature when splitting at all boosting decision tree nodes, the feature importance evaluation index directly related to the performance improvement of the prediction model is obtained. The extraction module 205 is used to perform mixed sorting of all features constituting the periodic components and fluctuation components based on the feature importance evaluation index, and to filter based on the dynamic association threshold to remove redundant features and extract the core feature subset.

[0101] Electronic device 3 may include, but is not limited to, processor 30 and memory 31. Those skilled in the art will understand that... Figure 3 This is merely an example of electronic device 3 and does not constitute a limitation on electronic device 3. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device may also include input / output devices, network access devices, buses, etc.

[0102] The processor 30 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0103] The memory 31 can be an internal storage unit of the electronic device 3, such as a hard disk or RAM. The memory 31 can also be an external storage device of the electronic device 3, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 31 can include both internal and external storage units of the electronic device 3. The memory 31 is used to store computer programs and other programs and data required by the electronic device. The memory 31 can also be used to temporarily store data that has been output or will be output.

[0104] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed. That is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above-described device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0105] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0106] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0107] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0108] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0109] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0110] If integrated modules / units are implemented as software functional units and sold or used as independent products, they can be stored in a storage medium. Based on this understanding, all or part of the processes in the above-described embodiments can also be implemented by a computer program instructing related hardware. The computer program for the traffic flow data processing method and feature selection method can be stored in a storage medium. When executed by a processor, this computer program can implement the steps of the various method embodiments described above, namely: constructing a generative model based on a temporal generative adversarial network and a long short-term memory neural network; using the autoencoding component in the generative model to encode and decode the input traffic flow spatiotemporal data; generating and identifying data through a generative adversarial component; learning the data distribution pattern through joint training; and generating simulation sequences for data augmentation; and performing quality enhancement processing on the original traffic flow data and simulation sequences based on a general temporal difference feature construction method to construct an enhanced dataset containing original features and newly added statistical features. The quality enhancement processing of the original traffic flow data and simulation sequences includes calculating the difference of temporal features. The system quantifies and groups features based on their differences and calculates statistics for each group. Using a local weighted smoothing method, the traffic flow data in the enhanced dataset is decomposed into trend components, periodic components, and residual components through an internal iterative process that includes de-trending and de-periodicization. The trend and residual components are then reconstructed into fluctuation components representing non-periodic changes, allowing for separate feature analysis and evaluation of the periodic and fluctuation components. Based on the extreme gradient boosting algorithm and a feature splitting criterion based on a "gain" mechanism, the feature importance of the periodic and fluctuation components is evaluated separately. The sum of the squared error risk estimation improvement brought by each feature during splitting at all boosting decision tree nodes is calculated to obtain a feature importance evaluation index directly related to the performance improvement of the prediction model. Based on the feature importance evaluation index, all features constituting the periodic and fluctuation components are mixed and ranked, and filtered based on a dynamic association threshold to remove redundant features and extract a subset of core features. The computer program includes computer program code, which can be in source code form, object code form, executable files, or some intermediate form. Storage media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the contents of storage media can be appropriately added to or removed according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, storage media may not include electrical carrier signals and telecommunication signals.

[0111] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application. The specific embodiments described above further illustrate the purpose, technical solutions, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the protection scope of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for processing traffic flow data and a feature selection method, characterized in that, The method includes: Step S1: Construct a generative model based on temporal generative adversarial network and long short-term memory neural network. Use the autoencoder component in the generative model to encode and decode the input traffic flow spatiotemporal data, and use the generative adversarial component to generate and identify the data. Learn the data distribution pattern in a joint training manner to generate simulation sequences for data augmentation. Step S2: Based on the general temporal difference feature construction method, perform quality enhancement processing on the original traffic flow data and the simulation sequence to construct an enhanced dataset containing the original features and newly added statistical features. The quality enhancement processing on the original traffic flow data and the simulation sequence includes calculating the difference components of the temporal features, grouping the features according to the difference components, and calculating the statistics of each group. Step S3: Based on the local weighted smoothing method, the traffic flow data in the enhanced dataset is decomposed into trend components, periodic components and residual components through an internal iterative process that includes detrending and deperiodicization. The trend components and the residual components are then reconstructed into fluctuation components that characterize non-periodic changes, so that the periodic components and the fluctuation components can be feature-analyzed and evaluated separately. Step S4: Based on the extreme gradient boosting algorithm and using a feature splitting criterion based on the "gain" mechanism, the feature importance of the periodic composition and the fluctuation composition obtained in Step S3 is evaluated respectively. The feature importance evaluation index directly related to the performance improvement of the prediction model is obtained by calculating the sum of the squared error risk estimation improvements brought by each feature when splitting at all boosting decision tree nodes. This includes: constructing multiple boosting decision trees for the periodic composition or the fluctuation composition respectively; when constructing any single boosting decision tree, when determining the optimal split for each internal node, traversing all candidate features and candidate splitting points, and selecting the feature and splitting point that maximizes the squared error risk estimation improvement for node splitting; for For any feature, the sum of the squared error risk estimation improvement brought by it at each internal node of all boosting decision trees is obtained, and the sum of the cumulative improvement is the feature importance evaluation index of that feature. Step S5: Based on the feature importance evaluation index, all features constituting the periodic and volatile components are mixed and sorted, and filtered based on the dynamic correlation threshold to remove redundant features and extract the core feature subset.

2. The traffic flow data processing method and feature selection method according to claim 1, characterized in that, The joint training of the generative model includes: Step S1a: Encode and decode the input traffic flow spatiotemporal data using the embedding and recovery functions in the autoencoding component, and calculate the recovery loss between the original data and the decoded data; Step S1b: The generator and discriminator in the generative adversarial component play a game. The generator receives random noise and generates simulated data. The discriminator identifies whether the input data is real data or generated data and calculates the discrimination loss. Step S1c: Calculate the intermediate state loss between the intermediate states generated by the autoencoder component and the intermediate states generated by the generator; Step S1d: Iteratively update the network parameters of the generative model by jointly optimizing the recovery loss, the discrimination loss, and the intermediate state loss until the model converges.

3. The traffic flow data processing method and feature selection method according to claim 1, characterized in that, The locally weighted smoothing method decomposes the traffic flow data in the enhanced dataset into trend components, periodic components, and residual components through an internal iterative process that includes detrending and deperiodicization. Step S3a: Initialize trend component data; Step S3b: Perform detrending processing and smooth the detrended subsequence using local weighted smoothing to obtain the initial periodic subsequence; Step S3c: Perform low-pass filtering and local weighted smoothing on the initial periodic subsequence to obtain the trend subsequence, and subtract the trend subsequence from the initial periodic subsequence to obtain the periodic composition of the current iteration; Step S3d: Perform deperiodicization processing and smooth the deperiodic data using local weighted smoothing to obtain the updated trend composition; Step S3e: Repeat steps S3b to S3d until the convergence condition is met, and output the final trend composition and period composition.

4. The traffic flow data processing method and feature selection method according to claim 1, characterized in that, The filtering based on dynamic correlation thresholds includes: Step S5b: Set the dynamic association threshold; Step S5c: Select features whose ranking is higher than the dynamic association threshold as the core feature subset.

5. The traffic flow data processing method and feature selection method according to claim 1, characterized in that, The method further includes: Step S6: Input the core feature subset into the downstream traffic flow prediction model or electric vehicle charging load prediction model to generate traffic control instructions or charging scheduling strategies.

6. The traffic flow data processing method and feature selection method according to claim 5, characterized in that, The generation of the traffic control instructions includes: inputting the core feature subset into a spatiotemporal prediction model to predict the electric vehicle traffic flow at a specific time period and a specific station in the future, and optimizing the timing scheme of traffic lights in the corresponding area or adjusting the display content of variable message signs based on the predicted traffic flow.

7. A method for processing traffic flow data and a feature selection device, characterized in that, The device includes: The training module is used to construct a generative model based on temporal generative adversarial network and long short-term memory neural network. It uses the autoencoder component in the generative model to encode and decode the input traffic flow spatiotemporal data, and uses the generative adversarial component to generate and identify the data. It learns the data distribution pattern in a joint training manner and generates simulation sequences for data augmentation. The construction module is used to perform quality enhancement processing on the original traffic flow data and the simulation sequence based on the general time difference feature construction method, so as to construct an enhanced dataset containing the original features and the newly added statistical features. The quality enhancement processing of the original traffic flow data and the simulation sequence includes calculating the difference components of the time series features, grouping the features according to the difference components, and calculating the statistics of each group. The reconstruction module is used to decompose the traffic flow data in the enhanced dataset into trend components, periodic components, and residual components through an internal iterative process that includes detrending and deperiodicization based on the local weighted smoothing method. The trend components and the residual components are then reconstructed into fluctuation components that characterize non-periodic changes, so that the periodic components and the fluctuation components can be feature-analyzed and evaluated separately. The evaluation module, based on the extreme gradient boosting algorithm and employing a feature splitting criterion based on a "gain" mechanism, evaluates the feature importance of the periodic composition and the volatile composition obtained in step S3. It calculates the sum of the improvements in squared error risk estimation brought by each feature when splitting at all boosting decision tree nodes, obtaining a feature importance evaluation index directly related to the performance improvement of the prediction model. This calculation involves: constructing multiple boosting decision trees for the periodic composition or the volatile composition; when constructing any single boosting decision tree, traversing all candidate features and candidate splitting points to determine the optimal split for each internal node, and selecting the feature and splitting point that maximizes the improvement in squared error risk estimation; for any feature, accumulating the improvement in squared error risk estimation brought by it at each internal node of all boosting decision trees to obtain a cumulative improvement sum, which is the feature importance evaluation index for that feature. The extraction module is used to perform mixed sorting of all features constituting the periodic and volatile components based on the feature importance evaluation index, and to filter based on the dynamic correlation threshold to remove redundant features and extract the core feature subset.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 6.

9. A storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images