E-commerce platform traffic anomaly detection method based on time-frequency potential diffusion model

Abstract

The invention discloses an e-commerce platform traffic anomaly detection method based on a time-frequency potential diffusion model, and belongs to the technical field of communication network traffic detection, and the method comprises the steps: S10, collecting an e-commerce platform traffic data set, and marking abnormal traffic data; s20, performing normalization processing on the traffic data, and dividing the traffic data into a training set, a verification set and a test set; s30, performing fast Fourier transform on the flow data to obtain frequency domain representation of the flow data; s40, constructing and training a time-frequency potential diffusion model; s50, training the model by using the training set, and adjusting the model; s60, calculating an exception score and determining an exception judgment threshold value; s70, verifying the performance of the model by using the test set, and judging the abnormal traffic through the abnormal score and the threshold value; s80, performing anomaly detection on the real-time flow data, and dynamically adjusting an anomaly judgment threshold value; the method has the beneficial effects that by fusing the time domain and frequency domain characteristics, the abnormal judgment threshold value is dynamically adjusted for real-time monitoring, the abnormal flow can be efficiently identified, the method adapts to flow fluctuation, and the detection performance is optimized.

Description

Technical Field

[0001] The present invention relates to the technical field of communication network traffic detection, and more specifically, to an e-commerce platform traffic anomaly detection method based on a time-frequency potential diffusion model. Background Art

[0002] With the rapid development of internet technology, e-commerce has become a vital component of the global economy, profoundly changing consumer shopping habits and business operations. As the core of e-commerce, e-commerce platforms bring together consumers and merchants worldwide, providing convenient transactions and shopping experiences. Driven by diverse business models such as cross-border e-commerce, B2C (business-to-consumer), and C2C (consumer-to-consumer), the e-commerce market continues to expand and competition intensifies. These models not only meet the diverse needs of different user groups but also present new growth opportunities and challenges for businesses. At the same time, with the growing user base and increasing transaction frequency, e-commerce platform network traffic has exhibited significant volatility. This volatility can be further categorized as normal and abnormal. For example, during major promotions, visits and transaction volumes to e-commerce platforms can surge within a short period of time; whereas, during off-seasons or when the platform encounters negative events, traffic may experience a significant drop. These types of fluctuations are considered normal. On the other hand, abnormal patterns caused by platform technical failures, misconfigurations, or malicious attacks are considered abnormal. Failure to promptly monitor and respond to traffic anomalies can lead to system crashes, financial losses, and even serious damage to brand reputation. Therefore, timely and accurate detection and response to these abnormal traffic fluctuations are crucial to ensuring the stable operation of the platform, improving user satisfaction and maintaining brand reputation.

[0003] Traditional traffic anomaly detection methods mostly rely on rule-based monitoring systems. These methods monitor traffic by matching known features and applying preset rules. However, with the continuous development of network environments and the increasing complexity of traffic patterns on e-commerce platforms, these rule-based detection methods often struggle to identify new or unknown anomalous traffic. Especially when dealing with large-scale, high-speed traffic, traditional methods face performance bottlenecks and are unable to respond to dynamic traffic changes in real time, resulting in unsatisfactory traffic anomaly detection results.

[0004] In recent years, data-driven anomaly detection methods, particularly the application of deep learning techniques, have significantly improved the accuracy and flexibility of traffic anomaly detection. Among these methods, autoencoders, as a deep learning model, have been widely used in traffic anomaly detection due to their advantages in unsupervised learning. Autoencoders effectively identify anomalous traffic by learning a low-dimensional representation of traffic data and reconstructing it back to the original data. While the normal traffic patterns of a platform are learned and successfully reconstructed by the autoencoder, anomalous traffic, due to its significant differences from the normal pattern, cannot be accurately reconstructed, thus enabling timely detection. While autoencoders have demonstrated great potential for traffic anomaly detection on e-commerce platforms, existing methods still face several challenges. For example, in dynamic and complex network environments, traditional autoencoder-based anomaly detection methods may experience false positives or false negatives, especially when the traffic data distribution changes, resulting in poor robustness and adaptability. Furthermore, autoencoders are susceptible to high-dimensional data and noise during training, resulting in unstable reconstruction errors and thus affecting the accuracy of anomalous traffic detection. At the same time, some autoencoder models may encounter the "identity mapping" problem when processing abnormal traffic. Even for abnormal traffic, the model can successfully reconstruct it, so that the reconstruction errors of normal traffic and abnormal traffic are almost the same, making it difficult to effectively distinguish between the two. Summary of the Invention

[0005] In order to overcome the shortcomings of existing technologies, an e-commerce platform traffic anomaly detection method based on the time-frequency potential diffusion model is proposed. By integrating time domain and frequency domain features and dynamically adjusting the anomaly judgment threshold for real-time monitoring, it can efficiently identify abnormal traffic, adapt to traffic fluctuations and optimize detection performance.

[0006] The technical solution adopted by the invention to solve its technical problem is: a method for detecting anomaly traffic on an e-commerce platform based on a time-frequency potential diffusion model, the improvement of which includes:

[0007] S10: Collecting a traffic data set of the e-commerce platform, the data set including traffic data in normal and abnormal states, and marking the abnormal traffic data;

[0008] S20: normalizing the traffic dataset and dividing it into a training set, a validation set, and a test set;

[0009] S30: performing fast Fourier transform on the flow data set to obtain a frequency domain representation of the flow data;

[0010] S40: Build a time-frequency latent diffusion model, including a time-domain encoder, a frequency-domain encoder, a time-domain denoising diffusion probability model, a frequency-domain denoising diffusion probability model, and a decoder, and perform training and optimization.

[0011] S50: Use the training set to train the time-frequency latent diffusion model and adjust the network parameters in the model;

[0012] S60: Input the training set, validation set, and test set into the trained time-frequency latent diffusion model, calculate the anomaly score, and determine the anomaly threshold;

[0013] S70: Use the test set to validate the time-frequency latent diffusion model, identify abnormal traffic using anomaly scores and thresholds, and evaluate model performance using validation metrics.

[0014] S80: Use the time-frequency latent diffusion model to detect anomalies in the real-time traffic data of the e-commerce platform, dynamically adjust the anomaly judgment threshold, and determine whether the traffic is abnormal in real time.

[0015] Furthermore, in step S10, the indicator information of the traffic data includes the number of visits, the number of visitors, and the number of page views.

[0016] Furthermore, the specific steps of step S20 include:

[0017] S201: normalizing the collected traffic data, using a minimum-maximum normalization method to normalize each characteristic value to the range of [0, 1];

[0018] S202: Divide the traffic data set generated by the e-commerce platform during normal operation into a training set and a validation set;

[0019] S203: Using a traffic data set generated by the e-commerce platform in both normal and abnormal states as a test set, and providing corresponding annotated data for the test set as a label for the test set.

[0020] Furthermore, the time domain encoder and the frequency domain encoder both include:

[0021] Linear embedding layer, used to map traffic data into a low-dimensional space;

[0022] Position encoding layer, used to add position information to the compressed representation;

[0023] The Encoder layer of Transformer, which includes a self-attention mechanism, a feedforward neural network, and a normalization layer, is used to learn feature representations of time domain or frequency domain data.

[0024] Furthermore, the time-domain denoising diffusion probability model and the frequency-domain denoising diffusion probability model are both composed of a U-Net network, which includes multiple convolutional layers. The U-Net network retains more detailed information through the set symmetric encoding-decoding structure and effectively restores the potential representation after noise addition.

[0025] Furthermore, the decoder comprises:

[0026] Linear embedding layer, compresses the traffic data representation into a low-dimensional space;

[0027] The Transformer's Decoder layer is used to reconstruct traffic data, including self-attention

[0028] Cross-attention layers, feed-forward neural networks, and normalization layers;

[0029] The fully connected layer converts the data output by the Decoder layer into reconstructed traffic data.

[0030] Furthermore, the specific steps of step S40 include:

[0031] S401: Construct a time domain encoder and a frequency domain encoder to map the time domain information and frequency domain information of the traffic data to the time domain latent space and the frequency domain latent space respectively;

[0032] S402: Constructing a denoising diffusion probability model in the time domain and frequency domain latent space, processing the time domain and frequency domain latent representations by adding noise and predicting the noise, and then reconstructing the original traffic data;

[0033] S403: Obtain a time-frequency fusion potential representation by fusing the features of the time-domain potential representation and the frequency-domain potential representation, and decode the potential representation using a decoder to output reconstructed traffic data.

[0034] Furthermore, the specific steps of step S50 include:

[0035] S501: Calculate the mean square error between the input flow data in the training set and the reconstructed flow data output by the model;

[0036] S502: Calculate the error between the noise added to the time domain latent representation and the noise predicted by the time domain denoising diffusion probability model; calculate the error between the noise added to the frequency domain latent representation and the noise predicted by the frequency domain denoising diffusion probability model, and accumulate the two errors calculated in this step;

[0037] S503: Perform a weighted summation of the error calculated in S501 and the error calculated in S502, use the training set to train the model, and adjust the parameters of the entire model.

[0038] Furthermore, the specific steps of step S60 include:

[0039] S601: Obtain a potential representation of the time-domain traffic data through a time-domain encoder, add noise to the data, and perform iterative denoising using a time-domain denoising diffusion probability model to ultimately obtain a time-domain potential representation.

[0040] S602: Perform FFT processing on the time-domain traffic data to obtain frequency-domain data, encode and denoise the frequency-domain data, and iteratively denoise using a frequency-domain denoising diffusion probability model to obtain a frequency-domain potential representation.

[0041] S603: Fusing the potential representations in the time domain and frequency domain to obtain a final feature vector;

[0042] S604: Input the final feature vector and the original traffic data into the decoder to obtain reconstructed data;

[0043] S605: Calculate the anomaly score by the mean square error between the sample and the reconstructed data.

[0044] S606: Based on the anomaly scores of the training set and the validation set, the SPOT algorithm is used to dynamically calculate the anomaly judgment threshold of the test set.

[0045] Furthermore, the e-commerce platform traffic anomaly detection method based on the time-frequency potential diffusion model according to claim 1 is characterized in that the specific steps of step S70 include:

[0046] S701: Calculate the anomaly score of the test set and determine whether each sample is abnormal traffic based on the set anomaly threshold;

[0047] S702: When the anomaly score is greater than the anomaly threshold, the sample is determined to be abnormal traffic, otherwise it is normal traffic;

[0048] S703: Compare the anomaly detection results of the test set with the test set labels, and use precision, recall, and F1 value to verify the model.

[0049] Furthermore, the specific steps of step S80 include:

[0050] S801: Collect network traffic data from the e-commerce platform in real time and calculate anomaly scores using the trained model;

[0051] S802: Dynamically update the anomaly determination threshold: update the anomaly threshold based on the anomaly scores of historical traffic data and real-time traffic data;

[0052] S803: Calculate the anomaly score for each real-time traffic sample. If the anomaly score is greater than the anomaly threshold, it is determined to be abnormal traffic; otherwise, it is normal traffic.

[0053] The beneficial effects of the present invention are: by fusing time domain and frequency domain features and dynamically adjusting the abnormality judgment threshold for real-time monitoring, abnormal traffic can be efficiently identified, recognition accuracy can be improved, and detection performance can be optimized. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1This is a flow chart of the method for detecting anomaly in e-commerce platform traffic based on the time-frequency potential diffusion model of the present invention;

[0055] Figure 2 This is a cluster architecture diagram of the e-commerce platform traffic anomaly detection method based on the time-frequency potential diffusion model of the present invention;

[0056] Figure 3 This is a schematic diagram of the time domain encoder / frequency domain encoder network structure of the e-commerce platform traffic anomaly detection method based on the time-frequency potential diffusion model of the present invention;

[0057] Figure 4 Schematic diagram of the noise prediction architecture after adding noise to the time domain potential representation / frequency domain potential representation of the e-commerce platform traffic anomaly detection method based on the time-frequency potential diffusion model of the present invention;

[0058] Figure 5 This is a schematic diagram of the decoder network structure of the e-commerce platform traffic anomaly detection method based on the time-frequency potential diffusion model of the present invention;

[0059] Figure 6 This is a schematic diagram of the network structure of the e-commerce platform traffic anomaly detection method based on the time-frequency potential diffusion model of the present invention. DETAILED DESCRIPTION

[0060] The present invention will be further described below with reference to the accompanying drawings and examples.

[0061] The following will clearly and completely describe the concept, specific structure and technical effects of the present invention in combination with the embodiments and drawings, so as to fully understand the purpose, characteristics and effects of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention. In addition, all the connection / connection relationships involved in the patent do not refer to the direct connection of components, but refer to the fact that a better connection structure can be formed by adding or reducing connection accessories according to the specific implementation situation. The various technical features in the invention can be combined interactively without conflicting with each other.

[0062] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indications will also change accordingly.

[0063] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of such features. Secondly, the technical solutions between the various embodiments can be combined with each other, but this must be based on the fact that ordinary technicians in this field can implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0064] See also Figure 1-Figure 2 As shown, the present invention provides an e-commerce platform traffic anomaly detection method based on a time-frequency potential diffusion model, comprising:

[0065] S10: Collecting a traffic data set of the e-commerce platform, the data set including traffic data in normal and abnormal states, and marking the abnormal traffic data;

[0066] S20: normalizing the traffic dataset and dividing it into a training set, a validation set, and a test set;

[0067] S30: performing a Fast Fourier Transform (FFT) on the flow data set to obtain a frequency domain representation of the flow data;

[0068] S40: Build a time-frequency latent diffusion model, including a time-domain encoder, a frequency-domain encoder, a time-domain denoising diffusion probability model, a frequency-domain denoising diffusion probability model, and a decoder, and perform training and optimization.

[0069] S50: Use the training set to train the time-frequency latent diffusion model and adjust the network parameters in the model so that the model can accurately reconstruct the traffic data and accurately predict the noise added by the time domain and frequency domain latent representations;

[0070] S60: Input the training set, validation set, and test set into the trained time-frequency latent diffusion model, calculate the anomaly score, and determine the anomaly threshold;

[0071] S70: Use the test set to validate the time-frequency latent diffusion model, identify abnormal traffic using anomaly scores and thresholds, and evaluate model performance using validation metrics.

[0072] S80: Use the time-frequency latent diffusion model to detect anomalies in the real-time traffic data of the e-commerce platform, dynamically adjust the anomaly judgment threshold, and determine whether the traffic is abnormal in real time.

[0073] In this invention, by combining time-domain and frequency-domain information, the time-frequency latent diffusion model can more comprehensively capture abnormal patterns in traffic data, especially in environments with large traffic fluctuations on e-commerce platforms. By training the model in real time and dynamically adjusting the threshold, it can adapt to changes in e-commerce platform traffic, ensuring that the model can still accurately identify abnormal traffic under different traffic conditions. By optimizing the model with a training set, the time-frequency latent diffusion model can accurately reconstruct traffic data and predict the noise of the potential representation, thereby improving the accuracy of anomaly score calculation. This method not only detects anomalies in historical traffic in an offline environment, but also enhances the ability to identify unknown abnormal traffic, significantly reducing false positives and false negatives, providing a more intelligent and automated solution for network security protection on e-commerce platforms.

[0074] Furthermore, in step S10, the indicator information of the traffic data includes the number of visits, the number of visitors and the number of page views.

[0075] In this embodiment, we plan to collect two sets of e-commerce platform traffic data: one set is the traffic data generated by the e-commerce platform during normal operation; the other set contains the traffic data generated by the e-commerce platform during normal and abnormal operation, and the abnormal traffic data is marked to distinguish it from normal traffic. To obtain this data, the e-commerce platform's traffic information, including key indicators such as the number of visits, number of visitors, and page views, can be collected through the e-commerce platform's backend management system or related website traffic analysis tools. Common e-commerce traffic indicator information is shown in Table 1:

[0076] Table 1 Common e-commerce traffic indicators

[0077] Bounce rate Page visit duration Page views Number of visitors

[0078] Furthermore, the specific steps of step S20 include:

[0079] S201: normalizing the collected traffic data, using a minimum-maximum normalization method to normalize each characteristic value to the range of [0, 1];

[0080] S202: Divide the traffic data set generated by the e-commerce platform during normal operation into a training set and a validation set;

[0081] S203: Using a traffic data set generated by the e-commerce platform in both normal and abnormal states as a test set, and providing corresponding annotated data for the test set as a label for the test set.

[0082] Normalize the collected traffic data set, and for each sample x={x1,x2,…,x n Each feature x in n, after normalization using the following formula, we get x′={x′1,x′2,…,x′ n}:

[0083]

[0084] in, is the minimum value of the nth feature in the traffic data set collected under normal operating conditions, is the maximum value of the nth feature in the traffic dataset collected under normal operating conditions.

[0085] The frequency domain data of the flow is obtained by fast Fourier transforming the time domain data. The collected and processed flow data X∈R L×D Perform Fourier transform on each dimension in, and for the feature of the dth dimension Use the following formula to transform and preserve the valid frequencies:

[0086]

[0087] in It is the complex value of the kth frequency of the dth dimension feature of the traffic data in the frequency domain;

[0088] The transformed complex value in and Representing the real and imaginary parts respectively, extract the real and imaginary parts from each frequency component:

[0089]

[0090] The real part information and imaginary part information of each feature calculated are spliced ​​in the corresponding feature dimension to obtain Finally, the frequency domain data FFT(X)∈R of the traffic is obtained L×D

[0091] like Figure 3 As shown, the time domain encoder and the frequency domain encoder both include:

[0092] Linear embedding layer, used to map traffic data into a low-dimensional space;

[0093] The position encoding layer is used to add position information to the compressed representation of the position encoding layer. The position information is implemented by sine and cosine functions.

[0094] The Transformer's Encoder layer, which includes a self-attention mechanism, a feedforward neural network, and a normalization layer, is used to learn the internal dependencies of the compressed representation and further extract key features of traffic data in the time or frequency domain as well as deeper feature representations.

[0095] The linear embedding layer effectively maps traffic data into a low-dimensional space, reducing data complexity while preserving important feature information. The position encoding layer, by introducing positional information and specifically encoding the compressed representation using sine and cosine functions, helps the model understand the order and temporal relationships of the input data, improving its ability to capture time-domain features. The Transformer's encoder layer, leveraging a self-attention mechanism, effectively captures long-term dependencies in the data, enabling the model to automatically learn key time-domain and frequency-domain features from traffic data.

[0096] Further, if Figure 4 As shown, the time domain denoising diffusion probability model and the frequency domain denoising diffusion probability model are both composed of a U-Net network. The U-Net network includes multiple convolutional layers. The symmetrical encoding-decoding structure is set to retain more detailed information and effectively restore the potential representation after noise addition.

[0097] like Figure 5 As shown, the decoder includes:

[0098] The linear embedding layer compresses the traffic data into a low-dimensional space. Through linear transformation, the original traffic data representation can be mapped to a lower-dimensional space, thereby reducing computational complexity while maintaining the integrity of key features and avoiding interference from redundant information.

[0099] The Transformer's Decoder layer is used to reconstruct traffic data, which includes self-attention, cross-attention layers, feedforward neural networks, and normalization layers;

[0100] The fully connected layer converts the data output by the Decoder layer into reconstructed traffic data.

[0101] The self-attention layer helps the model capture dependencies within the input data and can perform a weighted summation of features at each position to identify important temporal information or features. The cross-attention layer is used to interact with the outputs of the encoder and decoder, allowing the decoder to focus on the encoder's output and further strengthen feature transfer. The feedforward neural network facilitates nonlinear transformations at each position. The normalization layer helps accelerate the training process and ensures stable model training. Its input includes the output of the linear embedding layer and the aforementioned potential time-frequency fusion features. It fully utilizes the global contextual information of the traffic data and combines it with the potential representation that captures normal pattern features to more appropriately reconstruct the traffic data, improving the reconstruction quality of normal pattern traffic data while reducing the reconstruction quality of abnormal pattern traffic data. The fully connected layer takes the output of the Transformer's Decoder layer as input, and its output serves as the reconstructed traffic data.

[0102] Further, if Figure 6 As shown, the specific steps of step S40 include:

[0103] S401: Construct a time domain encoder and a frequency domain encoder to map the time domain information and frequency domain information of the traffic data to the time domain latent space and the frequency domain latent space respectively;

[0104] S402: Constructing a denoising diffusion probability model in the time domain and frequency domain latent space, processing the time domain and frequency domain latent representations by adding noise and predicting the noise, and then reconstructing the original traffic data;

[0105] S403: Obtain a time-frequency fusion potential representation by fusing the features of the time-domain potential representation and the frequency-domain potential representation, and decode the potential representation using a decoder to output reconstructed traffic data.

[0106] The time-domain encoder focuses on extracting data changes and dynamic information over time, making it suitable for capturing short-term fluctuations and time series characteristics in traffic data. The frequency-domain encoder is better able to process the periodicity and frequency information in traffic data, capturing long-term periodic changes or frequency-related anomalies. By processing time and frequency domain information separately, the model can more specifically extract features at different levels, thereby enhancing reconstruction quality.

[0107] In this embodiment, the time domain potential representation is noised and the added noise is predicted using the constructed model;

[0108] The output TZ of the time domain encoder is denoised according to the following formula to obtain TZ T :

[0109]

[0110] in, is a predefined hyperparameter, ∈ t is pure noise sampled from a standard normal distribution.

[0111] The added noise is predicted using the time diffusion model, and the input of the time diffusion model is the time domain potential representation TZ after the noise is added T and the noise time step T, the prediction result is expressed as

[0112] Add noise to the frequency domain latent representation and predict the added noise using the constructed model;

[0113] The frequency domain encoder output FZ is denoised according to the following formula to obtain FZ T :

[0114]

[0115] in, is a predefined hyperparameter, ∈ f is pure noise sampled from a standard normal distribution.

[0116] The added noise is predicted using the frequency domain diffusion model, the input of which is the frequency domain potential representation FZ after adding noise. T and the noise time step T, the prediction result is expressed as

[0117] Fusion of time-frequency potential representation: The output TZ of the time domain encoder and the output FZ of the frequency domain encoder described in the first step are subjected to feature fusion according to the following formula to obtain TFZ:

[0118] TFZ=TZ+FZ;

[0119] Fusion of the time-domain latent representation (TZ) and the frequency-domain latent representation (FZ) into a fused time-frequency latent representation (TFZ) combines the strengths of both. This fusion not only captures the temporal characteristics of the data but also fully accounts for patterns in periodicity and frequency. Real-world traffic data often contains noise, anomalies, or missing data. By employing a denoising and noise removal approach, the model can better cope with these imperfect data. This approach makes the model more stable during training and enhances its robustness in various scenarios.

[0120] Furthermore, the specific steps of step S50 include:

[0121] S501: Calculate the mean square error between the input flow data in the training set and the reconstructed flow data output by the model;

[0122] S502: Calculate the error between the noise added to the time domain potential representation and the noise predicted by the time domain denoising diffusion probability model; calculate the error between the noise added to the frequency domain potential representation and the noise predicted by the frequency domain denoising diffusion probability model; accumulate the two errors calculated in this step;

[0123] S503: Perform a weighted summation on the error calculated in S501 and the error calculated in S502, train the model using the training set, and adjust the parameters in the entire model.

[0124] The model is trained by calculating the error between the flow data input to the model and the reconstructed value of the model output, the error between the noise added to the time domain potential representation and the noise predicted by the time domain denoising diffusion probability model, and the error between the noise added to the frequency domain potential representation and the noise predicted by the frequency domain denoising diffusion probability model.

[0125] By adjusting the parameters of different network structures in the model, a model is obtained that can accurately reconstruct traffic data and accurately predict the noise added by the time domain potential representation and the noise added by the frequency domain potential representation.

[0126] The loss function for model training is defined by the following formula:

[0127]

[0128] Among them, Loss is the loss function used in training, X represents the traffic data after data preprocessing, Represents the reconstructed traffic data output by the encoder at the end of the model, λ represents the hyperparameter, and Loss t Represents the loss function of the time domain denoising diffusion probability model, Loss f Represents the loss function of the frequency domain denoising diffusion probability model

[0129]

[0130] in, represents the prediction noise of the time domain denoising diffusion probability model, ∈ t represents the true noise added to the time domain latent representation.

[0131]

[0132] in, represents the prediction noise of the frequency domain denoising diffusion probability model, ∈ f represents the true noise added to the frequency domain latent representation.

[0133] By separately calculating the noise prediction error for the latent representations in the time and frequency domains, the model optimizes for the characteristics of each domain, more accurately removing noise and restoring more refined and natural features. The training loss function combines the flow data reconstruction error and the noise prediction error. This allows the model to prioritize the accuracy of the denoising process while maintaining data reconstruction quality, thereby improving the model's robustness and reliability.

[0134] Next, the specific steps of step S60 include:

[0135] S601: Obtain a potential representation of the time-domain traffic data through a time-domain encoder, add noise to the data, and perform iterative denoising using a time-domain denoising diffusion probability model to ultimately obtain a time-domain potential representation.

[0136] S602: Perform FFT processing on the time-domain traffic data to obtain frequency-domain data, encode and denoise the frequency-domain data, and iteratively denoise using a frequency-domain denoising diffusion probability model to obtain a frequency-domain potential representation.

[0137] S603: Fusing the potential representations in the time domain and frequency domain to obtain a final feature vector;

[0138] S604: Input the final feature vector and the original traffic data into the decoder to obtain reconstructed data;

[0139] S605: Calculate the anomaly score by the mean square error between the sample and the reconstructed data.

[0140] S606: Based on the anomaly scores of the training set and the validation set, the SPOT algorithm is used to dynamically calculate the anomaly judgment threshold of the test set.

[0141] In this embodiment, the training set, validation set, and test set are input into the trained model to calculate the anomaly score. The calculation of the anomaly score consists of the following steps:

[0142] Calculate the frequency domain data FFT(X) of the flow rate through the preprocessed time domain data X;

[0143] The time domain data is encoded by the time domain encoder to obtain the time domain potential representation, and the time domain potential representation TZ is denoised according to the following formula to obtain TZ T :

[0144]

[0145] in, is a predefined hyperparameter, ∈ t is pure noise sampled from a standard normal distribution;

[0146] The noise of the time domain potential representation after adding noise is predicted by the time domain denoising diffusion probability model at the corresponding denoising time step T And sample the next denoising time step T-1 using the following formula:

[0147]

[0148] The sampled TZ T-1 The sample is obtained by adding noise using the following formula

[0149]

[0150] Among them, α T , are all predefined hyperparameters, z,∈ T-1 Sampling from a standard normal distribution;

[0151] right Also use the above denoising formula to obtain

[0152] Continue to Using the above noise addition formula, we can obtain

[0153] Repeat the above operation for u times to obtain u samples of the denoising time step T-1 feature space. Calculate the information entropy H of this u sample and select the sample with the largest information entropy as the representative of the denoising time step T-1 feature space to participate in the next step of denoising space Sampling calculation.

[0154] Iterate the above operation T times until TZ0 is calculated.

[0155] The frequency domain data is processed by the frequency domain encoder to obtain the frequency domain potential representation, and the frequency domain potential representation FZ is denoised according to the following formula to obtain FZ T :

[0156]

[0157] in, is a predefined hyperparameter, ∈ f is pure noise sampled from a standard normal distribution;

[0158] The frequency domain denoising diffusion probability model is used to predict the noise of the frequency domain potential representation after adding noise at the corresponding denoising time step T. And sample the next denoising time step T-1 using the following formula:

[0159]

[0160] The sampled FZ T-1 The sample is obtained by adding noise using the following formula

[0161]

[0162] Among them, α T , are all predefined hyperparameters, z,∈ T-1 Sampling from a standard normal distribution;

[0163] right Also use the above denoising formula to obtain

[0164] Continue to Using the above noise addition formula, we can obtain

[0165] Repeat the above operation for u times to obtain u samples of the denoising time step T-1 feature space. Calculate the information entropy H of this u sample and select the sample with the largest information entropy as the representative of the denoising time step T-1 feature space to participate in the next step of denoising space Sampling calculation.

[0166] Iterate the above operation T times until FZ0 is calculated.

[0167] TFZ is obtained by fusing the time domain potential representation TZ0 and FZ0 after adding noise and iterative denoising:

[0168] TFZ=TZ0+FZ0;

[0169] The reconstructed data is obtained by inputting the fusion feature TFZ and the preprocessed traffic data X into the decoder

[0170] The diffusion model for adding and denoising data simulates the true distribution of data and gradually restores the underlying representation over multiple time steps of the denoising process, enabling the model to learn more stable and universal features. During the denoising process, information entropy is used to select optimal samples, ensuring that key information is not lost during the denoising process, improving denoising quality, and making the ultimately extracted features more reliable.

[0171] The anomaly score of each traffic data sample is composed of the data sample x and the reconstructed data sample The mean square error of is determined by:

[0172] The above method is used to obtain the anomaly score Train_Score for the training set, the anomaly score Valid_Score for the validation set, and the anomaly score Test_Score for the test set. Based on the anomaly scores of the training and validation sets, the SPOT algorithm dynamically determines the threshold for the test set. The core concept of the SPOT algorithm is to dynamically set the threshold using the statistical characteristics of the data. As the data flow changes, the threshold is continuously updated to adapt to different network environments or application scenarios. This makes the SPOT algorithm particularly suitable for traffic monitoring and other scenarios that require real-time dynamic threshold adjustment.

[0173] δ=SPOT(concat(Train_Score,Valid_Score),Test_Score);

[0174] The SPOT algorithm dynamically calculates the anomaly threshold, ensuring that it remains static and continuously adjusts based on the statistical characteristics of the data. The SPOT algorithm adapts to traffic distribution in different time periods and network conditions. Traditional fixed threshold methods often struggle to adapt to complex traffic scenarios, while the SPOT algorithm improves detection accuracy while maintaining a low false alarm rate.

[0175] The specific steps of step S70 include:

[0176] S701: Calculate the anomaly score of the test set and determine whether each sample is abnormal traffic based on the set anomaly threshold;

[0177] S702: When the anomaly score is greater than the anomaly threshold, the sample is determined to be abnormal traffic, otherwise it is normal traffic;

[0178] S703: Compare the anomaly detection results of the test set with the test set labels, and use precision, recall, and F1 value to verify the model.

[0179] Using the obtained test set anomaly score and the determined anomaly threshold, we determine abnormal traffic on the test set. For each sample x in the test set, if the anomaly score s > δ, we determine that the sample x is an abnormal traffic data sample; otherwise, it is a normal traffic data sample. The anomaly detection results of the test set are combined with the labeled data of the test set to verify the model's effectiveness. The verification metrics are affiliation-precision, affiliation-recall, and affiliation-F1.

[0180] The specific steps of step S80 include:

[0181] S801: Collect network traffic data from the e-commerce platform in real time and calculate anomaly scores using the trained model;

[0182] S802: Dynamically update the anomaly determination threshold: update the anomaly threshold based on the anomaly scores of historical traffic data and real-time traffic data;

[0183] S803: Calculate the anomaly score for each real-time traffic sample. If the anomaly score is greater than the anomaly threshold, it is determined to be abnormal traffic; otherwise, it is normal traffic.

[0184] Furthermore, network traffic data generated by the e-commerce platform is collected in real time, and the trained model is sampled to calculate the anomaly score. The anomaly judgment threshold is dynamically updated based on the anomaly score of historical traffic data and the anomaly score of real-time collected traffic data:

[0185] δ new =SPOT(History_Score,New_Score);

[0186] Among them, History_Score represents the anomaly score of historical traffic data, and New_Score represents the anomaly score of real-time collected traffic data.

[0187] For each sample x in the real-time collected traffic data new The anomaly score s new , when s new >δ new When the sample x new If the value is not equal to 0, it is abnormal traffic data, otherwise it is normal traffic data. The above operation is performed on each sample data to complete the real-time anomaly detection of traffic data.

[0188] By combining the anomaly scores of historical and real-time traffic data to adjust the threshold, we can, to a certain extent, balance the differences between historical patterns and current traffic. This approach effectively avoids relying solely on historical data to establish patterns, while also avoiding over-reliance on local fluctuations in current traffic. The anomaly scores in historical data provide a reliable reference for real-time traffic data, helping to determine whether there are abnormal traffic patterns. E-commerce platforms often experience significant traffic fluctuations, especially during peak periods such as promotions and holidays. This approach ensures that the system can accurately identify abnormal data even in high-traffic environments by detecting and dynamically adjusting the anomaly threshold in real time.

[0189] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

[0190] Finally, it should be noted that the above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. An e-commerce platform traffic anomaly detection method based on the time-frequency potential diffusion model is characterized by: include: S10: Collecting a traffic data set of the e-commerce platform, the data set including traffic data in normal and abnormal states, and marking the abnormal traffic data; S20: normalizing the traffic dataset and dividing it into a training set, a validation set, and a test set; S30: performing fast Fourier transform on the flow data set to obtain a frequency domain representation of the flow data; S40: Build a time-frequency latent diffusion model, including a time-domain encoder, a frequency-domain encoder, a time-domain denoising diffusion probability model, a frequency-domain denoising diffusion probability model, and a decoder, and perform training and optimization. S50: Use the training set to train the time-frequency latent diffusion model and adjust the network parameters in the model; S60: Input the training set, validation set, and test set into the trained time-frequency latent diffusion model, calculate the anomaly score, and determine the anomaly threshold; S70: Use the test set to validate the time-frequency latent diffusion model, identify abnormal traffic using anomaly scores and anomaly thresholds, and evaluate model performance using validation metrics. S80: Use the trained time-frequency latent diffusion model to detect anomalies in the real-time traffic data of the e-commerce platform, dynamically adjust the anomaly judgment threshold, and determine in real time whether the traffic is abnormal.

2. The e-commerce platform traffic anomaly detection method based on the time-frequency potential diffusion model according to claim 1 is characterized in that: In step S10, the indicator information of the traffic data includes the number of visits, the number of visitors, and the number of page views.

3. The e-commerce platform traffic anomaly detection method based on the time-frequency potential diffusion model according to claim 1 is characterized in that: The specific steps of step S20 include: S201: normalizing the collected traffic data, using a minimum-maximum normalization method to normalize each characteristic value to the range of [0, 1]; S202: Divide the traffic data set generated by the e-commerce platform during normal operation into a training set and a validation set; S203: Using a traffic data set generated by the e-commerce platform in both normal and abnormal states as a test set, and providing corresponding annotated data for the test set as a label for the test set.

4. The e-commerce platform traffic anomaly detection method based on the time-frequency potential diffusion model according to claim 1 is characterized in that: The time domain encoder and the frequency domain encoder both include: Linear embedding layer, used to map traffic data into a low-dimensional space; Position encoding layer, used to add position information to the compressed representation; The Encoder layer of Transformer, which includes a self-attention mechanism, a feedforward neural network, and a normalization layer, is used to learn feature representations of time domain or frequency domain data. The time domain denoising diffusion probability model and the frequency domain denoising diffusion probability model are both composed of a U-Net network. The U-Net network includes multiple convolutional layers. The symmetrical encoding-decoding structure is set to retain more detailed information and effectively restore the potential representation after noise addition.

5. The e-commerce platform traffic anomaly detection method based on the time-frequency potential diffusion model according to claim 1 is characterized in that: The decoder comprises: Linear embedding layer, compresses the traffic data representation into a low-dimensional space; The Transformer's Decoder layer is used to reconstruct traffic data, including self-attention Cross-attention layers, feed-forward neural networks, and normalization layers; The fully connected layer converts the data output by the Decoder layer into reconstructed traffic data.

6. The e-commerce platform traffic anomaly detection method based on the time-frequency potential diffusion model according to claim 1 is characterized in that: The specific steps of step S40 include: S401: Construct a time domain encoder and a frequency domain encoder to map the time domain information and frequency domain information of the traffic data to the time domain latent space and the frequency domain latent space respectively; S402: Constructing a denoising diffusion probability model in the time domain and frequency domain latent space, processing the time domain and frequency domain latent representations by adding noise and predicting the noise, and then reconstructing the original traffic data; S403: Obtain a time-frequency fusion potential representation by fusing the features of the time-domain potential representation and the frequency-domain potential representation, and decode the potential representation using a decoder to output reconstructed traffic data.

7. The e-commerce platform traffic anomaly detection method based on the time-frequency potential diffusion model according to claim 1 is characterized in that: The specific steps of step S50 include: S501: Calculate the mean square error between the input flow data in the training set and the reconstructed flow data output by the model; S502: Calculate the error between the noise added to the time domain potential representation and the noise predicted by the time domain denoising diffusion probability model; calculate the error between the noise added to the frequency domain potential representation and the noise predicted by the frequency domain denoising diffusion probability model; accumulate the two errors calculated in this step; S503: Perform a weighted sum of the error calculated in S501 and the error calculated in S502, train the model using the training set, and adjust the parameters of the entire model.

8. The e-commerce platform traffic anomaly detection method based on the time-frequency potential diffusion model according to claim 1 is characterized in that: The specific steps of step S60 include: S601: Obtain a potential representation of the time-domain traffic data through a time-domain encoder, add noise to the data, and perform iterative denoising using a time-domain denoising diffusion probability model to ultimately obtain a time-domain potential representation. S602: Perform FFT processing on the time-domain traffic data to obtain frequency-domain data, encode and denoise the frequency-domain data, and iteratively denoise using a frequency-domain denoising diffusion probability model to obtain a frequency-domain potential representation. S603: Fusing the potential representations in the time domain and frequency domain to obtain a final feature vector; S604: Input the final feature vector and the original traffic data into the decoder to obtain reconstructed data; S605: Calculate the anomaly score by using the mean square error between the sample and the reconstructed data; S606: Based on the anomaly scores of the training set and the validation set, the SPOT algorithm is used to dynamically calculate the anomaly judgment threshold of the test set.

9. The e-commerce platform traffic anomaly detection method based on the time-frequency potential diffusion model according to claim 1 is characterized in that: The e-commerce platform traffic anomaly detection method based on the time-frequency potential diffusion model according to claim 1 is characterized in that the specific steps of step S70 include: S701: Calculate the anomaly score of the test set and determine whether each sample is abnormal traffic based on the set anomaly threshold; S702: When the anomaly score is greater than the anomaly threshold, the sample is determined to be abnormal traffic, otherwise it is normal traffic; S703: Compare the anomaly detection results of the test set with the test set labels, and use precision, recall, and F1 value to verify the model.

10. The e-commerce platform traffic anomaly detection method based on the time-frequency potential diffusion model according to claim 1 is characterized in that: The specific steps of step S80 include: S801: Collect network traffic data from the e-commerce platform in real time and calculate anomaly scores using the trained model; S802: Dynamically update the anomaly determination threshold: update the anomaly threshold based on the anomaly scores of historical traffic data and real-time traffic data; S803: Calculate the anomaly score for each real-time traffic sample. If the anomaly score is greater than the anomaly threshold, it is determined to be abnormal traffic; otherwise, it is normal traffic.

Images