An anomaly detection method for e-commerce platform traffic based on a time-frequency latent diffusion model

Abstract

This invention discloses a method for detecting traffic anomalies on e-commerce platforms based on a time-frequency latent diffusion model, belonging to the field of communication network traffic detection technology. The method includes the following steps: S10: Collecting e-commerce platform traffic datasets and labeling abnormal traffic data; S20: Normalizing the traffic data and dividing it into training, validation, and test sets; S30: Performing a Fast Fourier Transform on the traffic data to obtain its frequency domain representation; S40: Constructing and training a time-frequency latent diffusion model; S50: Training and adjusting the model using the training set; S60: Calculating anomaly scores and determining anomaly judgment thresholds; S70: Verifying model performance using the test set and judging abnormal traffic based on anomaly scores and thresholds; S80: Detecting anomalies in real-time traffic data and dynamically adjusting the anomaly judgment thresholds. The beneficial effects of this invention are: by fusing time-domain and frequency-domain features and dynamically adjusting the anomaly judgment thresholds for real-time monitoring, it can efficiently identify abnormal traffic, adapt to traffic fluctuations, and optimize detection performance.

Description

Technical Field

[0001] This invention relates to the field of communication network traffic detection technology, and more specifically, to a method for detecting traffic anomalies on e-commerce platforms based on a time-frequency potential diffusion model. Background Technology

[0002] With the rapid development of internet technology, e-commerce has become an important part of the global economy, profoundly changing consumers' shopping habits and business operation models. As the core of e-commerce, e-commerce platforms bring together consumers and merchants worldwide, providing convenient transaction and shopping experiences. Especially driven by various business models such as cross-border e-commerce, B2C (business-to-consumer), and C2C (consumer-to-consumer), the e-commerce market continues to expand, and competition is becoming increasingly fierce. These models not only meet the diverse needs of different user groups but also bring new growth opportunities and challenges to businesses. At the same time, with the continuous expansion of the user base and the increasing frequency of transactions, the network traffic of e-commerce platforms exhibits significant volatility. This volatility can be further divided into normal and abnormal fluctuations. For example, during large-scale promotional events, the number of visits and transactions on e-commerce platforms can surge in a short period; while during off-seasons or when the platform encounters negative events, traffic may drop sharply. Such fluctuations can be considered normal. On the other hand, abnormal patterns caused by platform technical failures, misconfigurations, or malicious attacks are considered abnormal fluctuations. Failure to monitor and respond to abnormal traffic situations in a timely manner may lead to system crashes, financial losses, and even serious damage to brand reputation. Therefore, timely and accurate detection and response to these abnormal traffic fluctuations are crucial for ensuring the stable operation of the platform, improving user satisfaction, and maintaining brand reputation.

[0003] Traditional traffic anomaly detection methods mostly rely on rule-based monitoring systems. These methods monitor traffic by matching known features and applying preset rules. However, with the continuous development of the network environment and the increasing complexity of e-commerce platform traffic patterns, these rule-based detection methods often fall short when identifying new or unknown abnormal traffic. Especially when facing large-scale, high-speed traffic, traditional methods suffer from performance bottlenecks and cannot respond to dynamic changes in traffic in real time, resulting in unsatisfactory traffic anomaly detection results.

[0004] In recent years, data-driven anomaly detection methods, especially those based on deep learning, have significantly improved the accuracy and flexibility of traffic anomaly detection. Among these methods, autoencoders, as a deep learning model, are widely used in traffic anomaly detection due to their advantages in unsupervised learning. Autoencoders effectively identify abnormal traffic by learning a low-dimensional representation of traffic data and reconstructing it back to the original data. When the normal traffic pattern of a platform is learned and successfully reconstructed by the autoencoder, abnormal traffic, due to its significant differences from the normal pattern, cannot be accurately reconstructed, thus enabling timely detection. Although autoencoders have demonstrated strong potential in e-commerce platform traffic anomaly detection, existing methods still face several challenges. For example, in dynamic and complex network environments, traditional autoencoder-based anomaly detection methods may experience false positives or false negatives, especially when the distribution of traffic data changes, resulting in poor robustness and adaptability of the model. Furthermore, autoencoders are susceptible to high-dimensional data and noise during training, leading to unstable reconstruction errors and affecting the accuracy of abnormal traffic detection. Meanwhile, some autoencoder models may encounter the "identity mapping" problem when dealing with abnormal traffic. Even with abnormal traffic, the model can successfully reconstruct it, making the reconstruction error of normal traffic and abnormal traffic almost indistinguishable and difficult to effectively differentiate between the two. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, an e-commerce platform traffic anomaly detection method based on a time-frequency potential diffusion model can efficiently identify abnormal traffic, adapt to traffic fluctuations, and optimize detection performance by integrating time-domain and frequency-domain features, dynamically adjusting the anomaly judgment threshold, and monitoring in real time.

[0006] The technical solution adopted by the invention to solve its technical problem is: a method for detecting traffic anomalies on e-commerce platforms based on a time-frequency latent diffusion model, the improvement of which includes:

[0007] S10: Collect traffic datasets from e-commerce platforms, including traffic data under normal and abnormal conditions, and label abnormal traffic data.

[0008] S20: Normalize the traffic dataset and divide it into a training set, a validation set, and a test set;

[0009] S30: Perform a fast Fourier transform on the traffic dataset to obtain the frequency domain representation of the traffic data;

[0010] S40: Construct a time-frequency potential diffusion model, including a time-domain encoder, a frequency-domain encoder, a time-domain denoising diffusion probability model, a frequency-domain denoising diffusion probability model, and a decoder, and perform training and optimization.

[0011] S50: Train the time-frequency potential diffusion model using the training set and adjust the network parameters in the model;

[0012] S60: Input the training set, validation set, and test set into the trained time-frequency latent diffusion model, calculate the anomaly score, and determine the anomaly threshold;

[0013] S70: Validate the time-frequency potential diffusion model using a test set, identify abnormal traffic through anomaly scores and thresholds, and evaluate model performance through validation metrics.

[0014] S80: Uses a time-frequency potential diffusion model to detect anomalies in real-time traffic data of e-commerce platforms, dynamically adjusts the anomaly judgment threshold, and determines in real time whether the traffic is abnormal.

[0015] Furthermore, in step S10, the traffic data metrics include the number of visits, the number of visitors, and the number of page views.

[0016] Furthermore, step S20 specifically includes the following steps:

[0017] S201: Normalize the collected traffic data by using the min-max normalization method to normalize each feature value to the range of [0,1].

[0018] S202: Divide the traffic dataset generated by the e-commerce platform under normal operating conditions into a training set and a validation set;

[0019] S203: Use the traffic dataset that includes traffic generated by the e-commerce platform under both normal and abnormal conditions as the test set, and provide corresponding labeled data for the test set as the test set's labels.

[0020] Furthermore, both the time-domain encoder and the frequency-domain encoder include:

[0021] Linear embedding layer is used to map traffic data to a low-dimensional space;

[0022] The positional encoding layer is used to add positional information to the compressed representation;

[0023] The Transformer's Encoder layer, which includes a self-attention mechanism, a feedforward neural network, and a normalization layer, is used to learn feature representations of time-domain or frequency-domain data.

[0024] Furthermore, both the temporal denoising diffusion probability model and the frequency domain denoising diffusion probability model are composed of a U-Net network. The U-Net network includes multiple convolutional layers and retains more detailed information through a symmetric encoder-decoder structure, effectively restoring the latent representation after noise addition.

[0025] Furthermore, the decoder includes:

[0026] The linear embedding layer compresses the traffic data representation into a low-dimensional space;

[0027] The Transformer's Decoder layer is used to reconstruct traffic data, including self-attention.

[0028] Cross-attention layer, feedforward neural network, and normalization layer;

[0029] The fully connected layer transforms the data output from the Decoder layer into reconstructed traffic data.

[0030] Furthermore, step S40 specifically includes the following steps:

[0031] S401: Construct a time-domain encoder and a frequency-domain encoder to map the time-domain information and frequency-domain information of traffic data to the time-domain latent space and the frequency-domain latent space, respectively.

[0032] S402: Construct a denoised diffusion probability model in the time and frequency domain latent spaces, process the time and frequency domain latent representations by adding noise and predicting noise, and then reconstruct the original traffic data;

[0033] S403: By fusing the features of the time-domain latent representation and the frequency-domain latent representation, a time-frequency fusion latent representation is obtained, and the latent representation is decoded using a decoder to output the reconstructed traffic data.

[0034] Furthermore, step S50 specifically includes the following steps:

[0035] S501: Calculate the mean square error between the input flow data in the training set and the reconstructed flow data output by the model;

[0036] S502: Calculate the error between the noise added to the time-domain latent representation and the noise prediction value of the time-domain denoising diffusion probability model; calculate the error between the noise added to the frequency-domain latent representation and the noise prediction value of the frequency-domain denoising diffusion probability model, and sum the two errors calculated in this step;

[0037] S503: The errors calculated in S501 and S502 are weighted and summed, the model is trained using the training set, and the parameters of the entire model are adjusted.

[0038] Furthermore, step S60 specifically includes the following steps:

[0039] S601: The potential representation of the time-domain traffic data is obtained by the time-domain encoder, and noise is added to it. The time-domain denoising diffusion probability model is used for iterative denoising to finally obtain the time-domain potential representation.

[0040] S602: Perform FFT processing on the time-domain flow data to obtain frequency-domain data, encode the frequency-domain data and add noise, use the frequency-domain denoising diffusion probability model to perform iterative denoising, and obtain the frequency-domain latent representation;

[0041] S603: The latent representations in the time domain and frequency domain are fused to obtain the final feature vector;

[0042] S604: Input the final feature vector and the original traffic data into the decoder to obtain the reconstructed data;

[0043] S605: The outlier score is calculated by the mean square error between the sample and the reconstructed data.

[0044] S606: Based on the anomaly scores of the training and validation sets, the SPOT algorithm is used to dynamically calculate the anomaly detection threshold for the test set.

[0045] Furthermore, step S70 specifically includes the following steps:

[0046] S701: Calculate the anomaly score of the test set and determine whether each sample is abnormal traffic based on the set anomaly threshold;

[0047] S702: When the abnormal score is greater than the abnormal threshold, the sample is determined to be abnormal traffic; otherwise, it is normal traffic.

[0048] S703: Compare the anomaly detection results on the test set with the test set labels, and use precision, recall, and F1 score to validate the model.

[0049] Furthermore, step S80 specifically includes the following steps:

[0050] S801: Collects network traffic data from e-commerce platforms in real time and calculates anomaly scores using the trained model;

[0051] S802: Dynamically update the anomaly judgment threshold: Update the anomaly threshold based on the anomaly scores of historical traffic data and real-time traffic data;

[0052] S803: Calculate an anomaly score for each real-time traffic sample. If the anomaly score is greater than the anomaly threshold, it is determined to be abnormal traffic; otherwise, it is normal traffic.

[0053] The beneficial effects of this invention are: by fusing time-domain and frequency-domain features and dynamically adjusting the anomaly detection threshold for real-time monitoring, abnormal traffic can be efficiently identified, improving identification accuracy and optimizing detection performance. Attached Figure Description

[0054] Figure 1 This is a flowchart of the method for detecting traffic anomalies on e-commerce platforms based on a time-frequency latent diffusion model, as described in this invention.

[0055] Figure 2 This is a cluster architecture diagram of the e-commerce platform traffic anomaly detection method based on the time-frequency potential diffusion model of the present invention;

[0056] Figure 3 This is a schematic diagram of the time-domain encoder / frequency-domain encoder network structure of the e-commerce platform traffic anomaly detection method based on the time-frequency potential diffusion model of the present invention;

[0057] Figure 4 This is a schematic diagram of the time-domain latent representation / frequency-domain latent representation with added noise and prediction noise architecture of the e-commerce platform traffic anomaly detection method based on the time-frequency latent diffusion model of the present invention.

[0058] Figure 5 This is a schematic diagram of the decoder network structure of the e-commerce platform traffic anomaly detection method based on the time-frequency latent diffusion model of the present invention;

[0059] Figure 6 This is a schematic diagram of the network structure of the e-commerce platform traffic anomaly detection method based on the time-frequency latent diffusion model of the present invention. Detailed Implementation

[0060] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0061] The following will clearly and completely describe the concept, specific structure, and technical effects of the present invention in conjunction with embodiments and accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are all within the scope of protection of the present invention. Furthermore, all connections / linkages involved in the patent do not simply refer to direct contact between components, but rather to the ability to form a better connection structure by adding or reducing connecting accessories according to specific implementation conditions. The various technical features in this invention can be combined interactively without contradicting each other.

[0062] It should be noted that if the embodiments of the present invention involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of the components in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0063] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Secondly, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0064] See Figures 1-2 As shown, this invention provides a method for detecting traffic anomalies on e-commerce platforms based on a time-frequency latent diffusion model, including:

[0065] S10: Collect traffic datasets from e-commerce platforms, including traffic data under normal and abnormal conditions, and label abnormal traffic data.

[0066] S20: Normalize the traffic dataset and divide it into a training set, a validation set, and a test set;

[0067] S30: Perform a Fast Fourier Transform (FFT) on the traffic dataset to obtain the frequency domain representation of the traffic data;

[0068] S40: Construct a time-frequency potential diffusion model, including a time-domain encoder, a frequency-domain encoder, a time-domain denoising diffusion probability model, a frequency-domain denoising diffusion probability model, and a decoder, and perform training and optimization.

[0069] S50: Train the time-frequency potential diffusion model using the training set, adjust the network parameters in the model so that the model can accurately reconstruct traffic data and accurately predict the noise added to the potential representation in the time and frequency domains;

[0070] S60: Input the training set, validation set, and test set into the trained time-frequency latent diffusion model, calculate the anomaly score, and determine the anomaly threshold;

[0071] S70: Validate the time-frequency potential diffusion model using a test set, identify abnormal traffic through anomaly scores and thresholds, and evaluate model performance through validation metrics.

[0072] S80: Uses a time-frequency potential diffusion model to detect anomalies in real-time traffic data of e-commerce platforms, dynamically adjusts the anomaly judgment threshold, and determines in real time whether the traffic is abnormal.

[0073] In this invention, by combining time-domain and frequency-domain information, the time-frequency latent diffusion model can more comprehensively capture abnormal patterns in traffic data, especially in environments with large traffic fluctuations on e-commerce platforms. Through real-time training and dynamic threshold adjustment, the model can adapt to changes in e-commerce platform traffic, ensuring accurate identification of abnormal traffic under different traffic conditions. Optimizing the model using the training set enables it to accurately reconstruct traffic data and predict noise in the latent representation, thereby improving the accuracy of anomaly score calculation. This method not only enables anomaly detection of historical traffic in offline environments but also enhances the ability to identify unknown abnormal traffic, significantly reducing false positives and false negatives, providing a more intelligent and automated solution for network security protection of e-commerce platforms.

[0074] Furthermore, in step S10, the traffic data metrics include access count, visitor count, and page view count.

[0075] In this embodiment, two sets of e-commerce platform traffic data are planned to be collected: one set is the traffic data generated by the e-commerce platform under normal operating conditions; the other set includes the traffic data generated by the e-commerce platform under both normal and abnormal operating conditions, with abnormal traffic data marked to distinguish it from normal traffic. To obtain this data, traffic information of the e-commerce platform can be collected through the e-commerce platform's backend management system or relevant website traffic analysis tools, including key indicators such as access count, visitor count, and page views. Common e-commerce traffic indicator information is shown in Table 1:

[0076] Table 1 Common E-commerce Traffic Metrics

[0077] bounce rate Page visit duration Page views Visitor count

[0078] Furthermore, step S20 specifically includes the following steps:

[0079] S201: Normalize the collected traffic data by using the min-max normalization method to normalize each feature value to the range of [0,1].

[0080] S202: Divide the traffic dataset generated by the e-commerce platform under normal operating conditions into a training set and a validation set;

[0081] S203: Use the traffic dataset that includes traffic generated by the e-commerce platform under both normal and abnormal conditions as the test set, and provide corresponding labeled data for the test set as the test set's labels.

[0082] The collected traffic dataset is normalized, and for each sample x = { , ,…, Each feature in} After normalization using the following formula, we get ={ ,…, } :

[0083] = ;

[0084] in, Let n be the minimum value of the nth feature in the traffic dataset collected under normal operating conditions. This represents the maximum value of the nth feature in the traffic dataset collected during normal operation.

[0085] The frequency domain data of the flow rate is obtained by performing a Fast Fourier Transform on the time domain data. This is used to collect and process flow rate data X of duration L. Perform a Fourier transform on each dimension of the feature, for the d-th dimension... The following formula is used to transform and preserve the effective frequencies:

[0086] ;

[0087] in It is the complex value of the d-th dimension feature of the traffic data at the k-th frequency in the frequency domain;

[0088] For the transformed complex values = ,in and Representing the real and imaginary parts respectively, extracting the real and imaginary parts from each frequency component:

[0089] =

[0090] =

[0091] The real and imaginary parts of each calculated feature are concatenated along the corresponding feature dimension to obtain the result. Finally, the frequency domain data of the traffic is obtained via FFT(X).

[0092] like Figure 3 As shown, both the time-domain encoder and the frequency-domain encoder include:

[0093] Linear embedding layer is used to map traffic data to a low-dimensional space;

[0094] The positional coding layer is used to add positional information to the compressed representation. The positional information is implemented using sine and cosine functions.

[0095] The Transformer's Encoder layer, which includes a self-attention mechanism, a feedforward neural network, and a normalization layer, is used to learn the internal dependencies of the compressed representation and further extract key features and deeper feature representations of traffic data in the time or frequency domain.

[0096] Linear embedding layers effectively map traffic data to a low-dimensional space, reducing data complexity while preserving important feature information. Positional encoding layers, by introducing positional information, particularly using sine and cosine functions to encode the compressed representation, help the model understand the order and temporal relationships of the input data, improving its ability to capture temporal features. The Transformer's Encoder layer, utilizing self-attention, effectively captures long-term dependencies in the data, enabling the model to automatically learn key temporal and frequency domain features from traffic data.

[0097] Furthermore, such as Figure 4 As shown, both the temporal denoising diffusion probability model and the frequency domain denoising diffusion probability model are composed of a U-Net network. The U-Net network includes multiple convolutional layers and retains more detailed information through a symmetric encoder-decoder structure, effectively restoring the latent representation after noise addition.

[0098] like Figure 5 As shown, the decoder includes:

[0099] The linear embedding layer compresses traffic data into a low-dimensional space. Through linear transformation, the original traffic data representation can be mapped to a lower-dimensional space, thereby reducing computational complexity while maintaining the integrity of key features and avoiding interference from redundant information.

[0100] The Transformer's Decoder layer is used to reconstruct traffic data, and includes self-attention, cross-attention layers, feedforward neural networks, and normalization layers;

[0101] The fully connected layer transforms the data output from the Decoder layer into reconstructed traffic data.

[0102] The self-attention layer helps the model capture the dependencies within the input data and can perform weighted summation of features at each position to identify important temporal information or features. The cross-attention layer interacts with the encoder and decoder outputs, allowing the decoder to focus on the encoder's output and further enhance feature transmission. The feedforward neural network facilitates non-linear transformations at each position. The normalization layer helps accelerate the training process and ensures model training stability. Its input includes the output of the linear embedding layer and the aforementioned latent time-frequency fusion features. It fully utilizes the global contextual information of the traffic data, combined with latent representations capturing normal pattern features, to perform more appropriate reconstruction of the traffic data, improving the reconstruction quality of normal pattern traffic data while reducing the reconstruction quality of abnormal pattern traffic data. The fully connected layer takes the output of the Transformer's Decoder layer as input, and its output becomes the reconstructed traffic data.

[0103] Furthermore, such as Figure 6 As shown, step S40 specifically includes the following steps:

[0104] S401: Construct a time-domain encoder and a frequency-domain encoder to map the time-domain information and frequency-domain information of traffic data to the time-domain latent space and the frequency-domain latent space, respectively.

[0105] S402: Construct a denoised diffusion probability model in the time and frequency domain latent spaces, process the time and frequency domain latent representations by adding noise and predicting noise, and then reconstruct the original traffic data;

[0106] S403: By fusing the features of the time-domain latent representation and the frequency-domain latent representation, a time-frequency fusion latent representation is obtained, and the latent representation is decoded using a decoder to output the reconstructed traffic data.

[0107] Temporal encoders focus on extracting changes and dynamic information in data over time, making them suitable for capturing short-term fluctuations and time-series characteristics in traffic data. Frequency-domain encoders, on the other hand, are better able to handle periodic features and frequency information in traffic data, capturing long-term periodic changes or frequency-related anomalies. By processing time-domain and frequency-domain information separately, the model can extract features at different levels more effectively, thereby enhancing reconstruction quality.

[0108] In this embodiment, the temporal latent representation is denoised and the constructed model is used to predict the added noise;

[0109] The output TZ of the time-domain encoder is noise-added according to the following formula. :

[0110] = ;

[0111] in, For predefined hyperparameters, This is pure noise sampled from a standard normal distribution.

[0112] The added noise is predicted using a time-domain diffusion model, whose input is the noisy time-domain latent representation. Given the noisy time step T, the prediction result is expressed as ;

[0113] Add noise to the frequency domain latent representation and use the constructed model to predict the added noise;

[0114] The frequency domain encoder output FZ is noise-added using the following formula. :

[0115] = ;

[0116] in, For predefined hyperparameters, This is pure noise sampled from a standard normal distribution.

[0117] The added noise is predicted using a frequency-domain diffusion model, where the input to the frequency-domain diffusion model is the noisy latent representation. Given the noisy time step T, the prediction result is expressed as .

[0118] By fusing the time-frequency latent representation, the output TZ of the time-domain encoder and the output FZ of the frequency-domain encoder described in the first step are fused according to the following formula to obtain TFZ:

[0119] TFZ = TZ + FZ;

[0120] Fusing the temporal latent representation (TZ) and frequency latent representation (FZ) into a time-frequency fusion latent representation (TFZ) combines the advantages of both. This fusion not only captures the temporal characteristics of the data but also fully considers periodicity and frequency regularity. Real-world traffic data often contains noise, anomalies, or missing data. Through noise addition and denoising design, the model can better handle these imperfect data. This approach makes the model more stable during training and enhances its robustness in various scenarios.

[0121] Furthermore, step S50 specifically includes the following steps:

[0122] S501: Calculate the mean square error between the input flow data in the training set and the reconstructed flow data output by the model;

[0123] S502: Calculate the error between the noise added to the time-domain latent representation and the noise prediction value of the time-domain denoising diffusion probability model; calculate the error between the noise added to the frequency-domain latent representation and the noise prediction value of the frequency-domain denoising diffusion probability model; sum the two errors calculated in this step;

[0124] S503: The errors calculated in S501 and S502 are weighted and summed, the model is trained using the training set, and the parameters in the entire model are adjusted.

[0125] The model is trained based on the error between the traffic data input to the model and the reconstructed value output by the model, the error between the noise added to the time-domain latent representation and the noise prediction value of the time-domain denoising diffusion probability model, and the error between the noise added to the frequency-domain latent representation and the noise prediction value of the frequency-domain denoising diffusion probability model.

[0126] By adjusting the parameters of different network structures in the model, a model is obtained that can accurately reconstruct traffic data and accurately predict the noise added to the time-domain latent representation and the noise added to the frequency-domain latent representation.

[0127] The loss function for model training is defined by the following formula:

[0128] + ;

[0129] Where Loss is the loss function used for training, and X represents the traffic data after data preprocessing. This represents the reconstructed traffic data output by the encoder at the end of the model. Indicates hyperparameters, This represents the loss function of the temporal denoising diffusion probability model. Represents the loss function of the frequency domain denoising diffusion probability model.

[0130] ;

[0131] in, This represents the prediction noise of the time-domain denoised diffusion probability model. This represents the real noise added to the latent representation in the time domain.

[0132] ;

[0133] in, This represents the predicted noise of the frequency domain denoising diffusion probability model. This represents the real noise added to the potential representation in the frequency domain.

[0134] By calculating the noise prediction error of the latent representations in the time and frequency domains separately, the model can be optimized for the characteristics of each domain, resulting in more accurate noise removal and the recovery of more refined and natural features. The training loss function integrates the traffic data reconstruction error and the noise prediction error, allowing the model to focus more on the accuracy of the denoising process while ensuring the quality of data reconstruction, thereby improving the robustness and reliability of the model.

[0135] Next, step S60 specifically includes the following steps:

[0136] S601: The potential representation of the time-domain traffic data is obtained by the time-domain encoder, and noise is added to it. The time-domain denoising diffusion probability model is used for iterative denoising to finally obtain the time-domain potential representation.

[0137] S602: Perform FFT processing on the time-domain flow data to obtain frequency-domain data, encode the frequency-domain data and add noise, use the frequency-domain denoising diffusion probability model to perform iterative denoising, and obtain the frequency-domain latent representation;

[0138] S603: The latent representations in the time domain and frequency domain are fused to obtain the final feature vector;

[0139] S604: Input the final feature vector and the original traffic data into the decoder to obtain the reconstructed data;

[0140] S605: The outlier score is calculated by the mean square error between the sample and the reconstructed data.

[0141] S606: Based on the anomaly scores of the training and validation sets, the SPOT algorithm is used to dynamically calculate the anomaly detection threshold for the test set.

[0142] In this embodiment, after inputting the training set, validation set, and test set into the trained model, the anomaly score is calculated. The calculation of the anomaly score consists of the following steps:

[0143] Calculate the frequency domain data FFT(X) of the traffic using the preprocessed traffic time domain data X;

[0144] The temporal latent representation is obtained by encoding the temporal data using a temporal encoder. Noise is then added to the temporal latent representation TZ according to the following formula. :

[0145] = ;

[0146] in, For predefined hyperparameters, Pure noise sampled from a standard normal distribution;

[0147] The noise level at the corresponding denoising time step T is predicted using a temporal denoising diffusion probability model to represent the denoised temporal latent representation. And sample the next denoising time step T-1 using the following formula:

[0148] )+ z;

[0149] The sampled The following formula is used to add noise to obtain the sample. :

[0150] = ;

[0151] in, , , All are predefined hyperparameters, z, Sampling from a standard normal distribution;

[0152] right Similarly, the above noise reduction formula is used to obtain... ;

[0153] Continue to Obtain using the above noise addition formula ;

[0154] Repeat the above operation u times to obtain u samples in the feature space at time step T-1. , , , ..., }, calculate the information entropy H of these u samples and select the sample with the largest information entropy as the representative of the feature space in the denoising time step T-1, to participate in the next denoising space { , , , ..., Sampling calculations for}.

[0155] Iterate the above operations T times until the result is obtained. .

[0156] The frequency domain data is processed by a frequency domain encoder to obtain the frequency domain latent representation. Noise is then added to the frequency domain latent representation FZ according to the following formula. :

[0157] = ;

[0158] in, For predefined hyperparameters, Pure noise sampled from a standard normal distribution;

[0159] The noise level at the corresponding denoising time step T is predicted using a frequency domain denoising diffusion probability model to represent the latent frequency domain representation after denoising. And sample the next denoising time step T-1 using the following formula:

[0160] )+ z;

[0161] The sampled The following formula is used to add noise to obtain the sample. :

[0162] = ;

[0163] in, , , All are predefined hyperparameters, z, Sampling from a standard normal distribution;

[0164] right Similarly, the above noise reduction formula is used to obtain... ;

[0165] Continue to Obtain using the above noise addition formula ;

[0166] Repeat the above operation u times to obtain u samples in the feature space at time step T-1. , , , ..., }, calculate the information entropy H of these u samples and select the sample with the largest information entropy as the representative of the feature space in the denoising time step T-1, to participate in the next denoising space { , , , ..., Sampling calculations for}.

[0167] Iterate the above operations T times until the result is obtained. .

[0168] By adding noise and iteratively denoising the temporal latent representation and TFZ is obtained by feature fusion:

[0169] TFZ = + ;

[0170] Reconstructed data is obtained by inputting the fused feature TFZ and preprocessed traffic data X into the decoder. ;

[0171] The diffusion model for adding and denoising can simulate the true distribution of data and gradually recover the latent representation during the denoising process at multiple time steps, enabling the model to learn more stable and general features. During denoising, information entropy is used to select the optimal samples, ensuring that key information is not lost and improving denoising quality, resulting in more reliable extracted features.

[0172] The anomaly score for each traffic data sample is determined by the data sample x and the reconstructed data sample. The mean square error is determined by:

[0173] The above method is used to obtain the anomaly score (Train_Score) for the training set, the anomaly score (Valid_Score) for the validation set, and the anomaly score (Test_Score) for the test set. Based on the anomaly scores from the training and validation sets, the SPOT algorithm is used to dynamically determine the threshold for the test set. The core idea of ​​the SPOT algorithm is to dynamically set the threshold using the statistical characteristics of the data. As the data flow changes, the threshold is continuously updated to adapt to different network environments or application scenarios. This makes the SPOT algorithm particularly suitable for traffic monitoring and other scenarios that require real-time dynamic adjustment of thresholds.

[0174] ;

[0175] The SPOT algorithm is used to dynamically calculate the anomaly detection threshold, ensuring that the threshold is not fixed but continuously adjusted according to the statistical characteristics of the data. The SPOT algorithm can adapt to traffic distribution under different time periods and network conditions. Traditional fixed threshold methods often struggle to adapt to complex traffic scenarios, while the SPOT algorithm can improve detection accuracy while maintaining a low false alarm rate.

[0176] The specific steps of step S70 include:

[0177] S701: Calculate the anomaly score of the test set and determine whether each sample is abnormal traffic based on the set anomaly threshold;

[0178] S702: When the abnormal score is greater than the abnormal threshold, the sample is determined to be abnormal traffic; otherwise, it is normal traffic.

[0179] S703: Compare the anomaly detection results on the test set with the test set labels, and use precision, recall, and F1 score to validate the model.

[0180] Using the obtained anomaly scores in the test set, anomaly thresholds are determined to identify abnormal traffic in the test set; for each sample x in the test set, anomaly score s is calculated, and when s > 0. If the anomaly detection result is not found, the sample x is determined to be an abnormal traffic data sample; otherwise, it is considered a normal traffic data sample. The anomaly detection results of the test set are combined with the label data of the test set to verify the effectiveness of the model. The verification metrics are affiliation-precision, affiliation-recall, and affiliation-F1.

[0181] The specific steps of step S80 include:

[0182] S801: Collects network traffic data from e-commerce platforms in real time and calculates anomaly scores using the trained model;

[0183] S802: Dynamically update the anomaly judgment threshold: Update the anomaly threshold based on the anomaly scores of historical traffic data and real-time traffic data;

[0184] S803: Calculate an anomaly score for each real-time traffic sample. If the anomaly score is greater than the anomaly threshold, it is determined to be abnormal traffic; otherwise, it is normal traffic.

[0185] Furthermore, real-time network traffic data generated by e-commerce platforms is collected, and the trained model is sampled to calculate anomaly scores. The anomaly detection threshold is dynamically updated based on the anomaly scores from historical traffic data and the real-time collected traffic data.

[0186] ;

[0187] in, This indicates the score for abnormal historical traffic data. This indicates the score for abnormal traffic data collected in real time.

[0188] For each sample in the real-time collected traffic data abnormal scores ,when > When determining the sample If the data is abnormal, it is considered normal traffic data; otherwise, it is considered normal traffic data. The above steps are performed on each sample data to complete real-time anomaly detection of the traffic data.

[0189] By adjusting thresholds based on anomaly scores from both historical and real-time traffic data, the differences between historical patterns and current traffic can be balanced to some extent. This approach effectively avoids relying solely on fixed patterns derived from historical data, while also preventing over-reliance on localized fluctuations in current traffic. Utilizing anomaly scores from historical data provides a reliable reference for real-time traffic data, helping to determine the presence of abnormal traffic. E-commerce platforms typically face significant traffic fluctuations, especially during peak periods such as promotions and holidays. This method, through real-time detection and dynamic adjustment of anomaly detection thresholds, ensures that the system can accurately identify abnormal data even in high-traffic environments.

[0190] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

[0191] Finally, it should be noted that the above is a detailed description of the preferred embodiments of the present invention, but the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A method for detecting traffic anomalies on e-commerce platforms based on a time-frequency latent diffusion model, characterized in that, include: S10: Collect traffic datasets from e-commerce platforms, including traffic data under normal and abnormal conditions, and label abnormal traffic data. S20: Normalize the traffic dataset and divide it into a training set, a validation set, and a test set; S30: Perform a fast Fourier transform on the traffic dataset to obtain the frequency domain representation of the traffic data; S40: Construct a time-frequency potential diffusion model, including a time-domain encoder, a frequency-domain encoder, a time-domain denoising diffusion probability model, a frequency-domain denoising diffusion probability model, and a decoder, and perform training and optimization. S50: Train the time-frequency potential diffusion model using the training set and adjust the network parameters in the model; S60: Input the training set, validation set, and test set into the trained time-frequency latent diffusion model, calculate the anomaly score, and determine the anomaly threshold; Specifically, when calculating the anomaly score, the time-domain denoising diffusion probability model and the frequency-domain denoising diffusion probability model are used to iteratively denoise the noisy time-domain latent representation and the frequency-domain latent representation, respectively. During the iterative denoising process, for each denoising time step, the following operations are performed: the denoising and denoising operations are cyclically executed a preset number of times to generate multiple candidate latent representation samples for that time step, the information entropy of these samples is calculated, and the sample with the largest information entropy is selected as the latent representation for that time step to participate in the denoising calculation of the next time step. S70: Validate the time-frequency potential diffusion model using a test set, identify abnormal traffic through anomaly scores and anomaly thresholds, and evaluate model performance through validation metrics. S80: Use a trained time-frequency latent diffusion model to detect anomalies in real-time traffic data of e-commerce platforms, dynamically adjust the anomaly judgment threshold, and determine in real time whether the traffic is abnormal.

2. The method for detecting traffic anomalies on e-commerce platforms based on a time-frequency latent diffusion model according to claim 1, characterized in that, In step S10, the traffic data metrics include the number of visits, the number of visitors, and the number of page views.

3. The method for detecting traffic anomalies on e-commerce platforms based on a time-frequency latent diffusion model according to claim 1, characterized in that, The specific steps of step S20 include: S201: Normalize the collected traffic data by using the min-max normalization method to normalize each feature value to the range of [0,1]. S202: Divide the traffic dataset generated by the e-commerce platform under normal operating conditions into a training set and a validation set; S203: Use the traffic dataset that includes traffic generated by the e-commerce platform under both normal and abnormal conditions as the test set, and provide corresponding labeled data for the test set as the test set's labels.

4. The method for detecting traffic anomalies on e-commerce platforms based on a time-frequency latent diffusion model according to claim 1, characterized in that, Both the time-domain encoder and the frequency-domain encoder include: Linear embedding layer is used to map traffic data to a low-dimensional space; The positional encoding layer is used to add positional information to the compressed representation; The Transformer's Encoder layer contains a self-attention mechanism, a feedforward neural network, and a normalization layer, used to learn feature representations of time-domain or frequency-domain data. Both the temporal denoising diffusion probability model and the frequency denoising diffusion probability model are composed of a U-Net network. The U-Net network includes multiple convolutional layers and retains more detailed information through a symmetric encoder-decoder structure, effectively restoring the latent representation after noise addition.

5. The method for detecting traffic anomalies on e-commerce platforms based on a time-frequency latent diffusion model according to claim 1 is characterized in that, The decoder includes: The linear embedding layer compresses the traffic data representation into a low-dimensional space; The Transformer's Decoder layer is used to reconstruct traffic data, including self-attention. Cross-attention layer, feedforward neural network, and normalization layer; The fully connected layer transforms the data output from the Decoder layer into reconstructed traffic data.

6. The method for detecting traffic anomalies on e-commerce platforms based on a time-frequency latent diffusion model according to claim 1, characterized in that, The specific steps of step S40 include: S401: Construct a time-domain encoder and a frequency-domain encoder to map the time-domain information and frequency-domain information of traffic data to the time-domain latent space and the frequency-domain latent space, respectively. S402: Construct a denoised diffusion probability model in the time and frequency domain latent spaces, process the time and frequency domain latent representations by adding noise and predicting noise, and then reconstruct the original traffic data; S403: By fusing the features of the time-domain latent representation and the frequency-domain latent representation, a time-frequency fusion latent representation is obtained, and the latent representation is decoded using a decoder to output the reconstructed traffic data.

7. The method for detecting traffic anomalies on e-commerce platforms based on a time-frequency latent diffusion model according to claim 1, characterized in that, The specific steps of step S50 include: S501: Calculate the mean square error between the input flow data in the training set and the reconstructed flow data output by the model; S502: Calculate the error between the noise added to the time-domain latent representation and the noise prediction value of the time-domain denoising diffusion probability model; calculate the error between the noise added to the frequency-domain latent representation and the noise prediction value of the frequency-domain denoising diffusion probability model; sum the two errors calculated in this step; S503: The error calculated in S501 and the error calculated in S502 are weighted and summed, the model is trained using the training set, and the parameters of the entire model are adjusted.

8. The method for detecting traffic anomalies on e-commerce platforms based on a time-frequency latent diffusion model according to claim 1, characterized in that, The specific steps of step S60 include: S601: The potential representation of the time-domain traffic data is obtained by the time-domain encoder, and noise is added to it. The time-domain denoising diffusion probability model is used for iterative denoising to finally obtain the time-domain potential representation. S602: Perform FFT processing on the time-domain flow data to obtain frequency-domain data, encode the frequency-domain data and add noise, use the frequency-domain denoising diffusion probability model to perform iterative denoising, and obtain the frequency-domain latent representation; S603: The latent representations in the time domain and frequency domain are fused to obtain the final feature vector; S604: Input the final feature vector and the original traffic data into the decoder to obtain the reconstructed data; S605: The outlier score is calculated by the mean square error between the sample and the reconstructed data; S606: Based on the anomaly scores of the training and validation sets, the SPOT algorithm is used to dynamically calculate the anomaly detection threshold for the test set.

9. The method for detecting traffic anomalies on e-commerce platforms based on a time-frequency latent diffusion model according to claim 1, characterized in that, The e-commerce platform traffic anomaly detection method based on the time-frequency potential diffusion model according to claim 1, wherein step S70 specifically includes: S701: Calculate the anomaly score of the test set and determine whether each sample is abnormal traffic based on the set anomaly threshold; S702: When the abnormal score is greater than the abnormal threshold, the sample is determined to be abnormal traffic; otherwise, it is normal traffic. S703: Compare the anomaly detection results on the test set with the test set labels, and use precision, recall, and F1 score to validate the model.

10. The method for detecting traffic anomalies on e-commerce platforms based on a time-frequency latent diffusion model according to claim 1, characterized in that, The specific steps of step S80 include: S801: Collects network traffic data from e-commerce platforms in real time and calculates anomaly scores using the trained model; S802: Dynamically update the anomaly judgment threshold: Update the anomaly threshold based on the anomaly scores of historical traffic data and real-time traffic data; S803: Calculate an anomaly score for each real-time traffic sample. If the anomaly score is greater than the anomaly threshold, it is determined to be abnormal traffic; otherwise, it is normal traffic.

Images