A network traffic prediction method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose a network traffic prediction method and device, electronic equipment and storage medium. The method comprises: decomposing obtained first network traffic data into first data and second data, performing Fourier transform processing on the second data to obtain third data representing frequency domain representation of the second data; the first data represents long-term trend characteristics of the first network traffic data, and the second data represents periodic fluctuation characteristics of the first network traffic data; performing encoding processing on the first network traffic data through a transformer encoder to obtain fourth data; performing inverse Fourier transform processing on at least part of the third data to obtain fifth data representing time domain representation of the at least part of the data; performing feature fusion processing on the first data, the fourth data and the fifth data, and performing decoding processing on the data after the fusion processing based on a transformer decoder to obtain predicted second network traffic data.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically to a network traffic prediction method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the rapid development of networks, the services and applications carried on them are becoming increasingly diverse. Strengthening network management and construction remains a challenge for operators. Effectively improving network speed and utilization hinges on network traffic prediction. Predicting network traffic overload and analyzing and resolving issues before it occurs can significantly improve network and service availability. However, current network traffic prediction models still suffer from insufficient prediction accuracy. Summary of the Invention

[0003] To address the existing technical problems, embodiments of the present invention provide a network traffic prediction method, apparatus, electronic device, and storage medium.

[0004] To achieve the above objectives, the technical solution of this invention is implemented as follows:

[0005] This invention provides a network traffic prediction method, which is applied in an electronic device; the method includes:

[0006] The obtained first network traffic data is decomposed into first data and second data. The second data is subjected to Fourier transform processing to obtain third data that represents the frequency domain representation of the second data. The first data represents the long-term trend characteristics of the first network traffic data, and the second data represents the periodic fluctuation characteristics of the first network traffic data.

[0007] The first network traffic data is encoded using a transformer encoder to obtain the fourth data.

[0008] At least a portion of the third data is subjected to inverse Fourier transform to obtain a fifth data representing the time-domain representation of the at least a portion of the data;

[0009] The first data, the fourth data, and the fifth data are subjected to feature fusion processing. The fused data is then decoded using a transformer decoder to obtain the predicted second network traffic data.

[0010] In the above scheme, performing inverse Fourier transform processing on at least a portion of the third data includes: processing the third data through an attention model to determine the data to be retained and / or the data to be zeroed in the third data;

[0011] The data retained in the third set of data is subjected to inverse Fourier transform.

[0012] In the above scheme, the step of processing the third data using an attention model to determine the data to be retained and / or the data to be zeroed out includes:

[0013] The attention model is used to assign a corresponding attention value to each data point in the third data.

[0014] Based on the attention value corresponding to each data point in the third data, determine the data to be retained in the first part and / or the data to be zeroed out;

[0015] The step of performing inverse Fourier transform processing on the data retained in the third data includes: performing inverse Fourier transform processing on the data retained in the first part.

[0016] In the above scheme, determining the first part of retained data and / or zeroed data based on the attention value corresponding to each data point in the third data includes:

[0017] When the attention value corresponding to the data is greater than or equal to the first threshold, the data is determined to be retained;

[0018] When the attention value corresponding to the data is less than the first threshold, the data is determined to be zero.

[0019] In the above scheme, the step of processing the third data using an attention model to determine the data to be retained and / or the data to be zeroed out includes:

[0020] The attention model is used to assign a corresponding attention value to each data point in the third data.

[0021] Based on the attention value corresponding to each data point in the third data, determine the data to be retained in the first part and / or the data to be zeroed out;

[0022] The neural network units in the attention model are deactivated with a first probability. Based on the deactivated neural network units, the data retained in the first part of the retained data and / or the data that are returned to zero are determined to obtain the second part of the retained data.

[0023] The step of performing inverse Fourier transform processing on the data retained in the third data includes: performing inverse Fourier transform processing on the data retained in the second part.

[0024] In the above scheme, before performing inverse Fourier transform processing on at least a portion of the third data, the method further includes:

[0025] Missing data in the data retained after processing by the attention model is filled in.

[0026] In the above scheme, the step of decomposing the first network traffic data into first data and second data includes: performing average pooling processing on the first network traffic data, and obtaining the first data based on the first network traffic data after average pooling processing.

[0027] The second data is obtained by subtracting the first network traffic data from the first data.

[0028] In the above scheme, the feature fusion processing of the first data, the fourth data, and the fifth data includes: summing the first data, the fourth data, and the fifth data to perform feature fusion.

[0029] This invention also provides a network traffic prediction device, the device comprising: an encoder and a decoder; the encoder comprising a Fourier encoder and a transformer encoder; the decoder comprising a Fourier decoder and a transformer decoder; wherein,

[0030] The Fourier encoder is used to decompose the obtained first network traffic data into first data and second data, and to perform Fourier transform processing on the second data to obtain third data that represents the frequency domain representation of the second data; the first data represents the long-term trend characteristics of the first network traffic data, and the second data represents the periodic fluctuation characteristics of the first network traffic data.

[0031] The transformer encoder is used to encode the first network traffic data to obtain the fourth data;

[0032] The Fourier decoder is used to perform inverse Fourier transform processing on at least a portion of the third data to obtain fifth data representing the time-domain representation of the at least a portion of the data; it is also used to perform feature fusion processing on the first data, the fourth data, and the fifth data;

[0033] The transformer decoder is used to decode the data fused by the Fourier decoder to obtain the predicted second network traffic data.

[0034] In the above scheme, the Fourier encoder includes a decomposition module and a Fourier transform module; the Fourier decoder includes an inverse Fourier transform module and a feature vector fusion module; wherein,

[0035] The decomposition module is used to decompose the obtained first network traffic data into the first data and the second data, send the first data to the feature vector fusion module, and send the second data to the Fourier transform module.

[0036] The Fourier transform module is used to perform Fourier transform processing on the second data to obtain third data that characterizes the frequency domain representation of the second data;

[0037] The inverse Fourier transform module is used to perform inverse Fourier transform processing on at least a portion of the third data to obtain the fifth data, which is a time-domain representation of the at least a portion of the data, and send the fifth data to the feature vector fusion module.

[0038] The feature vector fusion module is used to perform feature fusion processing on the first data, the fourth data, and the fifth data.

[0039] In the above scheme, the device further includes: a frequency domain component selection module, used to process the third data through an attention model, determine the data to be retained and / or the data to be returned to zero in the third data, and send the data to be retained in the third data to the Fourier decoder for inverse Fourier transform processing.

[0040] In the above scheme, the frequency domain component selection module is used to assign a corresponding attention value to each data in the third data through an attention model; based on the attention value corresponding to each data in the third data, the first part of the retained data and / or the data returned to zero is determined, and the first part of the retained data is sent to the Fourier decoder for inverse Fourier transform processing.

[0041] In the above scheme, the frequency domain component selection module is used to assign a corresponding attention value to each data in the third data through an attention model; based on the attention value corresponding to each data in the third data, determine the first part of retained data and / or zeroed data; deactivate the neural network unit in the attention model with a first probability; determine the retained data and / or zeroed data in the first part of retained data based on the deactivated neural network unit to obtain the second part of retained data; and send the second part of retained data to the Fourier decoder for inverse Fourier transform processing.

[0042] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the network traffic prediction method described in this invention.

[0043] This invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the network traffic prediction method described in this invention.

[0044] The network traffic prediction method, apparatus, electronic device, and storage medium provided in this invention include: an electronic device decomposing acquired first network traffic data into first data and second data; performing Fourier transform processing on the second data to obtain third data representing the frequency domain representation of the second data; the first data representing the long-term trend characteristics of the first network traffic data, and the second data representing the periodic fluctuation characteristics of the first network traffic data; encoding the first network traffic data using a transformer encoder to obtain fourth data; performing inverse Fourier transform processing on at least a portion of the third data to obtain fifth data representing the time domain representation of the at least a portion of the data; performing feature fusion processing on the first data, the fourth data, and the fifth data; and decoding the fused data using a transformer decoder to obtain predicted second network traffic data. By decomposing the first network traffic data into first data representing long-term trend characteristics and second data representing periodic fluctuation characteristics during the encoding stage, global spectral information can be added to the model, enabling better modeling of long-term network traffic data and thus improving the prediction accuracy of network traffic. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the network traffic prediction device structure and data flow direction according to an embodiment of the present invention;

[0046] Figure 2 This is a flowchart illustrating the network traffic prediction method according to an embodiment of the present invention. Figure 1 ;

[0047] Figure 3 This is a flowchart illustrating the network traffic prediction method according to an embodiment of the present invention. Figure 2 ;

[0048] Figure 4 This is a schematic diagram of the frequency domain component selection module in the network traffic prediction device according to an embodiment of the present invention;

[0049] Figure 5 This is a schematic diagram of the hardware composition structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0050] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0051] The technical solutions of this invention can be applied to various communication systems, such as GSM (Global System of Mobile communication), LTE (Long Term Evolution), or 5G systems. Optionally, a 5G system or 5G network can also be referred to as a New Radio (NR) system or NR network.

[0052] It should be understood that devices with communication functions in the network / system of this application embodiment can be referred to as communication devices. Communication devices may include network devices and terminals with communication functions. Network devices and terminal devices can be the specific devices described above, which will not be repeated here. Communication devices may also include other devices in the communication system, such as network controllers, mobility management entities, and other network entities. This embodiment of the present invention does not limit these.

[0053] It should be understood that the terms "system" and "network" are often used interchangeably in this document. The term "and / or" in this document merely describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Furthermore, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0054] The terms “first,” “second,” etc., used in the specification and claims of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0055] In traditional time prediction models, the encoder represents the input sequence (x1,...x) as symbols. n The mapping is to a continuous representation sequence (z1,...z2) n Given z, the decoder generates an output sequence (y1,...y) of one symbol at a time. nIn each step, the model is autoregressive, taking the previously generated symbol sequence as additional input when generating the next step. This results in predictions for each time step being made separately and independently. Therefore, traditional time-based prediction models cannot maintain the global properties of time series and cannot treat statistical data as a whole, leading to insufficient prediction accuracy of network traffic.

[0056] Figure 1 This is a schematic diagram of the network traffic prediction device structure and data flow direction according to an embodiment of the present invention; as shown. Figure 1 As shown, the network traffic prediction device of this embodiment includes: an encoder 11 and a decoder 12; the encoder 11 includes a Fourier encoder 111 and a transformer encoder 112; the decoder 12 includes a Fourier decoder 121 and a transformer decoder 122.

[0057] Optionally, the Fourier encoder 111 includes a decomposition module 1111 and a Fourier transform module 1112; the Fourier decoder 121 includes an inverse Fourier transform module 1211 and a feature vector fusion module 1212. In the encoding stage, the decomposition module 1111 decomposes the first network traffic data into first data and second data, where the first data represents the long-term trend characteristics of the first network traffic data, and the second data represents the periodic fluctuation characteristics of the first network traffic data. The Fourier transform module 1112 performs Fourier transform processing on the second data, mapping the second data from the time domain to the frequency domain, thereby obtaining third data representing the frequency domain component of the second data. In the decoding stage, the inverse Fourier transform module 1211 performs inverse Fourier transform processing on at least a portion of the third data, mapping at least a portion of the third data from the frequency domain to the time domain, obtaining fifth data representing the time domain component of the at least a portion of the data. The feature vector fusion module 1212 performs feature fusion on the received first data, fourth data, and fifth data, and then the transformer decoder 122 performs decoding processing on the feature-fused data. This approach adds global spectral information to the model, enabling better modeling of long-term network traffic data and thus improving the accuracy of network traffic prediction.

[0058] Optionally, the device may further include a frequency domain component selection module 13, used to process the third data output by the Fourier encoding module through an attention model (or attention layer), zeroing out data that is not helpful for prediction and retaining data that is helpful for prediction. The combination of the inverse Fourier transform module in the Fourier decoder 121 for time-domain to frequency-domain mapping processing of the data, and the feature vector fusion module 1212 for fusion processing of the data, makes this approach suitable for time series prediction. This is because some trend changes in the time series are related to important events, and each frequency domain component may contain information about the predicted trend. Compared to retaining low-frequency components and discarding high-frequency components to represent the time series, this embodiment of the invention is more suitable for time series prediction, preserving as much information as possible that contains the predicted trend, thereby improving the accuracy of network traffic prediction.

[0059] Figure 2 This is a flowchart illustrating the network traffic prediction method according to an embodiment of the present invention. Figure 1 ,like Figure 2 As shown, the method includes:

[0060] Step 101: Decompose the obtained first network traffic data into first data and second data, perform Fourier transform processing on the second data to obtain third data that represents the frequency domain representation of the second data; the first data represents the long-term trend characteristics of the first network traffic data, and the second data represents the periodic fluctuation characteristics of the first network traffic data.

[0061] Step 102: Encode the first network traffic data using a transformer encoder to obtain the fourth data;

[0062] Step 103: Perform inverse Fourier transform on at least a portion of the third data to obtain the fifth data representing the time-domain representation of the at least a portion of the data;

[0063] Step 104: Perform feature fusion processing on the first data, the fourth data, and the fifth data, and decode the fused data using a transformer decoder to obtain the predicted second network traffic data.

[0064] The network traffic prediction method of this embodiment is applied to an electronic device, which may be a personal device or server with data processing capabilities. In this embodiment, the network traffic data (such as first network traffic data, second network traffic data, etc.) refers to the amount of data transmitted over the network. For example, this network traffic data can be represented by access volume, click volume, etc., such as access volume and click volume for a website or application (APP). In this embodiment, the first network traffic data is specifically historical network traffic data obtained in advance, such as access volume and click volume over a past period. The second network traffic data is network traffic data predicted based on the first network traffic data, such as predicted access volume and click volume over a future period. It can be considered that both the first network traffic data and the second network traffic data can be called long-term network traffic data, that is, the first network traffic data may include network traffic values ​​(such as access volume, click volume, etc.) corresponding to multiple historical moments (or historical time points) within a past period, and the second network traffic data may include network traffic values ​​(such as access volume, click volume, etc.) corresponding to multiple future moments (or future time points) within a future period, etc.

[0065] Combination Figure 1 As shown, in this embodiment, the obtained first network traffic data can be decomposed into first data and second data through a decomposition module. The first data represents the long-term trend characteristics of the first network traffic data, and the second data represents the periodic fluctuation characteristics of the first network traffic data. In other optional embodiments, the first data can also be called a long-term trend term, representing the development trend of traffic data in the time domain; the second data can also be called a periodic fluctuation term, representing the change trend of traffic data in the frequency domain. Further, the first data is input into the feature vector fusion module in the Fourier decoder, and the second data is input into the Fourier transform module.

[0066] For example, the first network traffic data can be generated by multiple... The data composition, or in other words, the first network traffic data can be multiple, such as... The data can form the first network traffic data. Based on this, by decomposing the first network traffic data, it can be decomposed into... Data in the form of, then, b represents the second data point indicating cyclical fluctuation characteristics, and b represents the first data point indicating long-term trend characteristics.

[0067] In some optional embodiments, decomposing the first network traffic data into first data and second data includes: performing average pooling processing on the first network traffic data to obtain the first data based on the average pooled first network traffic data; and performing a difference operation between the first network traffic data and the first data to obtain the second data. In other optional embodiments, the first network traffic data after average pooling processing may also be padded (e.g., with zeros) with reference to the first network traffic data before processing to maintain the consistency of data length.

[0068] Specifically, if the first network traffic data is represented by vector x, then the first data can be represented as:

[0069] x t =Padding(AvgPool(x)) (1)

[0070] Where, x t The first data is represented by "Padding", which indicates padding of the first network traffic data, and "AvgPool" indicates average pooling. For example, the padding process could be to maintain the same scale as the input item (such as the first network traffic data) and symmetrically pad with zeros on both sides.

[0071] Accordingly, the second data can be represented as:

[0072] x s =xx t (2)

[0073] Where, x s This refers to the second data.

[0074] In this embodiment, the second data is processed by a Fourier transform module to project the second data from the input sequence in the time domain to the frequency domain. Specifically, the third data can be represented as:

[0075] X s (ω)=∑ n x(n)e -iωn (3)

[0076] Among them, X s (ω) represents the third data, and x(n) is x s This represents the nth value in the second set of data.

[0077] As can be seen from the above expression (3), the frequency domain representation of the periodic fluctuation term (i.e., the third data) is composed of a finite number of periodic functions of different frequencies, including a series of lower frequencies and a series of higher frequencies.

[0078] On the other hand, in this embodiment of the invention, the input first network traffic data is encoded using a transformer encoder to obtain the fourth data. It should be noted that the execution order of steps 101 and 102 is not limited. Step 101 can be executed first, followed by step 102, or vice versa, with step 102 executed first and then step 101. Steps 101 and 102 can also be executed simultaneously. This embodiment does not impose any limitations on this.

[0079] The network traffic prediction method in this embodiment can be implemented based on the improved transformer model. The transformer model is essentially an encoder-decoder structure. That is, this embodiment can improve the transformer model by adding a Fourier encoder (including a decomposition module and a Fourier transform module) to the encoder and a Fourier decoder (including an inverse Fourier transform module and a feature vector fusion module) to the decoder.

[0080] In specific implementations, the transformer encoder and transformer decoder are the encoder and decoder in a conventional transformer model. For example, the transformer encoder may include a multi-layer encoder structure, and correspondingly, the transformer decoder may include a multi-layer decoder structure. Each transformer encoder may have two sub-layers: an attention layer and a feed-forward network (FFN). The attention layer may be, for example, a multi-head attention layer, a self-attention layer, etc. Each transformer encoder in the multi-layer encoder included in the transformer encoder has the same structure, using different weight parameters. The transformer decoder includes the two sub-layers of the transformer encoder and also includes another attention layer.

[0081] In some optional embodiments, the present invention performs inverse Fourier transform processing on at least a portion of the third data using an inverse Fourier transform module to obtain fifth data representing the time-domain components of the at least a portion of the data; then, a feature vector fusion module performs feature fusion processing on the first data, the fourth data, and the fifth data, and the data after feature fusion processing is recorded as the sixth data; the sixth data is then decoded by a transformer decoder to output the predicted second network traffic data.

[0082] In some alternative embodiments, the third data output by the Fourier coding module can be processed by an attention model (or attention layer) through a frequency domain component selection module in the device. Data that is not helpful in predicting the data is reset to zero, while data that is helpful in predicting the data is retained, thereby improving the accuracy of network traffic prediction. Based on this, such as... Figure 3 As shown, step 103 above may include:

[0083] Step 1031: Process the third data using an attention model to determine the data to be retained and / or the data to be zeroed out in the third data;

[0084] Step 1032: Perform inverse Fourier transform on the data retained in the third data to obtain the fifth data representing the time-domain components;

[0085] In this embodiment, the attention model can also be called an attention layer, and the attention model (or attention layer) is a fully connected layer structure. The third data is processed through this attention model to filter it, retaining data that is helpful for prediction and zeroing out (or deleting) data that is of little or no help to prediction. Further, the retained data undergoes an inverse Fourier transform to obtain the fifth data.

[0086] In some optional embodiments of the present invention, the step of processing the third data through an attention model to determine the data to be retained and / or the data to be zeroed in the third data includes: assigning a corresponding attention value to each data in the third data through an attention model; determining a first portion of the data to be retained and / or the data to be zeroed based on the attention value corresponding to each data in the third data; correspondingly, the step of performing inverse Fourier transform processing on the data to be retained in the third data includes: performing inverse Fourier transform processing on the first portion of the data to be retained.

[0087] In this embodiment, attention values ​​(or attention weights) are assigned to the data in the third data through the attention model (or attention layer), so that data that is of great help to the prediction data are assigned large weights, data that is of little help to the prediction data are assigned small weights, and the data to be retained and / or data to be zeroed in the third data are determined according to the assigned weights. Data that is of no help or of very little help to the prediction data are zeroed, and data that is helpful to the prediction data are retained.

[0088] Specifically, in this embodiment, the backpropagation mechanism of a neural network can be used to analyze which frequency domain data are helpful for the final network traffic prediction output, which are noisy data, and which frequency data are less helpful for predicting future network traffic data. This allows for the optimization of an attention table (or attention set), which may include multiple frequency domain data and the attention value (or attention weight) corresponding to each frequency domain data. Further, the fifth data output by the frequency domain component selection module can be obtained according to the following expression:

[0089] Y = WX s +B (4)

[0090] Where Y is the set of selected data output by the frequency domain component selection module (i.e., the data output by the frequency domain component selection module), W represents the set of attention values ​​(or attention weights), and X... s B represents the data output by the Fourier transform module (i.e., the third data), and B is the offset set. In some optional embodiments, expression (4) can be used as the structure of a fully connected layer of an attention model (or attention layer). Optionally, the offsets for different frequency domain data (i.e., the third data) can be pre-optimized to obtain the offset set B; in other optional embodiments, the offset set may not be required, i.e., the offset for each frequency domain data (i.e., the third data) is 0.

[0091] In some optional embodiments, determining the first portion of retained data and / or zeroed data based on the attention value corresponding to each data in the third data includes: determining that data is retained when the attention value corresponding to the data is greater than or equal to a first threshold; and determining that data is zeroed when the attention value corresponding to the data is less than the first threshold.

[0092] In this embodiment, by setting a first threshold related to the attention value (or attention weight), data with an attention value (or attention weight) lower than the first threshold are reset to zero, while data with an attention value (or attention weight) greater than or equal to the first threshold are retained. This removes data that is of little or no help to the prediction data, thereby reducing computational complexity, speeding up data processing, and improving data processing efficiency.

[0093] Specifically, the following expression can be used to determine whether data is retained and / or zeroed out:

[0094]

[0095] Among them, X s (ω) represents the frequency domain data set of the third data, W n Represents the set of attention values ​​(or attention weights); The same meaning as expression (3).

[0096] This embodiment of the invention uses an attention mechanism to assign greater weight to frequency data that is more helpful in predicting future data, and less weight to frequency data that is less helpful in predicting future data. This is more conducive to focusing the attention of the network computation on the frequencies that are more helpful in predicting future data, which can improve the accuracy of future data prediction. Furthermore, the frequency domain component selection module 13 inputs the first part of the retained data into the inverse Fourier transform module 1211 for inverse Fourier transform processing.

[0097] In some optional embodiments of the present invention, the step of processing the third data through an attention model to determine the data to be retained and / or the data to be zeroed in the third data includes: assigning a corresponding attention value to each data in the third data through an attention model; determining a first part of the data to be retained and / or the data to be zeroed based on the attention value corresponding to each data in the third data; deactivating the neural network units in the attention model with a first probability; and determining the data to be retained and / or the data to be zeroed in the first part of the data to be retained based on the deactivated neural network units to obtain a second part of the data to be retained; the step of performing inverse Fourier transform processing on the data to be retained in the third data includes: performing inverse Fourier transform processing on the second part of the data to be retained.

[0098] In this embodiment, after determining the first portion of retained data as described in the previous embodiments, a dropout layer is added after the attention model (or attention layer) to increase the model's generalizability and robustness. Figure 4 As shown, by removing (dropout) layers to deactivate or discard neural network units in the attention model with a certain probability (such as a first probability), the length of the input data (represented as a vector, which can be called the input vector) is greatly reduced, thus lowering the computational complexity. Although this process results in a loss of input information, experiments show that this loss has little impact on the final accuracy. This is because signals are generally sparser in the frequency domain than in the time domain. Furthermore, a large amount of information in the high-frequency region is so-called noise, which can often be discarded in time series prediction problems because noise often represents randomly generated parts and is therefore unpredictable. In contrast, in the image domain, the "noise" in the high-frequency region may represent image details and cannot be ignored.

[0099] Specifically, the above processing can be represented by the following expression:

[0100] r i(l) ~Bernoulli(p) (6)

[0101]

[0102]

[0103] Here, expression (6) represents a Bernoulli distribution with probability p, that is, according to the set probability p, the data is randomly set to 0 with probability p following a Bernoulli distribution, in order to deactivate the neural network units of the attention model (or attention layer), r (l) Let r represent the zeroing vector randomly generated following a Bernoulli distribution. Expression (7) represents the frequency domain data processed based on the deactivated neural network units, r (l) With X s Perform point multiplication on X s Some positions are set to zero; since neural network units can be deactivated with probability p, it is possible that r (l) The case where X equals 0; s X is the data retained after weight allocation and threshold selection. s After r (l) The processing yielded The result could be 0, meaning it could be discarded or retained. The retained data is processed by expression (8) to obtain the data output by the frequency domain component selection module 13 (i.e., the second part of the retained data). In some alternative embodiments, expression (8) can serve as the structure of a fully connected layer of another attention model (or attention layer).

[0104] In some optional embodiments of the present invention, before performing inverse Fourier transform processing on at least a portion of the third data, the method further includes: imputing missing data in the data retained after processing by the attention model.

[0105] In this embodiment, due to the processing by the frequency domain component selection module, some data is zeroed out, and some data may be lost due to the dropout layer. This results in the frequency domain representation of data Y after processing by the frequency domain component selection module inevitably lacking some information compared to the unsampled data. To ensure data dimensionality consistency, this embodiment performs padding on the data retained after processing by the frequency domain component selection module. For example, zeros are padded to the sampled frequency domain representation, and unsampled frequency points are represented as 0. This ensures that the signal can be restored to its original length during the mapping from the frequency domain back to the time domain. The padded data set is represented as follows: Furthermore, an inverse Fourier transform is performed on the padded data. The inverse Fourier transform can be expressed by the following expression:

[0106]

[0107] in, This represents the fifth data point obtained after inverse Fourier transform processing. This represents the data set after completion.

[0108] In the above embodiments, the frequency domain data after sampling and padding is mapped to the time domain (i.e., inverse Fourier transform processing), resulting in data with the same dimension as the original long-series first network traffic data. However, after attention allocation and frequency filtering, the remapped time domain data filters out some noise such as extremely high frequencies or extremely low frequency sporadic events, and applies more attention values ​​(or attention weights) to frequency data that are more helpful in predicting future data. Furthermore, the fifth data after inverse Fourier transform processing is input into the feature vector fusion module.

[0109] In some optional embodiments of the present invention, the feature fusion processing of the first data, the fourth data, and the fifth data includes: summing the first data, the fourth data, and the fifth data to perform feature fusion.

[0110] In this embodiment, the data input to the transformer decoder is obtained by fusing and superimposing the first, fourth, and fifth data. Specifically, this is done by adding them position-by-position to obtain a new feature vector. The advantage of this approach is that it reduces the computational complexity of the model and allows for the superposition of a new vector with attention weights onto the original feature vector. This results in a new superimposed feature vector that includes data from the entire spectrum and receives greater weight and attention at frequencies that are more helpful in predicting future data, thus improving the accuracy of future data predictions. This can be specifically processed using the following expression:

[0111]

[0112] Where, x merge This represents the data after feature fusion processing; x t Indicates the first data; Indicates the fifth data; x e This indicates the fourth data point.

[0113] Furthermore, the fused feature vector x merge The data is input into the transformer decoder for feature decoding, thereby obtaining the predicted second network traffic data.

[0114] This invention is based on the current Transformer model, and adds a Fourier encoder (including a decomposition module and a Fourier transform module), a frequency domain component selection module, and a Fourier decoder (including an inverse Fourier transform module and a feature vector fusion module). Adding a Fourier encoder can add global spectral information to the model; adding a frequency domain component selection module and a Fourier decoder can effectively retain frequency point information that is helpful for future data prediction, thereby effectively improving the accuracy of the network traffic prediction model.

[0115] Based on the above embodiments, this invention also provides a network traffic prediction device, which is applied in an electronic device. (Refer to...) Figure 1 As shown, the device includes: an encoder 11 and a decoder 12; the encoder 11 includes a Fourier encoder 111 and a transformer encoder 112; the decoder 12 includes a Fourier decoder 121 and a transformer decoder 122; wherein,

[0116] The Fourier encoder 111 is used to decompose the obtained first network traffic data into first data and second data, and perform Fourier transform processing on the second data to obtain third data that represents the frequency domain representation of the second data; the first data represents the long-term trend characteristics of the first network traffic data, and the second data represents the periodic fluctuation characteristics of the first network traffic data.

[0117] The transformer encoder 112 is used to encode the first network traffic data to obtain the fourth data;

[0118] The Fourier decoder 121 is used to perform inverse Fourier transform processing on at least a portion of the third data to obtain fifth data representing the time-domain representation of the at least a portion of the data; it is also used to perform feature fusion processing on the first data, the fourth data, and the fifth data;

[0119] The transformer decoder 122 is used to decode the data fused by the Fourier decoder 121 to obtain the predicted second network traffic data.

[0120] In some optional embodiments of the present invention, the Fourier encoder 111 includes a decomposition module 1111 and a Fourier transform module 1112; the Fourier decoder 121 includes an inverse Fourier transform module 1211 and a feature vector fusion module 1212; wherein,

[0121] The decomposition module 1111 is used to decompose the obtained first network traffic data into the first data and the second data, send the first data to the feature vector fusion module 1212, and send the second data to the Fourier transform module 1112.

[0122] The Fourier transform module 1112 is used to perform Fourier transform processing on the second data to obtain third data that characterizes the frequency domain representation of the second data.

[0123] The inverse Fourier transform module 1211 is used to perform inverse Fourier transform processing on at least a portion of the data in the third data to obtain the fifth data in the time domain representation of the at least a portion of the data, and send the fifth data to the feature vector fusion module 1212.

[0124] The feature vector fusion module 1212 is used to perform feature fusion processing on the first data, the fourth data, and the fifth data.

[0125] In some optional embodiments of the present invention, the apparatus further includes: a frequency domain component selection module 13, configured to process the third data through an attention model, determine the data to be retained and / or the data to be returned to zero in the third data, and send at least a portion of the retained third data to the Fourier decoder 121 for inverse Fourier transform processing.

[0126] In some optional embodiments of the present invention, the frequency domain component selection module 13 is used to assign a corresponding attention value to each data in the third data through an attention model; based on the attention value corresponding to each data in the third data, determine the first part of retained data and / or the data that has been returned to zero, and send the first part of retained data to the Fourier decoder for inverse Fourier transform processing.

[0127] In some optional embodiments of the present invention, the frequency domain component selection module 13 is used to determine that the data is retained when the attention value corresponding to the data is greater than or equal to a first threshold, and to determine that the data is zeroed when the attention value corresponding to the data is less than the first threshold.

[0128] In some optional embodiments of the present invention, the frequency domain component selection module 13 is used to assign a corresponding attention value to each data in the third data through an attention model; determine the first part of retained data and / or zeroed data based on the attention value corresponding to each data in the third data; deactivate the neural network unit in the attention model with a first probability; determine the retained data and / or zeroed data in the first part of retained data based on the deactivated neural network unit to obtain the second part of retained data; and send the second part of retained data to the Fourier decoder for inverse Fourier transform processing.

[0129] In some optional embodiments of the present invention, the inverse Fourier transform module 1211 is used to fill in the missing data in the data retained after processing by the attention model before performing inverse Fourier transform processing on at least a portion of the data in the third data.

[0130] In some optional embodiments of the present invention, the decomposition module 1111 is used to perform average pooling processing on the first network traffic data, obtain the first data based on the first network traffic data after average pooling processing, and perform subtraction processing on the first network traffic data and the first data to obtain the second data.

[0131] In some optional embodiments of the present invention, the Fourier decoder 121 (or the feature vector fusion module 1212) is used to perform summation processing on the first data, the fourth data and the fifth data to perform feature fusion.

[0132] In this embodiment of the invention, the encoder 11, decoder 12, and frequency domain component selection module 13 in the device, as well as the modules included in the encoder 11 and decoder 11, can all be implemented by a central processing unit (CPU), a digital signal processor (DSP), a microcontroller unit (MCU), or a field-programmable gate array (FPGA) in practical applications.

[0133] It should be noted that the network traffic prediction device provided in the above embodiments is only illustrated by the division of the above program modules. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the network traffic prediction device and the network traffic prediction method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0134] This invention also provides an electronic device. Figure 5 This is a schematic diagram of the hardware composition structure of the electronic device according to an embodiment of the present invention, such as... Figure 5 As shown, the electronic device includes a memory 22, a processor 21, and a computer program stored in the memory 22 and executable on the processor 21. When the processor 21 executes the program, it implements the steps of the network traffic prediction method.

[0135] Optionally, the electronic device also includes at least one network interface 23. The various components in the electronic device are coupled together via a bus system 24. It is understood that the bus system 24 is used to implement communication between these components. In addition to a data bus, the bus system 24 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 5 The general labeled all buses as Bus System 24.

[0136] It is understood that memory 22 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 22 described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0137] The methods disclosed in the above embodiments of the present invention can be applied to or implemented by processor 21. Processor 21 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 21 or by instructions in software form. The processor 21 may be a general-purpose processor, DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 21 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of the present invention can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 22. Processor 21 reads the information in memory 22 and completes the steps of the aforementioned method in conjunction with its hardware.

[0138] In an exemplary embodiment, the electronic device may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), FPGAs, general-purpose processors, controllers, MCUs, microprocessors, or other electronic components to perform the aforementioned method.

[0139] In an exemplary embodiment, the present invention also provides a computer-readable storage medium, such as a memory 22 including a computer program, which can be executed by a processor 21 of an electronic device to perform the steps described in the foregoing method. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM; or it may be various devices including one or any combination of the above-mentioned memories.

[0140] The computer-readable storage medium provided in this embodiment of the invention stores a computer program thereon, which, when executed by a processor, implements the steps of the network traffic prediction method described in this embodiment of the invention.

[0141] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0142] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0143] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0144] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0145] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0146] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0147] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0148] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0149] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A network traffic prediction method, characterized in that, The method is applied to an electronic device; the method includes: The obtained first network traffic data is decomposed into first data and second data. The second data is subjected to Fourier transform processing to obtain third data that represents the frequency domain representation of the second data. The first data represents the long-term trend characteristics of the first network traffic data, and the second data represents the periodic fluctuation characteristics of the first network traffic data. The first network traffic data is encoded using a transformer encoder to obtain the fourth data. At least a portion of the third data is subjected to inverse Fourier transform to obtain a fifth data representing the time-domain representation of the at least a portion of the data; The first data, the fourth data, and the fifth data are subjected to feature fusion processing. The fused data is then decoded using a transformer decoder to obtain the predicted second network traffic data.

2. The method according to claim 1, characterized in that, The inverse Fourier transform processing of at least a portion of the third data includes: The third data is processed using an attention model to determine the data to be retained and / or the data to be zeroed out. The data retained in the third set of data is subjected to inverse Fourier transform.

3. The method according to claim 2, characterized in that, The step of processing the third data using an attention model to determine the data to be retained and / or the data to be zeroed out includes: The attention model is used to assign a corresponding attention value to each data point in the third data. Based on the attention value corresponding to each data point in the third data, determine the data to be retained in the first part and / or the data to be zeroed out; The step of performing inverse Fourier transform processing on the data retained in the third data includes: performing inverse Fourier transform processing on the data retained in the first part.

4. The method according to claim 3, characterized in that, The step of determining the first portion of retained data and / or zeroed data based on the attention value corresponding to each data point in the third data includes: When the attention value corresponding to the data is greater than or equal to the first threshold, the data is determined to be retained; When the attention value corresponding to the data is less than the first threshold, the data is determined to be zero.

5. The method according to claim 2, characterized in that, The step of processing the third data using an attention model to determine the data to be retained and / or the data to be zeroed out includes: The attention model is used to assign a corresponding attention value to each data point in the third data. Based on the attention value corresponding to each data point in the third data, determine the data to be retained in the first part and / or the data to be zeroed out; The neural network units in the attention model are deactivated with a first probability. Based on the deactivated neural network units, the data retained in the first part of the retained data and / or the data that are returned to zero are determined to obtain the second part of the retained data. The step of performing inverse Fourier transform processing on the data retained in the third data includes: performing inverse Fourier transform processing on the data retained in the second part.

6. The method according to claim 3, characterized in that, Before performing inverse Fourier transform processing on at least a portion of the third data, the method further includes: Missing data in the data retained after processing by the attention model is filled in.

7. The method according to claim 1, characterized in that, The step of decomposing the first network traffic data into first data and second data includes: The first network traffic data is subjected to average pooling processing, and the first data is obtained based on the first network traffic data after average pooling processing. The second data is obtained by subtracting the first network traffic data from the first data.

8. The method according to claim 1, characterized in that, The feature fusion processing of the first data, the fourth data, and the fifth data includes: The first data, the fourth data, and the fifth data are summed to perform feature fusion.

9. A network traffic prediction device, characterized in that, The device includes: an encoder and a decoder; the encoder includes a Fourier encoder and a transformer encoder; the decoder includes a Fourier decoder and a transformer decoder; wherein, The Fourier encoder is used to decompose the obtained first network traffic data into first data and second data, and to perform Fourier transform processing on the second data to obtain third data that represents the frequency domain representation of the second data; the first data represents the long-term trend characteristics of the first network traffic data, and the second data represents the periodic fluctuation characteristics of the first network traffic data. The transformer encoder is used to encode the first network traffic data to obtain the fourth data; The Fourier decoder is used to perform inverse Fourier transform processing on at least a portion of the third data to obtain fifth data representing the time-domain representation of the at least a portion of the data; it is also used to perform feature fusion processing on the first data, the fourth data, and the fifth data; The transformer decoder is used to decode the data fused by the Fourier decoder to obtain the predicted second network traffic data.

10. The apparatus according to claim 9, characterized in that, The Fourier encoder includes a decomposition module and a Fourier transform module; the Fourier decoder includes an inverse Fourier transform module and a feature vector fusion module; wherein, The decomposition module is used to decompose the obtained first network traffic data into the first data and the second data, send the first data to the feature vector fusion module, and send the second data to the Fourier transform module. The Fourier transform module is used to perform Fourier transform processing on the second data to obtain third data that characterizes the frequency domain representation of the second data; The inverse Fourier transform module is used to perform inverse Fourier transform processing on at least a portion of the third data to obtain the fifth data, which is a time-domain representation of the at least a portion of the data, and send the fifth data to the feature vector fusion module. The feature vector fusion module is used to perform feature fusion processing on the first data, the fourth data, and the fifth data.

11. The apparatus according to claim 9, characterized in that, The device further includes a frequency domain component selection module, used to process the third data through an attention model, determine the data to be retained and / or the data to be returned to zero in the third data, and send the data to be retained in the third data to the Fourier decoder for inverse Fourier transform processing.

12. The apparatus according to claim 11, characterized in that, The frequency domain component selection module is used to assign a corresponding attention value to each data in the third data through an attention model; based on the attention value corresponding to each data in the third data, it determines the first part of retained data and / or data that has been returned to zero, and sends the first part of retained data to the Fourier decoder for inverse Fourier transform processing.

13. The apparatus according to claim 11, characterized in that, The frequency domain component selection module is used to assign a corresponding attention value to each data in the third data through an attention model; and to determine the first part of retained data and / or zeroed data based on the attention value corresponding to each data in the third data. The neural network units in the attention model are deactivated with a first probability. Based on the deactivated neural network units, the data to be retained and / or the data to be returned to zero in the first part of the retained data are determined to obtain the second part of the retained data. The second part of the retained data is then sent to the Fourier decoder for inverse Fourier transform processing.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method according to any one of claims 1 to 8.

15. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 8.

Images