Network traffic prediction method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a network traffic prediction method and device, electronic equipment and a storage medium. The method comprises the following steps: decomposing obtained first network flow data into first data and second data, and performing Fourier transform processing on the second data to obtain third data representing frequency domain representation of the second data; the first data represents a long-term trend characteristic of the first network traffic data, and the second data represents a periodic fluctuation characteristic of the first network traffic data; carrying out encoding processing on the first network flow data through a transformer encoder to obtain fourth data; performing inverse Fourier transform processing on at least part of data in the third data to obtain fifth data representing time domain representation of the at least part of data; and carrying out feature fusion processing on the first data, the fourth data and the fifth data, and carrying out decoding processing on the fused data based on a transformer decoder to obtain predicted second network flow data.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a network traffic prediction method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of the network, the services and applications carried on the network are becoming increasingly rich. Strengthening network management construction is a problem that operators continue to solve. To effectively improve the network operation speed and utilization rate, network traffic prediction is the key. If network traffic overload can be predicted, the problem can be analyzed and solved before the network traffic overload occurs, which can significantly improve the availability of the network and services. However, the current network traffic prediction model still has the problem of insufficient prediction accuracy. Summary of the invention

[0003] In order to solve the existing technical problems, the embodiments of the present invention provide a network traffic prediction method, device, electronic device and storage medium.

[0004] To achieve the above object, the technical solution of the embodiment of the present invention is implemented as follows:

[0005] An embodiment of the present invention provides a network traffic prediction method, which is applied in an electronic device; the method includes:

[0006] Decomposing the obtained first network traffic data into first data and second data, performing Fourier transform processing on the second data, and obtaining third data representing the frequency domain representation of the second data; the first data represents the long-term trend characteristics of the first network traffic data, and the second data represents the periodic fluctuation characteristics of the first network traffic data;

[0007] Encoding the first network traffic data through a transformer encoder to obtain fourth data;

[0008] Performing inverse Fourier transform processing on at least part of the third data to obtain fifth data representing a time domain representation of at least part of the data;

[0009] The first data, the fourth data and the fifth data are subjected to feature fusion processing, and the fused data are decoded based on a transformer decoder to obtain predicted second network traffic data.

[0010] In the above solution, the performing inverse Fourier transform processing on at least part of the third data includes: processing the third data through an attention model to determine the data to be retained and / or the data to be zeroed in the third data;

[0011] An inverse Fourier transform process is performed on the data retained in the third data.

[0012] In the above solution, the processing of the third data by the attention model to determine the data to be retained and / or the data to be zeroed in the third data includes:

[0013] Assigning a corresponding attention value to each data in the third data by using an attention model;

[0014] Determine the data to be retained in the first part and / or the data to be reset to zero based on the attention value corresponding to each data in the third data;

[0015] The performing inverse Fourier transform processing on the data retained in the third data includes: performing inverse Fourier transform processing on the data retained in the first part.

[0016] In the above solution, determining the data to be retained and / or the data to be reset to zero in the first part based on the attention value corresponding to each data in the third data includes:

[0017] When the attention value corresponding to the data is greater than or equal to the first threshold, determining that the data is retained;

[0018] When the attention value corresponding to the data is less than the first threshold, it is determined that the data is reset to zero.

[0019] In the above solution, the processing of the third data by the attention model to determine the data to be retained and / or the data to be zeroed in the third data includes:

[0020] Assigning a corresponding attention value to each data in the third data by using an attention model;

[0021] Determine the data to be retained in the first part and / or the data to be reset to zero based on the attention value corresponding to each data in the third data;

[0022] Deactivating the neural network unit in the attention model with a first probability, determining the data retained and / or the data reset to zero in the first part of the retained data based on the deactivated neural network unit, and obtaining the second part of the retained data;

[0023] The performing inverse Fourier transform processing on the data retained in the third data includes: performing inverse Fourier transform processing on the data retained in the second part.

[0024] In the above solution, before performing inverse Fourier transform processing on at least part of the third data, the method further includes:

[0025] The missing data in the data retained after being processed by the attention model is supplemented.

[0026] In the above solution, decomposing the first network traffic data into the first data and the second data includes: performing average pooling processing on the first network traffic data, and obtaining the first data based on the first network traffic data after the average pooling processing;

[0027] The first network traffic data is subtracted from the first data to obtain the second data.

[0028] In the above scheme, the feature fusion processing of the first data, the fourth data and the fifth data includes: adding the first data, the fourth data and the fifth data to perform feature fusion.

[0029] The embodiment of the present invention further provides a network traffic prediction device, the device comprising: an encoder and a decoder; the encoder comprises a Fourier encoder and a transformer encoder; the decoder comprises a Fourier decoder and a transformer decoder; wherein,

[0030] The Fourier encoder is used to decompose the obtained first network traffic data into first data and second data, and perform Fourier transform processing on the second data to obtain third data representing the frequency domain representation of the second data; the first data represents the long-term trend characteristics of the first network traffic data, and the second data represents the periodic fluctuation characteristics of the first network traffic data;

[0031] The transformer encoder is used to encode the first network traffic data to obtain fourth data;

[0032] The Fourier decoder is used to perform inverse Fourier transform processing on at least part of the third data to obtain fifth data representing the time domain representation of at least part of the data; and is also used to perform feature fusion processing on the first data, the fourth data and the fifth data;

[0033] The transformer decoder is used to decode the data after fusion processing by the Fourier decoder to obtain predicted second network traffic data.

[0034] In the above scheme, the Fourier encoder includes a decomposition module and a Fourier transform module; the Fourier decoder includes an inverse Fourier transform module and a feature vector fusion module; wherein,

[0035] The decomposition module is used to decompose the obtained first network traffic data into the first data and the second data, send the first data to the feature vector fusion module, and send the second data to the Fourier transform module;

[0036] The Fourier transform module is used to perform Fourier transform processing on the second data to obtain third data representing the frequency domain representation of the second data;

[0037] The inverse Fourier transform module is used to perform inverse Fourier transform processing on at least part of the third data to obtain fifth data represented by the time domain of at least part of the data, and send the fifth data to the feature vector fusion module;

[0038] The feature vector fusion module is used to perform feature fusion processing on the first data, the fourth data and the fifth data.

[0039] In the above scheme, the device also includes: a frequency domain component selection module, which is used to process the third data through an attention model, determine the data retained and / or the data returned to zero in the third data, and send the data retained in the third data to the Fourier decoder for inverse Fourier transform processing.

[0040] In the above scheme, the frequency domain component selection module is used to assign a corresponding attention value to each data in the third data through the attention model; based on the attention value corresponding to each data in the third data, determine the data retained in the first part and / or the data returned to zero, and send the data retained in the first part to the Fourier decoder for inverse Fourier transform processing.

[0041] In the above scheme, the frequency domain component selection module is used to assign a corresponding attention value to each data in the third data through the attention model; determine the data retained in the first part and / or the data reset to zero based on the attention value corresponding to each data in the third data; deactivate the neural network unit in the attention model with a first probability, determine the data retained and / or the data reset to zero in the first part of the data retained based on the neural network unit after the deactivation, obtain the data retained in the second part, and send the data retained in the second part to the Fourier decoder for inverse Fourier transform processing.

[0042] The embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the network traffic prediction method described in the embodiment of the present invention are implemented.

[0043] An embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the network traffic prediction method described in the embodiment of the present invention are implemented.

[0044] The network traffic prediction method, device, electronic device and storage medium provided by the embodiment of the present invention include: the electronic device decomposes the obtained first network traffic data into first data and second data, performs Fourier transform processing on the second data, and obtains third data representing the frequency domain representation of the second data; the first data represents the long-term trend characteristics of the first network traffic data, and the second data represents the periodic fluctuation characteristics of the first network traffic data; the first network traffic data is encoded by a transformer encoder to obtain fourth data; at least part of the third data is inversely Fourier transformed to obtain fifth data representing the time domain representation of at least part of the data; the first data, the fourth data and the fifth data are feature fused, and the fused data are decoded based on a transformer decoder to obtain the predicted second network traffic data. The technical solution of the embodiment of the present invention is adopted, and by decomposing the first network traffic data into first data representing long-term trend characteristics and second data identifying periodic fluctuation characteristics in the encoding stage, global spectrum information can be added to the model, and long-time series network traffic data can be better modeled, thereby improving the prediction accuracy of network traffic. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A schematic diagram of the structure and data flow of a network traffic prediction device according to an embodiment of the present invention;

[0046] Figure 2 The network traffic prediction method of the present invention is shown in FIG. Figure 1 ;

[0047] Figure 3 The flow chart of the network traffic prediction method according to the embodiment of the present invention is as follows Figure 2 ;

[0048] Figure 4 It is a structural schematic diagram of a frequency domain component selection module in a network traffic prediction device according to an embodiment of the present invention;

[0049] Figure 5 The figure is a schematic diagram of the hardware structure of the electronic device according to the embodiment of the present invention. DETAILED DESCRIPTION

[0050] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0051] The technical solution of the embodiment of the present invention can be applied to various communication systems, for example: a global system of mobile communication (GSM) system, a long term evolution (LTE) system or a 5G system, etc. Optionally, the 5G system or 5G network may also be referred to as a new radio (NR) system or NR network.

[0052] It should be understood that the device with communication function in the network / system in the embodiment of the present application can be referred to as a communication device. The communication device may include a network device and a terminal with communication function, and the network device and the terminal device may be the specific devices described above, which will not be repeated here; the communication device may also include other devices in the communication system, such as a network controller, a mobile management entity and other network entities, which are not limited in the embodiment of the present invention.

[0053] It should be understood that the terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0054] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable in appropriate circumstances, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, the process, method, system, product or equipment comprising a series of steps or units need not be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0055] In the traditional time prediction model, the encoder converts the input sequence (x1,...x n ) is mapped to a continuous representation sequence (z1,...z n ). Given z, the decoder generates an output sequence (y1,...y) one symbol at a time n). In each step, the model is auto-regressive, and when generating the next step, the previously generated symbol sequence is used as an additional input, so that the prediction of each time step is made separately and independently. Therefore, the traditional time prediction model cannot maintain the global properties of the time series and cannot treat the statistical data as a whole, resulting in insufficient prediction accuracy of network traffic.

[0056] Figure 1 FIG. 1 is a schematic diagram of the structure of a network traffic prediction device and data flow according to an embodiment of the present invention; Figure 1 As shown, the network traffic prediction device of the embodiment of the present invention includes: an encoder 11 and a decoder 12; the encoder 11 includes a Fourier encoder 111 and a transformer encoder 112; the decoder 12 includes a Fourier decoder 121 and a transformer decoder 122.

[0057] Among them, optionally, the Fourier encoder 111 includes a decomposition module 1111 and a Fourier transform module 1112; the Fourier decoder 121 includes an inverse Fourier transform module 1211 and a feature vector fusion module 1212. In the encoding stage, the first network traffic data is decomposed into first data and second data by the decomposition module 1111, the first data represents the long-term trend characteristics of the first network traffic data, and the second data represents the periodic fluctuation characteristics of the first network traffic data; the second data is Fourier transformed by the Fourier transform module 1112, and the second data is mapped from the time domain to the frequency domain, so as to obtain third data representing the frequency domain component of the second data; in the decoding stage, at least part of the third data is inversely Fourier transformed by the inverse Fourier transform module 1211, and at least part of the third data is mapped from the frequency domain to the time domain, so as to obtain fifth data of the time domain component of at least part of the data, and the received first data, fourth data and fifth data are feature fused by the feature vector fusion module 1212, and the feature fused data is decoded by the transformer decoder 122. This method can add global spectrum information to the model, which can better model long-term network traffic data and thus improve the prediction accuracy of network traffic.

[0058] Optionally, the device may also include a frequency domain component selection module 13, which is used to process the third data output by the Fourier encoding module through an attention model (or attention layer), return the data that is not helpful for the predicted data to zero, and retain the data that is helpful for the predicted data. The inverse Fourier transform module in the joint Fourier decoder 121 performs time domain-frequency domain mapping processing on the data, and the feature vector fusion module 1212 performs fusion processing on each data. This method is suitable for time series prediction, because some trend changes in the time series are related to important events, and each frequency domain component may contain a piece of information about the predicted trend; compared with the method of retaining low-frequency components and discarding high-frequency components to represent the time series, the embodiment of the present invention is more suitable for time series prediction, retaining information containing predicted trends as much as possible, thereby improving the prediction accuracy of network traffic.

[0059] Figure 2 The flow chart of the network traffic prediction method according to the embodiment of the present invention is as follows Figure 1 ,like Figure 2 As shown, the method includes:

[0060] Step 101: decomposing the obtained first network traffic data into first data and second data, performing Fourier transform processing on the second data to obtain third data representing the frequency domain representation of the second data; the first data represents the long-term trend characteristics of the first network traffic data, and the second data represents the periodic fluctuation characteristics of the first network traffic data;

[0061] Step 102: encoding the first network traffic data through a transformer encoder to obtain fourth data;

[0062] Step 103: performing inverse Fourier transform processing on at least part of the third data to obtain fifth data representing the time domain representation of at least part of the data;

[0063] Step 104: Perform feature fusion processing on the first data, the fourth data and the fifth data, and obtain predicted second network traffic data by decoding the fused data based on a transformer decoder.

[0064] The network traffic prediction method of this embodiment is applied to an electronic device, and the electronic device may be a personal device or a server with a data processing function. Among them, the network traffic data (such as the first network traffic data, the second network traffic data, etc.) in this embodiment is the amount of data transmitted by the network. Exemplarily, the network traffic data can be represented by visits, clicks, etc., such as visits, clicks, etc. for a certain website, application (APP). In this embodiment, the first network traffic data is specifically the historical network traffic data obtained in advance, such as visits, clicks, etc. in the past period of time. The second network traffic data is the network traffic data predicted based on the first network traffic data, such as visits, clicks, etc. predicted in the future period of time. It can be considered that both the first network traffic data and the second network traffic data can be called long-term network traffic data, that is, the first network traffic data may include network traffic values ​​(such as visits, clicks, etc.) corresponding to multiple historical moments (or historical time points) within a period of time in the past, and the second network traffic data may include network traffic values ​​(such as visits, clicks, etc.) corresponding to multiple future moments (or future time points) within a period of time in the future, etc.

[0065] Combination Figure 1 As shown, in this embodiment, the first network traffic data obtained can be decomposed into first data and second data through a decomposition module, wherein the first data represents the long-term trend characteristics of the first network traffic data, and the second data represents the periodic fluctuation characteristics of the first network traffic data. In other optional embodiments, the first data can also be called a long-term trend item, which represents the development trend of the traffic data in the time domain; the second data can also be called a periodic fluctuation item, which represents the change trend of the traffic data in the frequency domain. Further, the first data is input into the feature vector fusion module in the Fourier decoder, and the second data is input into the Fourier transform module.

[0066] Exemplarily, the first network traffic data may include multiple Or, in other words, the first network traffic data may be composed of multiple The data can form the first network traffic data. Based on this, by decomposing the first network traffic data, it can be decomposed into The data in the form of a represents the second data of periodic fluctuation characteristics, b represents the first data of long-term trend characteristics.

[0067] In some optional embodiments, the decomposing the first network traffic data into the first data and the second data includes: performing average pooling processing on the first network traffic data, obtaining the first data based on the first network traffic data after the average pooling processing; performing difference processing between the first network traffic data and the first data to obtain the second data. In other optional embodiments, the first network traffic data after the average pooling processing can also be padded (such as filling with 0) with reference to the first network traffic data before processing to keep the data length consistent.

[0068] Specifically, if the first network traffic data is represented by a vector x, the first data can be represented as:

[0069] x t =Padding(AvgPool(x)) (1)

[0070] Among them, x t represents the first data, Padding represents padding processing of the first network traffic data, and AvgPool represents average pooling processing. Exemplarily, the padding processing may be, for example, maintaining the same scale as the input item (such as the first network traffic data) and padding 0 to both sides symmetrically.

[0071] Accordingly, the second data can be expressed as:

[0072] x s =xx t (2)

[0073] Among them, x s represents the second data.

[0074] In this embodiment, the second data is subjected to Fourier transform processing by a Fourier transform module, with the purpose of projecting the second data from an input sequence in the time domain to the frequency domain. Specifically, the third data can be expressed as:

[0075] X s (ω)=∑ n x(n)e -iωn (3)

[0076] Among them, X s (ω) represents the third data, x(n) is x s The nth value in the second data represented by .

[0077] It can be seen from the above expression (3) that the frequency domain representation of the periodic fluctuation term (ie, the third data) is composed of a finite number of periodic functions of different frequencies, which includes a series of lower frequencies and a series of higher frequencies.

[0078] On the other hand, the embodiment of the present invention encodes the input first network traffic data through a transformer encoder to obtain fourth data. It should be noted that the execution of step 101 and step 102 is not limited to the order, and step 101 can be executed first and then step 102, or vice versa, step 102 can be executed first and then step 101, or step 101 and step 102 can be executed simultaneously, which is not limited in this embodiment.

[0079] The network traffic prediction method in this embodiment can be implemented on the basis of an improved transformer model. The transformer model is essentially an encoder-decoder structure, that is, this embodiment can be improved on the transformer model by adding a Fourier encoder (including a decomposition module and a Fourier transform module) on the basis of the encoder, and adding a Fourier decoder (including an inverse Fourier transform module and a feature vector fusion module) on the basis of the decoder.

[0080] In a specific embodiment, the transformer encoder and the transformer decoder are the encoder and decoder in a conventional transformer model. For example, the transformer encoder may include a multi-layer encoder structure, and correspondingly, the transformer decoder may include a multi-layer decoder structure. Each transformer encoder may have two sublayers: an attention layer and a feed forward network (FFN). The attention layer may be, for example, a multi-head attention layer (Multi-Head Attention), a self-attention layer (Self-Attention), etc. The structure of each transformer encoder in the multi-layer encoder included in the transformer encoder is the same, using different weight parameters. The transformer decoder includes two sublayers in the transformer encoder and also includes another attention layer (Attention).

[0081] In some optional embodiments, the embodiments of the present invention perform inverse Fourier transform processing on at least part of the third data through an inverse Fourier transform module to obtain fifth data representing the time domain component of at least part of the data; then perform feature fusion processing on the first data, the fourth data and the fifth data through a feature vector fusion module, the data after feature fusion processing is recorded as sixth data, and then the sixth data is decoded through a transformer decoder to output predicted second network traffic data.

[0082] In some other optional embodiments, the third data output by the Fourier encoding module can also be processed by the frequency domain component selection module in the device through the attention model (or attention layer), the data that is not helpful for the predicted data is reset to zero, and the data that is helpful for the predicted data is retained, so as to improve the accuracy of network traffic prediction. Based on this, if Figure 3 As shown, the above step 103 may include:

[0083] Step 1031: Process the third data by using an attention model to determine the data to be retained and / or the data to be reset to zero in the third data;

[0084] Step 1032: performing inverse Fourier transform processing on the data retained in the third data to obtain fifth data representing the time domain component;

[0085] In this embodiment, the attention model may also be referred to as an attention layer, and the attention model (or attention layer) is a structure of a fully connected layer. The third data is processed by the attention model to screen the third data, retain the data that is helpful for predicting the data, and zero (or delete) the data that is not helpful or very little helpful for predicting the data. Further, the retained data is subjected to inverse Fourier transform processing to obtain the fifth data.

[0086] In some optional embodiments of the present invention, the processing of the third data through the attention model to determine the data retained and / or the data reset to zero in the third data includes: assigning a corresponding attention value to each data in the third data through the attention model; based on the attention value corresponding to each data in the third data, determining the data retained and / or the data reset to zero in the first part; accordingly, performing inverse Fourier transform processing on the data retained in the third data includes: performing inverse Fourier transform processing on the data retained in the first part.

[0087] In this embodiment, attention values ​​(or attention weights) are assigned to data in the third data through the attention model (or attention layer), so that large weights are assigned to data that are of great help in predicting the data, and small weights are assigned to data that are of little help in predicting the data. The data to be retained and / or reset to zero in the third data is determined based on the assigned weights, and the data that is not helpful or of very little help to the predicted data is reset to zero, and the data that is helpful to the predicted data is retained.

[0088] Specifically, in this embodiment, the back propagation mechanism of the neural network can be used to analyze which frequency domain data is helpful for the final network traffic prediction output, which is noise data, and which frequency data is less helpful for predicting future network traffic data, thereby optimizing the attention table (or attention set), which may include multiple frequency domain data and the attention value (or attention weight) corresponding to each frequency domain data. Further, the fifth data output by the frequency domain component selection module can be obtained according to the following expression:

[0089] Y=WX s +B (4)

[0090] Where Y is the selected data set output by the frequency domain component selection module (i.e., the data output by the frequency domain component selection module), W represents the attention value (or attention weight) set, and X s represents the data output by the Fourier transform module (i.e., the third data), and B is an offset set. In some optional embodiments, expression (4) can be used as the structure of a fully connected layer of an attention model (or attention layer). Optionally, the offsets for different frequency domain data (i.e., the third data) can be pre-optimized to obtain an offset set B; in other optional embodiments, the offset set may not be required, that is, the offset for each frequency domain data (i.e., the third data) is 0.

[0091] In some optional embodiments, the data to be retained and / or the data to be reset to zero in the first part is determined based on the attention value corresponding to each data in the third data, including: when the attention value corresponding to the data is greater than or equal to a first threshold, determining that the data is retained; when the attention value corresponding to the data is less than the first threshold, determining that the data is reset to zero.

[0092] In this embodiment, by setting a first threshold value related to the attention value (or attention weight), data with an attention value (or attention weight) lower than the first threshold value is reset to zero, and data with an attention value (or attention weight) greater than or equal to the first threshold value is retained, thereby deleting data that is not helpful or very helpful for predicting data, which can reduce computational complexity, speed up data processing, and improve data processing efficiency.

[0093] Specifically, the determination of data retention and / or zeroing can be referred to the following expression:

[0094]

[0095] Among them, X s (ω) represents the frequency domain data set of the third data, W n Represents a set of attention values ​​(or attention weights); The meaning is the same as that of expression (3).

[0096] The embodiment of the present invention uses the attention mechanism to give a larger weight to the frequency data that is more helpful in predicting future data, and a smaller weight to the frequency data that is less helpful in predicting future data. This is more conducive to focusing the attention in network calculations on the frequencies that are more helpful in predicting future data, and can better improve the accuracy of predicting future data. Furthermore, the frequency domain component selection module 13 inputs the data retained in the first part into the inverse Fourier transform module 1211 for inverse Fourier transform processing.

[0097] In some optional embodiments of the present invention, the processing of the third data through the attention model to determine the data retained and / or the data reset to zero in the third data includes: assigning a corresponding attention value to each data in the third data through the attention model; determining the data retained and / or the data reset to zero in the first part based on the attention value corresponding to each data in the third data; deactivating the neural network units in the attention model with a first probability, and determining the data retained and / or the data reset to zero in the first part of the data retained based on the deactivated neural network units to obtain the data retained in the second part; performing inverse Fourier transform processing on the data retained in the third data includes: performing inverse Fourier transform processing on the data retained in the second part.

[0098] In this embodiment, after determining the first part of the retained data in the aforementioned embodiment, in order to increase the generalizability and robustness of the model, a dropout layer is added after the attention model (or attention layer), referring to Figure 4 As shown, by removing the (dropout) layer with a certain probability (such as the first probability), the neural network units in the attention model are inactivated or discarded, and the neural network units on the attention model (or attention layer) are discarded according to the first probability, which greatly reduces the length of the input data (the input data is represented by a vector, so the input data can be called an input vector) and thus reduces the computational complexity. Although this processing causes loss of input information. However, experiments have shown that this loss has little effect on the final accuracy. Because the general signal is sparser in the frequency domain than in the time domain. And a large amount of information in the high-frequency part is the so-called noise, which can often be discarded in the problem of time series prediction, because noise often represents a randomly generated part and cannot be predicted. In contrast, in the field of images, the "noise" in the high-frequency part may represent the details of the picture, which cannot be ignored.

[0099] Specifically, the above processing process can be expressed by the following expression:

[0100] r i(l) ~Bernoulli(p) (6)

[0101]

[0102]

[0103] Wherein, expression (6) represents the Bernoulli distribution with probability p, that is, according to the set probability p, the data is randomly set to 0 with probability p in accordance with the Bernoulli distribution, so as to inactivate the neural network unit of the attention model (or attention layer), r (l) represents the zero processing vector randomly generated by Bernoulli distribution. Expression (7) represents the frequency domain data processed based on the neural network unit after deactivation, r (l) With X s Perform point multiplication, X s Some positions of are set to zero; since the neural network unit has inactivation with probability p, it is possible that r (l) The situation equal to 0 occurs; X s is the data retained after weight allocation and threshold selection, then X s After r (l) The processing obtained It may be 0, that is, it may be discarded, or it may be retained. The retained data is processed by expression (8) to obtain the data output by the frequency domain component selection module 13 (that is, the data retained in the second part). In other optional embodiments, expression (8) can be used as the structure of a fully connected layer of another attention model (or attention layer).

[0104] In some optional embodiments of the present invention, before performing inverse Fourier transform processing on at least part of the third data, the method further includes: completing missing data in the data retained after processing by the attention model.

[0105] In this embodiment, due to the processing by the frequency domain component selection module, some data is reset to zero, and some data may be discarded by the removal (dropout) layer, resulting in the data Y processed by the frequency domain component selection module. Its frequency domain representation must be missing some information compared to the unsampled state. In order to ensure the consistency of the data dimension, in this embodiment, the data retained after being processed by the frequency domain component selection module is padded, for example, the sampled frequency domain representation is padded with 0, and the unsampled frequency points are represented by 0, so that in the process of mapping from the frequency domain back to the time domain, it can be ensured that the signal can be restored to its original length. The data set after padding is represented as Furthermore, an inverse Fourier transform operation is performed on the padded data. The inverse Fourier transform can be specifically expressed by the following expression:

[0106]

[0107] in, represents the fifth data obtained after inverse Fourier transform processing, Represents the data set after padding.

[0108] In the above embodiment, the mapping of the frequency domain data in the time domain after sampling and padding (i.e., inverse Fourier transform processing) can be obtained, and then data with the same dimension as the first network traffic data of the original input long time series can be obtained. However, after the attention allocation and frequency screening, the remapped time domain data will filter out some such as ultra-high frequency noise or extremely low frequency incidental events, and will apply more attention values ​​(or attention weights) to the frequency data that is more helpful for predicting future data. Further, the fifth data after the inverse Fourier transform processing is input into the feature vector fusion module.

[0109] In some optional embodiments of the present invention, the performing feature fusion processing on the first data, the fourth data and the fifth data includes: performing addition processing on the first data, the fourth data and the fifth data to perform feature fusion.

[0110] In this embodiment, the data input to the transformer decoder is obtained by fusion and superposition of the first data, the fourth data and the fifth data, specifically by performing positional addition to obtain a new feature vector. The advantage of this is that it can reduce the computational complexity of the model, and on the basis of the original feature vector, by superimposing a new vector with attention weights, the new superimposed feature vector not only contains the data of the full spectrum, but also obtains greater weighted attention on certain frequency points that are of great help in predicting future data, which can better improve the accuracy of predicting future data. It can be processed specifically through the following expression:

[0111]

[0112] Among them, x merge Represents the data after feature fusion processing; x t represents the first data; Indicates the fifth data; x e Indicates the fourth data.

[0113] Furthermore, the fused feature vector x merge Input into the transformer decoder for feature decoding to obtain the predicted second network traffic data.

[0114] The present invention is based on the current Transformer model, and adds a Fourier encoder (including a decomposition module and a Fourier transform module), a frequency domain component selection module and a Fourier decoder (including an inverse Fourier transform module and a feature vector fusion module) on this basis. The addition of the Fourier encoder can add global spectrum information to the model; the addition of the frequency domain component selection module and the Fourier decoder can effectively retain the frequency point information that is helpful for future data prediction, thereby effectively improving the accuracy of the network traffic prediction model.

[0115] Based on the above embodiments, the present invention also provides a network traffic prediction device, which is applied in an electronic device. Figure 1 As shown, the device includes: an encoder 11 and a decoder 12; the encoder 11 includes a Fourier encoder 111 and a transformer encoder 112; the decoder 12 includes a Fourier decoder 121 and a transformer decoder 122; wherein,

[0116] The Fourier encoder 111 is used to decompose the obtained first network traffic data into first data and second data, and perform Fourier transform processing on the second data to obtain third data representing the frequency domain representation of the second data; the first data represents the long-term trend characteristics of the first network traffic data, and the second data represents the periodic fluctuation characteristics of the first network traffic data;

[0117] The transformer encoder 112 is used to encode the first network traffic data to obtain fourth data;

[0118] The Fourier decoder 121 is used to perform inverse Fourier transform processing on at least part of the third data to obtain fifth data representing the time domain representation of at least part of the data; and is also used to perform feature fusion processing on the first data, the fourth data and the fifth data;

[0119] The transformer decoder 122 is used to decode the data after fusion processing by the Fourier decoder 121 to obtain predicted second network traffic data.

[0120] In some optional embodiments of the present invention, the Fourier encoder 111 includes a decomposition module 1111 and a Fourier transform module 1112; the Fourier decoder 121 includes an inverse Fourier transform module 1211 and a feature vector fusion module 1212; wherein,

[0121] The decomposition module 1111 is used to decompose the obtained first network traffic data into the first data and the second data, send the first data to the feature vector fusion module 1212, and send the second data to the Fourier transform module 1112;

[0122] The Fourier transform module 1112 is used to perform Fourier transform processing on the second data to obtain third data representing the frequency domain representation of the second data;

[0123] The inverse Fourier transform module 1211 is used to perform inverse Fourier transform processing on at least part of the third data to obtain fifth data represented by the time domain of at least part of the data, and send the fifth data to the feature vector fusion module 1212;

[0124] The feature vector fusion module 1212 is used to perform feature fusion processing on the first data, the fourth data and the fifth data.

[0125] In some optional embodiments of the present invention, the device also includes: a frequency domain component selection module 13, which is used to process the third data through an attention model, determine the retained data and / or zeroed data in the third data, and send at least part of the retained third data to the Fourier decoder 121 for inverse Fourier transform processing.

[0126] In some optional embodiments of the present invention, the frequency domain component selection module 13 is used to assign a corresponding attention value to each data in the third data through an attention model; based on the attention value corresponding to each data in the third data, determine the data retained in the first part and / or the data reset to zero, and send the data retained in the first part to the Fourier decoder for inverse Fourier transform processing.

[0127] In some optional embodiments of the present invention, the frequency domain component selection module 13 is used to determine that the data is retained when the attention value corresponding to the data is greater than or equal to a first threshold; and to determine that the data is reset to zero when the attention value corresponding to the data is less than the first threshold.

[0128] In some optional embodiments of the present invention, the frequency domain component selection module 13 is used to assign a corresponding attention value to each data in the third data through an attention model; determine the data retained in the first part and / or the data reset to zero based on the attention value corresponding to each data in the third data; deactivate the neural network unit in the attention model with a first probability, determine the data retained and / or the data reset to zero in the first part of the data retained based on the neural network unit after the deactivation, obtain the data retained in the second part, and send the data retained in the second part to the Fourier decoder for inverse Fourier transform processing.

[0129] In some optional embodiments of the present invention, the inverse Fourier transform module 1211 is used to fill in missing data in the data retained after processing by the attention model before performing inverse Fourier transform processing on at least part of the third data.

[0130] In some optional embodiments of the present invention, the decomposition module 1111 is used to perform average pooling processing on the first network traffic data, and obtain the first data based on the first network traffic data after the average pooling processing; and perform difference processing between the first network traffic data and the first data to obtain the second data.

[0131] In some optional embodiments of the present invention, the Fourier decoder 121 (or the feature vector fusion module 1212) is used to perform a summing process on the first data, the fourth data and the fifth data to perform feature fusion.

[0132] In the embodiment of the present invention, the encoder 11, the decoder 12 and the frequency domain component selection module 13 in the device, as well as the modules included in the encoder 11 and the decoder 11, can be implemented by a central processing unit (CPU), a digital signal processor (DSP), a microcontroller unit (MCU) or a programmable gate array (FPGA) in practical applications.

[0133] It should be noted that: the network traffic prediction device provided in the above embodiment only uses the division of the above program modules as an example when performing network traffic prediction. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the network traffic prediction device provided in the above embodiment and the network traffic prediction method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0134] An embodiment of the present invention further provides an electronic device. Figure 5 FIG. 1 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Figure 5 As shown, the electronic device includes a memory 22, a processor 21, and a computer program stored in the memory 22 and executable on the processor 21. When the processor 21 executes the program, the steps of the network traffic prediction method are implemented.

[0135] Optionally, the electronic device further includes at least one network interface 23. The components in the electronic device are coupled together via a bus system 24. It is understood that the bus system 24 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 24 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 5 Various buses are labeled as bus system 24 .

[0136] It can be understood that the memory 22 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 22 described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0137] The method disclosed in the above embodiment of the present invention can be applied to the processor 21, or implemented by the processor 21. The processor 21 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 21 or the instruction in the form of software. The above processor 21 can be a general-purpose processor, a DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 21 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general-purpose processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 22. The processor 21 reads the information in the memory 22 and completes the steps of the above method in combination with its hardware.

[0138] In an exemplary embodiment, the electronic device may be implemented by one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), FPGAs, general purpose processors, controllers, MCUs, microprocessors, or other electronic components to execute the aforementioned method.

[0139] In an exemplary embodiment, the present invention also provides a computer-readable storage medium, such as a memory 22 including a computer program, and the computer program can be executed by a processor 21 of an electronic device to complete the steps of the aforementioned method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disk, or CD-ROM; or it can be various devices including one or any combination of the above memories.

[0140] The computer-readable storage medium provided by the embodiment of the present invention stores a computer program, which, when executed by a processor, implements the steps of the network traffic prediction method described in the embodiment of the present invention.

[0141] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0142] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0143] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0144] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0145] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0146] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0147] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium, which, when executed, executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, disks or optical disks.

[0148] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0149] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A network traffic prediction method, characterized in that: The method is applied in an electronic device; the method comprises: Decomposing the obtained first network traffic data into first data and second data, performing Fourier transform processing on the second data, and obtaining third data representing the frequency domain representation of the second data; the first data represents the long-term trend characteristics of the first network traffic data, and the second data represents the periodic fluctuation characteristics of the first network traffic data; Encoding the first network traffic data through a transformer encoder to obtain fourth data; Performing inverse Fourier transform processing on at least part of the third data to obtain fifth data representing a time domain representation of at least part of the data; The first data, the fourth data and the fifth data are subjected to feature fusion processing, and the fused data are decoded based on a transformer decoder to obtain predicted second network traffic data.

2. The method according to claim 1, characterized in that The performing inverse Fourier transform processing on at least part of the third data comprises: Processing the third data by using an attention model to determine data to be retained and / or data to be zeroed in the third data; An inverse Fourier transform process is performed on the data retained in the third data.

3. The method according to claim 2, characterized in that The processing of the third data by using the attention model to determine the data to be retained and / or the data to be zeroed in the third data includes: Assigning a corresponding attention value to each data in the third data by using an attention model; Determine the data to be retained in the first part and / or the data to be reset to zero based on the attention value corresponding to each data in the third data; The performing inverse Fourier transform processing on the data retained in the third data includes: performing inverse Fourier transform processing on the data retained in the first part.

4. The method according to claim 3, characterized in that The determining, based on the attention value corresponding to each data in the third data, the data retained in the first part and / or the data returned to zero comprises: When the attention value corresponding to the data is greater than or equal to the first threshold, determining that the data is retained; When the attention value corresponding to the data is less than the first threshold, it is determined that the data is reset to zero.

5. The method according to claim 2, characterized in that: The processing of the third data by using the attention model to determine the data to be retained and / or the data to be zeroed in the third data includes: Assigning a corresponding attention value to each data in the third data by using an attention model; Determine the data to be retained in the first part and / or the data to be reset to zero based on the attention value corresponding to each data in the third data; Deactivating the neural network unit in the attention model with a first probability, determining the data retained and / or the data reset to zero in the first part of the retained data based on the deactivated neural network unit, and obtaining the second part of the retained data; The performing inverse Fourier transform processing on the data retained in the third data includes: performing inverse Fourier transform processing on the data retained in the second part.

6. The method according to claim 3, characterized in that Before performing inverse Fourier transform processing on at least part of the third data, the method further includes: The missing data in the data retained after being processed by the attention model is supplemented.

7. The method according to claim 1, characterized in that Decomposing the first network traffic data into first data and second data includes: Performing average pooling processing on the first network traffic data, and obtaining the first data based on the first network traffic data after the average pooling processing; The first network traffic data is subtracted from the first data to obtain the second data.

8. The method according to claim 1, characterized in that The performing feature fusion processing on the first data, the fourth data and the fifth data includes: The first data, the fourth data and the fifth data are added to perform feature fusion.

9. A network traffic prediction device, characterized in that: The device comprises: an encoder and a decoder; the encoder comprises a Fourier encoder and a transformer encoder; the decoder comprises a Fourier decoder and a transformer decoder; wherein, The Fourier encoder is used to decompose the obtained first network traffic data into first data and second data, and perform Fourier transform processing on the second data to obtain third data representing the frequency domain representation of the second data; the first data represents the long-term trend characteristics of the first network traffic data, and the second data represents the periodic fluctuation characteristics of the first network traffic data; The transformer encoder is used to encode the first network traffic data to obtain fourth data; The Fourier decoder is used to perform inverse Fourier transform processing on at least part of the third data to obtain fifth data representing the time domain representation of at least part of the data; and is also used to perform feature fusion processing on the first data, the fourth data and the fifth data; The transformer decoder is used to decode the data after fusion processing by the Fourier decoder to obtain predicted second network traffic data.

10. The device according to claim 9, characterized in that The Fourier encoder includes a decomposition module and a Fourier transform module; the Fourier decoder includes an inverse Fourier transform module and a feature vector fusion module; wherein, The decomposition module is used to decompose the obtained first network traffic data into the first data and the second data, send the first data to the feature vector fusion module, and send the second data to the Fourier transform module; The Fourier transform module is used to perform Fourier transform processing on the second data to obtain third data representing the frequency domain representation of the second data; The inverse Fourier transform module is used to perform inverse Fourier transform processing on at least part of the third data to obtain fifth data represented by the time domain of at least part of the data, and send the fifth data to the feature vector fusion module; The feature vector fusion module is used to perform feature fusion processing on the first data, the fourth data and the fifth data.

11. The device according to claim 9, characterized in that The device also includes: a frequency domain component selection module, which is used to process the third data through an attention model, determine the data retained and / or the data returned to zero in the third data, and send the data retained in the third data to the Fourier decoder for inverse Fourier transform processing.

12. The device according to claim 11, characterized in that The frequency domain component selection module is used to assign a corresponding attention value to each data in the third data through an attention model; based on the attention value corresponding to each data in the third data, determine the data retained in the first part and / or the data returned to zero, and send the data retained in the first part to the Fourier decoder for inverse Fourier transform processing.

13. The device according to claim 11, characterized in that The frequency domain component selection module is used to assign a corresponding attention value to each data in the third data through an attention model; based on the attention value corresponding to each data in the third data, determine the data retained in the first part and / or the data returned to zero; The neural network unit in the attention model is deactivated with a first probability, and the retained data and / or zeroed data in the first part of the retained data are determined based on the deactivated neural network unit to obtain the second part of the retained data, and the second part of the retained data is sent to the Fourier decoder for inverse Fourier transform processing.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

15. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images