Data prediction method and device, storage medium and electronic equipment

Abstract

The invention provides a data prediction method and device, a storage medium and electronic equipment, and the method comprises the steps: obtaining to-be-predicted data, and determining a first projection result based on the to-be-predicted data; performing Fourier feature extraction on the first projection result to obtain a first Fourier feature, and performing nonlinear activation feature calculation on the first projection result to obtain a first nonlinear activation feature; based on the first Fourier feature and the first nonlinear activation feature, determining a first Fourier analysis network output feature; and based on the first Fourier analysis network output feature, determining a target Fourier analysis network output feature of the to-be-predicted data, and calculating a target predicted value of the to-be-predicted data by using the target Fourier analysis network output feature. According to the embodiment of the invention, the accuracy of data prediction can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular, to a data prediction method, apparatus, storage medium, and electronic device. Background Art

[0002] Currently, neural networks such as Multilayer Perceptron (MLP) and Transformer are widely used and are often used for data prediction, etc. However, the related technologies have serious limitations in the modeling and reasoning capabilities of periodic data, resulting in low accuracy of data prediction. Based on this, there is currently no good solution to how to improve the accuracy of data prediction. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a data prediction method, apparatus, storage medium, and electronic device to solve problems such as low accuracy of data prediction in related technologies. That is to say, embodiments of the present invention can learn the periodicity of data by extracting Fourier features, thereby breaking through the limitations of existing neural networks such as multilayer perceptrons in the modeling and reasoning capabilities of periodic data, and can effectively improve the accuracy of data prediction.

[0004] According to an aspect of an embodiment of the present invention, there is provided a data prediction method, the method comprising:

[0005] Obtain data to be predicted, and determine a first projection result based on the data to be predicted;

[0006] Extract Fourier features from the first projection result to obtain first Fourier features, and calculate non-linear activation features for the first projection result to obtain first non-linear activation features;

[0007] Determine first Fourier analysis network output features based on the first Fourier features and the first non-linear activation features;

[0008] Determine target Fourier analysis network output features of the data to be predicted based on the first Fourier analysis network output features, and calculate a target predicted value of the data to be predicted using the target Fourier analysis network output features.

[0009] According to another aspect of an embodiment of the present invention, there is provided a data prediction apparatus, the apparatus comprising:

[0010] An acquisition unit, configured to acquire data to be predicted;

[0011] A processing unit, configured to determine a first projection result based on the data to be predicted;

[0012] The processing unit is further configured to perform Fourier feature extraction on the first projection result to obtain a first Fourier feature, and perform non-linear activation feature calculation on the first projection result to obtain a first non-linear activation feature;

[0013] The processing unit is further configured to determine an output feature of the first Fourier analysis network based on the first Fourier feature and the first non-linear activation feature;

[0014] The processing unit is further configured to determine an output feature of the target Fourier analysis network for the data to be predicted based on the output feature of the first Fourier analysis network, and calculate a target prediction value for the data to be predicted by using the output feature of the target Fourier analysis network.

[0015] According to another aspect of the embodiments of the present invention, an electronic device is provided. The electronic device includes a processor and a memory storing a program. Wherein, the program includes instructions that, when executed by the processor, cause the processor to execute the method mentioned above.

[0016] According to another aspect of the embodiments of the present invention, a non-transitory computer-readable storage medium storing computer instructions is provided. The computer instructions are used to cause a computer to execute the method mentioned above.

[0017] Embodiments of the present invention can obtain data to be predicted and determine a first projection result based on the data to be predicted. Then, Fourier feature extraction can be performed on the first projection result to obtain a first Fourier feature, and non-linear activation feature calculation can be performed on the first projection result to obtain a first non-linear activation feature; thus, an output feature of the first Fourier analysis network can be determined based on the first Fourier feature and the first non-linear activation feature. Further, an output feature of the target Fourier analysis network for the data to be predicted can be determined based on the output feature of the first Fourier analysis network, and a target prediction value for the data to be predicted can be calculated by using the output feature of the target Fourier analysis network. It can be seen that embodiments of the present invention can learn the periodicity of data through the extraction of Fourier features, thereby breaking through the limitations of existing neural networks such as multi-layer perceptrons in the modeling and reasoning capabilities of periodic data, and can effectively improve the accuracy of data prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In the following description of exemplary embodiments with reference to the accompanying drawings, more details, features, and advantages of the present invention are disclosed. In the drawings:

[0019] Figure 1 A flowchart showing a data prediction method according to an exemplary embodiment of the present invention is shown;

[0020] Figure 2Shows a flowchart of another data prediction method according to an exemplary embodiment of the present invention;

[0021] Figure 3 Shows a schematic diagram of a model layer structure according to an exemplary embodiment of the present invention;

[0022] Figure 4 Shows a schematic block diagram of a data prediction device according to an exemplary embodiment of the present invention;

[0023] Figure 5 Shows a structural block diagram of an exemplary electronic device capable of implementing the embodiments of the present invention. Detailed implementation manners

[0024] Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not used to limit the protection scope of the present invention.

[0025] It should be understood that the various steps recorded in the method embodiments of the present invention can be executed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this regard.

[0026] As used herein, the term "including" and its variations are open-ended, that is, "including but not limited to". The term "based on" is "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description. It should be noted that the concepts such as "first" and "second" mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order of the functions executed by these devices, modules or units or their interdependence.

[0027] It should be noted that the modifications of "one" and "plural" mentioned in the present invention are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly specified in the context, it should be understood as "one or more".

[0028] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0029] It should be noted that the execution subject of the data prediction method provided in the embodiments of the present invention can be one or more electronic devices, and the embodiments of the present invention do not limit this; among them, the electronic device can be a terminal (i.e., a client) or a server. Then, when the execution subject includes multiple electronic devices, and at least one terminal and at least one server are included in the multiple electronic devices, the data prediction method provided in the embodiments of the present invention can be jointly executed by the terminal and the server. Correspondingly, the terminals mentioned here can include, but are not limited to: smart phones, tablet computers, laptop computers, desktop computers, intelligent voice interaction devices, and so on. The server mentioned here can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms, and so on.

[0030] Based on the above description, the embodiments of the present invention propose a data prediction method, which can be executed by the above-mentioned electronic devices (terminals or servers); or, this data prediction method can be jointly executed by the terminal and the server. For the convenience of description, in the following, it will be described by taking the electronic device executing this data prediction method as an example; as Figure 1 shown, this data prediction method may include the following steps S101-S104:

[0031] S101, obtain the data to be predicted, and determine the first projection result based on the data to be predicted.

[0032] Optionally, the data to be predicted can be any periodic data, and the embodiments of the present invention do not limit this; exemplarily, x ∈ [0, 2π], x can represent the data to be predicted, and so on. Optionally, the data periodicity (i.e., the period granularity) to which the data to be predicted belongs can be one week, or one month, or one year, etc., and the embodiments of the present invention do not limit this. It should be noted that the embodiments of the present invention do not limit the dimension of the data to be predicted.

[0033] In the embodiments of the present invention, the acquisition method of the data to be predicted can include, but is not limited to, at least one of the following:

[0034] The first acquisition method: At least one data can be stored in the own storage space of the electronic device. In this case, the electronic device can select (select in sequence, etc.) one data from the at least one data, and use the selected data as the data to be predicted.

[0035] The second acquisition method: The electronic device can obtain the download link of the data to be predicted, and use the data downloaded based on the download link of the data to be predicted as the data to be predicted.

[0036] The third acquisition method: The electronic device can have a data input interface. Then the user can perform a data input operation on the data input interface. Then the electronic device can respond to the data input operation performed by the user, and use the data input by the data input operation as the data to be predicted, so as to obtain the data to be predicted, and so on.

[0037] Optionally, the number of data to be predicted can be one or more, and the embodiments of the present invention do not limit this; when the number of data to be predicted is multiple, data prediction can be performed on each of the multiple data to be predicted respectively. For the convenience of description, hereinafter, only one data to be predicted will be used as an example for illustration.

[0038] S102, perform Fourier feature extraction on the first projection result to obtain the first Fourier feature, and perform non-linear activation feature calculation on the first projection result to obtain the first non-linear activation feature.

[0039] Optionally, when performing non-linear activation feature calculation on the first projection result to obtain the first non-linear activation feature, the electronic device can calculate the first activation projection data of the first projection result through the first activation projection matrix and the first bias in the target data prediction model; and call the first activation function to determine the first non-linear activation feature based on the first activation projection data, so as to perform non-linear activation feature calculation on the first projection result to obtain the first non-linear activation feature. Optionally, a data prediction model can include M layers of Fourier analysis networks. One layer of Fourier analysis network can include a feature extraction layer. One feature extraction layer can include an activation projection matrix and a bias, and one feature extraction layer can correspond to one activation function (that is, one layer of Fourier analysis network can correspond to one activation function); wherein, the first activation projection matrix and the first bias can be the activation projection matrix and the bias in the first layer of Fourier analysis network (that is, the first activation projection matrix and the first bias can be the activation projection matrix and the bias in the feature extraction layer included in the first layer of Fourier analysis network), and the first activation function can be the activation function corresponding to the first layer of Fourier analysis network (that is, the first activation function can be the activation function corresponding to the feature extraction layer included in the first layer of Fourier analysis network). Optionally, the activation function corresponding to one layer of Fourier analysis network can be set according to experience or according to actual needs, that is, the activation function corresponding to one layer of Fourier analysis network can be any activation function, and the embodiments of the present invention do not limit this; optionally, the activation functions corresponding to different layers of Fourier analysis networks can be the same or different, and the embodiments of the present invention do not limit this.

[0040] Exemplarily, the electronic device may use Equation 1.1 to calculate the first activation projection data of the first projection result:

[0041] z ′ = W q z + B q Equation 1.1

[0042] Where z' may represent the first activation projection data, z may represent the first projection result, W q may represent the first activation projection matrix, and B q may represent the first bias.

[0043] Optionally, assuming that the first activation function is the activation function GELU (Gaussian Error Linear Unit, a non-linear activation function based on the Gaussian distribution), then the electronic device may use Equation 1.2 to determine the first non-linear activation feature based on the first activation projection data:

[0044] σ(z ′ ) = GELU(z ′ ) Equation 1.2

[0045] Where σ(z ′ ) may represent the first non-linear activation feature. Optionally, the embodiments of the present invention do not limit the dimensions of the non-linear activation features and the Fourier feature dimensions under each layer of the Fourier analysis network, that is, the embodiments of the present invention do not limit the specific structure of the data prediction model; Exemplarily, the dimension of the non-linear activation features may be 2048 dimensions, and the dimension of a first non-linear activation feature may be 2048, and so on.

[0046] S103. Determine the output feature of the first Fourier analysis network based on the first Fourier feature and the first non-linear activation feature.

[0047] In the embodiments of the present invention, the output feature of the first Fourier analysis network may include the first Fourier feature and the first non-linear activation feature; Specifically, the electronic device may perform feature splicing on the first Fourier feature and the first non-linear activation feature to obtain the output feature of the first Fourier analysis network, so as to determine the output feature of the first Fourier analysis network.

[0048] Optionally, the electronic device may input the first projection result into the feature extraction layer in the first layer of the Fourier analysis network included in the target data prediction model to output the output feature of the first Fourier analysis network.

[0049] S104. Determine the target output feature of the Fourier analysis network for the data to be predicted based on the output feature of the first Fourier analysis network, and calculate the target prediction value of the data to be predicted using the target output feature of the Fourier analysis network.

[0050] Optionally, a data prediction model may further include a prediction output layer, and a prediction output layer may include a prediction output layer weight matrix and a prediction output layer bias; based on this, the electronic device may call the prediction output layer weight matrix and the prediction output layer bias in the target data prediction model to calculate the prediction value of the target Fourier analysis network output feature, and obtain the target prediction value of the data to be predicted. That is to say, the target Fourier analysis network output feature may be input into the prediction output layer in the target data prediction model to output the target prediction value of the data to be predicted.

[0051] Exemplarily, the electronic device may calculate the target prediction value of the data to be predicted by using the target Fourier analysis network output feature through Formula 1.3:

[0052] y = W out Φ(x) + B out Formula 1.3

[0053] where y may represent the target prediction value, W out may represent the prediction output layer weight matrix, B out may represent the prediction output layer bias, and Φ(x) may represent the target Fourier analysis network output feature.

[0054] In an embodiment of the present invention, the data to be predicted may be obtained, and the first projection result may be determined based on the data to be predicted. Then, Fourier feature extraction may be performed on the first projection result to obtain the first Fourier feature, and non-linear activation feature calculation may be performed on the first projection result to obtain the first non-linear activation feature; thus, the first Fourier analysis network output feature may be determined based on the first Fourier feature and the first non-linear activation feature. Further, the target Fourier analysis network output feature of the data to be predicted may be determined based on the first Fourier analysis network output feature, and the target prediction value of the data to be predicted may be calculated by using the target Fourier analysis network output feature. It can be seen that in an embodiment of the present invention, the periodicity of the data can be learned by extracting the Fourier feature, thereby breaking through the limitations of existing neural networks such as multi-layer perceptrons in the modeling and reasoning capabilities of periodic data, and effectively improving the accuracy of data prediction.

[0055] Based on the above description, another data prediction method is also proposed in an embodiment of the present invention. Correspondingly, this data prediction method may be executed by the above-mentioned electronic device (terminal or server); or, this data prediction method may be jointly executed by the terminal and the server. For the convenience of description, in the following, it is assumed that the electronic device executes this data prediction method as an example for illustration; please refer to Figure 2 , and this data prediction method may include the following steps S201 - S206:

[0056] S201, obtain the data to be predicted, and determine the first projection result based on the data to be predicted.

[0057] Optionally, when determining the first projection result based on the data to be predicted, the electronic device may normalize the data to be predicted to obtain the normalized result of the data to be predicted; for example, formula 2.1 may be used to normalize the data to be predicted to obtain the normalized result of the data to be predicted:

[0058]

[0059] where x norm may represent the normalized result of the data to be predicted, min(x) may represent the minimum value of each data dimension in the data range where the data to be predicted is located, and man(x) may represent the maximum value of each data dimension in the data range where the data to be predicted is located; based on this, each dimension of the data to be predicted can be normalized separately, that is, the normalized result of the data to be predicted may include the results of normalizing each dimension of the data to be predicted separately.

[0060] Furthermore, the projection linear transformation may be performed on the normalized result of the data to be predicted to obtain the first projection result. Optionally, the electronic device may use formula 2.2 to perform the projection linear transformation on the normalized result of the data to be predicted to obtain the first projection result:

[0061] z = W p x norm Formula 2.2

[0062] where W p may represent the projection matrix. At this time, the projection matrix can be used for linear transformation (i.e., projection linear transformation). Optionally, a one-layer Fourier analysis network may include a projection transformation layer, and a projection transformation layer can be represented by a projection matrix (i.e., may include a projection matrix); based on this, the first projection result can be obtained by performing a linear transformation through the projection matrix in the first layer of the Fourier analysis network. That is to say, the normalized result of the data to be predicted can be input into the projection transformation layer in the first layer of the Fourier analysis network included in the target data prediction model to output the first projection result. Optionally, the dimension of a projection result can be any dimension, that is, the model architecture can be set according to experience or actual requirements, and the embodiments of the present invention do not limit this; for example, the dimension of the first projection result can be 512, and so on.

[0063] Optionally, in other embodiments, the electronic device may also perform a projection linear transformation on the data to be predicted to obtain the first projection result. At this time, normalization processing may not be performed, and so on; the present invention does not limit this.

[0064] S202. Extract the cosine feature from the first projection result to obtain the first cosine feature.

[0065] Based on this, the first cosine feature can be cos(z).

[0066] S203. Perform sine feature extraction on the first projection result to obtain a first sine feature, so as to perform Fourier feature extraction on the first projection result to obtain a first Fourier feature; wherein, the first Fourier feature includes a first cosine feature and a first sine feature.

[0067] Based on this, the first sine feature can be sin(z).

[0068] Wherein, a Fourier feature may include a cosine feature and a sine feature.

[0069] S204. Perform non-linear activation feature calculation on the first projection result to obtain a first non-linear activation feature.

[0070] In an embodiment of the present invention, the electronic device may input the first projection result into a feature extraction layer in a first-layer Fourier analysis network included in a target data prediction model, so as to output a first sine feature, a first cosine feature, and a first non-linear activation feature.

[0071] S205. Determine an output feature of the first Fourier analysis network based on the first Fourier feature and the first non-linear activation feature.

[0072] In an embodiment of the present invention, the first Fourier feature includes a first sine feature and a first cosine feature; then correspondingly, the first sine feature, the first cosine feature, and the first non-linear activation feature may be subjected to feature splicing to obtain an output feature of the first Fourier analysis network, so as to implement feature splicing of the first Fourier feature and the first non-linear activation feature. It should be noted that the present invention embodiment does not limit the feature splicing order, that is, the feature splicing order may be set according to experience or actual requirements; exemplarily, the output feature of the first Fourier analysis network may be [cos(z)||sin(z)||σ(z’)], that is, at this time, the feature splicing may be performed in the order of the first cosine feature, the first sine feature, and the first non-linear activation feature, and so on. Based on this, assuming that the dimension of the first sine feature is 512, the dimension of the first cosine feature is 512, and the dimension of the first non-linear activation feature is 2048, then the dimension of the output feature of the first Fourier analysis network may be 3072.

[0073] S206. Determine an output feature of the target Fourier analysis network for the data to be predicted based on the output feature of the first Fourier analysis network, and calculate a target prediction value for the data to be predicted by using the output feature of the target Fourier analysis network.

[0074] In an embodiment of the present invention, the target prediction value may be predicted by a target data prediction model, and the target data prediction model may include M layers of Fourier analysis networks, where M is a positive integer; among them, the output features of the first Fourier analysis network are output by the first layer of Fourier analysis network in the target data prediction model. That is to say, the normalized result of the data to be predicted can be input into the first layer of Fourier analysis network in the target data prediction model to output the output features of the first Fourier analysis network. Specifically, the projection linear transformation of the normalized result of the data to be predicted can be performed through the projection transformation layer included in the first layer of Fourier analysis network in the target data prediction model to obtain the first projection result, and the feature extraction of the first projection result can be performed through the feature extraction layer included in the first layer of Fourier analysis network in the target data prediction model (which may include Fourier feature extraction and non-linear activation feature calculation (also referred to as non-linear activation feature extraction)) to obtain the first Fourier feature and the first non-linear activation feature. Then, the first Fourier feature and the first non-linear activation feature are feature-stitched to obtain the output features of the first Fourier analysis network, so as to output the output features of the first Fourier analysis network through the first layer of Fourier analysis network in the target data prediction model, as Figure 3 shown; optionally, one layer of Fourier analysis network may further include a feature stitching module to implement feature stitching, and so on.

[0075] Based on this, when determining the output features of the target Fourier analysis network for the data to be predicted based on the output features of the first Fourier analysis network, the electronic device may input the output features of the first Fourier analysis network into the second-layer Fourier analysis network in the target data prediction model, so as to output the output features of the second Fourier analysis network for the data to be predicted through the second-layer Fourier analysis network. That is to say, the projection linear transformation of the output features of the first Fourier analysis network (i.e., the output features of the data to be predicted under the first-layer Fourier analysis network) can be performed through the projection transformation layer included in the second-layer Fourier analysis network of the target data prediction model, and the feature extraction of the projection result here (i.e., the second projection result) can be performed through the feature extraction layer included in the second-layer Fourier analysis network until the output features of the second Fourier analysis network (i.e., the output features of the data to be predicted under the second-layer Fourier analysis network) are output. Optionally, the output features of the mth-layer Fourier analysis network for a piece of data can also be referred to as the output features of the corresponding data under the mth-layer Fourier analysis network or the output features of the mth-layer Fourier analysis network of the corresponding data, where m ∈ [1, M]; correspondingly, the projection result of the mth-layer Fourier analysis network for a model input data can also be referred to as the projection result of the corresponding model input data (such as the data to be predicted) under the mth-layer Fourier analysis network or the mth projection result of the corresponding model input data. For example, the first projection result can also be referred to as the first projection result of the data to be predicted, and so on. Optionally, a data prediction model may further include a normalization module to perform normalization processing on the input data, so as to input the normalization result into the corresponding first-layer Fourier analysis network, and so on.

[0076] Further, the output features of the second Fourier analysis network can be continuously input into the third Fourier analysis network in the target data prediction model (to output the output features of the data to be predicted under the third Fourier analysis network), until the output features of the Mth Fourier analysis network of the data to be predicted are output through the Mth Fourier analysis network in the target data prediction model (i.e., the output features of the data to be predicted under the Mth Fourier analysis network), and the output features of the Mth Fourier analysis network are used as the target Fourier analysis network output features of the data to be predicted. In other words, the features output by the ith Fourier analysis network can be input into the (i + 1)th Fourier analysis network to perform projection linear transformation, feature extraction, etc., so as to obtain the features output by the (i + 1)th Fourier analysis network, and so on; where i is a positive integer less than M. Exemplarily, when M is 3, the output features of the third Fourier analysis network can be used as the target Fourier analysis network output features. Based on this, the embodiments of the present invention can respectively perform projection linear transformation on the input data (such as the normalization result of the data to be predicted or the output features of the previous Fourier analysis network) through each layer of the Fourier analysis network, and perform feature extraction on the corresponding projection results, so as to sequentially obtain the output features of each Fourier analysis network, and further obtain the target Fourier analysis network output features.

[0077] Optionally, the dimensions of the output features of different Fourier analysis networks can be the same.

[0078] Optionally, the electronic device can also obtain N training data, where N is a positive integer; and call the initial data prediction model to respectively predict the training prediction values of each training data in the N training data; where the initial data prediction model includes M layers of Fourier analysis networks, that is, a data prediction model can include M layers of Fourier analysis networks. In other words, each training data can be respectively input into the initial data prediction model to output the training prediction values of each training data through the initial data prediction model.

[0079] Then correspondingly, the electronic device can calculate the model loss value of the initial data prediction model based on the training prediction values and label values of each training data; based on this, the model parameters in the initial data prediction model can be optimized in the direction of reducing the model loss value (i.e., optimizing the weights, biases, etc. in the model, an activation projection matrix or a projection matrix, etc. can include at least one weight, that is, a matrix can be composed of the corresponding at least one weight), to obtain the initial data prediction model with optimized model, and determine the target data prediction model based on the initial data prediction model with optimized model.

[0080] Optionally, the electronic device can use Equation 2.3 to calculate the model loss value of the initial data prediction model:

[0081]

[0082] Among them, L can represent the model loss value, and y n can represent the training prediction value of the nth training data among N training data, and r n can represent the label value (i.e., the true target value) of the nth training data among N training data. Optionally, the electronic device can update the model parameters using algorithms such as backpropagation and stochastic gradient descent to optimize the model parameters in the initial data prediction model in the direction of reducing the model loss value, etc.; the embodiments of the present invention do not limit this.

[0083] The embodiments of the present invention can obtain the data to be predicted and determine the first projection result based on the data to be predicted. Then, cosine feature extraction can be performed on the first projection result to obtain the first cosine feature; and sine feature extraction can be performed on the first projection result to obtain the first sine feature, so as to perform Fourier feature extraction on the first projection result to obtain the first Fourier feature; where the first Fourier feature includes the first cosine feature and the first sine feature. Correspondingly, non-linear activation feature calculation can be performed on the first projection result to obtain the first non-linear activation feature. Based on this, the first Fourier analysis network output feature can be determined based on the first Fourier feature and the first non-linear activation feature. Further, based on the first Fourier analysis network output feature, the target Fourier analysis network output feature of the data to be predicted can be determined, and the target prediction value of the data to be predicted can be calculated using the target Fourier analysis network output feature. It can be seen that the embodiments of the present invention propose an efficient hybrid layer structure based on sine and cosine (i.e., the structure of the data prediction model), which can directly integrate periodic information into the operation and structure of the network from the data level by introducing sine and cosine, thus getting rid of the limitations of existing neural networks, etc. when dealing with periodic patterns; based on this, the embodiments of the present invention can optimize the modeling of the input data by the model compared with the fully connected layer, and obtain a better feature representation than the original features (i.e., obtain a more accurate Fourier analysis network output feature), greatly exceeding the fully connected layer in performance, and greatly improving the accuracy in the prediction stage. Moreover, the layer structure mentioned in the embodiments of the present invention can achieve better performance with fewer parameters and lower computational costs (i.e., while enhancing the feature representation ability, reducing the parameters of the model). That is to say, the embodiments of the present invention can effectively avoid the situation where the number of parameters of the fully connected layer relied on by architectures such as multi-layer perceptrons increases rapidly with the increase of the input dimension and the number of hidden layer units, so as to effectively improve the accuracy of data prediction while improving the computational efficiency; in addition, the model structure proposed in the embodiments of the present invention can be seamlessly integrated into various architectures containing fully connected layers, and has extremely strong scalability.

[0084] Based on the description of the related embodiments of the above data prediction method, an embodiment of the present invention further provides a data prediction device, which may be a computer program (including program code) running in an electronic device; as Figure 4 shown, the data prediction device may include an acquisition unit 401 and a processing unit 402. The data prediction device can execute Figure 1 or Figure 2 the data prediction method shown, that is, the data prediction device can run the above units:

[0085] The acquisition unit 401 is used to acquire the data to be predicted;

[0086] The processing unit 402 is used to determine a first projection result based on the data to be predicted;

[0087] The processing unit 402 is further used to perform Fourier feature extraction on the first projection result to obtain a first Fourier feature, and perform non-linear activation feature calculation on the first projection result to obtain a first non-linear activation feature;

[0088] The processing unit 402 is further used to determine a first Fourier analysis network output feature based on the first Fourier feature and the first non-linear activation feature;

[0089] The processing unit 402 is further used to determine a target Fourier analysis network output feature of the data to be predicted based on the first Fourier analysis network output feature, and calculate a target prediction value of the data to be predicted by using the target Fourier analysis network output feature.

[0090] In one implementation, a Fourier feature includes a cosine feature and a sine feature; when the processing unit 402 performs Fourier feature extraction on the first projection result to obtain a first Fourier feature, it may specifically be used for:

[0091] Performing cosine feature extraction on the first projection result to obtain a first cosine feature;

[0092] Performing sine feature extraction on the first projection result to obtain a first sine feature, so as to implement performing Fourier feature extraction on the first projection result to obtain a first Fourier feature; wherein, the first Fourier feature includes the first cosine feature and the first sine feature.

[0093] In another implementation, when the processing unit 402 determines a first projection result based on the data to be predicted, it may specifically be used for:

[0094] Performing normalization processing on the data to be predicted to obtain a normalization result of the data to be predicted;

[0095] Perform a projection linear transformation on the normalization result of the data to be predicted to obtain a first projection result.

[0096] In another implementation, the target prediction value is predicted by a target data prediction model, and the target data prediction model includes M layers of Fourier analysis networks, where M is a positive integer; among them, the output features of the first Fourier analysis network are output by the first layer of Fourier analysis network in the target data prediction model; when the processing unit 402 determines the output features of the target Fourier analysis network of the data to be predicted based on the output features of the first Fourier analysis network, it can specifically be used for:

[0097] Input the output features of the first Fourier analysis network into the second layer of Fourier analysis network in the target data prediction model to output the output features of the second Fourier analysis network of the data to be predicted through the second layer of Fourier analysis network;

[0098] Continue to input the output features of the second Fourier analysis network into the third layer of Fourier analysis network in the target data prediction model until the output features of the Mth Fourier analysis network of the data to be predicted are output through the Mth layer of Fourier analysis network in the target data prediction model, and use the output features of the Mth Fourier analysis network as the output features of the target Fourier analysis network of the data to be predicted.

[0099] In another implementation, when the processing unit 402 calculates the target prediction value of the data to be predicted using the output features of the target Fourier analysis network, it can specifically be used for:

[0100] Call the weight matrix and bias of the prediction output layer in the target data prediction model to calculate the prediction value of the output features of the target Fourier analysis network to obtain the target prediction value of the data to be predicted.

[0101] In another implementation, when the processing unit 402 calculates the non-linear activation features of the first projection result to obtain the first non-linear activation features, it can specifically be used for:

[0102] Calculate the first activation projection data of the first projection result through the first activation projection matrix and the first bias in the target data prediction model;

[0103] Call the first activation function to determine the first non-linear activation features based on the first activation projection data, so as to implement the calculation of the non-linear activation features of the first projection result to obtain the first non-linear activation features.

[0104] In another implementation, the target prediction value is predicted by a target data prediction model, and the acquisition unit 401 can also be used for:

[0105] Obtain N training data, where N is a positive integer;

[0106] The processing unit 402 can also be used for:

[0107] Call the initial data prediction model to respectively predict the training prediction values of each of the N training data; wherein, the initial data prediction model includes M layers of Fourier analysis networks, and M is a positive integer;

[0108] Based on the training prediction values and label values of each of the training data, calculate the model loss value of the initial data prediction model;

[0109] According to the direction of reducing the model loss value, optimize the model parameters in the initial data prediction model to obtain the initial data prediction model with optimized model, and based on the initial data prediction model with optimized model, determine the target data prediction model.

[0110] According to an embodiment of the present invention, Figure 4 Each unit in the data prediction device shown can be respectively or all combined into one or several other units to form, or some of them can be further split into multiple smaller units with functional division to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present invention. The above units are divided based on logical functions. In practical applications, the function of one unit can also be realized by multiple units, or the functions of multiple units can be realized by one unit. In other embodiments of the present invention, any data prediction device can also include other units. In practical applications, these functions can also be assisted by other units and can be realized by the cooperation of multiple units.

[0111] According to another embodiment of the present invention, it can be achieved by running a computer program (including program code) capable of executing the respective steps involved in the corresponding method shown in Figure 1 or Figure 2 on a general electronic device such as a computer including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM), to construct the data prediction device shown in Figure 4 and to implement the data prediction method of the embodiments of the present invention. The computer program can be recorded on a computer storage medium such as a computer storage medium, loaded into the above-mentioned electronic device through the computer storage medium, and run therein.

[0112] Embodiments of the present invention can obtain data to be predicted and determine a first projection result based on the data to be predicted. Then, Fourier feature extraction can be performed on the first projection result to obtain first Fourier features, and non-linear activation feature calculation can be performed on the first projection result to obtain first non-linear activation features. Thus, based on the first Fourier features and the first non-linear activation features, the output features of the first Fourier analysis network can be determined. Further, based on the output features of the first Fourier analysis network, the target output features of the Fourier analysis network for the data to be predicted can be determined, and the target prediction value of the data to be predicted can be calculated using the target output features of the Fourier analysis network. It can be seen that the embodiments of the present invention can learn the periodicity of data through the extraction of Fourier features, thereby breaking through the limitations of existing neural networks such as multi-layer perceptrons in the modeling and reasoning capabilities of periodic data, and effectively improving the accuracy of data prediction.

[0113] Based on the descriptions of the above method embodiments and apparatus embodiments, an exemplary embodiment of the present invention further provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program that can be executed by the at least one processor, and when the computer program is executed by the at least one processor, it is used to cause the electronic device to execute the method according to the embodiments of the present invention.

[0114] An exemplary embodiment of the present invention further provides a non-transitory computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor of a computer, it is used to cause the computer to execute the method according to the embodiments of the present invention.

[0115] An exemplary embodiment of the present invention further provides a computer program product, including a computer program, wherein when the computer program is executed by a processor of a computer, it is used to cause the computer to execute the method according to the embodiments of the present invention.

[0116] Reference Figure 5 , the structural block diagram of an electronic device 500 that can be used as a server or a client of the present invention will now be described. It is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described herein and / or claimed.

[0117] As Figure 5As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to computer programs stored in a read-only memory (ROM) 502 or computer programs loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0118] Multiple components in the electronic device 500 are connected to the I / O interface 505, including: an input unit 506, an output unit 507, a storage unit 508, and a communication unit 509. The input unit 506 can be any type of device capable of inputting information into the electronic device 500. The input unit 506 can receive input digital or character information and generate key signal inputs related to the user settings and / or function controls of the electronic device. The output unit 507 can be any type of device capable of presenting information and can include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 508 can include, but is not limited to, a magnetic disk, an optical disk. The communication unit 509 allows the electronic device 500 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks and can include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a BluetoothTM device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0119] The computing unit 501 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 501 executes the various methods and processes described above. For example, in some embodiments, the data prediction method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 500 via the ROM 502 and / or the communication unit 509. In some embodiments, the computing unit 501 can be configured to execute the data prediction method by any other appropriate means (e.g., by means of firmware).

[0120] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to the processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing devices, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as an independent software package and partially on a remote machine, or entirely on a remote machine or server.

[0121] In the context of the present invention, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0122] As used in the present invention, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, apparatus, and / or device (e.g., a disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term “machine-readable signal” refers to any signal for providing machine instructions and / or data to a programmable processor.

[0123] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0124] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0125] A computer system can include clients and servers. The clients and servers are generally far from each other and typically interact through a communication network. The client - server relationship is created by computer programs that run on the respective computers and have a client - server relationship with each other.

[0126] Moreover, it should be understood that the above - disclosed are only the preferred embodiments of the present invention, and of course cannot be used to limit the scope of rights of the present invention. Therefore, equivalent changes made according to the claims of the present invention still fall within the scope covered by the present invention.

Claims

1. A data prediction method, characterized in that, Including: Obtain the data to be predicted, and determine a first projection result based on the data to be predicted; Perform Fourier feature extraction on the first projection result to obtain a first Fourier feature, and perform non-linear activation feature calculation on the first projection result to obtain a first non-linear activation feature; Determine an output feature of the first Fourier analysis network based on the first Fourier feature and the first non-linear activation feature; Determine an output feature of the target Fourier analysis network for the data to be predicted based on the output feature of the first Fourier analysis network, and calculate a target prediction value for the data to be predicted by using the output feature of the target Fourier analysis network.

2. The method according to claim 1, wherein A Fourier feature includes a cosine feature and a sine feature; the performing Fourier feature extraction on the first projection result to obtain a first Fourier feature includes: Perform cosine feature extraction on the first projection result to obtain a first cosine feature; Perform sine feature extraction on the first projection result to obtain a first sine feature, so as to perform Fourier feature extraction on the first projection result to obtain a first Fourier feature; wherein, the first Fourier feature includes the first cosine feature and the first sine feature.

3. The method according to claim 1 or 2, characterized in that, The determining a first projection result based on the data to be predicted includes: Perform normalization processing on the data to be predicted to obtain a normalization result of the data to be predicted; Perform projection linear transformation on the normalization result of the data to be predicted to obtain a first projection result.

4. The method according to claim 1 or 2, characterized in that The target prediction value is predicted by a target data prediction model, and the target data prediction model includes M layers of Fourier analysis networks, where M is a positive integer; wherein, the output feature of the first Fourier analysis network is output by the first layer of Fourier analysis network in the target data prediction model; the determining an output feature of the target Fourier analysis network for the data to be predicted based on the output feature of the first Fourier analysis network includes: Input the output feature of the first Fourier analysis network into the second layer of Fourier analysis network in the target data prediction model, so as to output an output feature of the second Fourier analysis network for the data to be predicted by the second layer of Fourier analysis network; Continue to input the output feature of the second Fourier analysis network into the third layer of Fourier analysis network in the target data prediction model until the output feature of the Mth Fourier analysis network for the data to be predicted is output by the Mth layer of Fourier analysis network in the target data prediction model, and use the output feature of the Mth Fourier analysis network as the output feature of the target Fourier analysis network for the data to be predicted.

5. The method according to claim 4, characterized in that The calculating a target prediction value for the data to be predicted by using the output feature of the target Fourier analysis network includes: Call a weight matrix and a bias of a prediction output layer in the target data prediction model, and calculate a prediction value for the output feature of the target Fourier analysis network to obtain a target prediction value for the data to be predicted.

6. The method according to claim 1 or 2, characterized in that, The performing non-linear activation feature calculation on the first projection result to obtain a first non-linear activation feature includes: Calculate the first activation projection data of the first projection result through the first activation projection matrix and the first bias in the target data prediction model; Call the first activation function to determine the first non-linear activation feature based on the first activation projection data, so as to perform non-linear activation feature calculation on the first projection result and obtain the first non-linear activation feature.

7. The method according to claim 1 or 2, characterized in that, The target prediction value is predicted by a target data prediction model, and the method further includes: Obtain N training data, where N is a positive integer; Call the initial data prediction model to predict the training prediction values of each of the N training data respectively; wherein, the initial data prediction model includes M layers of Fourier analysis networks, and M is a positive integer; Calculate the model loss value of the initial data prediction model based on the training prediction values and label values of the respective training data; Optimize the model parameters in the initial data prediction model in the direction of reducing the model loss value to obtain the initial data prediction model with optimized model, and determine the target data prediction model based on the initial data prediction model with optimized model.

8. A data prediction device, characterized in that, The device includes: An acquisition unit for acquiring data to be predicted; A processing unit for determining a first projection result based on the data to be predicted; The processing unit is further configured to perform Fourier feature extraction on the first projection result to obtain a first Fourier feature, and perform non-linear activation feature calculation on the first projection result to obtain a first non-linear activation feature; The processing unit is further configured to determine a first Fourier analysis network output feature based on the first Fourier feature and the first non-linear activation feature; The processing unit is further configured to determine the target Fourier analysis network output feature of the data to be predicted based on the first Fourier analysis network output feature, and calculate the target prediction value of the data to be predicted by using the target Fourier analysis network output feature.

9. An electronic device, characterized in that, Comprising: A processor; And A memory storing a program, wherein the program includes instructions that, when executed by the processor, cause the processor to execute the method according to any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause a computer to execute the method according to any one of claims 1-7.

Images