Revenue prediction method and system based on statistical fusion self-attention

Abstract

The invention relates to the technical field of income prediction, in particular to an income prediction method and system based on statistical fusion self-attention, and the method comprises the steps: obtaining to-be-predicted data in any unit time in real time; inputting the to-be-predicted data into an income prediction model to obtain a package income prediction result used for assisting service decision making in the next unit time; all attention layers of any encoder or decoder in the income prediction model are fusion self-attention layers; any fusion self-attention layer is used for performing feature extraction on a received first input vector according to a preset autoregressive moving average algorithm and a self-attention mechanism to obtain a corresponding time sequence prediction weight and at least one user portrait attention weight, and performing feature fusion on the time sequence prediction weight and the attention weight of each user portrait. The method has the beneficial effects that the model training cost is reduced while the user portrait of the user is concerned and future income prediction is accurately carried out.

Description

Technical Field

[0001] The present invention relates to the technical field of revenue prediction, and particularly to a revenue prediction method and system based on statistical fusion self-attention. Background Art

[0002] With the increasing demand of enterprises for accurate financial prediction, traditional prediction methods have been difficult to meet the complex and changeable market demands. Existing statistical models (such as autoregressive AR, moving average model MA, and autoregressive integrated moving average model ARIMA) are based on the basic assumption that the data to be predicted is linearly correlated. However, package revenue is affected by factors such as promotional activities, festivals, user habit preferences, and user account status, and the analysis accuracy of using pure statistical models is not high. In addition, pure statistical models cannot directly combine user portrait data, so when a user-level model is required to support user-level revenue prediction, the training cost is high. For existing neural network-based methods, such as LSTM, although they can better process non-linearly correlated data and can also combine user portrait attributes as features for revenue prediction, the network structure required by LSTM to achieve the goal of long-term memory is relatively complex. Under a large number of users, the training cost is high, and the training speed is slow due to the intelligent serial training of network layers, so the input-output risk is relatively large.

[0003] In addition, whether it is ARIMA or LSTM, there is a problem of error propagation when performing multi-step prediction. One mistake leads to subsequent mistakes, making it difficult to apply in actual scenarios, with a low ROI and high risk. Summary of the Invention

[0004] (I) Technical Problems to be Solved

[0005] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a revenue prediction method and system based on statistical fusion self-attention, which solves the technical problems that existing statistical models cannot consider the influence of factors such as promotional activities, festivals, user habit preferences, and user account status, resulting in low analysis accuracy, and the technical problems of high training cost, slow training speed, and relatively large input-output risk of neural network-based methods.

[0006] (II) Technical Solutions

[0007] To achieve the above object, the main technical solutions adopted by the present invention include:

[0008] In the first aspect, an embodiment of the present invention provides a revenue prediction method based on statistical fusion self-attention, including:

[0009] S11. Obtain the data to be predicted within any unit time in real time;

[0010] S12. Input the to-be-predicted data into a pre-set revenue prediction model to obtain a package revenue prediction result for assisting business decision-making in the next unit time;

[0011] The revenue prediction model is a prediction model constructed based on the transformer network;

[0012] All attention layers of any encoder or decoder in the revenue prediction model are fused self-attention layers;

[0013] Any of the fused self-attention layers is used to: perform feature extraction on the received first input vector according to a pre-set autoregressive moving average algorithm and self-attention mechanism respectively, obtain corresponding time-series prediction weights and at least one user portrait attention weight respectively, and perform feature fusion on the time-series prediction weights and each user portrait attention weight respectively;

[0014] The first input vector is the to-be-predicted data or the data generated after the to-be-predicted data is processed by any encoder / decoder.

[0015] Optionally, the first input vector includes a revenue vector and at least one user portrait vector;

[0016] Then, any of the fused self-attention layers, performing feature extraction on the received first input vector according to a pre-set autoregressive moving average algorithm and self-attention mechanism respectively, obtaining corresponding revenue time-series features and at least one user portrait attention weight respectively, and performing feature fusion on the revenue time-series features and each user portrait attention weight respectively, includes:

[0017] Performing time-series prediction on the received revenue vector according to a pre-set autoregressive moving average algorithm to obtain the time-series prediction weights corresponding to the first input vector;

[0018] Performing feature extraction on each received user portrait vector according to a pre-set self-attention mechanism to obtain the user portrait attention weights corresponding to each user portrait vector;

[0019] Performing feature fusion on the time-series prediction weights and the user portrait attention weights corresponding to each user portrait vector respectively.

[0020] Optionally, then any of the fused self-attention layers, performing trend feature extraction on the received first input vector according to a pre-set autoregressive moving average algorithm, obtaining the corresponding to the first input vector, includes:

[0021] Obtaining the time-series prediction weights corresponding to the first input vector according to the revenue vector and a pre-set formula one; the formula one is:

[0022] Attention(ARIMA) = softmax(p×q×d);

[0023] p is the AR order, d is the pre-set differencing order, and q is the MA order;

[0024] Wherein k is the income vector, and n and m are pre-set constants.

[0025] Optionally, any one of the fusion self-attention layers extracts features from each received user portrait vector according to a pre-set self-attention mechanism to obtain the user portrait attention weights corresponding to each user portrait vector, including:

[0026] Performing a linear transformation on each received user portrait vector to obtain a query vector, a key vector, and a value vector corresponding to each user portrait vector;

[0027] According to the query vectors, key vectors, and value vectors of all user portrait vectors, obtaining at least one similarity score corresponding to each user portrait vector; the similarity score is the similarity between any one user portrait vector and any other user portrait vector;

[0028] Based on a pre-set SoftMax function, performing normalization processing on each similarity score corresponding to each user portrait vector;

[0029] Performing a weighted sum of all the normalized similarity scores corresponding to each user portrait vector to obtain the user portrait attention weights corresponding to each user portrait vector.

[0030] Optionally, before S11, it further includes:

[0031] S10. Training a pre-constructed income prediction model based on a pre-constructed prediction training set to obtain a trained income prediction model;

[0032] The prediction training set includes at least one time-series panel data of different time periods, and the package income prediction results corresponding to each time-series panel data;

[0033] The time-series panel data includes customer basic data, business basic package data, basic portrait labels, business change data, and unit time income data.

[0034] Optionally, any one of the decoders in the income prediction model further includes a regression layer before the output layer;

[0035] Any one of the regression layers is used to: compress the received vector and use it as the output of the corresponding encoder;

[0036] The output of any one encoder is income prediction data.

[0037] Optionally, the first input vector includes an income vector;

[0038] Then, for any of the fusion self-attention layers, according to the preset autoregressive moving average algorithm and self-attention mechanism, feature extraction is performed on the received first input vector respectively to obtain corresponding income time-series features and at least one user profile attention weight, and feature fusion is performed on the income time-series features and each user profile attention weight respectively, including:

[0039] Performing time-series prediction on the received income vector according to the preset autoregressive moving average algorithm to obtain a time-series prediction weight corresponding to the first input vector;

[0040] Performing feature fusion on the time-series prediction weight and at least one preset historical user profile attention weight respectively.

[0041] Optionally, any of the encoders includes a connected fusion attention layer and a feed-forward neural network;

[0042] Any of the decoders includes a fusion attention layer, a fusion cross-attention layer, and a regression layer connected in sequence;

[0043] Among them, the feed-forward neural network of the encoder is connected to the fusion cross-attention layer of the decoder.

[0044] In a second aspect, an embodiment of the present invention provides an income prediction system based on statistical fusion self-attention, including: a central prediction module, and a data acquisition module communicatively connected to the central prediction module;

[0045] The data acquisition module is configured to obtain data to be predicted within any unit time in real time;

[0046] The central prediction module is configured to input the data to be predicted into a preset income prediction model to obtain a package income prediction result for assisting business decision-making in the next unit time;

[0047] Among them, the income prediction model is the above-mentioned income prediction model.

[0048] In a third aspect, an embodiment of the present invention provides a computer-readable medium, on which instructions are stored, and when executed by one or more processors, cause the processors to execute the above-mentioned income prediction method based on statistical fusion self-attention.

[0049] (III) Advantageous Effects

[0050] The beneficial effects of the present invention are as follows: A revenue prediction method based on statistical fusion self-attention of the present invention, due to the adoption of a fusion self-attention layer, compared with the prior art, can, while paying attention to the user profile of the user and accurately predicting future revenue, reduce the model training cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a schematic flowchart of a revenue prediction method based on statistical fusion self-attention provided by an embodiment of the present invention;

[0052] Figure 2 It is a schematic diagram of the transformer network architecture provided by an embodiment of the present invention;

[0053] Figure 3 It is a schematic diagram of the ARIMA architecture provided by an embodiment of the present invention;

[0054] Figure 4 It is a schematic diagram of the structure of the fusion attention layer provided by an embodiment of the present invention;

[0055] Figure 5 It is a schematic diagram of the structure of the revenue prediction model provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0056] In order to better explain the present invention for easy understanding, the present invention will be described in detail below with reference to the accompanying drawings through specific embodiments.

[0057] A revenue prediction method based on statistical fusion self-attention proposed in an embodiment of the present invention, due to the adoption of a fusion self-attention layer, compared with the prior art, can, while paying attention to the user profile of the user and accurately predicting future revenue, reduce the model training cost.

[0058] In order to better understand the above technical solution, the exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present invention can be understood more clearly and thoroughly, and the scope of the present invention can be fully conveyed to those skilled in the art.

[0059] Embodiment 1

[0060] This embodiment provides a revenue prediction method based on statistical fusion self-attention, including:

[0061] S11. Real-time obtain the data to be predicted within any unit time;

[0062] S12. Input the to-be-predicted data into a pre-set revenue prediction model to obtain a package revenue prediction result for assisting business decision-making in the next unit of time.

[0063] The revenue prediction model is a prediction model constructed based on the Transformer network.

[0064] All attention layers of any encoder or decoder in the revenue prediction model are fused self-attention layers.

[0065] Any of the fused self-attention layers is used to: extract features from the received first input vector according to the pre-set autoregressive integrated moving average algorithm and self-attention mechanism respectively, obtain corresponding time-series prediction weights and at least one user profile attention weight respectively, and perform feature fusion between the time-series prediction weights and each user profile attention weight.

[0066] The first input vector is the to-be-predicted data or the data generated after the to-be-predicted data is processed by any encoder / decoder.

[0067] A revenue prediction method based on statistical fused self-attention proposed in an embodiment of the present invention, due to the adoption of fused self-attention layers, compared with the prior art, can focus on the user profile of users, accurately predict future revenues, and at the same time reduce the model training cost.

[0068] Embodiment 2

[0069] This embodiment provides a method for training a revenue prediction model based on statistical fused self-attention, including:

[0070] In order to be able to combine the time series of package prices and user profile information, etc., to predict the technical feasibility of future revenues, and at the same time discuss at the framework level that the new architecture can solve the problem of error propagation in multi-step prediction, a revenue prediction model based on statistical fused self-attention is constructed, and this model is implemented based on the Transformer network of ARIMA fused self-attention. Among them, the weights of ARIMA fused self-attention are trained by package revenue data, and the weights of other sub-attention layers are fine-tuned or refined after being encoded by the existing model combined with user profile data.

[0071] In addition, the ordinary weight part of the new framework supports fine-tuning or refinement, the computing power cost is flexible and easy to control, the training speed is faster than that of LSTM, and this framework can be trained based on all users or a specific user group to train a sub-group model, so as to support revenue prediction at both the package level dimension and the user level dimension at the same time.

[0072] Among them, the ARIMA integrated self-attention (ARIMA, Autoregressive Integrated Moving Average) is used. This algorithm is relatively common in traditional time series analysis. The model consists of two parts: autoregressive (AR, Auto-Regressive) and moving average model (MA, Moving Average). The basic idea of ARIMA is that the data values fluctuate around the general trend of time, where the trend is formed by being affected by historical labels (suitable for AR prediction); the fluctuations are formed by being affected by accidental events within a period of time, and the general trend itself is not necessarily stable (suitable for MA prediction). For example, the price of a package is relatively stable within a long observation time range (considering that the usage period of a package is generally several years and users will not consider changing the package frequently), and the occurrence of accidental events will cause the price of the package to fluctuate, such as holiday promotions, package expiration, user group events (such as college students graduating and changing cities). From the perspective of the package itself, it is suitable to use the ARIMA model for prediction. Therefore, this embodiment uses ARIMA integrated self-attention to predict the time series characteristics of income data. The ARIMA integrated self-attention is as follows:

[0073]

[0074] Among them, p, d, and q are the AR order, the difference order, and the MA order respectively, Xt is the value to be predicted, L is the lag operator, that is, L i = X t-i , φ i and θ i are all learning parameters of the autoregressive moving average algorithm, and ε t is the error term. In summary, the separate ARIMA integrated self-attention is only applicable to the time series prediction of the price itself and cannot combine user portrait data.

[0075] Transformer network, such as Figure 2As shown in the figure, it has been very popular in the field of natural language processing and is applied to sequence input-sequence output tasks such as language generation and dialogue. Since the core of this model is the self-attention mechanism, and self-attention is essentially the extraction of cross-correlation features of data at different positions in a sequence. The Transformer model contains a self-attention encoder and a self-attention decoder. The encoder is used for the above-mentioned relevant feature extraction, and these features are output to the decoder in the form of tensors. The decoder calculates an output sequence from the above tensor features and the specified input. Because the Transformer architecture can achieve sequence input-sequence output and consider the correlation of sequence data at the same time, its architecture is also applied to the field of time series analysis, inputting the observed values corresponding to historical time series and outputting the observed values corresponding to the predicted time series. For example, income data (income vector) is used as the input, and at the same time, user portrait data can be encoded as the input together with the income data, and through the fusion feature extraction of the encoder, the model can consider historical price features and user portrait data at the same time when predicting income.

[0076] The calculation method of self-attention in the Transformer network is as follows:

[0077] Among them, Q, K, and V are weight matrices, dk is the vector dimension. The essence of attention calculation is a scalar obtained by taking the inner product of the vectorized input, and this scalar represents the degree of association between any part of the sequence and other parts. To sum up, the self-attention mechanism can be used to analyze the correlation between user portrait data (vector) and package historical price data.

[0078] ARIMA integrates self-attention, as Figure 3 shown. Let the historical package price sequence be X0, X1, …, X t , and the user portrait data is vectorized and encoded as u1, u2, …, u m .

[0079] Definition 1: attension(ARIMA) = softmax(p×q×d), where

[0080] The PACF function and ACF function will not be elaborated here. d is used to judge whether the sequence {X t} is stationary by the ADF test or KPSS test, which will not be elaborated in this article either. The design method of softmax in the above calculation normalizes the attention, which is convenient for fusing with the attention carrying user portrait data.

[0081] Secondly, endow the model with the ability to refer to the user portrait, which is defined using Transformers. For the convenience of the following discussion, fused attention is proposed.

[0082] That is, Definition 2:

[0083] Taking the encoded user portrait vector u1 as an example, obtain the attention a 1,1 , a 1,2 , a 1,3 , a 1,4 , and finally obtain the hidden layer output output(u1), that is, the user portrait attention weight corresponding to this user portrait vector. Finally, a fused attention calculation method will be proposed so that the model can consider both historical package data and user portrait data simultaneously.

[0084] That is: Definition 3:

[0085] attionsion(normal) = attension(ARIMA) × attension(normal);

[0086] For example Figure 4 As shown, multiply the ARIMA attention by the normal attention, and finally obtain the output of the new fused attention layer. The new fused attention layer combines the ARIMA historical features and the features of the encoded portrait dimension (taking u1 as an example). The income prediction model framework is as Figure 5 shown.

[0087] During training, the encoder of the above framework simultaneously receives historical package data (time series panel data, including historical package data and historical user portrait data), uses the fused attention mechanism described above for feature extraction, and passes it to the decoder.

[0088] At the same time, use the above fused attention layer, take the data of the month before the month to be predicted of the package as the input, and output the probability distribution of the data of the month to be measured. In addition, during prediction (when using), the user portrait data of the month to be predicted can not be input, that is, package-level prediction can be performed, and the income prediction result of the next month of this package can be obtained; if the user portrait data of the previous month is input, it is user-level prediction.

[0089] Therefore, a method for training an income prediction model based on statistical fused self-attention proposed in this embodiment can simultaneously combine information such as the time series of package prices and user portraits to predict future income. While paying attention to the user portrait of the user and accurately predicting future income, the model training cost is reduced.

[0090] Embodiment 3

[0091] This embodiment provides an income prediction method based on statistical fused self-attention, including:

[0092] S10. Train the pre - constructed revenue prediction model based on the pre - constructed prediction training set to obtain a trained revenue prediction model;

[0093] The prediction training set includes at least one time - series panel data of different time periods and the corresponding package revenue prediction results for each time - series panel data;

[0094] The time - series panel data includes customer basic data, business basic package data, basic portrait tags, business change data, and revenue data per unit time.

[0095] S11. Obtain the data to be predicted in any unit time in real - time;

[0096] S12. Input the data to be predicted into the pre - set revenue prediction model to obtain the package revenue prediction result for assisting business decision - making in the next unit time;

[0097] The revenue prediction model is a prediction model constructed based on the Transformer network;

[0098] All attention layers of any encoder or decoder in the revenue prediction model are fused self - attention layers;

[0099] Any of the fused self - attention layers is used to: extract features from the received first input vector according to the pre - set autoregressive moving average algorithm and self - attention mechanism respectively, obtain the corresponding time - series prediction results and at least one user portrait attention weight respectively, and perform feature fusion between the time - series prediction results and each user portrait attention weight;

[0100] The first input vector is the data to be predicted or the data generated after being processed by any encoder / decoder.

[0101] Among them, when the first input vector includes a revenue vector and at least one user portrait vector, any of the fused self - attention layers extracts features from the received first input vector according to the pre - set autoregressive moving average algorithm and self - attention mechanism respectively, obtains the corresponding revenue time - series features and at least one user portrait attention weight respectively, and performs feature fusion between the revenue time - series features and each user portrait attention weight, including:

[0102] Perform time - series prediction on the received revenue vector according to the pre - set autoregressive moving average algorithm to obtain the time - series prediction result corresponding to the first input vector;

[0103] Extract features from each received user portrait vector according to a preset self-attention mechanism to obtain the user portrait attention weights corresponding to each user portrait vector;

[0104] Perform feature fusion on the time series prediction result and the user portrait attention weights corresponding to each user portrait vector respectively.

[0105] When the first input vector includes an income vector;

[0106] Then, for any one of the fusion self-attention layers, perform feature extraction on the received first input vector according to a preset autoregressive moving average algorithm and self-attention mechanism respectively to obtain the corresponding income time series features and at least one user portrait attention weight, and perform feature fusion on the income time series features and each user portrait attention weight respectively, including:

[0107] Perform time series prediction on the received income vector according to a preset autoregressive moving average algorithm to obtain the time series prediction result corresponding to the first input vector;

[0108] Perform feature fusion on the time series prediction result and at least one preset historical user portrait attention weight respectively.

[0109] Further, for any one of the fusion self-attention layers, perform trend feature extraction on the received first input vector according to a preset autoregressive moving average algorithm to obtain the corresponding to the first input vector, including:

[0110] Obtain the time series prediction weight corresponding to the first input vector according to the income vector and a preset formula one; the formula one is:

[0111] Attension(ARIMA) = softmax(p×q×d);

[0112] Where p is the AR order, d is the difference order, and q is the MA order. Attension(ARIMA) = softmax(p×q×d);

[0113] p is the AR order, d is a preset difference order, and q is the MA order;

[0114] Where Income vector, n and m are preset constants.

[0115] For any one of the fusion self-attention layers, perform feature extraction on each received user portrait vector according to a preset self-attention mechanism to obtain the user portrait attention weights corresponding to each user portrait vector, including:

[0116] Perform a linear transformation on each received user portrait vector to obtain a query vector, a key vector, and a value vector corresponding to each user portrait vector;

[0117] Based on the query vectors, key vectors, and value vectors of all user portrait vectors, obtain at least one similarity score corresponding to each user portrait vector; the similarity score is the similarity between any one user portrait vector and any other user portrait vector;

[0118] Based on the preset SoftMax function, perform normalization processing on each similarity score corresponding to each user portrait vector;

[0119] Perform a weighted sum of all the normalized similarity scores corresponding to each user portrait vector to obtain the user portrait attention weight corresponding to each user portrait vector.

[0120] Further, any decoder in the income prediction model further includes a regression layer before the output layer;

[0121] Any of the regression layers is used to: perform vector compression on the received vector as the output of the corresponding encoder;

[0122] The output of any encoder is income prediction data.

[0123] Further, any of the encoders includes a fused attention layer and a feed-forward neural network connected in series;

[0124] Any of the decoders includes a fused attention layer, a fused cross-attention layer, and a regression layer connected in sequence;

[0125] Among them, the feed-forward neural network of the encoder is connected to the fused cross-attention layer of the decoder.

[0126] A method for income prediction based on statistical fusion self-attention provided by this embodiment, due to the adoption of a fused self-attention layer, compared with the prior art, it can focus on the user portrait of the user, accurately predict future income, and at the same time reduce the model training cost.

[0127] Embodiment 4

[0128] This embodiment provides an income prediction system based on statistical fusion self-attention, including: a central prediction module, and a data acquisition module communicatively connected to the central prediction module;

[0129] The data acquisition module is used to obtain the data to be predicted in any unit time in real time;

[0130] The central prediction module is configured to input the data to be predicted into a pre-set revenue prediction model, and obtain a package revenue prediction result for assisting business decision-making in the next unit time;

[0131] The revenue prediction model is a prediction model constructed based on the transformer network;

[0132] All attention layers of any encoder or decoder in the revenue prediction model are fused self-attention layers;

[0133] Any of the fused self-attention layers is configured to: perform feature extraction on the received first input vector according to a pre-set autoregressive moving average algorithm and self-attention mechanism respectively, obtain corresponding time series prediction weights and at least one user profile attention weight respectively, and perform feature fusion on the time series prediction weights and each user profile attention weight respectively;

[0134] The first input vector is the data to be predicted, or the data generated after being processed by any encoder / decoder of the data to be predicted.

[0135] Further, the first input vector includes a revenue vector and at least one user profile vector;

[0136] Then, any of the fused self-attention layers performs feature extraction on the received first input vector according to a pre-set autoregressive moving average algorithm and self-attention mechanism respectively, obtains corresponding revenue time series features and at least one user profile attention weight respectively, and performs feature fusion on the revenue time series features and each user profile attention weight respectively, including:

[0137] Performing time series prediction on the received revenue vector according to a pre-set autoregressive moving average algorithm to obtain the time series prediction weights corresponding to the first input vector;

[0138] Performing feature extraction on each received user profile vector according to a pre-set self-attention mechanism to obtain the user profile attention weights corresponding to each user profile vector;

[0139] Performing feature fusion on the time series prediction weights and the user profile attention weights corresponding to each user profile vector respectively.

[0140] Further, any of the fused self-attention layers performs trend feature extraction on the received first input vector according to a pre-set autoregressive moving average algorithm, and obtaining the corresponding to the first input vector, including:

[0141] Obtaining the time series prediction weights corresponding to the first input vector according to the revenue vector and a pre-set formula one; the formula one is:

[0142] Attention(ARIMA) = softmax(p×q×d);

[0143] p is the AR order, d is the preset differencing order, and q is the MA order;

[0144] where k is the income vector, and n and m are preset constants.

[0145] Furthermore, for any one of the fusion self-attention layers, feature extraction is performed on each received user portrait vector according to a preset self-attention mechanism to obtain the user portrait attention weight corresponding to each user portrait vector, including:

[0146] Performing a linear transformation on each received user portrait vector to obtain a query vector, a key vector, and a value vector corresponding to each user portrait vector;

[0147] Based on the query vectors, key vectors, and value vectors of all user portrait vectors, at least one similarity score corresponding to each user portrait vector is obtained; the similarity score is the similarity between any one user portrait vector and any other user portrait vector;

[0148] Based on a preset SoftMax function, normalization processing is performed on each similarity score corresponding to each user portrait vector;

[0149] Performing a weighted sum of all the normalized similarity scores corresponding to each user portrait vector to obtain the user portrait attention weight corresponding to each user portrait vector.

[0150] Furthermore, the central prediction module is further configured to: train a pre-constructed income prediction model based on a pre-constructed prediction training set to obtain a trained income prediction model;

[0151] The prediction training set includes at least one time-series panel data of different time periods, and the package income prediction result corresponding to each time-series panel data;

[0152] The time-series panel data includes customer basic data, business basic package data, basic portrait labels, business change data, and unit time income data.

[0153] Furthermore, any one decoder in the income prediction model further includes a regression layer before the output layer;

[0154] Any one of the regression layers is configured to: perform vector compression on the received vector as the output of the corresponding encoder;

[0155] The output of any one encoder is the income prediction data.

[0156] Further, the first input vector includes an income vector;

[0157] Then, for any of the fusion self-attention layers, according to the preset autoregressive moving average algorithm and self-attention mechanism, feature extraction is respectively performed on the received first input vector to respectively obtain corresponding income time series features and at least one user portrait attention weight, and feature fusion is performed between the income time series features and each user portrait attention weight respectively, including:

[0158] Perform time series prediction on the received income vector according to the preset autoregressive moving average algorithm to obtain the time series prediction weight corresponding to the first input vector;

[0159] Perform feature fusion between the time series prediction weight and at least one preset historical user portrait attention weight respectively.

[0160] The income prediction system based on statistical fusion self-attention provided in this embodiment, due to the adoption of the fusion self-attention layer, compared with the prior art, can focus on the user portrait of the user, accurately predict future income, and at the same time reduce the model training cost.

[0161] The core of the embodiment of the present invention is the improvement of the attention mechanism in the income prediction model, and thus the above technical effects can be achieved.

[0162] Embodiment 5

[0163] This embodiment provides a computer-readable medium, on which instructions are stored, and when executed by one or more processors, cause the processors to execute an income prediction method according to any one of Embodiment 1 or Embodiment 3.

[0164] In the description of the present invention, it should be understood that the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, "a plurality" means two or more, unless otherwise specifically defined.

[0165] In the present invention, unless otherwise clearly specified and limited, the terms "installation", "connection", "connection", "fixation" and other terms should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or integrated; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium; it may be the internal communication of two components or the interaction relationship between two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0166] In the present invention, unless otherwise clearly specified and defined, a first feature being "on" or "under" a second feature may mean that the first and second features are in direct contact, or that the first and second features are indirectly in contact via an intermediate medium. Further, a first feature being "above", "over" and "on top of" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature has a higher level than the second feature in terms of horizontal height. A first feature being "under", "below" and "beneath" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature has a lower level than the second feature in terms of horizontal height.

[0167] In the description of this specification, the descriptions of terms such as "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples", etc. mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0168] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art may make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A revenue prediction method based on statistical fusion self-attention, characterized in that Including: S11. Obtain the data to be predicted within any unit time in real time; S12. Input the data to be predicted into a pre-set revenue prediction model to obtain a package revenue prediction result for assisting business decision-making in the next unit time; The revenue prediction model is a prediction model constructed based on the transformer network; All attention layers of any encoder or decoder in the revenue prediction model are fused self-attention layers; Any of the fused self-attention layers is used to: extract features from the received first input vector according to the pre-set autoregressive moving average algorithm and self-attention mechanism respectively, obtain the corresponding time series prediction weights and at least one user profile attention weight respectively, and perform feature fusion between the time series prediction weights and each user profile attention weight respectively; The first input vector is the data to be predicted, or the data generated after being processed by any encoder / decoder of the data to be predicted.

2. The revenue prediction method based on statistical fusion self-attention according to claim 1, wherein The first input vector includes a revenue vector and at least one user profile vector; Then, any of the fused self-attention layers extracts features from the received first input vector according to the pre-set autoregressive moving average algorithm and self-attention mechanism respectively, obtains the corresponding revenue time series features and at least one user profile attention weight respectively, and performs feature fusion between the revenue time series features and each user profile attention weight respectively, including: Performing time series prediction on the received revenue vector according to the pre-set autoregressive moving average algorithm to obtain the time series prediction weights corresponding to the first input vector; Extracting features from each received user profile vector according to the pre-set self-attention mechanism to obtain the user profile attention weights corresponding to each user profile vector; Performing feature fusion between the time series prediction weights and the user profile attention weights corresponding to each user profile vector respectively.

3. The income prediction method based on statistical fusion self-attention according to claim 2, characterized in that Any of the fused self-attention layers extracts trend features from the received first input vector according to the pre-set autoregressive moving average algorithm, and obtaining the corresponding to the first input vector, including: Obtaining the time series prediction weights corresponding to the first input vector according to the revenue vector and the pre-set formula one; the formula one is: Attension(ARIMA)=softmax(p×q×d); p is the AR order, d is the pre-set difference order, and q is the MA order; wherein k is an income vector, and n and m are preset constants.

4. The income prediction method based on statistical fusion self-attention according to claim 2, wherein Any of the fused self-attention layers extracts features from each received user profile vector according to the pre-set self-attention mechanism to obtain the user profile attention weights corresponding to each user profile vector, including: Performing linear transformation on each received user profile vector to obtain the query vector, key vector and value vector corresponding to each user profile vector; Obtaining at least one similarity score corresponding to each user profile vector according to the query vectors, key vectors and value vectors of all user profile vectors; the similarity score is the similarity between any user profile vector and any other user profile vector; Performing normalization processing on each similarity score corresponding to each user profile vector based on the pre-set SoftMax function; Perform a weighted sum of all the normalized similarity scores corresponding to each user portrait vector to obtain the user portrait attention weight corresponding to each user portrait vector.

5. The revenue prediction method based on statistical fusion self-attention according to claim 1, wherein Before S11, it further includes: S10. Train a pre-constructed revenue prediction model based on a pre-constructed prediction training set to obtain a trained revenue prediction model. The prediction training set includes at least one time-series panel data of different time periods, and the corresponding package revenue prediction result for each time-series panel data. The time-series panel data includes customer basic data, business basic package data, basic portrait tags, business change data, and unit time revenue data.

6. The income prediction method based on statistical fusion self-attention according to claim 5, characterized in that Any decoder in the revenue prediction model further includes a regression layer before the output layer. Any of the regression layers is used to: compress the received vector as the output of the corresponding encoder. The output of any encoder is revenue prediction data.

7. The income prediction method based on statistical fusion self-attention according to claim 1, characterized in that The first input vector includes a revenue vector. Then, any of the fusion self-attention layers extracts features from the received first input vector according to a preset autoregressive moving average algorithm and self-attention mechanism, respectively obtains the corresponding revenue time-series features and at least one user portrait attention weight, and fuses the revenue time-series features with each user portrait attention weight, including: Perform time-series prediction on the received revenue vector according to a preset autoregressive moving average algorithm to obtain the time-series prediction weight corresponding to the first input vector. Fuse the time-series prediction weight with at least one preset historical user portrait attention weight respectively.

8. The income prediction method based on statistical fusion self-attention according to claim 1, characterized in that, Any of the encoders includes a connected fusion attention layer and a feed-forward neural network. Any of the decoders includes a fusion attention layer, a fusion cross-attention layer, and a regression layer connected in sequence. Among them, the feed-forward neural network of the encoder is connected to the fusion cross-attention layer of the decoder.

9. An income prediction system based on statistical fusion self-attention, characterized in that, It includes: A central prediction module, and a data acquisition module communicatively connected to the central prediction module. The data acquisition module is used to obtain the data to be predicted in any unit time in real time. The central prediction module is used to input the data to be predicted into a preset revenue prediction model to obtain a package revenue prediction result for assisting business decision-making in the next unit time. Among them, the revenue prediction model is the revenue prediction model described in any one of claims 1 to 8.

10. A computer-readable medium having instructions stored thereon that, when executed by one or more processors, cause the processors to execute the revenue prediction method based on statistical fusion self-attention described in any one of claims 1 to 8.

Images