Prediction method and device based on LSTM and Bayesian uncertainty and medium

Abstract

The invention discloses a prediction method and device based on LSTM and Bayesian uncertainty and a medium, and the method comprises the steps: obtaining original data, and generating an initial feature set through the original data; dividing the initial feature set into a training set and a prediction set; the training set and the prediction set are introduced into an optimization function, so that optimal parameters of an XGBoost model are obtained through Bayesian optimization; using the optimal parameter, the training set and the prediction set to train an XGBoost model, and extracting intermediate features; fusing the training set and the intermediate features to obtain a final feature set; training the initial prediction model by using the final feature set to obtain a prediction model; and performing prediction by using the prediction model to obtain a prediction result. Bayesian uncertainty is used, the uncertainty contained in prediction is considered in a prediction result, and then a more accurate prediction result containing most risks is obtained through searching for optimal parameters and a large amount of calculation reasoning.

Description

Technical Field

[0001] The present application relates to the field of data prediction, and specifically to a prediction method, device and medium based on LSTM and Bayesian uncertainty. Background Art

[0002] When dealing with prediction problems, in pursuit of certainty, it is usually necessary to use some historical data, and then use the principles of statistics to predict some data, so as to obtain the probability of some events developing in a certain direction, so as to increase the certainty in statistics.

[0003] In the prior art, linear regression models are usually used for prediction. However, although linear regression models have a place in many prediction application scenarios due to their theoretical simplicity and computational efficiency, their inherent limitations cannot be ignored. The model presupposes that there is a linear mapping relationship between the independent variable and the dependent variable, and this assumption is often difficult to hold in complex and changeable real-world systems. Specifically, linear regression is difficult to capture nonlinear dynamics that may exist between data, such as exponential surges and logarithmic decay. At the same time, it is not ideal for processing the long-term memory effect in time series data (the characteristic that the current state is affected by the states of multiple historical time points). Summary of the invention

[0004] In order to solve the above problems, the present application proposes a prediction method, device and medium based on LSTM and Bayesian uncertainty, wherein the method includes:

[0005] Obtain original data and generate an initial feature set through the original data; divide the initial feature set into a training set and a prediction set; pass the training set and the prediction set into the optimization function to obtain the optimal parameters of the XGBoost model through Bayesian optimization; use the optimal parameters, the training set and the prediction set to train the XGBoost model and extract intermediate features; fuse the training set and the intermediate features to obtain a final feature set; use the final feature set to train the initial prediction model to obtain a prediction model; the initial prediction model is an LSTM model that introduces a Bayesian uncertainty layer, a drop layer and a GRU attention layer; use the prediction model to perform predictions to obtain prediction results.

[0006] In one example, the initial prediction model includes: a time series input processing layer, two Bayesian convolution layers, a bidirectional LSTM layer, a bidirectional GRU layer, an attention mechanism, a GRU output layer, a Bayesian fully connected layer and an output layer; the input of the time series input processing layer is time series data; the Bayesian convolution layer has 128 convolution kernels, and the size of each convolution kernel is 5, and the activation function is a linear rectification function; the Bayesian fully connected layer uses Bayesian inference and uses probability distribution weights; the Bayesian fully connected layer contains 75 hidden units, and the activation function is a linear rectification function.

[0007] In one example, generating an initial feature set using the original data specifically includes: crawling sentiment articles related to the original data; analyzing the sentiment articles using a sentiment model to obtain sentiment features of the articles; and merging the sentiment features of the articles with the original data feature set corresponding to the original data to obtain an initial feature set.

[0008] In one example, the use of the optimal parameters, the training set, and the prediction set to train the XGBoost model specifically includes: dividing the initial feature set into K data subsets; selecting any subset from the K data subsets as a target prediction subset, and using the remaining data subsets as training subsets; using the target prediction subset and the training subset to repeatedly train the XGBoost model; traversing all subsets in the K data subsets and repeating the training process until each data subset has been used as the target prediction subset; during the training process, using the early stopping method to prevent the XGBoost model from overfitting.

[0009] In one example, the optimal parameter categories include: maximum depth of the tree, minimum loss reduction of splitting nodes, data sampling ratio, and feature sampling ratio.

[0010] In one example, the training set and the prediction set are passed into the optimization function to obtain the optimal parameters of the XGBoost model through Bayesian optimization, specifically including: defining an internal function of the optimization function, wherein the internal function is used to define the optimal parameter category and evaluate the performance of the XGBoost model under specific hyperparameter settings; setting a fixed learning rate, and converting the training set and the prediction set into a format using an XGBoost-specific data structure DMatrix; training and testing the converted training set and prediction set using a cross-validation method, and returning the RMSE mean on the test set; and configuring the Bayesian optimizer to obtain the optimal parameters of the XGBoost model based on the RMSE mean.

[0011] In one example, the configuration of the Bayesian optimizer to obtain the optimal parameters of the XGBoost model based on the RMSE mean specifically includes: obtaining the optimal parameters output by the Bayesian optimizer; determining whether the maximum depth value of the tree in the optimal parameters is an integer, if the maximum depth value of the tree in the optimal parameters is not an integer, reselecting the optimal parameters through the Bayesian optimizer.

[0012] In one example, the prediction is performed using the prediction model to obtain a prediction result, specifically including: inputting prediction data into the prediction model to obtain a prediction value and a statistical probability; denormalizing the prediction value to obtain a prediction result and a corresponding probability of occurrence; and sending the prediction result and the probability of occurrence to a user.

[0013] The present application also provides a prediction device based on LSTM and Bayesian uncertainty, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute: obtaining original data and generating an initial feature set through the original data; dividing the initial feature set into a training set and a prediction set; passing the training set and the prediction set into an optimization function to obtain the optimal parameters of the XGBoost model through Bayesian optimization; using the optimal parameters, the training set and the prediction set to train the XGBoost model and extract intermediate features; fusing the training set and the intermediate features to obtain a final feature set; using the final feature set to train an initial prediction model to obtain a prediction model; the initial prediction model is an LSTM model that introduces a Bayesian uncertainty layer, a drop layer and a GRU attention layer; using the prediction model to perform predictions to obtain prediction results.

[0014] The present application also provides a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured to: obtain raw data, and generate an initial feature set through the raw data; divide the initial feature set into a training set and a prediction set; pass the training set and the prediction set into an optimization function to obtain the optimal parameters of the XGBoost model through Bayesian optimization; train the XGBoost model using the optimal parameters, the training set, and the prediction set, and extract intermediate features; fuse the training set and the intermediate features to obtain a final feature set; use the final feature set to train an initial prediction model to obtain a prediction model; the initial prediction model is an LSTM model that introduces a Bayesian uncertainty layer, a drop layer, and a GRU attention layer; use the prediction model to perform predictions to obtain prediction results.

[0015] The method proposed in this application can bring the following beneficial effects: finding the best parameters through XGBoost and extracting more effective intermediate features. Bayesian neural network can provide uncertainty estimation to help risk management and decision-making. Adding drop layer and Bayesian uncertainty layer, using early stopping method and K-fold method to train the model, it is more robust to outliers and data noise, avoiding the risk of overfitting, and strengthening data features through article sentiment analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 A flowchart of a prediction method based on LSTM and Bayesian uncertainty in an embodiment of the present application;

[0018] Figure 2 This is a schematic diagram of the structure of a prediction device based on LSTM and Bayesian uncertainty in an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0020] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0021] Figure 1 A flowchart of a prediction method based on LSTM and Bayesian uncertainty provided for one or more embodiments of this specification. The method can be applied to different business fields, such as Internet financial business field, e-commerce business field, instant messaging business field, game business field, official business field, etc. The process can be executed by a computing device in the corresponding field (such as a risk control server or smart mobile terminal corresponding to the payment business), and some input parameters or intermediate results in the process allow manual intervention and adjustment to help improve accuracy.

[0022] The analysis method involved in the embodiments of the present application can be implemented by a terminal device or a server, and the present application does not impose any special restrictions on this. For the convenience of understanding and description, the following embodiments are described in detail by taking a server as an example.

[0023] It should be noted that the server can be a single device or a system composed of multiple devices, that is, a distributed server, and this application does not make any specific limitations on this.

[0024] like Figure 1 As shown, the embodiment of the present application provides a prediction method based on LSTM and Bayesian uncertainty, including:

[0025] S101: Acquire original data, and generate an initial feature set using the original data.

[0026] First, obtain the original data, that is, the data feature set. If there is enough data, you can directly divide the data set. If there is a need for sentiment analysis of the article, you can use the sentiment model to perform sentiment feature analysis. If there is no need for sentiment analysis, you can directly ignore it. The data value of the original data can be directly regarded as the original feature.

[0027] In one embodiment, when performing sentiment feature analysis, it is necessary to crawl sentiment articles related to the original data; and analyze the sentiment articles through the sentiment model to obtain the sentiment features of the article and other available features as data, and merge the sentiment features of the article with the original data feature set corresponding to the original data to obtain the initial feature set. After analyzing the sentiment, a sentiment score will be generated, such as 90 points for positive and 20 points for negative, and then this score will be merged with the original data. The quality of sentiment analysis depends on the quality of the trained sentiment model. In terms of training, it is trained with a large amount of data. If the sentiment analysis is wrong, the training set data and verification machine data can be manually modified and optimized. Other features here depend on the specific content that the user wants to predict, and only a prediction framework is provided here. For example, if the user wants to predict the weather, the feature data can be the current temperature, the region, and whether it is cloudy.

[0028] S102: Divide the initial feature set into a training set and a prediction set.

[0029] After obtaining the initial feature set, use the create_dataset function to divide the data set into a training set and a prediction set. That is, when the model is trained with historical data, a part of the data is used for training, and a part of the data is used for prediction to determine the accuracy of the current prediction, and then the feature weights are repeatedly adjusted based on the prediction results. After the division is completed, the divided training set and prediction set need to be normalized, that is, large data is reduced, which helps to improve training efficiency.

[0030] S103: passing the training set and the prediction set into an optimization function to obtain the optimal parameters of the XGBoost model through Bayesian optimization.

[0031] Among them, the optimization function is an innovation function, which has a built-in Bayesian optimization process. This function is used to find super parameters, such as predicting whether it will rain tomorrow. Based on temperature, humidity, geographic location and other characteristics, the probability of temperature and humidity affecting rainfall may be relatively large in some places, and the correlation between wind and rainfall may be relatively large in some places. The optimization function finds some parameters with relatively large correlation through continuous attempts, and will use these parameters for subsequent training.

[0032] Specifically, the optimization function is called optimize_xgboost, and the function includes an internal function: define the objective function xgb_evaluate. This is an internal function specifically used to evaluate the performance of the XGBoost model under specific hyperparameter settings. It receives four hyperparameters: the maximum depth of the tree, the minimum loss reduction of the split node, the data sampling ratio, and the feature sampling ratio. These parameters are used to build the XGBoost model, and a fixed learning rate (eta=0.001) is set. The dataset is converted to the format of the XGBoost-specific data structure DMatrix, and then trained and tested using the 3-fold cross-validation method. The model will be trained for 100 rounds, but if the RMSE does not improve significantly in 10 consecutive rounds, the training will stop. Finally, the function returns the negative RMSE mean on the test set, so that when optimizing, it can be maximized by maximizing this value, which is equivalent to minimizing the RMSE.

[0033] The optimization function uses the Bayesian Optimization tool to perform hyperparameter search. The goal of optimization is to make xgb_evaluate (i.e., RMSE mean) as small as possible. The search range of the four hyperparameters to be optimized is defined: Maximum tree depth: ranges from 3 to 10, controlling the depth of the tree. Minimum loss reduction for splitting nodes: ranges from 0 to 5, determining the degree of conservatism of node splitting. Data sampling ratio: ranges from 0.5 to 1, controlling the proportion of data used by each tree, and lower values ​​can prevent overfitting. Feature sampling ratio: ranges from 0.3 to 1, controlling the proportion of features used by each tree.

[0034] When performing hyperparameter optimization, we first perform 10 random sampling initializations, which is equivalent to randomly selecting some parameter combinations for evaluation. Then, we perform 50 iterations of optimization, and Bayesian optimization intelligently selects the next most likely parameter combination based on the inferences of the previous evaluation results to improve optimization efficiency. After the optimization is completed, the function extracts the best performing parameter combination from the optimizer. At the same time, ensure that the maximum depth parameter of the tree is an integer. The function finally returns the best parameters found, which can be used for subsequent model training or evaluation.

[0035] S104: Train the XGBoost model using the optimal parameters, the training set, and the prediction set, and extract intermediate features.

[0036] After obtaining the optimal parameters, the XGBoost model can be trained using the training set and the prediction set, and intermediate features can be extracted. This step obtains intermediate features that have a greater impact on the results and enhances predictable regularity.

[0037] In one embodiment, when training the XGBoost model, the initial feature set needs to be divided into K data subsets, and any subset is selected from the K data subsets as the target prediction subset, and the remaining data subsets are used as training subsets. Then, the target prediction subset and the training subset are used to repeatedly train the XGBoost model, traversing all subsets in the K data subsets, and repeating the training process until each data subset has been used as the target prediction subset. At the same time, during the training process, the early stopping method is used to prevent the XGBoost model from overfitting.

[0038] S105: Fusing the training set and the intermediate features to obtain a final feature set.

[0039] S106: Using the final feature set to train an initial prediction model to obtain a prediction model; the initial prediction model is an LSTM model that introduces a Bayesian uncertainty layer, a drop layer, and a GRU attention layer.

[0040] After obtaining the final feature set, the initial prediction model is trained using the final feature set to obtain a prediction model. The initial prediction model is an innovative structure, which is an LSTM model that introduces a Bayesian uncertainty layer, a drop layer, and a GRU attention layer.

[0041] In one embodiment, the initial prediction model includes: a time series input processing layer, two Bayesian convolutional layers, a bidirectional LSTM layer, a bidirectional GRU layer, an attention mechanism, a GRU output layer, a Bayesian fully connected layer, and an output layer, as described below:

[0042] The relevant codes of the timing input processing layer are as follows:

[0043] combined_input=Input(shape=(time_step, combined_feature_dim)).

[0044] The input is a 3D tensor with shape (time_step, combined_feature_dim), which means that the input is time series data with combined_feature_dim features at each time step.

[0045] Bayesian convolution layer: The relevant code is as follows:

[0046] x=tfp.layers.Convolution1DFlipout(filters=128,kernel_size=5,padding='same',activation='relu')(combined_input),

[0047] x=tfp.layers.Convolution1DFlipout(filters=128, kernel_size=5, padding='same', activation='relu')(x).

[0048] Among them, tfp.layers.Convolution1DFlipout is the Bayesian convolution layer provided by TensorFlow Probability. The convolution layer here uses the "Flipout" variant of Bayesian inference, allowing the distribution uncertainty of the weights to be propagated through sampling, enhancing the robustness of the model in the face of noisy data. This layer has 128 convolution kernels, each with a size of 5, padding = 'same' means that the output size is the same as the input, and the activation function is ReLU. Two convolution layers are used to extract local patterns in time series, and the Bayesian convolution layer can also model prediction uncertainty.

[0049] The relevant codes for the bidirectional LSTM and GRU layers are as follows:

[0050] x=Bidirectional(LSTM(150,return_sequences=True))(x);

[0051] x=Bidirectional(GRU(150, return_sequences=True))(x).

[0052] Bidirectional LSTM and bidirectional GRU are classic architectures for processing time series data. The bidirectional structure allows the model to extract features not only from past information but also from future information. Long short-term memory networks and gated recurrent units are two common variants of recurrent neural networks. Long short-term memory networks are good at capturing long-term dependencies, while gated recurrent units are relatively more computationally efficient. Here, stacking long short-term memory networks and gated recurrent unit layers can further enhance the model's ability to capture time series dependencies.

[0053] The relevant code of the attention mechanism is as follows:

[0054] attention=Attention()([x,x]);

[0055] x=Add()([x,attention]).

[0056] The attention mechanism is designed to make the model pay more attention to the key information of certain time steps. In the sequence model, some time steps may be more important than other time steps, and the attention mechanism can enhance the model's attention to these key time steps. Here, the attention mechanism calculates the correlation of the sequence itself through self-attention, and then adds it to the original input, which not only retains the original information, but also introduces new global dependency information, enhancing the model's feature expression ability.

[0057] The relevant code of the GRU output layer is as follows:

[0058] x=GRU(150)(x)

[0059] Finally, a GRU layer that does not return a sequence is used for aggregation to map the time series features to a vector of a fixed size. This step further compresses the high-dimensional features processed by the convolution, loop and attention mechanisms, making them suitable for the subsequent prediction task.

[0060] The relevant code of the Bayesian fully connected layer is as follows:

[0061] x=tfp.layers.DenseFlipout(75,activation='relu')(x)

[0062] Among them, DenseFlipout is a Bayesian fully connected layer. Unlike ordinary fully connected layers, it uses Bayesian inference, and the weights are probability distributions rather than definite values. This makes the model more advantageous when dealing with uncertainty, and is particularly suitable for tasks with high noise or high uncertainty. This layer has 75 hidden units and the activation function is ReLU.

[0063] The output layer related codes are as follows:

[0064] mean_output=tfp.layers.DenseFlipout(1,activation='linear',name='mean_output')(x)

[0065] This output is used for regression tasks to predict continuous values, and the activation function is a linear function. The predicted mean is modeled using a Bayesian DenseFlipout layer.

[0066] up_prob_output = Dense(1, activation = 'sigmoid', name = 'up_prob_output')(x). This output is a classification task, used to predict the probability of an event (such as whether a state in a time series rises). The activation function is Sigmoid, and the output value is between 0 and 1.

[0067] up_rate_output=tfp.layers.DenseFlipout(1,activation='linear',name='up_rate_output')(x)

[0068] This output is also a regression task, predicting the specific magnitude of the increase (such as the change in price or indicator), using a linear activation function to model the magnitude, and using a Bayesian layer to handle uncertainty.

[0069] S107: Use the prediction model to perform prediction to obtain a prediction result.

[0070] Because the data will be normalized before training, that is, large data will be reduced, which helps improve training efficiency, so after the prediction is completed, the reduced data needs to be enlarged, that is, denormalized. Specifically, after the prediction data is input into the prediction model, the prediction value and statistical probability are obtained, and then the prediction value is denormalized to obtain the prediction result and the corresponding probability of occurrence; finally, the prediction result and probability of occurrence are sent to the user.

[0071] like Figure 2 As shown, the embodiment of the present application also provides a prediction device based on LSTM and Bayesian uncertainty, including: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to:

[0072] Obtain original data and generate an initial feature set through the original data; divide the initial feature set into a training set and a prediction set; pass the training set and the prediction set into the optimization function to obtain the optimal parameters of the XGBoost model through Bayesian optimization; use the optimal parameters, the training set and the prediction set to train the XGBoost model and extract intermediate features; fuse the training set and the intermediate features to obtain a final feature set; use the final feature set to train the initial prediction model to obtain a prediction model; the initial prediction model is an LSTM model that introduces a Bayesian uncertainty layer, a drop layer and a GRU attention layer; use the prediction model to perform predictions to obtain prediction results.

[0073] The embodiment of the present application further provides a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured as follows:

[0074] Obtain original data and generate an initial feature set through the original data; divide the initial feature set into a training set and a prediction set; pass the training set and the prediction set into the optimization function to obtain the optimal parameters of the XGBoost model through Bayesian optimization; use the optimal parameters, the training set and the prediction set to train the XGBoost model and extract intermediate features; fuse the training set and the intermediate features to obtain a final feature set; use the final feature set to train the initial prediction model to obtain a prediction model; the initial prediction model is an LSTM model that introduces a Bayesian uncertainty layer, a drop layer and a GRU attention layer; use the prediction model to perform predictions to obtain prediction results.

[0075] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0076] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0077] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0078] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0079] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0080] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0081] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0082] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0083] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include transitory media such as modulated data signals and carrier waves.

[0084] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0085] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A prediction method based on LSTM and Bayesian uncertainty, characterized in that: include: Acquire raw data, and generate an initial feature set using the raw data; Dividing the initial feature set into a training set and a prediction set; Passing the training set and the prediction set into the optimization function to obtain the optimal parameters of the XGBoost model through Bayesian optimization; Using the optimal parameters, the training set, and the prediction set to train the XGBoost model, and extract intermediate features; The training set and the intermediate features are merged to obtain a final feature set; Using the final feature set to train an initial prediction model to obtain a prediction model; the initial prediction model is an LSTM model that introduces a Bayesian uncertainty layer, a drop layer, and a GRU attention layer; The prediction model is used to perform prediction to obtain a prediction result.

2. The method according to claim 1, characterized in that The initial prediction model comprises: Time series input processing layer, two Bayesian convolutional layers, bidirectional LSTM layer, bidirectional GRU layer, attention mechanism, GRU output layer, Bayesian fully connected layer, and output layer; The input of the time series input processing layer is time series data; The Bayesian convolution layer has 128 convolution kernels, and the size of each convolution kernel is 5, and the activation function is a linear rectification function; The Bayesian fully connected layer uses Bayesian inference and probability distribution weights; the Bayesian fully connected layer includes 75 hidden units, and the activation function is a linear rectification function.

3. The method according to claim 1, characterized in that Generating an initial feature set through the original data specifically includes: Crawl sentiment articles related to the raw data; Analyze the sentiment article through a sentiment model to obtain sentiment features of the article; The sentiment features of the article are combined with the original data feature set corresponding to the original data to obtain an initial feature set.

4. The method according to claim 1, characterized in that: The training of the XGBoost model using the optimal parameters, the training set, and the prediction set specifically includes: Dividing the initial feature set into K data subsets; Select any one of the K data subsets as a target prediction subset, and use the remaining data subsets as training subsets; Using the target prediction subset and the training subset, repeatedly training the XGBoost model; Traversing all the K data subsets, and repeating the training process until each data subset has been used as the target prediction subset; During training, early stopping was used to prevent the XGBoost model from overfitting.

5. The method according to claim 1, characterized in that The optimal parameter categories include: maximum depth of the tree, minimum loss reduction of splitting nodes, data sampling ratio, and feature sampling ratio.

6. The method according to claim 5, characterized in that The training set and the prediction set are passed into the optimization function to obtain the optimal parameters of the XGBoost model through Bayesian optimization, specifically including: Defining an internal function of the optimization function, wherein the internal function is used to define the optimal parameter category and evaluate the performance of the XGBoost model under specific hyperparameter settings; Set a fixed learning rate, and convert the training set and the prediction set into a format using a DMatrix data structure specific to XGBoost; The converted training set and prediction set are trained and tested using the cross-validation method, and the RMSE mean on the test set is returned; Configure the Bayesian optimizer to obtain the optimal parameters for the XGBoost model based on the mean RMSE.

7. The method according to claim 6, characterized in that The configuration of the Bayesian optimizer to obtain the optimal parameters of the XGBoost model according to the RMSE mean specifically includes: Obtaining the optimal parameters output by the Bayesian optimizer; It is determined whether the maximum depth value of the tree in the optimal parameter is an integer. If the maximum depth value of the tree in the optimal parameter is not an integer, the optimal parameter is reselected by the Bayesian optimizer.

8. The method according to claim 1, characterized in that The using the prediction model to perform prediction to obtain a prediction result specifically includes: Inputting the prediction data into the prediction model to obtain the prediction value and statistical probability; Denormalizing the predicted value to obtain a predicted result and a corresponding probability of occurrence; The prediction result and the probability of occurrence are sent to the user.

9. A prediction device based on LSTM and Bayesian uncertainty, characterized in that: include: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to execute: Acquire raw data, and generate an initial feature set using the raw data; Dividing the initial feature set into a training set and a prediction set; Passing the training set and the prediction set into the optimization function to obtain the optimal parameters of the XGBoost model through Bayesian optimization; Using the optimal parameters, the training set, and the prediction set to train the XGBoost model, and extract intermediate features; The training set and the intermediate features are merged to obtain a final feature set; Using the final feature set to train an initial prediction model to obtain a prediction model; the initial prediction model is an LSTM model that introduces a Bayesian uncertainty layer, a drop layer, and a GRU attention layer; The prediction model is used to perform prediction to obtain a prediction result.

10. A non-volatile computer storage medium storing computer executable instructions, characterized in that: The computer executable instructions are configured to: Acquire raw data, and generate an initial feature set using the raw data; Dividing the initial feature set into a training set and a prediction set; Passing the training set and the prediction set into the optimization function to obtain the optimal parameters of the XGBoost model through Bayesian optimization; Using the optimal parameters, the training set, and the prediction set to train the XGBoost model, and extract intermediate features; The training set and the intermediate features are merged to obtain a final feature set; Using the final feature set to train an initial prediction model to obtain a prediction model; the initial prediction model is an LSTM model that introduces a Bayesian uncertainty layer, a drop layer, and a GRU attention layer; The prediction model is used to perform prediction to obtain a prediction result.

Images