Variable weight combination prediction method for battery manufacturing capacity based on triple reinforcement learning

Abstract

The present invention discloses a variable weight combination prediction method for battery manufacturing capacity based on triple reinforcement learning, which belongs to the field of power battery manufacturing prediction. This method targets the periodicity, mutation and seasonality of lithium battery manufacturing capacity, selects corresponding models to respectively mine the linear and nonlinear features in the time series to form a new combination prediction model; in view of the problem that the existing methods are difficult to obtain the optimal combination prediction weight, a double-layer reinforcement learning algorithm is designed to solve the optimal weight matrix under this time to reduce the prediction error; in addition, since the prediction effect of each single prediction model is different at different time points, this application integrates reinforcement learning and entropy ideas, explores the optimal sliding window length, uses sliding windows to divide the prediction sequence, and uses a double-layer reinforcement learning algorithm in each window to determine the optimal combination weight, and finally constructs a triple reinforcement learning battery manufacturing capacity combination prediction model, which further improves the prediction accuracy and reliability of battery manufacturing capacity.

Description

Technical Field

[0001] The present invention relates to a variable weight combination prediction method for battery manufacturing capacity based on triple reinforcement learning, and belongs to the field of power battery manufacturing prediction. Background Art

[0002] In recent years, research on high-quality manufacturing of lithium-ion batteries, core components of new energy vehicles, has garnered increasing attention. Current research focuses primarily on analyzing battery quality, such as power monitoring and battery health status estimation. However, accurately understanding future battery manufacturing capacity is crucial for battery manufacturers to rationally plan production and formulate market strategies. However, research findings in this area are relatively scarce.

[0003] Battery manufacturing capacity can be described as the number of qualified batteries produced per unit time. Since battery manufacturing capacity data is arranged in chronological order, and is obtained at a fixed sampling frequency over equally spaced time periods, it exhibits a certain regularity. Therefore, predicting lithium battery manufacturing capacity is essentially a time series prediction problem.

[0004] In actual engineering, lithium battery manufacturing capacity is often influenced by production factors such as machine performance and staff scheduling per unit time, as well as market factors such as battery order volume and delivery time. Therefore, existing single forecasting methods cannot fully capture the time series characteristics of lithium battery manufacturing capacity. Therefore, combined model forecasting methods have emerged, but these methods involve the issue of weighting each model. Obtaining the optimal weight coefficient directly affects the accuracy of the forecast results, so it is necessary to consider how to solve the problem of optimal weight coefficient.

[0005] In the existing research results on time-varying weight selection strategies, setting a sliding window is a common way to achieve time-varying weights. Choosing an appropriate window length has a great impact on prediction accuracy. In previous studies, the selection of window length is often based on observation or empirical methods, and it is usually difficult to obtain the optimal sliding window length. Relatively speaking, the entropy method can solve the problem of the optimal sliding window length, but it requires high computing power and is not suitable for practical applications.

[0006] In addition, traditional methods for determining the weights of combined models include the hierarchical analysis method and the adaptive weight method. However, these methods often find it difficult to obtain the optimal weight coefficient, resulting in suboptimal prediction accuracy. Summary of the Invention

[0007] In order to achieve accurate prediction of battery manufacturing capacity, help enterprises rationally arrange production plans and formulate market strategic plans, and improve economic benefits. The present invention provides a battery manufacturing capacity variable weight combination prediction method based on triple reinforcement learning. When predicting battery manufacturing capacity, the method first defines a new combination prediction form, then introduces a sliding window and designs the first reinforcement learning: reinforcement learning-entropy algorithm determines the optimal window length. Finally, in order to fully integrate the performance of a single algorithm, a double-layer reinforcement learning algorithm is designed under each window to explore the optimal weight of a single prediction result, and the obtained weights are combined and added to obtain the optimal prediction value at that time point. The sliding window is continued until the optimal weighting is achieved for the entire prediction sequence, thereby realizing the variable weight combination prediction of battery manufacturing capacity.

[0008] A battery manufacturing capacity variable weight combination prediction method based on triple reinforcement learning, the method comprising:

[0009] Step S1: construct a combined prediction model based on LSTM, GRU and SARIMA single prediction models, and use LSTM, GRU and SARIMA single prediction models to obtain the prediction results at time t according to the historical production data before time t.

[0010] Step S2: Based on the prediction results of each single prediction model in step S1, a reinforcement learning-entropy algorithm is designed to determine the sliding window length and obtain the optimal sliding window length l;

[0011] Step S3, based on the optimal window length l obtained in step S2, the single prediction sequence and the true value sequence Y t Perform window division, with k representing the time point after division, and the window with time k as the first data is called the kth window; the prediction result and the true value at time k are respectively: Y k ;

[0012] Step S4, based on the window data divided in step S3, design the second layer of reinforcement learning to optimize the combined weight of the LSTM prediction model and the GRU prediction model to obtain the optimal combined weight [ω l ,ω g ], and the optimal combination weight [ω l ,ω g ] is assigned to the first time point k in the window, and the LSTM and GRU combined prediction value at time point k is obtained by adding the weight combination

[0013] Step S5: The optimal combination of LSTM and GRU obtained in step S4 and SARIMA forecast results Together, we conduct the third layer reinforcement learning combination weight optimization to obtain the optimal combination weight [ω nn ,ω s ] and calculate the battery manufacturing capacity forecast;

[0014] Step S6: The sliding window continuously updates the data backward, and steps S4 and S5 are repeated to calculate the optimal combination weight of the current window until the last window is reached to obtain the optimal combination prediction sequence.

[0015] Optionally, step S2 includes:

[0016] Step S2.1, determine the state matrix S, and the state matrix selects the window length l;

[0017] S=[l] (5)

[0018] Step S2.2: Set the action matrix A to the state of the previous moment and perform fixed-amplitude addition and subtraction:

[0019] A=[Δω1,-Δω1] (6)

[0020] Δω1 represents the amplitude of addition or subtraction;

[0021] Step S2.3, the true value Y t and predicted values The absolute prediction error of manufacturing capacity of the i-th single prediction algorithm at time t is calculated as:

[0022]

[0023] Where t=1,2,...,N,i=1,2,3;

[0024] Step S2.4: Calculate the mean absolute error of manufacturing capability of K single-item prediction algorithms at time t using formula (3):

[0025]

[0026] Step S2.5, calculate the average absolute error of the manufacturing capacity at the moment of the average absolute error of the lithium battery manufacturing capacity prediction data in the kth window according to formula (4):

[0027]

[0028] And normalize the mean absolute error:

[0029]

[0030] Step S2.6, calculate the entropy value of the manufacturing capacity prediction data under this window according to formula (6):

[0031]

[0032] Step S2.7, introduce equation (7) to set the loss function L and the reward and punishment function R. The loss function L is set to:

[0033]

[0034] When L l+1 <L l When , the reward and punishment function R is set to:

[0035] R=1+L t -L t+1 (13)

[0036] When L l+1 >L l When , the reward and punishment function R is set to:

[0037] R=-1+L t -L t+1 (14)

[0038] Step S2.8, construct a reinforcement learning process to obtain the optimal sliding window length.

[0039] Optionally, step S4 includes:

[0040] Step S4.1, for the problem of optimizing the weight of LSTM and GRU combination, set the target state matrix S = [ω l ,ω g ], where ω l +ω g =1;

[0041] Step S4.2, set the action matrix A = [Δω2, -Δω2], select the action, and l Increase and decrease, and get ω g , forming the next state, Δω2 represents the amplitude of addition and subtraction;

[0042] In step S4.3, the minimum root mean square error of the data in the sliding window is taken as the exploration goal, and the loss function L for weighted optimization in the k-th window is set:

[0043]

[0044] According to the loss function L, the reward and punishment function is set to R = 1 / L;

[0045] Step S4.4: Construct the second-layer reinforcement learning optimization process according to steps S4.1 to S4.3 until the optimal number of iterations is reached. At this time, explore the optimal combination weight of LSTM and GRU under the current window, assign the weight to the first time point in the window, and obtain the LSTM and GRU combination prediction value at the time point k according to the weight combination.

[0046]

[0047] According to formula (16), the LSTM and GRU combined prediction value at time point k is obtained: When ω l and ω g is the optimal combined weight value of LSTM and GRU in the current window.

[0048] Optionally, step S5 includes:

[0049] Step S5.1, for the optimal combination results based on LSTM and GRU and SARIMA forecast results Together we conduct the combined weight optimization problem and set the target state matrix S = [ω nn ,ω s ], where ω nn +ω s =1;

[0050] Step S5.2, set the action matrix A = [Δω2, -Δω2], select the action, and nn Increase and decrease, and get ω s , forming the next state;

[0051] Step S5.3, set the loss function L:

[0052]

[0053] According to the loss function L, the reward and punishment function is set to R = 1 / L;

[0054] Step S5.4: Construct the third layer of reinforcement learning optimization process according to steps S5.1 to S5.3 until the optimal number of iterations is reached; at this time, explore the optimal combination weight of the three single prediction models under the current window, assign the weight to the first time point in the window, and obtain the optimal combination prediction value at the time point k according to the weight combination.

[0055]

[0056] According to formula (18), the optimal combination prediction value at time point k is obtained: When ω nn and ω s is the optimal combined weight value of the three single prediction models in the current window.

[0057] Optionally, when the sliding window in step S6 continuously updates data backward, it slides according to the optimal window length l obtained in step S2.

[0058] Optionally, the historical production data before time t in step S1 is production data collected at equal intervals according to a predetermined sampling frequency.

[0059] Optionally, the historical production data before time t in step S1 is production and manufacturing data collected every hour within one year before time t.

[0060] Optionally, in the combined prediction model, the LSTM and GRU model parameters include the number of training rounds, the initial learning rate, the number of learning rate decay rounds, the learning rate decay speed and the number of hidden layer nodes; the SARIMA model parameters include the autoregressive order of the trend, the non-seasonal difference number, the moving average order of the trend, the seasonal autoregressive order, the seasonal difference number and the seasonal moving average order.

[0061] Optionally, in step S4, the second layer reinforcement learning performs weight optimization on the combination of the LSTM prediction model and the GRU prediction model with an addition or subtraction amplitude of Δω1=1.

[0062] Optionally, in step S5, the addition and subtraction amplitude Δω2=0.001 is used when optimizing the third-layer reinforcement learning combination weight.

[0063] The beneficial effects of the present invention are:

[0064] In view of the fact that lithium battery manufacturing capacity has characteristics such as periodicity, mutation and seasonality, this application selects a prediction algorithm that meets the characteristics of the prediction object to respectively mine the linear and nonlinear characteristics in the time series to form a new combination prediction algorithm. Secondly, compared with the traditional weight determination method, it is difficult to obtain the optimal combination prediction weight. This application designs a double-layer reinforcement learning algorithm to solve the optimal weight matrix at this time to reduce the prediction error; in addition, since the prediction effect of each single prediction algorithm is different at different time points, this application integrates reinforcement learning and entropy ideas, explores the optimal sliding window length, uses sliding windows to divide the prediction sequence, and uses a double-layer reinforcement learning algorithm in each window to determine the optimal combination weight, and finally constructs a triple reinforcement learning battery manufacturing capacity combination prediction model. Compared with the traditional fixed weight combination prediction, it can better utilize the characteristics of each single prediction algorithm, integrate their respective advantages, and further improve the prediction accuracy and reliability of battery manufacturing capacity prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0066] Figure 1 It is a triple reinforcement learning combined prediction model diagram disclosed in one embodiment of the present invention.

[0067] Figure 2 This is a schematic diagram of a principle for determining the sliding window length by reinforcement learning-entropy disclosed in one embodiment of the present invention.

[0068] Figure 3 This is a comparison chart of prediction accuracies using different window lengths disclosed in one embodiment of the present invention.

[0069] Figure 4 This is a comparison chart of the lithium battery manufacturing capacity prediction results disclosed in one embodiment of the present invention.

[0070] Figure 5 2 is a comparison chart of prediction errors of a triple reinforcement learning algorithm and a long short-term memory network disclosed in one embodiment of the present invention.

[0071] Figure 6 2 is a comparison chart of prediction errors between the triple reinforcement learning algorithm and the gated recurrent unit disclosed in one embodiment of the present invention.

[0072] Figure 7 This is a comparison chart of prediction errors between the triple reinforcement learning algorithm and the reinforcement learning-long short-term memory network-gated recurrent unit disclosed in one embodiment of the present invention.

[0073] Figure 8 2 is a comparison chart of prediction errors of a triple reinforcement learning algorithm and a fixed weight combination disclosed in one embodiment of the present invention. DETAILED DESCRIPTION

[0074] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0075] Example 1:

[0076] This embodiment provides a battery manufacturing capacity variable weight combination prediction method based on triple reinforcement learning, see Figure 1 The method predicts production capacity for a period of time in the future based on historical production data, and the method includes:

[0077] Step 1: Based on the characteristics of lithium battery manufacturing capacity sequences, the Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), and Seasonal Autoregressive Integrated Moving Average (SARIMA) models are selected as individual forecasting models to form a combined forecasting model. Based on historical production data, the forecast results of each individual forecasting model at time t are obtained: Among them, the superscripts l, g, and s represent the LSTM prediction model, the GRU prediction model, and the SARIMA prediction model, respectively. The historical production data refers to the production and manufacturing capacity in the period before time t. The production and manufacturing data can be collected at equal intervals according to a determined sampling frequency.

[0078] Step 2: Based on the single prediction result of step 1, a reinforcement learning-entropy algorithm is designed to determine the sliding window length and obtain the optimal sliding window length l;

[0079] Step 3: Based on the optimal window length l obtained in step 2, and the true value sequence Y t Perform window division, with k representing the time point after division, and the window with time k as the first data is called the kth window; the prediction result and the true value at time k are respectively: Y k ;

[0080] Step 4: Based on the window data divided in step 3, design the second layer of reinforcement learning to optimize the combined weights of the LSTM prediction model and the GRU prediction model to obtain the optimal combined weight [ω l ,ω g ];

[0081] Step 5: Optimal combination of LSTM prediction model and GRU prediction model according to step 4 The prediction results of the SARIMA forecast model Together, we conduct the third layer reinforcement learning combination weight optimization to obtain the optimal combination weight [ω nn ,ω s ] and calculate the battery manufacturing capacity forecast;

[0082] Step 6: Slide the window backward to continuously update the data, and repeat the algorithm of steps 4 and 5 to calculate the optimal combination weight of the window until the last window, and obtain the optimal combination prediction sequence

[0083] In this method, during the first layer of reinforcement learning, a reinforcement learning-entropy algorithm is designed to determine the sliding window length, with the goal of finding the window length that minimizes the error fluctuation of the manufacturing capacity prediction subsequence; while ensuring the optimal sliding window length, the amount of computation brought by the enumeration method in the past when determining the window length is avoided; during the second layer of reinforcement learning, the LSTM prediction model and the GRU prediction model are combined and weighted to optimize for mining linear features in time series; during the third layer of reinforcement learning, based on the optimal combination results of LSTM and GRU, the third layer of reinforcement learning combined weight optimization is performed together with the prediction results of the SARIMA prediction model used to mine nonlinear features in time series. In the above method, the process of optimizing the weights of the three models is implemented separately in the second layer of reinforcement learning and the third layer of reinforcement learning, avoiding the simultaneous optimization of setting the target state matrix to S=[w l ,w g ,w s ] causes too many states, the agent cannot fully explore, state overload and dimensionality disaster occur, and thus the optimal result cannot be explored.

[0084] Example 2:

[0085] This embodiment provides a variable weight combination prediction method for battery manufacturing capacity based on triple reinforcement learning. The method is based on a new prediction algorithm combination model. For battery manufacturing capacity data with characteristics such as periodicity and mutation, LSTM, GRU and SARIMA are selected as single prediction models to form a new combination prediction model. The first layer of reinforcement learning is designed according to the prediction results of each single prediction model, that is, the reinforcement learning-entropy algorithm is used to determine the sliding window length to obtain the optimal sliding window length l. After determining the optimal sliding window length l, the single prediction sequence is and the true value sequence Y t Perform window division, and design the second layer of reinforcement learning to optimize the combination weight of LSTM and GRU according to the divided window data to obtain the optimal combination weight [ω l ,ω g ], and then according to the optimal combination weight [ω l ,ω g ]Get the prediction results of LSTM and GRU, combined with the SARIMA prediction results Jointly design the third layer of reinforcement learning combination weight optimization to obtain the optimal combination weight [ω nn ,ω s ], and calculate the optimal combination weight [ω nn ,ω s ] The battery manufacturing capacity prediction value is obtained; the sliding window updates the data, repeats the second layer reinforcement learning and the third layer reinforcement learning until the last window, and obtains the optimal combination prediction sequence The method comprises:

[0086] Step 1: Select LSTM, GRU and SARIMA as single prediction models to form a new combined prediction model;

[0087] In the combined prediction model, the LSTM model has three inputs at time t, where C t-1 is the long-term memory information at time t-1, h t-1 is the short-term memory information at time t-1, x t is the input sample at time t, and the output gate i t 、Forget Gate t and output gate o t The solution is shown in formula (1):

[0088]

[0089] Among them, σ is the Sigmoid activation function, w and b are the weight matrix and bias vector of the corresponding gate respectively. Then, the short-term memory information h can be obtained. t , long-term memory information C t And the model output y t , the calculation method is shown in formula (2):

[0090]

[0091] The model input of the GRU model is the memory information h at time t-1 t-1 and the input variable x at time t t , the corresponding input-output relationship is shown in formula (3):

[0092]

[0093] where x t is the input sample at time t, r t and z t are the output variables of the reset gate and update gate at time t respectively.

[0094] The LSTM and GRU model parameters are defined in Table 1:

[0095] Table 1: LSTM and GRU model parameters

[0096]

[0097] SARIMA(p,d,q)(P,D,Q) SThe model consists of seven parameters, where p and q represent the order of autoregression and moving average, d and D represent the order of trend difference and seasonal difference, P and Q represent the order of seasonal autoregression and seasonal moving average, and S represents the number of seasonal cycles. The general expression of SARIMA is shown in Equation (4):

[0098] φ(B)Φ(B S )(1-B) d (1-B S ) D y t =c+θ(B)θ(B S )ε t (4)

[0099] Among them, B is the delay operator, that is, By t =y t-1 , Φ(B) and Θ(B) are the p-order and q-order characteristic polynomials of B, φ(B) is an autoregressive process, θ(B) is a moving average process, and y t is the input sequence, ε t is white noise.

[0100] SARIMA model parameters are shown in Table 2:

[0101] Table 2: SARIMA model parameters

[0102]

[0103] Step 2: Based on the pre-collected historical production data, obtain the prediction results of each individual prediction model at time t:

[0104] Among them, the superscripts l, g, and s represent the LSTM prediction model, the GRU prediction model, and the SARIMA prediction model, respectively. The historical production data refers to the production and manufacturing capacity in the period before time t. The production and manufacturing data can be collected at equal intervals according to a determined sampling frequency.

[0105] Step 3: Based on the prediction results of each individual prediction model, design the first layer of reinforcement learning: the reinforcement learning-entropy algorithm determines the sliding window length and obtains the optimal sliding window length l;

[0106] See also Figure 2 , the process of determining the optimal sliding window length l includes:

[0107] Step 3.1: Determine the state matrix S. In this problem, the goal is to find the window length that minimizes the error fluctuation of the manufacturing capability prediction subsequence. Therefore, the state matrix can be selected as the window length l.

[0108] S=[l] (5)

[0109] Step 3.2: In order to allow the agent to traverse all possible states during the optimization process, the action matrix A is set to perform fixed-amplitude additions and subtractions on the state of the previous moment, so it is set to:

[0110] A=[Δω1,-Δω1] (6)

[0111] Step 3.3: From the true value Y t and predicted values The absolute prediction error of manufacturing capacity of the i-th single prediction algorithm at time t is calculated as:

[0112]

[0113] Where t=1,2,...,N,i=1,2,3.

[0114] Step 3.4: Calculate the average absolute error of manufacturing capability of K single-item prediction algorithms at time t using formula (3):

[0115]

[0116] Step 3.5: Calculate the average absolute error of the manufacturing capacity at the moment of the average absolute error of the lithium battery manufacturing capacity prediction data in the kth window according to formula (4):

[0117]

[0118] And normalize the mean absolute error:

[0119]

[0120] Step 3.6: Calculate the entropy value of the manufacturing capacity prediction data under this window according to formula (6):

[0121]

[0122] Step 3.7: Introduce equation (7) to set the loss function L and the reward and punishment function R. The loss function L is set to:

[0123]

[0124] In order to explore the state value that minimizes the entropy value, this application sets the reward and penalty function R to:

[0125] When L l+1 <L l hour

[0126] R=1+L t -L t+1 (12)

[0127] When L l+1 >L l hour

[0128] R=-1+L t -L t+1 (14)

[0129] Step 3.8: Construct a reinforcement learning process to obtain the optimal sliding window length.

[0130] By running the code in Example 1, we determine that the optimal sliding window length is 5.

[0131] To verify the effectiveness of the reinforcement learning-entropy algorithm proposed in this example, the root mean square error of the variable weight combination prediction algorithm under different sliding window lengths is calculated for comparison. Figure 3 When the sliding window length is 5, the root mean square error of the prediction result is the smallest, which verifies the effectiveness and accuracy of the reinforcement learning-entropy algorithm designed in this application to determine the sliding window length, while avoiding the computational complexity brought by the enumeration method in the past when determining the window length.

[0132] The parameter settings for the reinforcement learning-entropy algorithm constructed in this embodiment are shown in Table 3:

[0133] Table 3: Reinforcement Learning-Entropy Algorithm Parameters

[0134]

[0135]

[0136] Step 4: Based on the optimal window length l obtained in step 3, and the true value sequence Y t Perform window division, with k representing the time point after division, and the window with time k as the first data is called the kth window; the prediction result and the true value at time k are respectively: Y k ;

[0137] Step 5: Based on the window data divided in step 4, the second layer of reinforcement learning is designed to optimize the combined weights of LSTM and GRU to obtain the optimal combined weight [ω l ,ω g ];

[0138] Considering the large amount of lithium battery manufacturing time series data collected, the high frequency of collection, and the characteristics of periodicity and seasonality, this application uses deep learning models LSTM and GRU and a statistical model-based prediction method SARIMA to respectively mine the nonlinear and linear features in lithium battery production data to form a combined prediction model;

[0139] First, based on the optimal window length determined in step 3, reinforcement learning is used within each window period to solve the optimal weights of the deep learning model LSTM and the GRU single prediction model. The method includes:

[0140] Step 5.1: Set the target state matrix S. The exploration goal is to obtain the optimal weight combination, so the state matrix is ​​set to S = [ω l ,ω g ], where ω l +ω g =1.

[0141] Step 5.2: Set the action matrix A = [Δω2, -Δω2], select the action, and l Increase and decrease, and get ω g , forming the next state.

[0142] Step 5.3: Since the enterprise needs to achieve a stable forecast of future manufacturing capacity and does not want to pursue a certain point of forecast accuracy at the expense of large errors at other time points, this application takes the minimum root mean square error of the data in the sliding window as the exploration goal and sets the loss function L for weighted optimization in the kth window:

[0143]

[0144] According to the loss function L, the reward and punishment function is set to R = 1 / L.

[0145] Step 5.4: Construct the second layer of reinforcement learning optimization process according to steps 5.1 to 5.3 until the optimal number of iterations is reached. At this time, explore the optimal combination weight of LSTM and GRU under this window, assign this weight to the first time point in the window, and add the weight combination to obtain the LSTM and GRU combination prediction value at this time point k.

[0146]

[0147] The results of the combined prediction weights of LSTM and GRU determined by the second layer of reinforcement learning are shown in Table 4:

[0148] Table 4: Optimal calculation results of the second layer reinforcement learning

[0149]

[0150] Table 4 shows the weight values ​​at each time point obtained after the second-layer reinforcement learning weight optimization, as well as the combined prediction results of the LSTM and GRU algorithms obtained using the combined weights (due to space limitations, only some points are shown here).

[0151] Step 6: Optimal combination results based on LSTM and GRU and SARIMA forecast results Jointly conduct the third layer of reinforcement learning combination weight optimization.

[0152] Step 6.1: Set the target state matrix S. The exploration goal is to obtain the optimal weight combination, so the state matrix is ​​set to S = [ω nn ,ω s ], where ω nn +ω s =1.

[0153] Step 6.2: Set the action matrix A = [Δω2, -Δω2], select the action, and nn Increase and decrease, and get ω s , forming the next state.

[0154] Step 6.3: Refer to step 5 to optimize the combined weights of LSTM and GRU. The company under consideration needs to achieve a stable forecast of future manufacturing capacity and does not want to pursue a certain point of prediction accuracy at the expense of excessive errors at other time points. Set the loss function L:

[0155]

[0156] According to the loss function L, the reward and punishment function is set to R = 1 / L.

[0157] Step 6.4: Construct the third layer of reinforcement learning optimization process according to steps 6.1 to 6.3 until the optimal number of iterations is reached. At this time, explore the optimal combination weight of the three single prediction algorithms under this window, assign this weight to the first time point in the window, and add up the weight combination to obtain the optimal combination prediction value at this time point k.

[0158]

[0159] The results of the combined prediction weights determined by the third layer of reinforcement learning are shown in Table 5:

[0160] Table 5: Results of optimal weight values ​​for the third layer reinforcement learning

[0161]

[0162] The parameter settings for the reinforcement learning combination weight optimization process in steps 5 and 6 are shown in Table 6:

[0163] Table 6: Parameters for optimizing combined weights in reinforcement learning

[0164]

[0165] Step 7: Based on the optimal window length l obtained in step 3, the sliding window is continuously updated backward, and the optimization process of steps 5 and 6 is repeated to calculate the optimal combination weight of the window until the last window is obtained to obtain the optimal combination prediction sequence.

[0166] In order to verify the prediction performance of the method of the present application (hereinafter referred to as the Triple-RL method), the estimation results of the method of the present application are compared with those of four existing methods to judge the pros and cons of the method. The four existing methods are the long short-term memory network method (hereinafter referred to as the LSTM method), the gated recurrent unit (hereinafter referred to as the GRU method), the long short-term memory network and gated recurrent unit combination prediction method based on reinforcement learning (hereinafter referred to as the RL-LSTM-GRU method), and the fixed weight combination model prediction method based on the long short-term memory network method, the gated recurrent unit and the seasonal difference autoregressive sliding average model (hereinafter referred to as the LSTM-GRU-SARIMA method).

[0167] During the experiment, the production data of the 18650 battery production workshop of Zhejiang Tianneng Battery Co., Ltd. was used for engineering verification. The workshop counted the number of qualified batteries produced on an hourly basis, generating 24 observations per day. This application collected 300 consecutive days of historical data, totaling 7,200 data points as data samples, and selected the first 85% as training set samples and the last 15% as test set samples.

[0168] Figure 4 The chart shows the prediction results for 72 future time points using five different algorithms. The diamond dashed line represents the actual manufacturing capacity, the star dashed line represents the manufacturing capacity prediction using the RL-LSTM-GRU algorithm, the hexagonal dashed line represents the manufacturing capacity prediction using the LSTM algorithm, the circular dashed line represents the manufacturing capacity prediction using the GRU algorithm, the × dashed line represents the manufacturing capacity prediction using the LSTM-GRU-SARIMA algorithm, and the + dashed line represents the manufacturing capacity prediction using the Triple-RL algorithm.

[0169] Figure 5-Figure 8The error comparison results of the present application method Triple-RL and four comparison algorithms, LSTM, GRU, RL-LSTM-GRU, and LSTM-GRU-SARIMA, are respectively shown. The dot marks are the error percentages of the present application method Triple-RL algorithm when predicting points, and the diamond marks are the error percentages of each comparison algorithm when predicting points. The analysis found that most of the time, the prediction error of the time-varying weight combination prediction algorithm proposed by the present application method is smaller than that of the comparison algorithm, and the overall prediction accuracy is significantly improved. At some points, the error is slightly larger than that of the comparison algorithm. This is because in the process of reinforcement learning optimization, the reward and punishment function is set to the root mean square error within each window, with the aim of making the overall prediction result more accurate. Therefore, the accuracy at some points will be sacrificed, but the overall error is reduced and the prediction accuracy is improved.

[0170] The comparison of battery manufacturing capacity prediction errors under different prediction methods is shown in Table 7:

[0171] Table 7: Comparison of battery manufacturing capacity prediction errors under different prediction methods

[0172]

[0173] By analyzing Table 7, compared with the single prediction model LSTM, the root mean square error of the proposed method is reduced from 86.2824 to 40.2137, a decrease of 53%. The mean absolute error and mean absolute percentage error are reduced from 71.8703 and 0.00622 to 28.4285 and 0.00245, respectively, a decrease of 60%.

[0174] Based on deep learning time series combination prediction, this application adds the SARIMA algorithm to further explore the linear characteristics of the time series. Compared with the RL-LSTM-GRU algorithm, the root mean square error of the algorithm proposed in this application is reduced from 73.8737 to 40.2137, a decrease of 45%, and the mean absolute error and mean absolute percentage error are reduced from 57.892 and 0.00501 to 28.4285 and 0.00245, respectively, a decrease of 52%.

[0175] Compared with fixed-weight combination prediction, this application introduces sliding windows and reinforcement learning to implement variable-weight combination prediction. The root mean square error is reduced from 65.8867 to 40.2137, a decrease of 38%. The mean absolute error and mean absolute percentage error are reduced from 53.9252 and 0.00466 to 28.4285 and 0.00245, respectively, a decrease of 47%. This demonstrates the effectiveness and superiority of the battery manufacturing capacity prediction algorithm proposed in this application.

[0176] Some steps in the embodiments of the present invention may be implemented using software, and the corresponding software program may be stored in a readable storage medium, such as a CD or a hard disk.

[0177] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A variable weight combination prediction method for battery manufacturing capacity based on triple reinforcement learning, characterized in that: The method comprises: Step S1: construct a combined prediction model based on LSTM, GRU and SARIMA single prediction models, and use LSTM, GRU and SARIMA single prediction models to obtain the prediction results at time t according to the historical production data before time t. Step S2: Based on the prediction results of each single prediction model in step S1, a reinforcement learning-entropy algorithm is designed to determine the sliding window length and obtain the optimal sliding window length l; Step S3, based on the optimal window length l obtained in step S2, the single prediction sequence and the true value sequence Y t Perform window division, with k representing the time point after division, and the window with time k as the first data is called the kth window; the prediction result and the true value at time k are respectively: Y k ; Step S4, based on the window data divided in step S3, design the second layer of reinforcement learning to optimize the combined weight of the LSTM prediction model and the GRU prediction model to obtain the optimal combined weight [ω l ,ω g ], and the optimal combination weight [ω l ,ω g ] is assigned to the first time point k in the window, and the LSTM and GRU combined prediction value at time point k is obtained by adding the weight combination Step S5: The optimal combination of LSTM and GRU obtained in step S4 and SARIMA forecast results Together, we conduct the third layer reinforcement learning combination weight optimization to obtain the optimal combination weight [ω nn ,ω s ] and calculate the battery manufacturing capacity forecast; Step S6: The sliding window continuously updates the data backward, and steps S4 and S5 are repeated to calculate the optimal combination weight of the current window until the last window is reached to obtain the optimal combination prediction sequence.

2. The method according to claim 1, characterized in that The step S2 comprises: Step S2.1, determine the state matrix S, and the state matrix selects the window length l; S＝[l] (5) Step S2.2: Set the action matrix A to the state of the previous moment and perform fixed-amplitude addition and subtraction: A=[Δω1,-Δω1] (6) Δω1 represents the amplitude of addition or subtraction; Step S2.3, the true value Y t and predicted values The absolute prediction error of manufacturing capacity of the i-th single prediction algorithm at time t is calculated as: Where t=1,2,...,N,i=1,2,3; Step S2.4: Calculate the mean absolute error of manufacturing capability of K single-item prediction algorithms at time t using formula (3): Step S2.5, calculate the average absolute error of the manufacturing capacity at the moment of the average absolute error of the lithium battery manufacturing capacity prediction data in the kth window according to formula (4): And normalize the mean absolute error: Step S2.6, calculate the entropy value of the manufacturing capacity prediction data under this window according to formula (6): Step S2.7, introduce equation (7) to set the loss function L and the reward and punishment function R. The loss function L is set to: When L l+1 <L l When , the reward and punishment function R is set to: R=1+L t -L t+1 (13) When L l+1 >L l When , the reward and punishment function R is set to: R=-1+L t -L t+1 (14) Step S2.8, construct a reinforcement learning process to obtain the optimal sliding window length.

3. The method according to claim 2, characterized in that The step S4 comprises: Step S4.1, for the problem of optimizing the weight of LSTM and GRU combination, set the target state matrix S = [ω l ,ω g ], where ω l +ω g =1; Step S4.2, set the action matrix A = [Δω2, -Δω2], select the action, and l Increase and decrease, and get ω g , forming the next state, Δω2 represents the amplitude of addition and subtraction; In step S4.3, the minimum root mean square error of the data in the sliding window is taken as the exploration goal, and the loss function L for weighted optimization in the k-th window is set: According to the loss function L, the reward and punishment function is set to R = 1 / L; Step S4.4: Construct the second-layer reinforcement learning optimization process according to steps S4.1 to S4.3 until the optimal number of iterations is reached. At this time, explore the optimal combination weight of LSTM and GRU under the current window, assign the weight to the first time point in the window, and obtain the LSTM and GRU combination prediction value at the time point k according to the weight combination. According to formula (16), the LSTM and GRU combined prediction value at time point k is obtained: When ω l and ω g is the optimal combined weight value of LSTM and GRU in the current window.

4. The method according to claim 3, characterized in that The step S5 comprises: Step S5.1, for the optimal combination results based on LSTM and GRU and SARIMA forecast results Together we conduct the combined weight optimization problem and set the target state matrix S = [ω nn ,ω s ], where ω nn +ω s =1; Step S5.2, set the action matrix A = [Δω2, -Δω2], select the action, and nn Increase and decrease, and get ω s , forming the next state; Step S5.3, set the loss function L: According to the loss function L, the reward and punishment function is set to R = 1 / L; Step S5.4: Construct the third layer of reinforcement learning optimization process according to steps S5.1 to S5.3 until the optimal number of iterations is reached; at this time, explore the optimal combination weight of the three single prediction models under the current window, assign the weight to the first time point in the window, and obtain the optimal combination prediction value at the time point k according to the weight combination. According to formula (18), the optimal combination prediction value at time point k is obtained: When ω nn and ω s is the optimal combined weight value of the three single prediction models in the current window.

5. The method according to claim 4, characterized in that When the sliding window in step S6 continuously updates data backward, it slides according to the optimal window length l obtained in step S2.

6. The method according to claim 5, characterized in that The historical production data before time t in step S1 is production data collected at equal intervals according to a predetermined sampling frequency.

7. The method according to claim 6, characterized in that The historical production data before time t in step S1 is production data collected every hour within one year before time t.

8. The method according to claim 6, characterized in that In the combined prediction model, the LSTM and GRU model parameters include the number of training rounds, the initial learning rate, the number of learning rate attenuation rounds, the learning rate attenuation speed, and the number of hidden layer nodes; the SARIMA model parameters include the autoregressive order of the trend, the non-seasonal difference number, the moving average order of the trend, the seasonal autoregressive order, the seasonal difference number, and the seasonal moving average order.

9. The method according to claim 7, characterized in that In the step S4, the second layer reinforcement learning performs weight optimization on the LSTM prediction model and the GRU prediction model, and the addition and subtraction amplitude Δω1=1.

10. The method according to claim 8, characterized in that In the step S5, the addition and subtraction amplitude Δω2 is 0.001 when optimizing the third layer reinforcement learning combination weight.

Images