Time series data prediction method, device, and medium based on federal reserve pool model
Patent Information
- Application Number
- CN202410547856.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-06
- Publication Date
- 2026-10-30
- Estimated Expiration
- 2044-05-06
AI Technical Summary
然而，这增加了模型的复杂性和计算成本
[0026]本发明方法基于储备池将待预测的时序数据映射到高维可计算空间数据，通过联邦学习的方法联合更新输出层参数，达到跨域数据孤岛和减小每个用户训练代价的目的。最后，通过高斯牛顿方法提高客户端模型的迭代收敛的速度和效率，并减小计算量。本发明方法将类脑计算中的储备池算法应用到联邦学习场景中，实现时序数据的快速分析和预测。
Smart Images
Figure CN118410338B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of time series information prediction technology, and in particular to a method, device, and medium for time series data prediction based on a federal reserve pool model. Background Technology
[0002] There are various methods for time series data prediction, some of which include statistical methods, deep learning methods, support vector machines, decision tree methods, and model fusion methods.
[0003] 1. Statistical methods: such as linear regression, autoregressive models (AR), autoregressive filters (ARIMA), and seasonal autoregressive models (SARIMA). These statistical methods typically predict future trends based on past observations. However, these methods only utilize limited past observations, may ignore other influencing factors, are sensitive to outliers, and do not consider the nonlinear relationships in the data.
[0004] 2. Deep learning methods: such as Long Short-Term Memory (LSTM), Bidirectional Long Short-Term Memory (BiLSTM), and Recurrent Neural Networks (RNN). These methods are particularly suitable for complex nonlinear time series forecasting and can capture long-term dependencies. However, deep learning models typically require large amounts of data and computational resources.
[0005] 3. Support Vector Machine (SVM): SVM can perform classification and regression prediction on time-series data. However, its performance is affected by the choice of kernel function, and training on large-scale datasets can be time-consuming.
[0006] 4. Decision Tree Method: This method uses decision tree models (such as random forests) for classification and regression prediction of time series data. While easy to understand and interpret, it may suffer from overfitting and may not perform well for complex nonlinear relationships. Model Fusion Method: This method combines the predictions of multiple models to obtain more accurate predictions. However, this increases the complexity of the model and the computational cost.
[0007] In summary, although there are various methods for time series data prediction, some disadvantages still exist. Summary of the Invention
[0008] To address the shortcomings of existing technologies, this invention provides a method, device, and medium for time-series data prediction based on a federal reserve pool model.
[0009] In a first aspect, the present invention provides a time-series data prediction method based on a federal reserve pool model, the method comprising:
[0010] Obtain the time series data to be predicted;
[0011] The time series data to be predicted is input into a pre-trained client to obtain the time series prediction results;
[0012] The client-side training process includes:
[0013] A federal reserve pool model is constructed, which includes a reserve pool input layer, a reserve pool implicit layer, and several reserve pool output layers; the reserve pool input layer and the reserve pool implicit layer are used as servers, and each reserve pool output layer is used as a client.
[0014] After initializing the federal reserve pool model, the client receives the initialization parameters sent by the server;
[0015] Each client is iteratively trained using the Gauss-Newton method, and the parameters of all reservoir output layers are obtained and uploaded to the server.
[0016] After the server aggregates the parameters of all the output layers of the reserve pools, the client receives the aggregated global model parameters sent by the server.
[0017] The client uses the global model for updates, thus completing the client's training.
[0018] Secondly, the present invention provides a time-series data prediction system based on a federal reserve pool model, used to implement the aforementioned time-series data prediction method based on a federal reserve pool model, the system comprising:
[0019] The time series data acquisition module is used to acquire the time series data to be predicted;
[0020] The time series prediction result acquisition module is used to input the time series data to be predicted into a pre-trained client to obtain the time series prediction result;
[0021] The client training module is used to construct a federated reserve pool model, which includes a reserve pool input layer, a reserve pool hidden layer, and several reserve pool output layers. The reserve pool input layer and the reserve pool hidden layer act as servers, and each reserve pool output layer acts as a client. After initializing the federated reserve pool model, the client receives initialization parameters from the server. Each client is iteratively trained using the Gauss-Newton method to obtain the parameters of all reserve pool output layers, which are then uploaded to the server. After the server aggregates the parameters of all reserve pool output layers, the client receives the aggregated global model parameters from the server. The client then updates the global model, thus completing the client's training.
[0022] Thirdly, the present invention provides an electronic device including a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above-described time-series data prediction method based on the federal reserve pool model.
[0023] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described time-series data prediction method based on the federal reserve pool model.
[0024] Fifthly, the present invention provides a computer program product, including a computer program / instruction that, when executed by a processor, implements the aforementioned time-series data prediction method based on the federated reserve pool model.
[0025] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0026] This invention's method maps the time-series data to be predicted to a high-dimensional computable space based on a reservoir. It then uses federated learning to jointly update the output layer parameters, achieving the goals of overcoming data silos and reducing the training cost for each user. Finally, the Gauss-Newton method is used to improve the speed and efficiency of iterative convergence of the client model while reducing computational load. This invention applies the reservoir algorithm from neuromorphic computing to a federated learning scenario, enabling rapid analysis and prediction of time-series data. Attached Figure Description
[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0028] Figure 1 This is a schematic diagram of the federal reserve pool model provided in an embodiment of the present invention;
[0029] Figure 2 This is a schematic diagram of a time-series data prediction method based on a federal reserve pool model provided in an embodiment of the present invention;
[0030] Figure 3 The function curve plotted by Van-der-Pol in solving the ordinary differential equation provided in the embodiments of the present invention;
[0031] Figure 4 This is a curve of the solution to the ordinary differential equation Lorenz63, as provided in the embodiment of the present invention.
[0032] Figure 5This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0034] It should be noted that, unless otherwise specified, the features in the following embodiments and implementation methods can be combined with each other.
[0035] like Figure 1 and Figure 2 As shown, this invention provides a time series data prediction method based on a federated reserve pool model. Each client holds its own unique input data, which cannot go out of the domain. The server has a common reserve pool layer that does not need to be trained. Each client holds its own reserve pool readout layer, which undergoes federated learning based on the Gauss-Newton iterative method. Finally, each client has its own output layer, which is the predicted value of its own time series data.
[0036] The method specifically includes the following steps:
[0037] Step S1: Obtain the time series data to be predicted.
[0038] Specifically, time-series data includes various types of data, such as numerical data (e.g., sensor data such as temperature, humidity, and pressure, stock prices, etc.), Boolean data (e.g., switch status, alarm status, etc.), string data (e.g., log data, device status, etc.), geographic location data (e.g., GPS data, earthquake data, etc.), and multimedia data such as images, audio, and video (e.g., surveillance video, audio signals, etc.).
[0039] In this example, the time series data can be meteorological data (such as temperature, precipitation, and humidity) to predict temperature change trends; or traffic data (such as vehicle flow and pedestrian flow) to predict traffic congestion; or oscillator state variables to predict oscillation phenomena; or convection terms of vertical velocity, horizontal wind speed differences, temperature, or humidity in two-dimensional space during atmospheric convection to predict atmospheric convection processes.
[0040] Step S2: Input the time series data to be predicted into the pre-trained client to obtain the time series prediction result.
[0041] The client-side training process includes:
[0042] Step S100: Construct a federal reserve pool model, which includes a reserve pool input layer, a reserve pool hidden layer, and several reserve pool output layers; use the reserve pool input layer and the reserve pool hidden layer as servers, and use each reserve pool output layer as a client.
[0043] It's important to note that reservoir computing maps input signals to high-dimensional computable data space using a fixed recurrent neural network. Reservoir computing can rapidly process time-series data and capture non-linear features within the data. Federated learning, on the other hand, uses methods to jointly update output layer parameters, achieving cross-domain data islanding and reducing the training cost per user. Federated learning allows multiple participants to share model updates while maintaining data privacy.
[0044] Step S200: Initialize the federal reserve pool model and send the server's initialization parameters to each client;
[0045] Step S300: Iteratively train each client based on the Gauss-Newton method to obtain the parameters of all reservoir output layers;
[0046] Specifically, step S300 includes the following steps:
[0047] Step S301: Set an initial estimate x0, which can be arbitrarily selected.
[0048] Step S302: For the k-th iteration, k∈{1,2,…,K}, the current estimated value is known to be x. k Calculate the objective function f(x) in x k Jacobian matrix J(x) at the location k Jacobian matrix J(x) k ) is a matrix consisting of the partial derivatives of f(x). Calculate the residual r. k =f(x) k Solve the incremental equation J(x) k ) T J(x k )Δx k =-J(x) k ) T r k To obtain the increment Δx k Updated estimate: x k+1=x k +Δx k ;
[0049] In this example, the objective function f(x) is set according to the time series data to be predicted.
[0050] For example, when the time series data to be predicted is selected as oscillator state variables to predict oscillation phenomena, the objective function f(x) can be set as the Van der Pol equation, as shown below:
[0051]
[0052] In the formula, x(t) represents the oscillator state variable (usually displacement or voltage) at time t. and Let these represent their derivatives with respect to time, i.e., velocity and acceleration, respectively.
[0053] μ is a positive real parameter, namely the van der Pol coefficient, which determines the strength of the nonlinear effect. When μ = 0, the system simplifies to an undamped simple harmonic oscillator; as μ increases, the nonlinear effect becomes significant.
[0054] (1-x 2 The term is nonlinear, introducing state-dependent negative damping. When x approaches zero, the damping approaches zero; however, when x reaches a large absolute value, the damping becomes positive, resulting in amplitude suppression. This characteristic allows the VdP oscillator to spontaneously maintain stable periodic oscillations without external drive, forming a so-called "limit cycle".
[0055] It should be noted that the Van der Pohl (VdP) oscillation ator is an important model in classical dynamics and circuit theory, used to describe the behavior of certain electronic oscillators, especially self-excited oscillations in the presence of negative damping effects. VdP oscillators have wide applications in physics and engineering, such as modeling oscillation phenomena in radio technology and biophysics.
[0056] For example, when the time series data to be predicted uses the convection terms of vertical velocity, horizontal wind speed difference, temperature, or humidity in a two-dimensional space during atmospheric convection processes, the objective function f(x) can be set as the Lorenz 63 and Lorenz 96 ordinary differential equations, expressed as follows:
[0057]
[0058] In the formula, x represents the convection term of vertical velocity in two-dimensional space, y represents the convection term of temperature (or humidity), z represents the convection term of horizontal wind speed difference, σ represents the relative strength of horizontal diffusion, β represents the ratio of vertical thermal diffusivity to the convection term, and ρ represents the intensity and stability of the convection process.
[0059] Step S303, check the increment Δx k The size of the estimate is determined by checking if the change in the estimate between two consecutive iterations is small enough. If the convergence condition is met, the iteration stops and the current estimate x is returned. k+1 As a solution.
[0060] In step S400, all clients upload the parameters of the corresponding reserve pool output layer to the server. The server aggregates the parameters of all reserve pool output layers using the average aggregation method to obtain the global model parameters.
[0061] In step S500, the server sends the global model parameters to each client, and the client updates the global model to obtain the trained client.
[0062] Figure 3 The diagram shows the function curves plotted by Van-der-Pol for solving the ordinary differential equation; different curves correspond to different hyperparameters. Figure 4 The figure shows the curves of the solutions to the ordinary differential equation Lorenz63, with different curves corresponding to different parameters describing the ordinary differential equation.
[0063] This invention also provides a time-series data prediction system based on a federal reserve pool model, used to implement the above-described time-series data prediction method based on a federal reserve pool model. The system includes:
[0064] The time series data acquisition module is used to acquire the time series data to be predicted;
[0065] The time series prediction result acquisition module is used to input the time series data to be predicted into a pre-trained client to obtain the time series prediction result;
[0066] The client training module is used to construct a federated reserve pool model, which includes a reserve pool input layer, a reserve pool hidden layer, and several reserve pool output layers. The reserve pool input layer and the reserve pool hidden layer act as servers, and each reserve pool output layer acts as a client. After initializing the federated reserve pool model, the client receives initialization parameters from the server. Each client is iteratively trained using the Gauss-Newton method to obtain the parameters of all reserve pool output layers, which are then uploaded to the server. After the server aggregates the parameters of all reserve pool output layers, the client receives the aggregated global model parameters from the server. The client then updates the global model, thus completing the client's training.
[0067] Regarding the system in the above embodiments, the specific ways in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.
[0068] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0069] In summary, the method of this invention maps the time-series data to be predicted to a high-dimensional computable space based on a reservoir, and jointly updates the output layer parameters through federated learning, thereby overcoming data silos and reducing the training cost for each user. Finally, the Gauss-Newton method is used to improve the speed and efficiency of iterative convergence of the client model and reduce the computational load. This invention applies the reservoir algorithm from neuromorphic computing to the federated learning scenario, enabling rapid analysis and prediction of time-series data.
[0070] like Figure 5 As shown, this application provides an electronic device including a memory 101 for storing one or more programs and a processor 102. When the one or more programs are executed by the processor 102, they implement the method as described in any of the first aspects above.
[0071] The system also includes a communication interface 103. The memory 101, processor 102, and communication interface 103 are electrically connected directly or indirectly to each other to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory 101 can be used to store software programs and modules, and the processor 102 executes various functional applications and data processing by executing the software programs and modules stored in the memory 101. The communication interface 103 can be used for signaling or data communication with other node devices.
[0072] The memory 101 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.
[0073] The processor 102 can be an integrated circuit chip with signal processing capabilities. The processor 102 can be a general-purpose processor 102, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0074] In the embodiments provided in this application, it should be understood that the disclosed methods and systems can also be implemented in other ways. The method and system embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
[0075] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.
[0076] On the other hand, embodiments of this application provide a computer-readable storage medium storing a computer program thereon. When executed by processor 102, the computer program implements the methods described in any of the first aspects above. If the functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory 101 (ROM), random access memory 101 (RAM), magnetic disks, or optical disks.
[0077] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only.
[0078] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.
Claims
1. A time-series data forecasting method based on a federal reserve pool model, characterized in that, The method includes: Obtain the time series data to be predicted; The time series data to be predicted is input into a pre-trained client to obtain the time series prediction results; The client-side training process includes: A federal reserve pool model is constructed, which includes a reserve pool input layer, a reserve pool implicit layer, and several reserve pool output layers; the reserve pool input layer and the reserve pool implicit layer are used as servers, and each reserve pool output layer is used as a client. After initializing the federal reserve pool model, the client receives the initialization parameters sent by the server; Each client is iteratively trained using the Gauss-Newton method to obtain the parameters of all reservoir output layers, which are then uploaded to the server; including: Set a random initial estimate x0; For the k-th iteration, k∈{1,2,…,K}, the current estimate is known to be x. k Calculate the objective function f(x) in x k Jacobian matrix J(x) at the location k ); Calculate the residual r k =f(x k ); Solve the incremental equation J(x) k ) T J(x k )Δx k =−J(x k ) T r k Obtain the increment Δx k ; Update the estimated value x k+1 =x k +Δx k； When the increment Δx k If the value of the estimate is less than a preset threshold, the iteration stops and the current estimate x is returned. k+1 ; When the time series data to be predicted uses oscillator state variables to predict oscillation phenomena, the objective function f(x) is set as the Van der Pol equation; or, When the time series data to be predicted is selected from the convection terms of vertical velocity, horizontal wind speed difference, temperature or humidity in two-dimensional space during atmospheric convection, the objective function f(x) is set as Lorenz63 and Lorenze96 ordinary differential equations. After the server aggregates the parameters of all the output layers of the reserve pools, the client receives the aggregated global model parameters sent by the server. The client uses the global model for updates, thus completing the client's training.
2. The time-series data forecasting method based on the federal reserve pool model according to claim 1, characterized in that, The time-series data can be selected from meteorological data, which includes temperature, precipitation, and humidity, to predict temperature change trends. Alternatively, the time-series data may be traffic data, including vehicle flow and pedestrian flow, to predict traffic congestion. Alternatively, the time-series data may be selected from oscillator state variables to predict oscillation phenomena; Alternatively, the time-series data may be selected from the convection terms of vertical velocity, horizontal wind speed difference, temperature or humidity in two-dimensional space during atmospheric convection processes to predict atmospheric convection processes.
3. A time-series data forecasting system based on a federal reserve pool model, characterized in that, The system is used to implement the time-series data prediction method based on the federal reserve pool model as described in any one of claims 1-2, the system comprising: The time series data acquisition module is used to acquire the time series data to be predicted; The time series prediction result acquisition module is used to input the time series data to be predicted into a pre-trained client to obtain the time series prediction result; The client training module is used to construct a federated reserve pool model, which includes a reserve pool input layer, a reserve pool hidden layer, and several reserve pool output layers. The reserve pool input layer and the reserve pool hidden layer act as servers, and each reserve pool output layer acts as a client. After initializing the federated reserve pool model, the client receives initialization parameters from the server. Each client is iteratively trained using the Gauss-Newton method to obtain the parameters of all reserve pool output layers, which are then uploaded to the server. After the server aggregates the parameters of all reserve pool output layers, the client receives the aggregated global model parameters from the server. The client then updates the global model, thus completing the client's training.
4. An electronic device, comprising a memory and a processor, characterized in that, The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the time series data prediction method based on the federal reserve pool model as described in any one of claims 1-2.
5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the time-series data prediction method based on the federal reserve pool model as described in any one of claims 1-2.
6. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the time-series data prediction method based on the federal reserve pool model as described in any of claims 1-2.