A time series data statistical prediction method based on reinforcement learning

Abstract

The application discloses a kind of time series data statistical prediction methods based on reinforcement learning, the present application relates to electric digital data processing technical field, by to real-time standardization time series data is three-level hierarchical management and control, constructs double-track isolation processing architecture and executes differentiated authority processing, in combination with the backtracking check of critical precursor signal data and authority dynamic adjustment, the advantages of the present application are that: by constructing the double-track isolation processing architecture of adaptive reinforcement learning prediction system, with the three-level hierarchical management and control of time series data and the differentiated authority access mechanism, break through the bottom logic shackles of long-term prediction experience memory and real-time new data learning update depth binding in prior art.Model different characteristics of time series data are isolated by authority, which cuts off the interference path of invalid noise data on the long-term core parameters of the model from the root, avoids the pollution of noise data to the long-term effective prediction experience of model sedimentation, and effectively suppresses the error accumulation problem in long-time series prediction process.

Description

Technical Field

[0001] This invention relates to the field of electronic digital data processing technology, specifically to a time-series data statistical prediction method based on reinforcement learning. Background Technology

[0002] Existing time-series data is the core data carrier for multiple core areas such as industrial production, energy dispatch, urban governance, environmental monitoring, and financial risk control under the industrial internet and Internet of Things system. The sequence data with continuous time dimension fully carries the operating rules, changing trends and potential characteristics of the monitored objects. Accurate time-series data statistical analysis and trend prediction are the core technical support for realizing intelligent management and control, refined operation and forward-looking risk prevention and control in various fields. It is also one of the key research directions in the current process of digital economy and industrial digital transformation. Existing statistical forecasting techniques for time series data mainly rely on traditional statistical analysis models, conventional machine learning algorithms, and reinforcement learning prediction methods to predict trends and control risks in scenarios such as industrial equipment operation monitoring, power grid load scheduling, and environmental quality monitoring. However, these methods have certain shortcomings. First, existing reinforcement learning-based time series forecasting methods generally deeply bind the model's long-term prediction experience memory with the learning and updating of real-time new data. New data indiscriminately updates the model's core parameters, and noisy data easily contaminates long-term effective prediction experience, leading to the continuous accumulation of long-term prediction errors and insufficient long-term stability of the prediction results. Second, existing conventional improvements to address these problems only employ pre-filtering and outlier removal, which can easily mistakenly delete weak precursor signals reflecting trend changes in the time series data, failing to adapt to scenarios with sudden changes in time series data and resulting in insufficient prediction accuracy. Therefore, we propose a reinforcement learning-based statistical forecasting method for time series data. Summary of the Invention

[0003] The purpose of this invention is to provide a method for statistical prediction of time-series data based on reinforcement learning.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a time-series data statistical prediction method based on reinforcement learning, the prediction method comprising the following steps: Step 1: Obtain the raw time-series data output by the entity acquisition device in the target technical scenario, perform standardized preprocessing such as missing value imputation, normalization, time-series alignment and resampling to obtain a standardized time-series dataset. The standardized time-series dataset contains the real-time standardized time-series data to be processed, and simultaneously constructs a historical valid time-series dataset verified by the actual running results in the target scenario. Step 2: For the real-time standardized time-series data obtained in Step 1, quantitatively calculate the degree of matching between its statistical distribution and that of the historical valid time-series dataset. Correlation with trends Two core judgment indicators; Step 3: Based on the preset grading threshold rules, and using two core judgment indicators as the sole basis, the real-time standardized time series data is divided into three control levels: strong effective trend data, critical precursor signal data, and invalid noise data, and the corresponding control level labels are marked. Step 4: Construct a dual-track isolation processing architecture adapted to the reinforcement learning prediction model, set up a long-term experience memory and a real-time prediction cache pool with complete logical isolation, physical isolation of storage partitions, and independent control of operation permissions, and clarify the access rules for data at each control level. Step 5: Based on the control level tags bound to the real-time standardized time series data, perform differentiated access control and processing operations, and match the corresponding permissions to complete parameter updates and policy fine-tuning. Step 6: Perform sliding time window backtracking verification on the critical precursor signal data in the real-time prediction cache pool, and dynamically adjust the data permissions based on the verification results; Step 7: Combine the basic prediction strategy of the long-term experience memory bank with the effective fine-tuning strategy of the real-time prediction cache pool to generate and output the time-series data prediction results, thus completing the entire closed loop.

[0005] As a further aspect of the present invention: In step one, the target technical scenario includes any one of industrial equipment operation status monitoring, power grid load scheduling, environmental quality monitoring, and traffic flow control. The original time series data is continuous numerical sequence data with equally spaced timestamps. A full-process standardized preprocessing operation is performed on the original time series data. First, linear interpolation is used to fill in the missing values ​​of the original time series data. Then, the min-max normalization method is used to map the filled time series data to the [0,1] numerical range to complete the extreme value normalization processing. Subsequently, based on a fixed sampling frequency consistent with the acquisition frequency of the original time series data, time series alignment and resampling operations are performed on the normalized data to remove redundant data corresponding to duplicate timestamps. Finally, a standardized time series dataset with continuous time dimension, uniform numerical range, and consistent sampling interval is obtained. The historical valid time series dataset is a collection of historical time series data that has been verified by the actual operation results of the target scenario and conforms to the real change law of the time series data in the scenario. The time span of the dataset is not less than 30 complete time series data sampling periods.

[0006] As a further aspect of the present invention: In step two, multi-dimensional statistical feature quantification calculations are performed on the real-time standardized time-series data obtained in step one to obtain the statistical distribution matching degree between the set of real-time standardized time-series data and the historical valid time-series dataset. Correlation with trends Two core judgment indicators; Among them, statistical distribution matching degree The calculation formula is as follows: ; in, To standardize time series data in real time and historical valid time series datasets The test statistic characterizes the difference in the cumulative probability distributions of the two groups of data, and its value ranges from [0,1]. To standardize time series data in real time relative to historical valid time series datasets Divergence, representing the difference in distributional information entropy between two sets of data, is valued based on the historical valid time-series dataset. Based on the maximum and minimum divergence values, after min-max normalization, they are fixedly mapped to the interval [0,1]. and The preset weighting coefficients satisfy... ,and , All are greater than 0, and the preset value range is [value range missing]. ; Trend correlation The calculation formula is as follows: ; in, To standardize the Pearson correlation coefficient between real-time time-series data and the corresponding trend sequences of historical valid time-series datasets, representing the degree of linear trend correlation between the two sets of data, the value range is [-1, 1]. Granger causality test on trend sequences of historical valid time series data for real-time standardization of time series data. The value represents the causal explanatory power of real-time data on historical trends, and its range is [0,1]. and The preset weighting coefficients satisfy... ,and , All are greater than 0, and the preset value range is [value range missing]. After calculating the two core judgment indicators, the indicator results are uniquely bound and stored with the corresponding real-time standardized time series data to provide a quantitative judgment basis for subsequent hierarchical management and control.

[0007] As a further aspect of the present invention: In step three, based on a preset hierarchical threshold rule, the calculated statistical distribution matching degree is... Correlation with trends Using this as the sole criterion, a three-tiered hierarchical control system is implemented for real-time standardized time-series data. Preset first distribution threshold Second distribution threshold ,satisfy Preset first trend threshold Second trend threshold ,satisfy In the hierarchical partitioning rules, when real-time standardized time-series data simultaneously meet the following conditions... ≥ and ≥ When real-time standardized time series data simultaneously meet the following conditions, it is classified as strongly effective trend data. When the real-time standardized time series data does not meet any of the above two judgment conditions, it is classified as invalid noise data. After the classification is completed, a unique control level label is marked for each group of real-time standardized time series data. The label is bound to the data one by one, and there is no overlap or omission in the classification intervals.

[0008] As a further aspect of the present invention: In step four, the dual-track isolation processing architecture comprises two core processing units that are logically completely isolated, physically isolated in storage partitions, and independently managed in terms of operation permissions. These are a long-term experience memory and a real-time prediction cache pool, respectively. The long-term experience memory is the sole permanent storage unit for the core parameters of the reinforcement learning prediction model. It stores the core weight parameters for basic time-series prediction, accumulated from long-term training with valid data. It only accepts parameter update operations triggered by data that meets preset permission requirements. Data without permission cannot be written to or modified. The real-time prediction cache pool is a temporary storage unit for the reinforcement learning prediction model. The strategy parameter storage unit stores temporary fine-tuning parameters that are only effective for a single or short-term prediction. Its storage partition is completely physically isolated from the long-term experience memory. All parameters written to the real-time prediction cache pool are only effective within a preset temporary session period and cannot be directly written to or modified in any form. The temporary session period is preset to 1-5 time-series sampling periods. At the same time, the unique permission control rules for the two processing units are clearly defined: only strong and effective trend data has write permission to the long-term experience memory, critical precursor signal data only has write permission to the real-time prediction cache pool, and invalid noise data does not have any write permission to either unit.

[0009] As a further aspect of the present invention: In step five, based on the control level label marked in step three and uniquely bound to the real-time standardized time series data, differentiated access and processing operations are performed on the real-time standardized time series data bound to the corresponding label. Specifically, the processing rules are as follows: For real-time standardized time series data marked as strong effective trend data, access to its long-term experience memory is granted, this data is input into the reinforcement learning prediction model, the core weight parameters of the model stored in the long-term experience memory are updated, and the updated core parameters are synchronously mapped to the real-time prediction cache pool as a basic strategy for short-term prediction. For data marked as critical precursor signals… The real-time standardized time-series data is only granted access to its real-time prediction cache pool. This set of data is input into the reinforcement learning prediction model and is only used to generate temporary fine-tuning parameters that adapt to the current data trend. These parameters are then written to the real-time prediction cache pool. No parameter modification operations are triggered in the long-term experience memory throughout the process. The temporary fine-tuning parameters only take effect for no more than 5 subsequent time-series prediction tasks. For real-time standardized time-series data marked as invalid noise data, an interception operation is directly performed. Only the original information and classification results of this set of data are logged. It is not input into any processing unit of the reinforcement learning prediction model and does not trigger any parameter update or strategy adjustment operations.

[0010] As a further aspect of the present invention: In step six, for the critical precursor signal data written to the real-time prediction cache pool, a sliding time window backtracking verification and dynamic permission adjustment operation is performed. Firstly, a preset length of... A sliding time window for each time-series sampling period The time frame is a positive integer ≥3 and ≤10. Starting from the moment when the critical precursor signal data is written into the real-time prediction buffer, the real time-series data of the subsequent output is continuously collected within the sliding time window. The trend matching degree between the critical precursor signal data and the real time-series data within the sliding time window is calculated. Trend matching degree The calculation formula is as follows: ; in, The Pearson correlation coefficient is the sum of the critical precursor signal data and the actual time series data within the sliding time window, with a value range of [-1, 1]. This is the trend direction consistency coefficient. A value of 1 is assigned when the increasing / decreasing trends of two sets of data are completely consistent in each sampling period within the sliding time window. A value of 0.5 is assigned when the increasing / decreasing trends of two sets of data are consistent in more than 50% but less than 100% of the sampling periods within the sliding time window. A value of 0 is assigned when the increasing / decreasing trends of two sets of data are consistent in 50% or less of the sampling periods within the sliding time window. A value of 0 is assigned when the trends are completely opposite. and The preset weighting coefficients satisfy... ,and , All are greater than 0, and the preset value range is [value range missing]. When the trend matches continuous When all sampling periods are ≥ the preset upgrade threshold of 0.85, the critical precursor signal data is upgraded to strongly effective trend data, its long-term empirical memory is granted access, and the core weight parameters of the model are updated. When the trend matching degree... continuous When all sampling periods are less than the preset clearing threshold of 0.4, the temporary fine-tuning parameters corresponding to this set of critical precursor signal data are automatically expired and cleared from the real-time prediction buffer, stopping their impact on the strategy of subsequent prediction tasks. ≥2 and less than The positive integers, where the upgrade threshold and the clearing threshold are both fixed values, satisfying 1 > upgrade threshold > clearing threshold > 0.

[0011] As a further aspect of the present invention: In step seven, when receiving a time-series prediction task request for the target scenario, the core weight parameters of the model stored in the long-term experience memory are called to generate a basic prediction strategy. At the same time, the effective temporary fine-tuning parameters stored in the real-time prediction cache pool are called to adapt and adjust the basic prediction strategy to generate the final time-series prediction strategy. Based on the final time-series prediction strategy, the statistical prediction calculation of the target time-series data is completed, and a time-series data prediction result sequence with corresponding timestamps is generated. The prediction result sequence is output to the technical execution system corresponding to the target scenario for technical operations corresponding to equipment early warning, scheduling optimization, and risk prevention and control in the target scenario. At the same time, the real feedback time-series data corresponding to this prediction task is added to the historical effective time-series dataset to complete the entire closed loop of the prediction method.

[0012] Compared with the prior art, the beneficial effects of the present invention by adopting the above technical solution are as follows: 1. This invention, by constructing a dual-track isolation processing architecture adapted to the reinforcement learning prediction system, coupled with a three-level hierarchical management and differentiated access control mechanism for time series data, breaks through the underlying logical constraints of the deep binding between the model's long-term prediction experience memory and real-time new data learning and updating in existing technologies. The solution clearly isolates time series data with different characteristics by eliminating the interference path of invalid noise data on the model's long-term core parameters from the root, avoiding the pollution of the model's long-term effective prediction experience by noise data, effectively suppressing the problem of continuous error accumulation in the long-term time series prediction process, and improving the long-term stability of time series data prediction results. At the same time, it does not require structural modifications to the reinforcement learning algorithm itself. The solution has strong adaptability and can be widely applied to time series data prediction work in various industrial and livelihood scenarios. 2. This invention avoids the inherent flaws of existing technologies' one-size-fits-all data screening in pre-filtering and outlier removal schemes by setting up a real-time prediction cache pool that is completely physically and logically isolated from the long-term experience memory, coupled with a sliding time window backtracking verification and dynamic permission adjustment mechanism for critical precursor signal data. For critical precursor signals with trend correlation, the scheme uses isolated caching to fine-tune short-term prediction strategies, fully preserving weak precursor signals reflecting trend changes in time series data without interfering with the model's long-term core parameters. This effectively improves prediction accuracy in scenarios of sudden trend changes in time series data. Simultaneously, the backtracking verification mechanism dynamically adjusts data permissions, balancing the timeliness and operational stability of the prediction model. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the dual-track isolation processing architecture and data flow in an embodiment of the present invention; Figure 2 This is a schematic diagram of the method steps in an embodiment of the present invention; Figure 3 This is a schematic diagram of the logic flow for time-series data statistical feature quantification and three-level hierarchical control in an embodiment of the present invention. Detailed Implementation

[0014] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings. It should be noted that the description of these embodiments is for the purpose of helping to understand the present invention, but does not constitute a limitation of the present invention.

[0015] Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0016] Please see the appendix Figure 1 -Appendix Figure 3 This invention discloses a statistical prediction method for time-series data based on reinforcement learning. The prediction method includes the following steps: Step 1: Obtain the raw time-series data output by the entity acquisition device in the target technical scenario, perform standardized preprocessing such as missing value imputation, normalization, time-series alignment and resampling to obtain a standardized time-series dataset. The standardized time-series dataset contains the real-time standardized time-series data to be processed, and simultaneously constructs a historical valid time-series dataset verified by the actual running results in the target scenario. Step 2: For the real-time standardized time-series data obtained in Step 1, quantitatively calculate the degree of matching between its statistical distribution and that of the historical valid time-series dataset. Correlation with trends Two core judgment indicators; Step 3: Based on the preset grading threshold rules, and using two core judgment indicators as the sole basis, the real-time standardized time series data is divided into three control levels: strong effective trend data, critical precursor signal data, and invalid noise data, and the corresponding control level labels are marked. Step 4: Construct a dual-track isolation processing architecture adapted to the reinforcement learning prediction model, set up a long-term experience memory and a real-time prediction cache pool with complete logical isolation, physical isolation of storage partitions, and independent control of operation permissions, and clarify the access rules for data at each control level. Step 5: Based on the control level tags bound to the real-time standardized time series data, perform differentiated access control and processing operations, and match the corresponding permissions to complete parameter updates and policy fine-tuning. Step 6: Perform sliding time window backtracking verification on the critical precursor signal data in the real-time prediction cache pool, and dynamically adjust the data permissions based on the verification results; Step 7: Combine the basic prediction strategy of the long-term experience memory bank with the effective fine-tuning strategy of the real-time prediction cache pool to generate and output the time-series data prediction results, thus completing the entire closed loop. Example

[0017] The application scenario of this embodiment is daily load dispatching and forecasting for a regional power grid. The goal is to achieve time-series forecasting of hourly load for the regional power grid using the method of this invention, providing data support for power grid dispatching. The specific implementation process is as follows: Data Acquisition and Preprocessing The target scenario is a power grid in a prefecture-level city in East China. Hourly load data of the regional power grid from January 1, 2025 to March 31, 2025 is collected through the power grid SCADA system. The sampling period is 1 hour, resulting in a total of 2160 sets of raw time-series data, which are used as a historical valid time-series dataset. Hourly load data from April 1, 2025 to April 30, 2025, totaling 720 sets of data, were collected as the real-time test dataset. Standardization preprocessing was performed on the raw time-series data: missing values ​​were imputed using linear interpolation. In this embodiment, the original data missing rate was 0.32%, and 7 sets of missing data were imputed. The load data is mapped to the [0,1] interval using the min-max normalization method. The normalization formula is as follows: ; In the formula, This is the raw load data. The minimum load in the historical dataset. The maximum load in the historical dataset; Time series alignment and resampling are performed using a fixed sampling frequency of 1 hour, and duplicate timestamp data are removed to obtain a standardized time series dataset.

[0018] Core parameter presets In this embodiment, the preset parameters are as follows: statistical distribution matching degree weighting coefficient. =0.5、 =0.5; Trend correlation weighted coefficient =0.6、 =0.4; Grading threshold =0.85、 =0.6, =0.8、 =0.5; The temporary session period consists of 3 timing sampling periods; Sliding time window length =5, continuous verification period =3; Trend matching degree weighted coefficient =0.7、 =0.3; The upgrade threshold is 0.85, and the clear threshold is 0.4.

[0019] Statistical Feature Quantification and Data Classification Taking the real-time standardized time-series data at 00:00 on April 1, 2025 as an example, the results of comparing this data set with historical valid time-series datasets were calculated. Test statistic =0.11, after normalization divergence =0.08, substitute into the formula to calculate the statistical distribution matching degree: ; The Pearson correlation coefficient between this set of data and the historical trend series was calculated. =0.92, Granger causality test value =0.03, substitute into the formula to calculate the trend correlation: ; This set of data satisfies ≥0.85 and Data with a value ≥0.8 is classified as having a strong trend and is labeled accordingly.

[0020] Taking real-time data at 8:00 AM on April 15, 2025 as an example, the following calculations were performed: =0.72, =0.68, satisfying 0.6≤ Data with values ​​<0.85 and 0.5≤R<0.8 are classified as critical precursor signals and labeled accordingly.

[0021] Taking the real-time data at 12:00 on April 20, 2025 as an example, the calculation is as follows: =0.41, R=0.32, which does not meet the above two interval conditions, so it is classified as invalid noise data and labeled accordingly.

[0022] Dual-track isolation processing and differentiated data operations A dual-track isolated processing architecture adapted to the DQN reinforcement learning prediction model is constructed, with a long-term experience memory and a real-time prediction cache pool set up. The two units are physically isolated in storage partitions and have independent access control. For strong and effective trend data, write permissions to the long-term experience memory are granted, and the data is input into the DQN model to update the core weight parameters. The updated parameters are then mapped to the real-time prediction cache pool. For critical precursor signal data, only write permissions to the real-time prediction cache pool are granted. The data is input into the model to generate temporary fine-tuning parameters, which are then written to the cache pool. This only applies to the subsequent three prediction tasks and does not modify the long-term core parameters. For invalid and noisy data, it is directly intercepted, only the log is retained, and it is not used in any learning process.

[0023] Retrospective verification and dynamic permission adjustment For the critical precursor signal data written to the cache pool at 8:00 on April 15, 2025, a sliding time window with a length of 5 hours is opened starting from the time of writing, and subsequent real load data is continuously collected. The trend matching degree S of each sampling period is calculated. If S is ≥0.85 for 3 consecutive periods, the data is upgraded to strong effective trend data, the long-term experience memory access is opened, and the core parameters of the model are updated. For the critical precursor signal data written to the cache pool at 10:00 on April 22, 2025, if S < 0.4 for three consecutive periods, the corresponding temporary fine-tuning parameters are cleared from the cache pool to stop their impact on subsequent predictions.

[0024] Prediction results output and the entire closed loop When receiving a request for daily load forecasting of the power grid, the system calls the core parameters of the long-term experience memory to generate a basic forecasting strategy, combines them with the effective fine-tuning parameters of the real-time forecasting cache pool to generate the final forecasting strategy, outputs the power grid load time-series forecast results for the next 24 hours, and transmits them to the power grid dispatching system; and supplements the actual load data to the historical effective time-series dataset every day to complete the closed loop of the entire process.

[0025] Example Effect Verification In this embodiment, the method of the present invention has a mean absolute error (MAE) of 1.28% and a root mean square error (RMSE) of 1.67% for 24-hour power grid load prediction. The MAE volatility of the 30-day long-term forecast is 0.35%, with no significant error accumulation. For the power grid load change scenario on April 25 (load jump before May Day), the predicted peak deviation was 2.12%, the success rate of capturing precursor signals was 96.4%, and the load change trend was accurately captured. Example

[0026] The application scenario of this embodiment is the vibration status monitoring and fault early warning of asynchronous motors in industrial production workshops. The goal is to use the method of this invention to predict the trend of motor vibration acceleration time series data and identify abnormal vibration trends of the motor in advance. The specific implementation process is as follows: Data Acquisition and Preprocessing Vibration time-series data during motor operation were collected using a vibration acceleration sensor on the motor bearing housing. The sampling frequency was 1Hz. Continuous operation data from February 1, 2025 to February 28, 2025 were collected, totaling 6720 sets of data as a historical valid time-series dataset. The running data from March 1, 2025 to March 31, 2025 was collected as the real-time test dataset; the raw data was subjected to standardized preprocessing, including linear interpolation missing value imputation, min-max normalization, time series alignment and resampling, to obtain a standardized time series dataset.

[0027] Core parameter presets The preset parameters in this embodiment are as follows: =0.45、 =0.55; =0.65、 =0.35; the grading threshold is the same as in Example 1; the temporary session period is 5 sampling periods; sliding time window =6, =3; =0.75、 =0.25; the upgrade threshold and the clear threshold are the same as in Example 1.

[0028] The method execution process follows the complete process of statistical feature calculation, data classification, dual-track isolation processing, backtracking verification, and prediction output in Example 1. It is adapted to the SAC reinforcement learning prediction model to complete the statistical prediction of motor vibration time series data.

[0029] In this embodiment, the method of the present invention achieves a MAE of 1.56% and an RMSE of 1.92% for predicting motor vibration trends over 12 hours; the MAE fluctuation range for predicting over 15 consecutive days is 0.41%, with no significant error accumulation; for the scenario of sudden vibration trend changes caused by motor bearing wear, the prediction peak deviation is 2.35%, the success rate of capturing precursor signals is 94.7%, and abnormal precursor signals are captured 18 hours in advance, meeting the accuracy requirements for industrial equipment fault early warning.

[0030] Comparative Example 1 This comparative example uses the exact same power grid load dataset, DQN reinforcement learning prediction model, sampling period, and prediction duration as Example 1. It adopts the conventional reinforcement learning time series prediction method in the prior art. The specific process is as follows: perform 3σ criterion pre-filtering and outlier removal on the original data, input the preprocessed data into the reinforcement learning model in full, update the core weight parameters of the model without discrimination, and complete the time series prediction.

[0031] In the comparative study, the traditional method achieved a MAE of 2.76% and an RMSE of 3.42% for 24-hour power grid load prediction. The MAE fluctuation for 30-day long-term prediction was 1.89%, showing significant error accumulation. For the power grid load abrupt change scenario on April 25, the prediction peak deviation was 7.86%, and the success rate of capturing precursor signals was 62.8%, failing to capture the load abrupt change trend in advance.

[0032] Comparative Example 2 This comparative example uses the exact same motor vibration dataset, SAC reinforcement learning prediction model, sampling period, and prediction duration as Example 2. It adopts the conventional reinforcement learning time series prediction method in the prior art. The specific process is as follows: perform 3σ criterion pre-filtering and outlier removal on the original data, input the preprocessed data into the reinforcement learning model in its entirety, update the core weight parameters of the model without discrimination, and complete the time series prediction.

[0033] In the comparative study, the traditional method predicted the MAE of motor vibration trend over 12 hours to be 3.21% and the RMSE to be 3.85%. The MAE fluctuation range for 15 consecutive days was 2.17%, showing significant error accumulation. For the scenario of sudden vibration trend changes caused by motor bearing wear, the prediction peak deviation was 8.52%, and the success rate of capturing precursor signals was 58.3%, failing to capture abnormal vibration trends in advance.

[0034] A comprehensive comparison table of the effects of the examples and comparative examples is shown in Table 1:

[0035] Table 1 Through comprehensive comparison, it can be seen that the method of the present invention achieves a doubling improvement in prediction accuracy compared with existing traditional methods in two completely different time series prediction scenarios: power grid load and industrial motor vibration. The error accumulation problem of long time series prediction is fundamentally suppressed, and the ability to capture precursor signals in sudden change scenarios is significantly enhanced. It has stable and excellent technical effects in different application scenarios, which fully verifies the universality, inventiveness and practical value of the present invention.

[0036] While the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Any variations and modifications can be made by those skilled in the art without departing from the spirit and scope of the invention. Therefore, any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention, without departing from the scope of the invention, fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for statistical prediction of time-series data based on reinforcement learning, characterized in that, The prediction method includes the following steps: Step 1: Obtain the raw time-series data output by the entity acquisition device in the target technical scenario, perform standardized preprocessing such as missing value imputation, normalization, time-series alignment and resampling to obtain a standardized time-series dataset. The standardized time-series dataset contains the real-time standardized time-series data to be processed, and simultaneously constructs a historical valid time-series dataset verified by the actual running results in the target scenario. Step 2: For real-time standardized time-series data, quantitatively calculate the degree of matching between its statistical distribution and that of historical valid time-series datasets. Correlation with trends Two core judgment indicators; Step 3: Based on the preset grading threshold rules, and using two core judgment indicators as the sole basis, the real-time standardized time series data is divided into three control levels: strong effective trend data, critical precursor signal data, and invalid noise data, and the corresponding control level labels are marked. Step 4: Construct a dual-track isolation processing architecture adapted to the reinforcement learning prediction model, set up a long-term experience memory and a real-time prediction cache pool with complete logical isolation, physical isolation of storage partitions, and independent control of operation permissions, and clarify the access rules for data at each control level. Step 5: Based on the control level tags bound to the real-time standardized time series data, perform differentiated access control and processing operations, and match the corresponding permissions to complete parameter updates and policy fine-tuning. Step 6: Perform sliding time window backtracking verification on the critical precursor signal data in the real-time prediction cache pool, and dynamically adjust the data permissions based on the verification results; Step 7: Combine the basic prediction strategy of the long-term experience memory bank with the effective fine-tuning strategy of the real-time prediction cache pool to generate and output the time-series data prediction results, thus completing the entire closed loop.

2. The method for statistical prediction of time-series data based on reinforcement learning according to claim 1, characterized in that: In step one, the target technical scenario includes any one of industrial equipment operation status monitoring, power grid load scheduling, environmental quality monitoring, and traffic flow control. The original time series data is continuous numerical sequence data with equally spaced timestamps. A full-process standardized preprocessing operation is performed on the original time series data. First, linear interpolation is used to fill missing values ​​in the original time series data. Then, the min-max normalization method is used to map the filled time series data to the [0,1] numerical range to complete the extreme value normalization processing. Subsequently, based on a fixed sampling frequency consistent with the original time series data acquisition frequency, time series alignment and resampling operations are performed on the normalized data to remove redundant data corresponding to duplicate timestamps. Finally, a standardized time series dataset with continuous time dimension, uniform numerical range, and consistent sampling interval is obtained. The historical valid time series dataset is a collection of historical time series data that has been verified by the actual operation results of the target scenario and conforms to the real change law of the scenario time series data. The time span of the dataset is no less than 30 complete time series data sampling periods.

3. The method for statistical prediction of time-series data based on reinforcement learning according to claim 1, characterized in that: In step two, multi-dimensional statistical feature quantification calculations are performed on the real-time standardized time-series data obtained in step one to obtain the statistical distribution matching degree between the real-time standardized time-series data and the historical valid time-series dataset. Correlation with trends Two core judgment indicators; Among them, statistical distribution matching degree The calculation formula is as follows: ； in, To standardize time series data in real time and historical valid time series datasets The test statistic characterizes the difference in the cumulative probability distributions of the two groups of data, and its value ranges from [0,1]. To standardize time series data in real time relative to historical valid time series datasets Divergence, representing the difference in distributional information entropy between two sets of data, is valued based on the historical valid time-series dataset. Based on the maximum and minimum divergence values, after min-max normalization, they are fixedly mapped to the interval [0,1]. and The preset weighting coefficients satisfy... ,and , All are greater than 0, and the preset value range is [value range missing]. ; Trend correlation The calculation formula is as follows: ； in, To standardize the Pearson correlation coefficient between real-time time-series data and the corresponding trend sequences of historical valid time-series datasets, representing the degree of linear trend correlation between the two sets of data, the value range is [-1, 1]. Granger causality test on trend sequences of historical valid time series data for real-time standardization of time series data. The value represents the causal explanatory power of real-time data on historical trends, and its range is [0,1]. and The preset weighting coefficients satisfy... ,and , All are greater than 0, and the preset value range is [value range missing]. After calculating the two core judgment indicators, the indicator results are uniquely bound and stored with the corresponding real-time standardized time series data.

4. The method for statistical prediction of time-series data based on reinforcement learning according to claim 1, characterized in that: In step three, based on the preset hierarchical threshold rules, the calculated statistical distribution matching degree is... Correlation with trends Using this as the sole criterion, a three-tiered hierarchical control system is implemented for real-time standardized time-series data. Preset first distribution threshold Second distribution threshold ,satisfy Preset first trend threshold Second trend threshold ,satisfy In the hierarchical partitioning rules, when real-time standardized time-series data simultaneously meet the following conditions... ≥ and ≥ When real-time standardized time series data simultaneously meet the following conditions, it is classified as strongly effective trend data. When the real-time standardized time series data does not meet any of the above two judgment conditions, it is classified as invalid noise data. After the classification is completed, a unique control level label is marked for each group of real-time standardized time series data. The label is bound to the data one by one, and there is no overlap or omission in the classification intervals.

5. The method for statistical prediction of time-series data based on reinforcement learning according to claim 1, characterized in that: In step four, the dual-track isolation processing architecture comprises two core processing units: a long-term experience memory (LTM) and a real-time prediction cache. The LTM serves as the sole permanent storage unit for the core parameters of the reinforcement learning prediction model, receiving only parameter update operations triggered by data meeting preset permission requirements. Data without permission cannot be written to or modified. The real-time prediction cache serves as the temporary policy parameter storage unit for the reinforcement learning prediction model, storing temporary fine-tuning parameters that only affect single or short-term predictions. Its storage partition is completely physically isolated from the LTM. All parameters written to the real-time prediction cache are only effective within a preset temporary session period and cannot be directly written to or modified in any form. The preset temporary session period is 1-5 time-series sampling periods. Unique permission control rules are defined for both processing units: only strongly valid trend data has write permission to the LTM, critical precursor signal data only has write permission to the real-time prediction cache, and invalid noise data has no write permission to either unit.

6. The method for statistical prediction of time-series data based on reinforcement learning according to claim 1, characterized in that: In step five, based on the control level label uniquely bound to the real-time standardized time series data marked in step three, differentiated access and processing operations are performed on the real-time standardized time series data bound to the corresponding label. Specifically, for real-time standardized time series data marked as strong effective trend data, access to its long-term experience memory is granted. This data is then input into the reinforcement learning prediction model, updating the core weight parameters of the model stored in the long-term experience memory. Simultaneously, the updated core parameters are synchronously mapped to the real-time prediction cache pool as the basic strategy for short-term prediction. For real-time standardized data marked as critical precursor signal data… For time-series data, access is granted only to its real-time prediction cache pool. This set of data is input into the reinforcement learning prediction model and is used only to generate temporary fine-tuning parameters adapted to the current data trend. These parameters are then written to the real-time prediction cache pool. No parameter modification operations are triggered in the long-term experience memory throughout the process. The temporary fine-tuning parameters only take effect for no more than five subsequent time-series prediction tasks. For real-time standardized time-series data marked as invalid noise data, an interception operation is directly performed. Only the original information and classification results of this set of data are logged. It is not input into any processing unit of the reinforcement learning prediction model and does not trigger any parameter update or strategy adjustment operations.

7. The method for statistical prediction of time-series data based on reinforcement learning according to claim 1, characterized in that: In step six, for the critical precursor signal data written to the real-time prediction cache pool, a sliding time window backtracking verification and dynamic permission adjustment operation are performed. First, a preset length of... A sliding time window for each time-series sampling period The time frame is a positive integer ≥3 and ≤10. Starting from the moment when the critical precursor signal data is written into the real-time prediction buffer, the real time-series data of the subsequent output is continuously collected within the sliding time window. The trend matching degree between the critical precursor signal data and the real time-series data within the sliding time window is calculated. Trend matching degree The calculation formula is as follows: ； in, The Pearson correlation coefficient is the sum of the critical precursor signal data and the actual time series data within the sliding time window, with a value range of [-1, 1]. This is the trend direction consistency coefficient. A value of 1 is assigned when the increasing / decreasing trends of two sets of data are completely consistent in each sampling period within the sliding time window. A value of 0.5 is assigned when the increasing / decreasing trends of two sets of data are consistent in more than 50% but less than 100% of the sampling periods within the sliding time window. A value of 0 is assigned when the increasing / decreasing trends of two sets of data are consistent in 50% or less of the sampling periods within the sliding time window. A value of 0 is assigned when the trends are completely opposite. and The preset weighting coefficients satisfy... ,and , All are greater than 0, and the preset value range is [value range missing]. When the trend matches continuous When all sampling periods are ≥ the preset upgrade threshold of 0.85, the critical precursor signal data is upgraded to strongly effective trend data, its long-term empirical memory is granted access, and the core weight parameters of the model are updated. When the trend matching degree... continuous When all sampling periods are less than the preset clearing threshold of 0.4, the temporary fine-tuning parameters corresponding to this set of critical precursor signal data are automatically expired and cleared from the real-time prediction buffer, stopping their impact on the strategy of subsequent prediction tasks. ≥2 and less than The positive integers, where the upgrade threshold and the clearing threshold are both fixed values, satisfying 1 > upgrade threshold > clearing threshold > 0.

8. The method for statistical prediction of time-series data based on reinforcement learning according to claim 1, characterized in that: In step seven, when receiving a time-series prediction task request for the target scenario, the core weight parameters of the model stored in the long-term experience memory are called to generate a basic prediction strategy. At the same time, the effective temporary fine-tuning parameters stored in the real-time prediction cache pool are called to adapt and adjust the basic prediction strategy to generate the final time-series prediction strategy. Based on the final time-series prediction strategy, the statistical prediction calculation of the target time-series data is completed, and a time-series data prediction result sequence with corresponding timestamps is generated. The prediction result sequence is output to the technical execution system corresponding to the target scenario. At the same time, the real feedback time-series data corresponding to this prediction task is added to the historical effective time-series dataset to complete the entire closed loop of the prediction method.

Images