Economic trend prediction system and method based on big data statistical analysis

Abstract

The invention belongs to the technical field of economic trend prediction, and provides an economic trend prediction system and method based on big data statistical analysis, and the system comprises a diversified data integration module; an intelligent data preprocessing module; a dynamic feature extraction module; an integrated prediction model module; a visual result display module; the system can integrate economic related data from a plurality of data sources in real time or regularly, and perform efficient and accurate data preprocessing and feature extraction, thereby performing economic trend prediction by using an advanced prediction model; finally, through a visual result display module, a prediction result and uncertainty analysis are displayed in a visual and understandable mode, powerful decision support is provided for decision makers and researchers, the accuracy and efficiency of economic trend prediction are remarkably improved, economic development trends can be better grasped, and the economic development efficiency is improved. And a scientific basis is provided for policy making and economic planning.

Description

Technical Field

[0001] The present invention belongs to the technical field of economic trend forecasting, and specifically relates to an economic trend forecasting system and method based on big data statistical analysis. Background Art

[0002] In the economic field, accurate prediction of economic trends has always been the focus of decision makers and researchers. Traditional economic trend prediction methods mainly rely on economic theories and historical data, and make predictions by building mathematical models.

[0003] However, with the advent of the big data era, economic data has shown explosive growth, and data types and sources have become more diversified. Currently, traditional forecasting methods have many limitations when processing big data, such as slow processing speed, low forecasting accuracy, and difficulty in capturing dynamic changes in economic data.

[0004] To this end, those skilled in the art have proposed an economic trend forecasting system and method based on big data statistical analysis to solve the problems raised by the background technology. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides an economic trend forecasting system and method based on big data statistical analysis, so as to solve the many limitations of the forecasting methods in the prior art when processing big data, such as slow processing speed, low forecasting accuracy, and difficulty in capturing dynamic changes in economic data.

[0006] An economic trend forecasting system and method based on big data statistical analysis, comprising:

[0007] A diversified data integration module for real-time or periodic integration of economic data from macroeconomic, microeconomic, financial market, social media and industry report data sources;

[0008] Intelligent data preprocessing module, used for anomaly detection, missing value filling, data standardization and noise reduction of integrated data;

[0009] Dynamic feature extraction module, which automatically extracts key features that affect economic trends based on historical change patterns and correlation analysis of economic indicators;

[0010] The integrated forecasting model module combines the fusion algorithm of the long short-term memory network (LSTM) and the attention mechanism to build an economic trend forecasting framework with multi-model fusion to improve the overall forecasting performance;

[0011] The visualization result display module intuitively displays the prediction results and uncertainty analysis in the form of charts, reports, etc.

[0012] Preferably, the intelligent data preprocessing module further includes an adaptive algorithm for dynamically adjusting the preprocessing strategy according to data characteristics and changes.

[0013] Preferably, the dynamic feature extraction module adopts a feature selection algorithm to select the most representative features, reduce feature dimensions, improve model generalization ability, and thus improve the accuracy of the prediction model.

[0014] Preferably, in the integrated prediction model module, the formula of the fusion algorithm of the long short-term memory network (LSTM) and the attention mechanism is as follows:

[0015]

[0016] Among them, e t is the attention score, a t is the attention weight and T is the time step.

[0017] Preferably, the integrated prediction model module also includes a model evaluation and selection submodule, which is used to evaluate the performance of each prediction model based on historical data and dynamically select the optimal model combination for prediction, that is, to predict by using a Bayesian optimization algorithm to efficiently search the model parameter space and find the optimal parameter combination.

[0018] Preferably, in the visualization result display module, the intuitive display of prediction results and uncertainty analysis adopts a visualization interpretation algorithm to provide an intuitive explanation of the prediction results and enhance the interpretability and credibility of the model.

[0019] An economic trend forecasting method based on big data statistical analysis, using the above system, includes:

[0020] Integrate and preprocess multi-dimensional economic data;

[0021] Dynamically extract key economic features;

[0022] Build and apply integrated forecasting models to predict economic trends;

[0023] Visualize the prediction results.

[0024] A processor is configured to execute the above-mentioned economic trend forecasting system based on big data statistical analysis.

[0025] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned economic trend forecasting system based on big data statistical analysis.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] 1. The present invention can integrate economic-related data from multiple data sources such as macroeconomics, microeconomics, financial markets, social media and industry reports in real time or regularly by constructing a diversified data integration module, thereby achieving the comprehensiveness and diversity of data, helping to capture the dynamic changes and potential trends of economic data and improve the accuracy of predictions.

[0028] 2. The present invention adopts an intelligent data preprocessing module to perform anomaly detection, missing value filling, data standardization and noise reduction on the integrated data, which effectively improves the quality and availability of the data; at the same time, the application of adaptive algorithms can dynamically adjust the preprocessing strategy according to data characteristics and changes, further enhancing the flexibility and accuracy of data processing.

[0029] 3. The present invention uses a dynamic feature extraction module to automatically extract key features that affect economic trends based on historical change patterns and correlation analysis of economic indicators, which helps to reduce feature dimensions, improve model generalization capabilities, and thus improve the accuracy of the prediction model; at the same time, the application of feature selection algorithms can select the most representative features, further enhancing the prediction performance of the model.

[0030] 4. The present invention adopts an integrated prediction model module, combines the fusion algorithm of long short-term memory network (LSTM) and attention mechanism, and constructs a multi-model fusion economic trend prediction framework, which helps to improve the overall prediction performance and accurately capture the changing laws of economic trends; at the same time, the application of model evaluation and selection submodules can evaluate the performance of each prediction model based on historical data, and dynamically select the optimal model combination for prediction, further improving the accuracy and stability of the prediction.

[0031] 5. The present invention uses a visualization result display module to intuitively display the prediction results and uncertainty analysis in the form of charts, reports, etc., which helps decision makers and researchers to better understand the prediction results, evaluate the uncertainty of the prediction, and make more informed decisions; at the same time, the application of the visualization explanation algorithm can provide an intuitive explanation of the prediction results and enhance the interpretability and credibility of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a framework diagram of the economic trend forecasting system based on big data statistical analysis of the present invention. DETAILED DESCRIPTION

[0033] The following embodiments of the present invention are described in further detail in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0034] Embodiment: The present invention provides an economic trend forecasting system based on big data statistical analysis, such as Figure 1As shown, it includes a diversified data integration module, an intelligent data preprocessing module, a dynamic feature extraction module, an integrated prediction model module and a visualization result display module, and the diversified data integration module, the intelligent data preprocessing module, the dynamic feature extraction module, the integrated prediction model module and the visualization result display module are electrically connected in sequence:

[0035] A diversified data integration module for real-time or periodic integration of economic data from macroeconomic, microeconomic, financial market, social media and industry report data sources;

[0036] Intelligent data preprocessing module, used for anomaly detection, missing value filling, data standardization and noise reduction of integrated data;

[0037] Dynamic feature extraction module, which automatically extracts key features that affect economic trends based on historical change patterns and correlation analysis of economic indicators;

[0038] The integrated forecasting model module combines the fusion algorithm of the long short-term memory network (LSTM) and the attention mechanism to build an economic trend forecasting framework with multi-model fusion to improve the overall forecasting performance;

[0039] The visualization result display module intuitively displays the prediction results and uncertainty analysis in the form of charts, reports, etc.

[0040] From the above, it can be seen that through the diversified data integration modules, intelligent data preprocessing modules, dynamic feature extraction modules, integrated prediction model modules and visual result display modules that are electrically connected in sequence, the whole chain of intelligent processing from data integration to result display is realized; the system can integrate economic-related data from multiple data sources in real time or regularly, and perform efficient and accurate data preprocessing and feature extraction, and then use advanced prediction models to predict economic trends; finally, through the visual result display module, the prediction results and uncertainty analysis are displayed in an intuitive and easy-to-understand way, providing strong decision-making support for decision makers and researchers. The system has significantly improved the accuracy and efficiency of economic trend forecasting, helped to better grasp the dynamics of economic development, and provided a scientific basis for policy making and economic planning.

[0041] Furthermore, the intelligent data preprocessing module further includes an adaptive algorithm for dynamically adjusting the preprocessing strategy according to data characteristics and changes. The formula of the adaptive algorithm is as follows:

[0042]

[0043] Among them, y i is the response variable, x i is the feature vector, βis a coefficient vector, and λ is a regularization parameter; by adjusting λ, features can be selected adaptively.

[0044] From the above, we can see that the adaptive algorithm can dynamically adjust the preprocessing strategy according to the characteristics and changes of the data, and adaptively select features by adjusting the regularization parameter λ; this dynamic adjustment capability not only improves the efficiency and flexibility of data preprocessing, but also effectively enhances the quality and availability of the data, providing a more accurate and reliable data basis for subsequent economic trend forecasts.

[0045] Furthermore, the dynamic feature extraction module adopts a feature selection algorithm to select the most representative features, reduce feature dimensions, improve model generalization ability, and thus improve the accuracy of the prediction model. The formula of the feature selection algorithm is as follows:

[0046]

[0047] Among them, X is the original feature matrix, W is the feature selection matrix, ||·|| F represents the Frobenius norm, ||·||1 represents the L1 norm, and λ is the regularization parameter.

[0048] From the above, we can see that the feature selection algorithm used in the dynamic feature extraction module can effectively select the most representative features from the original feature matrix. By introducing regularization parameters and balancing the influence of the Frobenius norm and the L1 norm, the feature dimension can be reduced. This not only improves the generalization ability of the prediction model and avoids overfitting, but also significantly improves the accuracy of the prediction model, making the prediction of economic trends more accurate and reliable.

[0049] Furthermore, in the integrated prediction model module, the formula of the fusion algorithm of the long short-term memory network (LSTM) and the attention mechanism is as follows:

[0050]

[0051] Among them, e t is the attention score, a t is the attention weight and T is the time step.

[0052] From the above, we can see that in the integrated forecasting model module, the fusion algorithm of the long short-term memory network (LSTM) and the attention mechanism can dynamically focus on important parts of economic data at different time steps by calculating attention scores and attention weights; this fusion strategy not only improves the model's ability to process time series data, but also effectively captures long-term dependencies and key change points in economic trends, thereby significantly improving the overall performance and accuracy of economic trend forecasting.

[0053] Furthermore, the integrated prediction model module also includes a model evaluation and selection submodule, which is used to evaluate the performance of each prediction model based on historical data and dynamically select the optimal model combination for prediction, that is, to predict by using the Bayesian optimization algorithm to efficiently search the model parameter space and find the optimal parameter combination. The formula of the Bayesian optimization algorithm is as follows:

[0054] P(yx)=N(m(x),k(x,x′));

[0055] Among them, m(x) is the mean function and k(x,x′) is the kernel function.

[0056] From the above, we can see that the Bayesian optimization algorithm can be used to efficiently search the model parameter space and evaluate the performance of each prediction model based on historical data, so as to dynamically select the optimal model combination for prediction; it not only greatly improves the accuracy and efficiency of model selection, but also helps to discover prediction model combinations with better performance, further enhancing the accuracy and stability of economic trend predictions; the introduction of the Bayesian optimization algorithm makes the model parameter tuning process more scientific and efficient, providing strong technical support for economic trend predictions.

[0057] Furthermore, in the visualization result display module, the intuitive display of prediction results and uncertainty analysis adopts a visualization explanation algorithm to provide an intuitive explanation of the prediction results and enhance the interpretability and credibility of the model. The formula of the visualization explanation algorithm is as follows:

[0058]

[0059] Among them, φ i (x) is feature x i , f(S) is the predicted value of the model on subset S.

[0060] From the above, we can see that by using a visual explanation algorithm to intuitively display the prediction results and uncertainty analysis, and using the SHAP value of the feature and the prediction value of the model on the subset, a clear and intuitive explanation of the prediction results is provided; this method not only significantly enhances the interpretability of the model, allowing decision makers and researchers to have a deeper understanding of the basis and logic of the prediction results, but also further improves the credibility of the prediction results; through the visual explanation algorithm, the prediction results of economic trends become more transparent and easy to understand, providing more reliable and powerful support for economic decision-making.

[0061] An economic trend forecasting method based on big data statistical analysis, using the above system, includes:

[0062] Integrate and preprocess multi-dimensional economic data;

[0063] Dynamically extract key economic features;

[0064] Build and apply integrated forecasting models to predict economic trends;

[0065] Visualize the prediction results.

[0066] Working principle: Integrate multi-dimensional economic data by building a diversified data integration module, use the intelligent data preprocessing module to clean and standardize the data, and then extract key economic features through the dynamic feature extraction module. Then, apply the fusion algorithm of the long short-term memory network (LSTM) and the attention mechanism in the integrated prediction model module to predict economic trends, and select the optimal model combination through the model evaluation and selection sub-module. Finally, use the visualization result display module to intuitively display the prediction results and uncertainty analysis in the form of charts, reports, etc. to provide support for economic decision-making.

[0067] Furthermore, the economic trend forecasting system based on big data statistical analysis of the embodiment is compared with the current traditional forecasting method (comparative example), and the following table is obtained:

[0068]

[0069]

[0070] It can be seen from the above table that the economic trend forecasting system based on big data statistical analysis is superior to traditional forecasting methods in terms of data processing capability, data preprocessing effect, feature extraction capability, forecasting model performance, result presentation method and decision support capability.

[0071] The embodiment of the present application provides an electronic device, which is applicable to the above-mentioned economic trend prediction system based on big data statistical analysis, including:

[0072] Memory, used to protect computer programs and data;

[0073] Processor, used to run system programs.

[0074] An embodiment of the present application provides a computer storage medium, which is applicable to the above-mentioned economic trend forecasting system based on big data statistical analysis, and performs hierarchical confidentiality management on the above-mentioned system and data in accordance with confidentiality management requirements.

[0075] Those skilled in the art will appreciate that the embodiments of the present application may be provided as a system or a computer program product. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0076] The present application is described with reference to the flowcharts and / or block diagrams of the devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0077] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0078] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0079] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0080] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0081] Computer readable media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0082] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, commodity or device including the elements.

[0083] The embodiments of the present invention are provided for the purpose of illustration and description. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations of the present invention. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. An economic trend forecasting system based on big data statistical analysis, characterized by: include: A diversified data integration module for real-time or periodic integration of economic data from macroeconomic, microeconomic, financial market, social media and industry report data sources; Intelligent data preprocessing module, used for anomaly detection, missing value filling, data standardization and noise reduction of integrated data; Dynamic feature extraction module, which automatically extracts key features that affect economic trends based on historical change patterns and correlation analysis of economic indicators; The integrated forecasting model module combines the fusion algorithm of the long short-term memory network (LSTM) and the attention mechanism to build an economic trend forecasting framework that integrates multiple models; The visualization result display module intuitively displays the prediction results and uncertainty analysis in the form of charts, reports, etc.

2. The economic trend forecasting system and method based on big data statistical analysis as claimed in claim 1, characterized in that: The intelligent data preprocessing module further includes an adaptive algorithm for dynamically adjusting the preprocessing strategy according to data characteristics and changes.

3. The economic trend forecasting system and method based on big data statistical analysis as claimed in claim 1, characterized in that: The dynamic feature extraction module adopts a feature selection algorithm to select the most representative features and reduce the feature dimension.

4. The economic trend forecasting system and method based on big data statistical analysis as claimed in claim 1, characterized in that: In the integrated prediction model module, the formula of the fusion algorithm of the long short-term memory network and the attention mechanism is as follows: Among them, e t is the attention score, a t is the attention weight and T is the time step.

5. The economic trend forecasting system and method based on big data statistical analysis as claimed in claim 1, characterized in that: The integrated prediction model module also includes a model evaluation and selection submodule, which is used to evaluate the performance of each prediction model based on historical data and dynamically select the optimal model combination for prediction, that is, to perform prediction by adopting a Bayesian optimization algorithm.

6. The economic trend forecasting system and method based on big data statistical analysis as claimed in claim 1, characterized in that: In the visualization result display module, the intuitive display of prediction results and uncertainty analysis adopts a visualization explanation algorithm to provide an intuitive explanation of the prediction results and enhance the interpretability and credibility of the model.

7. An economic trend forecasting method based on big data statistical analysis, using the system as described in any one of claims 1 to 6, characterized in that: include: Integrate and preprocess multi-dimensional economic data; Dynamically extract key economic features; Build and apply integrated forecasting models to predict economic trends; Visualize the prediction results.

8. A processor, characterized in that: The invention is configured to execute an economic trend forecasting system based on big data statistical analysis according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, an economic trend forecasting system based on big data statistical analysis as described in any one of claims 1 to 6 is implemented.

Images