Big data prediction analysis method, system and device and storage medium

Abstract

The invention discloses a big data prediction analysis method, system and device and a storage medium. The method comprises the steps of collecting data to obtain a data set; generating a corresponding rule by using a rule fitting algorithm; performing regularization processing on the original attributes of the data in the data set; generating a prediction model according to the corresponding rules and the original attributes; calculating parameters of the prediction model; obtaining a weight value set of the prediction model according to the parameters; and calculating to obtain a corresponding prediction analysis result according to the weight value set. According to the invention, a unified and effective prediction analysis method is provided for each industry analysis data set according to the characteristics of the industry data set; relationships and rules existing in data are discovered through a big data analysis method, and the future development trend of things is predicted,so that the scientificity of decision making can be improved; meanwhile, enterprises can be helped to analyze future data information, and risks are effectively avoided. The method is widely applied to the technical field of data mining.

Description

technical field [0001] The present invention relates to the technical field of data mining, in particular to a method, system, device and storage medium for predictive analysis of big data. Background technique [0002] Predictive analytics is a common method in data mining where the goal is to predict unknown values ​​using known attributes. Predictive analysis can also evaluate the influence of certain attributes in the data set on the changes of certain numerical attributes. Generally, the influence is evaluated by predicting the change trend of the target numerical attribute and determining the weighting factors of other attributes on the change trend. [0003] At present, there are mainly two types of methods for predictive analysis of industry big data, one is a classic research method that follows the combination of expert knowledge and statistics, and the other is a research method based on machine learning. [0004] For the classical approach, the variables for eac...

AI Technical Summary

Problems solved by technology

Classical methods have some inherent defects that need to be resolved: (1) the research of classical methods is time-consuming and requires the efforts of many experts; (2) because data sets generally contain a large number of interrelated and interacting data attributes, traditional statistical algorithms Not smart enough to fully mine the correlation and interaction between data attributes, so the predicted results may be simple and limited; (3) If the data comes from a questionnaire survey, it means that the survey form is designed by the surveyor himself , so the attributes of the data in the dataset may not be sufficient for researchers to create models based on their knowledge and inferences

[0005] For machine learning methods, most models (such as neural networks) cannot be explained. For interpretable models (such as decision trees), the analysis accuracy is generally not guaranteed. In addition, extracting patterns (knowledge we need) from the model needs to be done manually. , which requires the efforts of many professionals

For example, the linear regression algorithm assumes that the predicted value y and the attribute x=(x 1 ,x 2 …x n ) is linear, the predicted value y can be calculated as y=a 0 +a 1 x 1 +a 2 x 2 +…+a n x n , where a 0 is the error term for all other factors affecting the variable y except attribute x, however the linear regression model does not consider the interaction between features

Images