Mixing multi-interpolation method and system for missing data in big data analysis
A technology of missing data and multiple interpolation, applied in neural learning methods, electrical digital data processing, digital data information retrieval, etc., can solve problems such as overfitting bias, improve accuracy, avoid overfitting and bias estimation, and improve The effect of variability
Pending Publication Date: 2022-06-03
CHONGQING UNIV OF EDUCATION
View PDF0 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0005] The purpose of the present invention is to provide a missing data mixed multiple interpolation method and system in big data analysis to solve the technical problems of overfitting and biased estimation in the prior art
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0066] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work, all belong to the protection scope of the present invention.
[0067] like figure 1 As shown, the present invention provides a mixed multiple interpolation method for missing data in big data analysis, comprising the following steps:
[0068] Step S1, take the missing data as the interpolation center, use multiple sets of non-missing data located in the same horizontal and vertical direction as the interpolation center in the data matrix where the missing data is located as the training data for the m...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a missing data mixed multi-interpolation method and system in big data analysis, and the method comprises the following steps: S1, taking missing data as an interpolation center, taking multiple groups of non-missing data located in the same transverse direction and the same vertical direction with the interpolation center in a data matrix where the missing data is located as multiple groups of interpolation model training data, respectively setting interpolation directions of the plurality of groups of interpolation model training data according to the direction of the interpolation center of the plurality of groups of interpolation model training data; and S2, sorting the plurality of groups of interpolation model training data according to the interpolation direction to obtain a plurality of groups of interpolation model training sequences, and performing model training based on the plurality of groups of interpolation model training sequences by using a neural network model to obtain a plurality of groups of interpolation models. According to the method, multi-direction interpolation is carried out, a plurality of missing results are estimated through modeling, so that weighted interpolation is carried out, the variability of missing data is improved, the optimal interpolation result is obtained through weighting of interpolation values in multiple directions, and over-fitting and deviation estimation are avoided.
Description
technical field [0001] The invention relates to the technical field of data interpolation, in particular to a method and system for mixed multiple interpolation of missing data in big data analysis. Background technique [0002] At present, people have entered a new era of big data. Big data has an important influence in more and more fields, and at the same time, the requirements for data quality are getting higher and higher. As the amount of data continues to increase, the problems of data quality and data availability are also increasing, such as data missing and data redundancy. Analysis will introduce impact and error. How to solve missing data and eliminate the impact of missing data on big data analysis is a crucial issue to ensure the quality of original data under big data analysis. [0003] The problem that cannot be ignored in big data is the existence of incomplete data, which is called missing data. There are three missing mechanisms for missing data, missin...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/215G06N3/04G06N3/08
CPCG06F16/215G06N3/04G06N3/08G06N3/084Y02D10/00
Inventor 施成湘
Owner CHONGQING UNIV OF EDUCATION