Continuous large-scale water quality missing data filling method based on transfer learning
A transfer learning and missing data technology, applied in neural learning methods, electrical digital data processing, special data processing applications, etc., can solve problems such as the inability to fill in large-scale continuous water quality missing data, and achieve the effect of improving the filling accuracy.
Inactive Publication Date: 2021-05-07
HANGZHOU DIANZI UNIV
View PDF5 Cites 1 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0006] The present invention provides a large-scale continuous water quality missing data filling method based on migration learning for existing technologies that cannot fill in large-scale continuous water quality missing data
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0026] Depend on figure 1 As shown, the missing data filling method framework proposed by the patent of the present invention can be divided into two parts: data preprocessing and filling algorithm execution.
[0027] In the process of data preprocessing, firstly, the incomplete data sequence collected from a water quality monitoring station sensor is cleaned, standardized and defined as experimental data. Secondly, use the method of time series similar query (in the invention, use the dynamic time warping algorithm (DTW)) to find out the data of the monitoring station most similar to the incomplete data sequence and set it as the reference data. Finally, the training and testing samples are constructed using the sliding window algorithm (Sliding Window).
[0028] During the execution of the filling algorithm, the present invention proposes a new filling algorithm TrAdaBoost-LSTM, which combines an example-based migration learning algorithm: TrAdaBoost and an advanced deep le...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention relates to a continuous large-scale water quality missing data filling method based on transfer learning. The method comprises the following steps: firstly, preprocessing data, and constructing training and testing samples by using a sliding window algorithm; filling data, specifically, fusing the training samples of the target domain and the training samples of the source domain into a new mixed training sample set; in each iteration, creating a new weak learner for filling data; calculating an average prediction filling error on the newly mixed training sample; respectively calculating weight iteration update coefficients of the training sample of the source domain and the training sample of the target domain; updating new weights of the training samples of the source domain and the target domain at the next moment; and carrying out weighted average on the output values of all the weak learners to obtain a final prediction filling value of a strong learner. Filling accuracy is improved by about 15%-25% in the process of processing large-scale continuous missing data.
Description
technical field [0001] The invention relates to a method for filling missing data of water quality, in particular to a method for filling missing data of large-scale continuous water quality based on migration learning. Background technique [0002] With the rapid development of industrialization and urbanization, water resource protection and water pollution control have become the hottest and most worrying hot topics in the world. In order to control water pollution and reduce its adverse effects on water ecosystems and human society, a large number of researchers have done a lot of work (including spatiotemporal prediction of water quality, assessment of water quality pollutants’ impact factors and data-driven water quality models, etc.) to improve the quality of small watersheds. Water quality monitoring level. [0003] When conducting these studies, valid and high-quality water quality datasets are an important prerequisite for producing plausible and reliable research...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/215G06N3/04G06N3/08
CPCG06N3/049G06N3/08G06F16/215
Inventor 蒋鹏陈锃许欢刘俊林广
Owner HANGZHOU DIANZI UNIV