Interpolation method for multi-dimensional time series monitoring data of concrete condition

Abstract

To provide an interpolation method for multi-dimensional time series monitoring data of concrete condition.SOLUTION: A method includes steps of: performing a Box-Cox transformation on collected multi-dimensional raw data; constructing a matrix Dn,m with multi-dimensional time series data after the Box-Cox transformation, and performing a reconstruction operation after cutting columns of the matrix Dn,m, so that new matrix after the reconstruction is Pn,m*n'; predicting missing values in the reconstructed matrix Pn,m*n', complementing the missing values, and extracting a matrix Dn,m from the complemented matrix; and performing an inverse transformation on the data in the extracted matrix Dn,m. The present invention performs a Box-Cox transformation on the raw data so that it follows a normal distribution, organizes the converted data into a sequence matrix, reconstructs the matrix by dividing the columns of the matrix, and fills in the missing values of the reconstructed matrix by using a multiple complementation method based on chain equations, thereby realizing the complementation of the multi-dimensional time series monitoring data.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to the field of water utilization and hydroelectric power generation, and more particularly to a method for complementing multi-dimensional time-series monitoring data of concrete condition. [Background technology]

[0002] Concrete, as a very important building material, has a very wide range of applications. In high-rise buildings, concrete plays the role of supporting the ground. In the water conservancy field, with the development and utilization of water resources, concrete dams are built all over China, and whether concrete dams can operate normally is directly related to the security of water conservancy projects and affects the safety of people's lives and property. With the increase in the construction time of concrete building materials, performing reliability analysis on concrete monitoring data contributes to understanding the normal state of concrete, thereby ensuring the stable and safe operation of buildings. However, due to the deterioration of the monitoring system, transmission and other failure problems, the loss of a large amount of monitoring data has become one of the challenges in the management of current concrete dams. The loss of monitoring data reduces the accuracy of concrete condition evaluation. Therefore, recovering the lost multi-dimensional time series monitoring data has become an important challenge in the analysis and management of concrete buildings.

[0003] The key to data imputation is to establish data dependency between missing data and collected data. A regression machine learning model may predict missing values ​​by learning the relationship between the missing values ​​and other variables. However, when there are missing data in all of multiple variables, directly applying the regression model cannot solve the multivariate data imputation problem.

[0004] Therefore, a new technical solution is needed to solve this problem. Summary of the Invention [Problem to be solved by the invention]

[0005] In order to overcome the shortcomings of the prior art, a method for complementing multi-dimensional time series monitoring data of concrete condition is provided, which mainly involves performing a Box-Cox transformation on the raw data so that it follows a normal distribution, organizing the transformed data into a sequence matrix, reconstructing the matrix by dividing the columns of the matrix, and filling in the missing values ​​of the reconstructed matrix using multiple complementation by chained equations to realize the complementation of multi-dimensional time series monitoring data. [Means for solving the problem]

[0006] In order to achieve the above object, the present invention provides Step S1 of performing Box-Cox transformation on the multi-dimensional raw data collected by the concrete automation monitoring system so that the data follows a normal distribution; Matrix D for multivariate time series data after Box-Cox transformation n,m In order to enhance the interdependence between the multi-dimensional data by using the time information, the matrix D n,m Then, we cut the columns of and perform the reconstruction operation, and the new matrix after the reconstruction is called P n,m*n′ Step S2: The reconstructed matrix P n,m*n′ Predict the missing values ​​in , impute the missing values, and convert the imputed matrix to matrix D. n,m Step S3 of extracting Extracted matrix D n,m and step S4 of performing inverse transformation on the data in order to bring the data and the observation data into the same order of magnitude, thereby finally achieving the completion of the multi-dimensional time series data.

[0007] Furthermore, the multi-dimensional raw data in step S1 includes temperature, pH value, resistivity, chloride ion concentration, calcium ion concentration, and osmotic flow.

[0008] Furthermore, the specific process of step S1 is as follows: Step S1-1: collecting multi-dimensional raw data including a total of six variables, namely, temperature, pH value, resistivity, chloride ion concentration, calcium ion concentration, and seepage flow, from a concrete automation monitoring system, where the system measurement frequency is 15 minutes; Step S1-2 performs a Box-Cox transformation on the collected multi-dimensional raw data so that the data follows a normal distribution, thereby ensuring the regression characteristics of the time series dimension of the variables, and the formula is:

number

number

number

number

[0009] Furthermore, the specific process of step S2 is as follows: Matrix D for multivariate time series data after Box-Cox transformation n,m each row of the matrix represents a data record on a time stamp, n,m teeth,

number

number

number

number

[0010] Furthermore, the method for solving n in step S2-2 is as follows: Whether or not all elements in a row can be missing is determined by the matrix D n,m Observe each row of A1 and A2, which, if it exists, establishes a set S that contains all the divisors of n whose corresponding quotients are greater than or equal to column m of the original matrix; If it does not exist, A3 executes A4 as is. n′ is an element in the set S, and the length of the new sequence n / n′ is the length of the matrix D n,m is greater than the length of the sequence of n, i.e., n / n′>m, and n′ is

number

number

[0011] Further, the specific steps of step S3 are as follows: Step S3-1 of predicting missing values ​​in the reconstructed matrix P using a multiple imputation method based on a chain of equations and imputing the missing values; After the missing values ​​in the new matrix P are imputed, the imputed matrix is ​​converted to the matrix D. n,m and step S3-2 of extracting the

[0012] Furthermore, the specific steps of the chained equation algorithm in step S3-1 are as follows: a step B1 of estimating missing values ​​in a matrix; Step B2: select one variable in turn and delete its estimate, use the deleted estimate variable as the dependent variable of the imputation model, and use the remaining variables as independent variables of the model, and select an appropriate imputation model based on the characteristics of the dependent variable to predict the missing value; Step B3 of filling the data columns of the matrix and replacing the previous estimates with the imputed results; step B4 of repeating steps B2-B3 for each variable containing missing values ​​until all missing values ​​have been replaced; and step B5 of training the imputation model multiple times until it converges.

[0013] Furthermore, the method for estimating missing values ​​in the matrix in step B1 is to replace missing values ​​in variables with the mean value of the collected data.

[0014] Furthermore, the method of selecting the imputation model in step B2 is to select an appropriate imputation model based on the data characteristics of the variables of the selected column, and if the relationship between the data is linear, select a linear regression model and estimate the missing values ​​by fitting a linear function; if the variable data is continuous or discrete data, select a K-nearest neighbor model and estimate the missing values ​​using nearest observed values ​​based on the similarity between the data.

[0015] Furthermore, the specific steps of step S4 are as follows: Extracted matrix D n,m Step S4-1 performs an inverse Box-Cox transformation on the data in

number

[0016] The chained equation method uses a series of regression models for multiple imputation and allows the previous estimated values ​​to be used to predict subsequent variables, which can solve the multivariate data imputation problem, but the chained equation is not designed for time series data imputation, but is specifically designed for general data imputation. In the present invention, in order to strengthen the time association between data in the chained equation method, the matrix is ​​reconstructed to strengthen the time recurring patterns in the time series data and improve the association between missing values ​​and observed values. Effect of the Invention

[0017] Compared with the prior art, the present invention has the following advantages:

[0018] 1. By using the Box-Cox transformation to convert the collected raw data into normally distributed variables, the accuracy of estimating missing values ​​by the imputation model used in the chained equations can be improved.

[0019] 2. Since the chained equation does not consider the time linkage between data, the matrix is ​​reconstructed before imputation to enhance the time repetition pattern of the time series data, thereby improving the association between missing values ​​and observed values ​​and improving the imputation accuracy.

[0020] 3. When variable data of the concrete automated monitoring instrument is lost within a certain period due to a failure problem, in the process of reconstructing the matrix, the matrix is ​​divided according to columns to ensure that each row has an observed value of the corresponding variable, and finally improve the accuracy of predicting the missing value. [Brief description of the drawings]

[0021] [Figure 1] 1 is a flow chart of a method of the present invention. [Diagram 2] FIG. 13 is a schematic diagram of splitting the original matrix according to columns and combining column blocks horizontally during matrix reconstruction. [Diagram 3] FIG. 2 is a schematic diagram of elements in a new matrix P. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] The present invention will be further described below with reference to the drawings and specific examples. However, these examples are merely for the purpose of illustrating the present invention and do not limit the scope of the present invention. It should be understood that any modifications made by those skilled in the art to various equivalent forms of the present invention after reading the present invention are within the scope limited by the appended claims of this application.

[0023] As shown in FIG. 1, the present invention provides a method for complementing multi-dimensional time-series monitoring data of concrete condition, including the following steps S1-S4.

[0024] S1: The multi-dimensional raw data collected by the automated concrete monitoring system, including temperature, pH value, resistivity, chloride ion concentration, calcium ion concentration, and seepage flow, were subjected to a Box-Cox transformation so that the data followed a normal distribution. The specific steps of the Box-Cox transformation are: Step S1-1: collecting multi-dimensional raw data including a total of six variables, namely, temperature, pH value, resistivity, chloride ion concentration, calcium ion concentration, and seepage flow, from a concrete automation monitoring system, where the system measurement frequency is 15 minutes; Step S1-2 performs a Box-Cox transformation on the collected multi-dimensional raw data so that the data follows a normal distribution, thereby ensuring the regression characteristics of the time series dimension of the variables, and the formula is:

number

number

number

number

[0025] S2: Matrix D with the transformed multi-dimensional time series data n,m In order to utilize time information to strengthen the interdependence between multi-dimensional data, the columns of the matrix are first cut, and then a reconstruction operation is performed. The new matrix after reconstruction is P n,m*n′ and Referring to FIG. 2, the specific process is as follows: Matrix D for multivariate time series data after Box-Cox transformation n,m each row of the matrix represents a data record on a time stamp, n,m teeth,

number

number

number

number

number

number

[0026] S3: Matrix P reconstructed using multiple imputation method with chained equations n,m*n′ After predicting the missing values ​​in and imputing the missing values, we convert the imputed matrix to the matrix D n,m Extract The specific steps are: A step S3-1 of predicting missing values ​​in the reconstructed matrix P using a multiple imputation method based on a chain of equations and imputing the missing values, The specific steps of the chained equation algorithm are: B1, which estimates missing values ​​in a matrix, B1, which replaces missing values ​​in variables with the mean value of the collected data; Select one variable in turn and delete its estimate, and use the deleted estimate variable as a dependent variable of the imputation model, and use the remaining variables as independent variables of the model, and select an appropriate imputation model based on the characteristics of the dependent variable to predict the missing value. B2. The method of selecting the imputation model is to select an appropriate imputation model based on the data characteristics of the variables in the selected column; if the relationship between the data is linear, select a linear regression model and estimate the missing values ​​by fitting a linear function; if the variable data is continuous or discrete data, select a K-nearest neighbor model and estimate the missing values ​​using the nearest observations based on the similarity between the data; B2; B3, which fills the data columns of the matrix and replaces the previous estimates with the completion results; B4: Repeat steps B2-B3 for each variable that contains missing values ​​until all missing values ​​have been replaced; Step S3-1, which is training the imputation model multiple times until it converges; and step B5. After the missing values ​​in the new matrix P are imputed, the imputed matrix is ​​converted to the matrix D. n,m and step S3-2 of extracting All the above steps are computer-implemented.

[0027] S4: By performing inverse data transformation on the data in the original matrix extracted from the new matrix, the data and the observed data are brought to the same digits, and finally the completion of the multi-dimensional time series data is realized. The specific steps of the inverse transformation are: Extracted matrix D n,m Step S4-1 performs an inverse Box-Cox transformation on the data in

number

[0028] This embodiment further provides a concrete condition multi-time series monitoring data complementation system, including a network interface for realizing signal transmission and reception in the process of transmitting and receiving information with other external network elements, a memory for storing computer program instructions executable by the processor, and a processor for performing steps of the above consensus method when executing the computer program instructions.

[0029] The present embodiment further provides a computer storage medium on which a computer program is stored, and when a processor executes the computer program, the above method can be realized. The computer readable medium may be considered tangible and non-transitory. Non-limiting examples of non-transitory tangible computer readable media include non-volatile memory circuits (e.g., flash memory circuits, erasable programmable read-only memory circuits or masked read-only memory circuits), volatile memory circuits (e.g., static random access memory circuits or dynamic random access memory circuits), magnetic storage media (e.g., analog or digital tape or hard disk drives), optical storage media (e.g., CDs, DVDs or Blu-ray discs), and the like. A computer program includes processor executable instructions stored on at least one non-transitory tangible computer readable medium. A computer program may further include or depend on stored data. A computer program may include a basic input output system (BIOS) that interacts with the hardware of a dedicated computer, a device driver program that interacts with specific devices of a dedicated computer, one or more operating systems, user applications, background services, background applications, and the like.

[0030] It should be understood by those skilled in the art that the embodiments of the present application can be provided as a method, a system, or a computer program product. Thus, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. And, the present application can take the form of a computer program product embodied in one or more computer usable storage mediums (including, but not limited to, magnetic disk memories, CD-ROMs, optical memories, etc.) containing computer usable program code.

[0031] (Additional Note) (Appendix 1) Step S1 of performing Box-Cox transformation on the multi-dimensional raw data collected by the concrete automation monitoring system so that the data follows a normal distribution; Matrix D for multivariate time series data after Box-Cox transformation n,m Construct the matrix D n,m Then, we cut the columns of P and perform the reconstruction operation, and the new matrix after reconstruction is n,m*n′ Step S2: The reconstructed matrix P n,m*n′ Predict the missing values ​​in , impute the missing values, and convert the imputed matrix to matrix D. n,m Step S3 of extracting Extracted matrix D n,m and step S4 of performing an inverse transformation on the data in order to make the data and the observation data have the same order of magnitude, thereby finally achieving the completion of the multi-dimensional time series data; The specific process of step S2 is as follows: Matrix D for multivariate time series data after Box-Cox transformation n,m each row of the matrix represents a data record on a time stamp, n,m teeth,

number

number

number

[0032] (Appendix 2) The method for complementing multi-dimensional time-series monitoring data of concrete condition described in Appendix 1, characterized in that the multi-dimensional raw data in step S1 includes temperature, pH value, resistivity, chloride ion concentration, calcium ion concentration, and seepage flow.

[0033] (Appendix 3) The specific process of step S1 is as follows: A step S1-1 of collecting multi-dimensional raw data from a concrete automation monitoring system; Step S1-2 performs a Box-Cox transformation on the collected multi-dimensional raw data so that the data follows a normal distribution, thereby ensuring the regression characteristics of the time series dimension of the variables, and the formula is:

number

number

number

number

[0034] (Appendix 4) The method for solving n' in step S2-2 is as follows: Whether or not all elements in a row can be missing is determined by the matrix D n,m Observe each row of A1 and A2, which, if it exists, establishes a set S that contains all the divisors of n whose corresponding quotients are greater than or equal to column m of the original matrix; If it does not exist, A3 executes A4 as is. n′ is an element in the set S, and the length of the new sequence n / n′ is the length of the matrix D n,m is greater than the length of the sequence of n, i.e., n / n′>m, and n′ is

number

number

[0035] (Appendix 5) The specific steps of step S3 are as follows: Step S3-1 of predicting missing values ​​in the reconstructed matrix P using a multiple imputation method based on a chain of equations and imputing the missing values; After the missing values ​​in the new matrix P are imputed, the imputed matrix is ​​converted to the matrix D. n,m and step S3-2 of extracting:

[0036] (Appendix 6) The specific steps of the chained equation algorithm in step S3-1 are as follows: a step B1 of estimating missing values ​​in a matrix; Step B2: select one variable in turn and delete its estimate, use the deleted estimate variable as the dependent variable of the imputation model, and use the remaining variables as independent variables of the model, and select an imputation model based on the characteristics of the dependent variable to predict the missing value; Step B3 of filling the data columns of the matrix and replacing the previous estimates with the imputed results; step B4 of repeating steps B2-B3 for each variable containing missing values ​​until all missing values ​​have been replaced; and step B5 of repeatedly training the complementation model multiple times until the complementation model converges.

[0037] (Appendix 7) The method for imputing multi-dimensional time-series monitoring data of concrete conditions according to Supplementary Note 6, wherein the method for estimating missing values ​​in the matrix in step B1 is to replace missing values ​​in variables with average values ​​of collected data.

[0038] (Appendix 8) The method for complementing multi-dimensional time-series monitoring data of concrete condition according to Appendix 6, wherein the method for selecting the complementation model in step B2 is to select the complementation model based on data characteristics of the variables of the selected column, and when the relationship between the data is linear, select a linear regression model and estimate the missing values ​​by fitting a linear function, and when the variable data is continuous or discrete data, select a K-nearest neighbor model and estimate the missing values ​​using nearest neighbor observed values ​​based on the similarity between the data.

[0039] (Appendix 9) The specific steps of step S4 are as follows: Extracted matrix D n,m Step S4-1 performs an inverse Box-Cox transformation on the data in

number

Claims

1. Step S1 of performing Box-Cox transformation on multi-dimensional raw data collected by the automated concrete monitoring system so that the data follows a normal distribution; Matrix D for multi-dimensional time series data after Box-Cox transformation ｎ，ｍ Construct a matrix D ｎ，ｍ The columns of P are cut and the reconstructed matrix is ｎ，ｍ＊ｎ′ Step S2: The reconstructed matrix P ｎ，ｍ＊ｎ′ Predict the missing values ​​in the matrix D, impute the missing values, and derive the matrix D from the imputed matrix. ｎ，ｍ Step S3 of extracting Extracted matrix D ｎ，ｍ and step S4 of performing an inverse transformation on the data in order to make the data and the observation data have the same order of magnitude, thereby finally achieving complementation of the multi-dimensional time series data; The specific process of step S2 is as follows: Matrix D for multi-dimensional time series data after Box-Cox transformation ｎ，ｍ , where each row of the matrix represents a data record on a time stamp, and ｎ，ｍ teeth, [0010] It is expressed as Here, the first column is the timestamp, and the remaining columns, i.e., s １ , s ２ , …, s ｍ－１ is a variable, step S2-1, matrix D ｎ，ｍ into n′ short column blocks of equal length, where n′ is calculated based on the autocorrelation values, and the matrix after division is [0025] It is expressed as The reconstructed matrix is ​​D' ｎ／ｎ′，ｍ＊ｎ′ = [T １ , T ２ , …, T ｎ′ , s １１ , s １２ , …, s １ｎ′ , …, s （ｍ－１）１ , s （ｍ－１）２ , …, s （ｍ－１）ｎ′ Step S2-2, Reconstructed matrix D' ｎ／ｎ′，ｍ＊ｎ′ Duplicate n' times, then create n' matrices D ｎ／ｎ′，ｍ＊ｎ′ Vertically combine the delay sequence matrix [0030] Step S2-3 of forming Delay sequence matrix H ｎ，ｍ＊ｎ′ For each row, ｎ，ｍ splice after H′, remove repeated elements in each row of the delay matrix, and create a new delay sequence matrix H′ ｎ，ｍ＊（ｎ′－１） Let D be the matrix ｎ，ｍ and the new delay sequence matrix to create a new matrix P ｎ，ｍ＊ｎ′ = [D ｎ，ｍ , H' ｎ，ｍ＊（ｎ′－１） and step S2-4 of constituting a step of:

2. 2. The method for complementing multi-dimensional time-series monitoring data of concrete condition according to claim 1, wherein the multi-dimensional raw data in step S1 includes temperature, pH value, resistivity, chloride ion concentration, calcium ion concentration, and seepage flow.

3. The specific process of step S1 is as follows: Step S1-1 of collecting multi-dimensional raw data from a concrete automation monitoring system; Step S1-2 performs a Box-Cox transformation on the collected multi-dimensional raw data so that the data follows a normal distribution, thereby ensuring the regression characteristics of the time series dimension of the variables, and the formula is [0045] and Here, x λ ｉ is the raw data, and a ｉ step S1-2, where σ is the transformed data, n is the number of data, and λ is the transformation coefficient, which is determined by the data sequence itself; A step S1-3 of estimating values ​​of the parameters λ using maximum likelihood estimation and establishing a likelihood function L(λ), the formula of which is [0050] and Where: [006] represents the maximum likelihood estimate of a, [0070] The method for complementing multi-dimensional time-series monitoring data of concrete condition according to claim 1, characterized in that: step S1-3, wherein represents an average value of the sequence.

4. The method for solving n' in step S2-2 is as follows: Whether or not all elements in a row can be missing is determined by the matrix D ｎ，ｍ A1 observes each row of A2 establishes a set S, if it exists, that contains all divisors of n whose corresponding quotients are greater than or equal to column m of the original matrix; If it does not exist, A3 executes A4 as is; n' is an element in the set S, and the length of the new sequence n / n' is the length of the matrix D ｎ，ｍ That is, n / n'>m, and n' is [0080] where [0090] The method for complementing multi-dimensional time-series monitoring data of concrete conditions according to claim 1, characterized in that: represents the smallest integer greater than x, and the number of rows and the number of columns of the matrix after division are both A4 and m, which are greater than m.

5. The specific steps of step S3 are as follows: Step S3-1 of predicting missing values ​​in the reconstructed matrix P using a multiple imputation method based on a chain equation and imputing the missing values; After the missing values ​​in the new matrix P are imputed, the matrix D is ｎ，ｍ and step S3-2 of extracting the above-mentioned.

6. The specific steps of the chained equation algorithm in step S3-1 are as follows: A step B1 of estimating missing values ​​in a matrix; Step B2: select one variable in turn and delete its estimate, set the deleted estimate variable as a dependent variable of the imputation model, and set the remaining variables as independent variables of the model, and select an imputation model based on the characteristics of the dependent variable to predict the missing value; Step B3 of filling the data columns of the matrix and replacing the previous estimates with the interpolated results; Step B4 of repeating steps B2-B3 for each variable containing missing values ​​until all missing values ​​have been replaced; and step B5 of repeatedly training the imputation model multiple times until the imputation model converges.

7. 7. The method for imputing multi-dimensional time-series monitoring data of concrete condition according to claim 6, wherein the method for estimating missing values ​​in the matrix in step B1 is to replace missing values ​​in variables with average values ​​of collected data.

8. The method for complementing multi-dimensional time-series monitoring data of concrete condition according to claim 6, characterized in that the method for selecting the complementation model in step B2 is to select the complementation model based on data characteristics of the variables of the selected column, and when the relationship between the data is linear, select a linear regression model and estimate the missing values ​​by fitting a linear function, and when the variable data is continuous or discrete data, select a K-nearest neighbor model and estimate the missing values ​​using nearest neighbor observation values ​​based on the similarity between the data.

9. The specific steps of step S4 are as follows: Extracted matrix D ｎ，ｍ Step S4-1 performs an inverse Box-Cox transformation on the data in [0010] and Here, λ is the optimal solution obtained in S1-3. Step S4-1; Matrix W consisting of the data after inverse transformation ｎ，ｍ and step S4-2 of setting the matrix after completion as follows:

Images