Interpolation method for multi-dimensional time-series monitoring data of concrete condition

Abstract

To provide an interpolation method for multi-dimensional time series monitoring data of concrete condition.SOLUTION: A method includes steps of: performing a Box-Cox transformation on collected multi-dimensional raw data; constructing a matrix Dn,m with multi-dimensional time series data after the Box-Cox transformation, and performing a reconstruction operation after cutting columns of the matrix Dn,m, so that new matrix after the reconstruction is Pn,m*n'; predicting missing values in the reconstructed matrix Pn,m*n', complementing the missing values, and extracting a matrix Dn,m from the complemented matrix; and performing an inverse transformation on the data in the extracted matrix Dn,m. The present invention performs a Box-Cox transformation on the raw data so that it follows a normal distribution, organizes the converted data into a sequence matrix, reconstructs the matrix by dividing the columns of the matrix, and fills in the missing values of the reconstructed matrix by using a multiple complementation method based on chain equations, thereby realizing the complementation of the multi-dimensional time series monitoring data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to the field of water utilization and hydroelectric power generation, and more particularly to a method for complementing multi-dimensional time-series monitoring data of concrete condition. [Background technology]

[0002] Concrete is an extremely important building material with a wide range of applications. In high-rise buildings, concrete supports the foundation. In the water conservancy sector, concrete dams are being constructed throughout China as water resources are developed and utilized. The normal operation of concrete dams directly affects the security of water conservancy projects and the safety of people's lives and property. As the construction time of concrete building materials increases, reliability analysis of concrete monitoring data contributes to understanding the normal condition of concrete and thereby ensuring the stable and safe operation of buildings. However, due to monitoring system degradation, transmission failures, and other issues, the loss of large amounts of monitoring data is a challenge in current concrete dam management. Missing monitoring data reduces the accuracy of concrete condition assessments. Therefore, recovering lost multi-dimensional time-series monitoring data has become an important issue in the analysis and management of concrete buildings.

[0003] The key to data imputation is to establish data dependency between missing data and collected data. A regression machine learning model may predict missing values ​​by learning the relationship between missing values ​​and other variables. However, when there are missing data for multiple variables, directly applying a regression model cannot solve the multivariate data imputation problem.

[0004] Therefore, a new technical solution is needed to solve this problem. Summary of the Invention [Problem to be solved by the invention]

[0005] In order to overcome the shortcomings of the prior art, a method for imputing multi-dimensional time series monitoring data of concrete conditions is provided. This method mainly involves performing a Box-Cox transformation on the raw data so that it follows a normal distribution, organizing the transformed data into a sequence matrix, reconstructing the matrix by dividing the columns of the matrix, and using multiple imputation by chain equations to fill in the missing values ​​of the reconstructed matrix, thereby realizing the imputation of multi-dimensional time series monitoring data. [Means for solving the problem]

[0006] In order to achieve the above object, the present invention provides: Step S1: performing a Box-Cox transformation on the multi-dimensional raw data collected by the automated concrete monitoring system so that the data follows a normal distribution; The matrix D is the multivariate time series data after Box-Cox transformation. n,m In order to strengthen the interdependence between the multiple data by using time information, the matrix D n,m The columns of are cut and the reconstruction operation is performed, and the new matrix after reconstruction is P n,m*n′ Step S2: The reconstructed matrix P n,m*n′ Predict missing values ​​in the matrix D, impute missing values, and derive matrix D from the imputed matrix. n,m Step S3 of extracting Extracted matrix D n,m and step S4 of performing an inverse transformation on the data in order to make the data and the observation data the same order of magnitude, thereby finally achieving the completion of the multi-dimensional time series data.

[0007] Furthermore, the multi-dimensional raw data in step S1 includes temperature, pH value, resistivity, chloride ion concentration, calcium ion concentration, and osmotic flow.

[0008] Furthermore, the specific process of step S1 is as follows: Step S1-1: collecting multi-dimensional raw data including a total of six variables, namely, temperature, pH value, resistivity, chloride ion concentration, calcium ion concentration, and seepage flow, from a concrete automation monitoring system, wherein the system measurement frequency is 15 minutes; Step S1-2 is to perform Box-Cox transformation on the collected multi-dimensional raw data so that it follows a normal distribution and ensures the regression characteristics of the time series dimension of the variable, and the formula is:

number

number

number

number

[0009] Furthermore, the specific process of step S2 is as follows: The matrix D is the multivariate time series data after Box-Cox transformation. n,m a step S2-1 of constructing a matrix D, where each row of the matrix represents a data record on a timestamp; n,m teeth,

number

number

number

number

[0010] Furthermore, the method for solving n in step S2-2 is as follows: Whether all elements in a row can be missing or not is determined by the matrix D. n,m Observe each row of A1 and A2 establishes a set S, if it exists, that contains all the divisors of n whose corresponding quotients are greater than or equal to column m of the original matrix; If it doesn't exist, A3 executes A4 as is. n' is an element in the set S, and the length of the new sequence n / n' is the length of the matrix D n,m is greater than the length of the sequence of n, i.e., n / n′>m, and n′ is

number

number

[0011] Furthermore, the specific steps of step S3 are as follows: Step S3-1: predicting missing values ​​in the reconstructed matrix P using a chained equation multiple imputation method and imputing the missing values; After the missing values ​​in the new matrix P are imputed, the imputed matrix is ​​converted to matrix D. n,m and step S3-2 of extracting the

[0012] Furthermore, the specific steps of the chained equation algorithm in step S3-1 are as follows: a step B1 of estimating missing values ​​in a matrix; Step B2: select one variable in turn and delete its estimated value, use the deleted estimated variable as the dependent variable of the imputation model, and use the remaining variables as independent variables of the model, and select an appropriate imputation model based on the characteristics of the dependent variable to predict the missing value; Step B3 of filling the data columns of the matrix and replacing the previous estimates with the imputed results; Step B4 repeats steps B2-B3 for each variable containing missing values ​​until all missing values ​​have been replaced; and step B5 of training the imputation model multiple times until it converges.

[0013] Furthermore, the method for estimating missing values ​​in the matrix in step B1 is to replace missing values ​​in variables with the mean value of the collected data.

[0014] Furthermore, the method for selecting an imputation model in step B2 is to select an appropriate imputation model based on the data characteristics of the variables in the selected column; if the relationship between the data is linear, a linear regression model is selected to estimate the missing values ​​by fitting a linear function; if the variable data is continuous or discrete data, a K-nearest neighbor model is selected to estimate the missing values ​​using the nearest observed values ​​based on the similarity between the data.

[0015] Furthermore, the specific steps of step S4 are as follows: Extracted matrix D n,m Step S4-1 of performing an inverse Box-Cox transformation on the data in

number

[0016] The chained equation method utilizes a series of regression models for multiple imputation and allows the prediction of subsequent variables using previously estimated values, which can solve multivariate data imputation problems. However, the chained equation method is not designed for time series data imputation, but is specifically designed for general data imputation. In the present invention, to strengthen the time correlation between data in the chained equation method, the matrix is ​​restructured to strengthen the time-recurring patterns in the time series data and improve the correlation between missing values ​​and observed values. [Effects of the Invention]

[0017] The present invention has the following advantages over the prior art.

[0018] 1. By using the Box-Cox transformation to convert the collected raw data into normally distributed variables, the accuracy of estimating missing values ​​by the imputation model used in the chained equations can be improved.

[0019] 2. Because the chained equation does not consider the time connection between data, the matrix is ​​reconstructed before imputation to enhance the time-recurring pattern of the time series data, thereby improving the correlation between missing values ​​and observed values ​​and improving the imputation accuracy.

[0020] 3. When the automated concrete monitoring instrument experiences a loss of variable data within a certain period due to a failure problem, the matrix is ​​divided into columns during the process of reconstructing the matrix to ensure that each row contains an observed value for the corresponding variable, ultimately improving the accuracy of predicting the missing values. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a flow chart of the method of the present invention. [Figure 2] FIG. 10 is a schematic diagram of dividing the original matrix according to columns and combining column blocks horizontally during matrix reconstruction. [Figure 3] 1 is a schematic diagram of elements in a new matrix P. DETAILED DESCRIPTION OF THE INVENTION

[0022] The present invention will be further described below with reference to the drawings and specific examples. However, these examples are merely for the purpose of illustrating the present invention and do not limit the scope of the present invention. It should be understood that after reading the present invention, any modifications made by those skilled in the art to various equivalent forms of the present invention are within the scope limited by the appended claims of this application.

[0023] As shown in FIG. 1, the present invention provides a method for imparting multi-dimensional time-series monitoring data of concrete condition, including the following steps S1 to S4.

[0024] S1: The multi-dimensional raw data collected by the automated concrete monitoring system, including temperature, pH value, resistivity, chloride ion concentration, calcium ion concentration, and seepage flow, was subjected to a Box-Cox transformation so that it would follow a normal distribution. The specific steps of the Box-Cox transformation are: Step S1-1: collecting multi-dimensional raw data including a total of six variables, namely, temperature, pH value, resistivity, chloride ion concentration, calcium ion concentration, and seepage flow, from a concrete automation monitoring system, wherein the system measurement frequency is 15 minutes; Step S1-2 is to perform Box-Cox transformation on the collected multi-dimensional raw data so that it follows a normal distribution and ensures the regression characteristics of the time series dimension of the variable, and the formula is:

number

number

number

number

[0025] S2: Matrix D with the transformed multidimensional time series data n,m In order to utilize the time information to strengthen the interdependence between the multi-dimensional data, the columns of the matrix are first cut, and then a reconstruction operation is performed. After the reconstruction, the new matrix is ​​P n,m*n′ and Referring to Figure 2, the specific process is as follows: The matrix D is the multivariate time series data after Box-Cox transformation. n,m a step S2-1 of constructing a matrix D, where each row of the matrix represents a data record on a timestamp; n,m teeth,

number

number

number

number

number

number

[0026] S3: Reconstructed matrix P using multiple imputation method with chained equations n,m*n′ After predicting the missing values ​​in and imputing the missing values, we extract the matrix D from the imputed matrix. n,m Extract The specific steps are: A step S3-1 of predicting missing values ​​in the reconstructed matrix P using a chained equation multiple imputation method and imputing the missing values, The specific steps of the chained equation algorithm are: B1, which estimates missing values ​​in a matrix, B1, which replaces missing values ​​in variables with the mean value of the collected data; B2, selecting one variable in turn and deleting its estimate, and using the deleted estimate variable as the dependent variable of the imputation model, and the remaining ones as independent variables of the model, and selecting an appropriate imputation model based on the characteristics of the dependent variable to predict the missing value; The imputation model selection method is to select an appropriate imputation model based on the data characteristics of the selected column variables; if the relationship between the data is linear, select a linear regression model and estimate the missing values ​​by fitting a linear function; if the variable data is continuous or discrete, select a K-nearest neighbor model and estimate the missing values ​​using the nearest observations based on the similarity between the data; B2. Fill the data columns of the matrix B3 and replace the previous estimates with the imputed results. B4: Repeat steps B2-B3 for each variable containing missing values ​​until all missing values ​​have been replaced; Step S3-1, which is training the imputation model multiple times until it converges; and step B5. After the missing values ​​in the new matrix P are imputed, the imputed matrix is ​​converted to matrix D. n,m Step S3-2 extracts All of the above steps are performed by a computer.

[0027] S4: By performing an inverse data transformation on the data in the original matrix extracted from the new matrix, the data and the observed data are made to have the same digits, and finally the completion of the multi-dimensional time series data is realized. The specific steps of the inverse transformation are: Extracted matrix D n,m Step S4-1 of performing an inverse Box-Cox transformation on the data in

number

[0028] This embodiment further provides a concrete condition multi-way time series monitoring data complementation system, including: a network interface for realizing signal transmission and reception in the process of transmitting and receiving information to and from other external network elements; a memory for storing computer program instructions executable by the processor; and a processor for performing the steps of the above consensus method when executing the computer program instructions.

[0029] This embodiment further provides a computer storage medium on which a computer program is stored, and when a processor executes the computer program, the above-described method can be realized. The computer-readable medium may be considered tangible and non-transitory. Non-limiting examples of non-transitory tangible computer-readable media include non-volatile memory circuits (e.g., flash memory circuits, erasable programmable read-only memory circuits, or mask read-only memory circuits), volatile memory circuits (e.g., static random access memory circuits or dynamic random access memory circuits), magnetic storage media (e.g., analog or digital tape or hard disk drives), optical storage media (e.g., CDs, DVDs, or Blu-ray discs), etc. A computer program includes processor-executable instructions stored on at least one non-transitory tangible computer-readable medium. A computer program may further include or depend on stored data. A computer program may include a basic input / output system (BIOS) that interacts with hardware in a special-purpose computer, device driver programs that interact with specific devices in a special-purpose computer, one or more operating systems, user applications, background services, background applications, etc.

[0030] Those skilled in the art should understand that the embodiments of the present application may be provided as a method, a system, or a computer program product. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. The present application may also take the form of a computer program product embodied in one or more computer-usable storage media (including, but not limited to, magnetic disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0031] (Addendum) (Appendix 1) Step S1: performing a Box-Cox transformation on the multi-dimensional raw data collected by the automated concrete monitoring system so that the data follows a normal distribution; The matrix D is the multivariate time series data after Box-Cox transformation. n,m and construct the matrix D n,m The columns of are cut and the reconstruction operation is performed, and the new matrix after reconstruction is n,m*n′ Step S2: The reconstructed matrix P n,m*n′ Predict missing values ​​in the matrix D, impute missing values, and derive matrix D from the imputed matrix. n,m Step S3 of extracting Extracted matrix D n,m and step S4 of performing an inverse transformation on the data in order to make the data and the observation data the same order of magnitude, thereby finally achieving completion of the multi-dimensional time series data; The specific process of step S2 is as follows: The matrix D is the multivariate time series data after Box-Cox transformation. n,m a step S2-1 of constructing a matrix D, where each row of the matrix represents a data record on a timestamp; n,m teeth,

number

number

number

[0032] (Appendix 2) The method for complementing multi-dimensional time-series monitoring data of concrete condition described in Appendix 1, characterized in that the multi-dimensional raw data in step S1 includes temperature, pH value, resistivity, chloride ion concentration, calcium ion concentration, and seepage flow.

[0033] (Appendix 3) The specific process of step S1 is as follows: Step S1-1 of collecting multi-dimensional raw data from a concrete automation monitoring system; Step S1-2 is to perform Box-Cox transformation on the collected multi-dimensional raw data so that it follows a normal distribution and ensures the regression characteristics of the time series dimension of the variable, and the formula is:

number

number

number

number

[0034] (Appendix 4) The method for solving n' in step S2-2 is as follows: Whether all elements in a row can be missing or not is determined by the matrix D. n,m Observe each row of A1 and A2 establishes a set S, if it exists, that contains all the divisors of n whose corresponding quotients are greater than or equal to column m of the original matrix; If it doesn't exist, A3 executes A4 as is. n' is an element in the set S, and the length of the new sequence n / n' is the length of the matrix D n,m is greater than the length of the sequence of n, i.e., n / n′>m, and n′ is

number

number

[0035] (Appendix 5) The specific steps of step S3 are: Step S3-1: predicting missing values ​​in the reconstructed matrix P using a chained equation multiple imputation method and imputing the missing values; After the missing values ​​in the new matrix P are imputed, the imputed matrix is ​​converted to matrix D. n,m and step S3-2 of extracting the above-mentioned data.

[0036] (Appendix 6) The specific steps of the chained equation algorithm in step S3-1 are as follows: a step B1 of estimating missing values ​​in a matrix; Step B2: select one variable in turn and delete its estimated value, use the deleted estimated variable as the dependent variable of the imputation model, and use the remaining variables as independent variables of the model, and select an imputation model based on the characteristics of the dependent variable to predict the missing value; Step B3 of filling the data columns of the matrix and replacing the previous estimates with the imputed results; Step B4 repeats steps B2-B3 for each variable containing missing values ​​until all missing values ​​have been replaced; and step B5 of repeatedly training the interpolation model multiple times until the interpolation model converges.

[0037] (Appendix 7) 7. The method for imputing multi-dimensional time-series monitoring data of concrete conditions according to claim 6, wherein the method for estimating missing values ​​in the matrix in step B1 is to replace missing values ​​in variables with average values ​​of collected data.

[0038] (Appendix 8) The method for imputing multi-dimensional time-series monitoring data of concrete condition according to Appendix 6, wherein the method for selecting an imputation model in step B2 is to select an imputation model based on data characteristics of the variables in the selected column, and if the relationship between the data is linear, select a linear regression model and estimate the missing values ​​by fitting a linear function, and if the variable data is continuous or discrete data, select a K-nearest neighbor model and estimate the missing values ​​using nearest observed values ​​based on the similarity between the data.

[0039] (Appendix 9) The specific steps of step S4 are as follows: Extracted matrix D n,m Step S4-1 of performing an inverse Box-Cox transformation on the data in

number

Claims

1. Step S1: performing Box-Cox transformation on multi-dimensional raw data collected by the automated concrete monitoring system so that the data follows a normal distribution; Matrix D is the multidimensional time series data after Box-Cox transformation. ｎ，ｍ and construct the matrix D ｎ，ｍ The columns of P are cut and the reconstruction operation is performed, and the new matrix after reconstruction is ｎ，ｍ＊ｎ′ Step S2: The matrix P after reconstruction using the multiple imputation method with chain equations ｎ，ｍ＊ｎ′ Predict missing values ​​in the matrix, impute missing values, and derive matrix D from the imputed matrix. ｎ，ｍ Step S3 of extracting Extracted matrix D ｎ，ｍ and step S4 of performing an inverse Box-Cox transformation on the data in order to make the data and the observed data have the same digits, thereby finally achieving complementation of the multidimensional time series data; The specific process of step S2 is as follows: Matrix D is the multidimensional time series data after Box-Cox transformation. ｎ，ｍ Step S2-1 of constructing a matrix D, where each row of the matrix represents a data record on a timestamp. ｎ，ｍ teeth, [Equation 1] is expressed as where the first column is the timestamp and the remaining columns, i.e., s １ , s ２ , ..., s ｍ－１ is a variable, step S2-1, matrix D ｎ，ｍ into n' short column blocks of equal length, where n' is calculated based on the autocorrelation values, and the matrix after division is [Equation 2] is expressed as The reconstructed matrix is ​​D' ｎ／ｎ′，ｍ＊ｎ′ = [T １ , T ２ , ..., T ｎ′ , s １１ , s １２ , ..., s １ｎ′ , ..., s （ｍ－１）１ , s （ｍ－１）２ , ..., s （ｍ－１）ｎ′ Step S2-2, Reconstructed matrix D' ｎ／ｎ′，ｍ＊ｎ′ Duplicate n' times, then create n' matrices D ｎ／ｎ′，ｍ＊ｎ′ Vertically combine the delay sequence matrix [Equation 3] Step S2-3 of forming Delay sequence matrix H ｎ，ｍ＊ｎ′ matrix D ｎ，ｍ and remove the repeated elements in each row of the delay matrix, and create a new delay sequence matrix H' ｎ，ｍ＊（ｎ′－１） Let matrix D ｎ，ｍ and the new delay sequence matrix to create a new matrix P ｎ，ｍ＊ｎ′ = [D ｎ，ｍ , H' ｎ，ｍ＊（ｎ′－１） and step S2-4 comprising the steps of:

2. 2. The method for complementing multi-dimensional time-series monitoring data of concrete condition according to claim 1, wherein the multi-dimensional raw data in step S1 includes temperature, pH value, resistivity, chloride ion concentration, calcium ion concentration, and seepage flow.

3. The specific process of step S1 is as follows: Step S1-1 of collecting multi-dimensional raw data from a concrete automation monitoring system; Step S1-2 is to perform Box-Cox transformation on the collected multidimensional raw data so that the data follows a normal distribution, thereby ensuring the regression characteristics of the time series dimension of the variables, and the formula is: [Equation 4] and Here, x λ ｉ is the raw data, and a ｉ Step S1-2, where σ is the transformed data, n is the number of data, and λ is the transformation coefficient, which is determined by the data sequence itself; A step S1-3 of estimating the value of the parameter λ using maximum likelihood estimation and establishing a likelihood function L(λ), the formula of which is: [Equation 5] and where: [Equation 6] represents the maximum likelihood estimate of a, [Equation 7] The method for imparting multi-dimensional time-series monitoring data of concrete condition according to claim 1, characterized in that step S1-3, wherein represents an average value of the sequence.

4. The method for solving n' in step S2-2 is as follows: Regarding whether all elements in a row can be missing, the matrix D ｎ，ｍ A1 observes each row of A2 establishes a set S, if it exists, that contains all the divisors of n whose corresponding quotients are greater than or equal to column m of the original matrix; If it does not exist, A3 executes A4 as is; n' is an element in the set S, and the length of the new sequence n / n' is the length of the matrix D ｎ，ｍ is greater than the length of the sequence of [Equation 8] where [Equation 9] represents the smallest integer greater than or equal to the real number x, and the number of rows and the number of columns of the matrix after division are both greater than m, and

5. The specific steps of step S3 are: Step S3-1: predicting missing values ​​in the reconstructed matrix P using a chained equation multiple imputation method and imputing the missing values; After the missing values ​​in the new matrix P are imputed, the imputed matrix is ​​converted to the matrix D. ｎ，ｍ and step S3-2 of extracting the above-mentioned data.

6. The specific steps of the chained equation algorithm in step S3-1 are: A step B1 of estimating missing values ​​in a matrix; Step B2: Select one variable in turn and delete its estimated value, use the deleted estimated variable as the dependent variable of the imputation model, and use the remaining variables as independent variables of the model, and select an imputation model based on the characteristics of the dependent variable to predict the missing value; Step B3 of filling the data columns of the matrix and replacing the previous estimates with the interpolated results; Step B4 repeats steps B2-B3 for each variable containing missing values ​​until all missing values ​​have been replaced; and step B5 of repeatedly training the imputation model multiple times until the imputation model converges.

7. 7. The method for imputing multi-dimensional time-series monitoring data of concrete condition according to claim 6, wherein the method for estimating missing values ​​in the matrix in step B1 is to replace missing values ​​in variables with average values ​​of collected data.

8. 7. The method for imputing data of multi-dimensional time-series monitoring of concrete conditions according to claim 6, wherein the method for selecting an imputation model in step B2 is to select an imputation model based on data characteristics of variables in the selected column, and if the relationship between the data is linear, select a linear regression model and estimate the missing values ​​by fitting a linear function, and if the variable data is continuous or discrete data, select a K-nearest neighbor model and estimate the missing values ​​using nearest observed values ​​based on the similarity between the data.

9. The specific steps of step S4 are as follows: Extracted matrix D ｎ，ｍ Step S4-1 performs an inverse Box-Cox transformation on the data in [Equation 10] and Here, λ is the optimal solution obtained in S1-3. Matrix W consisting of data after inverse transformation ｎ，ｍ and step S4-2 of setting the matrix after interpolation as follows:

Images