The application discloses a shield large data scalable warehousing and
processing method, comprising the following steps: based on the multivariate heterogeneous
data stream collected on the shield construction site, effective tunneling data in the tunneling stage is extracted; the statistical baseline corresponding to the
ring number where the effective tunneling data is located is acquired, and the effective tunneling data is dynamically tolerance detected and the abnormal value is eliminated according to the statistical baseline, so that the cleaned
time series data is obtained; the expected data amount of the current ring is calculated based on the ring boundary moment determined by the multivariate heterogeneous
data stream, and the data coverage of each physical parameter is calculated in combination with the actual effective data amount of the cleaned
time series data; the completeness
score of the current ring containing a short board penalty term is calculated; when the preconfigured hierarchical trigger threshold is satisfied, the cleaned
time series data is input into a preset construction analysis
algorithm model, and a construction state analysis result is obtained. The application breaks the dependence on discrete hardware signals, suppresses the high-frequency
jitter of the working condition state, and effectively prevents the false deletion of the physical
mutation data of the stratum transition section.