Shield tunneling big data scalable storage and processing method

Abstract

The application discloses a shield large data scalable warehousing and processing method, comprising the following steps: based on the multivariate heterogeneous data stream collected on the shield construction site, effective tunneling data in the tunneling stage is extracted; the statistical baseline corresponding to the ring number where the effective tunneling data is located is acquired, and the effective tunneling data is dynamically tolerance detected and the abnormal value is eliminated according to the statistical baseline, so that the cleaned time series data is obtained; the expected data amount of the current ring is calculated based on the ring boundary moment determined by the multivariate heterogeneous data stream, and the data coverage of each physical parameter is calculated in combination with the actual effective data amount of the cleaned time series data; the completeness score of the current ring containing a short board penalty term is calculated; when the preconfigured hierarchical trigger threshold is satisfied, the cleaned time series data is input into a preset construction analysis algorithm model, and a construction state analysis result is obtained. The application breaks the dependence on discrete hardware signals, suppresses the high-frequency jitter of the working condition state, and effectively prevents the false deletion of the physical mutation data of the stratum transition section.

Description

Technical Field

[0001] This invention belongs to the field of underground engineering informatization and data processing, and in particular, it is a method for scalable data storage and processing of shield tunneling big data. Background Technology

[0002] Tunnel boring machines (TBMs) operate in complex underground spaces, where their mechanical state is highly coupled with the geological environment. Real-time and accurate extraction and processing of massive amounts of time-series sensor data from multi-source, heterogeneous equipment is a technological prerequisite for predicting cutterhead wear, providing early warnings of geological risks, and intelligently optimizing tunneling parameters. High-quality spatiotemporal data flow control and structured processing capabilities directly determine the effectiveness of feature extraction in downstream intelligent analysis algorithms and the accuracy of engineering simulations.

[0003] Current shield tunneling construction data stream processing systems typically rely on discrete hardware signals output by programmable logic controllers (PLCs) to classify data batches and ring numbers, and use independent static numerical rules to label construction stages. In the data cleaning phase, mainstream solutions generally use statistical criteria based on fixed time windows to remove outliers, and then calculate a conventional arithmetic mean coverage rate based on the number of data arrivals from each sensor to assess the overall completeness of the current data matrix and trigger subsequent operations.

[0004] Existing solutions mainly suffer from three deep-seated technical problems: fragile temporal slices, jitter in condition identification, and rigid cleaning baselines. Specifically, they rely heavily on underlying hardware signals to divide data blocks. When signals are lost or corrupted, the system struggles to autonomously deduce loop boundaries from the continuous stream, potentially leading to the loss of the three-dimensional spatial mapping benchmark for the entire data segment. Furthermore, stateless static threshold criteria ignore irreversible physical process constraints, and minute sensor fluctuations can easily cause high-frequency interruptions and logical inconsistencies in condition tags. Additionally, cleaning strategies based on pure temporal neighborhoods lack geological spatial perception capabilities, making it prone to misinterpreting normal mechanical response abrupt changes as dirty data and erroneously deleting large amounts of data when the tunnel boring machine traverses transitional zones between different strata. Summary of the Invention

[0005] The purpose of this invention is to provide a method for scalable data storage and processing of tunnel boring machine (TBM) big data, in order to solve the aforementioned problems existing in the prior art.

[0006] The technical solution, including methods for scalable data storage and processing of tunnel boring machine (TBM) big data, includes:

[0007] Based on the diverse and heterogeneous data streams collected at the tunnel boring machine construction site, effective tunneling data in the tunneling stage is extracted;

[0008] Obtain the statistical baseline corresponding to the ring number where the valid tunneling data is located, and based on this, perform dynamic tolerance detection on the valid tunneling data and remove outliers to obtain cleaned time series data;

[0009] The expected data volume of the current ring is calculated based on the ring boundary time determined by the multi-heterogeneous data stream, and the data coverage of each physical parameter is calculated by combining the actual effective data volume of the cleaned time series data.

[0010] Calculate the completeness score of the current loop, including the shortcoming penalty item, based on data coverage.

[0011] When the completeness score meets the pre-configured hierarchical trigger threshold, the cleaned time series data is input into the preset construction analysis algorithm model to obtain the construction status analysis results.

[0012] Beneficial effects: This invention breaks the dependence on discrete hardware signals, suppresses high-frequency jitter in operating conditions, and effectively prevents the accidental deletion of physical abrupt changes in formation transition data. Attached Figure Description

[0013] Figure 1 A flowchart illustrating the steps of a method for scalable data storage and processing in tunnel boring machines, as provided in this application embodiment.

[0014] Figure 2 A flowchart illustrating the steps for extracting valid tunneling data during the tunneling stage, as provided in this embodiment of the application.

[0015] Figure 3 A flowchart illustrating the steps for determining the ring boundary time and the corresponding ring number, provided in an embodiment of this application.

[0016] Figure 4 A flowchart illustrating the steps for determining the corresponding construction stage in an embodiment of this application.

[0017] Figure 5 The overall flowchart of the shield tunneling big data scalable storage and processing method provided in the embodiments of this application is shown.

[0018] Figure 6 The basic structure diagram of the shield tunneling big data scalable storage and processing system provided in the embodiments of this application is shown.

[0019] Figure 7 The flowchart illustrates the data cleaning and algorithm processing provided in the embodiments of this application. Detailed Implementation

[0020] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0021] It should be noted that the terms include and have, and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or device.

[0022] like Figure 1 As shown, a method for scalable data storage and processing in tunnel boring machines includes the following steps:

[0023] Based on the diverse and heterogeneous data streams collected at the tunnel boring machine (TBM) construction site, effective tunneling data during the tunneling stage is extracted.

[0024] like Figure 2 As shown, in a preferred implementation, extracting valid tunneling data during the tunneling phase includes:

[0025] It receives diverse and heterogeneous data streams from the tunnel boring machine construction site and extracts continuous time-series data with timestamps.

[0026] Specifically, the diverse and heterogeneous data streams include data sets of different formats, protocols, and storage methods generated by various subsystems at the tunnel boring machine (TBM) construction site. These data sets include machine operating parameters from the TBM's programmable logic controller (PLC) and geological exploration results. In the system data access environment, a standardized data interface service is deployed to receive these diverse and heterogeneous data streams. This interface service is built on a concrete state transport application programming interface (API) and supports lightweight data exchange formats and binary streams. To implement access control, the header of the data upload request is configured with an access token issued by an authentication authority. The system verifies the token's validity before executing the data reception operation.

[0027] Furthermore, the system parses the header or metadata of the received data to identify the data format and structure type. For example, by extracting and comparing the content type field, the system determines whether the currently received data entity is file data or a sequence of sensor parameters with timestamps. After completing format identification and data routing, the system encapsulates the timestamped sensor data sequence into a standard message format and sends it to a designated topic in the stream processing message middleware, where the stream computing engine continuously consumes and extracts continuous time-series data.

[0028] Parameter jump feature extraction and physical tunneling distance constraint verification are performed on continuous time series data to determine the ring boundary time and the corresponding ring number.

[0029] In this embodiment, parameter jump feature extraction refers to calculating the numerical gradient changes of key parameters such as propulsion speed and cutterhead rotation speed in continuous time-series data within a preset time window. Physical tunneling distance constraint verification refers to performing numerical integration based on the propulsion speed data sequence and time parameters to obtain the calculated displacement value, and comparing the deviation of the calculated displacement value with the preset standard segment ring width value. Existing shield tunneling data management systems typically use the hardware signal status output by the programmable logic controller as the basis for ring number switching. When the hardware signal is missing or the status jumps, the data block logic fails. By performing parameter jump feature extraction and physical tunneling distance constraint verification, based on the changing pattern of continuous sensor readings and physical displacement constraints, the starting point of each segment assembly operation is calculated, the corresponding time is output as the ring boundary time, and the ring number counting and update operation is performed based on this.

[0030] Based on a pre-configured finite state machine, process logic identification is performed on continuous time-series data to determine the corresponding construction stage.

[0031] Specifically, a finite state machine (FSM) is a logical judgment model that defines state nodes, state transition conditions, and time constraint parameters. The construction phase is the current working condition identifier calculated and output by the FSM based on input data. If a static threshold based on a single value is used as the judgment condition, the output working condition will frequently switch when sensor data experiences short-term fluctuations due to interference. By using a pre-configured FSM, and setting state transition path constraints and minimum dwell time thresholds for each state node, logical judgments can be made on the continuous time-series input data, resulting in the output of a construction phase identifier that conforms to preset transition rules.

[0032] By associating the ring number and construction stage as spatiotemporal context labels with continuous time series data, tagged time series data is obtained.

[0033] The spatiotemporal context labels are used to identify the spatial location and process time attributes of the data. The ring number corresponds to the underground space segment at the time the data was generated, and the construction stage corresponds to the operational process status at the time the data was generated. The ring number and construction stage are written into the additional fields of each extracted continuous time series data to generate tagged time series data with structured characteristics, providing an index basis for subsequent data filtering based on different tag attributes.

[0034] Data is segmented based on the construction stage in the tagged time series data, and valid tunneling data in the tunneling stage is extracted.

[0035] In this embodiment, data routing refers to routing data to different processing branches based on the written tag content. A filtering operator is used to traverse the tagged time-series data, extracting datasets identified as tunneling stages during construction phases to form valid tunneling data, which serves as the basic input for subsequent anomaly detection and algorithm analysis.

[0036] In an optional implementation, data identified as either a shutdown phase or an assembly phase during the construction phase is routed to a separate storage area for subsequent calculation tasks such as equipment downtime statistics or segment assembly time analysis.

[0037] Obtain the statistical baseline corresponding to the ring number where the valid tunneling data is located. Based on the statistical baseline, perform dynamic tolerance detection on the valid tunneling data and remove outliers to obtain cleaned time series data.

[0038] Specifically, the statistical baseline includes statistical parameters describing the central tendency and dispersion of the data sequence, specifically the expected mean and standard deviation. Dynamic tolerance detection involves calculating upper and lower threshold values ​​based on the statistical baseline and comparing the values ​​of each sampling point in the valid tunneling data with these thresholds. If a sampling point value exceeds these thresholds, the system identifies it as an outlier and removes it from the data sequence. Since this embodiment only processes valid tunneling data, it excludes data with parameter drops caused by shutdowns or equipment unloading, improving the accuracy of sensor interference noise identification under normal operating conditions. The final output is cleaned time-series data after removing outliers.

[0039] The expected data volume of the current ring is calculated based on the ring boundary time determined by the multi-source heterogeneous data stream, and the data coverage of each physical parameter is calculated by combining the actual effective data volume of the cleaned time series data.

[0040] In other words, the expected data volume of the current ring is determined by combining the ring boundary time, and the data coverage of each physical parameter is calculated based on the expected data volume and the actual effective data volume of the cleaned time series data.

[0041] Specifically, the expected data volume refers to the total number of data points calculated according to the theoretical sampling frequency set by the system between two adjacent ring boundary times. The system calculates the time difference between the current ring boundary time and the previous ring boundary time, and multiplies it by the sampling frequency to obtain the expected data volume. For each type of physical parameter included in the cleaned time series data, the number of non-empty data points corresponding to each parameter is counted as the actual effective data volume. The ratio of the actual effective data volume to the expected data volume is further calculated to generate an independent data coverage rate for each type of physical parameter, which is used to quantify the data integrity of that parameter in the current time period.

[0042] The completeness score of the current loop, including the shortcoming penalty item, is calculated based on the data coverage.

[0043] In this embodiment, the bottleneck penalty term is calculated using a weighted calculation rule based on the minimum value variable. When calculating the comprehensive score for multiple parameters, if the arithmetic mean of the coverage rates of all parameters is used, the average value cannot reflect the impact of low-coverage parameters on the overall data usability when some key parameters have low coverage rates while others have high coverage rates. By introducing a multiplicative weight for the minimum data coverage value as a bottleneck penalty term during the calculation process, the completeness score output by the system will decrease proportionally as the minimum coverage value decreases. A comprehensive evaluation index reflecting the degree of deficiency of individual parameters within the dataset is set.

[0044] When the completeness score meets the pre-configured hierarchical trigger threshold, the cleaned time series data is input into the preset construction analysis algorithm model to obtain the construction status analysis results.

[0045] Alternatively, when the completeness score meets the pre-configured hierarchical trigger threshold, the cleaned time series data is input into the preset construction analysis algorithm model to obtain the construction status analysis results. The cleaned time series data, construction status analysis results, ring numbers and construction stage spatiotemporal labels are then encapsulated in a structured manner according to an extensible data model and written in parallel to the time series database, relational database and object storage system to complete the extensible data storage.

[0046] Specifically, the tiered trigger thresholds are set based on the data input integrity requirements of different analysis algorithm models. The construction analysis algorithm model is a pre-deployed numerical calculation model or rule-based logic program. The system compares the completeness score with the corresponding tiered trigger threshold for each model. If the completeness score is greater than or equal to the tiered trigger threshold, the system executes an interface call, formats the cleaned time-series data, inputs it into the corresponding construction analysis algorithm model for calculation, and receives the construction status analysis results output by the model.

[0047] like Figure 3 As shown, in an exemplary embodiment, determining the ring boundary time and the corresponding ring number includes:

[0048] Within a preset short window, calculate the normalized jump variable of the difference between the mean values ​​of the first and second halves of a predetermined set of key parameters in continuous time series data.

[0049] Specifically, the system sets a sliding data interception interval with a fixed time length on the time axis as a preset short-time window to reduce random noise interference in single-point sampled data. The predetermined key parameters include variables that reflect the state of the tunnel boring machine, specifically including propulsion speed, total thrust, cutterhead rotation speed, and segment assembly machine hydraulic pressure.

[0050] For example, key parameters include propulsion speed, and the formula for calculating the normalized jump variable is:

[0051] δ v (t)=max(0,(v* [t-τ，t-τ / 2] -v* [t-τ / 2，t] ) / (v* [t-τ，t-τ / 2] +ε));

[0052] Where, δ v (t) represents the normalized jump variable of the propulsion velocity at time t, τ is the preset short-time window length, and v* [t-τ，t-τ / 2] and v* [t-τ / 2，t] ε represents the average advancing speed of the first and second half of the window, respectively, and ε is a very small positive number used to prevent division by zero.

[0053] In this embodiment, taking the propulsion speed as an example, since its value shows a decreasing trend when switching processes, the system uses a forward calculation formula to calculate its normalized jump variable.

[0054] As an optional implementation, since the hydraulic pressure parameter of the segment assembly machine shows an upward trend at the beginning of the assembly process, the system uses a reverse calculation formula to calculate its normalized jump variable. That is, since the hydraulic pressure of the segment assembly machine shows an upward jump at the loop boundary, the reverse calculation formula for its normalized jump variable is:

[0055] δ pe (t)=max(0,(p* e_[t-τ / 2，t] -p* e_[t-τ，t-τ / 2] ) / (p* e_[t-τ / 2，t] +ε));

[0056] Where, δ pe (t) is the normalized jump variable of the hydraulic pressure of the segment assembly machine at time t, p* e_[t-τ / 2，t] and p* e_[t-τ，t-τ / 2] ε represents the average hydraulic pressure of the rear half window and the front half window, respectively, and ε is a very small positive number used to prevent division by zero.

[0057] The weighted summation of the normalized jump variables for each key parameter yields the comprehensive jump score.

[0058] In this embodiment, to reduce the identification error caused by anomalies in a single physical parameter, the system merges the feature values ​​of multiple parameters by assigning weight coefficients. The formula for calculating the jump comprehensive score is:

[0059] R(t) = w v *δ v (t)+w T *δ T (t)+w n *δ n (t)+w e *δ pe (t);

[0060] Where R(t) is the jump score, δ v (t), δ T (t), δ n (t) represents the positively normalized jump variables of the propulsion speed, total thrust, and cutterhead rotation speed, respectively, δ pe (t) represents the reverse normalized jump variable of the hydraulic pressure of the segment assembly machine, w v w T w n w e The contribution weights of each pre-configured key parameter to the discrimination of the ring boundary are assigned, and the sum of the weights is 1.

[0061] It should be noted that δ T (t) and δ n The formula for calculating (t) and the normalized jump variable δ of the propulsion speed. v The forward calculation formula (t) has the same structure, with the total thrust and the cutterhead rotation speed replacing the propulsion speed as the calculation objects.

[0062] When the overall score of the transition exceeds the preset transition threshold, the corresponding moment is marked as the candidate ring boundary moment.

[0063] Specifically, the system compares the real-time calculated transition score with a preset transition threshold. If the score is greater than the threshold, it indicates that the degree of synchronous change of multiple parameters has met the judgment criteria. The system records the current timestamp and stores it in memory as the candidate ring boundary moment. This is the preliminary screening stage of feature values.

[0064] Calculate the cumulative tunneling distance between the previously confirmed ring boundary time and the candidate ring boundary time.

[0065] In this embodiment, physical size variables are introduced as verification conditions. The system extracts the time information of the last completed ring boundary update based on historical records and performs numerical integration of the velocity parameters within the time interval. For example, the formula for calculating the cumulative tunneling distance is:

[0066] D k =∑_{j:t k-1 <t j ≤t k}[v(t j )*Δt];

[0067] Among them, D k Let t be the cumulative tunneling distance of the kth ring. k-1 For the previously confirmed ring boundary time, t k At the candidate ring boundary time, v(t) j (t) represents time tj The propulsion speed, Δt is the sampling period, ∑_{j:t} k-1 <t j ≤t k} represents time t j Falling on the previously confirmed ring boundary time t k-1 With the current candidate ring boundary time t k The system sums the values ​​between the points in between. It's important to note that the units of the propulsion speed and the sampling period must be dimensionally compatible to output the distance dimension. During the calculation, the system incorporates dimension conversion logic. Before performing the summation, the system reads the set values ​​for the propulsion speed and the sampling period. If their time dimensions are inconsistent, the system applies a conversion factor to either the speed or time period value to ensure the product has a standard distance dimension. For example, when the speed input unit is millimeters per minute and the sampling period unit is seconds, the system divides the speed value by 60 before multiplication, ensuring the physical validity of the integrated distance calculation result.

[0068] In some preferred embodiments, the formula for calculating the cumulative tunneling distance can also be:

[0069] D k =∑_{j:t k-1 <t j ≤t k}[max(0, v(t)] j ))·Δt];

[0070] Where max(0, v(t) j )) indicates taking the positive propulsion speed.

[0071] Determine whether the deviation between the cumulative tunneling distance and the preset standard segment ring width is less than the preset allowable deviation. If so, confirm the candidate ring boundary time as the ring boundary time and update the ring number.

[0072] Specifically, the system calculates the absolute value of the difference between the cumulative tunneling distance and the preset standard segment ring width as deviation data. The system compares the deviation data with the preset allowable deviation value. If the condition is met, it indicates that the time node calculated based on time-series characteristics conforms to the physical constraints of segment width. The system updates the corresponding time status from candidate to confirmed and increments the current ring number by one.

[0073] Through a dual constraint verification mechanism, the system can still autonomously deduce the ring boundary from the continuous data stream even when external hardware signals are missing, thus avoiding the problem of data segment failure caused by hardware signal loss.

[0074] To illustrate the specific calculation process of the above processing logic, the following normalized numerical examples are set. Assume the zero-prevention parameter ε is 0.001. Within the current short-term window, the normalized average propulsion speed in the first half of the window is 0.9, and the average in the second half becomes 0.1. Substituting these values ​​into the forward formula, the normalized jump variable for the propulsion speed is 0.888. The normalized average of the hydraulic pressure of the segment assembly machine acquired synchronously in the first half of the window is 0.1, and the average in the second half is 0.8. Substituting these values ​​into the reverse formula, the normalized jump variable is 0.874. At the same time, assuming that the total thrust and cutterhead speed have not yet undergone significant jumps within the current window, their normalized jump variables are both 0.00. With the weights of the four key parameters all preset to 0.25, substituting the normalized jump variables of each parameter into the weighted summation formula, the calculated comprehensive jump score is approximately 0.44. If the system's preset transition threshold is 0.40, and the overall score for the jump exceeds the threshold, the system records the current moment as the candidate ring boundary moment. It then proceeds to distance verification calculations, assuming the normalized time length of the integration interval is 500 since the previous ring boundary moment, and the normalized average advance speed within the interval is 0.002. Discrete integration is performed, yielding a cumulative tunneling distance of 1.0. Comparing this to the system's set normalized standard segment ring width of 1.0, the calculated deviation is 0. Since 0 is less than the preset allowable deviation threshold, the system ultimately confirms the current node as the ring boundary moment.

[0075] It should be noted that in actual tunnel boring machine (TBM) construction scenarios, the standard segment ring width is typically 1.2 to 2.0 meters, the advance speed is usually in the range of 20 to 80 millimeters per minute, and the sampling period is usually set to 1 second. The above normalized numerical examples are used to illustrate the calculation logic; during engineering deployment, the parameters can be substituted according to the actual dimensions.

[0076] As an alternative basic judgment method, the system can adopt judgment logic based on static thresholds. For example, setting thresholds for propulsion speed and cutterhead rotation speed, the system outputs a shutdown status indicator when the parameter values ​​collected by the sensors are both less than the corresponding thresholds. In actual operation, affected by fluctuations in sensor sampling, the collected parameters may drop below the threshold in a short period of time and then recover quickly, causing the system's output operating status label to change continuously between shutdown and tunneling. This frequent status jump phenomenon will reduce the accuracy of subsequent data classification.

[0077] To address the aforementioned state hopping problem, a preferred implementation method based on path constraints and time constraints is adopted. Specifically, as follows: Figure 4 As shown, the corresponding construction stage is determined, including the following steps:

[0078] Extract key mechanical parameters from continuous time-series data to characterize tunnel boring machine (TBM) movements.

[0079] Specifically, the system parses the input data stream and extracts the sensor measurement values ​​within the current sampling period. The key extracted mechanical parameters include total thrust, propulsion speed, cutterhead rotation speed, and assembly system return signal, which serve as input parameters for subsequent state machine logic decisions.

[0080] Obtain the previous running state recorded by the finite state machine as the historical state.

[0081] In this embodiment, the system reads the state results output by the state machine module in the previous calculation cycle from the storage unit or memory, as the historical state. The system determines the current state node based on the historical state, which serves as the starting reference for executing subsequent state transition logic judgments.

[0082] Determine whether the key mechanical parameters meet the legal state transition conditions preset by the finite state machine, and verify whether the dwell time of the historical state has reached the preset minimum dwell time constraint.

[0083] Specifically, the system inputs key mechanical parameters into the logic operation module, compares them with the pre-configured judgment rules in the system memory, and reads the time value of the current state recorded by the internal timer, and compares it with the set time threshold parameter.

[0084] In a preferred implementation, the finite state machine defines a set of states that includes at least a shutdown state, a preparation state, a tunneling state, and an assembly state. Legal state transition conditions define legal directional transition paths, including: a normal construction flow path from the shutdown state to the preparation state, from the preparation state to the tunneling state, from the tunneling state to the assembly state, and a path returning from the assembly state to the shutdown or preparation state. To prevent deadlock, the transition paths also include an abnormal handling path that returns from the preparation state to the shutdown state, triggered when an emergency stop signal or parameter anomaly is detected. A minimum dwell time constraint is configured independently for each state in the state set to forcibly block erroneous transitions caused by sensor data fluctuations before the dwell time of the current state is met.

[0085] In this embodiment, the judgment rules and constraint parameters involved are specifically defined. The state set of the finite state machine is divided into four designated operating condition nodes. The legal state transition conditions set a one-way allowed switching rule between nodes. According to the set rule, when the system is in a shutdown state, the set allowed switching target is only the ready state.

[0086] Furthermore, the exception handling path sets reverse state transition conditions. When the current node of the system is in the ready state, and the logic operation module receives an emergency stop signal variable that is valid, or detects that parameters such as temperature and pressure exceed the preset safety range, the exception handling path transition conditions are met, and the system changes its state to the shutdown state to maintain the continuity of control logic.

[0087] Meanwhile, the system assigns different fixed time values ​​to the shutdown state, ready state, and other states in the state set as the minimum dwell time constraint. If the timer value is less than the fixed time value, the system determines that the time check is invalid.

[0088] In a further embodiment, the specific triggering condition for the transition from the tunneling state to the assembly state in the legal state transition conditions is as follows:

[0089] The total thrust in the key mechanical parameters is less than the preset thrust unloading threshold, and the propulsion speed is less than the preset propulsion stop threshold.

[0090] When the finite state machine executes the transition from the tunneling state to the assembly state according to the transition trigger condition, it outputs a state transition trigger signal;

[0091] The state transition trigger signal is used to activate the steps of extracting parameter transition features and verifying physical tunneling distance constraints on continuous time series data, as a prerequisite time series context constraint for determining the ring boundary moment.

[0092] In other words, the state transition trigger signal is used to activate the steps of extracting parameter transition features and verifying physical tunneling distance constraints on continuous time-series data, so as to serve as the time-series trigger for the system to start the ring boundary identification calculation.

[0093] In this embodiment, the numerical triggering conditions and output control logic for a specific state transition path are defined. For the transition from the tunneling state to the assembly state, the system sets two synchronization criteria: the total thrust value acquired in the current sampling period is lower than a preset thrust unloading threshold, and the propulsion speed value is lower than a preset propulsion stop threshold. When both the above numerical conditions and dwell time conditions are met, the system executes a state change operation. Simultaneously with the change operation, the system control module generates a state transition trigger signal in the form of an electrical signal or a software message. The system sends this trigger signal to the processing module, which, upon receiving the signal, initiates subsequent numerical transition calculations and physical displacement integral calculations.

[0094] Optionally, among the legal state transition conditions, the transition trigger condition from the shutdown state to the preparation state is: the cutterhead rotation speed is greater than the preset idling start threshold; the transition trigger condition from the preparation state to the tunneling state is: the total thrust is greater than the preset propulsion start threshold and the propulsion speed is greater than zero; the transition trigger condition from the assembly state back to the shutdown state or the preparation state is: the assembly system return signal is valid.

[0095] If the key mechanical parameters meet the legal state transition conditions and the dwell time has reached the minimum dwell time constraint, then the state transition is executed, and the new state is output as the construction phase.

[0096] In this embodiment, when both the numerical logic judgment condition and the time constraint verification condition are met, the system changes the internal state record variable. Specifically, when the input parameters meet the set path restrictions and numerical conditions, and the time verification result shows that the time constraint has been met, the system modifies the value in the state record register, outputs the calculated new node result, and records it as the construction stage identifier of the current sampled data. Taking the transition judgment from the tunneling state to the assembly state as an example, the transition trigger condition is set as follows: the total thrust value in the key mechanical parameters is less than the set thrust unloading threshold, and the propulsion speed value is less than the set propulsion stop threshold. When the cumulative time in the tunneling state meets the time constraint, and the input physical parameters simultaneously meet the above two numerical conditions, the system performs a state transition operation and updates the calculation result to the assembly state.

[0097] During this operation, when the system transitions from the tunneling state to the assembly state, the control module generates a state transition trigger signal. This signal is then transmitted to other processing modules within the system to activate the computational processes for extracting parameter transition features and verifying physical tunneling distance constraints on continuous time-series data. Through this logical coupling mechanism, the system enables the state machine's decision results to trigger instructions for numerical integration calculations.

[0098] Otherwise, the state is suppressed and the historical state is maintained as the construction phase.

[0099] Specifically, if the currently extracted key mechanical parameters do not meet any of the transfer path setting conditions corresponding to the current node, or if the cumulative time of the current state is less than the set minimum dwell time constraint parameter, the system terminates the state change operation. In this case, the system's state recording variables remain unchanged, and the historical state of the previous calculation cycle is output as the construction stage of the current data point. By setting time and logic judgment thresholds, output state changes caused by transient changes in sensor data are reduced.

[0100] In other words, if the input key mechanical parameters do not meet the corresponding transfer path setting conditions, i.e., the input key mechanical parameters do not meet the corresponding legal state transfer conditions, or the calculated dwell time value is less than the preset minimum dwell time constraint parameter, the system terminates the operation of modifying the state record register. It directly calls the historical state value, outputs it, and records it as the construction stage identifier of the current sampled data.

[0101] For example, when the tunnel boring machine (TBM) is in the tunneling state, due to fluctuations in sensor sampling, the total thrust value within a certain sampling period may momentarily drop below the thrust unloading threshold, while the propulsion speed remains above the propulsion stop threshold. In a static threshold-based determination method, the system would immediately output a stop status flag in that sampling period and then re-output the tunneling status flag in the next sampling period as the total thrust recovers, resulting in a meaningless switch of the status label. In the finite state machine scheme of this embodiment, since the transition from the tunneling state to the assembly state requires both the total thrust and the propulsion speed to simultaneously meet the threshold conditions, and the dwell time of the current state must reach the minimum dwell time constraint, the system maintains the tunneling state unchanged in fluctuating scenarios, suppressing false state transitions caused by sensor noise.

[0102] This embodiment uses a combination of path restriction and dwell time constraint to ensure that the construction stage labels output by the system will not switch frequently due to short-term fluctuations in sensor data, thus guaranteeing the stability of subsequent data diversion operations.

[0103] According to one aspect of this application, a statistical baseline corresponding to the ring number where the valid tunneling data is located is obtained; dynamic tolerance detection is performed on the valid tunneling data based on the statistical baseline, and outliers are removed to obtain cleaned time-series data, including:

[0104] Analyze the statistical baseline to obtain the expected mean and tolerance standard deviation within the current reference interval.

[0105] Specifically, the system receives objects or data structures containing statistical attributes, i.e., a statistical baseline. By reading specific fields within this data structure, it extracts the central location parameter, i.e., the expected mean, which characterizes the central distribution trend of the data sequence; and simultaneously extracts the dispersion parameter, i.e., the tolerance standard deviation, which quantifies the degree to which the data sequence deviates from this central location.

[0106] Optionally, as an implementation method driven by recent time series data, the current reference interval is a sliding window of a preset fixed length; correspondingly, the expected mean is the arithmetic mean of historical data points within the sliding window; the tolerance standard deviation is the standard deviation of historical data points within the sliding window; and the anomaly judgment threshold is 3 times the tolerance standard deviation, so as to form a local anomaly removal mechanism based on the 3σ criterion.

[0107] In this embodiment, specific technical limitations are applied to the abstract parameters. The system configures a data extraction range, i.e., a sliding window, with a fixed number of sampling points along the time axis, for example, a length of 60 sampling points. When the system receives new sampled data, the sliding window shifts forward one step in the data sequence, removing the oldest data point and incorporating the newest one. Mathematical calculations are performed on the set of all data points currently within the sliding window. The system calculates the sum of all values ​​in the set and divides it by the window length; the result is the expected mean. Further, the system calculates the sum of the squares of the differences between each data point in the set and the expected mean, divides it by the total number of data points, and then performs a square root operation; the result is the tolerance standard deviation. A multiplication operation is performed, multiplying the extracted tolerance standard deviation by a coefficient of 3. The resulting product is defined as the anomaly detection threshold. This setting corresponds to the 3σ rule in statistics, used to define the numerical boundary for determining whether a data point deviates from normal physical conditions.

[0108] Calculate the absolute deviation between the current data point and the expected mean in the effective tunneling data.

[0109] In this embodiment, the stream computing engine acquires newly received sensor sampling values, i.e., the current data point. The system reads the expected mean that has just been updated, performs a subtraction operation to obtain the difference between the two, and then takes the absolute value of the difference to generate an absolute deviation value, which is used to characterize the offset of the current input data from the local historical baseline state.

[0110] Determine whether the absolute deviation value is greater than the anomaly detection threshold set based on the tolerance standard deviation.

[0111] Specifically, the system inputs the absolute deviation value into the numerical comparator and performs a size comparison instruction with the anomaly detection threshold.

[0112] If so, the current data point is marked as an outlier and removed, and the remaining normal data points are aggregated to generate cleaned time series data.

[0113] Alternatively, if yes, the current data point is marked as an outlier and removed; if no, the current data point is determined to be a normal data point and retained; the retained normal data points are aggregated to generate cleaned time series data.

[0114] Specifically, if the comparison instruction returns true, meaning the absolute deviation value is greater than the anomaly detection threshold, the system determines that the input value does not conform to the continuous physical quantity change pattern. The system modifies the attribute field of the data point in the memory object, writing an anomaly flag. After the filtering component in the data processing pipeline recognizes this flag, it performs a discard operation, preventing the data point from entering the downstream data table.

[0115] If the comparison command returns false, the system determines that the input value is within the set tolerance range and retains the data point. The system then reassembles and writes the continuous data sequence after the retention operation according to the timestamp order, and finally outputs a set of sequences that do not contain the aforementioned abnormal data points, i.e., the cleaned time series data.

[0116] This embodiment provides a data cleaning strategy based on continuous time series, which is suitable for data processing under continuous tunneling conditions in the same stratum.

[0117] In another alternative embodiment, as an implementation that takes into account the geological spatial context, the current reference interval is the target geological partition matched in a pre-constructed geological partition mapping table based on the ring number of the valid tunneling data; correspondingly, the expected mean is the segment mean of the valid historical data accumulated within the target geological partition; the tolerance standard deviation is the segment standard deviation of the valid historical data accumulated within the target geological partition, so as to achieve dynamic adaptation of the anomaly detection baseline as the strata change.

[0118] In this embodiment, the system sets the reference data range for baseline calculation. After receiving valid tunneling data with ring number tags, the system queries the geological zoning mapping table based on the ring number field, reads the corresponding geological interval record, and sets it as the target geological zoning. The system extracts the valid data set within the target geological zoning from the initial ring number record to the present. The arithmetic mean of the values ​​within the valid data set is calculated as the expected mean, and the standard deviation of the values ​​within the data set is calculated as the tolerance standard deviation. This ensures that the statistical parameters output by the system are updated synchronously as the geological attribute boundaries change.

[0119] In one possible embodiment, the pre-built geological partitioning map is obtained in advance through the following non-real-time construction steps:

[0120] Receive unstructured offline geological exploration data, including longitudinal profiles or borehole columnar sections; extract geological zoning information from the offline geological exploration data, including stratigraphic numbers and soil and rock type attributes.

[0121] Specifically, before performing data stream cleaning, the system performs offline data extraction. This involves acquiring files generated during the exploration phase, i.e., offline geological exploration data. The system extracts data fields representing the physical properties of the stratigraphy, i.e., geological zoning information, from drawings and reports through character extraction or input. This includes stratigraphic numbers used to distinguish stratigraphic blocks, as well as the corresponding rock and soil type attributes.

[0122] Establish a correspondence between each geological zone information and the shield tunnel design ring number interval to form a geological zone mapping table, so as to provide online query during data streaming cleaning.

[0123] In this embodiment, the system maps and associates the extracted geological spatial attributes with the sequence number of the tunnel boring machine (TBM) construction. The set soil and rock types are mapped to the start and end ring number pairs, forming the TBM design ring number range. By establishing this association, the system generates a geological zoning mapping table with a data table structure and configures it in the database for the calculation module to execute query commands.

[0124] In a further embodiment, when the sample size of the accumulated valid historical data within the target geological zone is less than the preset minimum statistical confidence sample size, it is determined that the current geological zone is in a stratigraphic transition zone.

[0125] To prevent the accidental rejection of normal formation transition parameters, the tolerance standard deviation is smoothed using the following progressive fusion formula:

[0126] σ blend (t)=α(t·σ g’ (t)+(1-α(t))·γ·σ g_prev ;

[0127] Where, σ blend (t) represents the corrected tolerance standard deviation, α(t) is the fusion coefficient that asymptotically increases from 0 to 1 with increasing sample size, and σ g’ (t) represents the segment standard deviation of the target geological zone at the current time, σ g_prev γ is the standard deviation of the geological zone before the tunnel boring machine leaves, and γ is a preset transition relaxation coefficient greater than 1.

[0128] Correspondingly, the anomaly detection threshold is the product of the corrected tolerance standard deviation and the preset anomaly multiple.

[0129] Specifically, for the identification data located in the stratigraphic transition zone, the system applies numerical correction logic. It reads the segment standard deviation calculated from historical data of the previous geological zone, combines it with the standard deviation of the current target geological zone and the fusion coefficient, and substitutes these values ​​into the progressive fusion formula to perform algebraic operations. It should be noted that the progressive fusion formula used in this implementation is executed as an engineering heuristic dynamic threshold adjustment algorithm; this calculation rule does not belong to the mixed distribution standard deviation derivation theorem within the strict statistical scope.

[0130] The typical range of the transition relaxation factor γ is 1.2 to 2.0. A larger γ value results in a more lenient anomaly detection threshold in the early stages of formation transition, increasing tolerance for normal abrupt changes, but simultaneously reducing the sensitivity to identifying true outliers. In engineering practice, 1.5 is usually chosen as a balance point.

[0131] It should be noted that when the tunnel boring machine is in the first geological zone at the start of the tunnel, and there is no historical statistical data for the previous geological zone, σ g_prevInitialization can be performed using a global default standard deviation value set based on experience from similar projects or equipment calibration data, or a suitable initial value can be determined based on actual geological conditions.

[0132] Furthermore, the mathematical construction mechanism of the fusion coefficient is represented by a piecewise linear truncation function, the specific calculation formula of which is as follows:

[0133] α(t)=min(1,N) g’ (t) / N min );

[0134] Where α(t) is the fusion coefficient, N g’ (t) represents the sample size of valid historical data accumulated within the target geological zone at the current time, N. min This is the preset minimum statistical confidence sample size;

[0135] When the sample size N g’ (t) gradually accumulates to a value greater than or equal to the minimum statistical confidence sample size N. min When the fusion coefficient α(t) is locked at 1, the progressive fusion formula automatically degenerates into a segment standard deviation that depends only on the current target geological zone, and the tolerance relaxation mechanism of the stratigraphic transition section is smoothly ended.

[0136] In this embodiment, the system counts the total number of data points stored within the target geological zone at the current moment, which is taken as the sample size. The sample size is compared with a preset threshold constant, i.e., the minimum statistical confidence sample size. If the sample size is less than this threshold constant, the system records the current status label as a stratigraphic transition section. The piecewise linear truncation formula is called to perform calculations, i.e., dividing the sample size by the minimum statistical confidence sample size, and then taking the minimum value of the quotient and the constant 1. The calculated numerical result is set as the fusion coefficient. In subsequent data input processes, if the sample size reaches or exceeds this threshold constant, and the division result of the formula is greater than or equal to 1, after the minimum value operation, the system-output fusion coefficient remains a constant value of 1.

[0137] This embodiment provides an adaptive data cleaning scheme that incorporates geological spatial context information to process physical parameter data generated when a tunnel boring machine crosses the boundary of different strata.

[0138] According to one aspect of this application, since the conventional arithmetic mean method is used to calculate data quality, it is impossible to intercept the bottleneck effect caused by the absence of a single core driving parameter, which easily leads to incomplete feature matrices with missing key parameters flowing into the downstream algorithm system. Therefore, a bottleneck penalty mechanism based on weighted minimum value calculation is provided to identify and handle the situation of missing data for a single key parameter. Specifically, the data coverage of each physical parameter is calculated, including:

[0139] Extract the time difference between two adjacent ring boundary moments as the current ring tunneling time; multiply the current ring tunneling time by the preset sampling frequency to obtain the desired data volume.

[0140] Specifically, the system reads the timestamp of the last state machine update of the ring number and the timestamp of the latest determined ring boundary. It performs a subtraction operation to obtain the difference variable, which is defined as the current ring excavation duration. It then reads the sensor hardware sampling frequency parameters recorded in the pre-configuration file, i.e., the preset sampling frequency. Finally, it multiplies the time span variable with the sampling frequency value, and the output product represents the theoretically total number of data points that should be collected within that time period, i.e., the expected data volume.

[0141] For each physical parameter in the cleaned time series data, the actual effective data volume corresponding to each physical parameter is calculated. The ratio of the actual effective data volume of each physical parameter to its corresponding expected data volume is determined as the data coverage rate of that physical parameter.

[0142] In this embodiment, the system performs a column-wise splitting operation on the input multidimensional data matrix to form multiple independent single-parameter data sequences, such as propulsion velocity sequences and total thrust sequences. For each data sequence, the system performs a traversal statistical analysis, calculating the total number of non-empty data points that are not marked as anomalies, and outputting the actual effective data volume. Further, a division operation is performed, using the actual effective data volume as the dividend and the expected data volume as the divisor. The calculated quotient is output as the data coverage rate for quantifying the integrity of independent physical parameters.

[0143] In one embodiment of this application, calculating the completeness score of the current loop, which includes a shortcoming penalty term, based on data coverage includes:

[0144] Extract the data coverage of each parameter from the preset key parameter subset, select the minimum coverage, and calculate the average coverage of the key parameter subset; multiply the minimum coverage by the preset core weight, and multiply the average coverage by the corresponding remaining weight, and add the two together to form a completeness score, thereby implementing a penalty for the shortcomings of a single key parameter with a large number of missing values.

[0145] In other words, the completeness score is obtained by multiplying the minimum coverage by the preset core weight, multiplying the average coverage by the corresponding residual weight, and then adding the value of the minimum coverage multiplied by the preset core weight to the value of the average coverage multiplied by the corresponding residual weight.

[0146] It should be noted that this embodiment sets up numerical calculation logic for comprehensively evaluating the quality of multi-parameter data. The system pre-specifies a set containing some important physical parameters as a subset of key parameters. The calculation is performed using a comprehensive completeness scoring formula that includes penalty terms.

[0147] S comp =β*min(C i )+(1-β)*(1 / m)*∑C i ;

[0148] Among them, S comp For completeness, C i Let C be the data coverage of the i-th physical parameter in the subset of key parameters, min(C i (1 / m)*∑C represents the minimum coverage obtained from extraction. i The calculated average coverage is given by m, where m is the total number of parameters in the subset, β is the core weight, and (1-β) is the calculated residual weight.

[0149] In practical engineering deployments, the typical value range for the core weight β is 0.5 to 0.7. The larger this parameter is, the more sensitive the completeness score is to the absence of a single parameter. When β is 0.5, the minimum coverage and average coverage contribute equally to the score; when β is 0.7, the score is mainly determined by the minimum coverage, and the penalty effect is more significant.

[0150] Furthermore, to illustrate the data processing effect of the penalty calculation mechanism, the following normalized comparison example is provided. System parameters are configured as follows: The extracted key parameter subset contains three variables, with a core weight β set to 0.6 and a corresponding residual weight of 0.4. The input data coverage array is [1.0, 1.0, 0.1]. If the conventional arithmetic mean method is used, the system calculates an average score of 0.7. In the calculation logic of this embodiment, the system extracts the value 0.1 as the minimum coverage. Multiplying 0.1 by the core weight 0.6 yields the first product term 0.06; multiplying the arithmetic mean 0.7 by the residual weight 0.4 yields the second product term 0.28. The system performs an addition operation on the two terms, outputting a completeness score of 0.34. Comparison shows that this calculation logic makes the output score value tend towards the single parameter value with the lowest coverage.

[0151] In another embodiment of this application, when the completeness score meets a pre-configured tiered trigger threshold, the cleaned time-series data is input into a preset construction analysis algorithm model, including:

[0152] For each predetermined construction analysis algorithm model with different timeliness and accuracy requirements, a differentiated hierarchical trigger threshold is configured independently. The completeness score is compared with each hierarchical trigger threshold, and only the cleaned time series data is routed to the target construction analysis algorithm model whose trigger conditions are met for calculation.

[0153] In this embodiment, the system stores multiple construction analysis algorithm models with different logics. An independent numerical judgment threshold, i.e., a graded trigger threshold, is assigned to each model variable. For example, the graded trigger threshold corresponding to the risk warning model is set to 0.6, and the graded trigger threshold corresponding to the wear prediction model is set to 0.9. The system receives the completeness score value and performs a greater than or equal to comparison logic with the graded trigger threshold corresponding to each model. When the comparison logic of a certain model returns true, that model is set as the target construction analysis algorithm model, and a data routing channel is opened to transmit the data sequence to the corresponding interface.

[0154] In a further embodiment, when the completeness score is less than all the graded trigger thresholds, it is determined that the data in the current ring is severely missing. At this time, the triggering of the construction analysis algorithm model is blocked, and the cleaned time series data of the current ring is marked as incomplete. Based on the data incompleteness mark, the system initiates a data retransmission request to the data interface service at the shield tunneling site. If the supplementary data is not received within the preset retransmission timeout period, the data incompleteness mark is retained and the data of the current ring is added to the supplementation queue. After receiving the supplementary missing data, the supplementary missing data is merged into the cleaned time series data of the current ring, and the data coverage and completeness score are recalculated to form a closed-loop guarantee mechanism for the quality of time series data.

[0155] Specifically, if the completeness score is lower than the minimum trigger threshold parameter configured within the system, the system terminates all calls to algorithm model interfaces. A data incompleteness flag containing a status description is written to the memory object. After reading the data incompleteness flag, the system communication module assembles a retransmission request message containing the current ring number and the timestamp of the missing time period according to the preset communication protocol, and sends it to the data interface service node at the field end. When the communication module receives the compensation data packet returned from the field end, it performs a data sequence merging operation based on the timestamp information, clears the original data coverage and completeness score register values, and re-triggers the calculation process to determine the data coverage and completeness score, thus realizing conditional feedback control of the data flow.

[0156] In summary, a scalable method for storing and processing tunnel boring machine (TBM) big data includes: receiving heterogeneous data streams and extracting continuous time-series data; performing dual constraint verification on parameter jump characteristics and physical tunneling distance to determine ring boundaries and ring numbers; identifying process logic based on a pre-configured finite state machine, associating spatiotemporal context labels for data, and extracting valid tunneling data; obtaining the corresponding statistical baseline and performing dynamic tolerance detection to remove outliers; calculating the coverage rate of each physical parameter based on the expected data volume, and synthesizing a completeness score with a weighted short-board penalty term; and inputting the score into the construction analysis algorithm model when the score meets the graded trigger threshold.

[0157] In another embodiment of this application, during the system's data receiving phase, the data routing module performs classification and storage operations for data with different structures. When the input data is identified as an unstructured file containing geological exploration reports or design drawings, the system extracts its file attributes and stores its physical file in a distributed object storage system. Simultaneously, the system extracts the file's original name and hash value as metadata records and inserts them into a relational database, achieving independent management of file content and metadata. For time-series data sequences output by the stream processing engine, the system writes them into a time-series database to provide time-series retrieval services.

[0158] Furthermore, when the completeness score meets the pre-configured tiered trigger threshold, the process of inputting the cleaned time-series data into the preset construction analysis algorithm model to obtain the construction status analysis results is as follows: To unify the data input format of different hardware devices, the system performs standardized data encapsulation operations before calling the construction analysis algorithm model. The data scheduling program extracts the corresponding mechanical parameters from the cleaned time-series data and concatenates them into a one-dimensional feature vector according to a fixed arrangement order. For example, the linear representation format of the feature vector is:

[0159] x = [T, M, n, α, v];

[0160] Where x is the input state feature vector, T is the total thrust collected by the system, M is the cutterhead torque, n is the cutterhead rotation speed, α is the penetration depth, and v is the propulsion speed. The system uses this formatted vector as a fixed parameter to input into the interfaces of each independently deployed algorithm model for computation.

[0161] Furthermore, the construction analysis algorithm model is internally configured with judgment rule logic. Taking the construction risk classification model as an example, the system compares the total thrust T in the input vector with the preset maximum thrust threshold T. max Simultaneously compare the propulsion speed v with the preset minimum speed threshold v. min When the total thrust T is greater than a preset threshold and the propulsion velocity v is less than a preset threshold, the model determines that the current data meets the formation jamming condition and outputs the corresponding jamming risk label. After the model calculation is completed, the system serializes the output result into a structured string in Lightweight Data Exchange (JSON) format. The structured string and metadata attribute fields are written together into the result data table of the relational database. The metadata attribute fields include a ring number identifier field for recording spatial location, an algorithm name field for indicating the model type, and a confidence level field for quantifying the reliability of the calculation results.

[0162] Optionally, after generating the construction status analysis results, the system provides a multi-source data joint query function based on interface interaction. When a query request containing parameters of a specific ring number is received, the server initiates a data retrieval from the underlying system using a concurrent processing mechanism. The first retrieval accesses the time-series database to extract the thrust and attitude angle parameter sequences corresponding to the ring number; the second retrieval accesses the relational database to extract the risk label and wear index result records associated with the ring number; and the third retrieval accesses the metadata table of the object storage system to obtain the storage path of the geological profile map file corresponding to the ring segment. The system server completes the aggregation and assembly of the above three returned data in memory and transmits the combined data packet to the front-end display interface to generate a dataset view containing time-series and spatial correlation information.

[0163] The front-end display interface calls the corresponding rendering component based on the received data type identifier. Document type files are loaded using a lightweight document rendering program, while 3D engineering model files are displayed using a 3D view component. When the data packet returned by the relational database contains records with specific risk labels, the interface control module triggers an alarm display command, controlling the display device to output text prompts in a specified area and changing the color display attributes of the corresponding time period data points in the time series curve. Furthermore, the system is configured with multi-interval data statistics functionality. When the system receives a calculation command containing multiple adjacent ring numbers, the data processing module extracts the set of valid tunneling point data for the corresponding ring number interval, calculates the arithmetic mean of the velocity or thrust parameters within that set, and outputs the calculation result to the radar chart generation component for graphics rendering.

[0164] like Figures 5 to 7 As shown, according to one aspect of this application, a method for scalable data storage and processing of tunnel boring machines (TBMs) can also be:

[0165] S1: Receive diverse and heterogeneous data from the tunnel boring machine construction site through a standardized data interface service. The data includes at least the operating data of key components of the tunnel boring machine, geological exploration data, and engineering measurement data.

[0166] The standardized data interface service provides a security authentication mechanism based on access tokens and supports both single data write interfaces and batch data write interfaces to address network transmission interruption issues.

[0167] S2: Perform type determination and routing on the received data, write structured time-series data into the time-series database, and store unstructured file data into the object storage system.

[0168] Specifically, the system parses the message headers or metadata of the input data to identify the data source device, data format, and structure type. If it is identified as a time-series data stream with timestamps generated by a sensor, it forwards it to the message middleware, preparing to write it to the time-series database. The time-series data uses the tunnel boring machine construction ring as the basic organizational unit. If it is identified as a document, drawing, or multimedia file, it extracts its project number, file type, and version information. According to the preset multi-level classification directory rules, it maps the storage path to the logical location in the object storage system and performs storage operations. Furthermore, the system deploys a background monitoring service that continuously scans the specified unstructured file source paths. When a file creation, modification, or deletion event is detected, a synchronization process is automatically triggered, uploading or updating the file to the corresponding logical location in the object storage system, realizing version management and automatic archiving of unstructured data.

[0169] S3: Based on the stream processing framework, the time-series data is cleaned and standardized in real time. The processing includes field mapping, unit unification and running status association.

[0170] In this embodiment, a message queue is used to continuously consume time-series data streams. Within the stream processing engine, based on predefined configuration rules, the following operations are performed: heterogeneous field names from different devices or protocols are mapped to unified model fields; numerical units are converted to system standard units; outliers and invalid data are filtered; and each data record is associated with its construction context, including at least the project it belongs to, the tunnel boring machine (TBM) number, the current ring number, and the construction stage corresponding to the time the data was generated. The construction stage is automatically identified by analyzing the joint state patterns of multiple key parameters of the TBM, including at least one of shutdown, preparation, tunneling, and assembly.

[0171] S4: Input the cleaned data into the preset construction analysis algorithm model for calculation, and store the algorithm results in a relational database.

[0172] The construction analysis algorithm model includes at least one of the following: cutterhead wear prediction algorithm, construction risk classification algorithm, and multi-objective optimization control algorithm.

[0173] S5: The data in the time-series database, object storage system and relational database are integrated, displayed and interactively queried through a visualization system.

[0174] In one detailed embodiment, the system receives diverse and heterogeneous data from the tunnel boring machine (TBM) construction site through a standardized data interface service. This data includes at least operational data of key TBM components (such as cutterhead torque and propulsion speed), geological exploration data (such as rock quality index (RQD) values), and engineering measurement data (such as TBM attitude and segment deviation).

[0175] The system receives diverse and heterogeneous data from the tunnel boring machine (TBM) construction site through a standardized data interface service. This data includes at least operational data of key TBM components, geological exploration data, and engineering survey data. For example, the interface service employs a RESTful API design, supporting multiple data formats such as JSON, XML, and binary streams. All data upload requests must include a valid access token issued by an authentication authority in the HTTP request header. The system verifies the token's validity before processing the data, ensuring secure access. The interface service also provides a single-write interface for real-time streaming reporting and reserves a batch write interface for historical data re-upload or large file uploads to address transmission interruptions caused by unstable on-site networks.

[0176] The system performs type determination and routing on the received data, writing structured time-series data into a time-series database and storing unstructured file data in an object storage system. For data collected by the tunnel boring machine, the system parses the metadata or content type fields of the data message to identify its source device, data format, and structure type. If it is identified as structured time-series data with a timestamp, it forwards it to the message middleware in preparation for writing to the time-series database; if it is identified as a document, drawing, or multimedia file, it extracts its project number, file type, and version information, maps its storage path to its logical location in the object storage system according to preset multi-level classification directory rules, and performs the storage operation.

[0177] Specifically, the system parses the header or metadata of the input data to identify the data source device, data format, and structure type. For example, it uses the Content-Type field to determine whether it is application / json (structured data) or multipart / form-data (file upload). For data without explicit identification, the system can use the file magic number or field keywords for auxiliary identification. Regarding structured data routing, if it is identified as a time-series data stream with timestamps generated by sensors, it is encapsulated as a message and pushed to a topic in the Kafka message queue, awaiting consumption by the downstream stream processing engine. Time-series data is organized using the tunnel boring machine (TBM) construction ring as the basic unit. Each record is forcibly associated with the project number, TBM ID, current ring number, and acquisition time to ensure that the data has complete contextual semantics. For unstructured data management, if it is identified as unstructured files such as geological survey reports, design drawings, and construction logs, the system generates a unique storage path according to preset rules. The path format is / project / work site / ring segment / file type / file name. Then, the SDK of the object storage system is called to upload the file and insert a metadata record into the relational database, which includes the file hash value, original name, storage path, ring number, uploader and time, thus achieving decoupled management of file content and metadata.

[0178] The system performs real-time cleaning and standardization of the time-series data based on a stream processing framework. The process includes field mapping, unit unification, and operational status association. The system continuously consumes the time-series data stream through a message queue and executes cleaning operations in the stream processing engine according to predefined configuration rules. The cleaning process maps heterogeneous field names from different devices or protocols to unified model fields; converts physical units to system standard units; filters invalid data generated during non-tunneling phases such as downtime and segment assembly; and associates each valid record with its construction context. Specifically, for real-time data access, the stream processing job continuously pulls raw time-series data streams from Kafka topics and processes them based on event time to ensure temporal consistency. All data is temporarily stored in memory as structured objects, awaiting processing by subsequent cleaning operators. For data cleaning and transformation, the stream processing engine performs the following operations based on predefined configuration rules: mapping heterogeneous field names from different devices or communication protocols to unified model fields; converting numerical units to system standard units; removing outliers that significantly deviate from the normal range; and removing invalid records generated during shutdowns and segment assembly, retaining only valid data under normal tunneling conditions. For example, if the advance speed v < 5 mm / min and lasts for more than 30 seconds, while the cutterhead rotation speed n = 0, it is considered a shutdown; if the support shoe pressure is released and there is no advance action, it is considered assembly. This type of data is not included in subsequent analysis. For unit unification, the system performs unit conversion on physical quantities using a linear transformation formula:

[0179] x std =k·x raw +b;

[0180] Where x raw x is the original value. std The value is the standardized value, k is the scaling factor, and b is the offset.

[0181] For outlier detection, the system employs a modified 3σ criterion for outlier identification. The mean μ is calculated within a sliding window. w With standard deviation σ w The formula is as follows:

[0182] μ w =(1 / N) ·∑ i=t-N+1 t x i ;

[0183] σ w =sqrt((1 / (N-1)) ·∑ i=t-N+1 t (x i -μ w ) 2 );

[0184] Where N is the window length, with a default of 60 sampling points; x i Let be the original data value of the i-th sampling point within the sliding window. If a data point satisfies |x t -μ w |≥3σ w If the value is not found, it is considered an outlier and will be removed.

[0185] Regarding motion state association, the system associates each valid data record with its construction context, which includes at least the project to which it belongs, the tunnel boring machine (TBM) number, the current ring number, and the construction stage corresponding to the time the data was generated. The construction stage is automatically identified by analyzing the joint state patterns of multiple key parameters of the TBM, including at least one of shutdown, preparation, tunneling, and assembly.

[0186] The cleaned data is input into a pre-defined construction analysis algorithm model for calculation, and the algorithm results are stored in a relational database. Specifically, the construction analysis algorithm model includes at least one of a cutterhead wear prediction algorithm, a construction risk classification algorithm, and a multi-objective optimization control algorithm. The system monitors the cleaned data stream through a scheduling service, and automatically triggers the corresponding algorithm task when it detects that the data for a certain loop has arrived completely. All algorithms are deployed as independent microservices, supporting on-demand invocation and horizontal scaling. Each algorithm model registers its own metadata with the service registry center at startup, including service name, IP address, port, and a list of supported input variables. The scheduler dynamically discovers available instances through service names, achieving load balancing and failover. The scheduler extracts the key parameter vector x=[T, M, n, α, v] of the current loop from the cleaned data, where T is the total thrust, M is the cutterhead torque, n is the cutterhead rotation speed, α is the penetration depth, and v is the propulsion speed. This vector is passed to the algorithm service as a standard input format to ensure interface consistency. The risk classification model receives current environmental geological attributes, such as the surrounding rock grade RQD and groundwater pressure, along with equipment response parameters, and determines the risk type using a rule engine or lightweight classifier. For example, if T>T... max And v <v min If M suddenly increases and is accompanied by abnormal vibration acceleration, it is marked as a tool breakage warning. All algorithm results are uniformly converted to JSON format and written to the MySQL algorithm_results table along with the following metadata:

[0187] ring_no: Associated ring number;

[0188] algorithm_name: Algorithm name;

[0189] result_json: The structured result body;

[0190] confidence: level of confidence or risk level;

[0191] exec_time: Execution timestamp;

[0192] model_version: Model version number.

[0193] Ensure that the results are traceable, comparable, and auditable.

[0194] A visualization system integrates and interactively displays data from the time-series database, object storage system, and relational database. Specifically, the visualization system employs a front-end / back-end separation architecture. The back-end provides high-performance asynchronous APIs based on the FastAPI (Fast Application Programming Interface), while the front-end uses the AMiS low-code framework to dynamically generate pages, enabling the construction of complex interactive interfaces without writing JavaScript. After a user selects a ring number on the front-end, the back-end initiates three parallel queries: querying all time-series parameters for that ring from the time-series database (InfluxDB), such as thrust, torque, and attitude angle; querying associated algorithm analysis results from the relational database (MySQL), such as wear index and risk labels; and querying relevant unstructured files from the object storage service (MinIO) metadata table, such as geological profiles and segment installation records. The three results are aggregated on the server and returned to the front-end, achieving integrated display of a single view for each ring. The front-end automatically generates real-time curves, bar charts, and status cards based on the returned data. When the algorithm results contain high-risk labels, the system automatically highlights the data points for the corresponding time period and displays an alarm banner at the top of the interface, supporting one-click navigation to the associated geological report to view the geological basis. Users can select multiple ring numbers, such as rings 120-1210, and the system will automatically aggregate key metrics for each ring from InfluxDB. Taking average propulsion speed v* as an example, the calculation formula is:

[0195] v*=(1 / N*) ·∑ i=1 N* v j，i ;

[0196] Where j is the ring number, N*=N j v represents the number of effective excavation points in this ring. j，i This represents the initial thrust velocity value for the j-th ring and the i-th effective tunneling point. The system generates a multi-ring comparison radar chart, displaying differences in thrust, torque, velocity, and other dimensions to assist engineers in optimizing tunneling parameters.

[0197] The system automatically associates PDF, DWG, MP4, and other files stored in MinIO based on the ring number. When a user clicks on a filename, the front end calls a common preview component: PDF files are rendered using PDF.js, DWG drawings are loaded using Autodesk Viewer, and video files are played using an HTML5 player, allowing for online viewing without downloading. The system implements data access control based on the RBAC (Role-Based Access Control) model. For example, operators can only view data for their specific section, engineers can export analysis results, and administrators can manage algorithm models. All sensitive operations, such as file deletion and parameter modification, are logged in an audit log, including the operator, time, IP address, and operation content, meeting engineering compliance requirements.

[0198] According to another aspect of this application, a scalable data storage and processing system for tunnel boring machines (TBMs) for implementing the method described in any of the above embodiments includes:

[0199] The data interface service module provides standardized and secure data access interfaces.

[0200] The data storage and routing module is used to distribute data to time-series databases or object storage systems according to data type.

[0201] The streaming data cleaning module is used to perform real-time cleaning, standardization, and operational condition correlation on time-series data.

[0202] The intelligent analysis and processing module is used to run construction analysis algorithm models and persistently store the results;

[0203] The visualization and management module provides a unified front-end interface for data retrieval, chart display, and file management.

[0204] In this embodiment, the data interface service module adopts a microservice architecture, and the system architecture reserves a distributed storage interface for expansion to multiple servers. The visualization and management module is built on the FastAPI backend framework and the AMis frontend low-code framework, supporting multi-dimensional retrieval and visualization of key parameters of tunnel boring machine construction by ring number and time interval. Optionally, the system also includes a tunnel boring machine engineering management system submodule, which is connected to the object storage system and provides functions for manual uploading, downloading, and directory management of unstructured files.

[0205] According to another aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the method described in any of the above embodiments.

[0206] This invention employs a dual constraint verification method, combining short-time jump characteristics of multiple sensing parameters with strict integration of tunnel segment physical dimensions. This allows the system to break free from passive dependence on underlying control signals, deducing loop boundaries solely based on continuous data flow, thus ensuring the unbroken foundation of the three-dimensional spatial mapping of data. By introducing a hard-coded directed graph of real physical processes and a finite state machine with minimum dwell time, transient fluctuations and illogical state transitions are forcibly shielded from logical common sense and physical inertia, giving the data flow smooth and coherent construction stage labels. A spatial mapping is established using offline exploration data, and a progressive fusion smoothing algorithm with a relaxation coefficient is introduced, enabling the anomaly detection baseline to possess geological context awareness. This effectively filters out hardware errors while preserving real physical mutation data generated when the tunnel boring machine traverses different strata. A completeness penalty gateway based on weighted minimum coverage is constructed, causing the score to decrease as the coverage of a single parameter decreases. It can also perform hierarchical routing and reverse retransmission closed loops according to the needs of different models, reducing the risk of incomplete data entering downstream advanced analysis algorithms.

[0207] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the specific details of the above embodiments. Within the scope of the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all fall within the protection scope of the present invention.

Claims

1. A method for scalable data entry and processing of tunnel boring machine (TBM) big data, characterized in that: include: Based on the diverse and heterogeneous data streams collected at the tunnel boring machine construction site, effective tunneling data in the tunneling stage is extracted; Obtain the statistical baseline corresponding to the ring number where the valid tunneling data is located, and based on this, perform dynamic tolerance detection on the valid tunneling data and remove outliers to obtain cleaned time series data; The expected data volume of the current ring is calculated based on the ring boundary time determined by the multi-heterogeneous data stream, and the data coverage of each physical parameter is calculated by combining the actual effective data volume of the cleaned time series data. Calculate the completeness score of the current loop, including the shortcoming penalty item, based on the data coverage. When the completeness score meets the pre-configured hierarchical trigger threshold, the cleaned time series data is input into the preset construction analysis algorithm model to obtain the construction status analysis results. The extraction of valid tunneling data during the tunneling phase includes: Receive diverse and heterogeneous data streams from the tunnel boring machine construction site and extract continuous time-series data with timestamps; Parameter jump feature extraction and physical tunneling distance constraint verification are performed on continuous time series data to determine the ring boundary time and the corresponding ring number; Determine the ring boundary time and the corresponding ring number, including: Within a preset short window, calculate the normalized jump variable of the difference between the mean values ​​of the first and second half windows of a predetermined set of key parameters in continuous time series data. The weighted summation of the normalized jump variables of each key parameter yields the comprehensive jump score. When the overall score of the transition exceeds the preset transition threshold, the corresponding time is marked as the candidate ring boundary time; Calculate the cumulative tunneling distance between the previously confirmed ring boundary time and the candidate ring boundary time; Determine whether the deviation between the cumulative tunneling distance and the preset standard segment ring width is less than the preset allowable deviation. If so, confirm the candidate ring boundary time as the ring boundary time and update the ring number. Key parameters include propulsion speed, and the formula for calculating the normalized jump variable is: δ v (t)=max(0,(v* [t-τ，t-τ / 2] -v* [t-τ / 2，t] ) / (v* [t-τ，t-τ / 2] +ε)); Where, δ v (t) represents the normalized jump variable of the propulsion velocity at time t, τ is the preset short-time window length, and v* [t-τ，t-τ / 2] and v* [t-τ / 2，t] These are the average propagation speeds of the first and second half of the window, respectively, and ε is a very small positive number used to prevent division by zero. The formula for calculating the cumulative tunneling distance is: D k =∑_{j:t k-1 <t j ≤t k }[v(t j )·Δt]; Among them, D k Let v(t) be the cumulative tunneling distance of the kth ring. j (t) represents time t j The propulsion speed, Δt is the sampling period, ∑_{j:t} k-1 <t j ≤t k } represents time t j Falling on the previously confirmed ring boundary time t k-1 With the current candidate ring boundary time t k Summing the points between them.

2. The method according to claim 1, characterized in that, Extract valid tunneling data during the tunneling phase, including: Based on a pre-configured finite state machine, process logic identification is performed on continuous time-series data to determine the corresponding construction stage; By associating the ring number and construction stage as spatiotemporal context labels with continuous time series data, tagged time series data is obtained. Data is segmented based on the construction stage in the tagged time series data, and valid tunneling data in the tunneling stage is extracted.

3. The method according to claim 2, characterized in that, Determine the corresponding construction stages, including: Extract key mechanical parameters from continuous time-series data to characterize tunnel boring machine (TBM) movements; Obtain the previous running state recorded by the finite state machine as the historical state; Determine whether the key mechanical parameters meet the legal state transition conditions preset by the finite state machine, and verify whether the dwell time of the historical state has reached the preset minimum dwell time constraint. If the key mechanical parameters meet the legal state transition conditions and the dwell time has reached the minimum dwell time constraint, then the state transition is executed, and the new state is output as the construction stage. Otherwise, the state is suppressed and the historical state is maintained as the construction phase.

4. The method according to claim 1, characterized in that, The cleaned time-series data includes: Analyze the statistical baseline to obtain the expected mean and tolerance standard deviation within the current reference interval; Calculate the absolute deviation between the current data point and the expected mean in the effective tunneling data; Determine whether the absolute deviation value exceeds the anomaly threshold set based on the tolerance standard deviation; If so, the current data point is marked as an outlier and removed, and the remaining normal data points are aggregated to generate cleaned time series data.

5. The method according to claim 4, characterized in that, The current reference interval is the target geological zone matched in a pre-built geological zone mapping table based on the ring number of the valid tunneling data; The expected mean is the segment mean of the accumulated valid historical data within the target geological zone; The tolerance standard deviation is the segmental standard deviation of the effective historical data accumulated within the target geological zone.

6. The method according to claim 5, characterized in that, When the sample size of the accumulated valid historical data within the target geological zone is less than the preset minimum statistical confidence sample size, it is determined that the current geological zone is in a stratigraphic transition zone. To prevent the accidental rejection of normal formation transition parameters, the tolerance standard deviation is smoothed using the following progressive fusion formula: s blend (t)=α(t)·σ g’ (t)+(1-α(t))·γ·σ g_prev ; Where, σ blend (t) represents the corrected tolerance standard deviation, α(t) is the fusion coefficient that asymptotically increases from 0 to 1 with increasing sample size, and σ g’ (t) represents the segment standard deviation of the target geological zone at the current time, σ g_prev γ is the standard deviation of the geological zone before the tunnel boring machine leaves, and γ is a preset transition relaxation coefficient greater than 1. The anomaly detection threshold is the product of the corrected tolerance standard deviation and the preset anomaly multiple.

7. The method according to claim 1, characterized in that, Calculate the data coverage for each physical parameter, including: Extract the time difference between two adjacent ring boundary moments as the current ring tunneling time; Multiply the current ring tunneling time by the preset sampling frequency to obtain the desired amount of data; For each physical parameter in the cleaned time series data, the corresponding actual effective data volume is calculated. The ratio of the actual effective data volume of each physical parameter to its corresponding expected data volume is determined as the data coverage rate of that physical parameter.

8. The method according to claim 7, characterized in that, The completeness score of the current loop, including the shortcoming penalty, is calculated based on data coverage, including: Extract the data coverage of each parameter from the preset key parameter subset, select the minimum coverage, and calculate the average coverage of the key parameter subset; The minimum coverage rate is multiplied by the preset core weight, and the average coverage rate is multiplied by the corresponding remaining weight. The two are then added together to form a completeness score.

Images