Excavation action generation method and system based on spatiotemporal depth fusion multi-task prediction

Abstract

The present application relates to a tunneling action generation method and system based on spatiotemporal deep fusion multi-task prediction in the technical field of shield engineering data processing, which integrates local and global time sequence information through a dynamic deep fusion network, realizes multi-spatiotemporal scale fusion, improves the perception ability of complex working conditions, simultaneously outputs future state prediction and control candidates, and performs action fusion at the strategy layer to reduce the performance loss caused by the fragmentation of prediction and control, realize prediction-control integration, introduce an uncertainty head and explicitly constrain it at the loss function and strategy fusion layer to effectively suppress the risk under stratum mutation or sensor noise and realize uncertainty constraint safety; based on experience replay, weight self-adaptation and noise removal, the online self-adaptive update demand of different strata and tunneling stages is met, online self-adaptive update is realized, and the action generation ability of tunneling stability control under complex strata and noise conditions is significantly improved.

Description

Technical Field

[0001] This invention belongs to the field of shield tunneling engineering data processing technology, and relates to a method and system for generating tunneling actions based on spatiotemporal depth fusion multi-task prediction. Background Technology

[0002] Shield tunneling is a typical complex construction process, which is affected by a variety of factors, including geological properties (water content, cohesion, internal friction angle and permeability, etc.), heterogeneity of surrounding rock (interlayers, boulders and karst, etc.), construction parameters (earth pressure, slurry pressure, rotation speed, cutterhead rotation speed and torque, propulsion cylinder thrust, grouting ratio and flow rate, etc.), and unit conditions (cutter wear, seal and bearing condition, drive and hydraulic system efficiency). Its controlled objects have the following essential engineering characteristics: strong nonlinearity, strong coupling, time-varying and non-stationary, multiple input and output variables coexisting, significant measurement noise, and multiple and rigid constraints (pressure, displacement, rate, energy consumption and safety red line, etc.).

[0003] In recent years, various representative solutions have emerged along three paths—fuzzy rule-based methods, data-driven prediction, and reinforcement learning decision-making—around parameter prediction, correction, and intelligent control in tunnel boring machine (TBM) excavation. Methods based on fuzzy rules and expert priors (such as an intelligent control method for earth pressure balance TBM tunneling parameters) establish topological and fuzzy correlation matrices of influencing factors to perform real-time analysis and correction of tunneling parameters. However, they lack adaptability to sudden geological changes and strongly coupled working conditions, and lack foresight and uncertainty measurement. Data-driven time-series prediction methods (such as an intelligent dynamic control method for earth pressure based on LSTM) use LSTM models to predict key parameters such as earth pressure in the short term and assist in control. However, engineering constraints and uncertainties are difficult to transmit to the control layer, resulting in limited online robustness and reliability.

[0004] In reinforcement learning decision-making, one approach directly outputs corrective actions by constructing a simulation environment and reward function (such as a reinforcement learning-based intelligent decision-making method for tunnel boring machine (TBM) correction). While this approach possesses end-to-end policy learning capabilities, it lacks integration with time-series prediction and uncertainty assessment, making it difficult to anticipate forward-looking changes in geological conditions and loads. Another approach focuses on using historical data for parameter prediction and guidance (such as a TBM tunneling parameter prediction method and equipment based on offline reinforcement learning). This approach only performs prediction without closed-loop control, or ignores action feasibility and safety domain constraints, making it unsuitable for directly driving PLCs (Programmable Logic Controllers). Therefore, the aforementioned existing solutions struggle to simultaneously achieve unified representation, future prediction, action decision-making, and risk constraints under complex geological and noisy conditions, exhibiting a technical problem of weak action generation capability for tunneling stability control under such conditions. Summary of the Invention

[0005] To address the problems existing in the traditional methods mentioned above, this invention proposes a method and system for generating tunneling actions based on spatiotemporal deep fusion multi-task prediction, which can significantly improve the action generation capability for tunneling stability control under complex strata and noise conditions.

[0006] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0007] On the one hand, a method for generating tunneling actions based on spatiotemporal deep fusion multi-task prediction is provided, including the following steps:

[0008] Raw time series data is collected from the tunnel boring machine and preprocessed. The preprocessed time series is then used to construct samples according to historical windows and prediction steps to obtain sample sequences with a unified time axis.

[0009] The sample sequence is represented by dual-path spatiotemporal coding and dynamic deep fusion to obtain sample-level semantic vectors; the dual-path spatiotemporal coding includes local dependency path and global dependency path.

[0010] After performing global dependency modeling and temporal feature extraction on sample-level semantic vectors using a multi-head self-attention module, multi-task prediction, model uncertainty estimation, and candidate continuous control action vector generation are performed using parallel multi-task output heads; the multi-task output heads include a prediction head, an uncertainty head, and a control head;

[0011] A policy fusion network is used to perform policy fusion on the output of the prediction head, the uncertainty head, and the control head. Based on the task weight vector and safety constraints, the final action that can be output to the tunnel boring machine actuator is generated. The safety constraints include action amplitude limit, rate of change constraint, working condition safety domain constraint and uncertainty penalty. Based on the reinforcement learning framework, experience replay and noise annealing mechanisms are used to explore policies and the final action is updated online adaptively.

[0012] On the other hand, a tunneling action generation system based on spatiotemporal deep fusion multi-task prediction is also provided, including:

[0013] The data preprocessing module is used to collect raw time series data from the tunnel boring machine and perform data preprocessing. The preprocessed time series is used to construct samples according to historical windows and prediction steps to obtain sample sequences with a unified time axis.

[0014] The encoding and representation module is used to perform dual-path spatiotemporal encoding and dynamic deep fusion representation on the sample sequence to obtain sample-level semantic vectors; the dual-path spatiotemporal encoding includes local dependency path and global dependency path.

[0015] The prediction uncertainty module is used to perform global dependency modeling and temporal feature extraction on sample-level semantic vectors using a multi-head self-attention module, and then use parallel multi-task output heads to perform multi-task prediction, model uncertainty estimation, and generation of candidate continuous control action vectors respectively; the multi-task output heads include a prediction head, an uncertainty head, and a control head;

[0016] The strategy fusion output module is used to perform strategy fusion on the prediction head output, uncertainty head output and control head output using a strategy fusion network. Based on the task weight vector and safety constraints, it generates the final action that can be output to the tunnel boring machine actuator. The safety constraints include action amplitude limit, rate of change constraint, working condition safety domain constraint and uncertainty penalty. Based on the reinforcement learning framework, the module uses experience replay and noise annealing mechanisms to explore strategies and adaptively update the final action online.

[0017] One of the above technical solutions has the following advantages and beneficial effects:

[0018] The aforementioned method and system for generating tunneling actions based on spatiotemporal deep fusion multi-task prediction integrates local and global temporal information through a dynamic deep fusion network, achieving multi-spatiotemporal scale fusion and improving the perception capability of complex working conditions. It also outputs future state predictions and control candidates, and performs action fusion at the strategy layer to reduce performance loss caused by prediction-control separation, achieving prediction-control integration. Furthermore, it introduces an uncertainty head and explicitly constrains it in the loss function and strategy fusion layer, effectively suppressing risks under geological abrupt changes or sensor noise, achieving uncertainty constraint safety. Based on experience playback, weight adaptation, and noise removal, it meets the online adaptive update requirements of different geological formations and tunneling stages, enabling online adaptive updates and significantly improving the action generation capability for tunneling stability control under complex geological and noise conditions. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the conventional technology, the drawings used in the description of the embodiments or the conventional technology will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of a tunneling action generation method based on spatiotemporal depth fusion multi-task prediction in one embodiment;

[0021] Figure 2 This is a schematic diagram of the design module of a tunneling action generation method based on spatiotemporal depth fusion multi-task prediction in one embodiment;

[0022] Figure 3This is a schematic diagram of the architecture of the feature fusion output section in one embodiment;

[0023] Figure 4 This is a schematic diagram of the Transformer architecture in one embodiment;

[0024] Figure 5 This is a schematic diagram of the architecture of the attention and multi-task head part in one embodiment;

[0025] Figure 6 This is a schematic diagram of the module composition of a tunneling action generation system based on spatiotemporal depth fusion multi-task prediction in one embodiment. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0027] It should be noted that, in this document, the reference to "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The presentation of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will understand that the embodiments described herein can be combined with other embodiments. The term "and / or" as used herein refers to any combination of one or more of the associated listed items, and all possible combinations, including such combinations.

[0028] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0029] To address the challenges of tunneling stability control in complex geological conditions and under noise interference, this invention proposes an integrated intelligent control method and system for shield tunneling. This system uses multi-channel parallel encoding to obtain robust representations of the collected shield tunneling data. It outputs prediction, control, and uncertainty-related multi-task results in a unified semantic space. Furthermore, it achieves online adaptive updates and closed-loop control within the safety boundary through strategy weighting and risk constraints. The generated actions can be linked with the shield machine's PLC to ensure stable, efficient, and low-energy construction within the safety domain.

[0030] In one embodiment, such as Figure 1As shown, a method for generating tunneling actions based on spatiotemporal depth fusion multi-task prediction is provided, which may include the following processing steps S12 to S18:

[0031] S12: Collect raw time series data from the tunnel boring machine and perform data preprocessing. Construct samples of the preprocessed time series according to the historical window and prediction step size to obtain sample sequences with a unified time axis.

[0032] S14, perform dual-path spatiotemporal coding and dynamic deep fusion representation on the sample sequence to obtain sample-level semantic vectors; dual-path spatiotemporal coding includes local dependency path and global dependency path;

[0033] S16, after performing global dependency modeling and temporal feature extraction on the sample-level semantic vector using a multi-head self-attention module, multi-task prediction, model uncertainty estimation, and candidate continuous control action vector generation are performed using parallel multi-task output heads; the multi-task output heads include a prediction head, an uncertainty head, and a control head;

[0034] S18, a policy fusion network is used to perform policy fusion on the prediction head output, uncertainty head output, and control head output. Based on the task weight vector and safety constraints, a final action that can be output to the tunnel boring machine's actuator is generated. The safety constraints include action amplitude limits, rate of change constraints, working condition safety domain constraints, and uncertainty penalties. Based on a reinforcement learning framework, experience replay and noise annealing mechanisms are used for policy exploration, and the final action is updated adaptively online.

[0035] Understandably, the first step is data acquisition and preprocessing: raw time-series data can be collected from the tunnel boring machine (TBM), which may include multi-dimensional time series data such as the TBM's propulsion speed, cutterhead torque, total propulsion thrust, cutterhead rotation speed, cutterhead penetration depth, and attitude angles (e.g., shield pitch and roll angles). Data preprocessing is then performed on the raw time-series data, including missing value handling, outlier removal, construction of rolling statistical features, construction of difference / lag features, and time window slicing. Feature selection can be performed using mutual information, variance filtering, or correlation constraints, and feature scale alignment can be achieved through standardization or normalization. The preprocessed time-series data is then used to construct samples according to historical windows and prediction step sizes to obtain sample sequences.

[0036] Then, spatiotemporal coding and dynamic deep fusion representation are performed: a multi-layer temporal feature extraction network is constructed to extract the local-global temporal patterns of the samples; features at different levels are dynamically weighted through interactive attention and channel attention to generate a fused temporal representation tensor; temporal pooling or attention readout is performed on the temporal representation tensor to obtain sample-level semantic vectors.

[0037] Next, multi-task prediction and uncertainty estimation are performed using sample-level semantic vectors: the forecast head performs point or interval predictions of target variables for multiple future steps; the uncertainty head outputs variance parameters or quantiles to characterize prediction and control risks; and the control head generates candidate continuous control action vectors.

[0038] Next, policy fusion and action output are performed: a policy fusion network is constructed to fuse the control head output, the prediction head output mapped by the action space, and the uncertainty head output; based on the task weight vector w=[w_f, w_c, w_u] and safety constraints, the final action that can be output to the tunnel boring machine's actuator is generated.

[0039] Safety constraints include action amplitude limits, rate of change constraints, operating condition safety domain constraints, and uncertainty penalties. Specifically, the task weight vector *w* represents the relative importance of different subtasks in the generation of the final control action; *w_f* is the weight coefficient of the trend-oriented control component obtained by mapping the prediction head output through the action space, used to measure the proportion of control conforming to the predicted trend in the final action; *w_c* is the weight coefficient of the real-time control component obtained by the control head output, used to measure the proportion of immediate candidate actions in the final action; and *w_u* is the weight coefficient of the conservative control component obtained by the uncertainty head risk assessment, used to reflect the degree of influence of uncertainty or safety constraints in policy fusion. All *w_f*, *w_c*, and *w_u* are non-negative real numbers and satisfy *w_f* + *w_c* + *w_u* = 1.

[0040] Finally, reinforcement learning optimization and online adaptive updates are performed: Experience replay and noisy annealing mechanisms are used for policy exploration, based on the reward function. R Minimize overall loss L :

[0041] (1)

[0042] Strategies such as Target Network, Early Stopping, and SlidingValidation are employed to improve training stability. During the online phase, task weights and noise intensity are adaptively adjusted based on real-time uncertainty and validation rewards. L control This represents the control-related loss term, which is generated by the candidate continuous control actions output by the control head and the target of the reinforcement learning policy, and is used to measure the deviation of the current policy at the action level. L forecastThis represents the multi-step prediction loss term, corresponding to the supervised loss of the prediction head in a multi-task scenario; P uncertainty This represents the uncertainty penalty term, derived from the variance parameter or quantile output by the uncertainty header, and is used to explicitly constrain high uncertainty scenarios in the loss. R reg Regularization terms include engineering-level regularization or penalties for network parameters, action out-of-bounds tendencies, etc. (i.e., regularization terms will control network parameters and action out-of-bounds tendencies through engineering-level constraints or control measures (penalties) to avoid the system from becoming overly complex, unstable or violating regulations), which are used to improve training stability and engineering security. , , and The weighting coefficients represent the weights of each loss term, used to balance control loss, prediction loss, uncertainty penalty, and regularization term in the overall loss. L The relative importance of [the subject / method].

[0043] Understandable, such as Figure 2 As shown, to enhance the integration of prediction and control, this specification will preferentially adopt: Spatiotemporal encoding, which utilizes multiple models to construct a dual-path parallel extraction of short-term and long-term dependencies, and employs cross-layer attention and gating mechanisms for dynamic deep fusion, outputting sample-level semantic vectors. Multi-task head: This involves parallel output of prediction results for multiple future steps, candidate continuous control actions, and uncertainty parameters (variance parameters / quantiles), where uncertainty is used for constraints and weighting. Policy fusion: After mapping the prediction results to the action space, they are input together with candidate continuous control actions and uncertainty parameters into the policy fusion network, outputting the final action based on task weights and operational safety domain constraints. Reinforcement learning optimization: This employs TD3 (Dual-Delay Deep Deterministic Policy Gradient Algorithm) with dual critics, target policy noise, and delayed policy updates, combined with priority experience replay and soft updates to improve stability and sample efficiency; rewards consider control error, smoothness, energy consumption, and constraint violation rate. Online adaptive update: Task weights and exploration noise are adaptively adjusted based on validation rewards; when uncertainty exceeds the threshold, degraded control and manual takeover prompts can be triggered.

[0044] The aforementioned method for generating tunneling actions based on spatiotemporal deep fusion multi-task prediction integrates local and global temporal information through a dynamic deep fusion network, achieving multi-spatiotemporal scale fusion and improving the perception capability of complex working conditions. It also outputs future state predictions and control candidates, and performs action fusion at the strategy layer to reduce performance loss caused by prediction-control separation, achieving prediction-control integration. Furthermore, it introduces an uncertainty head and explicitly constrains it in the loss function and strategy fusion layer, effectively suppressing risks under geological abrupt changes or sensor noise, achieving uncertainty constraint safety. Based on experience playback, weight adaptation, and noise removal, it meets the online adaptive update requirements of different geological formations and tunneling stages, enabling online adaptive updates and significantly improving the action generation capability for tunneling stability control under complex geological and noise conditions.

[0045] In some implementations, the above methods can be deployed on CPU, GPU or edge computing devices used for tunnel boring machine control, supporting batch training and real-time inference, which facilitates deployment and application on engineering sites.

[0046] In one embodiment, the original time series data includes the tunnel boring machine's propulsion speed, cutterhead torque, total propulsion thrust, cutterhead rotation speed, cutterhead penetration depth, and attitude angle. Data preprocessing includes: resampling the original time series data onto the same time axis; synchronously filling in short-term interruptions according to specified interpolation rules; the specified interpolation rules include linear interpolation rules and zero-order hold interpolation rules; using a quantile interval-based truncation strategy for outliers to limit the data of each channel to the range [Q1-1.5IQR, Q3+1.5IQR]; where IQR is the interquartile range, Q3 is the third quartile, and Q1 is the first quartile; and extracting feature representations based on mutual information scores on the aligned and cleaned time series; the feature representations include basic statistical features, dynamic difference features, and mutual information correlation features.

[0047] Specifically, such as Figure 3 As shown, through data acquisition and preprocessing, data access and time-series alignment for tunnel boring machine (TBM) projects can be achieved. Specifically, in TBM projects, interruptions and abnormal spikes caused by inconsistent sensor frequencies, cutterhead replacements, and shutdowns are common. To ensure the portability of the subsequent model across multiple lines and sections, this embodiment first establishes a unified time reference. It resamples and aligns the original time-series data, including TBM parameters and data signals from multiple sources, to uniformly resample the multi-source signals of the TBM (such as rotational speed, cutterhead torque, thrust, propulsion displacement, grouting volume, attitude angle, and grouting pressure) onto the same time axis. Short-term interruptions are synchronously filled in according to specified interpolation rules, thereby forming a continuous and computable input time series with time alignment. .

[0048] (2)

[0049] in, Indicates the first c Each channel in time t The signal value after upsampling and interpolation; t i This represents the observed time points in the original data, serving as nodes for interpolation; linear Indicates the linear interpolation rule; zero-order-hold Indicates the zero-order preserving interpolation rule; int ERP (·) represents an operator that interpolates the original discrete points according to a set rule. method The parameter is used to specify the interpolation rule: for continuous physical quantities in multi-source signals from a tunnel boring machine, method = linear That is, using linear interpolation rules, the signal value at the time of interruption is calculated based on the linear relationship in time between two adjacent observation points; for states or other discrete quantities, method = zero-order-hold That is, the zero-order hold interpolation rule is used to hold the most recent observation until the next observation time.

[0050] After completing the time alignment of the data, the cleaning and robustness processing of the field data can be performed according to the following established process: Outliers are truncated using a quantile-based truncation strategy, limiting the data for each channel to the range of [Q1-1.5IQR, Q3+1.5IQR] to reduce the impact of extreme values ​​on model updates; the corresponding interval pruning and saturation mapping are as follows:

[0051] (3)

[0052] in, This represents the clipped sequence values; IQR The interquartile range is defined as follows: IQR C = Q 3 C - Q 1 C ; Q 1 C Indicates the first c The first quartile of the channel; Q 3 C Indicates the first c The third quartile of the channel.

[0053] Subsequently, feature representations are extracted from the aligned and cleaned time series, including: basic statistical features to characterize the steady-state level within the window; dynamic difference features (such as first- and second-order differences, rates of change, and their accumulation) to measure short-term evolution trends; and mutual information correlation features to measure the information coupling between the channel and the control objective. In the feature selection stage, candidate features are ranked based on mutual information scoring criteria, and the top-ranked features are selected. TopK The scoring function for mutual information scoring, with each feature chosen as a factor. Score(c) Feature Filtering C k The formal definition is as follows:

[0054] (4)

[0055] in, Score(c) Representing candidate features c The scoring function value is used to measure the degree of information correlation between the feature and the target variable; MI ( x c , y ) indicates the first c Feature sequences x c With target variable y t Mutual information between them; y t This represents the control objective and the key target variables that need to be predicted, and is used as a reference output for mutual information. TopK Indicates by score Score The K features with the highest scores after sorting correspond to the filtering results. C k ; C k This indicates that the scoring was done through mutual information and TopK The feature subset that is retained after selection.

[0056] Based on this, the selected feature subsets are jointly searched according to the above process to finally obtain the feature set used for training.

[0057] (5)

[0058] in, Indicates time t The features constructed Indicates time t The preprocessed data; mean w Indicates that in the current window... The statistical characteristics of the distribution of the mean; std w Indicates to within the current window Statistical characteristics of standard deviation; , They represent First-order difference features and second-order difference features; Indicates the first i A feature constructor is used to modify the original time series. xGenerate statistical difference features; S This indicates that feature constructors are applied to each channel. f The feature subset obtained later z t The feature vector (i.e., the sample sequence) representing the final input model is generated by... Selected feature subset S constitute; KMI (·) is a mutual information index based on Kendall rank correlation, used to evaluate the degree of correlation between candidate feature subsets and target variables.

[0059] Furthermore, before constructing the samples, z-score standardization is performed on the time series of each channel, and a sliding window method is used to divide the continuous time series into sample sequences. It can be understood that, in order to avoid the deviation caused by different channel units in the optimization, z-score (i.e., Z-score) standardization can also be performed on the time series of each channel before entering the modeling stage. The calculation expression of the standardized time series value is shown in Equation (6). The statistical caliber and update strategy of the mean and standard deviation are based on the existing implementation method, thereby ensuring the consistency between the training and deployment stages.

[0060] (6)

[0061] in, x c ( t ) indicates the first c Channel in time t The original data values; This represents the standardized numerical value, used as input to the model. μ c This represents the mean of the training data; σ c It represents the standard deviation on the training data.

[0062] Finally, a sliding window method is used to divide the continuous time series into sample sequences of length . L A historical window of 0 is used to encode the current state, and the predicted horizon is... H The future window is used to supervise multi-step output. The window construction and the sample tensor can be described as follows:

[0063] (7)

[0064] in, S n Let n be the input history window tensor of the nth sample; (t n - L 0+1:t n ) indicates from time t- LFrom 0+1 to t L 0 time steps to the first n Multichannel feature matrix of segment sequence; K 0 indicates the input feature dimension; L 0 indicates the length of the history window; Y n Indicates the first n The future supervision signal window for each sample; y t ( t n +1: t n + H ) indicates from t +1 to t + H of H Predict the target sequence step by step; H This indicates the predicted horizon length (i.e., the number of future steps). D Indicates the dimension of the target variable; R L0×K0 , R H×D These represent the shapes of the corresponding tensors, respectively. L 0× K 0 and H × D A real matrix.

[0065] Furthermore, the following engineering values ​​can be suggested: the time interval of the sliding window. The value is taken for 1 to 2 seconds. L 0 can take values ​​from 32 to 128. H Values ​​range from 8 to 32. TopK The value can be taken from 16 to 128 to obtain the most suitable sample sequence.

[0066] Through the above data acquisition and preprocessing process, a closed loop can be achieved from raw multi-source signals, alignment and cleaning, feature extraction and screening, standardization to windowed samples, forming the input data foundation required by the subsequent dual-path spatiotemporal coding and dynamic deep fusion modules.

[0067] In one embodiment, step S14 described above may specifically include the following steps:

[0068] In the local dependency path, the input sample sequence is first passed through two layers of bidirectional LSTM to output the final encoded features and then attention convergence is performed. The attention weights are used to sum the hidden states at each time step to obtain the local representation vector. The first layer of bidirectional LSTM is used to output the complete intermediate sequence features, and the second layer of bidirectional LSTM is used to output the final encoded features.

[0069] In the global dependency path, the input sample sequence is encoded and embedded at position and then enters the multi-head self-attention module to obtain the temporal encoding tensor output by the global dependency path. The temporal encoding tensor is then processed by attention pooling to obtain the global representation vector.

[0070] By using adaptive gating to perform a convex combination of the local and global representation vectors in the vector space, the fused representation at the current time step is obtained.

[0071] After using cross-attention to exchange information between the local representation vector, the global representation vector and the fused representation at the current time step, the sample-level semantic vector is read out using cross-layer attention.

[0072] Understandable, such as Figure 3 As shown, for spatiotemporal coding and dynamic deep fusion representation, since shield control is affected by both instantaneous load and frictional changes, as well as slow variables with memory such as earth pressure evolution and grout diffusion; short-term transients such as parameter changes and attitude adjustments need to be captured in a timely manner, while long-term trends caused by changes in strata properties and grouting strategies determine the control upper limit and safety, this embodiment adopts dual-path spatiotemporal coding: the local dependency path is responsible for short-term dynamic extraction, and the global dependency path is responsible for long-term modeling and cross-channel correlation modeling; then, interpretable deep fusion is completed before decision-making using gating and attention mechanisms.

[0073] In the locally dependent pathway, the input sample sequence first passes through two layers of bidirectional LSTM (Long Short-Term Memory) networks. The first layer of bidirectional LSTM outputs complete intermediate sequence features, and the second layer of bidirectional LSTM outputs the final encoded features based on these intermediate sequence features. The recursive relationships for the two bidirectional LSTM layers are as follows:

[0074] (8)

[0075] (9)

[0076] in, This indicates that the first layer of bidirectional LSTM is in the range of 1 to 2. L A sequence of hidden states over a single time step; This represents the hidden state sequence of the second-layer bidirectional LSTM at each time step, serving as the final sequence encoding of the local path; L 1 represents the time step, which is the same as the window length; BiLSTM 1(·) represents the first layer of the bidirectional LSTM network, used to encode the input sample sequence; BiLSTM 2(·) represents the second-layer bidirectional LSTM network, used to encode the input sample sequence.

[0077] To obtain representative local semantics for the current control moment, the local dependency path performs attention convergence on the time series, and uses attention weights to weighted summation of the hidden states at each moment to obtain the local representation vector. :

[0078] (10)

[0079] in, Pool For Attention-Pooling: The hidden states at each time step are summed in weights according to the attention weights. LN (·) indicates layer normalization, used to stabilize values. This represents the attention weight at time step t. Represents the attention query vector. This represents the hidden state vector at time step t.

[0080] In the globally dependent path, the input sample sequence, after positional encoding and embedding, enters a multi-head self-attention module to capture long-term relationships and trends across time periods and channels. It then undergoes normalization via a feedforward network and a residual layer to improve numerical stability. The temporal encoded tensor output by the globally dependent path... H1 Its global representation vector obtained by pooling z glob The calculation formula is as follows:

[0081] (11)

[0082] in, MHA (·) indicates a multi-head self-attention module, which performs global dependency modeling on the sequence; Indicates the first n The tensor of the input feature sequence of each sample after being mapped by the embedding layer; E This represents the input embedding vector, which is related to the original feature map; P Represents positional encoding, incorporating temporal positional information; temporal coding tensor H1 It contains global context information for each time step; Pool ( H1 ) indicates to H 1. Perform attention pooling operations in the time dimension; FFN (·) indicates a feedforward network that further transforms the pooling results; z glob This represents the global representation vector, which indicates long-term trends and cross-channel correlation patterns.

[0083] After completing the independent encoding of the two paths, this embodiment introduces adaptive gating for the current operating condition during the fusion stage. First, scalar gating coefficients are obtained based on the linear transformation and activation (such as Sigmoid) of the concatenated local and global representation vectors. α Then, scalar gating coefficients were used. α The representation vectors obtained from the two paths are convexly combined in the vector space to form the fused representation at the current moment. z The gating calculation and representation weight calculation formulas are as follows:

[0084] (12)

[0085] Among them, [z loc ||z glob The symbol ] represents the concatenation of the local representation vector and the global representation vector; W T , b These represent the weight matrix and bias vector of the gated network, respectively. σ (·) represents the Sigmoid activation function, used to compress the linear output to the (0, 1) interval.

[0086] Under this mechanism, when sudden formation changes or short-term disturbances dominate, the gating automatically increases the LSTM weights; during stable tunneling or slowly changing working conditions, the Transformer (such as...) Figure 4 As shown, the dominant element (the element shown) enables automatic scene switching, which can be mathematically represented as:

[0087] (13)

[0088] in, H LSTM This represents the sequence-level representation from the LSTM. H Tr Represents the sequence-level representation from the Transformer; scalar gating coefficients α Used to weight the two pathways; W α and b α This represents the parameters of the linear transformation.

[0089] Furthermore, such as Figure 5 As shown, the gating results at the vector level and sequence level are obtained. Subsequently, to enhance cross-modal and cross-channel coupled modeling capabilities, the fusion stage employs cross-attention in the local representation vector, global representation vector, and prior fusion vector (i.e., the fusion representation at the current time in equation (12)). zInformation is exchanged between them to further enhance attention to key moments and key channels. The dynamic deep fusion representation of cross-attention output is denoted as... z cross The calculation relationship is as follows:

[0090] (14)

[0091] Subsequently, cross-layer attention is used to read out the fused representation, compressing the temporal information into sample-level semantic vectors. s This approach preserves key dependencies while suppressing noise accumulation, providing a homogeneous, stable, and interpretable input representation for subsequent multi-task prediction and uncertainty estimation. Attn (·) denotes the cross-attention operator, used to exchange information between the local representation vector, the global representation vector, and the prior fusion vector; z cross Used for unified output of subsequent multi-task heads.

[0092] After using a multi-head self-attention module to perform global dependency modeling and temporal feature extraction on sample-level semantic vectors, based on the encoding results, parallel multi-task output heads such as prediction head, uncertainty head, and control head are set up to perform multi-task prediction, model uncertainty estimation, and generation of candidate continuous control action vectors, respectively.

[0093] Specifically, such as Figure 5 As shown, for multi-task prediction and uncertainty estimation: after obtaining a unified representation, this embodiment provides three complementary results in parallel on the same semantic level, allowing these three complementary results to influence and constrain each other, so as to avoid situations where only measurement is performed without control or only control is performed without measurement.

[0094] Multi-step prediction head for the future H × D The key variables of each step generate conditional heteroscedasticity outputs. The network simultaneously regresses the mean, median statistic, and log-variance of each dimension at each step to characterize which type of variable is more uncertain at which time step. The corresponding network mapping is shown in Equation (15). During training, a heteroscedasticity negative log-likelihood objective is used, and the specific loss is shown in Equation (16). This heteroscedasticity negative log-likelihood objective encourages the model to be more confident in predictable areas and automatically widens in unpredictable areas at the sample level, which avoids overfitting noise and provides risk information with interval significance for subsequent decisions.

[0095] (15)

[0096] (16)

[0097] in, MLP f (s ) represents the multilayer perceptron corresponding to the prediction head; Indicates the future H × D Statistical prediction of the mean or median of the key variables in the step; σ 2 Indicates the future H × D The variance parameter corresponding to each step is used to characterize the uncertainty of conditional heteroscedasticity; L pred This indicates multi-step loss prediction; Indicates the first h The prediction step, the first d The true observed values ​​of the target variables Indicates the first h The prediction step, the first d The predicted mean of each target variable; σ 2 h,d Indicates the first h The prediction step, the first d The prediction variance of the target variable. Figure 5 In this model, GELU is the activation function, and Dropout is the regularization layer, which randomly discards some neuron outputs to avoid overfitting.

[0098] Secondly, the control head directly uses feature subsets S Candidate continuous control vectors are output in the physical action space to characterize immediate responses based on current perception. The mapping relationship and output domain constraints employ the hyperbolic tangent activation function tanh to limit actions to an engineering-acceptable normalization range, and its computational expression is as follows:

[0099] (17)

[0100] in, a c This represents the candidate continuous control action vector output by the control head; MLP C (·) indicates the multilayer sensor corresponding to the control head; superscript A Let be the dimension of the action.

[0101] The uncertainty head metric, based on the variance and logarithmic variance obtained from heteroscedasticity prediction, can be aggregated into a global risk indicator for the current period. This indicator is used to adjust weights in policy fusion, enabling interpretable control that automatically conserves resources in high-risk scenarios and prioritizes efficiency in low-risk scenarios. The aggregation and usage of the uncertainty metric are decoupled from the aforementioned prediction head and control head to maintain causal consistency between training and deployment.

[0102] By using the three parallel heads of prediction, uncertainty and control, the "where is difficult to predict" can be fed back to "how to control", avoiding prediction and control from acting independently; and heteroscedasticity training prevents the model from treating difficult samples as "bad samples", thus improving overall robustness.

[0103] It is understandable that for strategy fusion and action output, candidate continuous control actions alone are not enough to form an executable control loop. Therefore, this embodiment introduces a strategy fusion layer on a unified representation, which isomorphically expresses the three types of information—predicted trends, immediate candidate actions, and risk levels—within the action domain. Then, the fusion result is constrained to the engineering feasible domain through a differentiable safety layer, and finally, an executable control quantity is obtained.

[0104] Furthermore, regarding step S18 above, the process of generating the final action that can be output to the tunnel boring machine actuator based on the task weight vector and safety constraints may include the following processing: using a differentiable safety layer to project the fused action obtained based on the action fusion weight vector onto the engineering feasible set jointly defined by amplitude, rate of change and working condition domain to obtain the final action.

[0105] Specifically, based on the mapping relationship between predicted actions, the ensemble statistics of multi-step predictions are compressed into the action space to characterize trend-based action components. This mapping between predicted actions employs a lightweight MLP (a multilayer perceptron model that achieves efficient computation through simplified structure or optimized design) to vectorize and compress direction and amplitude information within the future horizon.

[0106] (18)

[0107] in, μ 1:H,1:D Indicates the future H × D The mean of the step prediction; vec (·) denotes a vectorized mapping operator that compresses multi-step prediction statistics into the action space; a f This represents the trend-based action component obtained from the predicted trend mapping, which reflects control recommendations in accordance with the predicted trend in the action space.

[0108] Subsequently, the fusion layer performs weighted synthesis of the three types of information within the same action domain. The weight vector is jointly determined by the representation and risk, and is normalized to a probability distribution using softmax, as calculated below:

[0109] (19)

[0110] in, w This represents the action fusion weight vector; W w and bw This represents the linear transformation parameters that generate the weight vector. z This represents the fused semantic vector (Formula 12); softmax(·) is the normalization function; R 3 It represents a three-dimensional real vector space.

[0111] Under this weighting, unconstrained fusion actions are obtained and numerical stability is maintained. This process achieves interpretable action source decomposition: when uncertainty increases or boundaries tighten, the weights automatically bias towards conservative components; when the prediction is reliable and the safety margin is sufficient, the prediction trend and immediate components dominate.

[0112] (20)

[0113] in, u This indicates a risk indicator derived from uncertainty, used to adjust for conservatism; a c || a f ||(1· u [)] represents the concatenated vector of the three types of information in the action domain; W a Indicates the linear transformation parameters of the fusion layer; a raw This refers to fusion actions that are not subject to safety constraints, and is based on a combination of three sources: predicted trends, immediate candidate actions, and risk levels.

[0114] Before being issued, the fusion action must satisfy the engineering boundary. This embodiment uses a differentiable safety layer to project the aforementioned fusion action onto an engineering feasible set jointly defined by amplitude, rate of change, and operating domain. C The equivalent approximate update form of the final action can be obtained as follows:

[0115] (twenty one)

[0116] in, g m This represents the operating condition constraint function (such as the upper limit of cavity pressure), which satisfies... The time constraint is satisfied; the feasible set of the project is met. C It is defined by the action amplitude limit, the rate of change limit, and the operating condition domain constraint; This indicates the final action after projection through the differentiable safety layer; ▽ a Constraints on Actions a The gradient is used to construct the penalty direction; η Indicates the step size coefficient for updating the differentiable safety layer; Δ ( a () indicates the change in motion between adjacent time steps, used to penalize excessive changes in motion between adjacent steps; λ This represents the penalty weight for the rate of change of action.

[0117] The fusion layer enables interpretability of actions (adapting to predicted trends and automatically conserving energy during high-risk situations), while the differentiable safety layer ensures that actions never cross boundaries or that crossing boundaries incurs high costs, thus meeting the conditions for going online. Through the aforementioned mapping of predicted actions, weight fusion, and the cascading design of differentiable safety projection, prediction reliability, immediate feasibility, and engineering safety boundaries are effectively unified into a single control solution. Simultaneously, the differentiable safety layer imposes higher costs on boundary crossings or near-boundary states, forming an auditable control chain that meets the conditions for going online.

[0118] In one embodiment, for reinforcement learning optimization, the control policy is ultimately driven by value learning. To continuously improve the robustness and economy of the control policy while satisfying engineering safety boundaries, this embodiment places the online control loop within a value-based continuous action reinforcement learning framework, and employs the TD3 algorithm's dual-commentator structure, target policy smoothing, delayed actor updates, and soft update mechanism to suppress... Q Overestimation and training oscillations. To ensure that scarce and critical challenging scenarios (such as formation jumps and attitude criticalities) are seen more frequently, Prioritized Experience Replay (PER) is introduced. Sampling is performed based on the magnitude of the TD error, and unbiasedness is corrected using importance weights. The TD error determines the priority, prioritizing samples with higher learning value (i.e., larger TD errors) for training to improve sample efficiency; simultaneously, importance weights are used to eliminate sampling bias, ensuring consistent final convergence. The priority is defined as:

[0119] (twenty two)

[0120] in, p i Indicates the first i The priority of each sample in the playback buffer; δ i Indicates the first i TD error for each sample; ε This indicates the prevention of small positive constants with a priority of 0. To prevent zero probability, we encourage exploration; This indicates a priority index, which controls the intensity of priority sampling. Control priority: Degenerate into uniform sampling; The larger the value, the more it is biased towards high-error samples.

[0121] The priority-based sampling probability is:

[0122] (twenty three)

[0123] in, j Represents the sample index in the buffer; ∑ jp j This represents the summation of the priorities of all samples.

[0124] We employ a segmented sum-tree or heap-based indexing approach to achieve a weighted sampling and priority update algorithm with a complexity of O(logN). Furthermore, we recommend setting the buffer size to 10. 5 Up to 10 6 .

[0125] Since non-uniform sampling introduces estimation bias, importance weights are assigned to samples during training to perform unbiased correction.

[0126] (twenty four)

[0127] in, w i Indicates the first i The importance weights of each sample are used to correct for biases caused by non-uniform sampling in the loss calculation; N Indicates the size of the playback buffer; β Indicating the importance sampling index, To compensate for estimation bias caused by non-uniform sampling, during the later stages of training... β Anneal to 1 to ensure unbiased convergence.

[0128] Based on the above sampling mechanism, the comprehensive expression of the TD target is:

[0129] (25)

[0130] in, y This represents the TD target value of the sample; r Indicates an immediate reward; γ Indicates the discount factor; Q This represents a network of two target commentators; and These represent the state and the target action at the next moment, respectively. P ( j ), w j This represents the sampling probability and importance weight; This represents the TD error of the j-th sample; ε It is a very small positive number, ensuring that all samples have a non-zero sampling frequency (as in Equation 22).

[0131] The actor strategy employs delayed updates combined with soft updates to improve stability. Offline training is conducted in parallel with prediction supervision, while online training continues with small steps to adapt to new strata. Soft updates refer to the way the target network parameters follow the updates of the main network, corresponding to formula (29): allowing the target network to slowly follow changes in the online network, rather than a one-time copy (i.e., hard updates).

[0132] To reduce the variance generated by the target policy and improve update stability, the TD target and update process employs methods such as target noise addition, minimum selection by two commentators, and delayed policy updates. Small noise is added to the target action and clipped, then adjusted according to amplitude. c Pruning, as a target strategy, smooths out overfitting in steep regions, thus mitigating overestimation.

[0133] (26)

[0134] in, Indicates the target policy network in state The action to be output; N (0, σ 2 () indicates a mean of 0 and a standard deviation of . σ 2 Gaussian noise; This indicates that the noise is clipped to [- c , c ] interval; Used to calculate the target Q value; σ , c These represent noise intensity and clipping threshold, respectively.

[0135] Two target critics and Minimize the value to systematically suppress Q-overestimation bias. Target Q value:

[0136] (27)

[0137] From PER w i The sample weights are multiplied by the mean squared error (MSE), and the gradients of the two critics are backpropagated separately to maintain parameter decoupling. Minimum range weight decay (L2 regularization) is applied to the output of Q, and the gradient norm is constrained to maintain numerical stability. Critics' loss (weighted MSE):

[0138] (28)

[0139] in, L Qk Indicates the first k The loss of a commentator; Indicates the first kA commentator on the sample Q-value estimation, This represents the expectation of the sampled state from the experience replay buffer. y i This represents the TD target value of the i-th sample.

[0140] Every training evaluator d 0 times (typical d (0=2), update the policy network and shared encoder again to maximize the first commenter's evaluation of the current policy. .

[0141] The target network parameters are soft-updated according to the following formula:

[0142] (29)

[0143] The target network slowly follows the main network to ensure the target's smoothness and stability; among which, θ Indicates the current network parameters; θ ' represents the corresponding target network parameters; τ This represents the soft update coefficient, with a value range of (0,1).

[0144] To unify prediction, control, and safety constraints under a single optimization objective, this embodiment superimposes task-level and engineering-level regularization terms onto the loss function of the TD3 algorithm, forming a joint objective function. By setting a multi-task joint loss function, it ensures that the predictor continuously learns trends and uncertainties, guarantees that value learning dominates control optimality, reduces counterintuitive behavior, and ensures that the final action does not deviate too far from the trend. Furthermore, the boundary and rate limits are written as differentiable penalties, forming a strong inhibition against out-of-bounds intentions. Under the combined effect of these four factors, the model achieves prediction, control, and safety constraints.

[0145] (30)

[0146] The combined effect of the above four factors enables the model to continuously learn while remaining strictly constrained during the online phase. It improves efficiency in predictable areas and becomes more conservative due to increased costs in unpredictable areas, ultimately achieving coordinated optimization and deployable convergence of prediction, control, and safety. Here, the... L This indicates the final combined total loss; λ pred Indicates the prediction weights. L pred This indicates the loss of the predicted head; L Qk This indicates a loss for the critics; a raw The fusion action corresponding to the time step or sample; a fThis represents the trend action component at the corresponding time step; This indicates a penalty for deviation between the action and the trend recommendation; P ( a ) represents the out-of-bounds and rate-limit penalties generated by the differentiable safety layer; λ Q , λ cons and λ safety These are respectively represented as the commentator loss weight, consistency penalty weight, and safety penalty weight.

[0147] In one embodiment, the reward function during policy exploration comprises: tracking accuracy, action smoothing plus energy consumption, constraint violation, uncertainty penalty, and prediction consistency.

[0148] Specifically, for the reward function, the principle is to prioritize engineering objectives, followed by behavioral smoothness, and with absolute priority given to safety red lines. The core objective is to stably reduce the deviations and fluctuations of key indicators such as attitude and propulsion speed without breaking constraints, while suppressing large and frequent control actions to avoid increased energy consumption and mechanical shock. It consists of the following five parts: tracking accuracy, minimizing the deviation of key production line indicators (such as attitude, cavity pressure, and propulsion speed) relative to reference values; action smoothing and energy consumption, penalizing action jumps and excessive amplitude to reduce chattering and energy consumption; constraint violation, imposing strong penalties for out-of-bounds actions, with heavier penalties for longer out-of-bounds periods; uncertainty penalty, reducing aggression in high-uncertainty regions; and prediction consistency, encouraging the control direction to align with short-term prediction gradients to avoid going against the grain. This step incorporates the engineering objectives of safety, stability, and efficiency into a single differentiable optima, ensuring that policy learning considers convergence, smoothness, and constraint conservation at every step. Based on this, the comprehensive expression of the reward function is:

[0149] (31)

[0150] in, r t Indicates time t Total instant rewards; Indicates time t Key production line indicator vectors; This represents the corresponding reference value vector; Indicates time t The actual control action vector; Indicates time t With time t The change in motion between -1 is used to measure control jumps and energy consumption; max (0, g m () indicates that only the portion exceeding the boundary will be penalized; Ut This represents a risk indicator obtained by aggregating predictive uncertainty. Indicates the direction of change in short-term forecasts. It represents the cosine of the angle between the control direction and the predicted trend, encouraging them to move in the same direction. λ q , λ s , λ c , λ u and λ pc These represent the weight coefficients of each of the above reward sub-items.

[0151] In one embodiment, the online adaptive update process may include:

[0152] Sliding window statistics are performed on uncertainty quantiles, out-of-bounds rate, and duration of missing data; the results of the sliding window statistics are used to indicate whether the system enters the safety priority mode.

[0153] While keeping the control period unchanged, samples are drawn from the playback buffer at a predetermined frequency, and small-step incremental learning is performed using the TD3 algorithm and the target and loss of priority experience playback.

[0154] Increase the priority and importance annealing coefficient of priority experience replay when there is a significant shift in the data domain; automatically back off the learning rate and exploration intensity when the model is detected to have stabilized again.

[0155] It is understandable that for online adaptive updates, the state of the formation and equipment may change at any time in actual production. This embodiment adds an adaptive and degradation mechanism to the inference end to improve operational robustness and controllability without changing the real-time nature of the online update.

[0156] First, the system performs sliding window statistics on three types of signals: uncertainty quantile, out-of-bounds rate, and duration of missing measurements. When the mean uncertainty or high quantile within the sliding window exceeds a set threshold, or out-of-bounds events are frequently triggered (e.g., the number of triggers within a set time exceeds a set threshold), or continuous missing measurements exceed the allowed duration, the system immediately enters the safety priority mode. The set threshold can be set according to the needs of engineering applications. In the safety priority mode, the safety penalty weight in the reward function is increased online (consistent with formula (31)), the constraint coefficient of the differentiable safety layer is increased (as in formula (21)), and the aggressive weight is reduced and the exploration noise is decreased in the policy fusion. At the same time, a rate cap and a return-to-median strategy are applied to the critical channels.

[0157] Secondly, to cope with slow drift, the system performs delayed online adaptive updates while keeping the control cycle unchanged: samples are extracted from the playback buffer at a predetermined frequency (e.g., 0.5Hz, 1Hz or other update frequencies set according to engineering requirements), and small-step incremental learning is performed using the target and loss of TD3+PER (e.g., formula (22)–(30)); the commentator and shared encoder are updated first, the policy network is updated after a delay of one round, and soft updates are used to pass to the target network to ensure that the inference end is unblocked and the parameters are smooth.

[0158] Furthermore, when there is a significant shift in the data domain (e.g., long-segment stratigraphic switching), the priority and importance annealing factor (i.e., importance sampling index) of PER can be appropriately increased. β This makes complex and newly distributed samples easier to see without over-biasing the estimates; when the model is detected to have re-stabilized, it automatically reverts to the normal (i.e., non-biased, stable) learning rate and exploration intensity.

[0159] Through the above adaptive and degradation design, the system maintains efficient control in the stable distribution region and automatically switches to safety priority and prompts for takeover when the distribution drifts or the observation deteriorates. Combined with the reward of formula (31) and the learning objectives of formulas (22) to (30), a predictive-control closed-loop strategy is formed, which meets the practical requirements of interpretability and verifiability for engineering implementation.

[0160] It should be understood that, although Figure 1 The steps are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order in which these steps are executed; they can be performed in other orders. Figure 1 At least some of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0161] In one embodiment, such as Figure 6As shown, a tunneling action generation system based on spatiotemporal deep fusion multi-task prediction is also provided, including a data preprocessing module 11, an encoding and representation module 13, a prediction uncertainty module 15, and a strategy fusion output module 17. The data preprocessing module 11 collects raw time-series data from the tunnel boring machine and performs data preprocessing. The preprocessed time series is then used to construct samples according to historical windows and prediction step sizes to obtain sample sequences with a unified time axis. The encoding and representation module 13 performs dual-path spatiotemporal encoding and dynamic deep fusion representation on the sample sequences to obtain sample-level semantic vectors; the dual-path spatiotemporal encoding includes local dependency paths and global dependency paths. The prediction uncertainty module 15 uses a multi-head self-attention module to perform global dependency modeling and temporal feature extraction on the sample-level semantic vectors, and then uses parallel multi-task output heads to perform multi-task prediction, model uncertainty estimation, and candidate continuous control action vector generation; the multi-task output heads include a prediction head, an uncertainty head, and a control head. The strategy fusion output module 17 is used to perform strategy fusion on the prediction head output, uncertainty head output and control head output using the strategy fusion network, and generate the final action that can be output to the shield machine actuator according to the task weight vector and safety constraints. Among them, the safety constraints include action amplitude limit, rate of change constraint, working condition safety domain constraint and uncertainty penalty. Based on the reinforcement learning framework, the strategy exploration is carried out by experience replay and noise annealing mechanism, and the final action is updated online adaptively.

[0162] The aforementioned tunneling action generation system based on spatiotemporal deep fusion multi-task prediction integrates local and global temporal information through a dynamic deep fusion network to achieve multi-spatiotemporal scale fusion, improving the perception capability of complex working conditions. It also outputs future state predictions and control candidates, and performs action fusion at the strategy layer to reduce performance loss caused by prediction-control separation, achieving prediction-control integration. Furthermore, it introduces an uncertainty head and explicitly constrains it in the loss function and strategy fusion layer, effectively suppressing risks under geological abrupt changes or sensor noise, achieving uncertainty constraint safety. Based on experience playback, weight adaptation, and noise removal, it meets the online adaptive update requirements of different geological formations and tunneling stages, enabling online adaptive updates and significantly improving the action generation capability for tunneling stability control under complex geological and noise conditions.

[0163] In one embodiment, the encoding representation module 13 includes: a local representation submodule, used in the local dependency path to first output the final encoded features of the input sample sequence through two layers of bidirectional LSTM, and then perform attention pooling, using attention weights to weighted summation of the hidden states at each time step to obtain the local representation vector; the first layer of the two-layer bidirectional LSTM is used to output the complete intermediate sequence features, and the second layer of the bidirectional LSTM is used to output the final encoded features. A global representation submodule, used in the global dependency path to send the input sample sequence after positional encoding and embedding to a multi-head self-attention module to obtain the temporal encoding tensor output by the global dependency path, and then use attention pooling to obtain the global representation vector. A representation fusion submodule, used to use adaptive gating to perform convex combination of the local representation vector and the global representation vector in the vector space to obtain the fused representation at the current time step. An information interaction submodule, used to use cross attention to perform information interaction between the local representation vector, the global representation vector, and the fused representation at the current time step, and then use cross-layer attention to read out the sample-level semantic vector.

[0164] It is understood that the explanations of the features in the above-mentioned tunneling action generation system based on spatiotemporal deep fusion multi-task prediction can be understood by referring to the corresponding explanations in the various embodiments of the tunneling action generation method based on spatiotemporal deep fusion multi-task prediction.

[0165] Each module in the aforementioned spatiotemporal deep fusion multi-task prediction tunneling action generation system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of a device with data processing capabilities, or stored in software within the memory of the aforementioned device, so that the processor can call and execute the operations corresponding to each module. The aforementioned device can be, but is not limited to, various types of data processing system devices already existing in the field.

[0166] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), memory bus DRAM (RDRAM), and interface DRAM (DRDRAM), etc.

[0167] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0168] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of protection of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and all such modifications and improvements fall within the scope of protection of the present invention.

Claims

1. A method for generating tunneling actions based on spatiotemporal deep fusion multi-task prediction, characterized in that, Including the following steps: Raw time series data is collected from the tunnel boring machine and preprocessed. The preprocessed time series is then used to construct samples according to historical windows and prediction steps to obtain sample sequences with a unified time axis. The sample sequence is represented by dual-path spatiotemporal coding and dynamic deep fusion to obtain sample-level semantic vectors; the dual-path spatiotemporal coding includes local dependency path and global dependency path. After performing global dependency modeling and temporal feature extraction on sample-level semantic vectors using a multi-head self-attention module, multi-task prediction, model uncertainty estimation, and candidate continuous control action vector generation are performed using parallel multi-task output heads; the multi-task output heads include a prediction head, an uncertainty head, and a control head; A policy fusion network is used to perform policy fusion on the prediction head output, uncertainty head output, and control head output. Based on the task weight vector and safety constraints, the final action that can be output to the tunnel boring machine actuator is generated. The safety constraints include action amplitude limit, rate of change constraint, working condition safety domain constraint, and uncertainty penalty. Based on the reinforcement learning framework, experience replay and noise annealing mechanisms are used to explore policies and the final action is updated online adaptively. The steps involved in performing dual-path spatiotemporal coding and dynamic deep fusion representation on the sample sequence to obtain the sample-level semantic vector include: In the local dependency path, the input sample sequence is first passed through two layers of bidirectional LSTM to output the final encoded features and then attention convergence is performed. The attention weights are used to sum the hidden states at each time step to obtain the local representation vector. The first layer of bidirectional LSTM is used to output the complete intermediate sequence features, and the second layer of bidirectional LSTM is used to output the final encoded features. In the global dependency path, the input sample sequence is encoded and embedded at position and then enters the multi-head self-attention module to obtain the temporal encoding tensor output by the global dependency path. The temporal encoding tensor is then processed by attention pooling to obtain the global representation vector. By using adaptive gating to perform a convex combination of the local and global representation vectors in the vector space, the fused representation at the current time step is obtained. After using cross-attention to exchange information between the local representation vector, the global representation vector and the fused representation at the current time step, the sample-level semantic vector is read out using cross-layer attention.

2. The method for generating tunneling actions based on spatiotemporal deep fusion multi-task prediction according to claim 1, characterized in that, The raw time series data includes the tunnel boring machine's advance speed, cutterhead torque, total thrust, cutterhead rotation speed, cutterhead penetration depth, and attitude angle; Data preprocessing includes: The original time series data is resampled onto the same time axis, and short-term interruptions are synchronously filled in according to specified interpolation rules; the specified interpolation rules include linear interpolation rules and zero-order hold interpolation rules; Outliers are truncated using a quantile interval-based truncation strategy to limit the data of each channel to the range of [Q1-1.5IQR, Q3+1.5IQR]; IQR is the interquartile range, Q3 is the third quartile, and Q1 is the first quartile. Feature representations are extracted from the aligned and cleaned time series based on mutual information scores; the feature representations include basic statistical features, dynamic difference features, and mutual information correlation features.

3. The method for generating tunneling actions based on spatiotemporal deep fusion multi-task prediction according to claim 2, characterized in that, Before constructing the samples, z-score standardization is performed on the time series of each channel, and the continuous time series is divided into sample sequences using a sliding window method.

4. The method for generating tunneling actions based on spatiotemporal deep fusion multi-task prediction according to claim 1, characterized in that, The process of generating the final actions that can be output to the tunnel boring machine's actuators based on the task weight vector and safety constraints includes: A differentiable safety layer is used to project the fused action obtained based on the action fusion weight vector onto the engineering feasible set jointly defined by amplitude, rate of change and operating domain, to obtain the final action.

5. The method for generating tunneling actions based on spatiotemporal deep fusion multi-task prediction according to claim 1, characterized in that, The reinforcement learning framework employs the dual-commentator structure, target policy smoothing, delayed actor update, and soft update mechanism of the TD3 algorithm to suppress Q-value overestimation and training oscillations; it uses summation trees or heap indexes to achieve weighted sampling and priority update of algorithm complexity.

6. The method for generating tunneling actions based on spatiotemporal deep fusion multi-task prediction according to claim 5, characterized in that, The reward function during policy exploration consists of: tracking accuracy, action smoothing plus energy consumption, constraint violation, uncertainty penalty, and prediction consistency.

7. The method for generating tunneling actions based on spatiotemporal deep fusion multi-task prediction according to claim 5, characterized in that, The online adaptive update process includes: Sliding window statistics are performed on uncertainty quantiles, out-of-bounds rate, and duration of missing data; the results of the sliding window statistics are used to indicate whether the system enters the safety priority mode. While keeping the control period unchanged, samples are drawn from the playback buffer at a predetermined frequency, and small-step incremental learning is performed using the TD3 algorithm and the target and loss of priority experience playback. Increase the priority and importance annealing coefficient of priority experience replay when there is a significant shift in the data domain; automatically back off the learning rate and exploration intensity when the model is detected to have stabilized again.

8. A tunneling action generation system based on spatiotemporal deep fusion multi-task prediction, characterized in that, include: The data preprocessing module is used to collect raw time series data from the tunnel boring machine and perform data preprocessing. The preprocessed time series is used to construct samples according to historical windows and prediction steps to obtain sample sequences with a unified time axis. The encoding and representation module is used to perform dual-path spatiotemporal encoding and dynamic deep fusion representation on the sample sequence to obtain sample-level semantic vectors; the dual-path spatiotemporal encoding includes local dependency path and global dependency path. The prediction uncertainty module is used to perform global dependency modeling and temporal feature extraction on sample-level semantic vectors using a multi-head self-attention module, and then use parallel multi-task output heads to perform multi-task prediction, model uncertainty estimation, and generation of candidate continuous control action vectors respectively; the multi-task output heads include a prediction head, an uncertainty head, and a control head; The strategy fusion output module is used to perform strategy fusion on the prediction head output, uncertainty head output and control head output using a strategy fusion network. Based on the task weight vector and safety constraints, it generates the final action that can be output to the tunnel boring machine actuator. The safety constraints include action amplitude limit, rate of change constraint, working condition safety domain constraint and uncertainty penalty. Based on the reinforcement learning framework, the module uses experience replay and noise annealing mechanism to explore strategies and adaptively update the final action online. The encoding representation module includes: The local representation submodule is used in the local dependency path to first output the final encoded features of the input sample sequence through two layers of bidirectional LSTM, and then perform attention convergence. The attention weights are used to sum the hidden states at each time step to obtain the local representation vector. The first layer of bidirectional LSTM is used to output the complete intermediate sequence features, and the second layer of bidirectional LSTM is used to output the final encoded features. The global representation submodule is used to input the sample sequence in the global dependency path, after position encoding and embedding, and then feed it into the multi-head self-attention module to obtain the temporal encoding tensor output by the global dependency path. The temporal encoding tensor is then processed by attention pooling to obtain the global representation vector. The fusion submodule is used to perform a convex combination of the local and global representation vectors in the vector space using adaptive gating to obtain the fused representation at the current time. The information interaction submodule is used to perform information interaction between the local representation vector, the global representation vector and the fused representation at the current time using cross attention, and then read out the sample-level semantic vector using cross-layer attention.

Images