Multi-modal data fusion shield muck character monitoring and over-excavation prediction method and system

Abstract

The invention belongs to the related technical field of tunnel engineering and artificial intelligence, and discloses a multi-modal data fusion shield muck property monitoring and over-excavation prediction method and system. The method comprises the following steps: S1, performing data fusion on muck information and shield tunneling machine information in the tunneling process of a shield tunneling machine to form a feature vector; s2, performing dimensionality reduction on the feature vector, and inputting the feature vector after dimensionality reduction into a Bi-LSTM model to obtain a future prediction value of tunneling overexcavation of the shield tunneling machine; wherein each layer or the last layer in the Bi-LSTM model is provided with a Monte Carlo Dropout layer, so that different neurons are randomly discarded; and S3, returning to the step S1 until a preset number of times is reached, thereby obtaining a plurality of future prediction values of tunneling overexcavation of the shield tunneling machine, and taking a mean value of the plurality of prediction values as a final prediction result. According to the invention, the problems of an overfitting phenomenon and insufficient generalization ability in a neural network model prediction process are solved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of tunnel engineering and artificial intelligence, and more particularly relates to a shield spoil property monitoring and overbreak prediction method and system based on multi-modal data fusion. BACKGROUND

[0002] During construction, the stratum through which the shield machine passes is often complex and changeable, and accurate and timely judgment of the improvement state of the spoil is a guarantee for the safe and stable construction of the earth pressure balance shield machine. The spoil is difficult to improve during shield tunneling, and the cutter torque and tunneling speed fluctuate greatly, making it difficult to control the spoil amount during tunneling. Overbreakage of the spoil amount during tunneling will form a stratum cavity, which will then cause ground subsidence or damage to surrounding buildings and underground pipelines, etc. The judgment of whether overbreakage occurs during shield construction and the corresponding stratum conditions obtained through spoil analysis can prevent accidents caused by stratum instability at the tunnel excavation face. Accurate and timely acquisition of spoil information plays a crucial role in the safety of shield construction, but the detection of spoil information requires a large amount of manual work and time, making it difficult to perform real-time detection at a high frequency. Therefore, a shield spoil property monitoring and overbreak prediction method based on multi-modal data fusion is of great help to shield construction.

[0003] Traditional shield spoil property monitoring and overbreakage discrimination mainly rely on on-site human calculation and experience-based discrimination, and limited intelligent monitoring methods only measure a single spoil property. Traditional prediction methods fail to capture the relationship between multiple shield spoil property data and overbreakage-related influencing factors and overbreakage, and lack comprehensive analysis of multi-modal data, resulting in low prediction accuracy, short early warning time, etc. With the continuous development of neural network models, overbreakage prediction of shields through neural network modeling and multi-modal data fusion can improve the accuracy and generalization ability of overbreakage prediction, overcome the shortcomings of traditional methods, and has a wide application prospect.

[0004] However, how to efficiently acquire the spoil information of the shield is a difficult problem, and how to well fuse and process multi-modal data is a big problem. In addition, in neural networks, due to the complexity of the model and the large number of parameters, overfitting is a serious problem, which leads to insufficient generalization ability of the model, so the uncertainty and confidence evaluation of the neural network model is also an important problem. Confidence evaluation and uncertainty estimation of the prediction results of the model are particularly important. Therefore, new neural network models and methods need to consider how to reduce the risk of overfitting while enhancing the uncertainty estimation and confidence evaluation ability of the model. SUMMARY

[0005] In order to solve the above defects or improvement needs of the prior art, the present application provides a shield muck property monitoring and overbreak prediction method and system based on multi-modal data fusion, which solves the problems of overfitting and insufficient generalization ability in the neural network model prediction process.

[0006] To achieve the above-mentioned purpose, according to one aspect of the present application, a shield muck property monitoring and overbreak prediction method based on multi-modal data fusion is provided, which comprises the following steps: S1 fusing the muck information and the shield machine information during the shield machine tunneling process to form a feature vector; S2 dimensionally reducing the feature vector and inputting the dimensionally reduced feature vector into a Bi-LSTM model to obtain a future prediction value of the shield machine tunneling overbreak; wherein each layer or the last layer of the Bi-LSTM model is provided with a Monte Carlo Dropout layer to randomly discard different neurons; S3 returning to step S1 until a predetermined number of times are reached, thereby obtaining a plurality of future prediction values of the shield machine tunneling overbreak, and the mean of the plurality of prediction values is taken as the final prediction result.

[0007] Further preferably, in step S2, the training data in the training process of the Bi-LSTM model is the muck information and the shield machine information corresponding to each ring in the subway tunnel construction process, and the overbreak amount corresponding to each ring.

[0008] Further preferably, the Bi-LSTM model is provided with an Adam optimizer, and the loss function of the Bi-LSTM model is the mean square error.

[0009] Further preferably, in step S2, the principal component analysis method is used for dimensionally reducing the feature vector.

[0010] Further preferably, in step S1, the muck information includes muck volume, muck weight, muck flow plasticity, muck theoretical volume and muck theoretical weight, and the shield machine information includes shield machine tunneling speed and shield machine tunneling parameters, wherein the muck volume and the muck weight are numerical data, the muck flow plasticity is image data, and the muck theoretical volume, the muck theoretical weight, the shield machine tunneling speed and the shield machine tunneling parameters are text data.

[0011] Further preferably, in step S1, the data fusion is performed by using a convolutional neural network, and the numerical data, the image data and the text data are respectively input into independent convolution branches of the convolutional neural network to obtain the required feature vector after merging the extracted feature values.

[0012] More preferably, in step S1, before the data fusion, the information on slag and tunnel boring machine needs to be preprocessed, specifically by denoising the numerical data, removing duplicate observation data, interpolating and supplementing missing values, standardizing and normalizing them; adjusting the size of the images in the image data; and quantifying the text data.

[0013] According to another aspect of the present invention, a multimodal data fusion-based system for monitoring the properties of tunnel boring machine excavated soil and predicting over-excavation is provided. This system includes a data acquisition module, a data processing module, and a prediction module, wherein: The data acquisition module is used to acquire information on excavated soil and tunnel boring machine (TBM) during the tunneling process; The data processing module is used to fuse the information acquired by the data acquisition module to form a feature vector, and then perform dimensionality reduction on the feature vector; The prediction module is used to predict the data from the data processing module to obtain the future predicted value of over-excavation during tunnel boring machine excavation, and to calculate the average value of multiple predicted values.

[0014] According to another aspect of the present invention, a shield tunneling muck property monitoring and over-excavation prediction system based on multimodal data fusion is provided. The system includes an actuator for executing the aforementioned method for monitoring and predicting shield tunneling muck property based on multimodal data fusion.

[0015] According to another aspect of the present invention, a computer storage medium is provided having a computer program stored thereon, which, when executed, is used to implement the above-described method for monitoring the properties of tunnel boring machine excavated soil and predicting over-excavation through multimodal data fusion.

[0016] In summary, the technical solutions conceived by this invention have the following beneficial effects compared with the prior art: 1. This invention incorporates Monte Carlo Dropout layers in each or the last layer of a Bi-LSTM model, randomly discarding different neurons. The Monte Carlo Dropout technique is used for uncertainty assessment and confidence scoring. This technique can evaluate the uncertainty of neural network models, including the variability of input data, uncertainties in model parameters and structure, etc., obtaining reliable prediction confidence assessments. Compared with traditional prediction methods, it more accurately identifies and assesses uncertainty, resulting in better prediction performance and effectively solving the problems of overfitting and insufficient generalization ability in the prediction process of neural network models.

[0017] 2. In this invention, a convolutional neural network is used to fuse numerical data, image data, and text data. On the one hand, it can integrate multiple influencing factors related to the prediction of tunnel boring machine over-excavation, thereby more comprehensively describing the complexity of over-excavation prediction. On the other hand, since each CNN branch corresponds to a different type of input data, the interaction relationship and spatiotemporal features between different types of data can be further extracted, which can more accurately reflect the regularity of tunnel boring machine over-excavation and greatly improve the accuracy and generalization ability of tunnel boring machine over-excavation.

[0018] 3. This invention proposes an intelligent monitoring and measurement system for real-time measurement of the properties of excavated soil during the tunneling process of an earth pressure balance shield machine. Its function is to acquire data for over-excavation prediction of the shield machine in real time, improve the efficiency of data collection and preprocessing, and add parameters such as excavated soil volume, weight, and fluidity compared to traditional shield machine parameter data, making the subsequent prediction results more accurate. Attached Figure Description

[0019] Figure 1 This is a flowchart of a method for monitoring the properties of tunnel boring machine excavation soil and predicting over-excavation, based on a preferred embodiment of the present invention, using multimodal data fusion.

[0020] Figure 2 This is a flowchart of data fusion constructed according to a preferred embodiment of the present invention.

[0021] Figure 3 This is a schematic diagram of the location of a measuring device in the transportation of construction waste, constructed according to a preferred embodiment of the present invention.

[0022] Figure 4 This is a schematic diagram of the layout of the measurement system during the transportation of construction waste, constructed according to a preferred embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0024] like Figure 1 As shown, a method for monitoring the properties of shield tunnel excavation waste and predicting over-excavation based on multimodal data fusion is presented. This method includes the following steps: S1. Data Acquisition Data from multiple dimensions, including the volume and weight of excavated soil, images of soil flow plasticity, tunneling parameters, geological survey report data, engineering design document data, and historical construction data, were acquired during the tunneling process of the tunnel boring machine (TBM). Preprocessing was performed on the data from these dimensions. Specifically, the excavated soil volume, weight, and TBM tunneling parameters (propulsion force, cutting force, cutterhead rotation speed, propulsion speed, and earth pressure) were obtained from sensor numerical data; the soil flow plasticity images were obtained from image data; and the geological survey report data (soil characteristics, groundwater level, and potential risk areas), engineering design document data (theoretical earthwork volume and theoretical soil weight), and historical construction data (textual records of construction under similar conditions) were obtained from textual data.

[0025] 1.1 The specific steps for collecting initial data are as follows: (1) The line laser emitter and monocular industrial camera are fixed above the conveyor belt using the line laser three-dimensional measurement method. Appropriate supplementary lighting facilities are constructed based on the lighting environment at the tunnel boring machine site. To address potential soil splashing during transport, the industrial camera lens is oriented towards the conveyor belt's direction of movement, and a baffle is installed. After construction, the system parameters need to be calibrated, including camera calibration and optical plane calibration, to determine the camera's built-in parameters and optical plane coordinate data. For example... Figure 4 The measurement system layout is shown in the figure.

[0026] (2) During the measurement of slag and soil, the slag and soil move at a belt speed v along the direction perpendicular to the laser plane. The belt speed is obtained by the speed sensor. The camera collects the laser stripe image projected on the material surface by the laser at a specific frame rate and transmits the image data to the computer in the form of video. Since there may be occlusion, cameras with different angles can be set.

[0027] (3) The video captured by the camera, which includes laser scanning, is processed using image processing algorithms to extract the pixel coordinate data of the center of the laser stripe in each frame. Combined with the parameters obtained from calibration, the actual spatial coordinate data of the slag contour is obtained through external computer algorithms. A 3D point cloud model of the slag under test is reconstructed, and the point cloud data results formed from multiple perspectives are fused using point cloud processing software. The obtained point cloud model is then preprocessed to obtain a more accurate point cloud model that includes the slag under test and the conveyor belt.

[0028] (4) When the conveyor belt is unloaded, the same method is used to measure the conveyor belt to obtain the point cloud model of the unloaded conveyor belt. By fitting the two point cloud models, a point cloud model with only the measured slag is obtained. Then, the point cloud data is directly manipulated. The Delaunay triangulation method is used to segment the point cloud to calculate the volume, and then the volume of the measured slag is obtained.

[0029] (5) During the transportation of excavated soil, a gravity sensor generates an analog voltage signal, a speed sensor measures the speed of the conveyor belt, and the controller fuses the collected weight and speed information to calculate the instantaneous weight and cumulative weight, thus obtaining the amount of excavated soil. Figure 3 The measuring device shown.

[0030] (6) Using the video captured by the industrial camera, extract the relevant slag images at a certain frame rate, use machine vision methods and engineering experience to classify and label the slag images to form a dataset for slag flow plasticity detection, and use machine learning methods to determine the slag flow plasticity (dry, wet, suitable).

[0031] 1.2 Data Preprocessing (1) Numerical data processing After the initial data collection is completed, data cleaning is performed on different types of data to remove outliers and noise and extract effective features.

[0032] A threshold-based denoising method is used, with the following formula:

[0033] Where, x t Represents the original data, y t The data is denoised using a threshold-based denoising method. ε is the set threshold. If the difference between adjacent data exceeds the set threshold, the data is considered noise and assigned a value of 0.

[0034] To determine if duplicate observations exist, use the duplicated method to remove duplicate observations.

[0035] To determine if missing values ​​exist, interpolation is used to handle missing values ​​in each dimension. Specifically, handling missing data is necessary because some data may be considered noise (zero) after processing for acquisition errors, and missing data can also occur during modeling. Linear interpolation can be used to handle these missing data. Given the data at time t and t+mΔt, and the data at t+nΔt representing the data lost between time t and t+mΔt, the data at t+nΔt can be calculated using linear interpolation as follows:

[0036] Numerical data needs to be standardized, which is achieved through the Z-score standardization method, i.e.: Z = X - μ / σ, where X is the original data, μ is the mean, and σ is the standard deviation.

[0037] The numerical data were normalized using the Min-max method. Specifically, x...g x represents the normalized data; x represents the original data in the sample set; x min and x max These are the maximum and minimum values ​​in the sample set, respectively.

[0038]

[0039] Perform a reshape operation to convert the sensor data from a one-dimensional array into a column vector.

[0040] (2) Image data processing First, adjust the pixels: adjust the image to 224*224 pixels, then convert the format to NumPy array format so that it can be input into the neural network. Finally, normalize the image by dividing the image array by 255, scaling the pixel values ​​of the image from the range of [0,255] to [0, 1]; thus creating a normalized array.

[0041] (3) Processing of text data The original text is converted into a format that can be processed by the neural network, and appropriate numerical processing is performed. Specifically: a tokenizer is created and the vocabulary size is limited to 100 (only the top 100 most frequent words are considered). The text data is trained using `fit_on_texts` to extract the corresponding words from the text, and a vocabulary is built based on the frequency of each word in the text, assigning a unique index number. The `texts_to_sequences` method replaces each word in the text data with its corresponding numerical index, converting the text data into a numerical sequence. Each element of this numerical sequence represents the index of a word in the vocabulary, thus converting it into a vector form. Finally, `pad_sequences` pads or truncates all text sequences to the same length, forming vectors of the same length. Each sentence in the text is converted into a vector, with the number of vectors equal to the number of sentences, and each sentence's converted vector has the same length.

[0042] S2 data fusion The preprocessed data is then processed using a Convolutional Neural Network (CNN) to extract features from multiple dimensions and fuse multimodal data. Specifically, the multidimensional data of parameters and soil properties obtained during the tunnel boring machine's excavation process are treated as different influencing factors. Multiple parallel convolutional branches are used to process these different influencing factors, and the feature vector lists of all convolutional branches are then merged and fused.

[0043] like Figure 2As shown, a Convolutional Neural Network (CNN) model is built, comprising three convolutional branches, with independent branches designed for each type of input data (image, numerical, and text data). These branches operate independently during feature extraction, focusing on extracting the most representative features from their respective modalities. These convolutional branches are parallel, merging after each branch has independently extracted its features. Specifically: (1) Image data processing convolutional neural network branch The branch input shape is (224, 224, 3), and the output is a 128-dimensional vector. The neural network layers are sequentially combined using the Sequential function. In the image branch, three convolutional layers (Conv2D) are set up with ReLU activation. The first convolutional layer has 32 neurons and a 3x3 kernel, the second has 64 neurons and a 3x3 kernel, and the third has 128 neurons and a 3x3 kernel. Each convolutional layer is followed by a 2x2 pooling layer (MaxPooling2D). After the three convolutional layers, a flattening layer is connected to convert the 2D feature map after convolution and pooling operations into a 1D vector, which serves as the input to the fully connected layer. The fully connected layer has 128 neurons and ReLU activation. Finally, a Dropout layer with a 50% dropout rate is connected to prevent overfitting during model training.

[0044] (2) Sensor data convolutional neural network branch After the reshape operation in the sensor data preprocessing, the input shape of this branch is (number of vectors processed by the CNN model at one time, number of elements in the vector, number of physical meanings represented by each element), where batch_size=1, sequence_length=7, and num_features=1. The neural network layers are sequentially combined using the Sequential function in the sensor branch. Two convolutional layers (Conv1D) are set up, using ReLU activation. The first convolutional layer has 64 neurons and a kernel size of 3, while the second convolutional layer has 128 neurons and a kernel size of 3. Each convolutional layer is followed by a pooling layer (MaxPooling1D) with a size of 2. After the two convolutional layers, a flattening layer is connected to convert the output of the convolution and pooling operations into a one-dimensional feature vector, which serves as the input to the fully connected layer. The fully connected layer has 128 neurons, using ReLU activation, and is finally connected to a Dropout layer with a dropout rate of 50%.

[0045] (3) Text Data Convolutional Neural Network Branch The input layer has a shape of (max_text_length,)max_text_length=20. The neural network layers are sequentially combined using the Sequential function. In the text data branch, a word embedding layer is set up, mapping the integer index of each word to a 128-dimensional dense vector. The word embedding dimension is 128, meaning each word is represented by a 128-dimensional vector. A convolutional layer (Conv1D) is set up, using ReLU activation. The first convolutional layer has 64 layers and a kernel size of 5. After the convolutional layer, a pooling layer (GlobalMaxPooling1D) is set up to extract global features of the text sequence. Finally, a Dropout layer is connected with a dropout rate of 50%.

[0046] Each branch is trained separately using its corresponding training data and labels, creating a convolutional neural network (CNN) to process image, sensor, and text data, extracting features for each type. Once each branch is trained, its output feature vector is obtained. These feature vectors are then merged to form a comprehensive feature vector containing multimodal information. Specifically, the image data, sensor data, and text data each yield a 128-dimensional feature vector. The text data and the text data each yield a 64-dimensional feature vector. The `Concatenate` function is used to concatenate these vectors into a larger vector, which serves as input to subsequent models. Finally, the three feature vectors are concatenated along their last dimension (the feature dimension) to obtain a 320-dimensional feature vector. A single time step results in a single total feature vector, and multiple time steps correspond to multiple total feature vectors.

[0047] S3 eigenvector dimensionality reduction processing The eigenvectors obtained from the data fusion are linearly reduced in dimensionality using Principal Component Analysis (PCA). The total eigenvectors at each time step are then reduced in dimensionality to obtain the dimensionality-reduced vectors at each time step.

[0048] S4 Prediction Results The feature vector, after multimodal data fusion via CNN, is dimensionality-reduced using PCA and then input into a bidirectional long short-term memory (Bi-LSTM) model for training. This model is bidirectional, consisting of two LSTMs: one processing the forward sequence and the other processing the backward sequence. This allows the Bi-LSTM to consider information from both the previous and next time steps when processing data at the current time step, thus better capturing the bidirectional dependencies in the sequence. The input vector to the Bi-LSTM model is the feature vector after multimodal data fusion and dimensionality reduction, and the output vector is the predicted future value of the tunnel boring machine's over-excavation.

[0049] (1) Constructing a Bi-LSTM model: After obtaining the feature vector through PCA dimensionality reduction, define an input layer of a Bi-LSTM model with the input shape being the feature dimension after dimensionality reduction. Add a fully connected layer (DenseLayer) with 128 neurons and ReLU activation function after the input layer. Add a Dropout layer after the fully connected layer to prevent overfitting with a dropout rate of 50%. Set up a bidirectional LSTM layer with 64 LSTM units to return the output layer of the last time step. Over-excavation prediction of the tunnel boring machine is a regression task. Set up a neuron in the output layer to represent the predicted value of the over-excavation, and use the linear activation function. Use the Model function to connect the input layer and the output layer to form a complete neural network model. At the same time, configure the Adam optimizer and loss function (using mean squared error MSE) in the network model for subsequent model training.

[0050] (2) Model dataset Training data source: Data from past time points is used. Label data source: The excavation volume after tunneling by the tunnel boring machine at each time point needs to be compared with the theoretical earthwork volume to determine the actual over-excavation volume. This data is recorded in real-time during construction and time-aligned with other data. Since this problem involves the tunneling process of the tunnel boring machine, the data is time-series data, and its statistical characteristics may change over time. In this case, time-series partitioning is used. Training set: Training is performed using data from earlier time periods.

[0051] Validation set: Data from the middle time period is used to validate the model's generalization ability.

[0052] Test set: Use the latest time period data for testing to ensure that the model performs well on future data.

[0053] During training, the `fit()` function is used, requiring the specification of the number of training epochs, the number of samples used per batch, and callback functions. The test set is then used to evaluate the model's final performance.

[0054] (3) Model improvement Monte Carlo Dropout technology is introduced into the Bi-LSTM model for over-excavation prediction of tunnel boring machines (TBMs). Dropout layers are introduced at each or the last layer, and different neurons are randomly dropped multiple times in the model. Multiple prediction results are sampled, and the uncertainty assessment and confidence score of the TBM over-excavation prediction results are performed. This allows for more accurate identification and evaluation of the uncertainty of TBM over-excavation, providing prediction and decision-making for TBM over-excavation early warning in tunnel construction.

[0055] Dropout is a regularization technique designed to prevent overfitting. During training, it prevents the model from depending on certain neurons by randomly dropping a subset of neurons. During testing, Dropout is typically disabled, allowing the model to utilize all neurons for prediction. However, the core of Monte Carlo Dropout lies in maintaining Dropout activation during the testing phase. This allows for multiple forward propagations to simulate different network structures, generating multiple predictions to assess the model's uncertainty.

[0056] To modify the Bi-LSTM model to support Dropout, a Monte Carlo Dropout function (monte_carlo_dropout) is first defined. This function contains the trained Bi-LSTM model, the input data, and the number of predictions, ultimately returning the results of multiple predictions. Then, a low-level API from the Keras backend, K.function, is used to create a function that maintains the Dropout layer's functionality during each forward propagation. The main idea is to set the training flag (K.learning_phase()) to 1, ensuring that Dropout is enabled during the prediction phase.

[0057] Monte Carlo sampling is performed, and the forward propagation is executed multiple times during the testing phase using the `monte_carlo_dropout` function, yielding different prediction results each time. Each input feature data generates multiple predicted values ​​for shield tunneling over-excavation.

[0058] The mean and standard deviation of the predictions are calculated. The average of multiple predictions represents the model's final prediction of the tunnel boring machine's over-excavation. The standard deviation of multiple predictions represents the model's uncertainty in that prediction.

[0059] Results Output and Evaluation: With the predicted mean and standard deviation of uncertainty, each prediction can be output, along with a confidence score. When evaluating predictions, a confidence threshold is defined. The uncertainty of the model's prediction is judged based on the standard deviation, converting uncertainty into confidence levels; the smaller the standard deviation, the higher the confidence level. Decisions are made based on the confidence levels, providing a reliable assessment for predicting tunnel boring machine over-excavation. Decisions are made based on the predicted values ​​and uncertainties, combined with the actual engineering conditions. For example, if the uncertainty of certain predictions is high, monitoring and early warning for these areas can be enhanced.

[0060] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for monitoring the properties of shield tunnel excavated soil and predicting over-excavation based on multimodal data fusion, characterized in that, The method includes the following steps: S1 fuses the information on excavated soil and tunnel boring machine (TBM) during the tunneling process to form a feature vector; S2 reduces the dimensionality of the feature vector and inputs the reduced feature vector into the Bi-LSTM model to obtain multiple predicted values ​​of shield machine over-excavation. The mean of these multiple predicted values ​​and the confidence value corresponding to the mean are calculated, and the mean is taken as the final prediction result of shield machine over-excavation. In the Bi-LSTM model, each layer or the last layer is equipped with a Monte Carlo Dropout layer to randomly discard different neurons.

2. The method for monitoring shield tunneling spoil properties and predicting over-excavation based on multimodal data fusion as described in claim 1, characterized in that, In step S2, the training data for the Bi-LSTM model training process includes the excavated soil information and tunnel boring machine information corresponding to each ring in the subway tunnel construction process, as well as the over-excavation amount corresponding to each ring.

3. A method for monitoring and predicting over-excavation of shield tunnel excavation waste soil using multimodal data fusion as described in claim 1 or 2, characterized in that, The Bi-LSTM model incorporates an Adam optimizer, and its loss function is the mean squared error.

4. The method for monitoring shield tunneling spoil properties and predicting over-excavation based on multimodal data fusion as described in claim 1, characterized in that, In step S2, principal component analysis is used to reduce the dimensionality of the feature vector.

5. The method for monitoring shield tunneling spoil properties and predicting over-excavation based on multimodal data fusion as described in claim 1, characterized in that, In step S1, the excavated soil information includes excavated soil volume, excavated soil weight, excavated soil fluidity, theoretical excavated soil volume, and theoretical excavated soil weight. The tunnel boring machine information includes tunnel boring machine speed and tunnel boring machine parameters. The excavated soil volume and weight are numerical data, the excavated soil fluidity is image data, and the theoretical excavated soil volume, theoretical excavated soil weight, tunnel boring machine speed, and tunnel boring machine parameters are text data.

6. The method for monitoring shield tunneling spoil properties and predicting over-excavation based on multimodal data fusion as described in claim 5, characterized in that, In step S1, the data fusion is performed using a convolutional neural network. Numerical data, image data, and text data are input into separate convolutional branches of the convolutional neural network, and after extracting feature values ​​separately, they are merged to obtain the required feature vector.

7. A method for monitoring and predicting over-excavation of shield tunnel excavation waste soil using multimodal data fusion as described in claim 5 or 6, characterized in that, In step S1, before the data fusion, the information on slag and tunnel boring machine needs to be preprocessed. Specifically, the numerical data is denoised, duplicate observation data is removed, and missing values ​​are interpolated, standardized, and normalized. The size of the images in the image data is adjusted, and the text data is numerically processed.

8. A multimodal data fusion-based system for monitoring the properties of tunnel boring machine excavated soil and predicting over-excavation, characterized in that, The system includes a data acquisition module, a data processing module, and a prediction module, wherein: The data acquisition module is used to acquire information on excavated soil and tunnel boring machine (TBM) during the tunneling process; The data processing module is used to fuse the information acquired by the data acquisition module to form a feature vector, and then perform dimensionality reduction on the feature vector; The prediction module is used to predict the data from the data processing module to obtain the future predicted value of over-excavation during tunnel boring machine excavation, and to calculate the average value of multiple predicted values.

9. A multimodal data fusion-based system for monitoring the properties of tunnel boring machine excavated soil and predicting over-excavation, characterized in that, The system includes an actuator for performing a method for monitoring the properties of tunnel boring machine excavated soil and predicting over-excavation based on multimodal data fusion as described in any one of claims 1-7.

10. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it is used to implement the method for monitoring the properties of shield tunnel excavated soil and predicting over-excavation based on multimodal data fusion as described in any one of claims 1-7.

Images