A method for identifying soil type at excavation surface of earth pressure balance shield

Abstract

The present invention relates to the field of earth pressure balance shield tunnel construction. A method for identifying soil types in an earth pressure balance shield excavation face is proposed, which is characterized by comprising the following steps: step 1, establishing a slag classification system; step 2, establishing a slag identification database; step 3, constructing a slag identification model. The method of the present invention can not only scientifically reflect the properties of the soil itself but also be applicable to the actual needs of shield construction. The constructed slag visual identification criteria provide a basis for the production of data set labels; the deep learning algorithm can be used to quickly and accurately obtain soil type information in front of the excavation face by identifying the slag image, and has the characteristics of real-time and low cost.

Description

Technical Field

[0001] The invention relates to the field of earth pressure balance shield tunnel construction. Background Art

[0002] With the continuous acceleration of China's urbanization process, the mileage and scale of subways have also expanded rapidly, which has also brought great challenges to the safety of tunnel construction and the control of the impact on the surrounding environment. The shield method is widely used in the construction of subway tunnels because of its fast construction speed, small impact on the surrounding environment, and high degree of mechanization. However, the construction decision during shield tunneling depends on the understanding of the stratum conditions ahead. When the actual soil quality does not match the known conditions, it will pose a great threat to the construction quality and construction safety. At present, the geological information along the tunnel project mastered by the construction personnel only comes from the geological description provided by the geological exploration before construction, and the geological conditions between the boreholes are only obtained by interpolation, which has a high degree of uncertainty. When the actual stratum does not match the stratum information provided in the geological survey report, the tunneling process may cause tunnel quality problems, and even lead to safety accidents in serious cases. Therefore, a system that uses slag images to identify the soil quality of the excavation face in real time can ensure that the construction personnel can grasp the stratum changes in time, adjust the construction decisions, and ensure the safe and stable excavation of the tunnel.

[0003] At present, there are mainly the following solutions for the real-time identification of soil quality on the excavation surface of shield machines:

[0004] The Chinese patent application with application number 202110183343.6 proposes a method for determining stratum characteristics based on real-time shield tunneling parameters. The stratum conditions are first pre-classified according to the survey report before tunnel construction, and the parameters collected by the shield machine in real time are transformed and processed. The processed index is drawn into a two-dimensional plane diagram to determine whether a new stratum type is generated and the number of stratum types is updated. Finally, the standardized parameters are input into the K-Means algorithm, and the stratum type determined by the corresponding parameters is output.

[0005] The Chinese patent application with application number 201911421797.1 proposes a method for inversion geology based on earth pressure balance shield machine parameter data. The historical operation data of the earth pressure balance shield machine is processed to prepare a data set, and the random forest model is learned on the training set and verified on the test set. The real-time excavation data of the earth pressure balance shield machine is extracted and input into the random forest model to obtain geological condition information.

[0006] The Chinese patent application with application number 202110152053.5 proposes a real-time soil classification method based on vibration signals. First, a sensor is installed on the excavation mechanism to collect the vibration signal generated by excavation, and then the average shear modulus of the contact soil is analyzed based on the vibration signal. Finally, the analysis result is compared with the data in the soil classification database to determine the soil classification.

[0007] The Chinese patent application with application number 202110558201.3 proposes a real-time prediction system and method for soil quality of the excavation face of shield tunnel construction. The prediction system includes a similar engineering data acquisition module, a soil information processing module, a construction data processing module, a soil quality predictor construction module, and a soil quality prediction module. The soil quality predictor construction module uses a convolutional neural network to construct a first soil quality predictor and a second soil quality predictor based on the soil feature grayscale image and slag image obtained from the construction data, and fuses the prediction results of the two predictors to obtain real-time soil quality detection results.

[0008] Disadvantages of existing technology

[0009] Shield construction parameters are related to various factors such as soil type, shield machine selection, tunnel burial depth, and slag improvement status. Therefore, the geological identification model established by the method of inverting geological types using real-time excavation parameters can only be used in tunnel projects with similar construction conditions, otherwise the identification effect is difficult to guarantee; the method of using sensors to collect vibration signals to analyze soil parameters requires additional sensors to be arranged, and the soil can only be classified based on the average shear modulus parameter indicator; in the method of using slag images for soil identification, the soil feature information corresponding to the image is calculated based on the analysis of geological survey results before tunnel construction, and the geological survey results themselves are characterized by incomplete and inaccurate information, and there is great uncertainty in the distribution of soil layers between boreholes. Inaccurate labels directly affect the recognition effect of the model. Summary of the invention

[0010] Technical Problems to be Solved by the Invention

[0011] A method for classifying slag and a system for identifying soil types in real time based on slag monitoring video are designed to solve the problem that shield drivers have difficulty in obtaining information about the strata ahead. The present invention can realize real-time and accurate identification of the soil type in front of the cutterhead, providing information for shield drivers to make subsequent construction decisions such as adjusting construction parameters and slag improvement plans.

[0012] Purpose of the invention

[0013] The core of the present invention is to develop a slag classification system that can combine the engineering properties of the soil itself and engineering experience, and a method for real-time identification of the soil type ahead based on the slag image captured by the shield machine monitoring camera. Using this method, it is possible to obtain slag classification results based on the characteristics of shield construction, and to locate the slag in real time based on the slag discharge video collected by the monitoring camera, and to make a real-time judgment of the geological type ahead. Features: low cost, automation, and suitable for tunnels excavated in various soil strata.

[0014] Technical solution of the present invention

[0015] A method for identifying soil types at an earth pressure balance shield excavation face, the method comprising classifying earth pressure balance shield slag based on shield construction characteristics, and performing image recognition of slag based on a deep learning algorithm.

[0016] This method determines the soil type of the excavation surface based on the identification of the slag, so it is necessary to first determine the classification of slag based on the classification of soil types. First, a slag classification system that takes into account both the engineering characteristics of the soil itself and the experience of shield construction is established. Then, the slag data is collected, and the data is selected and processed to obtain the earth pressure balance shield slag identification dataset. Then, a slag target detection model is constructed and trained. Finally, the slag monitoring video during the earth pressure balance shield construction is input to realize the real-time identification of slag and obtain the soil type information of the excavation surface.

[0017] The steps of this method are as follows:

[0018] Step 1: Establish a soil classification system

[0019] In order to identify the soil type during shield tunneling, the first step is to understand the engineering properties of the shallow strata in the area where the shield project is located. The cone penetration test (CPT) is an in-situ test method that can comprehensively reflect the physical and mechanical properties of the soil. By analyzing the statistical characteristic values ​​of the penetration resistance of different strata, the engineering characteristics of the soil in each stratum can be mastered, laying the foundation for the classification of slag.

[0020] 1.1 First, collect the geological survey reports of the area where the shield construction is located, organize the static penetration test layer parameter tables, count the penetration resistance values ​​of each borehole through the stratum, and eliminate abnormal values, that is, obviously unreasonable data.

[0021] 1.2 Then calculate the statistical characteristic values ​​of the penetration resistance index of each formation, including the mean, standard deviation and coefficient of variation.

[0022] First, the soil layers are classified into major categories according to their names, such as clay (CL), silt (SI), and sandy soil (SA).

[0023] Then, it is divided into detailed categories based on the average penetration resistance that can reflect the engineering properties of the soil. For example, clay (CL) is divided into soft clay (SC), common clay (NC), and hard clay (HC).

[0024] According to this method, the strata in a certain area can be combined and classified according to their engineering properties to obtain the soil type classification suitable for the adjustment of earth pressure balance shield parameters.

[0025] 1.3 After classifying the soil types in a certain area, the classification of slag soil is determined on this basis.

[0026] First, consider whether there are special soils or confined water that have a greater impact on the construction of the earth pressure balance shield. If there are such strata with special properties, and such strata are combined with other strata in the same category when classified according to the penetration resistance value, then such strata should be classified separately. In addition, considering that the excavation surface often passes through multiple layers of strata during shield construction, the excavated slag can be classified as a special type of "mixed soil". At this point, adjustments can be made based on the soil classification to obtain the slag classification results that take into account the engineering properties of the soil itself and construction experience.

[0027] 1.4 Since each slag image needs to be labeled with its slag type in the process of building the data set, the criteria for visually identifying the slag category are constructed based on the visual features of the slag to facilitate the subsequent labeling. The slag identification criteria of this method use four surface features of slag to determine the slag category, namely, slag shape (MS), slag cross-sectional morphology (CS), slag surface smoothness (SC), and slag color (MC).

[0028] Step 2: Establish a slag identification database

[0029] 2.1 First, extensively collect monitoring videos of construction waste in the area where this method is applied.

[0030] The general surveillance video resolution is 1920*1080, and it is best not less than 1280*720.

[0031] The slag monitoring camera should be installed directly above or diagonally above the belt conveyor, and aimed at the excavation port of the spiral excavator to ensure that the complete slag discharge process and slag shape are captured. Lighting equipment should be installed above the excavation port, and the lens surface should be kept clean so that the captured image is bright, clear and easy to identify.

[0032] 2.2 After collecting the video data, preprocess the data.

[0033] First, the effective video of the stable excavation stage is intercepted, and then the video frames are extracted at a frequency of one every 30 frames (the specific frame frequency can be adjusted according to the advancement speed of the shield machine). Images are selected based on the principle of making the slag shape and image background as diverse as possible, and images taken with damaged lenses, mosaic images caused by video freezes, and redundant images are eliminated.

[0034] 2.3 Use LabelImg image annotation software to manually annotate the slag images, use a rectangular frame to frame the slag body target in each image, and enter the slag category of each target frame. The label information is in PASCAL VOC format, and the actual target frame position information and corresponding slag category information of each image are saved as XML files. In the training set and validation set, each image must be labeled, and the information recorded in it is called the true value (ground truth, GT). This true value is provided to the training process of 3.3.4.

[0035] 2.4 The final obtained slag image file and the corresponding label file are the slag dataset, which is divided into training set, validation set and test set in a ratio of 8:1:1.

[0036] Step 3: Construction of slag identification model

[0037] The construction process of the slag soil recognition model is also a component of the code and the writing order. The present invention uses a convolutional neural network to establish a slag soil recognition model. Its task belongs to the target detection task, so it is necessary to build a target detection network. The model is built on the deep learning framework Pytorch and implemented in Python.

[0038] The slag identification model includes: a custom data set module, a slag detection network building module, a training process construction module, a model testing and evaluation index module and a slag identification result visualization module.

[0039] 3.1 In the custom dataset module

[0040] 3.1.1 It is necessary to load the muck image data and the corresponding label information from the dataset storage path and save them in a list respectively.

[0041] 3.1.2 Then process and convert the image data and label data. Convert the image data from PIL format to Tensor format with shape (C, H, W), where C represents the number of image channels, H represents the image height, and W represents the image width, and divide all pixel values ​​by 255 to normalize them to between 0 and 1.

[0042] 3.1.3 Then, before the image is input into the network, it is subjected to certain random image enhancement operations. Five data enhancements are performed on the image with a probability of 0.5, including horizontal flipping, grid masking, random cropping, color jittering, and blurring, to improve the diversity of database images and enhance the robustness of the model.

[0043] 3.1.4 Calculate the mean and standard deviation of the pixel values ​​of the three channels of the image in the entire data set, standardize the image to facilitate model learning, and provide it to step 3.3.

[0044] 3.2 In the slag detection network construction module, the network is mainly divided into three parts: backbone network, neck network and top network, which are provided to step 3.3.

[0045] The main function of the backbone network is to extract image features. Its basic structure is a combination of convolutional layers, batch normalization layers, and activation layers. The backbone network can be built by stacking this basic structure. Adding residual connections to the backbone network can increase the depth of the network and effectively improve its feature extraction capabilities.

[0046] The main functions of the neck network are feature enhancement and fusion. By way of example and not limitation, the spatial pyramid pooling (SPP) and path aggregation network (PANet) used in the present invention are commonly used modules of the neck network. Among them, the SPP module can effectively expand the receptive field and separate the most important context information, and PANet can enhance the extracted image features by fusing the parameters of different levels of the backbone network. The neck network is of great help to the performance improvement of the entire target detection network.

[0047] The top network is the detector, whose main function is to make the final regression prediction of the location and category information of the slag. The higher resolution output feature map contains more detailed features of the input image and is good at detecting small targets, while the lower resolution output feature map contains coarser features of the input image and is good at detecting large targets.

[0048] As an embodiment, the backbone network uses Darknet53, the neck network uses SPP and PANet, and the top network is composed of target detectors of three output scales. The picture is first input into the backbone network, which is composed of 53 layers of convolution combinations, in which a certain number of residual connections are inserted. The residual connection is to solve the phenomenon that the network may degenerate after the network reaches a certain number of layers when training a deep neural network, that is, it not only fails to improve the expression ability of the model, but makes the effect of the model worse. Each convolution combination can be regarded as a function F, and the output prediction value y=F(x) can be obtained by inputting the observation value. The residual connection is divided into two lines, one line is the observation value x input representing the convolution combination of function F (x), and the other line directly transmits the observation value x, and the final prediction value is the sum of the output results of the two lines, that is, F(x)+x. If the entire residual connection is regarded as a function H, and the input observation value is x, then the prediction value y=H(x)=F(x)+x. F(x) = H(x) - x is the residual, that is, the difference between the predicted value y and the observed value x. In this way, the next layer connected by the residual contains not only the information of the previous layer after nonlinear changes (convolution combination), but also the original information of the previous layer. This processing makes it possible for information to increase layer by layer, and the performance of the model will not decrease with the increase of network depth.

[0049] After the backbone network extracts features layer by layer, the feature map already contains high-level semantic information that can identify slag, but it inevitably loses a lot of detailed features, which is not conducive to the detection of small targets. Therefore, the feature map obtained by backbone network training is input into the neck network for further feature fusion and enhancement.

[0050] The feature map at the top of the backbone network is first input into the SPP structure of the neck network. The SPP network performs three different scales of pooling operations on the feature map and then splices them together in the channel dimension, which can solve the problem of repeated feature extraction by the convolutional network, greatly improve the speed of generating detection candidate frames, and save computing costs. Then, the feature map at the top of the backbone network is input into the PANet network of the neck network with the feature maps of two different levels in the middle of the backbone network for top-down and bottom-up bidirectional fusion, so that it has both deep semantic information and shallow basic information such as texture and color, ensuring the integrity and diversity of the features and improving the final prediction effect.

[0051] The feature maps of three different layers in the backbone network (“feature Figure 1 ,feature Figure 2 ,feature Figure 3”) are fused and enhanced by the neck network and then input into the final top network detector respectively. After a simple convolution combination, each of them is regressed to obtain the final prediction results "prediction output 1", "prediction output 2" and "prediction output 3". The three feature maps of different sizes contain features of different scales, and also correspond to the prediction of targets of different sizes. The prediction output 1 feature map is larger and contains the most detailed information, and is responsible for predicting small target objects; the prediction output 3 feature map is smaller and easier to distinguish the overall information, and is responsible for predicting large target objects. The results of the above three prediction outputs are combined together to obtain the prediction result of the entire network for the image.

[0052] 3.3 In the construction training process module

[0053] 3.3.1 First, import the custom dataset module and the slag detection network building module, instantiate the slag dataset and slag detection network, and build a data loader to customize the way images and label data are input into the network.

[0054] 3.3.2 Design the network loss function. The loss function of the target detection network used in this method is divided into three parts: localization loss, target box confidence loss, and classification loss, as shown in formula (1).

[0055] Loss=Localization loss+Confidence loss+Classification loss (1)

[0057] The target box confidence represents whether the target box contains the soil body and the size of the intersection and union ratio between the target box and the real box when it is contained.

[0058] 3.3.3 When training the network, you need to set many hyperparameters, including the initial value of the learning rate and how it changes with the number of training cycles, the optimizer and its parameters, the batch size of input images, the number of training cycles, the weights of each item in the loss function, etc. After setting the loss function and hyperparameters, you can proceed to step 3.3.4 to start training.

[0059] 3.3.4 Each batch of images input into the network will get a prediction result. The predicted value and the true value (ground truth, GT) in the label of step 2.3 are input into the loss function to calculate the current loss value, that is, the distance between the predicted value and the true value. Calculate the derivative of the loss value with respect to all network parameters, and use the optimizer to optimize and update the network parameters, which is a round of training iteration. When all the images in the training set are input into the network for training, it is a training cycle. After a training cycle is completed, the images of the validation set are input into the network to calculate the network accuracy. Based on this, the network training situation is observed and used as the basis for adjusting the hyperparameters for the next network training. When the loss decreases and converges to a certain stable value, the training can be terminated, and the trained network parameters are saved, and a recognition model that can accurately locate the slag target and judge the slag type is obtained.

[0060] 3.4 In the model testing and evaluation index module, load the trained model and parameters, input the muck image to be tested, and then you can get the regressed muck location information and category information. Write the evaluation index calculation code, and perform quantitative evaluation on the test results of the test set to evaluate the effect of the detection model.

[0061] 3.5 In the slag recognition result visualization module, the slag body target frame positioning information and category information obtained by network regression are drawn on the original image to visualize the recognition results. The target frame is drawn in the original image using the target frame center point position information x0, y0 and length and width information h, w regressed by the network, and the slag category information and corresponding confidence obtained by regression are written in the upper left corner of each target frame in text form. At the same time, the target frames and category information of different slag types are drawn and written in different colors.

[0062] Beneficial Effects

[0063] 1. The present invention proposes a slag classification method and a slag visual identification criterion based on the soil engineering properties and shield construction experience. The classification results obtained by this method can not only scientifically reflect the properties of the soil itself but also be applicable to the actual needs of shield construction. The constructed slag visual identification criterion provides a basis for the production of data set labels.

[0064] 2. Excavated soil is the most direct data reflecting the soil type ahead. The determining factor of its apparent characteristics is the soil condition ahead. Therefore, the deep learning algorithm can quickly and accurately obtain the soil type information ahead of the excavation face by identifying the excavated soil image. Moreover, with the expansion of the excavated soil image database, the updated excavated soil recognition model will have a wider and wider application range. There is no limitation that the model trained by similar projects can only be used to predict the soil type of a certain project.

[0065] 3. The method proposed in the present invention only needs to use the existing soil monitoring video data on site during data collection and practical application, without the need for additional sensors or other equipment. The reasoning speed of the algorithm itself is also higher than the frame rate of the monitoring video, so the method is real-time and low-cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 Flow chart of the method of the present invention

[0067] Figure 2 Establishing a process for the muck classification system of the present invention

[0068] Figure 3 Establishing process for the muck identification database of the present invention

[0069] Figure 4 Schematic diagram of soil image data acquisition in the present invention

[0070] Figure 5 The process of constructing the muck identification model of the present invention

[0071] Figure 6 A schematic diagram of the network structure of an embodiment of the present invention DETAILED DESCRIPTION

[0072] The technical solution provided by the present application will be further described below in conjunction with specific embodiments and accompanying drawings. The advantages and features of the present application will become more apparent with the following description.

[0073] Example

[0074] A method for identifying soil types at an earth pressure balance shield excavation face, the method comprising classifying earth pressure balance shield slag based on shield construction characteristics, and performing image recognition of slag based on a deep learning algorithm.

[0075] The overall process of the invention is as follows Figure 1 shown.

[0076] This method determines the soil type of the excavation surface based on the identification of the slag, so it is necessary to first determine the classification of slag based on the classification of soil types. First, a slag classification system that takes into account both the engineering characteristics of the soil itself and the experience of shield construction is established. Then, the slag data is collected, and the data is selected and processed to obtain the earth pressure balance shield slag identification dataset. Then, a slag target detection model is constructed and trained. Finally, the slag monitoring video during the earth pressure balance shield construction is input to realize the real-time identification of slag and obtain the soil type information of the excavation surface.

[0077] Step 1: Establish a soil classification system

[0078] The establishment process of the waste soil classification system is as follows: Figure 2 As shown. In order to identify the soil type during shield tunneling, understanding the engineering properties of the shallow strata in the area where the shield project is located is the first step. The cone penetration test (CPT) is an in-situ test method that can comprehensively reflect the physical and mechanical properties of the soil. By analyzing the statistical characteristic values ​​of the penetration resistance of different strata, the engineering characteristics of the soil in each stratum can be mastered, laying the foundation for the classification of slag.

[0079] 1.1 First, collect the geological survey reports of the area where the shield construction is located, organize the static penetration test layer parameter table, and count the penetration resistance values ​​of each borehole through the stratum in Excel, and eliminate abnormal values, that is, obviously unreasonable data.

[0080] 1.2 Then calculate the statistical characteristic values ​​of the penetration resistance index of each stratum, including the mean, standard deviation and coefficient of variation. First, the soil layers are classified into major categories according to the names of the layers, such as clay (CL), silt (SI), and sand (SA). Then, a detailed classification is made based on the mean penetration resistance that can reflect the engineering properties of the soil, such as clay (CL) is divided into soft clay (SC), common clay (NC), and hard clay (HC). According to this method, the strata in a certain area can be combined and classified according to their engineering properties to obtain the soil type classification suitable for the adjustment of earth pressure balance shield parameters.

[0081] 1.3 After the soil types in a certain area are classified, the classification of the slag is determined on this basis. First, consider whether there are special soils or pressurized water that have a greater impact on the construction of the earth pressure balance shield. If there are such strata with special properties, and such strata are combined into one category with other strata when classified according to the penetration resistance value, then such strata should be classified separately. In addition, considering that the excavation surface often passes through multiple layers of strata during shield construction, the slag discharged at this time can be classified as a special type of "mixed soil". At this point, adjustments can be made based on the soil classification to obtain the slag classification results that take into account the engineering properties of the soil itself and construction experience.

[0082] 1.4 Since each slag image needs to be labeled with its slag type in the process of building the data set, the criteria for visually identifying the slag category are constructed based on the visual features of the slag to facilitate the subsequent labeling. The slag identification criteria of this method use four surface features of slag to determine the slag category, namely, slag shape (MS), slag cross-sectional morphology (CS), slag surface smoothness (SC), and slag color (MC).

[0083] Step 2: Establish a slag identification database

[0084] The process of establishing the slag identification database is as follows: Figure 3 shown.

[0085] 2.1 First, collect a wide range of monitoring videos of slag in the area where this method is applied. The general monitoring video resolution is 1920*1080, preferably not less than 1280*720. Figure 4 As shown in the figure, the slag monitoring camera should be installed directly above or obliquely above the belt conveyor, and aimed at the excavation port of the spiral excavator to ensure that the complete slag discharge process and slag shape are captured. Lighting equipment should be installed above the excavation port, and the lens surface should be kept clean so that the captured image is bright, clear and easy to identify.

[0086] 2.2 After the video data is collected, the data is preprocessed. First, the effective video of the stable excavation stage is intercepted, and then the video frames are extracted at a frequency of one every 30 frames (the specific frame frequency can be adjusted according to the advancement speed of the shield machine). Images are selected based on the principle of making the slag shape and image background as diverse as possible, and images taken with damaged lenses, mosaic images caused by video freezes, and redundant images are eliminated.

[0087] 2.3 Use LabelImg image annotation software to manually annotate the slag images, use a rectangular frame to frame the slag body target in each image, and enter the slag category of each target frame. The label information is in PASCAL VOC format, and the actual target frame position information and corresponding slag category information of each image are saved as XML files. In the training set and validation set, each image must be labeled, and the information recorded in it is called the true value (ground truth, GT). This true value is provided to the training process of 3.3.4.

[0088] 2.4 The final obtained slag image file and the corresponding label file are the slag dataset, which is divided into training set, validation set and test set in a ratio of 8:1:1.

[0089] Step 3: Construction of slag identification model

[0090] The construction process of the slag identification model is as follows: Figure 5 As shown, it is also the components of the code and the writing order. The present invention uses a convolutional neural network to establish a slag recognition model. Its task belongs to the target detection task, so it is necessary to build a target detection network. The model is built on the deep learning framework Pytorch and implemented in Python.

[0091] 3.1 In the custom dataset module

[0092] 3.1.1 It is necessary to load the muck image data and the corresponding label information from the dataset storage path and save them in a list respectively.

[0093] 3.1.2 Then process and convert the image data and label data. Convert the image data from PIL format to Tensor format with shape (C, H, W), where C represents the number of image channels, H represents the image height, and W represents the image width. Divide all pixel values ​​by 255 to normalize them to between 0 and 1.

[0094] 3.1.3 Then, before the image is input into the network, it is subjected to certain random image enhancement operations. Five data enhancements are performed on the image with a probability of 0.5, including horizontal flipping, grid masking, random cropping, color jittering, and blurring, to improve the diversity of database images and enhance the robustness of the model.

[0095] 3.1.4 Calculate the mean and standard deviation of the pixel values ​​of the three channels of the image in the entire data set, standardize the image to facilitate model learning, and provide it to step 3.3.

[0096] 3.2 In the slag detection network construction module, the network is mainly divided into three parts: backbone network, neck network and top network, which are provided to step 3.3.

[0097] The main function of the backbone network is to extract image features. Its basic structure is a combination of convolutional layers, batch normalization layers, and activation layers. The backbone network can be built by stacking this basic structure. Adding residual connections to the backbone network can increase the depth of the network and effectively improve its feature extraction capabilities.

[0098] The main functions of the neck network are feature enhancement and fusion. By way of example and not limitation, the spatial pyramid pooling (SPP) and path aggregation network (PANet) used in the present invention are commonly used modules of the neck network. Among them, the SPP module can effectively expand the receptive field and separate the most important context information, and PANet can enhance the extracted image features by fusing the parameters of different levels of the backbone network. The neck network is of great help to the performance improvement of the entire target detection network.

[0099] The top network is the detector, whose main function is to make the final regression prediction of the location and category information of the slag. The higher resolution output feature map contains more detailed features of the input image and is good at detecting small targets, while the lower resolution output feature map contains coarser features of the input image and is good at detecting large targets.

[0100] like Figure 6The diagram shows the structure of the network of the embodiment. The backbone network uses Darknet53, the neck network uses SPP and PANet, and the top network is composed of target detectors of three output scales. The picture is first input into the backbone network, which is composed of 53 layers of convolution combinations, in which a certain number of residual connections are inserted. The residual connection is to solve the phenomenon that the network may degenerate after the network reaches a certain number of layers when training a deep neural network, that is, it not only fails to improve the expression ability of the model, but makes the effect of the model worse. Each convolution combination can be regarded as a function F, and the output prediction value y=F(x) can be obtained by inputting the observation value. The residual connection is divided into two lines, one line is the observation value x input representing the convolution combination of function F F(x), and the other line directly transmits the observation value x, and the final prediction value is the sum of the output results of the two lines, that is, F(x)+x. If the entire residual connection is regarded as a function H, and the input observation value is x, then the prediction value y=H(x)=F(x)+x. F(x) = H(x) - x is the residual, that is, the difference between the predicted value y and the observed value x. In this way, the next layer connected by the residual contains not only the information of the previous layer after nonlinear changes (convolution combination), but also the original information of the previous layer. This processing makes it possible for information to increase layer by layer, and the performance of the model will not decrease with the increase of network depth.

[0101] After the backbone network extracts features layer by layer, the feature map already contains high-level semantic information that can identify slag, but it inevitably loses a lot of detailed features, which is not conducive to the detection of small targets. Therefore, the feature map obtained by backbone network training is input into the neck network for further feature fusion and enhancement.

[0102] The feature map at the top of the backbone network is first input into the SPP structure of the neck network. The SPP network performs three different scales of pooling operations on the feature map and then splices them together in the channel dimension, which can solve the problem of repeated feature extraction by the convolutional network, greatly improve the speed of generating detection candidate frames, and save computing costs. Then, the feature map at the top of the backbone network is input into the PANet network of the neck network with the feature maps of two different levels in the middle of the backbone network for top-down and bottom-up bidirectional fusion, so that it has both deep semantic information and shallow basic information such as texture and color, ensuring the integrity and diversity of the features and improving the final prediction effect.

[0103] The feature maps of three different layers in the backbone network (“feature Figure 1 ,feature Figure 2 ,feature Figure 3”) are fused and enhanced by the neck network and then input into the final top network detector respectively. After a simple convolution combination, each of them is regressed to obtain the final prediction results "prediction output 1", "prediction output 2" and "prediction output 3". The three feature maps of different sizes contain features of different scales, and also correspond to the prediction of targets of different sizes. The prediction output 1 feature map is larger and contains the most detailed information, and is responsible for predicting small target objects; the prediction output 3 feature map is smaller and easier to distinguish the overall information, and is responsible for predicting large target objects. The results of the above three prediction outputs are combined together to obtain the prediction result of the entire network for the image.

[0104] 3.3 In the module of building the training process

[0105] 3.3.1 First, import the custom dataset module and the slag detection network building module, instantiate the slag dataset and slag detection network, and build a data loader to customize the way images and label data are input into the network.

[0106] 3.3.2 Design the loss function of the network. The loss function of the target detection network used in this method is mainly divided into three parts: localization loss, target box confidence loss and classification loss, as shown in formula (1).

[0107] Loss=Localization loss+Confidence loss+Classificaltion loss (1)

[0108] The target box confidence represents whether the target box contains the soil body and the size of the intersection and union ratio between the target box and the real box when it is contained.

[0109] 3.3.3 When training the network, you need to set many hyperparameters, including the initial value of the learning rate and how it changes with the number of training cycles, the optimizer and its parameters, the batch size of input images, the number of training cycles, the weights of each item in the loss function, etc. After setting the loss function and hyperparameters, you can proceed to step 3.3.4 to start training.

[0110] 3.3.4 Each batch of images input into the network will get a prediction result. The predicted value and the true value (ground truth, GT) in the label of step 2.3 are input into the loss function to calculate the current loss value, that is, the distance between the predicted value and the true value. Calculate the derivative of the loss value with respect to all network parameters, and use the optimizer to optimize and update the network parameters, which is a round of training iteration. When all the images in the training set are input into the network for training, it is a training cycle. After a training cycle is completed, the images of the validation set are input into the network to calculate the network accuracy. Based on this, the network training situation is observed and used as the basis for adjusting the hyperparameters for the next network training. When the loss decreases and converges to a certain stable value, the training can be terminated, and the trained network parameters are saved, and a recognition model that can accurately locate the slag target and judge the slag type is obtained.

[0111] 3.4 In the model testing and evaluation index module, load the trained model and parameters, input the muck image to be tested, and then you can get the regressed muck location information and category information. Write the evaluation index calculation code, and perform quantitative evaluation on the test results of the test set to evaluate the effect of the detection model.

[0112] 3.5 In the slag recognition result visualization module, the slag body target frame positioning information and category information obtained by network regression are drawn on the original image to visualize the recognition results. The target frame is drawn in the original image using the target frame center point position information x0, y0 and length and width information h, w regressed by the network, and the slag category information and corresponding confidence obtained by regression are written in the upper left corner of each target frame in text form. At the same time, the target frames and category information of different slag types are drawn and written in different colors.

[0113] The slag classification system and slag visual identification criteria obtained by the method proposed by the present invention will be updated with the continuous expansion of the scope of geological survey report collection and the continuous expansion of the slag database, and a slag classification system and visual identification criteria with a wider application range will be obtained. At the same time, the training slag identification model can be updated to obtain a slag identification model with a wider application range. Therefore, the method proposed by the present invention is theoretically applicable to the identification of soil types in front of the excavation face of the earth pressure balance shield in all regions.

[0114] Innovation of the present invention

[0115] 1. The present invention provides a method for classifying earth pressure balance shield slag based on soil engineering properties and shield construction experience. The slag type classification results obtained by this classification method can be identified in the image through a deep learning algorithm, and can also meet the actual engineering needs during shield construction, laying a classification foundation for subsequent slag image recognition.

[0116] 2. The present invention provides a method for visually identifying slag soil. It proposes to formulate identification criteria for different slag categories based on four important appearance characteristics of slag soil, namely, slag soil shape, slag soil cross-sectional morphology, slag soil surface flatness and slag soil color, so as to provide an identification basis for the subsequent production of data set labels.

[0117] 3. The present invention provides a method for identifying the soil type of the excavation face of an earth pressure balance shield, that is, to obtain the soil type information ahead by performing slag image recognition based on deep learning. The method can use monitoring video data to perform model training and real-time recognition during application, and is accurate, fast, and low-cost.

[0118] The above description is only a description of the preferred embodiments of the present application, and is not intended to limit the scope of the present application. Any changes or modifications made by any person skilled in the art based on the above disclosed technical contents shall be deemed as equivalent effective embodiments and shall fall within the scope of protection of the technical solution of the present application.

[0119] Appendix: Terminology Explanation:

[0120] Strata: A general term for all layered rocks, which can be consolidated rocks or unconsolidated sediments, that is, soil. It is a layer or a group of rock (soil) layers with certain unified characteristics and properties and are clearly distinguished from the upper and lower layers.

[0121] Soil quality: refers to the structure and properties of the soil. It also refers to the quality and structure of the soil.

[0122] Geological exploration: investigation and research work on the geological conditions such as rocks, stratigraphic structures, minerals, groundwater, landforms, etc. in a certain area using geological exploration methods such as surveying and mapping, geophysical exploration, geochemical prospecting, drilling, pit exploration, sampling testing, and geological remote sensing.

[0123] Static penetration test: The static penetration test uses static pressure to press a conical probe into the soil at a certain rate and uniformly, measuring its penetration resistance (including cone head resistance and side wall friction resistance or friction resistance ratio), and dividing the soil layers according to the size of the resistance they encounter to determine the engineering properties of the soil.

[0124] Statistical characteristic value: refers to the representative quantitative characteristics that can accurately describe the distribution of statistical data after sorting out the original data of the statistical survey.

[0125] Tunnel: An engineering structure buried in the ground, it is a form of human utilization of underground space.

[0126] Subway: A fast, high-capacity, electrically-powered rail transit system built in tunnels in cities.

[0127] Shield method: A tunnel construction method that uses a shield machine to excavate the strata and assemble the tunnel segments.

[0128] Shield machine: A construction machine consisting of an outer shell, a cutterhead, a pushing device, an assembly device and other supporting equipment. The outer shell is a column and serves as a protection, and other equipment is inside the outer shell.

[0129] Earth Pressure Balance Shield: A type of shield machine that uses the soil cut by the rotating cutterhead at the front end to fill the soil chamber. Its passive earth pressure is basically balanced with the earth pressure and water pressure on the excavation face, so that the excavation face and the shield face are in a balanced state.

[0130] Earth pressure balance shield waste: the waste soil excavated during the advancement of the earth pressure balance shield.

[0131] Cutterhead: A device used by shield machines to cut through strata. It is located at the front end of the shield machine and cuts off soil through rotation and extrusion.

[0132] Face: The working face where tunnel excavation continues to advance.

[0133] Spiral excavator: A machine that transports the soil cut by the cutter disc to the belt conveyor through the rotation of the spiral component. The rotation speed of the spiral component determines the excavation speed.

[0134] Belt conveyor: A belt device that transports soil from the screw excavator to the muck truck.

[0135] Shield construction parameters: Various parameters that need to be set during the construction of the shield machine, such as cutter head speed, etc. Whether the parameter setting is reasonable determines the safety and quality of shield construction.

[0136] Artificial intelligence: A new technological science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence.

[0137] Machine learning: a multi-disciplinary interdisciplinary subject involving probability theory, statistics, approximation theory, algorithm complexity theory and other disciplines. It is the core of artificial intelligence and the fundamental way to make computers intelligent.

[0138] Deep learning: It is a new research direction in the field of machine learning. It studies the inherent laws and representation levels of sample data. The information obtained is of great help in interpreting data such as text, images, and sounds.

[0139] Object detection: Object detection is an important application of deep learning, which is to identify objects in images and mark their locations.

[0140] Sample: The basic unit of data used by machine learning methods, which can be a one-dimensional matrix or a high-dimensional matrix.

[0141] Tags: real information of data.

[0142] LabelImg: A tool for labeling datasets for target detection tasks.

[0143] Dataset: The collection of training set, validation set and test set is called a dataset, including data and labels.

[0144] Training set: It is a data sample for model fitting and is used to debug the neural network.

[0145] Validation set: It is a set of samples set aside during model training. It can be used to adjust the model's hyperparameters and to make a preliminary assessment of the model's capabilities to see the training effect.

[0146] Test set: used to evaluate the generalization ability of the final model. But it cannot be used as a basis for algorithm-related choices such as parameter adjustment and feature selection. It is used to test the actual learning ability of the network.

[0147] Convolutional neural network: It is a type of feedforward neural network that includes convolution calculations and has a deep structure. It is one of the representative algorithms of deep learning.

[0148] Image channel: In RGB color mode, it refers to the three channels of red, green and blue.

[0149] Image enhancement: By certain means, some information is added to the original image or data is transformed to selectively highlight the features of interest in the image or suppress (mask) some unnecessary features in the image so that the image matches the visual response characteristics.

[0150] Grid mask: Generates a grid with the same resolution as the original image, with the gray area value being 1 and the black area value being 0. Multiplying it with the original image yields an enhanced image, which deletes information in specific areas. It can essentially be understood as a regularization method.

[0151] Color jitter: refers to generating a new image by randomly adjusting the saturation, brightness, and contrast of the original image.

[0152] Robustness: refers to the ability of a system to maintain certain performance even when subjected to disturbances or uncertainties.

[0153] Convolutional layer: The convolutional layer is composed of several convolutional units, and the parameters of each convolutional unit are optimized through the back propagation algorithm. The purpose of the convolution operation is to extract different features of the input. The first convolutional layer may only extract some low-level features such as edges, lines, and corners. More layers of the network can iteratively extract more complex features from low-level features.

[0154] Batch Normalization Layer: Batch Normalization (BN) is a technique used to improve the performance and stability of artificial neural networks. It can normalize the input of any layer in the neural network and fix the mean and variance of the input signal of each layer.

[0155] Activation layer: performs activation operation on the input data, that is, nonlinear transformation, maps the features to a high-dimensional nonlinear interval for interpretation, and solves problems that linear models cannot solve.

[0156] Receptive field: The feature vector of a certain position in a feature map of a certain layer is calculated from the input of a fixed area in the previous layer. This area is the receptive field of this position. The receptive field of the input image corresponding to a feature map of a certain layer is commonly used.

[0157] Feature map: The layer obtained after the image is extracted through the convolutional layer.

[0158] Loss function: used to evaluate the degree to which the model's predicted value differs from the true value.

[0159] Hyperparameters: Hyperparameters are parameters whose values ​​are set before starting the learning process, rather than parameter data obtained through training. Usually, hyperparameters need to be optimized to select a set of optimal hyperparameters for the model to improve learning performance and results.

[0160] Learning rate: is the step size for updating model parameters each time.

[0161] Batch size: The number of samples input into the network each time training.

Claims

1. A method for identifying soil type of an earth pressure balance shield excavation surface, comprising the following steps: Step 1: Establish a soil classification system 1.1 Collect geological survey reports of the area where the shield construction is located, organize the static penetration test layer parameter table, calculate the penetration resistance value of each borehole through the stratum, and eliminate abnormal values, that is, obviously unreasonable data; 1.2 Calculate the statistical characteristic values ​​of the penetration resistance index of each stratum, including the mean, standard deviation and coefficient of variation; firstly, classify the soil layers into major categories according to the names of the soil layers; then classify them in detail according to the mean value of the penetration resistance that can reflect the engineering properties of the soil; According to this method, the strata in a certain area are combined and classified according to their engineering properties to obtain the soil type classification suitable for the adjustment of earth pressure balance shield parameters; 1.3 After classifying the soil types in a certain area, the classification of slag soil is determined on this basis; First, consider whether there are special soils or confined water that have a greater impact on the construction of the earth pressure balance shield. If there are such strata with special properties, and such strata are combined with other strata in the classification based on the penetration resistance value, then such strata should be classified separately; in addition, considering that the excavation surface often passes through multiple strata during shield construction, the excavated soil can be classified as a special type of "mixed soil"; so far, adjustments can be made on the basis of soil classification to obtain the soil classification results that take into account the engineering properties of the soil itself and construction experience; 1.4 Based on the visual features of the slag, a criterion for visually identifying the type of slag is established to facilitate the subsequent labeling; Step 2: Establish a slag identification database 2.1 Collect extensive monitoring videos of landfill in the area where this method is applied; 2.2 After collecting the video data, pre-process the data; The effective video of the stable excavation stage is intercepted, and then the video frames are extracted at a frequency of one every 30 frames. The specific frame frequency can be adjusted according to the advancement speed of the shield machine; the images are selected based on the principle of making the slag shape and image background as diverse as possible, and the images taken with the lens contaminated, as well as the mosaic bad images and redundant images caused by video freezes are eliminated; 2.3 Use LabelImg image annotation software to manually annotate the soil images, use a rectangular frame to frame the soil objects in each image, and enter the soil category of each target frame; The label information is in PASCAL VOC format, and the actual target box location information and corresponding soil category information of each image are saved as XML files; In the training set and validation set, each image must be labeled. The information recorded in it is called the true value, which is provided to the training process of 3.3.4; 2.4 The obtained muck image files and corresponding label files are the muck dataset, which is divided into training set, validation set and test set in a ratio of 8:1:1; Step 3: Construction of slag identification model The slag soil recognition model includes: a custom data set module, a slag soil detection network building module, a training process building module, a model testing and evaluation index module and a slag soil recognition result visualization module; 3.1 In the custom dataset module 3.1.1 It is necessary to load the soil image data and the corresponding label information from the dataset storage path and save them in a list respectively; 3.1.2 Then process and convert the image data and label data; convert the image data from PIL format to Tensor format with shape (C, H, W), where C represents the number of image channels, H represents the image height, and W represents the image width, and divide all pixel values ​​by 255 to normalize them to between 0 and 1; 3.1.3 Then, before the image is input into the network, a certain random image enhancement operation is performed on it. Five data enhancement operations are performed on the image with a probability of 0.5: horizontal flipping, grid masking, random cropping, color jittering, and blurring. This improves the diversity of the database images and enhances the robustness of the model. 3.1.4 Calculate the mean and standard deviation of the pixel values ​​of the three channels of the image in the entire data set, standardize the image to facilitate model learning, and provide it to step 3.3; 3.2 In the soil slag detection network building module, the network is divided into three parts: backbone network, neck network and top network, which are provided to step 3.3; The main function of the backbone network is to extract image features. Its basic structure is a combination of convolutional layers, batch normalization layers, and activation layers. The backbone network can be built by stacking this basic structure. Adding residual connections to the backbone network can increase the depth of the network and effectively improve its feature extraction capabilities. The functions of the neck network are feature enhancement and fusion; The top network is a detector, which is used to make the final regression prediction of the location and category information of the slag. The higher resolution output feature map contains more detailed features of the input image and is good at detecting small targets. The lower resolution output feature map contains coarser features of the input image and is good at detecting large targets. 3.3 In the construction training process module 3.3.1 First, import the custom dataset module and the muck detection network building module, instantiate the muck dataset and the muck detection network, and build a data loader to customize the way images and label data are input into the network; 3.3.2 Designing the loss function of the network. The loss function of the target detection network used is divided into three parts: muck positioning loss, target box confidence loss and muck classification loss, as shown in formula (1). （1） The target box confidence represents whether the target box contains the soil body and the size of the intersection and union ratio between the target box and the real box when it is contained; 3.3.3 Hyperparameters need to be set during network training, including: the initial value of the learning rate and how it changes with the number of training cycles, the optimizer and its parameters, the input image batch size, the number of training cycles, and the weights of each item in the loss function; after setting the loss function and hyperparameters, proceed to step 3.3.4 to start training; 3.3.4 Each batch of images input into the network will get a prediction result. The predicted value and the true value in the label of step 2.3 are input into the loss function to calculate the current loss value, that is, the distance between the predicted value and the true value; the derivative of the loss value with respect to all network parameters is calculated, and the network parameters are optimized and updated using the optimizer, which is a round of training iteration; when all the images in the training set are input into the network for training, it is a training cycle; after a training cycle, the images of the validation set are input into the network to calculate the network accuracy, and the network training situation is observed based on this and used as the basis for adjusting the hyperparameters for the next network training; when the loss decreases and converges to a certain stable value, the training ends, and the trained network parameters are saved, and a recognition model that can accurately locate the slag target and judge the slag type is obtained; 3.4 In the model testing and evaluation index module, load the trained model and parameters, input the muck image to be tested, and then obtain the regressed muck location information and category information; write the evaluation index calculation code, conduct quantitative evaluation on the test results of the test set, and evaluate the effect of the detection model; 3.5 In the slag recognition result visualization module, the slag target frame positioning information and category information obtained by network regression are drawn on the original image to visualize the recognition results; the target frame is drawn in the original image using the target frame center point position information x0, y0 and length and width information h, w regressed by the network, and the slag category information and corresponding confidence obtained by regression are written in text in the upper left corner of each target frame. At the same time, the target frames and category information of different slag types are drawn and written in different colors.

2. A method for identifying soil type of an earth pressure balance shield excavation surface as claimed in claim 1, characterized in that: The slag identification criteria use four surface features of slag to determine the slag category, namely, slag shape, slag cross-sectional morphology, slag surface flatness and slag color.

3. A method for identifying soil type of an earth pressure balance shield excavation surface as claimed in claim 1, characterized in that: The resolution of the slag monitoring video is 1920*1080, not less than 1280*720; The slag monitoring camera should be installed directly above or diagonally above the belt conveyor, aimed at the discharge port of the spiral excavator, to ensure that the complete slag discharge process and slag shape are captured; install lighting equipment above the discharge port, and keep the lens surface clean, so that the captured image is bright, clear and easy to identify.

4. The method for identifying soil type of an earth pressure balance shield excavation surface according to claim 1, characterized in that: The slag recognition model is built on the deep learning framework Pytorch and implemented in Python.

5. The method for identifying soil type of an earth pressure balance shield excavation surface according to claim 1, characterized in that: Step 3.2 The spatial pyramid pooling and path aggregation network used in the construction module of the slag detection network are the neck network modules; among them, the SPP module can effectively expand the receptive field and separate the most important contextual information, and PANet can enhance the extracted image features by fusing the parameters of different levels of the backbone network; the neck network is of great help to improve the performance of the entire target detection network; The backbone network uses Darknet53, the neck network uses SPP and PANet, and the top network is composed of target detectors with three output scales. The image is first input into the backbone network, which is composed of 53 layers of convolution combinations, in which a certain number of residual connections are inserted. The residual connection is to solve the problem that when training deep neural networks, the network may degenerate after reaching a certain number of layers, that is, it not only fails to improve the expression ability of the model, but makes the effect of the model worse. Each convolution combination is regarded as a function F, and the output prediction value y=F (x) can be obtained by inputting the observation value. The residual connection is divided into two lines, one line is the observation value x input representing the convolution combination of function F F(x), and the other line directly transmits the observation value x. The final prediction value is the sum of the output results of the two lines, that is, F(x)+x. If the entire residual connection is regarded as a function H, and the input observation value is x, then the prediction value y=H(x)=F(x)+x; F(x)= H(x)-x is the residual, that is, the difference between the predicted value y and the observed value x. In this way, the next layer connected by the residual contains not only the information of the previous layer after nonlinear changes (convolution combination), but also the original information of the previous layer. This processing makes it possible for information to increase layer by layer, and the performance of the model will not decrease with the increase of network depth. After layer-by-layer feature extraction by the backbone network, the feature map already contains high-level semantic information that can identify the slag, but it inevitably loses a lot of detailed features, which is not conducive to the detection of small targets; therefore, the feature map obtained by backbone network training is input into the neck network for further feature fusion and enhancement; The feature map at the top of the backbone network is first input into the SPP structure of the neck network. The SPP network performs three different scales of pooling operations on the feature map and then splices them together in the channel dimension, which can solve the problem of repeated feature extraction by the convolutional network, greatly improve the speed of generating detection candidate frames, and save computing costs. Then, the feature map at the top of the backbone network is input into the PANet network of the neck network with the feature maps of two different levels in the middle of the backbone network for top-down and bottom-up bidirectional fusion, so that it has both deep semantic information and shallow basic information such as texture and color, ensuring the integrity and diversity of the features and improving the final prediction effect. The feature maps of three different levels in the backbone network, "Feature Map 1, Feature Map 2, Feature Map 3", are fused and enhanced by the neck network and then input into the final top network detector respectively. After a simple convolution combination, each of them is regressed to obtain the final prediction results "Prediction Output 1", "Prediction Output 2", and "Prediction Output 3". The three feature maps of different sizes contain features of different scales, and also correspond to the prediction of targets of different sizes. The prediction output 1 feature map is larger and contains the most detailed information, and is responsible for predicting small target objects. The prediction output 3 feature map is smaller and easier to judge the overall information, and is responsible for predicting large target objects. The results of the above three prediction outputs are combined together to obtain the prediction result of the entire network for the image.

Images