A method and system for automatic detection of soil flow plasticity based on deep learning

Abstract

The present invention discloses a method and system for automatic detection of soil flow plasticity based on deep learning, the method comprising: classifying soil flow plasticity to establish different types of training sets; building a deep convolutional neural network model with Resnet network as the basic network; inputting the training set into the input layer of the model to obtain a three-dimensional original feature map S∈R c×h×w , input the original feature map S into three convolutional layers with the same filter respectively to generate three-dimensional arrays A, B and C, convert the three-dimensional array A into a two-dimensional array and transpose it to form a two-dimensional array D, convert the three-dimensional arrays B and C into two-dimensional arrays E and F respectively, perform logical operations to form a two-dimensional array G, #imgabs0# perform logical operations to form a two-dimensional array H, #imgabs1# convert the two-dimensional array H into a three-dimensional array and sum it with the original feature map S to update the model; train the model for soil flow plasticity classification and obtain an automatic detection model for identifying soil flow plasticity categories.

Description

Technical Field

[0001] The present invention belongs to the technical field of tunnel construction safety, and more specifically, relates to a method and system for automatic detection of soil flow plasticity based on deep learning. Background Art

[0002] With the development of rail transit, tunnel construction has become a common mode of transportation. Earth pressure balance shield has become the main method of urban subway tunnel construction due to its fast construction speed, small impact on the surrounding environment, and high degree of automation. The key to earth pressure balance shield is to achieve pressure balance inside and outside the soil bin and balance the amount of soil in and out of the soil bin. The pressure balance inside and outside the soil bin is mainly achieved by controlling the amount of soil discharge. During the construction process, the strata that the shield machine passes through are often complex and changeable. It is difficult to change its hardware device after the shield process begins. Therefore, it is necessary to improve the slag in the soil bin in real time during the shield process to make the slag have good flow plasticity and reduce the change range of the total thrust and total torque of each ring during the excavation process, thereby improving the stability of the shield. If the improvement is improper, it will cause "mud cake" and "gushing" and other phenomena, which will seriously affect the construction efficiency. Therefore, it is necessary to monitor the improved slag to determine whether the current slag improvement control is appropriate. Once it is found that the quality of the improved slag does not meet the requirements, the improvement control method should be adjusted in time. The current monitoring method is to conduct manual monitoring of the slag outlet. However, the manual monitoring method is inefficient and is greatly affected by subjective factors. The monitoring results are prone to misjudgment, which poses a hidden danger to the safety of shield machine construction. Summary of the invention

[0003] In view of the above defects or improvement needs of the prior art, the present invention provides a method and system for automatic detection of soil flow plasticity based on deep learning, which aims to provide intelligent soil identification technology to improve the monitoring efficiency and accuracy of improved soil during the shield machine's passage through the strata, thereby improving the safety and stability of shield machine construction.

[0004] To achieve the above object, according to one aspect of the present invention, a method for automatic detection of soil flow plasticity based on deep learning is provided, which comprises:

[0005] Acquire the slag image, extract the characteristic information of the slag image and classify the slag flow plasticity to establish different types of training sets;

[0006] A deep convolutional neural network model is constructed using the Resnet network as a basic network, wherein the deep convolutional neural network model includes an input layer, a hidden layer, and an output layer;

[0007] Input the training set into the input layer of the model to obtain the three-dimensional original feature map S∈R from the input layer to the hidden layer c×h×w, where w is the array width, h is the array height, and c is the number of channels. The original feature map S is input into three convolutional layers with the same filter to generate three-dimensional arrays A, B, and C. The three-dimensional array A is converted into a two-dimensional array and then transposed to form a two-dimensional array D. The three-dimensional arrays B and C are converted into two-dimensional arrays E and F, respectively. D, E, F ∈ R c×n , n = h × w, perform logical operations on the two-dimensional array D and the two-dimensional array E to form a two-dimensional array G. The value of the i-th row and j-th column of the two-dimensional array G is Perform logical operations on the two-dimensional array G and the two-dimensional array F to form a two-dimensional array H. The data in the i-th row of the two-dimensional array H Where θ is the weight learned from 0, the two-dimensional array H is converted into a three-dimensional array and summed with the original feature map S to update the feature map of the model;

[0008] The updated model is trained for soil flow plasticity classification until the loss function reaches a preset convergence level, and an automatic detection model for identifying soil flow plasticity categories is obtained.

[0009] Preferably, the method further comprises acquiring a slag image and inputting the image into the automatic detection model, and using the automatic detection model to identify and classify the flow plasticity of the slag.

[0010] Preferably, the slag is divided into dry soil, wet soil and suitable soil according to its fluidity and plasticity.

[0011] Preferably, the layer convolutional neural network model also includes a softmax layer, and the performing logical operations on the two-dimensional array D and the two-dimensional array E includes performing logical operations on the two-dimensional array D and the two-dimensional array E through the softmax layer.

[0012] Preferably, the filter is a 3*3 dimensional filter.

[0013] Preferably,

[0014] The method also includes: extracting multiple feature maps S, performing region of interest pooling and L2 normalization, connecting and rescaling the generated result features to adapt to the original scale of the features, and using a 1×1 convolution filter to match the number of layers of the original network to extract the final region of interest features.

[0015] Preferably, for the soil and slag image captured during the movement, before extracting the characteristic information, the method further comprises: restoring the soil and slag image by using Wiener filtering technology.

[0016] Preferably, for the soil image whose contrast does not meet the condition, before extracting the characteristic information, it also includes: using a histogram equalization technology to change the grayscale of each pixel in the image by changing the histogram of the image, thereby enhancing the contrast of the image.

[0017] Preferably, the updated model is trained until the loss function reaches a preset convergence level, including:

[0018] Set multiple groups of different learning rates, train the model at different learning rates to reach the preset number of times, compare the convergence of the loss function under each group of learning rates, and use the learning rate with the best convergence effect as the initial learning parameter of the model.

[0019] According to another aspect of the present invention, a system for automatically detecting soil flow plasticity based on deep learning is provided, which comprises:

[0020] Training set establishment unit: used to obtain slag soil images, extract characteristic information of slag soil images and establish training sets of different flow plasticity types;

[0021] An initial model building unit: used to build a deep convolutional neural network model using a Resnet network as a basic network, wherein the deep convolutional neural network model includes an input layer, a hidden layer, and an output layer;

[0022] Mapping update unit: used to input the training set into the input layer of the model and obtain the three-dimensional original feature map S∈R from the input layer to the hidden layer c×h×w , where w is the array width, h is the array height, and c is the number of channels. The original feature map S is input into three convolutional layers with the same filter to generate three-dimensional arrays A, B, and C. The three-dimensional array A is converted into a two-dimensional array and then transposed to form a two-dimensional array D. The feature map B and feature map C are converted into two-dimensional arrays E and two-dimensional arrays F, respectively. D, E, F ∈ R c×n , n = h × w, perform logical operations on the two-dimensional array D and the two-dimensional array E to form a two-dimensional array G. The value of the i-th row and j-th column of the two-dimensional array G is Perform logical operations on the two-dimensional array G and the two-dimensional array F to form a two-dimensional array H. The data in the i-th row of the two-dimensional array H Where θ is the weight learned from 0, the two-dimensional array H is converted into a three-dimensional array and summed with the original feature map S to update the feature map of the model;

[0023] The model training unit is used to perform soil flow plasticity classification training on the updated model until the loss function reaches a preset convergence degree, thereby obtaining an automatic detection model for identifying the soil flow plasticity category.

[0024] In general, through the above technical solutions conceived by the present invention, a deep convolutional neural network model is established, and the model is trained with images of slag with different flow plasticity as a training set to obtain a detection model that can automatically identify the flow plasticity category of slag. At the same time, after the initial convolutional neural network model is established, the feature map is processed by logical operations to update the feature map of the model, thereby improving the accuracy and reliability of feature extraction during deep learning, and then improving the accuracy and reliability of model recognition. The obtained model can automatically detect and identify the category of slag, and improve the monitoring efficiency and accuracy of improved slag during the shield machine's passage through the stratum, thereby improving the safety and stability of shield machine construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A flowchart of the steps of the automatic detection method of soil flow plasticity based on deep learning provided in an embodiment of the present invention;

[0026] Figure 2 A diagram showing the pretreatment process of three different types of improved slag provided by an embodiment of the present invention;

[0027] Figure 3 A logical operation process diagram of updating feature mapping provided by an embodiment of the present invention;

[0028] Figure 4 A diagram of a region of interest feature extraction process provided by an embodiment of the present invention;

[0029] Figure 5 A Loss-Epoch curve diagram under different initial learning rates provided by an embodiment of the present invention;

[0030] Figure 6 This is a graph showing the test results of three different types of improved slag provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0032] like Figure 1 FIG. 1 is a flowchart of a method for automatically detecting soil flow plasticity based on deep learning in one embodiment of the present invention, wherein the method includes:

[0033] Step S100: Acquire a slag image, extract feature information of the slag image and classify the slag flow plasticity to establish different types of training sets.

[0034] In one embodiment, an image of improved slag is collected through video, and the original image is cropped, restored and enhanced by preprocessing to obtain a feature-enhanced image, and a data set is established, which includes a training set. The training set includes image features of different types of slag, and the trained model can perform feature recognition and classification on slag images by using the different types of training sets. In one embodiment, the improved slag can be divided into wet soil, dry soil and suitable soil.

[0035] In one embodiment, the initial image is cropped by a screenshot method to retain only the main part and exclude interference caused by non-main part.

[0036] In one embodiment, for the soil and slag image captured during the movement, before extracting the feature information, the soil and slag image is restored by using the Wiener filtering technology. Specifically, based on the minimum mean square error criterion, the mean square error between the original image and the restored image is minimized, and the calculation formula is as follows:

[0037]

[0038]

[0039] Among them, e 2 stands for MSE (mean square error); E is the expected value of the parameter; f(x,y) and represent the original image and the restored image respectively; H(u,v) is the degradation function; G(u,v) is the degraded image; S n (u,v) is the noise power spectrum; S f (u,v) is the power spectrum of the undegraded image; is the frequency domain estimate. Figure 2 Shown are three different types of slag images restored using the Wiener filtering technology.

[0040] In one embodiment, for soil images whose contrast does not meet the requirements, for example, for problems such as image information being biased and having low contrast due to factors such as illumination and vibration, before extracting feature information, a histogram equalization technique is used to change the grayscale of each pixel in the image by changing the histogram of the image to enhance the contrast of the image. The specific calculation formula is as follows:

[0041]

[0042]

[0043] Where r and s represent the normalized original image grayscale and the image grayscale after histogram equalization, respectively; T(r) is the transformation function from r to s; p r (r) is the probability density of the random variable r; p s (s) is the probability density of random variable s; F s (s) is the distribution function of the assumed random variable s.

[0044] Step S200: Using the Resnet network as the basic network, a deep convolutional neural network model (CNN) is constructed. The deep convolutional neural network model includes an input layer, a hidden layer, and an output layer. The hidden layer is used to abstract the features of the input data into another dimensional space. Each layer of the network model has a number of neurons. Each neuron in the latter layer between two adjacent layers is connected to the neurons in the previous layer. In the image recognition problem, each neuron in the input layer may represent the grayscale value of a pixel. The neurons in the input layer need to be connected to the neurons in the hidden layer. This mapping from the input layer to the hidden layer is called feature mapping.

[0045] Step S300: Input the training set into the input layer of the model to obtain the three-dimensional original feature map S∈R from the input layer to the hidden layer c×h×w , where w is the array width, h is the array height, and c is the number of channels. The original feature map S is input into three convolutional layers with the same filter to generate three-dimensional arrays A, B, and C. The three-dimensional array A is converted into a two-dimensional array and then transposed to form a two-dimensional array D. The feature map B and feature map C are converted into two-dimensional arrays E and two-dimensional arrays F, respectively. D, E, F ∈ R c×n , n = h × w, perform logical operations on the two-dimensional array D and the two-dimensional array E to form a two-dimensional array G. The value of the i-th row and j-th column of the two-dimensional array G is Perform logical operations on the two-dimensional array G and the two-dimensional array F to form a two-dimensional array H. The data in the i-th row of the two-dimensional array H Among them, θ is the weight learned from 0, the two-dimensional array H is converted into a three-dimensional array and summed with the original feature map S to update the feature map of the model.

[0046] like Figure 3 As shown, the training set is input into the input layer of the deep convolutional neural network model to obtain the original feature map S∈R from the input layer to the hidden layer. c×h×w , where the original feature map S is a three-dimensional array, w is the array width, h is the array height, and c is the number of channels. c×h×wThe input is fed into three convolutional layers with the same filter size, which produces three new feature maps, namely three-dimensional arrays A, B, and C (A, B, C ∈ R c×h×w ). In one embodiment, the filter is a 3*3 dimensional filter.

[0047] Since three-dimensional arrays cannot be directly used for logical operations, it is necessary to reshape the three-dimensional arrays A, B, and C. The three-dimensional array of size h*w*c is converted into a c*n two-dimensional array, where n=h*w. The converted two-dimensional array still retains the characteristic information of the original three-dimensional array, but the data is converted from a three-dimensional form to a two-dimensional form for the convenience of logical operations. Take c=3 as an example. There are 3 channels, each channel is a h*w two-dimensional array, and the h*w two-dimensional array of each channel is converted into a 1*n one-dimensional array, where n=h*w, and then the 1*n one-dimensional arrays of the three channels are assembled into a 3*n two-dimensional array. Similarly, if you want to convert the c*n two-dimensional array into a h*w*c three-dimensional array, it is the inverse process of the above process.

[0048] like Figure 3 As shown, the three-dimensional array A is reshaped into a two-dimensional array and then transposed to form an n*c two-dimensional array D. The three-dimensional array B is converted to a c*n two-dimensional array E, and the three-dimensional array C is converted to a c*n two-dimensional array F, D, E, F∈R c×n , n = h × w. Perform logical operations on the two-dimensional array D and the two-dimensional array E to form a two-dimensional array G. The value of the i-th row and j-th column of the two-dimensional array G is Perform logical operations on the two-dimensional array G and the two-dimensional array F to form a two-dimensional array H. The data in the i-th row of the two-dimensional array H Where θ is the weight learned from 0. The weight is a learnable scalar, which is initialized to 0. The introduction of learnable θ allows the network to first rely on clues in the local neighborhood and then gradually learn to assign more weight to non-local evidence. The two-dimensional array H is converted to a three-dimensional array and summed with the original feature map S to update the feature map of the model. The updated feature map is In this embodiment, since Resnet is used as the basic network of Faster R-CNN, which has the problem of translation invariance and its local features are prone to misclassification of targets and content, the position attention feature is introduced by performing the above logical operation, and a wider range of contextual information is encoded into the local features, thereby improving the accuracy of the model's classification and recognition of features.

[0049] In one embodiment, if Figure 4As shown, in order to obtain different feature details of regions of interest of different granularities, the present invention is also based on combining shallow and deep features in multiple convolutional layers to improve the region of interest pooling layer. Specifically, by extracting multiple convolutional feature maps S, and then performing region of interest pooling and L2 normalization on them, the resulting features are then connected and rescaled to adapt to the original scale of the features, and finally, a 1×1 convolution filter is used to match the number of layers of the original network. In general, some intermediate results and the final feature map in the region generation network (RPN) are combined to produce the final region of interest pool features for object detection.

[0050] Step S400: Perform soil flow plasticity classification training on the updated model until the loss function reaches a preset convergence degree, thereby obtaining an automatic detection model for identifying soil flow plasticity categories.

[0051] In one embodiment, after updating the feature map, the model is trained for soil flow plasticity classification. Specifically, the number of training times can be set, for example, the maximum number of training times MaxEpochs parameter is set to 30, and the convergence of the loss function under multiple groups of different learning rates is tested respectively, and the convergence of the loss function under each group of learning rates is compared to obtain the Loss-Epoch curve respectively, and the learning rate with the best convergence effect is used as the initial learning parameter of the model. For example, 5×10 -3 , 10 -3 , 10 -4 and 10 -5 The loss function convergence under four different learning rates, the test results are as follows Figure 5 The test results show that when the initial learning rate is set to 10 -4 and 10 -5 When , the training gradient oscillates around the minimum value and eventually fails to converge, indicating that 10 -4 and 10 -5 The learning rate is set too large. When the initial learning rate is set to 5×10 -3 and 10 -3 When the learning rate is 5×10 -3 The network converges after about 21,000 iterations. When the learning rate is 10 -3 The network converges after about 11,000 iterations, and the learning rate is 10 -3 The convergence speed is about 5×10 -3 The convergence speed is nearly twice as fast. In summary, choosing 10 -3 As the initial learning rate, the initial learning parameters of the network model are obtained.

[0052] In one embodiment, the data set includes not only a training set but also a validation set. The validation set is substituted into the network to test the accuracy of the model, and the detection results of some three types of improved slag are obtained as follows: Figure 6 As shown, this shows that the model can accurately identify the type of slag.

[0053] In one embodiment, after obtaining the trained model, the model can be put into practical application. During the process of tunnel excavation by a pressure-balanced shield machine, the slag image is obtained and input into the automatic detection model. The automatic detection model is used to identify and classify the flow plasticity of the slag. When the slag type is identified as unsuitable soil, the monitoring system automatically feeds back to the control system to adjust the slag improvement method.

[0054] The present application also relates to an automatic detection system for soil flow plasticity based on deep learning, which comprises:

[0055] Training set establishment unit: used to obtain slag soil images, extract characteristic information of slag soil images and establish training sets of different flow plasticity types;

[0056] An initial model building unit: used to build a deep convolutional neural network model using a Resnet network as a basic network, wherein the deep convolutional neural network model includes an input layer, a hidden layer, and an output layer;

[0057] Mapping update unit: used to input the training set into the input layer of the model and obtain the three-dimensional original feature map S∈R from the input layer to the hidden layer c×h×w , where w is the array width, h is the array height, and c is the number of channels. The original feature map S is input into three convolutional layers with the same filter to generate three-dimensional arrays A, B, and C. The three-dimensional array A is converted into a two-dimensional array and then transposed to form a two-dimensional array D. The feature map B and feature map C are converted into two-dimensional arrays E and two-dimensional arrays F, respectively. D, E, F∈R c×n , n = h × w, perform logical operations on the two-dimensional array D and the two-dimensional array E to form a two-dimensional array G. The value of the i-th row and j-th column of the two-dimensional array G is Perform logical operations on the two-dimensional array G and the two-dimensional array F to form a two-dimensional array H. The data in the i-th row of the two-dimensional array H Where θ is the weight learned from 0, the two-dimensional array H is converted into a three-dimensional array and summed with the original feature map S to update the feature map of the model;

[0058] The model training unit is used to perform soil flow plasticity classification training on the updated model until the loss function reaches a preset convergence degree, thereby obtaining an automatic detection model for identifying the soil flow plasticity category.

[0059] The deep learning-based automatic detection system for slag flow plasticity is used to execute the above-mentioned deep learning-based automatic detection method for slag flow plasticity. The functions of each unit of the system correspond to the various steps in the above-mentioned method and will not be repeated here.

[0060] By building a deep convolutional neural network model and training the model with images of slag with different flow plasticity as a training set, a detection model that can automatically identify the flow plasticity category of slag is obtained. At the same time, after the initial convolutional neural network model is established, the feature map is updated by performing logical operations on the feature map, thereby improving the accuracy and reliability of feature extraction during deep learning, and then improving the accuracy and reliability of model recognition. The obtained model can automatically detect and identify the category of slag, and improve the monitoring efficiency and accuracy of improved slag during the shield machine's passage through the stratum, thereby improving the safety and stability of shield machine construction.

[0061] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An improved Faster R-CNN method for automatic detection of soil flow plasticity, characterized in that: include: Acquire the slag image, extract the characteristic information of the slag image and classify the slag flow plasticity to establish different types of training sets; The Resnet network is used as a basic network of the improved Faster R-CNN, and the improved Faster R-CNN includes an input layer, a hidden layer and an output layer; Input the training set into the input layer to obtain the three-dimensional original feature map S∈R from the input layer to the hidden layer c×h×w , where w is the array width, h is the array height, and c is the number of channels. The original feature map S is input into the position attention feature module. The position attention feature module includes: Three convolutional layers with the same filter are used to generate three-dimensional arrays A, B and C from the original feature map S respectively. Logical operation module, used to convert the three-dimensional array A into a two-dimensional array and then transpose it to form a two-dimensional array D, and convert the three-dimensional array B and three-dimensional array C into two-dimensional array E and two-dimensional array F respectively, D∈R n×c , E, F∈R c×n , n = h × w, perform logical operations on the two-dimensional array D and the two-dimensional array E to form a two-dimensional array G. The value of the i-th row and j-th column in the two-dimensional array G is , perform logical operations on the two-dimensional array G and the two-dimensional array F to form a two-dimensional array H. The data of the i-th row in the two-dimensional array H , where θ is the weight learned from 0, converts the two-dimensional array H into a three-dimensional array and sums it with the original feature map S to update the feature map of the model; The improved Faster R-CNN also includes an improved region of interest pooling layer for extracting multiple feature maps S, performing region of interest pooling and L2 normalization, connecting and rescaling the generated result features to adapt to the original scale of the features, and using a 1×1 convolution filter to match the number of layers of the original network to extract the final region of interest features; The updated model is trained for soil flow plasticity classification until the loss function reaches a preset convergence level, and an automatic detection model for identifying soil flow plasticity categories is obtained.

2. The method for automatically detecting flow plasticity of slag as claimed in claim 1, characterized in that: The method also includes acquiring a slag image and inputting the image into the automatic detection model, and using the automatic detection model to identify and classify the flow plasticity of the slag.

3. The method for automatically detecting flow plasticity of slag as claimed in claim 1, characterized in that: According to its plasticity, the slag can be divided into dry soil, wet soil and suitable soil.

4. The method for automatically detecting flow plasticity of slag as claimed in claim 1, characterized in that: The improved Faster R-CNN also includes a softmax layer, and the performing a logical operation on the two-dimensional array D and the two-dimensional array E includes performing a logical operation on the two-dimensional array D and the two-dimensional array E through the softmax layer.

5. The method for automatically detecting flow plasticity of slag as claimed in claim 1, characterized in that: The filter is a 3*3 dimensional filter.

6. The method for automatically detecting flow plasticity of slag as claimed in claim 1, characterized in that: For the soil slag image captured during the movement, before extracting the feature information, the method further includes: restoring the soil slag image by using the Wiener filtering technology.

7. The method for automatically detecting flow plasticity of slag as claimed in claim 1, characterized in that: For the muck image whose contrast does not meet the condition, before extracting the characteristic information, it also includes: using the histogram equalization technology to change the grayscale of each pixel in the image by changing the histogram of the image, so as to enhance the contrast of the image.

8. The method for automatically detecting flow plasticity of slag as claimed in claim 1, characterized in that: The updated model is trained until the loss function reaches a preset convergence degree, including: setting multiple groups of different learning rates, training the model at different learning rates to reach a preset number of times, comparing the convergence of the loss function under each group of learning rates, and using the learning rate with the best convergence effect as the initial learning parameter of the model.

9. An improved Faster R-CNN automatic detection system for soil flow plasticity, characterized in that: include: at least one memory for storing a program; At least one processor is used to enter the program stored in the memory, and when the program stored in the memory is entered, the processor is used to enter the method according to any one of claims 1 to 8.

Images