Shield tunneling dumping volume intelligent monitoring method based on image acquisition and laser scanning equipment

Abstract

The invention discloses an image acquisition and laser scanning equipment-based shield tunneling soil displacement intelligent monitoring method, which comprises the following steps of: performing real-time monitoring on operation influence factors of a shield tunneling machine, including environmental factors, the state of the shield tunneling machine and the like, predicting whether the shield tunneling machine is over-displacement or not, and giving an instruction in time; learning is continuously optimized through a neural network model, and the shield tunneling machine is controlled in real time through a control system. According to the invention, through identification of the muck elimination image of the shield tunneling machine, whether the muck is over-discharged or not is predicted, and an instruction is given in time. Meanwhile, a method for accurately measuring the dumping amount is provided, a two-dimensional model of the side section of the muck on the conveying belt is generated through a three-dimensional laser scanner, point cloud data are obtained, and the volume of the shield dumping is accurately measured in combination with a muck plane image recognized by a neural network. According to the method, the influence of factors such as geological environment actual construction is comprehensively considered, the muck discharge capacity is dynamically calculated and estimated in combination with real-time monitoring data, and the construction safety and efficiency are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of shield tunnel construction, and relates to a calculation and control method for the discharge volume of shield tunneling muck, and specifically to an intelligent monitoring method for the discharge volume of shield tunneling muck based on intelligent recognition of collected images and laser scanning of discharged muck. Background Art

[0002] During the excavation of subway tunnels, shield tunneling is an important core link. The amount of slag discharged during the shield excavation process not only affects the efficiency of the entire subway construction, but also directly affects the safety of the construction and the stability of the surrounding rock mass. Therefore, the control and monitoring of slag discharge and the prediction of the possible over-discharge risk are crucial in the entire construction process.

[0003] As the scale of subway construction gradually expands and construction continues to extend to areas with complex geological conditions, in actual projects, the amount of soil discharged by shield tunneling will interact with the environment, geological conditions, various construction parameters and the machine itself, causing complex changes in the stress field and deformation field of the surrounding rock mass. If the amount of soil excavation cannot be estimated and predicted in a timely and accurate manner, and the tunneling of the shield machine cannot be effectively controlled in a timely manner, it may lead to stress concentration, aggravated deformation, and even destruction of the rock and soil around the construction site, resulting in collapse, groundwater upwelling and other problems, which will have an adverse impact on the project and the environment.

[0004] At the same time, subway construction is becoming more and more extensive, and the geological environment conditions it faces are becoming more and more complex and changeable, with more and more types, and the environmental conditions will also change due to excavation. The amount of slag discharged and the judgment of whether it is over-discharged are closely related to environmental conditions, geological conditions, actual construction parameters, and the type and characteristics of shield machinery. There are many factors that need to be considered. The existing calculation of slag discharge is usually based on the actual excavated slag itself, by measuring the volume or weight of the slag actually transported on the slag truck or conveyor belt, and lacks real-time consideration and adjustment of various influencing factors in the actual specific construction process. The traditional measurement method ignores the dynamic changes of influencing factors during the construction process, has low accuracy and efficiency, and is difficult to ensure construction safety.

[0005] In addition, the traditional method of calculating the displacement of slag lacks a real-time monitoring and early warning mechanism, making it difficult to obtain and analyze the real-time displacement in a timely manner, and the calculation efficiency is very low. In addition, the lack of a complete system makes it impossible to make effective adjustments according to the actual situation, which poses construction risks and easily causes uncontrollable rock damage, threatening the safety of construction workers. Therefore, it is necessary to create an optimized method for measuring and estimating the amount of slag, and use non-contact means to dynamically adjust the parameters of factors such as influencing factors in real time to improve construction efficiency and ensure construction safety, in order to meet the construction needs of various complex shield tunneling. Summary of the invention

[0006] In view of the above-mentioned problems existing in the background technology, the present invention provides an intelligent monitoring method for shield tunneling discharge volume based on image acquisition and laser scanning equipment. The method predicts whether the discharge is over-discharged and gives instructions in time by identifying the shield machine's slag discharge image. At the same time, a method for accurately measuring the discharge volume is also provided. Through a three-dimensional laser scanner, a two-dimensional model of the slag side profile on the conveyor belt is generated and point cloud data is obtained. The volume of the shield discharge is accurately measured in combination with the slag plane image identified by the neural network. The present invention comprehensively considers the influence of factors such as the actual construction of the geological environment, and dynamically calculates and estimates the slag discharge volume in combination with real-time monitoring data, thereby improving construction safety and efficiency.

[0007] The objective of the present invention is achieved through the following technical solutions:

[0008] An intelligent monitoring method for shield tunneling soil volume based on image acquisition and laser scanning equipment includes the following steps:

[0009] Step 1: Required equipment and equipment installation

[0010] Step 1.1: Install a high-definition camera in front of the shield machine to ensure that the high-definition camera can fully collect images of the shield machine construction area, capture a complete picture of the slag and the real-time status of the slag, and set up a laser scanning device on the side of the shield machine to scan the side profile of the slag;

[0011] Step 1.2: Connect the high-definition camera to the image preprocessing device, configure the device driver and corresponding software, and prepare for operation. After the equipment is installed, perform functional testing to ensure the integrity of image acquisition, the stability of data transmission, and the accuracy of measurement results. According to the test results, make corresponding adjustments and optimization measures to achieve the optimal system performance.

[0012] Step 2: Collect the soil image

[0013] When the shield machine is running, start the high-definition camera to collect real-time muck images, save the images in the specified folder, use Python to process the images, and pre-set the resolution, frame rate, and exposure time parameters to ensure image quality;

[0014] Step 3: Image Preprocessing

[0015] Step 3.1: Perform Gaussian filtering and denoising on the image to reduce the impact of noise on edge detection results;

[0016] Step 3.2: grayscale the image and convert the soil color image into a grayscale image;

[0017] Step 3.3: Convert the image into a binary image and automatically determine the threshold using the two-dimensional Otsu algorithm optimized based on the wolf pack algorithm;

[0018] Step 3.4: Enhance the contrast of the image by histogram equalization.

[0019] Step 3.5: Use the Canny edge detection algorithm to extract the soil boundary, and use the Sobel operator to convert the gradient direction into a color map for visualization;

[0020] Step 3.6: Extract the specific color area of ​​the slag in the RGB space, identify the slag area in the image and determine its edge;

[0021] Step 3.7: Store the preprocessed images and transfer them to the machine learning device for subsequent machine learning model training;

[0022] Step 4: Import the preprocessed images into the deep learning model based on the YOLOv5 target monitoring algorithm, and through continuous learning and optimization, the purpose of judging the discharge status of the shield set by identifying the muck image can be achieved;

[0023] Step 5: Use the deep learning model trained in step 4 to classify the images captured by the high-definition camera and determine the emission status. If the normal emission requirements are not met, an alarm will be issued in time;

[0024] Step 6: Set up the early warning module to receive the image classification determined by the deep learning model. In normal conditions, no alarm is issued. In cases of slight over-discharge or under-discharge, only record but no alarm is issued. In case of severe over-discharge, an alarm is issued immediately.

[0025] Step 7: Establish a feedback system to dynamically adjust the operation of the shield machine: Based on the data analysis results obtained by deep learning model recognition, including the judgment of whether the slag is over-discharged and the calculation of the actual discharge amount, the feedback system decides whether it is necessary to adjust the parameters of the shield machine operation or stop the operation for inspection, and guides and predicts the operation and construction adjustment of the shield machine in the future based on the system's simulation estimation;

[0026] Step 8: Record and summarize the construction: Organize the simulation results into a report, record the operation status of the shield machine, the mechanical status, and the possible or existing construction risks to evaluate the construction process, and serve as a reference for future related or similar projects.

[0027] Compared with the prior art, the present invention has the following advantages:

[0028] 1. Real-time monitoring of environmental factors by image acquisition equipment can make the simulation results more realistic, take into account more comprehensive factors, and provide timely feedback and adjustments, thereby reducing engineering risks.

[0029] 2. By establishing a neural network learning system based on the optimized YOLOv5 algorithm to conduct in-depth analysis of the collected data, more accurate and convenient analysis and recording of data that is difficult to detect in real time by humans can be performed in the project, so as to have a more accurate and in-depth understanding of the construction environment, discover risks in a timely manner, and make accurate response decisions quickly.

[0030] 3. The feedback warning module can dynamically adjust or suspend construction in a timely manner according to the analysis results to avoid potential risks, making the construction process more accurate and efficient, and guiding the subsequent excavation process by predicting future construction.

[0031] 4. Through learning and optimizing image recognition judgment, the accuracy is gradually improved through machine learning. The amount of slag emissions can be obtained by monitoring the environment and shield status, reducing human calculations and resource waste.

[0032] 5. By recording images of various construction environment conditions and applying them to machine learning, the system can be applied to shield tunneling under various construction environment conditions, expanding the application scope of the monitoring and control system and improving construction efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a flow chart of the intelligent monitoring method for shield tunneling discharge based on image acquisition and laser scanning equipment. DETAILED DESCRIPTION

[0034] The technical solution of the present invention is further described below in conjunction with the accompanying drawings, but is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention should be included in the protection scope of the present invention.

[0035] The present invention provides an intelligent monitoring method for shield tunneling soil volume based on image acquisition and laser scanning equipment. The method monitors the factors affecting the operation of the shield machine, including environmental factors, the status of the shield machine, etc. in real time, predicts whether there is over-discharge and gives instructions in time, and continuously optimizes and learns through a neural network model, and achieves the purpose of real-time control of the shield machine through a control system. Figure 1 As shown, the specific steps include:

[0036] Step 1: Required equipment and equipment installation.

[0037] Step 1.1: Equipment selection. Select a high-definition camera that is suitable for the site environment. Make sure that the high-definition camera has sufficient resolution (1920×1080) and frame rate (30 frames per second), is waterproof and dustproof, and can provide good imaging in low-light environments. Set up an image transmission system and image preprocessing equipment. The image preprocessing equipment is mainly a computer device equipped with a Python program, which can process the edges of the image and more accurately identify the image of the slag.

[0038] Step 1.2: Determine the location of the image acquisition equipment and laser scanning. Install the high-definition camera in front of the shield machine to ensure that the high-definition camera can fully collect images of the shield machine construction area, and can capture complete pictures of the slag and the real-time status of the slag to avoid being blocked by obstacles. Generally, it is not moved at will, and the high-definition camera is reinforced and protected to prevent the camera from being disturbed and damaged. Set up protective and shockproof equipment to ensure that the high-definition camera remains stable during operation, avoid affecting the image quality, and reduce the error of image acquisition. In addition, a laser scanning device is set on the side of the shield equipment to scan the side profile of the slag.

[0039] Step 1.3: Set up the connection equipment. Ensure that the HD camera has a stable power supply, connect the HD camera to the image preprocessing equipment, configure the device driver and corresponding software, and prepare for operation. In order to ensure that the data can be transmitted quickly and smoothly, choose a suitable connection method, such as HDMI, USB or network connection, use a high-speed data transmission interface (Ethernet), and use the image transmission system to quickly and smoothly transmit image laser scanning and other data to the image preprocessing equipment. After the equipment is installed, a series of functional tests are required to ensure the integrity of image acquisition, the stability of data transmission, and the accuracy of measurement results. According to the test results, make corresponding adjustments and optimization measures to achieve the optimal system performance.

[0040] Step 2: Collect the soil image.

[0041] Step 2.1: Connect the HD camera to the image preprocessing device equipped with Python to ensure that the HD images captured by the HD camera are transmitted smoothly in real time.

[0042] Step 2.2: Install the driver that matches the HD camera used in the image preprocessing device to ensure that the image preprocessing device can recognize the HD camera and transmit the data to the image processing program in Python.

[0043] Step 2.3: When the shield machine is running, start the high-definition camera to collect real-time muck images, save the images in the specified folder, process the images using Python, and pre-set parameters such as resolution, frame rate, exposure time, etc. to ensure image quality.

[0044] Step 3: Image preprocessing:

[0045] The collected images are preprocessed to eliminate the interference of the external environment, so as to better extract the image feature information.

[0046] Step 3.1: Gaussian filtering denoising: Before edge detection, the image is denoised to reduce the impact of noise on the edge detection results.

[0047] Step 3.1.1: Determine the Gaussian kernel: Determine the two-dimensional array of the Gaussian kernel. The size of the generated kernel is set to 7×7. The calculation formula of the Gaussian kernel is as follows:

[0048]

[0049] Here, x and y are the distances from the center of the kernel, and σ is the standard deviation, which controls the width of the Gaussian distribution, i.e. how “blurry” the kernel is.

[0050] Step 3.1.2: Normalized Gaussian kernel: In order to keep the brightness of the image unchanged, the kernel needs to be normalized so that the sum of all elements is 1.

[0051] Step 3.1.3: Apply the Gaussian kernel: For each pixel in the image, multiply the pixel value in its neighborhood by the value of the corresponding position of the Gaussian kernel, and then sum them to get the new pixel value. This process can be expressed by the following formula:

[0052]

[0053] Among them, I′(x,y) is the pixel value of the filtered image at point (x,y), I(x+i,y+j) is the pixel value of the original image at point (x+i,y+j), G(i,j) is the value of the Gaussian kernel at position (i,j), and k is the radius of the Gaussian kernel.

[0054] Step 3.2: Perform grayscale processing on the image, convert the soil color image into a grayscale image, and convert the RGB value into a grayscale value by weighted summation:

[0055] I=0.299R+0.587G+0.114BI=0.299R+0.587G+0.114B

[0056] Among them, I is the grayscale value, and R, G, and B are the values ​​of the red, green, and blue channels respectively.

[0057] Use the OpenCV function cv2.cvtColor to convert the image from color to grayscale. The code is as follows: import cv2

[0058] #Read the image

[0059] image=cv2.imread('path_to_image.jpg')

[0060] #Convert to grayscale image

[0061] gray_image=cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)

[0062] #Display grayscale image

[0063] cv2.imshow('Gray Image',gray_image)

[0064] cv2.waitKey(0)

[0065] cv2.destroyAllWindows()

[0066] #Save grayscale image

[0067] cv2.imwrite('path_to_save_gray_image.jpg',gray_image)

[0068] Step 3.3: Perform image threshold processing: Convert the image into a binary image to reduce the interference of the image under external conditions such as lighting on the recognition, thereby enhancing the important features in the image and removing background noise. The threshold is automatically determined using the two-dimensional Otsu algorithm optimized based on the wolf pack algorithm.

[0069] Step 3.3.1: Use the two-dimensional Otsu algorithm to search all pixels in the image and find the most appropriate segmentation threshold to improve and stabilize edge detection accuracy. The specific operation can be done by using the `cv2.threshold` function in image processing libraries such as OpenCV, and automatically calculate and apply the Otsu threshold by setting the `type` parameter to `cv2.THRESH_BINARY+cv2.THRESH_OTSU`.

[0070] Step 3.3.2: Introduce the wolf pack algorithm into the two-dimensional Otsu algorithm, and use the powerful search ability of the wolf pack algorithm to find the optimal threshold in the image. The specific steps are: Initialize the gray wolf group: randomly generate a certain number of gray wolf individuals, and assign an initial position to each gray wolf; each gray wolf individual is an image threshold; calculate the fitness of each gray wolf individual according to the objective function of the gray wolf optimization algorithm; update the gray wolf position: according to the fitness of each gray wolf in the gray wolf group, use the collaborative behavior of the gray wolf group and the competitive behavior of the individual to update the gray wolf individual; finally, check whether the maximum number of iterations is reached or the maximum fitness meets the requirements. If the maximum number of iterations is not reached or the maximum fitness does not meet the requirements, repeat the above steps. If it meets the requirements, directly output the gray wolf individual with the largest fitness as the optimal threshold of the image.

[0071] Step 3.4: Enhance the contrast of the image through histogram equalization. The code is as follows: import cv2

[0072] import numpy as np

[0073] #Read the image

[0074] image=cv2.imread('path_to_image.jpg',cv2.IMREAD_GRAYSCALE)

[0075] # Apply histogram equalization

[0076] equ_image=cv2.equalizeHist(image)

[0077] # Display the original image and the equalized image

[0078] cv2.imshow('Original Image',image)

[0079] cv2.imshow('Equalized Image',equ_image)

[0080] cv2.waitKey(0)

[0081] cv2.destroyAllWindows()

[0082] Step 3.5: Extract the boundary of the denoised and contrast-enhanced image: Use the Canny edge detection algorithm to extract the boundary of the slag, and use the Sobel operator to convert the gradient direction into a color map for visualization, so as to better judge the type of slag and distinguish the edges based on the color.

[0083] Step 3.5.1: Calculate the magnitude and direction of the gradient, use the Sobel operator to calculate the gradient of the image in the x and y directions, and get the horizontal direction (G x ) and vertical direction (G y ) to find the edge gradient and direction of each pixel. For each pixel, the magnitude of the gradient is calculated using the following formula:

[0084]

[0085] For each pixel, the direction of the gradient is calculated using the following formula: θ represents the angle between the gradient vector and the horizontal axis.

[0086] Step 3.5.2: Normalize the gradient magnitude to a range of 0 to 255 for image display. This can be achieved using the OpenCV cv2.normalize function:

[0087] gradient_magnitude=cv2.normalize(gradient_magnitude,None,0,255,

[0088] cv2.NORM_MINMAX,dtype=cv2.CV_8U)

[0089] Step 3.5.3: The gradient direction is a range from -π to π. To convert these angles to colors, the gradient direction can be mapped to the hue (H) component of the HSV color space, where the hue ranges from 0 to 360 degrees or 0 to 255 (if 8-bit representation).

[0090] First, convert the angle from radians to degrees:

[0091] gradient_direction_degrees=np.rad2deg(gradient_direction)

[0092] Then, scale the angle to the range of 0 to 360 degrees or 0 to 255 and use it as the H component of the HSV image:

[0093] h=(gradient_direction_degrees+180) / 360*255 #Convert the angle range from -180 to 180 to 0 to 255.

[0094] Step 3.5.4: Map the normalized gradient direction value to the hue (H) component of the HSV color space. The H component of the HSV color space can represent the type of color, and the range is usually 0 to 360 degrees or 0 to 255 (if it is 8-bit representation).

[0095] Step 3.5.5: Set saturation and brightness. The saturation (S) and brightness (V) components can be set to fixed values ​​or adjusted according to the gradient amplitude. Saturation is usually set to the maximum value to obtain vivid colors, and brightness can be adjusted according to the gradient amplitude to reflect the strength of the gradient.

[0096] Step 3.5.6: Convert the color value in HSV color space to RGB color space. To convert the given HSV value H, S, V to RGB, first normalize the HSV value, that is, convert the value of H, S, V to a value between 0 and 1, and then calculate the intermediate variable:

[0097] Color purity: C = V × SC = V × S

[0098] The middle value of R, G, B: X = C × (1-abs ((H / 60)) X = C × (1-abs ((H / 60)

[0099] Offset of R, G, B: m = V - Cm = VC

[0100] According to the value of H, determine which interval it is in and calculate R, G, B:

[0101] When 0≤H<60 degrees: R=CG=XB=0

[0102] When 60≤H<120 degrees: R=XG=CB=0

[0103] When 120≤H<180 degrees: R=0G=CB=X

[0104] When 180≤H<240 degrees: R=0G=XB=C

[0105] When 240≤H<300 degrees: R=XG=0B=C

[0106] When 300≤H<360 degrees: R=CG=0B=X

[0107] Finally, add the calculated R, G, B values ​​to the offset m: R′=R+m, G′=G+m, B′=B+m to ensure that the RGB values ​​are within the valid range of 0 to 1.

[0108] Step 3.6: Extract the specific color area of ​​the slag in the RGB space, identify the slag area in the image and determine its edge.

[0109] Step 3.6.1: Use an image processing library such as OpenCV to read the image to be processed.

[0110] import cv2

[0111] img=cv2.imread('path_to_image.jpg')

[0112] Step 3.6.2: Find a suitable threshold range. Determine the RGB threshold of the color you want to extract. Setting thresholds in RGB space usually involves distinguishing or extracting specific areas in the image based on the values ​​of the three components of the color: red (R), green (G), and blue (B). In RGB space, the value range of each color component is usually 0 to 255. Define the minimum and maximum values ​​for each color component based on the color characteristics of the dirt and non-dirt areas of the identified image.

[0113] #For example, extract the red range:

[0114] lower_red=np.array([0,120,70])

[0115] upper_red=np.array([10,255,255])

[0116] Step 3.6.3: Create a mask based on the defined color range using cv2.inRange() function. This mask will be used to extract the region of a specific color from the image.

[0117] mask=cv2.inRange(img,lower_red,upper_red)

[0118] Step 3.6.4: Apply the mask to the original image to extract the color regions that meet the threshold.

[0119] result=cv2.bitwise_and(img,img,mask=mask)

[0120] Step 3.6.5: Display or save the extracted color area, and gradually adjust and optimize these parameters by observing the extraction effect. It is necessary to continuously adjust the thresholds of the values ​​of the three components of red (R), green (G), and blue (B) to continuously improve the recognition effect and accuracy.

[0121] cv2.imshow('Extracted Color Region',result)

[0122] cv2.waitKey(0)

[0123] cv2.destroyAllWindows()

[0124] Step 3.7: Store the preprocessed images and transfer them to the machine learning device for subsequent machine learning model training.

[0125] Step 4: Import the preprocessed images into the deep learning model based on the YOLOv5 target monitoring algorithm, and through continuous learning and optimization, the purpose of judging the discharge status of the shield set by identifying the slag image can be achieved.

[0126] Step 4.1: Collect as many images of discharged slag as possible. The images need to take into account different working conditions, different lighting conditions, different geological environmental conditions, different construction methods and construction environments, etc. Images should be collected under different factors affecting the construction conditions so that the sample images cover a variety of site conditions, improve the recognition and judgment capabilities of the model, and ensure that the collected sample size is large enough. At least hundreds of images should be collected under each discharge state of the shield machine, including under-discharge, normal discharge, slight over-discharge, and severe over-discharge, to improve the generalization ability of the model and reduce overfitting.

[0127] Step 4.2: Label the collected images, classify and label the collected slag images into under-discharge, normal, slightly over-discharge, and severely over-discharge, and store them in files.

[0128] Step 4.2.1: According to the project requirements and actual construction conditions, the soil slag status is divided into multiple categories, namely under-emission status (the soil slag does not reach the required discharge volume and the construction is slow), normal status (the soil slag discharge is consistent with the requirements and there is no possibility of danger), slight over-emission (the soil slag discharge slightly exceeds the expected value and there may be a possibility of danger) and severe over-emission (the soil slag discharge obviously exceeds the standard and will affect the construction safety).

[0129] Step 4.2.2: Import the collected images into the annotation tool and use Labelmg software to create classification labels. Before annotation, it is recommended to create a classes.txt file to list all the object categories that need to be annotated. In this way, you can directly select the category from the drop-down list when annotating without manual input. If needed, different construction sites and different types of slag can be annotated more carefully, such as the size and color of the slag particles, the rock properties of the site, the hydrological conditions, etc., for subsequent analysis.

[0130] Step 4.2.3: After data annotation, save the labeled .txt file and the corresponding source image in the YOLOv5 dataset format. The data folder gw is created in the same directory as the training code train.py, that is, the first-level directory YOLOv5-master folder. The datasets file is created under the gw file, and two folders are created in the datasets folder: images and labels; the inages folder stores the source images, and the labels folder stores the annotation box training information.

[0131] Step 4.3: Use data augmentation techniques to generate additional training samples and expand the training set.

[0132] Step 4.3.1: Perform geometric transformation on the original image, including:

[0133] Flip: Flip the image horizontally or vertically to increase data diversity.

[0134] Rotation: Rotate the image by a certain angle (within 45 degrees), which is suitable for image recognition tasks.

[0135] Cropping: Randomly cropping a portion of the image helps the model learn different parts of the image.

[0136] Scaling: Adjust the size of the image appropriately and increase the number of training samples at different scales.

[0137] Translation: Moving an image in the image plane to simulate a change in the position of an object.

[0138] Step 4.3.2: Perform pixel transformation on the image: including adjusting the brightness, contrast, and saturation of the image, simulating different lighting conditions, adding Gaussian noise and salt and pepper noise to the image to improve the model's robustness to noise, and using Gaussian blur, motion blur, etc. to simulate poor image quality.

[0139] Step 4.3.3: Randomly crop an area in the image and fill it with 0 so that the model learns not to rely on local features or mix the two images in proportion to generate new training samples and mix the labels of the two images.

[0140] Step 4.4: Normalize and standardize the data to make the input data consistent.

[0141] Step 4.4.1: Image normalization: Identify and eliminate errors, omissions, and anomalies in the data. Here is a method for normalizing sample data:

[0142]

[0143] Where, X n is the data between -1 and 1 after the change; X max is the maximum value of the data; X min is the minimum value of the data; Xi is the i-th input data; if the sample data does not meet the stationarity requirements during the processing, repeat the above steps until it meets the stationarity requirements.

[0144] Step 4.4.2: Image standardization: Through standardization, data of different units or magnitudes in the data set are compared and analyzed, and the influence of outliers or extreme values ​​is removed to better discover the laws and trends in the data. All types of data are reduced in dimension so that their values ​​are not affected by the measurement unit, and become a standardized data set with a mean of 0, a standard deviation of 1, and a range of [-1, 1], so as to improve the comparability of the data. The formula for Z-score standardization is as follows:

[0145] Z=(X-μ)\σ

[0146] Where Z is the standardized score, X is the original data value, μ is the mean of the original data, and σ is the standard deviation of the original data.

[0147]

[0148] Where X is the original data point, that is, an observation or sample in the data set; μ is the average value (mean) of the data set, which is calculated as the sum of all data points divided by the number of data points; X i is the ith data point in the data set, n is the total number of data points; σ is the standard deviation of the data set, which measures the distribution or dispersion of the data points around the mean; Z is the standardized data point, also known as the Z-score, which represents the difference between the original data point X and the mean μ, in units of standard deviation σ; the Z-score represents how many standard deviations the original data point is from the mean.

[0149] Step 4.5: Construct training set, validation set and test set. Create three folders in the images and labels folders in the datasets folder: train, test and verify; the source images and annotation box training information of the training set are stored in the train folder, the source images and annotation box training information of the test set are stored in the test folder, and the source images and annotation box training information of the verification set are stored in the verify folder. Datasets are usually divided into training set, validation set and test set. The training set is used to train the model, the validation set is used for model selection and hyperparameter adjustment, and the test set is used to finally evaluate the generalization ability of the model. 70% of the generated training sample data is used for training, 15% for validation, and 15% for testing.

[0150] Step 4.5.1: When dividing the data set, the randomness of the samples must be ensured. Therefore, the data set is randomly shuffled before division, and a stratified sampling method is used to ensure that the proportions of the categories of landfill emissions in the training set, validation set, and test set are similar. If the category distribution is uneven, some techniques (such as resampling, category weights, etc.) may be needed to deal with it.

[0151] Step 4.5.2: After completing the data set division, the sample size and category proportion of each data set need to be verified to ensure the rationality of the division results and their effectiveness for subsequent model training and evaluation. Subsequently, the model training, verification and final testing phases can be carried out to ensure that the model can show expected performance on different data sets.

[0152] Step 4.6: Establish an improved YOLOv5 model. The YOLOv5 model includes a Backbone network, a Neck network, and a Head network. The Backbone network of YOLOv5 incorporates the attention mechanism module SE of the convolution module, and inserts the attention mechanism module of the convolution module between the last C3 module and the SPPF module in the Backbone of YOLOv5 to obtain the improved YOLOv5 model.

[0153] Step 4.6.1: Based on the high accuracy and efficiency of the network EfficientNet, replace the feature extraction network of YOLOv5 to extract the features of the muck image. First, perform a 1×1 convolution on the input feature map to increase the dimension and standardize it; after adding the activation function, perform a 3×3 or 5×5 depth-separable convolution to introduce the SE attention mechanism module; then connect a Conv1×1 ordinary convolution to reduce the dimension, perform a Dropout operation, and complete the feature map fusion. To fuse EfficientNet with YOLOv5 as a feature extraction network. Add the network structure code of EfficientNet in the `models / common.py` file of YOLOv5. This includes defining the `stem` class and `MBConvBlock` class of EfficientNet, which are the basic modules that constitute EfficientNet; in the `models / yolo.py` file, add the class name of EfficientNet to the network structure of YOLOv5. This involves setting the parameter transmission details of the network structure to ensure that the EfficientNet module can be recognized and used by YOLOv5; copy the `yolov5s.yaml` file in the `models` folder and rename it to, for example, `yolov5s_EfficientNetv2.yaml`. Then modify the configuration file according to the network structure of EfficientNetv2 to ensure that the size of the feature map is correctly transformed, such as / 8, / 16, / 32, etc.; finally, modify the `--cfg` default parameter in the `train.py` file to specify the new model structure configuration file during training.

[0154] Step 4.6.2: Introduce the CBAM attention mechanism, combine the CBAM (Convolutional Block Attention Module) convolutional block attention module with the channel and spatial attention modules to enhance the model's ability to express shallow target feature information, and improve the recognition accuracy of cracks. Given an intermediate feature map F as input, first perform maximum pooling and average pooling operations on it, merge the results of the two pooling layers into the fully connected layer for calculation, and finally add them together to generate the attention MC on the channel dimension, multiply MC with the input feature map to obtain the feature map F′ after the channel dimension is adjusted. Then perform maximum pooling and average pooling operations on F′ in space, concatenate the obtained two-dimensional vectors, and finally convolve to generate the spatial attention MS.

[0155] The calculation formula of CAM is: Mc(F) = σ(MLP(AvgPool(F))+MLP(MaxPool(F)))

[0156] The calculation formula of SAM is: Ms(F) = σ(f7×7([AvgPool(F); MaxPool(F)]))

[0157] The calculation process of the CBAM module is: CBAM(F)=Ms(Mc(F)⊙F)⊙F

[0158] In Python, the CBAM module can be defined and integrated into a convolutional neural network.

[0159] Step 4.6.3: Add an improved image attention mechanism based on the LSTM model to the downsampling process of the YOLOv5 network layer. LSTM controls the flow of information by introducing a gating mechanism (gates), so that it can learn long-term dependencies. The LSTM model can process the relationship between data and time through long-term memory to predict the shield emission state at future moments. In the LSTM model, in addition to the traditional input layer, output layer, and hidden layer, a memory unit (MemoryCel1) and three gating units (Porget Gate, Input Gate, Output Gate) are added. t is the input gate, f t is the forget gate, g t For input supply, o t is the output gate.

[0160] The forget gate is used to control how much past information will be selected and forgotten, which is determined by the output h of the previous cell. t-1 and the current input x t Make a Sigmoid nonlinear mapping to get a vector f with each dimension value between 0 and 1 t, the formula is as follows:

[0161] f t =σ(W f [h t-1 ,x t ]+b f )

[0162] Where: f t is the output of the forget gate, which is a vector with values ​​of each dimension between 0 and 1; σ is the Sigmoid activation function, which maps the input to between 0 and 1; W f is the weight matrix of the forget gate; h t-1 is the hidden state of the previous time step; x t is the input of the current time step; b f is the bias vector of the forget gate. This formula indicates that the output of the forget gate is the hidden state h of the previous time step. t-1 and the input x at the current time step t The concatenation of f , plus the bias vector b of the forget gate f , and finally obtained through the Sigmoid activation function.

[0163] The forget gate vector will be combined with the cell state C t-1 When the value of the cell state is multiplied by 0, the result is close to 0, indicating that the information is forgotten; when it is multiplied by 1, the value itself is obtained, indicating that the information is completely stored. Through this mechanism, the invitation-forget gate realizes the memory of important information and the forgetting of unimportant information.

[0164] The input gate is used to control the storage ratio of the current input information, which is determined by the output h of the previous cell. t-1 and the current input x t A Sigmoid nonlinear mapping function layer is used to determine which part of the cell state value to update, and then a tanh function layer is used to generate new candidate information a. t , the formula is as follows:

[0165] i t =σ(W f ·[h t-1 ,x t ]+b f )

[0166] a t =σ(W a ·[h t-1 ,x t ]+b a )

[0167] The update formula of the cell state is as follows:

[0168] C t =f t ☉C t-1 +i t ☉a t

[0169] Among them, the old state of the cell C t-1 and f t Multiply and discard the information to be forgotten, and add the main i t ☉a t As new information is added to the cell state, the cell state is updated.

[0170] The function of the output gate is to decide how much current information to output. The output ht-1 of the previous cell and the current input xt are used to determine which parts of the cell state to output through a Signoid function layer. The cell state Ct is then processed through a tanh function layer and multiplied by the output of the Sigmoid layer. Finally, the part that is determined to be output is output. The formula is as follows:

[0171] ot=σ(Wo·[ht-1,xt]+bo)

[0172] ht=ot☉tanh(Ct)

[0173] Step 4.7: Set parameters for model training. The parameters for model training are as follows: nc=1 in the YOLOv5s.yaml file; set the initial weight in the training code train.py, that is, set the parameter weights, set the training model file and data set parameter file, that is, set the parameters cfg and data, set the number of iterations, that is, set the parameter epochs, set the number of batch processing files, that is, set the parameter batch-size, set the image size, that is, set the parameter imgsz, set GPU acceleration, that is, set the parameter device, set the multi-threading setting, that is, set the parameter workers; save the generated weight files last.pt and best.pt, where last.pt represents the weight obtained in the last round of training, and best.py represents the weight with the best effect during training.

[0174] Step 4.8: Hyperparameter optimization for model training: YOLOv5 provides a method for hyperparameter evolution, which can be set in the hyp.scratch.yaml file. However, due to its long time and high cost, other genetic algorithms can be combined to optimize the hyperparameters.

[0175] Step 4.8.1: Use the Bayesian optimization algorithm and the Adam algorithm to optimize the improved YOLOv5 model established in step 4.6. When optimizing the Bayesian algorithm, the hyperparameters are selected using the following formula:

[0176] Posterior mean:

[0177] mpost(x)=k(x,X)[K(X,X)+σ2noiseI]-1y

[0178] Where k(x,X) is the covariance vector of the kernel function between point x and the training dataset X, K(X,X) is the covariance matrix on the training dataset X, σ2noise is the noise variance, I is the identity matrix, and y is the target value vector of the training dataset.

[0179] Posterior covariance:

[0180] kpost(x,x')=k(x,x')-k(x,X)[K(X,X)+σ2noiseI]-1k(X,x')

[0181] Where k(x,x′) is the covariance of the kernel function between points x and x′, k(x,X) and k(X,x′) are the covariance vectors between point x and training dataset X and between training dataset X and point x′, respectively.

[0182] Probability Improvement:

[0183]

[0184] Where Φ is the CDF of the standard normal distribution, μ(x) and σ(x) are the predicted mean and standard deviation of the Gaussian process at x, respectively, and f(x+) is the best objective function value currently known.

[0185] In the framework of Bayesian optimization, the Adam algorithm can be used as the optimizer of the objective function. The Adam algorithm can be used to optimize the objective function in each step of Bayesian optimization. By using the adaptive learning rate feature of the Adam algorithm, the learning rate can be dynamically adjusted for different parameters in each iteration of Bayesian optimization to find the optimal solution of the objective function more quickly. The specific fusion method is as follows:

[0186] 1) Initialization: Select a set of standardized initial parameters and use the Adam algorithm to optimize these parameters to obtain the initial objective function value.

[0187] 2) Build a proxy model: Use the Gaussian process model in Bayesian optimization to approximate the objective function.

[0188] 3) Define the acquisition function: In Bayesian optimization, the acquisition function (such as EI, PI or UCB) is used to select the next most promising parameter combination. In the case of Adam fusion, the acquisition function can be combined with the gradient information in the Adam algorithm to guide the selection of parameters.

[0189] 4) Optimize the acquisition function: Find the parameter point that maximizes the acquisition function. This point is the candidate point of the next objective function to be evaluated.

[0190] 5) Evaluate the objective function: Use the Adam algorithm to evaluate the objective function at the newly selected parameter points to obtain new observations.

[0191] 6) Update the proxy model: Add new data points to the dataset and update the Gaussian process model.

[0192] 7) Repeat: Repeat the above steps 1) to 6) until a certain stop condition is met.

[0193] Step 4.8.2: Use mean absolute error (MAE), root mean square error (RMSE) and coefficient of determination R2 to evaluate the performance of the model and determine whether the hyperparameters have converged.

[0194] In order to make a more comprehensive and accurate evaluation of the established prediction model, the prediction accuracy is evaluated by a variety of evaluation methods. The present invention uses three indicators, namely, mean absolute error (MAE), root mean square error (RMSE) and determination coefficient R2, to evaluate the performance of the model. The lower the MAE and RMSE values ​​of the model's prediction results, the closer the R2 value is to 1, which means that the deviation between the posture parameter prediction result and the true value is lower, that is, the higher the prediction accuracy. Its calculation is shown in the following formula:

[0195]

[0196] Among them, y i is the actual value, x i is the predicted value and n is the number of samples.

[0197]

[0198] Among them, y i Actual value, x i is the predicted value and n is the number of samples.

[0199]

[0200] Among them, Y i is the actual value, f i is the predicted value, is the mean of the actual values, and n is the number of samples.

[0201] Step 4.9: Input the divided training set into the improved YOLOv5 model, evaluate the accuracy of the model, and continuously optimize the hyperparameters. Save the model structure when the prediction effect is optimal, use the optimal prediction model to predict the discharge of slag, and use the test set parameters to test the trained model to determine whether the model meets expectations. When the model meets the training requirements, the model can be used to automatically identify different slag states: under-discharge, normal, slightly over-discharge, and severely over-discharge.

[0202] Step 5: Put the established neural network model into practical application: According to the deep learning model trained in step 4, the images taken by the high-definition camera are classified and the emission conditions are judged. If the normal emission requirements are not met, an alarm is issued in time.

[0203] Step 5.1: Set the unit time length and collect a specified number of photos within a certain period of time. Use the trained YOLOv5 model to classify the collected images and calculate the proportion of each category of slag in the photos. Set the threshold to 0.7. If the probability of "severe over-discharge" exceeds the set threshold of 0.7, the current slag status is determined to be severely over-discharged.

[0204] Step 5.2: Measure the actual volume of the slag discharge. The point cloud data of the slag profile corresponding to each frame of the image is obtained by laser scanning equipment to obtain the slag profile map, and the average accumulation height of the slag area is calculated. The slag plane area obtained by extracting the edge of the slag plane image using the YOLOv5 model is multiplied by the average accumulation height to obtain the discharge volume of the slag corresponding to each frame of the image.

[0205] The volume of soil discharge per unit time is:

[0206]

[0207] Where n is the total number of frames per unit time.

[0208] Step 6: Set up the early warning module to receive the image classification determined by the deep learning model. No alarm will be issued in normal conditions. Minor over-discharge and under-discharge will only be recorded but no alarm will be issued. Severe over-discharge will immediately issue an alarm.

[0209] Step 7: Establish a feedback system to dynamically adjust the operation of the shield machine: Based on the data analysis results obtained by deep learning model recognition, including the judgment of whether the slag is over-discharged and the calculation of the actual discharge amount, the feedback system decides whether it is necessary to adjust the parameters of the shield machine operation or stop the operation for inspection, and can provide guidance and prediction for the subsequent operation and construction adjustments of the shield machine based on the system's simulation estimates.

[0210] Step 8: Record and summarize the construction: Organize the simulation results into a report, record the operation status of the shield machine, the mechanical status, possible or existing construction risks, etc. to evaluate the construction process and serve as a reference for future related or similar projects.

Claims

1. An intelligent monitoring method for shield tunneling soil discharge based on image acquisition and laser scanning equipment, characterized in that The method comprises the following steps: Step 1: Required equipment and equipment installation Step 1.1: Install a high-definition camera in front of the shield machine to ensure that the high-definition camera can fully collect images of the shield machine construction area, capture a complete picture of the slag and the real-time status of the slag, and set up a laser scanning device on the side of the shield machine to scan the side profile of the slag; Step 1.2: Connect the high-definition camera to the image preprocessing device, configure the device driver and corresponding software, and prepare for operation. After the equipment is installed, perform functional testing to ensure the integrity of image acquisition, the stability of data transmission, and the accuracy of measurement results. According to the test results, make corresponding adjustments and optimization measures to achieve the optimal system performance. Step 2: Collect the soil image When the shield machine is running, start the high-definition camera to collect real-time muck images, save the images in the specified folder, use Python to process the images, and pre-set the resolution, frame rate, and exposure time parameters to ensure image quality; Step 3: Image Preprocessing Step 3.1: Perform Gaussian filtering and denoising on the image to reduce the impact of noise on edge detection results; Step 3.2: grayscale the image and convert the soil color image into a grayscale image; Step 3.3: Convert the image into a binary image and automatically determine the threshold using the two-dimensional Otsu algorithm optimized based on the wolf pack algorithm; Step 3.4: Enhance the contrast of the image by histogram equalization. Step 3.5: Use the Canny edge detection algorithm to extract the soil boundary, and use the Sobel operator to convert the gradient direction into a color map for visualization; Step 3.6: Extract the specific color area of ​​the slag in the RGB space, identify the slag area in the image and determine its edge; Step 3.7: Store the preprocessed images and transfer them to the machine learning device for subsequent machine learning model training; Step 4: Import the preprocessed images into the deep learning model based on the YOLOv5 target monitoring algorithm, and through continuous learning and optimization, the purpose of judging the discharge status of the shield set by identifying the muck image can be achieved; Step 5: Use the deep learning model trained in step 4 to classify the images captured by the high-definition camera and determine the emission status. If the normal emission requirements are not met, an alarm will be issued in time; Step 6: Set up the early warning module to receive the image classification determined by the deep learning model. In normal conditions, no alarm is issued. In cases of slight over-discharge or under-discharge, only record but no alarm is issued. In case of severe over-discharge, an alarm is issued immediately. Step 7: Establish a feedback system to dynamically adjust the operation of the shield machine: Based on the data analysis results obtained by deep learning model recognition, including the judgment of whether the slag is over-discharged and the calculation of the actual discharge amount, the feedback system decides whether it is necessary to adjust the parameters of the shield machine operation or stop the operation for inspection, and guides and predicts the operation and construction adjustment of the shield machine in the future based on the system's simulation estimation; Step 8: Record and summarize the construction: Organize the simulation results into a report, record the operation status of the shield machine, the mechanical status, and the possible or existing construction risks to evaluate the construction process, and serve as a reference for future related or similar projects.

2. The intelligent monitoring method for shield tunneling soil discharge based on image acquisition and laser scanning equipment according to claim 1 is characterized in that The specific steps of step 3.3 are as follows: Step 3.3.1: Use the two-dimensional Otsu algorithm to optimally search all pixels in the image and find the most appropriate segmentation threshold to improve and stabilize edge detection accuracy; Step 3.3.2: Introduce the wolf pack algorithm into the two-dimensional Otsu algorithm and use the powerful search capability of the wolf pack algorithm to find the optimal threshold in the image.

3. The intelligent monitoring method for shield tunneling soil discharge based on image acquisition and laser scanning equipment according to claim 1 is characterized in that The specific steps of step 3.5 are as follows: Step 3.5.1: Calculate the magnitude and direction of the gradient, use the Sobel operator to calculate the gradient of the image in the x and y directions, and obtain the horizontal direction G x and vertical direction G y The first-order derivative of finds the edge gradient and direction of each pixel; Step 3.5.2: Normalize the gradient amplitude to the range of 0 to 255 for image display; Step 3.5.3: Map the gradient direction to the hue H component of the HSV color space, where the hue range is 0 to 360 degrees or 0 to 255; Step 3.5.4: Map the normalized gradient direction value to the hue H component of the HSV color space; Step 3.5.5: Set saturation and brightness; Step 3.5.6: Convert the color value in HSV color space to RGB color space: First, normalize the HSV values, that is, convert the values ​​of H, S, and V into values ​​between 0 and 1, and then calculate the intermediate variables: Color purity: C = V × SC = V × S The middle value of R, G, B: X = C × (1-abs ((H / 60)) X = C × (1-abs ((H / 60) Offset of R, G, B: m = V - Cm = VC According to the value of H, determine which interval it is in and calculate R, G, B: When 0≤H<60 degrees: R=CG=XB=0 When 60≤H<120 degrees: R=XG=CB=0 When 120≤H<180 degrees: R=0G=CB=X When 180≤H<240 degrees: R=0G=XB=C When 240≤H<300 degrees: R=XG=0B=C When 300≤H<360 degrees: R=CG=0B=X Finally, add the calculated R, G, B values ​​to the offset m: R′=R+m, G′=G+m, B′=B+m to ensure that the RGB values ​​are within the valid range of 0 to 1.

4. The intelligent monitoring method for shield tunneling soil discharge based on image acquisition and laser scanning equipment according to claim 1 is characterized in that The specific steps of step 3.6 are as follows: Step 3.6.1: Use the image processing library to read the image to be processed; Step 3.6.2: Determine the RGB threshold of the color you want to extract; Step 3.6.3: Create a mask based on the defined color range using cv2.inRange() function. This mask will be used to extract the region of a specific color from the image. Step 3.6.4: Apply the mask to the original image to extract the color regions that meet the threshold. Step 3.6.5: Display or save the extracted color area, and gradually adjust and optimize these parameters by observing the extraction effect.

5. The intelligent monitoring method for shield tunneling soil discharge based on image acquisition and laser scanning equipment according to claim 1 is characterized in that The specific steps of step 4 are as follows: Step 4.1: Collect as many images of the discharged soil as possible. The images need to take into account different working conditions, different lighting conditions, different geological environment conditions, and different construction methods and environments; Step 4.2: Label the collected images, classify and mark the collected slag images into under-discharge, normal, slightly over-discharge, and severely over-discharge, and store them in files; Step 4.3: Use data augmentation techniques to generate additional training samples and expand the training set; Step 4.4: Normalize and standardize the data to make the input data consistent; Step 4.5: Construct training set, validation set and test set; Step 4.6: Establish an improved YOLOv5 model. The YOLOv5 model includes a Backbone network, a Neck network, and a Head network. The Backbone network of YOLOv5 incorporates the attention mechanism module SE of the convolution module, and the attention mechanism module of the convolution module is inserted between the last C3 module and the SPPF module in the Backbone of YOLOv5 to obtain the improved YOLOv5 model. Step 4.7: Set parameters for model training: nc=1 in the YOLOv5s.yaml file; In the training code train.py, set the initial weight, that is, set the parameter weights, set the training model file and data set parameter file, that is, set the parameters cfg and data, set the number of iterations, that is, set the parameter epochs, set the number of batch processing files, that is, set the parameter batch-size, set the image size, that is, set the parameter imgsz, set GPU acceleration, that is, set the parameter device, set the multi-threading setting, that is, set the parameter workers; save the generated weight files last.pt and best.pt, where last.pt represents the weight obtained in the last round of training, and best.py represents the weight with the best effect during training; Step 4.8: Hyperparameter optimization for model training: Step 4.8.1: Use the Bayesian optimization algorithm and the Adam algorithm to optimize the improved YOLOv5 model established in step 4.6; Step 4.8.2: Use mean absolute error, root mean square error and coefficient of determination R2 to evaluate the performance of the model and determine whether the hyperparameters have converged; Step 4.9: Input the divided training set into the improved YOLOv5 model, evaluate the accuracy of the model, continuously optimize the hyperparameters, save the model structure when the prediction effect is optimal, use the optimal prediction model to predict the emission of slag, use the test set parameters to test the trained model to determine whether the model meets the expectations. When the model meets the training requirements, it is used to automatically identify different slag states: under-emission, normal, slightly over-emission and severely over-emission.

6. The intelligent monitoring method for shield tunneling discharge based on image acquisition and laser scanning equipment according to claim 5 is characterized in that The specific steps of step 4.2 are as follows: Step 4.2.1: According to the project requirements and the actual construction situation, the soil status is divided into multiple categories, namely under-discharge status, normal status, slight over-discharge status and severe over-discharge status; Step 4.2.2: Import the collected images into the annotation tool and use Labelmg software to create classification labels; Step 4.2.3: After data annotation, save the labeled .txt file and the corresponding source image in the YOLOv5 dataset format.

7. The intelligent monitoring method for shield tunneling soil discharge based on image acquisition and laser scanning equipment according to claim 5 is characterized in that The specific steps of step 4.3 are as follows: Step 4.3.1: Perform geometric transformation on the original image, including flipping, rotating, cropping, scaling, and translating; Step 4.3.2: Perform pixel transformation on the image, including adjusting the brightness, contrast, and saturation of the image to simulate different lighting conditions, adding Gaussian noise and salt and pepper noise to the image to improve the model's robustness to noise, and using Gaussian blur and motion blur to simulate poor image quality; Step 4.3.3: Randomly crop an area in the image and fill it with 0 so that the model learns not to rely on local features or mix the two images in proportion to generate new training samples and mix the labels of the two images.

8. The intelligent monitoring method for shield tunneling earth displacement based on image acquisition and laser scanning equipment according to claim 5 is characterized in that The specific steps of step 4.6 are as follows: Step 4.6.1: Based on the high precision and efficiency of the network EfficientNet, replace the feature extraction network of YOLOv5 to extract the features of the muck image: first, perform a 1×1 convolution on the input feature map, perform a dimensionality increase operation, and standardize it; after adding the activation function, perform a 3×3 or 5×5 depth-separable convolution, and introduce the SE attention mechanism module; then connect a Conv1×1 ordinary convolution to reduce the dimension, perform a Dropout operation, and complete the feature map fusion; fuse EfficientNet as the feature extraction network with YOLOv5, and add Ef in the `models / common.py` file of YOLOv5 EfficientNet's network structure code, including the definition of EfficientNet's `stem` class and `MBConvBlock` class. In the `models / yolo.py` file, add the EfficientNet class name to the YOLOv5 network structure. Copy the `yolov5s.yaml` file in the `models` folder and rename it. Then modify the configuration file according to the EfficientNetv2 network structure to ensure that the size of the feature map is transformed correctly. Finally, modify the `--cfg` default parameter in the `train.py` file to specify the new model structure configuration file during training. Step 4.6.2: Introduce the CBAM attention mechanism, combine the CBAM convolutional block attention module with the channel and spatial attention modules to enhance the model's ability to express shallow target feature information. Given an intermediate feature map F as input, first perform maximum pooling and average pooling operations on it, merge the results of the two pooling layers into the fully connected layer for calculation, and finally add them together to generate the attention MC on the channel dimension. Multiply MC with the input feature map to obtain the feature map F′ after the channel dimension is adjusted. Then perform maximum pooling and average pooling operations on F′ in space, concatenate the obtained two-dimensional vectors, and finally convolve to generate the spatial attention MS. Step 4.6.3: In the downsampling process of the YOLOv5 network layer, an improved image attention mechanism based on the LSTM model is added. In the LSTM model, in addition to the traditional input layer, output layer and hidden layer, a memory unit and three gating units are added, where i t is the input gate, f t is the forget gate, g t For input supply, o t is the output gate; The forget gate is used to control how much past information will be selected and forgotten, which is determined by the output h of the previous cell. t-1 and the current input x t Make a Sigmoid nonlinear mapping to get a vector f with each dimension value between 0 and 1 t , the formula is as follows: f t =σ(W f [h t-1 ,x t ]+b f ) Where: f t is the output of the forget gate, σ is the Sigmoid activation function, W f is the weight matrix of the forget gate, h t-1 is the hidden state of the previous time step, x t is the input of the current time step, b f is the bias vector of the forget gate; The input gate is used to control the storage ratio of the current input information, which is determined by the output h of the previous cell. t-1 and the current input x t A Sigmoid nonlinear mapping function layer is used to determine which part of the cell state value to update, and then a tanh function layer is used to generate new candidate information a. t , the formula is as follows: i t =σ(W f ·[h t-1 ,x t ]+b f ) a t =σ(W a ·[h t-1 ,x t ]+b a ) The update formula of the cell state is as follows: C t =f t ☉C t-1 +i t ☉a t Among them, the old state of the cell C t-1 and f t Multiply and discard the information to be forgotten, and add the main i t ☉a t As new information is added to the cell state, the cell state is updated; The function of the output gate is to decide how much current information to output. The output ht-1 of the previous cell and the current input xt are used to determine which parts of the cell state to output through a Signoid function layer. The cell state Ct is then processed through a tanh function layer and multiplied by the output of the Sigmoid layer. Finally, the part that is determined to be output is output. The formula is as follows: ot=σ(Wo·[ht-1,xt]+bo) ht=ot☉tanh(Ct).

9. The intelligent monitoring method for shield tunneling soil discharge based on image acquisition and laser scanning equipment according to claim 1 is characterized in that The specific steps of step 5 are as follows: Step 5.1: Set the unit time length, collect a specified number of photos within a certain period of time, use the trained YOLOv5 model to classify the collected images, and calculate the proportion of each category of slag in the photos, set the threshold, and if the probability of "severe over-discharge" exceeds the set threshold, determine that the current slag status is severely over-discharged; Step 5.2: Measure the actual volume of the slag discharge. Use laser scanning equipment to obtain the point cloud data of the slag profile corresponding to each frame of the image to obtain the slag profile map. Calculate the average accumulation height of the slag area. Use the YOLOv5 model to extract the edge of the slag plane image and multiply the slag plane area obtained by the average accumulation height to obtain the discharge volume of the slag corresponding to each frame of the image.

10. The intelligent monitoring method for shield tunneling earth discharge based on image acquisition and laser scanning equipment according to claim 9 is characterized in that In step 5.2, the volume of soil discharge per unit time is: Where n is the total number of frames per unit time.

Images