Shield muck improvement state evaluation method and system based on machine vision

Abstract

The invention belongs to the technical field of shield construction, and provides a shield muck improvement state evaluation method and system based on machine vision, during enhancement processing, for a low-illumination image data set, a contrast-limited adaptive histogram equalization method is adopted to adjust the overall brightness, and by introducing contrast limitation, excessive enhancement can be prevented; a color transformation method is adopted for other image data sets, detail description is enhanced through hue, saturation and brightness, and the color appearance of the image can be controlled more accurately; meanwhile, in the aspect of enhancing fine volume muck recognition, a multi-branch receptive field module is introduced, a Basic RFB layer is constructed, the receptive field of the model is expanded, and the recognition capability of the model is improved. In addition, by introducing a C3CA attention mechanism, PRNU noise fingerprints generated when a camera generates a video are suppressed, and the feature extraction capability is further enhanced, so that the model is more focused on global important information, and the muck improvement state can be fully reflected in subtle links and global aspects.

Description

Technical Field

[0001] This invention belongs to the field of shield tunneling technology, and in particular relates to a method and system for assessing the improved state of shield tunnel excavated soil based on machine vision. Background Technology

[0002] During tunnel boring machine (TBM) construction, the complex and variable engineering geological and hydrological conditions often make it difficult to achieve optimal results in on-site spoil modification. Typical problems frequently arise during tunneling, such as cutterhead "mud cake" formation, cutter wear, screw conveyor gushing, and surface subsidence. Therefore, it is crucial to scientifically measure and rapidly and accurately evaluate the physical and mechanical properties of modified spoil in order to adjust spoil modification parameters in real time and ensure the safety and efficiency of TBM tunneling.

[0003] Currently, machine vision-based methods for improving tunnel boring machine (TBM) excavated soil utilize image information and network models such as YOLOv8 to achieve real-time classification and detection. However, in tunnel construction inspection scenarios, due to the special environment, lighting conditions are often insufficient. Despite additional lighting measures, the acquired images still suffer from recognition difficulties. Images acquired under low-light conditions have relatively low clarity and contrast, making the boundaries of excavated soil in the image blurred, and the position and shape of targets difficult to accurately segment and locate, thus greatly reducing the accuracy of excavated soil detection. Under normal lighting conditions, low color contrast often affects the detection efficiency of image targets. If traditional feature enhancement is used directly, over-enhancement may occur, deviating from the overall appearance of the excavated soil and affecting the detection results. Furthermore, traditional network models such as YOLOv8 cannot effectively identify small volumes of excavated soil and lack detection of globally important information about the excavated soil, resulting in detection results that cannot fully reflect the excavated soil improvement status in both subtle and global aspects. Summary of the Invention

[0004] To address the aforementioned problems, this invention proposes a machine vision-based method and system for assessing the improved state of tunnel boring machine (TBM) slag. For low-light image datasets, this invention introduces contrast limiting to prevent over-enhancement. For other image datasets, it enhances detail description through hue, saturation, and brightness, allowing for more precise control over image color appearance. Furthermore, to enhance the identification of fine-volume slag, a multi-branch receptive field module is introduced, constructing a BasicRFB layer to expand the model's receptive field and improve its recognition capabilities. In addition, by introducing the C3_CA attention mechanism, PRNU noise fingerprints generated by the camera during video generation are suppressed, and feature extraction capabilities are further enhanced, enabling the model to focus more on globally important information and fully reflect the slag improvement state in both minute and global aspects.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solution: In a first aspect, the present invention provides a machine vision-based method for assessing the improved state of tunnel boring machine excavated soil, comprising: Obtain video data from the slag outlet; Keyframes are extracted from the video data to obtain image information; the image information is then cropped, restored, and enhanced; during enhancement, for low-light image datasets, a contrast-limited adaptive histogram equalization method is used to adjust the overall brightness; for other image datasets, a color transformation method is used to enhance detail description through hue, saturation, and brightness. Based on the processed image information and the pre-trained YOLOv8 network model, the soil and waste detection results are obtained. Specifically, in the YOLOv8 network model, deformable convolution is introduced into the C2f module to replace the standard convolutional network, a dynamic snake-shaped convolutional network is introduced in the Head layer, and a DySnakeConv layer is added. A multi-branch receptive field module is introduced to construct the BasicRFB layer, expanding the model's receptive field and enhancing the recognition of fine-volume soil and waste. A C3_CA attention mechanism is introduced to suppress noisy fingerprints generated during video generation and focus on global information. Based on the test results, the average flow velocity index is obtained; the improvement status is determined based on the average flow velocity index.

[0006] Furthermore, a three-frame difference method is used to extract keyframes. The inter-frame difference between every three frames is determined sequentially based on the video data, thereby obtaining the average inter-frame difference intensity. The frame with the maximum average inter-frame difference intensity is selected as the keyframe of the video.

[0007] Furthermore, image cropping includes: selecting regions with large numerical differences based on the mean square error of adjacent pixels, and obtaining the image range to be cropped using array slicing; obtaining the image cropping dataset, and expanding the dataset by rotating some images.

[0008] Furthermore, the contrast-limited adaptive histogram equalization method introduces a contrast limit while performing histogram equalization on each region. Specifically, it includes: dividing the original image into several small blocks; performing histogram equalization on the pixel values ​​within each small block, calculating the gray-level histogram, and generating a cumulative distribution function; setting a threshold to introduce a contrast limit when calculating the cumulative distribution function, and reducing the cumulative distribution function value of a certain gray level when it exceeds the threshold to limit excessive contrast enhancement; interpolating the small blocks after contrast-limited histogram equalization; and stitching the interpolated results of each small block to generate the final enhanced image.

[0009] Furthermore, the color transformation enhancement method controls the color appearance of an image by individually adjusting the hue, saturation, and brightness components in the color space during image processing. Specifically, it includes: converting the image from the original RGB color space to the HSV color space; adjusting the S and V components in the HSV color space to enhance the color saturation and brightness of the image; and converting the adjusted HSV color space image back to the RGB color space.

[0010] Furthermore, deformable convolution replaces the convolution in the C2f module to extract shallow features of the image; through the deformable convolutional network, DCN automatically adapts and adjusts according to the shape and proportion of the soil object to be tested. By learning the offset of the receptive field center position, the DCN network makes the position of the grid sampling move relative to the input feature map, thereby overcoming the inherent finite geometric transformations in CNN.

[0011] Furthermore, a DySnakeConv layer is added after each convolutional layer in the YOLOv8 Head network to enhance the model's ability to perceive the morphology of construction waste and to enhance the network's ability to express target features by expanding the receptive field. In the attention mechanism, the C3 module is responsible for the channel interaction of features, and the CA attention mechanism is responsible for focusing on the spatial location of the feature map.

[0012] Furthermore, the average flow velocity index for: ; in, The quantity of soil in the dry type at the slag outlet; s represents the quantity of slightly damp soil at the slag outlet; s represents the calculation duration; t represents the time.

[0013] Secondly, the present invention also provides a machine vision-based shield tunneling spoil improvement status assessment system, comprising: The data acquisition module is configured to acquire video data from the slag discharge port. The data processing module is configured to: extract keyframes from the video data to obtain image information; perform cropping, restoration, and enhancement processing on the image information; wherein, during enhancement processing, for low-light image datasets, a contrast-limited adaptive histogram equalization method is used to adjust the overall brightness; for other image datasets, a color transformation method is used to enhance detail description through hue, saturation, and brightness. The detection module is configured to: obtain the soil and waste detection result based on the processed image information and the pre-trained YOLOv8 network model; wherein, in the YOLOv8 network model, deformable convolution is introduced into the C2f module to replace the standard convolutional network, a dynamic snake convolutional network is introduced into the Head layer, and a DySnakeConv layer is added; a multi-branch receptive field module is introduced to construct the BasicRFB layer to expand the receptive field of the model and enhance the recognition of fine-volume soil and waste; and a C3_CA attention mechanism is introduced to suppress the noise fingerprint generated when generating the video and focus on global information; The evaluation module is configured to: obtain the average flow velocity index based on the detection results; and determine the improvement status based on the average flow velocity index.

[0014] Thirdly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the machine vision-based shield tunneling spoil improvement state assessment method described in the first aspect.

[0015] Fourthly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the program to implement the steps of the machine vision-based shield tunneling spoil improvement state assessment method described in the first aspect.

[0016] Fifthly, the present invention also provides a computer program product, the computer program product comprising a computer program, which, when executed by a processor, implements the steps of the machine vision-based shield tunneling slag improvement state assessment method described in the first aspect.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: In this invention, for low-light image datasets, a contrast-limited adaptive histogram equalization method is used to adjust the overall brightness. By introducing contrast limitations, over-enhancement can be prevented. For other image datasets, a color transformation method is used to enhance detail description through hue, saturation, and brightness, allowing for more precise control over the color appearance of the image. Simultaneously, to enhance the recognition of fine-volume construction waste, a multi-branch receptive field module is introduced, constructing a BasicRFB layer to expand the model's receptive field and improve its recognition ability. Furthermore, by introducing the C3_CA attention mechanism, PRNU noise fingerprints generated by the camera during video generation are suppressed, and feature extraction capabilities are further enhanced, enabling the model to focus more on globally important information and fully reflect the state of construction waste improvement in both subtle and global aspects. Attached Figure Description

[0018] The accompanying drawings, which form part of this embodiment, are used to provide a further understanding of this embodiment. The illustrative embodiments and their descriptions are used to explain this embodiment and do not constitute an improper limitation of this embodiment.

[0019] Figure 1 This is a flowchart of the method in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the three-frame difference method in Embodiment 1 of the present invention; Figure 3 This is a diagram of the original YOLOv8 network architecture in Embodiment 1 of the present invention; Figure 4 This is a diagram of the improved YOLOv8 network architecture of Embodiment 1 of the present invention. Detailed Implementation

[0020] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0021] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0022] Example 1: The application of shield tunneling in urban tunnel and underground engineering construction is becoming increasingly widespread, especially in urban subway projects. Earth pressure balance shield tunneling is widely used due to its advantages such as high construction efficiency and minimal impact on the surrounding environment. During earth pressure balance shield tunneling, the geological strata traversed by the shield machine are complex and variable. Construction personnel need to use soil improvement methods to maintain the soil in a plastic flow state to achieve stable soil inflow and outflow and pressure maintenance within the shield machine's soil chamber.

[0023] The mechanical properties of tunnel boring machine (TBM) excavation soil have a significant impact on the safety and efficiency of TBM tunneling. The properties of the excavation soil directly affect the smoothness of muck removal and the establishment of soil chamber pressure. To improve the safety and construction efficiency of the TBM, the excavation soil needs to be modified to possess suitable plasticity, low permeability, low shear strength, suitable compressibility, and low cohesive strength. The quality of excavation soil modification can be evaluated through indoor tests such as slump tests, flowability tests, consistency tests, and mixing tests. However, due to variations in geological formations and the efficiency of testing, the method of guiding the adjustment of excavation soil modification parameters through indoor experiments has a certain lag. Therefore, improving the efficiency and accuracy of excavation soil plasticity testing is crucial for the stable control of TBM parameters. Currently, in actual projects, construction personnel generally evaluate the morphology of the excavation soil at the muck opening through manual on-site observation or monitoring, and combine this with TBM tunneling parameters to comprehensively assess the current state of excavation soil modification. This requires dedicated personnel, is time-consuming and labor-intensive, and is greatly affected by the worker's skill level and subjective work attitude. Failure to provide timely feedback on the excavation soil condition will lead to safety and quality problems in TBM construction.

[0024] For earth pressure balance (EPB) tunnel boring machines (TBMs), accurate and timely assessment of the modified soil state is crucial for safe and stable construction. However, experimental research on modified soil inevitably faces certain challenges. On one hand, due to changes in geological strata, indoor testing of modified soil lags behind on-site construction guidance, making it difficult to synchronize adjustments of modification parameters with changes in the preceding strata. On the other hand, influenced by factors such as soil particle size and tunneling methods, there is no unified standard for the ideal fluid plasticity definition of modified soil. Most studies obtain a relatively optimal definition by examining the morphological responses of soil in different strata under various tests. Therefore, to accurately assess the modified soil state, reduce judgment errors caused by subjective human factors, and eliminate the adverse effects of prolonged harsh environments on the operator's physical and mental well-being, research is needed on the identification of modified soil fluid plasticity and the comprehensive assessment of the modified state in traditional EPB TBMs, developing effective methods for evaluating the modified state. Developing intelligent soil fluid plasticity detection and state assessment technologies is of great significance for adapting to changes in geological strata and synchronizing adjustments to TBM parameters during tunneling.

[0025] As described in the background section, in tunnel construction inspection scenarios, due to the special environment, lighting conditions are often insufficient. Even with additional lighting, the acquired images still suffer from recognition difficulties. Images acquired under low-light conditions have relatively low clarity and contrast, making the boundaries of the excavated soil blurry and the location and shape of targets difficult to accurately segment and locate, thus significantly reducing the accuracy of excavated soil detection. Under normal lighting conditions, low color contrast often affects the detection efficiency of image targets. Directly using traditional feature enhancement methods can lead to over-enhancement, deviating from the overall appearance of the excavated soil and affecting the detection results. Furthermore, traditional network models such as YOLOv8 cannot effectively identify small volumes of excavated soil and lack the ability to detect globally important information about the excavated soil, resulting in detection results that cannot fully reflect the state of excavated soil improvement in both minute and global aspects.

[0026] To address at least one of the aforementioned problems, this embodiment provides a machine vision-based method for assessing the modified state of tunnel boring machine (TBM) excavated soil. Based on an improved YOLOv8 network, it achieves morphological detection and modified state assessment of the excavated soil. First, images of the modified excavated soil at the TBM's discharge port are captured using monitoring cameras. The video image data undergoes preprocessing methods such as keyframe extraction, cropping, restoration, and enhancement to strengthen data features and construct a morphological dataset. Second, the dataset is labeled, and the YOLOv8 network structure is improved to train a target detection model. Finally, a modified soil state assessment model based on slump tests is constructed to predict and classify the modified soil state levels. The specific process of the method is as follows: Figure 1 As shown, the specific steps of the method are as follows: S1, Construction of soil morphology dataset.

[0027] A specific engineering project under research was selected, and video data was collected through equipment deployed at the slag discharge port of the left auger of the tunnel boring machine. The dataset was constructed by extracting key frames from the video, preprocessing the image sequence, and labeling the slag image dataset.

[0028] S1.1, Video Keyframe Extraction: Each video is an image sequence, highly expressive and rich in information. Video analysis is usually based on video frames, but video frames often contain a lot of redundancy, and frame extraction also suffers from missing frames and redundancy. This embodiment uses a shot segmentation method for the video sequence, then extracts keyframes from the shots, and uses these keyframes to obtain underlying features such as shape, texture, and color. To ensure a sufficient dataset of slag and waste soil images, video sequences from the first 100 rings of the project's slag discharge outlets each day were selected. Combined with the interval geological survey data, the inter-frame difference method was used to extract keyframes from the videos. If the intensity change of the geological interface is not significant at a certain moment, only one image from the keyframes in the slow-moving interval is analyzed; if the change is drastic at a certain moment, all extracted keyframes within that time period are considered.

[0029] Frame difference analysis is primarily used to detect motion in video recording or shooting. This method identifies motion by comparing pixel differences between two or more consecutive frames and identifies the frame with the largest difference. Considering that the speed of the dump conveyor belt changes due to variations in tunnel boring machine parameters, when the movement is fast, the target's position differs significantly between adjacent frames, meaning that subtracting two frames may not yield a complete picture of the moving target. Therefore, a three-frame difference method is chosen to extract keyframes. For example... Figure 2 As shown, the video is first read, and the inter-frame difference between every three frames is calculated sequentially to obtain the average inter-frame difference intensity. Then, the frame with the local maximum of the average inter-frame difference intensity is selected as the keyframe of the video.

[0030] S1.2 Image sequence preprocessing: This embodiment categorizes the improved slag soil into three types: slightly dry soil, suitable soil, and slightly moist soil. After keyframe extraction, the selected images undergo preprocessing such as cropping, restoration, and enhancement. On one hand, this enhances image features, facilitating manual annotation and network training. On the other hand, it expands the data sample, enhancing the model's robustness.

[0031] Among them, the humidity of slightly dry soil is less than that of suitable soil, and the humidity of suitable soil is less than that of slightly wet soil. The specific judgment boundary can be determined according to target identification, experimental methods, etc., and can also be adjusted according to the actual situation. For example, the humidity range of suitable soil is set to 80%~85%.

[0032] S1.2.1 Image cropping and rotation: Since the soil on the conveyor belt occupies a small proportion of the image, to reduce the amount of data during network computation, minimize interference from non-essential content, and improve network processing efficiency, image cropping is used to retain the main soil portion. This embodiment first uses the `imread` method to read the image to be cropped. Then, based on the mean square error of adjacent pixels, regions with significant numerical differences (e.g., differences greater than a preset value) are selected. The image range to be cropped is obtained using array slicing. An image cropping dataset is acquired, and some images are rotated at angles of 90°, 160°, and 180° to expand the dataset.

[0033] S1.2.2 Image Restoration: During the process of a camera capturing images, it is inevitable that the images will be affected by various noises and motion blur, resulting in images that are noisy, blurry, have low contrast, and lack clear details and textures, thus greatly reducing image quality. Therefore, this embodiment uses the Wiener filter algorithm to suppress noise points in moving images, improve image quality, and preserve image detail information.

[0034] Wiener filtering is an algorithm for processing images in the frequency domain. It is a classic image enhancement algorithm that can not only reduce image noise but also eliminate image blur caused by motion. This method is based on the minimum mean square error criterion to find an estimate of the uncontaminated image, thereby minimizing the mean square error between the original image and the restored image, as shown in formula (1), where E is the expected value of the parameter.

[0035] (1) Assuming the noise is independent of the image, one of the original or restored images has zero mean, and the estimated gray level is a linear function of the gray level in the degraded image, the minimum value of the mean square error function in the frequency domain is expressed by the following expression.

[0036] (2) in, This represents MSE (mean squared error). and These represent the original image and the restored image, respectively. It is a degenerate function; The Fourier transform of the degraded image; Noise power spectrum; The power spectrum of the undegraded image; This is for frequency domain estimation.

[0037] S1.2.3 Feature Enhancement: In tunnel construction inspection scenarios, lighting conditions are often insufficient due to the unique environment. Despite additional lighting measures, the acquired images still suffer from difficulty in identification. Images acquired under low-light conditions have relatively low clarity and contrast, making object boundaries blurred and the location and shape of targets difficult to accurately segment and locate, thus significantly reducing the performance of target detection algorithms. Under normal lighting conditions, low color contrast often affects the detection efficiency of image targets. Therefore, this embodiment innovatively proposes using the Contrast Limited Adaptive Histogram Equalization (CLAHE) algorithm to adjust the overall brightness for low-light image datasets (such as image datasets with light intensity below a preset intensity or image datasets with light intensity below the indoor standard intensity); and using the HSV transform algorithm for other image datasets to enhance detail description through hue, saturation, and brightness.

[0038] The contrast-limited adaptive histogram equalization algorithm introduces a contrast limit while performing histogram equalization on each region to prevent over-enhancement. The implementation principle of this algorithm is as follows: Image division into smaller blocks: The original image is divided into several smaller blocks, which are usually overlapping to ensure the continuity of the image.

[0039] Histogram equalization is performed on the pixel values ​​within each small block, grayscale histograms are calculated, and cumulative distribution function (CDF) is generated.

[0040] Contrast limiting: A threshold is set when calculating the cumulative distribution function to introduce contrast limiting. When the cumulative distribution function value of a certain gray level exceeds the threshold, that portion is cut off to limit excessive contrast enhancement. This can prevent the introduction of too much noise or make the image look unnatural in some cases.

[0041] Interpolation: Interpolation is performed on the small blocks after contrast-limited histogram equalization. The purpose of interpolation is to eliminate gray-level discontinuities between adjacent blocks, ensuring a smooth transition in the final CLAHE-enhanced image.

[0042] Generating the final enhanced image: The interpolated patches are stitched together to generate the final CLAHE-enhanced image. This image has a better balance in contrast compared to the original image, while maintaining an overall natural appearance.

[0043] HSV color transformation enhancement refers to adjusting the hue, saturation, and value of an image in the HSV color space to enhance its visual effect. In image processing, the HSV color space is commonly used for color space transformation and enhancement. By adjusting the hue, saturation, and value components individually in the HSV color space, the color appearance of an image can be controlled more precisely. The principle of HSV color transformation enhancement involves the following steps: Color space conversion: Converting an image from the original RGB color space to the HSV color space.

[0044] Component Adjustment: Adjusts the S and V components in the HSV color space to enhance the color saturation and brightness of the image.

[0045] Reconversion: Convert the adjusted HSV color space image back to RGB color space for display or further processing.

[0046] S1.3, Labeling of the slag and waste soil image dataset: This technical solution uses LabelImg as the annotation tool to accurately annotate a dataset of excavated soil images from tunnel construction scenarios. Three types of excavated soil targets—dry soil, wet soil, and suitable soil—are labeled separately. LabelImg provides an intuitive and easy-to-use interface, facilitating the annotation of image datasets. Furthermore, LabelImg supports classification labels for each bounding box, allowing users to specify the target's category information. These tools are particularly useful when processing large-scale image datasets, improving the efficiency and accuracy of annotation.

[0047] S2. Construction of an improved YOLOv8-based model for detecting the morphology of construction waste.

[0048] We introduce a fusion method of YOLOv8 and snake convolution, and use the multi-branch convolution module RFB to enhance the receptive field of the model and improve the feature extraction capability. We also add the CBAM self-attention mechanism to improve the attention to the target region and the localization accuracy.

[0049] S2.1, YOLOv8 original model: The YOLOv8 model network architecture mainly consists of four parts: Input, Backbone, Neck, and Head. Building upon the excellent features of previous versions, it adds new improvements and implements a new function: instance segmentation of the target object. Its network architecture is as follows: Figure 2 As shown.

[0050] Backbone Layer: YOLOv8 replaces the C3 module with the C2f module in the backbone network. The C2f module connects the originally concatenated Bottleneck module through gradient splitting and introduces more parallel branches, thereby optimizing the module structure and effectively solving the problem of convergence degradation with increasing depth. Compared with the C3 module in YOLOv5, the C2f module establishes a smooth transition between convolutional layers and fully connected layers, realizing parameter sharing to improve the model's efficiency and generalization ability, enabling the model to achieve higher accuracy while maintaining more reasonable latency.

[0051] In the YOLO series, Spatial Pyramid Pooling (SPP) is used to capture feature maps of multi-scale receptive fields, helping the model acquire richer information. The SPPF module, short for Spatial Pyramid Pooling-Fast, is an improvement and optimization of the SPP module. This module contains one CBS convolutional layer and three max-pooling layers. Its main advantage is its ability to handle input images of arbitrary sizes while capturing feature information at multiple scales, thus improving the performance of the network model.

[0052] Neck Layer: Located between the feature extraction network and the final output layer, the Neck layer primarily processes and enhances the features extracted from the backbone to improve network performance and generalization ability. YOLOv8's Neck layer still uses an FPN+PAN structure to construct the feature pyramid. FPN achieves multi-scale fusion feature extraction by adding feature layers of different scales to the backbone. Its core idea is to construct a feature pyramid through cross-layer connections combined with the top-level feature pyramid, thus fully preserving the semantic information of high-level features without losing the spatial information of low-level features. To address the problems of FPN and realize the process of transferring low-level information to high-level features, YOLOv8 adds a PAN (Path Aggregation Network) module to the feature fusion network. By introducing a downsampling process combined with FPN, upsampling and downsampling operations are applied to feature maps at different levels, thereby achieving multi-scale feature fusion.

[0053] Head Layer: The main function of the Head layer is to transform the features extracted from the backbone network into the final prediction result. It represents the most significant structural change in YOLOv8, replacing the original coupled head with a decoupled head, separating the classification and detection heads and using different branches for computation, thus improving detection performance. It also changes from anchor-based to anchor-free, reducing computational complexity. Furthermore, YOLOv8 improves the design of its loss function, employing an improved cross-entropy loss for classification and DFLLoss and CIOU Loss for regression. The DFL Loss introduces asymmetric weighting, making it more flexible than the traditional FocalLoss (FL) and QualityFocalLoss (QFL).

[0054] S2.2, YOLOv8 Improved Model: First, deformable convolutions are introduced into the C2f module to replace the standard convolutional network. Simultaneously, a dynamic snake-like convolutional network is introduced into the Head layer, adding a DySnakeConv layer to further improve the feature fitting ability. Second, to enhance the recognition of fine-volume soil debris, a multi-branch receptive field module is introduced, constructing a BasicRFB layer to expand the model's receptive field and improve its recognition capability. Furthermore, by introducing the C3_CA attention mechanism, PRNU noise fingerprints generated by the camera during video generation are suppressed, and feature extraction capabilities are further enhanced, allowing the model to focus more on globally important information. The improved model structure is as follows: Figure 3 As shown.

[0055] This embodiment explores two aspects: deformable convolutional networks and dynamic serpentine convolutional networks, aiming to expand the model's deformability. Specifically, on the one hand, to reduce useless contextual information in backbone feature extraction caused by poor distinguishability between detected soil targets and the background, deformable convolution is used to replace the conventional convolution of standard CBS in the C2f module, thereby more effectively extracting shallow features of the image. On the other hand, the dynamic serpentine convolutional network is studied in depth, especially its application in the field of segmentation.

[0056] S2.2.1 Deformable Convolutional Networks: Deformable convolutional networks (DCNs) aim to dynamically adjust the shape of convolutional kernels by learning the offset of convolutional sampling points in two-dimensional space. Compared to traditional fixed convolutional kernels, DCNs can automatically adapt and adjust according to the shape and proportion of the soil sampled object, exhibiting stronger adaptability. By learning the offset of the receptive field center position, the DCN network can make the grid sampling position move relative to the input feature map, thus overcoming the inherent finite geometric transformations in CNNs. This embodiment introduces a deformable convolutional module into YOLOv8, replacing the traditional convolution in YOLOv8's C2f module with DCNv2, designing a C2f-DCNv2 module. This operation introduces a more powerful nonlinear modeling capability into the model.

[0057] S2.2.2 Dynamic Serpentine Convolutional Network The core idea behind DSConv is to introduce deformable offsets to increase the flexibility of convolutional kernels in standard 2D convolutional operations. While the concept of incorporating deformable offsets is inspired by previous related work, DSConv makes significant and crucial improvements in this area. In traditional convolutional operations, keeping the receptive field unchanged can prevent the model from capturing certain detailed features; DSConv addresses this by allowing the model to learn deformable offsets. However, to avoid the receptive field deviating excessively from the target, DSConv employs an iterative strategy, ensuring that only one target is processed at a time, thus maintaining continuity of attention. Specifically, this embodiment adds a DySnakeConv layer after each convolutional layer in the YOLOv8 Head network to enhance the model's perception of slag morphology.

[0058] Furthermore, the rich information in low-level feature maps in object detection networks is often overlooked, leading to poor detection performance for small objects and affecting the accuracy of target localization. To address this issue, this embodiment introduces a receptive field block (RFB) module, which enhances the network's ability to express target features by expanding the receptive field. The RFB module is a multi-branch structure used to expand the model's receptive field and enhance feature representation. It uses the Inception structure as a reference network and employs three branches of different scales: 1×1, 3×3, and 5×5. Subsequently, it combines dilated convolutions with different dilation rates to extract features from convolutions at different scales. Finally, it uses a 1×1 convolutional layer to adjust the feature dimensions and fuse the features from the three multi-scale branches.

[0059] Finally, the C3_CA attention mechanism is introduced. In this attention mechanism, the C3 module is responsible for the channel interactions of features, while the CA attention mechanism focuses on the spatial location of the feature map, enabling the model to more accurately capture the target's location information. Integrating the CA attention mechanism with the C3 module combines their advantages, effectively improving the model's detection accuracy. The C3 module consists of multiple convolutional modules and a BottleNeck module, which can enhance the model's feature representation capabilities by adjusting the number of channels and depth of the feature map.

[0060] S3. Research on the assessment method of slag improvement state based on slump and average flow velocity experiments.

[0061] This embodiment combines slump with the average flow velocity index P of the slag soil at a certain moment to comprehensively and objectively evaluate the improvement status of coarse-grained soil.

[0062] S3.1, Indoor Test: Indoors, an improved slag plasticity detection model based on deep learning (i.e., the slag improvement state assessment model established using machine vision) was used to identify and calculate the average plasticity value. The formula for calculating the average flow velocity is as follows: (3) in, The quantity of "drier" soil at the slag outlet; The quantity of "wet" soil at the slag outlet; s is the calculation duration; t is the time, generally taken as 20-40 minutes. Based on the reasonable range of slump F value proposed by researchers, the relationship between the P value range and the soil improvement state is shown in Table 1.

[0063] Table 1 presents the relationship between the range of P-values ​​and the state of soil improvement.

[0064] S3.3 Outdoor Experiment: However, the P-value depends on the target detection results, and the detection efficiency and extent will increase the uncertainty of the P-value. Therefore, in earth pressure balance shield tunneling, the slump test is used to verify the final state of the modified soil. The slump test quantifies the fluidity and plasticity of concrete samples by measuring the degree of deformation during free fall. This test not only helps determine the optimal modification scheme but also provides a reliable basis for real-time monitoring during construction. Generally, soil with a slump value between 150 mm and 200 mm is considered to have good fluidity, suitable for the requirements of shield tunneling. The basic steps of the slump test are as follows: S3.1.1 Mixing of slag and soil amendment: First, thoroughly mix the excavated soil with the selected amendment (such as foaming agent, bentonite slurry, etc.). Common amendments include foaming agents, bentonite, polymers, etc.

[0065] S3.1.2, Filling slump cone: The mixed slag was filled into the standard slump cone in three layers, and each layer was tamped 25 times with a tamping rod to ensure that the slag was densely filled into the cone.

[0066] S3.1.3, Lift the slump cone: Within 5 to 10 seconds, quickly and smoothly lift the slump cone, ensuring the movement is vertical throughout the process. The total time for loading and lifting the cone should not exceed 150 seconds.

[0067] S3.1.4 Record slump values ​​and apparent characteristics: After the slump cone is lifted, the modified slag is allowed to collapse freely. Once the deformation has stabilized, the height of the collapsed material is measured and the slump value is recorded.

[0068] Each different geological stratum has a unique and reasonable slump range. The rationality of the P-value is verified based on the slump in millimeters to determine the final improvement state. If the improved excavated soil is in an over-improved state, the water injection and foam injection parameters of the tunnel boring machine are adjusted according to different geological types. If the improved excavated soil is in an under-improved state, the water injection or foam injection is increased until the excavated soil's fluidity and slump are ideal. If the improved excavated soil is in a suitable state, no adjustment of the excavated soil improvement parameters is required. Based on the current research status and actual construction conditions in the field of excavated soil improvement both domestically and internationally, this embodiment establishes a classification of excavated soil improvement state levels during earth pressure balance shield tunneling, as shown in Table 2.

[0069] Table 2 Classification of Slag and Soil Improvement Status During Earth Pressure Balance Shield Tunneling

[0070] In summary, this embodiment collects video data through equipment deployed at the muck outlet of the left auger of the tunnel boring machine. Dataset construction is achieved through video keyframe extraction, image sequence preprocessing, and muck image dataset annotation. An improved YOLOv8 algorithm model is established. Based on the original YOLOv8 network structure, deformable convolution is introduced into the C2f module to replace the standard convolutional network, and a dynamic snake-shaped convolutional network is introduced in the Head layer, along with a DySnakeConv layer to further enhance the feature fitting ability. Furthermore, to suppress the influence of image-generated fingerprint noise PRNU on feature extraction, a C3_CA attention mechanism is introduced, making the model more focused on globally important information. The dataset is divided into training and testing sets at a 7:3 ratio to train the optimal discrimination model for muck morphology classification, solving the problem of difficult muck morphology identification. An average flow velocity detection formula is constructed, and combined with outdoor slump experiments, the muck modification state is classified, addressing the difficulty in assessing the muck modification state.

[0071] Example 2: This embodiment provides a machine vision-based shield tunneling spoil improvement status assessment system, including: The data acquisition module is configured to acquire video data from the slag discharge port. The data processing module is configured to: extract keyframes from the video data to obtain image information; perform cropping, restoration, and enhancement processing on the image information; wherein, during enhancement processing, for low-light image datasets, a contrast-limited adaptive histogram equalization method is used to adjust the overall brightness; for other image datasets, a color transformation method is used to enhance detail description through hue, saturation, and brightness. The detection module is configured to: obtain the soil and waste detection result based on the processed image information and the pre-trained YOLOv8 network model; wherein, in the YOLOv8 network model, deformable convolution is introduced into the C2f module to replace the standard convolutional network, a dynamic snake convolutional network is introduced into the Head layer, and a DySnakeConv layer is added; a multi-branch receptive field module is introduced to construct the BasicRFB layer to expand the receptive field of the model and enhance the recognition of fine-volume soil and waste; and a C3_CA attention mechanism is introduced to suppress the noise fingerprint generated when generating the video and focus on global information; The evaluation module is configured to: obtain the average flow velocity index based on the detection results; and determine the improvement status based on the average flow velocity index.

[0072] The working method of the system is the same as the machine vision-based shield tunneling muck improvement status assessment method in Example 1, and will not be repeated here.

[0073] Example 3: This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the machine vision-based shield tunneling spoil improvement state assessment method described in Embodiment 1.

[0074] Example 4: This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the program, it implements the steps of the machine vision-based shield tunneling muck improvement state assessment method described in Embodiment 1.

[0075] Example 5: This embodiment provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the steps of the shield tunneling slag improvement state assessment method based on machine vision described in Embodiment 1.

[0076] The above description is merely a preferred embodiment of this practice and is not intended to limit the scope of this practice. Various modifications and variations can be made to this practice by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this practice should be included within the protection scope of this practice.

Claims

1. A method for evaluating the improvement state of a shield muck based on machine vision, characterized by, The method comprises the following steps: acquiring video data of a slag outlet; key frame extraction is performed on the video data to obtain image information; the image information is subjected to cropping, restoration and enhancement processing; when the enhancement processing is performed, for low-light image data sets, a limited contrast adaptive histogram equalization method is used to adjust the overall brightness; for other image data sets, a color transformation method is used to enhance the details through hue, saturation and brightness; a slag detection result is obtained according to the processed image information and a pre-trained YOLOv8 network model; in the YOLOv8 network model, deformable convolution is introduced into a C2f module to replace a standard convolution network, a dynamic snake convolution network is introduced into a Head layer, and a DySnakeConv layer is added; a multi-branch receptive field module is introduced to construct a BasicRFB layer, expand the receptive field of the model and strengthen the identification of fine volume slag; a C3_CA attention mechanism is introduced to suppress noise fingerprints generated when a video is generated and focus on global information; an average flow rate index is obtained according to the detection result; and a modified state is determined according to the average flow rate index.

2. The machine vision-based shield muck improvement state evaluation method according to claim 1, wherein, The key frame is extracted by using a three-frame difference method; first, the video data is determined, and the interframe difference between every three frames is determined in turn, and then the average interframe difference intensity is obtained; the frame with the maximum average interframe difference intensity is selected as the key frame of the video. 3.The machine vision-based evaluation method of a shield muck improvement state according to claim 1, wherein, The image cropping comprises the following steps: selecting a region with a large numerical difference according to the mean square error of adjacent pixel points, obtaining the image range to be cropped by using the array slicing method, and obtaining an image cropping data set; and the data set is expanded by setting the rotation angle of part of the images.

4. The machine vision-based shield muck improvement state evaluation method according to claim 1, wherein, The limited contrast adaptive histogram equalization method introduces a contrast limit while performing histogram equalization on each region, specifically including: dividing the original image into small blocks; performing histogram equalization on the pixel values in each small block, calculating the gray histogram and generating the cumulative distribution function; when calculating the cumulative distribution function, a threshold is set to introduce a contrast limit, and when the cumulative distribution function value of a certain gray level exceeds the threshold, it is reduced to limit the excessive enhancement of contrast; interpolating the small blocks after the contrast limited histogram equalization processing; and splicing the interpolated small block results to generate the final enhanced image. 5.The machine vision-based evaluation method of a shield muck improvement state according to claim 4, wherein, The color transformation enhancement method controls the color appearance of the image by separately adjusting the hue, saturation and brightness components in the color space during image processing, specifically including: converting the image from the original RGB color space to the HSV color space; adjusting the S and V components in the HSV color space to enhance the color saturation and brightness of the image; and converting the adjusted HSV color space image back to the RGB color space. 6.The machine vision-based evaluation method of a shield muck improvement state according to claim 1, wherein, The deformable convolution replaces the convolution in the C2f module to extract the shallow features of the image; through the deformable convolution network, the DCN automatically adapts and adjusts according to the shape and scale of the slag object to be detected, learns the offset of the center position of the receptive field, and makes the grid sampling position move relative to the input feature map, thereby overcoming the inherent limited geometric transformation in the CNN. 7.The machine vision-based evaluation method of a shield muck improvement state according to claim 1, wherein, A DySnakeConv layer is added after each convolutional layer in the Head network of YOLOv8 to enhance the model's perception of slag morphology and enhance the network's expression of target features by expanding the receptive field; in the attention mechanism, the C3 module is responsible for channel interaction of features, and the CA attention mechanism is responsible for focusing on the spatial position of the feature map. 8.The machine vision-based evaluation method of a shield muck improvement state according to claim 1, wherein, The average flow rate property is: ； wherein, is the number of dry type soil bodies for the taphole; is the number of wet type soil bodies for the taphole;s is the calculation duration;t is the time.

9. A machine vision-based shield muck improvement state evaluation system, characterized by, Comprise: A data acquisition module configured to obtain video data of a slag outlet; A data processing module configured to extract key frames from the video data to obtain image information, and to perform cropping, restoration and enhancement processing on the image information; wherein, when performing enhancement processing, for low-light image datasets, a limited contrast adaptive histogram equalization method is used to adjust the overall brightness; for other image datasets, a color transformation method is used to enhance the detail description through hue, saturation and brightness; A detection module configured to obtain slag detection results according to the processed image information and a pre-trained YOLOv8 network model; wherein, in the YOLOv8 network model, deformable convolution is introduced into the C2f module to replace the standard convolution network, a dynamic snake convolution network is introduced into the Head layer, and a DySnakeConv layer is added; a multi-branch receptive field module is introduced to construct a BasicRFB layer, expand the receptive field of the model, and strengthen the identification of fine volume slag; a C3_CA attention mechanism is introduced to suppress noise fingerprints generated when generating a video and focus on global information; An evaluation module configured to obtain an average flow rate index according to the detection results, and determine the improvement state according to the average flow rate index.

10. A computer program product, characterised in that, The computer program product comprises a computer program which, when executed by a processor, implements the steps of the machine vision-based shield slag improvement state evaluation method according to any one of claims 1-8. The computer program product comprises a computer program which, when executed by a processor, implements the steps of the machine vision-based shield slag improvement state evaluation method according to any one of claims 1-8.

Images