A content-aware based TBM rock debris lithology identification method and system

Abstract

The application provides a content-aware-based TBM rock debris lithology identification method and system, and belongs to the technical fields of tunnel TBM construction intelligent perception, computer vision and video intelligent analysis. The method comprises the following steps: inputting the image of a target rock debris area into a trained rock debris lithology identification network to obtain the lithology category and confidence of a real-time image; the trained rock debris lithology identification network; the training process comprises the following steps: collecting historical rock debris video image sequences and TBM tunneling parameters; synchronizing the historical rock debris video image sequences and the TBM tunneling parameters in time; pre-processing the synchronized data; dividing the rock debris data set into a training set, a verification set and a test set; training the rock debris lithology identification network by using the training set to obtain the trained rock debris lithology identification network. The method improves the identification effect of rock blocks with different particle sizes, improves the robustness of the model under distribution changes, and improves the reliability of engineering deployment.

Description

Technical Field

[0001] This invention belongs to the field of intelligent perception, computer vision and video intelligent analysis technology in tunnel TBM construction, specifically involving a content-aware TBM rock debris lithology identification method and system. Background Technology

[0002] With the increasing scale and complexity of deep-buried tunnel projects, geological disasters such as rock bursts and collapses occur frequently during TBM (Tunnel Boring Machine) excavation, seriously threatening personnel safety and equipment operation. The integrity of the surrounding rock is highly correlated with the risk of rock bursts, but due to the obstruction caused by the TBM's shield structure, construction personnel find it difficult to obtain timely information about the actual condition of the surrounding rock at the tunnel face. On-site, the rock debris transported by conveyor belts directly reflects the degree of rock fragmentation and geomechanical properties of the excavated area. Therefore, accurately sensing and identifying the characteristics of the rock debris on the conveyor belts and conducting real-time, automated lithological identification is an important scientific problem.

[0003] Existing research on rock debris lithology identification methods is relatively limited, with most popular methods based on static rock images or data constructed from laboratory conditions. However, these methods are primarily based on ideal environments, exhibiting limitations and failing to fully utilize the rich information in real-world field data. Furthermore, the high-speed movement of rock debris on conveyor belts leads to blurred image edges, and the varying sizes and non-uniform distribution of rock fragments easily cause occlusion and overlap. Added to this are interferences from complex backgrounds such as mud, water stains, dust, and strong reflections, making it difficult for existing techniques to effectively model and analyze the characteristics of rock debris under large amounts of complex and interference-laden data. This easily leads to over-reliance on shortcut features, such as background co-occurrence and local salient regions, resulting in insufficient generalization ability across working conditions and an inability to accurately identify rock debris lithology. Given the widespread application of artificial intelligence and computer vision, lithology identification methods for complex working conditions are still inadequate, necessitating the development of a robust and interpretable rock debris lithology identification method and system that can maintain robustness under complex disturbances and distribution variations. Summary of the Invention

[0004] To address the shortcomings of existing technologies, a content-aware TBM rock slag lithology identification method and system is proposed. This method aims to solve the problems of unstable lithology identification and insufficient generalization ability across working conditions caused by background interference such as motion fuzziness, scale inconsistencies, occlusion and overlap, conveyor belt texture, water stain reflection, and lighting changes under high-speed conveyor belt conditions. This will improve the robustness, interpretability, and on-site usability of the lithology identification results.

[0005] In a first aspect, the present invention provides a content-aware TBM slag lithology identification method, comprising: Real-time acquisition of video image sequences of rock slag on belt conveyors; The rock debris targets in the video image sequence of rock debris on the conveyor belt are located, and the images of the rock debris target areas corresponding to the target boxes are extracted. The image of the target area of ​​rock debris is input into the trained rock debris lithology identification network to obtain the lithology category and confidence level of the real-time image. The training process of the trained rock debris lithology identification network includes: collecting historical rock debris video image sequences on the belt conveyor and TBM tunneling parameters; synchronizing the historical rock debris video image sequences with the TBM tunneling parameters in time to obtain synchronization data; preprocessing the synchronization data to obtain a rock debris dataset; dividing the rock debris dataset into a training set, a validation set, and a test set; and using the training set to train the rock debris lithology identification network to obtain the trained rock debris lithology identification network.

[0006] The step of training the rock slag lithology identification network using a training set to obtain the trained rock slag lithology identification network includes: The rock debris targets in the historical rock debris video image sequence in the training set are located, and the images of the rock debris target areas corresponding to the target boxes are extracted. Texture features are extracted from images of target areas of rock debris in the training set. Channel attention and spatial attention are calculated for the texture features respectively. The channel attention and spatial attention are fused together. The fused features are input into the classification head to obtain the lithology category and confidence of historical images. The main task loss function is constructed based on the lithology categories of historical images and the actual lithology categories. The rock debris lithology identification network was updated using spatial intervention, background consistency, and occlusion reconstruction methods, respectively, and spatial intervention loss function, background consistency loss function, occlusion reconstruction loss function, and rule constraint terms were constructed. A total loss function is established based on the main task loss function, spatial intervention loss function, background consistency loss function, occlusion reconstruction loss function, and rule constraint terms. The total loss function is then used to iterate the rock slag video image sequence to obtain the trained rock slag lithology recognition network.

[0007] The space intervention method includes: Construct a set of candidate boxes based on the center coordinates of the target box, the width of the target box, the height of the target box, the translation amount, and the scaling factor; Each candidate bounding box in the candidate set is scored, and the candidate bounding box with the highest score is used as the updated bounding box in the rock slag lithology identification network.

[0008] The process of scoring each candidate bounding box in the candidate set, with the highest-scoring candidate bounding box being used as the updated bounding box in the rock debris lithology identification network, includes: ; ; in, For the k-th candidate bounding box in the candidate set, For the scoring function, For the candidate box set, As the first weight, As the second weight, As the third weight, To measure the extent to which the dominant evidence is weakened, As a stable term, For geometric validity constraints, This represents the updated target bounding box in the rock debris lithology identification network.

[0009] The background consistency method includes: Extract a frame from the historical rock debris video image sequence as the current rock debris image; Input the current rock debris image into the corresponding backbone network feature map to obtain the spatial attention map; The spatial attention map is then normalized to 0 to 1. The normalized spatial attention map is used as a foreground soft mask; Background features are obtained by using a foreground soft mask in the current rock debris image; Based on the background features, calculate the mean and standard deviation of each channel feature, and then concatenate the mean and standard deviation to form a background statistical descriptor; Construct a donor candidate set, which consists of samples from the training set other than the current rock debris image; Calculate the distance between the background statistical descriptor of the current rock debris image and the background statistical descriptor of each sample in the donor candidate set, and select the sample with the smallest distance as the background donor; The foreground of the current rock debris image remains unchanged, while the background is replaced by a background donor to obtain counterfactual features. The spatial attention map is used as the original feature. By inputting the original features and counterfactual features into the same classifier, we can obtain the original prediction results and the prediction results after background replacement, respectively. The background consistency loss function is used to ensure consistency between the original prediction results and the prediction results after background replacement.

[0010] The foreground of the current rock debris image remains unchanged, while the background is replaced by a background donor, resulting in counterfactual features, expressed as follows:

[0011] ; in, Background features that serve as a background donor. Foreground soft mask for background donor, The original features of the background donor, For the selected background donor sample, For the first Foreground soft mask for each sample, For the first The original features of each sample; The counterfactual features obtained by preserving the foreground of the current rock debris image and replacing the background of the current rock debris image with a background donor; The background consistency loss function is expressed as follows: ; in, This represents the category probability distribution corresponding to the original features. The category probability distribution corresponding to the counterfactual feature. The background consistency loss function is used. The divergence is Kullback-Leibler.

[0012] The occlusion reconstruction method includes: The channel vector corresponding to each spatial location in the fused features is used as the original spatial feature unit; The significance score of the spatial feature unit is obtained based on the original classification score corresponding to the original spatial feature unit and the actual lithology category of the sample. The spatial feature units are sorted from high to low based on their significance scores. According to the preset occlusion ratio, the original spatial feature units corresponding to the saliency scores of the first N spatial feature units are selected, and the N corresponding original spatial feature units are replaced with learnable mask vectors to obtain multiple occluded positions. Based on the unmasked context features and semantic conditions, the features of the masked locations are recovered to obtain the reconstructed features. The semantic conditions include: lithological attribute encoding. Calculate the mean squared error between the features at the occluded location and the reconstructed features; An occlusion reconstruction loss function is constructed based on the mean squared error between the features of the occluded location and the reconstructed features.

[0013] The significance score of the spatial feature unit is obtained based on the original classification score corresponding to the original spatial feature unit and the actual lithology category of the sample. The expression is as follows: ; in, For the first The significance score of each spatial feature unit For the first One original spatial feature unit, For the actual lithology category of the sample The corresponding original score for the category.

[0014] The content-aware TBM slag lithology identification method further includes: A pre-built symbol rule base is used to perform rule checks on the lithology category and confidence level of real-time images to obtain the check results; When the inspection result violates the corresponding rules, the scaling factor is determined according to the degree of rule violation. The original probability of the lithology category of the real-time image is suppressed, and the probability of the lithology category of all real-time images is renormalized to obtain the category probability after rule calibration. The category with the highest probability after rule calibration is taken as the lithology category of the final real-time image.

[0015] Secondly, the present invention also provides a content-aware TBM slag lithology identification system, comprising: The data acquisition module is used to acquire real-time video image sequences of rock slag on the belt conveyor; The target image extraction module is used to locate rock debris targets in the video image sequence of rock debris on the belt conveyor and extract the image of the rock debris target area corresponding to the target box; The image recognition module is used to input images of the target area of ​​rock debris into the trained rock debris lithology recognition network to obtain the lithology category and confidence level of the real-time image. The training process of the trained rock debris lithology recognition network includes: collecting historical rock debris video image sequences on the conveyor belt and TBM tunneling parameters; synchronizing the historical rock debris video image sequences with the TBM tunneling parameters in time to obtain synchronization data; preprocessing the synchronization data to obtain a rock debris dataset; dividing the rock debris dataset into a training set, a validation set, and a test set; and using the training set to train the rock debris lithology recognition network to obtain the trained rock debris lithology recognition network.

[0016] Thirdly, this application proposes an electronic device comprising: one or more processors, and a memory for storing instructions that, when executed by the one or more processors, cause the one or more processors to perform the content-aware TBM slag lithology identification method.

[0017] Fourthly, this application proposes a computer-readable storage medium storing executable instructions that, when executed, cause a processor to perform the content-aware TBM slag lithology identification method.

[0018] Fifthly, this application proposes a computer program product, including a computer program or instructions that, when executed by a processor, implement the aforementioned content-aware TBM slag lithology identification method.

[0019] Beneficial effects: This application proposes a content-aware TBM slag lithology identification method, which has the following beneficial effects: (1) To address the issues of image blurring, scale imbalance, and non-uniform distribution caused by the high-speed movement of TBMs at the site, we enhance the ability to extract lithological features through content-aware capture and content-aware attention mechanisms, and improve the recognition effect of rock blocks of different grain sizes through multi-scale context fusion.

[0020] (2) Through rule-guided spatial counterfactual intervention, counterfactual background decoupling, and saliency-driven semantic regularization training, the model’s dependence on geometric, background and saliency shortcut evidence is actively reduced, and the robustness of the model under the distribution changes of illumination, dust, mud, water stain reflection, occlusion and detection box jitter is improved.

[0021] (3) By using neural symbol prior constraints and reliability calibration, it is possible to perform gating suppression or confidence adjustment on predictions that violate the rules, and output reliability scores and uncertainty labels, thereby reducing the negative impact of abnormal samples, noise labels and out-of-distribution samples on training and inference, facilitating on-site manual review and feedback loop, and improving the reliability of engineering deployment.

[0022] (4) By introducing time series fusion and reliability threshold strategies, the continuous frame lithology output is smoothed and uncertainty is suppressed, thereby reducing the impact of misjudgment caused by single frame noise, instantaneous reflection and short-term occlusion on the proportion statistics, and improving the stability and engineering usability of lithology time series curves.

[0023] (5) The training and evaluation process of the present invention can be adapted to no less than six types of lithology system and supports the expansion of categories according to engineering needs; the model performance can be quantitatively verified through standardized data division and index calculation, and the rule base or model parameters can be incrementally updated on site through feedback mechanism to continuously maintain cross-working condition recognition performance. Attached Figure Description

[0024] Figure 1 A flowchart of a content-aware TBM slag lithology identification method according to an embodiment of the present invention; Figure 2 A schematic diagram of a multi-scale rock slag lithology identification network based on content-aware attention according to an embodiment of the present invention; Figure 3 A schematic diagram of rule-guided spatial counterfactual intervention candidate box construction according to an embodiment of the present invention; Figure 4 A schematic diagram illustrating background decoupling, structure matching, and background replacement according to an embodiment of the present invention; Figure 5 A schematic diagram of saliency-driven semantic regularization masking and semantic condition reconstruction in an embodiment of the present invention; Figure 6 A schematic diagram of neural symbol prior constraints and reliability calibration according to an embodiment of the present invention. Detailed Implementation

[0025] The specific implementation methods of this application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0026] Example 1: This embodiment provides a content-aware TBM slag lithology identification method, such as... Figure 1 ,include: Step S1: Real-time acquisition of video image sequences of rock slag on the conveyor belt; In this embodiment, the device for acquiring video image sequences of rock debris on the conveyor belt includes: an industrial camera, a mounting bracket, a supplementary lighting device, and a polarizing filter. The industrial camera can be a global shutter camera and fixedly mounted above or to the side of the conveyor belt. The supplementary lighting device can be a constant-brightness supplementary light source or a strobe light source to shorten exposure time and reduce motion blur. The polarizing filter is used to suppress water stain reflections and localized highlight interference. The frame rate and resolution can be set according to the conveyor belt speed, rock debris particle size distribution, and target size requirements. It is understood that the same device is used for both real-time acquisition of video image sequences of rock debris on the conveyor belt and acquisition of historical video image sequences of rock debris on the conveyor belt.

[0027] Step S2: Locate the rock debris targets in the video image sequence of rock debris on the conveyor belt, and extract the image of the rock debris target area corresponding to the target box; Step S3: Input the image of the target area of ​​rock debris into the trained rock debris lithology identification network to obtain the lithology category and confidence level of the real-time image. The training process of the trained rock debris lithology identification network includes: collecting historical rock debris video image sequences on the belt conveyor and TBM tunneling parameters; synchronizing the historical rock debris video image sequences with the TBM tunneling parameters in time to obtain synchronization data; preprocessing the synchronization data to obtain a rock debris dataset; dividing the rock debris dataset into a training set, a validation set, and a test set; and using the training set to train the rock debris lithology identification network to obtain the trained rock debris lithology identification network.

[0028] In this embodiment, historical rock debris video image sequences on the conveyor belt are collected, and TBM tunneling parameters can be collected as auxiliary information; the historical rock debris video image sequences and TBM tunneling parameters are synchronized in time, and the synchronized image frame sequence and its timestamp information are output.

[0029] The acquired video frames are preprocessed, and sample selection, annotation management and dataset partitioning are performed offline to output a rock slag lithology dataset for training and evaluation.

[0030] Furthermore, the synchronized data is preprocessed, including: target area localization and cropping submodule, noise reduction and distortion correction submodule, motion blur suppression submodule, brightness and color normalization submodule, annotation and quality inspection submodule, and dataset partitioning and augmentation submodule; among which data augmentation includes random motion blur, random occlusion, brightness and contrast perturbation, color jitter and background perturbation to simulate differences in on-site working conditions such as dust, water stains, and changes in lighting.

[0031] Step S3.1.1: Deploy image acquisition equipment and acquire historical rock debris video image sequences on the belt conveyor, and acquire TBM tunneling parameters (including: thrust, torque, rotational speed, and penetration depth) as auxiliary information; Step S3.1.2: Alignment is performed based on timestamps. Nearest neighbor matching or linear interpolation can be used to complete parameter alignment. Output the synchronization frame sequence and the corresponding parameter sequence, and use the synchronization frame sequence and the corresponding parameter sequence as synchronization data.

[0032] Step S3.2.1: Preprocess the synchronization data by denoising, correcting distortion, and normalizing brightness and color of the video frames; for high-speed motion blur, a deblurring strategy based on temporal consistency constraints, frequency domain filtering, or convolutional filtering can be used to suppress it; and the resolution and scale are unified.

[0033] Step S3.2.2: Annotation and Dataset Construction. Using annotation tools, target bounding boxes are created for rock debris targets in the video frames, and the corresponding lithology category is labeled for each target. Based on the target bounding boxes, target region images are extracted from the video frames to form an annotated dataset, which is then divided into training, validation, and test sets according to a preset ratio. Further class balancing and annotation quality control can be performed, such as double-checking, consistency checks, and weak noise filtering. Data augmentation can also be introduced to simulate differences in various working conditions.

[0034] It is understandable that subsequent lithology identification, spatial counterfactual intervention, and target region extraction in the reasoning stage are all based on the target bounding box.

[0035] The step of training the rock slag lithology identification network using a training set to obtain the trained rock slag lithology identification network includes: Step S100: Locate the rock debris targets in the historical rock debris video image sequence in the training set, and extract the image of the rock debris target area corresponding to the target box; Step S101: Extract texture features from images of target areas of rock debris in the training set, calculate channel attention and spatial attention for the texture features respectively, fuse the channel attention and spatial attention, input the fused features into the classification head to obtain the lithology category and confidence of historical images; In this embodiment, the image of the target area of ​​the rock debris is further classified into lithological categories based on content-aware attention and multi-scale fusion, and the corresponding confidence level is output. Content-aware capture is used to enhance the lithological texture and grain structure features under fuzzy conditions, content-aware attention is used to highlight the effective areas related to lithology, and multi-scale fusion is used to improve the ability to identify rock blocks of different grain sizes.

[0036] In this embodiment, a rock lithology identification network oriented towards motion fuzziness and scale imbalance is constructed, such as... Figure 2 As shown, it includes at least content-aware capture, content-aware attention mechanism and multi-scale context fusion, and outputs lithology category and confidence level.

[0037] Step S101.1: Content-aware capture, including: dividing the image of the target area of ​​rock debris in the training set into multiple local receptive fields, and adaptively responding to the blurred area based on the content-adaptive dynamic convolution kernel and dynamic weight generation mechanism, so as to simultaneously achieve lithological texture extraction and blur interference suppression. Step S101.2: Content-aware attention mechanism, including: calculating channel attention and spatial attention for texture features respectively, and generating a unique spatial importance map for each channel; fusing channel attention and spatial attention to enhance cross-channel and cross-space information interaction, highlighting lithological evidence such as texture, grain structure, and bedding bands; (1) Channel attention: ; in, For texture features, For global average pooling, It is a multilayer perceptron. For the Sigmoid function, This is the channel attention weight vector.

[0038] (2) Spatial attention: ; in, For average pooling along the channel dimension, For max pooling along the channel dimension, For splicing operations, For convolution operations, This is a spatial attention map.

[0039] Step S101.3: Multi-scale context fusion, including: upsampling / downsampling features at different levels and aligning them to the same resolution before fusion, in order to improve the ability to identify rock blocks of different grain sizes; (3) Attention fusion: ; in, This indicates channel broadcast multiplication. This indicates element-wise multiplication. This is an attention-enhancing feature.

[0040] (4) Multi-scale fusion: ; in, To align the resolution with upsampling or downsampling transformation, For learnable or preset weights, For feature-level indexing, The total number of feature levels participating in multi-scale fusion. The final multi-scale fused feature is obtained by resolution alignment and weighted fusion of attention enhancement features at each level. This is the attention enhancement feature for the l-th layer.

[0041] In this embodiment, the final input of the classifier is uniformly defined as the final multi-scale fused feature. The specific processing order is as follows: First, attention enhancement is applied to features at different levels to obtain attention-enhanced features corresponding to each level. Then, the features at each level are adjusted to a uniform spatial size, and multi-scale fusion is performed using a weighted method to obtain the final multi-scale fused features. Finally, Input a global pooling layer and a classifier, and output the lithology classification results.

[0042] Step S101.4: Identify the output, including: the final multi-scale fused features Input a classification head (or a combined detection and classification head) and output lithology category and confidence level; in video scenarios, temporal smoothing or majority voting can be performed on consecutive frame outputs to improve stability.

[0043] Step S102: Construct the main task loss function based on the lithology category of historical images and the actual lithology category; Step S103: The rock debris lithology identification network is updated using spatial intervention method, background consistency method and occlusion reconstruction method respectively, and spatial intervention loss function, background consistency loss function, occlusion reconstruction loss function and rule constraint terms are constructed. In this embodiment, three types of interventions are introduced during the training phase: spatial counterfactual, background decoupling consistency, and saliency occlusion reconstruction, in order to suppress shortcut dependencies of geometric position, background texture, and local salient regions and improve robustness across operating conditions.

[0044] The spatial intervention method, such as Figure 3 As shown, it includes: Step S103.1.1: Construct a candidate box set based on the center coordinates of the target box, the width of the target box, the height of the target box, the translation amount, and the scaling factor; In this embodiment, a set of restricted translation / scaling candidate boxes is constructed around the target candidate box. Counterfactual boxes are selected by a scoring function that comprehensively represents the degree of adversarial information attenuation, geometric effectiveness, and training stability, and a spatially perturbed training view is generated to enhance robustness to detection box jitter and local clipping errors.

[0045] Candidate box generation: ; in, The center coordinates of the target bounding box The width of the target box. The height of the target bounding box. The horizontal coordinate translation of the target bounding box. This represents the vertical translation of the target bounding box. This is the scaling factor.

[0046] The process of scoring each candidate bounding box in the candidate set, with the highest-scoring candidate bounding box being used as the updated bounding box in the rock debris lithology identification network, includes: ; ; in, For the k-th candidate bounding box in the candidate set, For the scoring function, For the candidate box set, As the first weight, As the second weight, As the third weight, To measure the degree to which the dominant evidence is weakened (this can be constructed using factors such as the difference in prediction probabilities). For stable terms (overlap / visibility ratio / outbound penalty, etc.) Geometric validity constraints (perturbation amplitude penalty / curriculum coefficient, etc.) are applied. This represents the updated target bounding box in the rock debris lithology identification network.

[0047] in, This measures the degree to which a candidate bounding box weakens the model's original dominant discriminative evidence. Let the original target bounding box be... , No. The candidate target boxes are The true category of the sample is The model outputs the predicted probability for the true class as follows: This measures the degree to which the dominant evidence is weakened. The calculation formula is: ; The more significantly the candidate box reduces the probability of the true class prediction, the stronger its weakening effect on the original discriminative evidence of the model. The larger.

[0048] in, This metric is used to ensure that the candidate bounding boxes do not deviate significantly from the original target region. It considers both the degree of overlap between the candidate and original bounding boxes and the proportion of the candidate bounding box within the valid image area. ; in, This indicates the intersection-union ratio (IoU) between the candidate bounding box and the original bounding box. This represents the proportion of the candidate box's area within the valid image region to the total area of ​​the candidate boxes. As the first non-negative weight, It is the second non-negative weight, and satisfies .

[0049] in, The formula is used to constrain the translation and scaling of the candidate bounding box relative to the original bounding box, preventing excessive changes in the position or scale of the candidate bounding box. ; in, To represent the horizontal offset of the candidate box center relative to the original box center, This represents the vertical offset of the candidate box center relative to the original box center; The width of the original frame. This is the original box height; This indicates the scaling ratio of the candidate box relative to the original box; Allowable parameters for horizontal translation This is the allowable parameter for vertical translation. This is the allowable parameter for scale variation.

[0050] Measuring the degree to which the dominant evidence is weakened Stable items Geometric validity constraints All three metrics are normalized to 0 to 1, with larger values ​​indicating that the candidate boxes better meet the corresponding requirements.

[0051] Step S103.1.2: Score each candidate target box in the candidate set, and use the candidate target box with the highest score as the updated target box in the rock slag lithology identification network.

[0052] In this embodiment, rule-guided spatial counterfactual intervention, such as... Figure 4 As shown, a constrained translation / scaling candidate box set is constructed around the target candidate box. Counterfactual boxes are selected by a scoring function that comprehensively represents the degree of adversarial information attenuation, geometric effectiveness, and training stability, generating a spatially perturbed training view to enhance robustness to detection box jitter and local clipping errors.

[0053] The background consistency method includes: Step S103.2.1: Extract one frame from the historical rock debris video image sequence as the current rock debris image; Step S103.2.2: Input the current rock debris image into the corresponding backbone network feature map to obtain the spatial attention map; Step S103.2.3: Normalize the spatial attention map to 0 to 1; Step S103.2.4: Use the normalized spatial attention map as a foreground soft mask; Step S103.2.5: Apply a foreground soft mask to the current rock debris image to obtain background features; Step S103.2.6: Based on the background features, calculate the mean and standard deviation of each channel feature, and concatenate the mean and standard deviation to form a background statistical descriptor; Step S103.2.7: Construct a donor candidate set, which consists of samples from the training set other than the current rock debris image; Step S103.2.8: Calculate the distance between the background statistical descriptor of the current rock debris image and the background statistical descriptor of each sample in the donor candidate set, and select the sample with the smallest distance as the background donor; Step S103.2.9: The foreground of the current rock debris image remains unchanged, while the background is replaced by a background donor to obtain counterfactual features. The spatial attention map is used as the original feature. Step S103.2.10: Input the original features and counterfactual features into the same classifier to obtain the original prediction results and the prediction results after background replacement, respectively; Step S103.2.11: Use the background consistency loss function to make the original prediction results and the prediction results after background replacement consistent.

[0054] In this embodiment, the counterfactual background is decoupled, such as Figure 4As shown, features are divided into foreground and background based on attention / salience. Background donors are matched within the batch and replaced with background features to construct a counterfactual view. Consistency supervision is applied to the original view and the counterfactual view to reduce background co-occurrence dependency.

[0055] First, a frame is extracted from the historical rock debris video image sequence as the current rock debris image. A spatial attention map or saliency map is obtained from the backbone network feature map corresponding to the current rock debris image and normalized to 0 to 1. This attention map represents the importance of different locations to the current lithology identification result; locations with higher values ​​usually correspond to the main body of the rock debris or areas with significant lithology discrimination. After adjusting the attention map to the same spatial size as the feature map, it can be used as a foreground soft mask.

[0056] Subsequently, a foreground soft mask is used to weaken the features of the main rock debris area while preserving the background features, thus obtaining the background features of the current sample. To facilitate comparison of the background features of different samples, this embodiment calculates the mean and standard deviation of the features of each channel at the effective background location, and concatenates the two into a background statistical descriptor. The background statistical descriptor is used to summarize the overall distribution of the background across different feature channels.

[0057] Next, a donor candidate set is constructed, consisting of samples from the training set excluding the current sample. Candidate samples should have sufficient effective background regions, and samples from different video clips, different acquisition times, or different working conditions are preferred to avoid directly selecting adjacent frames that highly overlap with the current sample. Then, the distance between the background statistical descriptor of the current rock debris image and the background statistical descriptors of each sample in the donor candidate set is calculated. Among the samples that meet the selection criteria, the sample with the smallest background statistical descriptor distance is selected as the background donor.

[0058] After obtaining the background donor, the main features of the rock debris are removed using the foreground mask of the donor sample itself, and the retained background features are adjusted to the same spatial size as the current sample's feature map. The rock debris foreground of the current sample remains unchanged, while the background is replaced by the donor background, thus forming a counterfactual feature.

[0059] The original features and counterfactual features are input into the same classifier, yielding the original prediction result and the prediction result after background replacement, respectively. By constraining the two prediction results to be consistent, the model reduces its dependence on non-lithological information such as background, lighting, and equipment environment, and focuses more on the color, texture, grain size, and structural characteristics of the rock slag itself.

[0060] The sample with the smallest background statistical descriptor distance from the candidate donor set is selected as the background donor for the current sample for matching: ; ; Among them, let the first The original features of each sample are The foreground mask is Then its background features are Background features For statistical description vectors, For the candidate donor set.

[0061] in, Indicates the first The sample and the first Background statistical descriptor distance between candidate donor samples; Represented as the first The background donor sample index selected for each sample; Indicates the first Background features of each sample; Indicates the first The feature map is obtained by using a feature extraction network on each sample.

[0062] Background counterfactual characteristics and consistency: ; ; in, , The background features of the donor sample are used for matching and exchanging with the sample. and These are the foreground mask and original features of the donor sample, respectively. The selected background donor sample; in Formula 10, and The first Foreground mask and original features of each sample; The counterfactual features obtained by preserving the foreground of the current sample and replacing the background of the current rock debris image with a background donor; and These are the category probability distributions corresponding to the original features and the counterfactual features, respectively; Use the background consistency loss function; The divergence is Kullback-Leibler.

[0063] The occlusion reconstruction method, such as Figure 5 As shown, it includes: The occlusion reconstruction method includes: Step S103.3.1: Use the channel vector corresponding to each spatial location in the fused features as the original spatial feature unit; Step S103.3.2: Based on the original spatial feature units and the original classification scores corresponding to the actual lithology categories of the samples, obtain the significance scores of the spatial feature units; Step S103.3.3: Sort the saliency scores of spatial feature units from high to low; Step S103.3.4: Select the original spatial feature units corresponding to the saliency scores of the first N spatial feature units according to the preset occlusion ratio, and replace the N corresponding original spatial feature units with learnable mask vectors to obtain multiple occluded positions. Step S103.3.5: Based on the unmasked context features and semantic conditions, restore the features of the masked location to obtain the reconstructed features, wherein the semantic conditions include: lithological attribute encoding; Step S103.3.6: Calculate the mean squared error between the features at the occluded location and the reconstructed features; Step S103.3.7: Construct the occlusion reconstruction loss function based on the mean squared error between the features of the occluded location and the reconstructed features.

[0064] In this embodiment, the saliency score of the token or feature unit is calculated, the top-k salient units are selected for masking, and the masked units are reconstructed under the constraints of semantic conditional information. The reconstruction error forms a regularized loss, which forces the model to focus on more stable lithological evidence.

[0065] ; ; ; in, For the first One original spatial feature unit; For the actual lithology category of the sample The corresponding original classification score, i.e., the target class logit before Softmax normalization; For the first The significance score of each spatial feature unit; This represents the total number of spatial feature units. The number of spatial feature units to be masked; The most significant A set of indices for spatial feature units; To obtain the first under semantic conditional information constraints One reconstructed feature unit; Let be the occlusion reconstruction loss function at the occluded location.

[0066] In this embodiment, the feature map after multi-scale fusion but before global pooling is first used as the processing object. This feature map contains multiple spatial locations, each corresponding to a vector composed of all channel feature values. In this embodiment, the channel vector corresponding to each spatial location is called a spatial feature unit, or token. Therefore, the feature unit is not additionally generated data, but is directly obtained by expanding the fused feature map according to its spatial location.

[0067] During training, the original classification score corresponding to the true class of the current sample is selected, and the gradient of this score with respect to each spatial feature unit is calculated. Then, the L2 norm of the gradient vector at each spatial location is calculated, and the resulting scalar is used as the significance score for that location. The larger the significance score, the more significant the influence of that location on the prediction result of the current true class.

[0068] Select the most significant feature units according to a preset occlusion ratio, and replace these feature units with learnable mask vectors. The occlusion ratio can be determined through a validation set to avoid insufficient training constraints when occlusion is too small, or severe loss of original semantic information when occlusion is too large.

[0069] The reconstruction module recovers features from occluded locations based on unoccluded contextual features and semantic conditions. Semantic conditions may include at least lithological attribute encoding, and may further include TBM operating parameters such as propulsion force, cutterhead torque, rotational speed, and penetration depth acquired synchronously with the current image.

[0070] Finally, the mean squared error between the original features and the reconstructed features at the occluded location is calculated to form the reconstruction loss. This training process enables the model to recover key features using surrounding texture, granularity, and structural information even when local salient evidence is occluded, thereby reducing the model's over-reliance on a few locally high-response regions.

[0071] This branch is mainly used during the training phase and can be removed after training is complete without increasing the computational load of the actual inference process.

[0072] Step S104: Establish a total loss function based on the main task loss function, spatial intervention loss function, background consistency loss function, occlusion reconstruction loss function, and rule constraint terms. Use the total loss function to iterate the rock slag video image sequence to obtain the trained rock slag lithology recognition network.

[0073] Total loss function: ; in, For the total loss function, Loss to the main task This refers to spatial counterfactual related losses or constraints induced by the scoring strategy. For rule constraints, As the first weight, As the second weight, As the third weight, As the fourth weight, Let be the occlusion reconstruction loss function at the occluded location. This is the background consistency loss function.

[0074] The main task loss function is used to constrain the classification results of the rock slag lithology identification network for the original rock slag target area to be consistent with the actual lithology category, and is expressed as: ; in, This represents the total number of lithological categories. For the first The sample corresponds to the first The actual category indicator value of lithology, when the first The sample belongs to the first When the lithology is similar, ,otherwise ; For rock cinder lithology identification network prediction, the first The sample belongs to the first The probability of lithological similarity; The loss function is the main task loss function.

[0075] In defining the spatial counterfactual target frame Then, the corresponding target area of ​​the rock debris is extracted and input into the rock debris lithology identification network. Based on the prediction results and the actual lithology category, a spatial intervention loss function is constructed: ; in, For rock cinder lithology identification network prediction, the first The probability that the spatial counterfactual target region of a sample belongs to lithology type c; This represents the spatial intervention loss function; the meanings of the other symbols are the same as those of the main task loss function.

[0076] The content-aware TBM slag lithology identification method further includes: A pre-built symbol rule base is used to perform rule checks on the lithology category and confidence level of real-time images to obtain the check results; When the inspection result violates the corresponding rules, the scaling factor is determined according to the degree of rule violation. The original probability of the lithology category of the real-time image is suppressed, and the probability of the lithology category of all real-time images is renormalized to obtain the category probability after rule calibration. The category with the highest probability after rule calibration is taken as the lithology category of the final real-time image.

[0077] Specifically, such as Figure 6 As shown, it also includes symbol rule base construction, rule checking and confidence adjustment, and prototype distance reliability calibration.

[0078] Based on the color, texture, grain size, bedding bands, and mineral reflectivity characteristics corresponding to different lithologies, color constraints and texture constraints are constructed, and mutual exclusion constraints, implied constraints, and short-time window stability constraints are established between different lithology categories and their attributes, forming a symbol rule library.

[0079] The rock debris lithology identification network outputs the raw probabilities of each lithology category. A symbolic rule base checks the prediction results for each category. When a category violates a rule, a scaling factor is determined based on the degree of violation to suppress the raw probability of that category. All category probabilities are then re-normalized to obtain rule-calibrated category probabilities. The category with the highest rule-calibrated probability is then selected as the predicted category for the current image.

[0080] The lithology identification network for slag extracts sample embeddings from multi-scale fused features. The category prototype is obtained by averaging the embedding features of samples of the same lithology category in the training set. During the training phase, the distance between the sample embedding and the actual labeled category prototype is calculated to obtain the reliability weight used for weighted learning; during the inference phase, the distance between the sample embedding and the predicted category prototype is calculated to obtain the reliability weight of the identification result. The smaller the distance, the higher the reliability; when the reliability weight is below a set threshold, the identification result is marked as pending confirmation.

[0081] During the training phase, the degree of rule violation is used to construct the rule constraint loss and participates in the network parameter optimization. During the real-time inference phase, symbolic rules are used to adjust the class probabilities, while prototype distance is used to evaluate the reliability of the identification results without updating the network parameters. The final output includes the predicted lithology class, the confidence level after rule calibration, and the reliability score.

[0082] In this embodiment, a symbol rule base is constructed, which includes color constraints, texture constraints, mutual exclusion constraints, implication constraints, and short-time window stationarity constraints. The prediction results that violate the rules are subjected to rule checks, gating suppression, or confidence adjustments. Reliability weights are generated based on sample embedding, class prototype distance, and uncertainty to improve training stability and output credibility.

[0083] Step S5.1: Construction of the symbol rule base. Based on the color characteristics, texture characteristics, grain size characteristics, bedding banding characteristics, and mineral reflectivity characteristics corresponding to different lithologies, color constraints and texture constraints are established; furthermore, mutual exclusion constraints, implied constraints, and short-time window stability constraints are established between different lithology categories and their attributes to form a symbol rule base for rule checking.

[0084] Step S5.2: Rule Check and Gating Adjustment. A rule check is performed on the probability distribution of lithology categories output by the model. When the prediction results violate color constraints, texture constraints, mutual exclusion constraints, implication constraints, or short-time window stationarity constraints, gating suppression or confidence adjustment is applied to the corresponding categories to reduce the output weight of unreasonable prediction results.

[0085] Step S5.3: Reliability Calibration. Calculate the distance from the sample embedding to the corresponding class prototype and map it to a reliability weight, which is used for weighted learning during the training phase or uncertainty labeling during the inference phase.

[0086] To facilitate the implementation of step S5, a set of optional expressions that do not constitute restrictions are provided: Confidence adjustment: ; in, Norm is the scaling factor obtained from the rule check, and Norm is the normalization operator; Indicates the category before rule adjustment The corresponding calibration probability; This represents the category after rule scaling and renormalization. The corresponding probability.

[0087] Prototype distance reliability: ; in, For the first Embedding features of each sample; For the first For each sample, the actual labeled category is used during the training phase, and the predicted category is used during the inference phase. For category The category prototype is obtained by averaging the embedding features of samples of the same category in the training set; The Euclidean distance between the sample embedding and the class prototype; The prototype distance scale parameter is determined using a validation set. It is a natural exponential function; This is used as a reliability weight. The closer the sample embedding is to the class prototype, the higher the reliability weight. The closer the result is to 1, the higher the reliability of the recognition result.

[0088] The training phase and the inference phase need to be explained separately: The actual labeled categories of the samples are known during the training phase, therefore Take the actual label The corresponding category prototype should be written as .

[0089] No categories are actually labeled during the inference phase, therefore Take the model's predicted category The corresponding category prototype should be written as .

[0090] After obtaining the trained rock slag lithology recognition network, during the inference phase, real-time video image sequences of rock slag on the conveyor belt are acquired. The rock slag targets in the video image sequences of rock slag on the conveyor belt are located, and the images of the rock slag target areas corresponding to the target boxes are extracted. The images of the rock slag target areas are input into the trained rock slag lithology recognition network to obtain the lithology category and confidence level of the real-time images. The recognition results are then statistically analyzed for proportion and time series.

[0091] Furthermore, it includes the following sub-steps: Real-time inference and time-series fusion. For continuous frame output, sliding window majority voting or exponential moving average can be used for fusion; when the reliability score is lower than the set threshold, the result can be marked as pending confirmation and delayed until subsequent windows accumulate evidence before making a decision, thereby improving the stability of alarm and percentage curves.

[0092] Furthermore, in other embodiments, the rock debris target can also be segmented and labeled at the pixel level; when segmentation and labeling are used, a bounding box can be generated based on the segmented region, and target region extraction, spatial counterfactual intervention and lithology identification can be completed based on the bounding box.

[0093] (1) The lithology identification module can use a convolutional network or a visual Transformer as the backbone network; the content-aware capture module can be replaced by dynamic convolution, deformable convolution or temporal convolution. (2) Counterfactual background decoupling can be achieved in the pixel domain, feature domain, or a combination of both; background donor matching can be based on activation structure similarity, texture statistical similarity, or a combination thereof; (3) Semantic condition information can be provided by lithological attribute vocabulary, engineer-annotated description, TBM tunneling parameters or their fusion vectors, and injected into the decoder through splicing, gating or cross-attention; (4) The symbol rule library can be formulated by experts or automatically generated by historical data statistics, and incrementally updated through the engineer feedback mechanism to adapt to different engineering sections and different working conditions.

[0094] Example 2: This embodiment also provides a content-aware TBM slag lithology identification system, including: a data acquisition module, a target image extraction module, and an image recognition module, wherein the data acquisition module, the target image extraction module, and the image recognition module are connected in sequence; The data acquisition module is used to acquire real-time video image sequences of rock slag on the belt conveyor; The target image extraction module is used to locate rock debris targets in the video image sequence of rock debris on the belt conveyor and extract the image of the rock debris target area corresponding to the target box; The image recognition module is used to input images of the target area of ​​rock debris into the trained rock debris lithology recognition network to obtain the lithology category and confidence level of the real-time image. The training process of the trained rock debris lithology recognition network includes: collecting historical rock debris video image sequences on the conveyor belt and TBM tunneling parameters; synchronizing the historical rock debris video image sequences with the TBM tunneling parameters in time to obtain synchronization data; preprocessing the synchronization data to obtain a rock debris dataset; dividing the rock debris dataset into a training set, a validation set, and a test set; and using the training set to train the rock debris lithology recognition network to obtain the trained rock debris lithology recognition network.

[0095] Example 3: This embodiment proposes an electronic device, including: one or more processors, and a memory, wherein the memory is used to store instructions, and when the instructions are executed by the one or more processors, the one or more processors execute the content-aware TBM slag lithology identification method.

[0096] The electronic device may be a mobile phone, computer, or tablet computer, etc., and includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements a content-aware TBM slag lithology identification method as described in the embodiments. It is understood that the electronic device may also include input / output (I / O) interfaces and communication components.

[0097] The processor is used to execute all or part of the steps in the content-aware TBM lithology identification method as described in the above embodiments. The memory is used to store various types of data, which may include, for example, instructions for any application or method in an electronic device, as well as application-related data.

[0098] The processor can be implemented as an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic components, and is used to execute the content-aware TBM lithology identification method described in the above embodiments.

[0099] Example 4: This embodiment proposes a computer-readable storage medium that stores executable instructions. When these instructions are executed, if they are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0100] The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the content-aware TBM slag lithology identification method described in the various embodiments of this application.

[0101] The aforementioned storage media include: flash memory, hard disk, multimedia card, card-type memory (e.g., SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory), random access memory (RAM), static random-access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, disk, optical disk, server, APP (Application) application store, and other media capable of storing program verification codes. These media store computer programs, which, when executed by a processor, can implement the various steps of the content-aware TBM lithology identification method described above.

[0102] Example 5: This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the aforementioned content-aware TBM slag lithology identification method.

[0103] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.

[0104] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0105] The scope of protection of this application is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of equivalent technology of this disclosure, then the intent of this disclosure also includes such modifications and variations.

Claims

1. A content-aware TBM slag lithology identification method, characterized in that, include: Real-time acquisition of video image sequences of rock slag on belt conveyors; The rock debris targets in the video image sequence of rock debris on the conveyor belt are located, and the images of the rock debris target areas corresponding to the target boxes are extracted. The image of the target area of ​​rock debris is input into the trained rock debris lithology identification network to obtain the lithology category and confidence level of the real-time image. The training process of the trained rock debris lithology identification network includes: collecting historical rock debris video image sequences on the belt conveyor and TBM tunneling parameters; synchronizing the historical rock debris video image sequences with the TBM tunneling parameters in time to obtain synchronization data; preprocessing the synchronization data to obtain a rock debris dataset; dividing the rock debris dataset into a training set, a validation set, and a test set; and using the training set to train the rock debris lithology identification network to obtain the trained rock debris lithology identification network.

2. The content-aware TBM slag lithology identification method according to claim 1, characterized in that, The step of training the rock slag lithology identification network using a training set to obtain the trained rock slag lithology identification network includes: The rock debris targets in the historical rock debris video image sequence in the training set are located, and the images of the rock debris target areas corresponding to the target boxes are extracted. Texture features are extracted from images of target areas of rock debris in the training set. Channel attention and spatial attention are calculated for the texture features respectively. The channel attention and spatial attention are fused together. The fused features are input into the classification head to obtain the lithology category and confidence of historical images. The main task loss function is constructed based on the lithology categories of historical images and the actual lithology categories. The rock debris lithology identification network was updated using spatial intervention, background consistency, and occlusion reconstruction methods, respectively, and spatial intervention loss function, background consistency loss function, occlusion reconstruction loss function, and rule constraint terms were constructed. A total loss function is established based on the main task loss function, spatial intervention loss function, background consistency loss function, occlusion reconstruction loss function, and rule constraint terms. The total loss function is then used to iterate the rock slag video image sequence to obtain the trained rock slag lithology recognition network.

3. The content-aware TBM slag lithology identification method according to claim 2, characterized in that, The space intervention method includes: Construct a set of candidate boxes based on the center coordinates of the target box, the width of the target box, the height of the target box, the translation amount, and the scaling factor; Each candidate bounding box in the candidate set is scored, and the candidate bounding box with the highest score is used as the updated bounding box in the rock slag lithology identification network.

4. The content-aware TBM slag lithology identification method according to claim 3, characterized in that, The process of scoring each candidate bounding box in the candidate set, with the highest-scoring candidate bounding box being used as the updated bounding box in the rock debris lithology identification network, includes: ； ； in, For the k-th candidate bounding box in the candidate set, For the scoring function, For the candidate box set, As the first weight, As the second weight, As the third weight, To measure the extent to which the dominant evidence is weakened, As a stable term, For geometric validity constraints, This represents the updated target bounding box in the rock debris lithology identification network.

5. The content-aware TBM slag lithology identification method according to claim 2, characterized in that, The background consistency method includes: Extract a frame from the historical rock debris video image sequence as the current rock debris image; Input the current rock debris image into the corresponding backbone network feature map to obtain the spatial attention map; The spatial attention map is then normalized to 0 to 1. The normalized spatial attention map is used as a foreground soft mask; Background features are obtained by using a foreground soft mask in the current rock debris image; Based on the background features, calculate the mean and standard deviation of each channel feature, and then concatenate the mean and standard deviation to form a background statistical descriptor; Construct a donor candidate set, which consists of samples from the training set other than the current rock debris image; Calculate the distance between the background statistical descriptor of the current rock debris image and the background statistical descriptor of each sample in the donor candidate set, and select the sample with the smallest distance as the background donor; The foreground of the current rock debris image remains unchanged, while the background is replaced by a background donor to obtain counterfactual features. The spatial attention map is used as the original feature. By inputting the original features and counterfactual features into the same classifier, we can obtain the original prediction results and the prediction results after background replacement, respectively. The background consistency loss function is used to ensure consistency between the original prediction results and the prediction results after background replacement.

6. The content-aware TBM slag lithology identification method according to claim 5, characterized in that, The foreground of the current rock debris image remains unchanged, while the background is replaced by a background donor, resulting in counterfactual features, expressed as follows: ； in, Background features that serve as a background donor. Foreground soft mask for background donor, The original features of the background donor, For the selected background donor sample, For the first Foreground soft mask for each sample, For the first The original features of each sample; The counterfactual features obtained by preserving the foreground of the current rock debris image and replacing the background of the current rock debris image with a background donor; The background consistency loss function is expressed as follows: ； in, This represents the category probability distribution corresponding to the original features. The category probability distribution corresponding to the counterfactual feature. The background consistency loss function is used. The divergence is Kullback-Leibler.

7. A content-aware TBM slag lithology identification method according to claim 2, characterized in that, The occlusion reconstruction method includes: The channel vector corresponding to each spatial location in the fused features is used as the original spatial feature unit; The significance score of the spatial feature unit is obtained based on the original classification score corresponding to the original spatial feature unit and the actual lithology category of the sample. The spatial feature units are sorted from high to low based on their significance scores. According to the preset occlusion ratio, the original spatial feature units corresponding to the saliency scores of the first N spatial feature units are selected, and the N corresponding original spatial feature units are replaced with learnable mask vectors to obtain multiple occluded positions. Based on the unmasked context features and semantic conditions, the features of the masked locations are recovered to obtain the reconstructed features. The semantic conditions include: lithological attribute encoding. Calculate the mean squared error between the features at the occluded location and the reconstructed features; An occlusion reconstruction loss function is constructed based on the mean squared error between the features of the occluded location and the reconstructed features.

8. The content-aware TBM slag lithology identification method according to claim 7, characterized in that, The significance score of the spatial feature unit is obtained based on the original classification score corresponding to the original spatial feature unit and the actual lithology category of the sample. The expression is as follows: ； in, For the first The significance score of each spatial feature unit For the first One original spatial feature unit, For the actual lithology category of the sample The corresponding original score for the category.

9. A content-aware TBM slag lithology identification method according to claim 1, characterized in that, Also includes: A pre-built symbol rule base is used to perform rule checks on the lithology category and confidence level of real-time images to obtain the check results; When the inspection result violates the corresponding rules, the scaling factor is determined according to the degree of rule violation. The original probability of the lithology category of the real-time image is suppressed, and the probability of the lithology category of all real-time images is renormalized to obtain the category probability after rule calibration. The category with the highest probability after rule calibration is taken as the lithology category of the final real-time image.

10. A content-aware TBM slag lithology identification system, implemented using the content-aware TBM slag lithology identification method according to any one of claims 1 to 9, comprising: The data acquisition module is used to acquire real-time video image sequences of rock slag on the belt conveyor; The target image extraction module is used to locate rock debris targets in the video image sequence of rock debris on the belt conveyor and extract the image of the rock debris target area corresponding to the target box; The image recognition module is used to input images of the target area of ​​rock debris into the trained rock debris lithology recognition network to obtain the lithology category and confidence level of the real-time image. The training process of the trained rock debris lithology recognition network includes: collecting historical rock debris video image sequences on the conveyor belt and TBM tunneling parameters; synchronizing the historical rock debris video image sequences with the TBM tunneling parameters in time to obtain synchronization data; preprocessing the synchronization data to obtain a rock debris dataset; dividing the rock debris dataset into a training set, a validation set, and a test set; and using the training set to train the rock debris lithology recognition network to obtain the trained rock debris lithology recognition network.

Images