A method and system for identifying water leakage in shield tunnels based on visual language

Abstract

This invention relates to the field of tunnel defect detection technology, specifically to a method and system for identifying water leakage in shield tunnels based on visual language. The method includes the following steps: A detection network constructed based on a contrastive language-image pre-trained model is used. A clutter-gated adapter is used to perform domain adaptation on the visual features extracted by the detection network, generating position-wise clutter confidence scores, allowing visually significant but semantically normal clutter regions to bypass the domain adaptation transformation; a multi-scale feature projector is used to fuse shallow and deep visual features from different receptive fields according to the prior weighting of the leakage scale in the training set, obtaining multi-scale patch features; attention weighting is applied to the multi-scale patch features based on normal basis anchoring attention, using the normal basis prototype set obtained from offline clustering as a reference system to highlight leakage regions that deviate from the normal distribution; based on the cosine similarity between the multi-scale patch features and three preset sets of text anchor points, combined with position-wise clutter confidence scores, the water leakage detection result is obtained.

Description

Technical Field

[0001] This invention relates to the field of tunnel defect detection technology, specifically to a method and system for identifying water leakage in shield tunnels based on visual language. Background Technology

[0002] With the continuous development of underground space engineering, shield tunnels are playing an increasingly important role in urban rail transit, municipal utility tunnels, and other infrastructure. The efficiency of intelligent detection and identification of tunnel lining defects has gradually become a key focus of the industry. During tunnel operation and subsequent maintenance, various factors such as surrounding load disturbances, material performance degradation, and insufficient segment assembly precision can cause water leakage defects on the surface of shield tunnel segments. This leakage can seep into the segments, leading to corrosion of the reinforced concrete and ultimately endangering the overall stability of the shield tunnel structure. Therefore, accurate detection and location of water leakage in shield tunnel segments is a crucial step in ensuring the safe and reliable operation of underground spaces.

[0003] Currently, the main technical approaches and shortcomings for detecting water leakage in shield tunnels are as follows: One method is the manual inspection method, which relies on the visual interpretation of maintenance personnel. This method is highly subjective, inefficient, and is mostly carried out during nighttime maintenance windows, making it difficult to meet the requirements of large-scale tunnel operation for inspection accuracy and timeliness.

[0004] The second method is based on traditional machine vision, which uses threshold segmentation, edge detection, and template matching to identify areas of water leakage. This type of method is sensitive to changes in light and humidity, and lacks the ability to distinguish structural backgrounds commonly found on the surface of tunnel segments, such as bolts, pipeline joints, engraved numbers, and surface undulations. It is prone to false detections in complex lining backgrounds.

[0005] Thirdly, there is the supervised detection method based on deep learning, which uses convolutional neural networks or Transformer-type networks to segment or detect water leakage images. This type of method has good accuracy in scenarios consistent with the training distribution, but its generalization ability is weak in scenarios such as shield tunnels where the background texture varies greatly, the sample size is limited, and the training set contains a large number of visually similar structural backgrounds to the leakage, resulting in a decrease in detection accuracy across different working conditions.

[0006] Fourthly, there are zero-shot and few-shot anomaly detection methods based on visual language pre-trained models (such as CLIP) that have emerged in recent years. These methods, leveraging semantic priors obtained through large-scale image and text pre-training, demonstrate a certain generalization ability on clean backgrounds of general industrial anomaly detection benchmarks (such as MVTec AD). However, in scenarios with highly complex background textures, such as tunnel boring machines, the models often misclassify visually similar but semantically normal content such as bolt shadows, engraved serial numbers, old stains at pipeline joints, and surface undulation shadows as anomalies, resulting in significant false detection problems.

[0007] Therefore, there is a need for an intelligent water leakage detection method that can effectively suppress structural background clutter in shield tunnel scenarios while retaining the generalization ability of visual language models. This method should be able to collaboratively suppress structural clutter semantics at the text encoding end and the visual encoding end, and combine leakage area scale priors, normal base priors, and industrial domain difference priors to maintain stable and reliable detection performance in shield tunnel operation scenarios with complex background textures and limited training samples. Summary of the Invention

[0008] To address the shortcomings of existing methods and the needs of practical applications, this invention provides a visual language-based method for identifying water leakage in shield tunnels, comprising the following steps: Images of the shield tunnel segments to be detected are acquired and input into a detection network constructed based on a contrastive language-image pre-trained model. A clutter gating adapter is used to perform domain adaptation on the visual features extracted by the detection network, generating position-wise clutter confidence scores to allow visually significant but semantically normal clutter regions to bypass the domain adaptation transformation. A multi-scale feature projector is used to fuse shallow and deep visual features from different receptive fields according to the prior weighting of the leakage scale in the training set, resulting in multi-scale patch features. Attention weighting is applied to the multi-scale patch features based on normal basis anchoring attention, using the normal basis prototype set obtained from offline clustering as a reference system to highlight leakage regions that deviate from the normal distribution. Based on the cosine similarity between the multi-scale patch features and three preset sets of text anchor points, combined with the position-wise clutter confidence scores, the leakage detection result is obtained.

[0009] Optionally, the clutter gating adapter is deployed in the shallow layers of the text encoder and the vision encoder, respectively, and internally maintains a clutter template library for performing the following steps: The maximum cosine similarity between the input features and the clutter template library is calculated as the position-wise clutter confidence score; the input features are adapted by performing an adaptation transformation to obtain the adapted features; the original input features and the adapted features are convexly combined and fused using the position-wise clutter confidence score as the weight to obtain the output features; the clutter template library is obtained by K-means clustering of shallow patch features of normal samples in the training set, and is updated by exponential moving average during training.

[0010] Optionally, the multi-scale feature projector is deployed in the middle of the visual encoder to perform the following steps: The shallow and deep group features of the visual encoder are extracted, where the shallow group features correspond to a small effective receptive field and the deep group features correspond to a large effective receptive field. The shallow and deep group features are mapped to a shared semantic space through independent linear projection heads and then L2 normalized. The two normalized features are weighted and summed according to the fixed weights obtained from the scale distribution of the leakage annotation area in the offline statistical training set to obtain the multi-scale patch features.

[0011] Optionally, the normal substrate anchoring attention is deployed deep within the visual encoder to perform the following steps: During the model initialization phase, the deep patch features obtained from the normal samples in the training set through the original visual encoder are subjected to offline K-means clustering to obtain a fixed normal basis prototype set. The query-key similarity calculation of the original multi-head attention is replaced by the inner product between the value vector of each patch and the row-wise cosine similarity vector of the normal basis prototype set to obtain the attention weight. The value vector is weighted and summed with the attention weight to obtain the attention-weighted features.

[0012] Optionally, the three sets of text anchor points are preset, including the following steps: Three sets of text prompt word templates are preset, corresponding to the clean and normal state of shield tunnel segments, the cluttered and normal state with structural background, and the abnormal state of water leakage, respectively. These three sets of text prompt word templates are input into a text encoder, and the average of the encoded text is used to obtain normal text anchors, clutter text anchors, and leakage text anchors. During the training phase, a three-class anchor layer-level decoupling loss is used to constrain the geometric relationship of the three sets of text anchors. The constraint logic of the three-class anchor layer-level decoupling loss includes: The first layer of high-priority constraints makes the cosine similarity between the leaked text anchor and the normal text anchor and the clutter text anchor approach 0, thus separating the leaked semantics from the non-defect semantics; the second layer of low-priority constraints makes the cosine similarity between the normal text anchor and the clutter text anchor approach 1, thus classifying the structural clutter semantics into the normal cluster.

[0013] Optionally, the three types of anchor layer decoupling losses satisfy:

[0014] in, This represents the loss of three types of anchor layer decoupling. , , These represent leaky text anchors, normal text anchors, and noise text anchors, respectively. The vector dot product is represented by the L2 normalization of all three anchor points, making it equivalent to the cosine similarity. This represents the weighting coefficients of the two constraints.

[0015] Optionally, the detection network is trained using a two-stage, curriculum-based training strategy, including: Before training, the domain difference between the text branch and the visual branch relative to the original pre-training distribution is calculated separately. If the text domain difference is greater than the visual domain difference, the visual branch parameters are frozen in the first stage, and only the text-side clutter gating adapter is trained. In the second stage, the text branch parameters are frozen, and the visual-side clutter gating adapter and the multi-scale feature projector are trained. If the visual domain difference is greater than the text domain difference, the visual branch is corrected first, and then the text branch is corrected. The text domain difference is the Euclidean distance between the average embedding of the training text prompt words obtained by the original text encoder and the average embedding of the pre-trained natural language prompt word template pool. The visual domain difference is the Euclidean distance between the average embedding of the training set images obtained by the original visual encoder and the average embedding of the pre-trained natural image reference set.

[0016] Optionally, the attenuation based on the position-by-position clutter confidence includes the following steps: The cosine similarity between the multi-scale patch features and the three sets of text anchors is calculated to obtain a three-channel score map. The three-channel score map is then fused into a single-channel anomaly score map according to a preset fusion formula. The single-channel anomaly score map is then attenuated position by position using the position-by-position clutter confidence to obtain the final anomaly score map.

[0017] Optionally, the preset fusion formula satisfies:

[0018]

[0019] in, For the decayed outlier scores, This represents the clutter confidence attenuation coefficient. Indicates position Clutter confidence level at the location This represents the single-channel anomaly score obtained through fusion. Indicates position The cosine similarity score for the leakage anchor points. Indicates position The cosine similarity score relative to normal anchor points. Indicates position The cosine similarity score of the clutter anchor point.

[0020] Secondly, to efficiently execute the visual language-based shield tunnel leakage identification method provided by this invention, this invention also provides a visual language-based shield tunnel leakage identification system, comprising: an input device, an output device, a processor, and a memory, wherein the input device, output device, processor, and memory are interconnected, and the memory stores program instructions for the visual language-based shield tunnel leakage identification method. This visual language-based shield tunnel leakage identification system of this invention has a compact structure and stable performance, and can stably execute the visual language-based shield tunnel leakage identification method provided by this invention, further enhancing the overall applicability and practical application capability of this invention.

[0021] This invention introduces a clutter-gated adapter in collaboration between the visual and text encoding ends, and constructs three types of anchor-layer decoupling losses to semantically classify structural backgrounds such as bolts, pipeline joints, markings, and surface undulations on the surface of shield tunnel segments. This maintains stable discrimination capability even when facing visually significant non-defect areas, effectively reducing the false detection rate against complex lining backgrounds. A multi-scale feature projector is introduced in the middle of the visual encoder, with fixed fusion weights pre-determined according to the scale distribution of the leakage annotation area in the training set. This allows shallow, small receptive field features and deep, large receptive field features to each serve leakage prediction at their corresponding scales, balancing small-scale wet stains near bolt holes and large-scale watermarks along assembly joints, avoiding unstable scale weights learned under small sample conditions. Furthermore, a normal basis anchoring attention is deeply embedded in the visual encoder, replacing the original query-key similarity reference system with a normal basis prototype set obtained from offline clustering. This allows leakage patches deviating from the normal basis to naturally stand out on the attention map without introducing new trainable parameters, thereby suppressing the over-response of the original attention mechanism to structural backgrounds.

[0022] This invention employs a course-based two-stage training strategy, prioritizing the correction of branches with large domain differences to improve the training stability and final accuracy of the model in the industrial grayscale image domain. Meanwhile, the network only needs to be trained on lightweight parameters such as adapters, projectors, and template libraries, making it easy to deploy on common industrial cameras and GPU platforms, thus demonstrating good implementability and scalability. Attached Figure Description

[0023] Figure 1 A flowchart of a method for identifying water leakage in shield tunnels based on visual language, provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a water leakage detection result provided in an embodiment of the present invention; Figure 3 This is a framework diagram of a shield tunnel leakage identification system based on visual language, provided for an embodiment of the present invention. Detailed Implementation

[0024] Specific embodiments of the present invention will now be described in detail. It should be noted that the embodiments described herein are for illustrative purposes only and are not intended to limit the invention. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that these specific details are not necessary to practice the invention. In other instances, well-known circuits, software, or methods have not been specifically described to avoid obscuring the invention.

[0025] Throughout this specification, references to "an embodiment," "an embodiment," "an example," or "an example" mean that a particular feature, structure, or characteristic described in connection with that embodiment or example is included in at least one embodiment of the invention. Therefore, the phrases "in an embodiment," "in an embodiment," "an example," or "an example" appearing in various places throughout the specification do not necessarily refer to the same embodiment or example. Furthermore, specific features, structures, or characteristics can be combined in one or more embodiments or examples in any suitable combination and / or sub-combination. Moreover, those skilled in the art will understand that the illustrations provided herein are for illustrative purposes and are not necessarily drawn to scale.

[0026] Please see Figure 1 This invention provides a method for identifying water leakage in shield tunnels based on visual language, comprising the following steps: S1. Obtain images of the shield tunnel segments to be detected and input them into the detection network built based on the contrastive language-image pre-trained model.

[0027] In this embodiment, a shield tunnel inspection vehicle is used to collect images of the tunnel segment surface inside the tunnel. The acquisition device is equipped with an industrial camera. The original resolution of the images acquired in this embodiment is no less than 518×518 pixels, covering on-site scenes under different lighting conditions, different tunnel segment splicing seam shapes, and different environmental noise conditions.

[0028] Furthermore, images of pipe segments containing manually identifiable leaks and normal pipe segments are selected from the collected images. The leak images include leak samples with different shapes and color depths, as well as leak samples under complex background conditions such as different lighting, different pipe segment splices, and different pre-buried holes. Then, the above images are divided into training set and test set according to a preset ratio. The specific division ratio can be flexibly set according to actual engineering needs and does not constitute a limitation on the scope of protection of this invention.

[0029] Based on this, the selected shield tunnel segment images are classified and stored as follows: normal segment images are stored in the normal sample directory, and leakage images are stored in the leakage sample directory.

[0030] The edges of the leakage areas in the selected leakage images are labeled with polygons using an image annotation tool. The annotation results are then converted into real mask images of the leakage images and stored in the mask image directory. The training set and the test set are then converted into structured annotation files that can be read by the model, thus constructing an auxiliary dataset for leakage of shield tunnel segments.

[0031] In this embodiment, the structured annotation file includes the path, mask path, anomaly label (0 for normal, 1 for anomaly), category name, and image height and width data of the shield tunnel segment image; normal images do not contain mask paths, and the training set and test set are written into structured annotation files with different filenames.

[0032] The following principles are followed during the annotation process to ensure the training value of the auxiliary dataset for the noisy anomaly detection task: First, structural backgrounds such as bolts, pipeline joints, engraved numbers, and surface undulations that objectively exist on the surface of shield tunnel segments are strictly excluded from the leakage mask during annotation, so that the model can see visually significant but semantically normal clutter samples during the training phase. Secondly, for small-scale wet stains with similar hues and blurred boundaries, as well as transition areas of large watermarks, the outer contours are reasonably fitted based on the polygon vertices, and the mask is slightly smoothed at the edges to reduce pixel-level label noise. Third, it provides a basis for the offline statistics of subsequent scale prior weights, and the pixel area corresponding to each leakage polygon is synchronously recorded in the structured annotation file.

[0033] It should be noted that the original contrastive language-image pre-trained model consists of a text encoder and a visual encoder. The text encoder is based on the Transformer module, and the visual encoder is based on the ViT module. It is a pre-trained multimodal model for multimodal image-text data in contrastive learning. Due to the domain specialization of shield tunnel scenarios, the pre-trained model is difficult to directly generate high-precision anomaly perception capabilities for shield tunnel segment leakage during detection. Therefore, this embodiment needs to construct an auxiliary dataset on shield tunnel leakage to enable the model to detect and identify shield tunnel leakage more accurately.

[0034] In another embodiment, a detection network based on a contrastive language-image pre-trained model is constructed, including a clutter-gated adapter, a multi-scale feature projector, and a normal basis anchoring attention module, as detailed below: The left side represents the text encoder branch. Three sets of text prompts are tokenized and embedded, then sequentially passed through Transformer Blocks 1-3. A clutter-gated adapter is connected in series at the output of each Transformer Block. That is, Transformer Blocks and clutter-gated adapters are stacked alternately, and the output features of the clutter-gated adapters replace the original Block outputs in a substitution manner before being fed into the next layer, for a total of K stacks. T =3 layers; thereafter, the features flow sequentially through the 4th to 12th Transformer blocks, LayerNorm, Take [EOT] position, and Multilayer Perceptron (MLP) to output normal text anchor points. Clutter text anchor points With leaked text anchors .

[0035] The right side is the ViT Module branch. The input image is patch-embedded and then sequentially passed through ViT Blocks 1 to 8. Similarly, clutter gating adapters are connected in series at the output position of each ViT Block in a replacement manner, stacked in K layers. L =8 layers; then flows through the 9th to 24th ViT blocks (ViT Blocks 9~24), and multi-scale feature projectors and normal basal anchoring attention are deployed in the middle and deep layers of the visual encoder, respectively, to output patch-level visual features. With global visual features .

[0036] The bottom section shows feature fusion and discrimination: text anchor points { , , } and visual features { , First, cosine similarity is calculated, then clutter-gated score is applied and channel-wise softmax is applied to obtain a 3-channel score map.

[0037] The detection network in this embodiment is built on the original contrastive language-image pre-trained model: in the shallow layer (first K) of the text encoder TA clutter-gated adapter is introduced in the shallow layer of the visual encoder, and the geometric relationships of the three sets of text anchor points (normal, clutter, and leakage) are constrained by combining three types of anchor layer decoupling losses; in the shallow layer of the visual encoder (the first K layers)... L A clutter-gated adapter is introduced in the layer; a multi-scale feature projector is deployed in the middle of the visual encoder; and normal substrate anchor attention is embedded deep in the visual encoder.

[0038] The three types of anchor layer decoupling losses apply to normal text anchors in the L2-normalized text embedding space (i.e., the unit hypersphere). Clutter text anchor points Leaking text anchors The geometric relationships are constrained by two sub-constraints with priority: Item 1 (high priority) uses leakage anchors Compared with normal anchors clutter anchor The goal is to make the cosine similarity approach 0; item 2 (low priority) uses normal anchors. With clutter anchor The goal is to make the cosine similarity approach 1. This is achieved through weighting. (This embodiment) =0.1、 =0.05), so that the hierarchical priority of "first ensuring that abnormalities and non-defects can be separated, and then refining the internal attribution of non-defects" is explicitly reflected in the loss function.

[0039] This creates a geometric structure in the text embedding space where "leaking anchors are far from non-defect anchor clusters, and clutter anchors and normal anchors are grouped into the same non-defect cluster," allowing structural background clutter to be semantically classified into the normal side and remain separable from leakage semantics.

[0040] This embodiment addresses the decoupling loss through three types of anchor layering. , , The geometric relationships of the three sets of text anchor points are constrained. The design motivation is that although the structural background (bolts, pipeline joints, engraving numbers, surface undulations, etc.) in the shield tunnel segment image is visually similar to the leakage area, it belongs to the non-defect category at the semantic level and should be on the same side as the normal and clean area and on the opposite side of the leakage abnormal area. If only the alignment of normal and abnormal anchor points is used, the clutter text does not have an explicit classification in the text embedding space and is easily pushed to the abnormal side in the cosine similarity space, thereby amplifying the false detection response in pixel-level prediction.

[0041] To this end, this embodiment explicitly introduces three types of anchors in the text embedding space and performs hierarchical decoupling constraints on their geometric relationships: the first item enables leakage anchor points. With normal anchor point Clutter anchor points Both are widened, forcing anomalous semantics to remain separable from all non-defective semantics; the second term makes normal anchor points With clutter anchor points Zooming in, structural background clutter is relegated to the normal side of the text embedding space, satisfying:

[0042] in, This represents the loss of three types of anchor layer decoupling. , , These represent leaky text anchors, normal text anchors, and noise text anchors, respectively. The vector dot product is represented by the L2 normalization of all three anchor points, making it equivalent to the cosine similarity. This represents the weighting coefficients of the two constraints. In this embodiment, they are respectively taken as... =0.1、 =0.05; Strictly greater than This reflects the hierarchical priority of "first ensuring that abnormalities and non-defects can be distinguished, and then refining the internal attribution of non-defects".

[0043] Understandable, minimizing Equivalent to simultaneously reducing and( , The correlation of ) and improvement and The correlation between these factors forms a geometric structure in the text embedding space where "leaky anchors are far from non-defective anchor clusters, while normal anchors and clutter anchors belong to the same cluster." Under this geometric structure, when calculating the cosine similarity between any detectable patch feature and the three groups of anchors, the structural background patch... and The response is naturally close to, to The response naturally decreases, making it less likely for clutter areas to be mistakenly pushed to the leakage side during the three-channel score map fusion stage, effectively reducing pixel-level false detection response in shield tunnel scenarios.

[0044] S2. The visual features extracted by the detection network are domain adapted through a clutter gating adapter to generate position-wise clutter confidence, so that visually significant but semantically normal clutter regions bypass the domain adaptation transformation.

[0045] In this embodiment, the internal connection relationships of the clutter gating adapter are as follows: The input feature x is assigned and simultaneously fed into three parallel branches: Branch A (Template Match) sequentially performs L2 normalization, cosine similarity with the clutter template library (Cosine Sim with P), and maximum over K along K templates, outputting the clutter confidence g at each position; where the clutter template library (Clutter Template Library P, shape [K,D], K=32) is used as a side input to Cosine Sim with P.

[0046] Branch B (Adaptive Transform) sequentially performs L2 normalization (L2 Norm) and linear transformation (Linear W). n It uses GELU nonlinear activation and learnable scaling (Scale α·(·)) to output adaptive features a(x).

[0047] Branch C (Branch C: Bypass) will bypass input x as is.

[0048] The output g of the three branches Merge with x into a convex combination node (Convex Combination), according to Calculate and obtain the output features (Output y).

[0049] The clutter gating adapter described in this embodiment is deployed in a collaborative manner at the shallow layer of the text encoder and the visual encoder. The design motivation is that: in the image of shield tunnel segments, visually prominent structural backgrounds such as bolts, pipeline joints, engraving numbers, and surface undulations often present feature distributions that are similar to leakage areas in the shallow semantic space of the native visual language model; if the same set of industrial domain adaptation transformations is applied to all tile features indiscriminately, content that belongs to normal semantics will be pushed into the abnormal subspace, causing more false detections.

[0050] To this end, this embodiment introduces an explicit clutter prior in the shallow layer, and adaptively judges the degree of clutter affiliation for each input feature point: weakens the adaptation for positions with high clutter tendency and strengthens the adaptation for positions with low clutter tendency, thereby achieving collaborative clutter suppression between the visual and text ends at the source.

[0051] Furthermore, the clutter-gated adapter uses input characteristics As input, maintain a clutter template library internally. It is equipped with an independent adaptation transformation branch, and the whole is composed of two parallel branches: the "template library matching branch" and the "adaptation transformation branch". Finally, the output features are obtained through "convex combination fusion". The template matching branch sequentially performs L2 normalization → row-by-row cosine similarity with the template library → Max over K → outputs clutter confidence. The adaptation transformation branch sequentially performs L2 normalization → linear mapping. →GELU nonlinearity→ learnable scaling → Output adaptation features .

[0052] In the convex combination fusion stage, the clutter confidence level g is used to evaluate the original input features. (Directly fed into the fusion node via bypass) and adaptation features Perform a weighted summation to obtain the final output. .

[0053] The clutter template library P is obtained by performing a K-means clustering (K=32) on the shallow patch features of all normal samples in the training set before training, and is slowly updated during training using an exponential moving average with momentum m=0.99. The clutter-gated adapter is used in the first K steps of the text encoder. T =3 Transformer block output positions and the K-th layer of the visual encoder L =8 layers of ViT block output positions are inserted separately in an inter-layer stacking manner, with template library P and weights for different layers. Independent of each other, output features It is directly fed into the next Transformer / ViT block.

[0054] For template matching, the clutter template library It is obtained by performing a K-means clustering on the features of all normal sample patches in the training set before training, denoted as The shape is K×D, representing K typical structural background templates (such as the centers of subclusters of bolts, pipeline joints, markings, surface undulations, shadows, etc.) of shield tunnel segments in the shallow semantic space. For each input feature vector Combine it with the clutter template library For each template, calculate the cosine similarity and take the maximum value as the clutter confidence level at that location. The calculation method is as follows:

[0055] In the formula, The first clutter template library There are 1 cluster center with shape [D]. This represents the cosine similarity operation, where K represents the number of cluster centers in the clutter sample library. In this embodiment, K=32. This indicates the degree to which the feature at that location belongs to the clutter subspace. The larger the value, the closer the location is to a certain type of structured background clutter.

[0056] Clutter Template Library The estimation is completed in one go during the initialization phase and slowly updated during training using an exponential moving average (EMA). The update momentum is set to 0.99 to avoid training instability caused by drastic template drift.

[0057] For the adaptation transformation, in another branch running parallel to template library matching, the input features are... First, L2 normalization is performed to eliminate amplitude differences, followed by a trainable linear transformation. With nonlinear activation Get industrial domain adapted output The calculation method is as follows:

[0058] In the formula, This represents the L2 normalization operation. This embodiment uses GELU to represent a non-linear activation function. This represents the trainable weight matrix corresponding to the nth layer clutter-gated adapter at the insertion position, with a shape of D×D. This is a learnable overall scaling factor used to stabilize the feature scale in the early stages of training, preventing the adapted output from exceeding the norm distribution of the original CLIP features. The industrial domain semantic bias introduced through this branch enhances the model's ability to perceive the specific semantic category of water leakage in shield tunnels.

[0059] For gated fusion, after obtaining the clutter confidence level... With adapter output Subsequently, the clutter-gated adapter... As a gating signal, for the original features With adapter output Perform a convex combination to obtain the final output. The calculation method is as follows:

[0060] In the formula, The output feature after gated fusion has a shape similar to the input feature. Maintain consistency. It should be understood that when the clutter confidence level... When it is close to 1, the output is... Almost all input features are preserved As is, visually significant but semantically normal clutter regions are explicitly bypassed during the adaptation phase; when When it is close to 0, the output Almost equivalent to adapting output This allows for the injection of industrial domain anomaly detection capabilities into potential leakage areas. It enables bypassing cluttered areas and targeted adaptation to potentially abnormal areas, avoiding the misalignment of normal to abnormal states caused by traditional adapters applying the same transformation to all locations.

[0061] For inter-layer stacking, this embodiment respectively uses the first K layers of the text encoder. T Layers and the front K of the visual encoder L A clutter-gated adapter is inserted at the output position of the layer Transformer block, where K T K L In this embodiment, K is chosen as the preset number of shallow layers. T =3、K L =8; Clutter template library at different levels With trainable weights They are independent of each other, capturing the clutter distribution at different semantic depths on the text and visual ends respectively, thus forming a clutter suppression structure that is from shallow to deep and coordinated between text and vision.

[0062] After the feature vector traverses all Transformer blocks, the feature shape is transposed to [b, L, 768]. After layer normalization, the [EOT] token position vector is extracted, and then the text feature vector is mapped out through a learnable projection layer with a shape of [b, 768]. The text feature vectors output by the three sets of prompt words are averaged to obtain the normal text anchor points. Clutter text anchor points With leaked text anchors .

[0063] S3. Using a multi-scale feature projector, shallow and deep visual features of different receptive fields are fused according to the prior weighting of the training set leakage scale to obtain multi-scale patch features.

[0064] In one embodiment, in the visual encoding part, an input image with 518×518 pixels is processed by a conv1 convolutional layer with stride P=14 to embed patches and obtain a feature map. After flattening, a sequence of shape [b, N, 1024] is obtained (N is the number of patches, N=37×37=1369). A learnable class token is embedded at the beginning of the sequence and positional encoding is added, and the sequence is transposed into a sequence of shape [N+1, b, 1024]. Subsequently, it passes through 24 ViT blocks sequentially, and the first K... L =The output position of the 8-layer ViT block is inserted into the visual clutter gating adapter.

[0065] In the visual encoder, shallow feature groups (taken from layers 6 and 12, corresponding to small effective receptive fields, sensitive to small-scale wet stains) and deep feature groups (taken from layers 18 and 24, corresponding to large effective receptive fields, sensitive to large-scale watermarks) are extracted respectively.

[0066] In this embodiment, the multi-scale feature projector is deployed in the middle of the visual encoder to address the problem of large scale spans in water leakage areas of shield tunnels, where a single receptive field cannot simultaneously capture both small-scale wet stains and large-scale watermarks. The design motivation is that the water leakage patterns in shield tunnel operation scenarios include both small-scale anomalies such as star-shaped wet stains near bolt holes and thin, elongated wet bands on both sides of pipeline joints, and large-scale anomalies such as large-scale watermarks along assembly joints and continuous wet areas. If only features at a fixed depth of the visual encoder are used for prediction, shallow features have high spatial resolution but a small effective receptive field, are sensitive to small-scale wet stains but lack global semantics; deep features have strong semantic abstraction capabilities and a large effective receptive field, are sensitive to large-scale watermarks but have low spatial resolution. Therefore, this embodiment establishes a dual-projection structure in the middle of the visual encoder, consisting of shallow small-scale branches and deep large-scale branches, and pre-determines fixed fusion weights according to the scale distribution of the leakage labeled area in the training set, achieving scale division and stable fusion.

[0067] The multi-scale feature projector consists of three branches: The small-scale branch takes the outputs of layers 6 and 12 (ViT L6, ViT L12) of the visual encoder, concatenates them along the channel dimension, and then projects them through a small-scale projection head. (1024→768) and L2 normalization (L2Norm) are used to obtain small-scale patch features. The large-scale branch takes the outputs from layers 18 and 24 (ViT L18, ViT L24), and also concatenates them using a large-scale projection head. (1024→768) and L2 normalization are used to obtain large-scale patch features. , and According to the scale prior weight at the weighted fusion node =0.4、 Summing by 0.6 yields the final tile-level features. .

[0068] The global branch takes the last ViT layer of the visual encoder, and performs global average pooling and projection (GAP+). Then perform L2 normalization to obtain the global image visual features. The features, along with the tile-level features, are then fed into the subsequent cosine similarity calculation.

[0069] The overall processing of the multi-scale feature projector includes four sub-operations: "shallow branch extraction", "deep branch extraction", "channel alignment and normalization", and "prior weighted fusion".

[0070] For shallow branch extraction, the outputs of the ViT blocks at layers 6 and 12 of the visual encoder are taken as the shallow feature groups. The images are then concatenated along the channel dimension, resulting in a shape of [N, b, 1024]. After removing the class token, the corresponding patch features are retained, and the dimensions are rearranged to obtain [b, N, 1024], which is then fed into the small-scale branch projection head. The Composed of a linear mapping layer and an L2 normalization layer, it maps 1024-dimensional shallow features to a 768-dimensional shared semantic space, resulting in a small-scale branch of tile-level feature vectors. The shape is [b, N, 768], which mainly carries the geometric clues for small-scale wet stains.

[0071] For deep branch extraction, the outputs of the ViT blocks at layers 18 and 24 of the visual encoder are taken as the deep feature groups. The array is then concatenated along the channel dimension, forming a shape of [N, b, 1024]; after removing the class token and rearranging the dimensions, it is fed into the large-scale branch projection head. The and With identical structures but independent parameters, 1024-dimensional deep features are mapped to a 768-dimensional shared semantic space, resulting in large-scale branched tile-level feature vectors. Its shape is [b, N, 768], and it mainly carries the semantic abstraction of large-scale watermarks.

[0072] For channel alignment and normalization and The outputs are all L2 normalized to ensure a consistent norm scale in the shared semantic space, which facilitates cosine similarity comparison with text anchors (which are also L2 normalized 768-dimensional vectors). At the same time, this normalization operation can reduce the relative weight drift caused by the difference in original norms between shallow and deep features.

[0073] For prior-weighted fusion, after aligning the two feature paths, the scale prior weights are applied. , The two feature vectors are weighted and summed to obtain the final tile-level feature vector. ,satisfy:

[0074] In the formula, , These are the patch feature tensors for the shallow group (layers 6 and 12) and the deep group (layers 18 and 24), respectively. , These are trainable linear projection heads for small-scale and large-scale branches, respectively. , The fixed fusion weights are predetermined after offline statistical analysis of the leakage labeled area scale distribution, and... In this embodiment, after performing area statistics on all leakage mask annotations in the training set, the proportion of small-scale leakage samples (annotated area less than the threshold A0) is approximately 0.4%, and the proportion of large-scale leakage samples is approximately 0.6%. Accordingly, [the following is taken as an example]. =0.4、 =0.6; the threshold A0 is determined by taking a value near the median of the leakage labeled area in the training set.

[0075] It should be noted that, , The reason for using pre-determined fixed values ​​from offline statistics instead of learnable parameters is that the training sample size for shield tunnel water leakage tasks is relatively limited, and learnable scale fusion weights are prone to degradation or unstable convergence under small sample conditions; while the scale prior weights obtained from offline statistics directly reflect the distribution characteristics of the target leakage area on the pixel area, and have clear engineering physical meaning, which can stably guide the scale division of shallow and deep dual-path features without introducing additional training parameters.

[0076] On the other hand, the features from the last layer of the visual encoder are taken, subjected to global average pooling, projected onto a 768-dimensional shared semantic space, and then subjected to L2 normalization to obtain the global image visual feature vector. The shape is [b, 768], and finally the tile-level feature vectors are aligned. Global image feature vector The dimension of the text anchor vector facilitates subsequent cosine similarity calculation.

[0077] S4. Based on the normal basis anchor attention, the multi-scale patch features are weighted by attention, and the normal basis prototype set obtained by offline clustering is used as a reference to highlight the leakage areas that deviate from the normal distribution.

[0078] This embodiment embeds a normal base anchoring attention mechanism deep into the visual encoder to address the problem that the native visual language model's attention mechanism tends to focus on salient objects and over-respond to structural backgrounds in industrial scenarios. The design motivation is that: the original multi-head attention measures the coupling strength by the query-key inner product between pairs of image tiles, which can guide the model to focus on semantically salient regions in natural images; however, in shield tunnel segment images, structural backgrounds such as bolts and engraved numbers are visually salient objects, which often receive high responses on the original attention map and are thus misjudged as anomalies; while the real leakage areas, because their color tone is similar to the background and their spatial distribution is diffuse, actually have a lower response on the original attention map.

[0079] To address this, this embodiment introduces an external prior of "normal basis anchoring": the normal basis prototype set obtained by clustering the features of normal sample patches in the training set is used as a unified reference system for attention calculation, so that the coupling relationship of all patches is determined by the external metric of "similarity with the normal basis", rather than by the inner product of each pair of patches; patches falling on the normal basis are stably coupled with each other with high weights and are regarded as background by the attention mechanism; leakage patches that deviate from the normal basis have lower coupling weights to all patches, and naturally highlight abnormal regions on the attention map.

[0080] The normal basis anchoring attention takes the input features of the i-th and j-th deep patches of the visual encoder as its object, and while preserving the original multi-head attention Q / K / V linear mapping, it transforms the original multi-head attention... The query-key inner product in the code is uniformly replaced with the normal base prototype set. It is the inner product of the similarity vectors of the reference frames.

[0081] Specifically, for the value vector of each deep map patch First, with the normal base prototype set Each row of (M=64) Calculate the cosine similarity to obtain a similarity vector of length M. Coupling weights of any two tiles Depend on According to line Normalization yields the value vector, which is then used to apply the normalization method. Weighted summation yields the anchored attention output. .

[0082] Normal base prototype set In the model initialization phase, the normal samples in the training set are obtained by performing an offline K-means clustering operation on the deep patch features from the original ViT forward-backward direction. This clustering remains fixed during the training and inference phases, without introducing new trainable parameters. The original multi-head attention uses the QK inner product to measure coupling, and VV self-attention (DPAM) uses the VV inner product to measure coupling. In this invention, the normal basis anchoring attention is uniformly measured using "patch pair". Using the membership degree as a coupling reference system, leakage patches that deviate from the normal base naturally stand out on the attention map.

[0083] The overall processing of normal base anchoring attention includes three sub-operations: "offline prototype estimation", "tile-by-tile similarity vector", and "anchoring attention calculation".

[0084] For offline prototype estimation, during the model initialization phase, all normal samples in the training set are fed forward through the original visual encoder (excluding the visual clutter gate adapter) to obtain the deep patch feature tensor of the visual encoder. After flattening it according to the patch dimension, an offline K-means clustering is performed to obtain M=64 cluster centers, which constitute the normal basis prototype set. The shape is M×D, representing the M typical normal sub-distributions of shield tunnel segments in the deep semantic space of the visual encoder. Prototype set The estimation is completed once during the initialization phase and remains fixed during the training and inference phases, without introducing new trainable parameters.

[0085] For the per-patch similarity vector, in the deeper layers of the visual encoder (after layer 20 in this embodiment), the query for each patch is first obtained using the original multi-head attention method. ,key ,value Vector; the value vector for each tile , respectively with the normal base prototype set Calculate the cosine similarity of each row of prototypes to obtain a similarity vector of length M. ,satisfy:

[0086] In the formula, Let be the value vector of the i-th deep patch of the visual encoder, with shape [D]. Let be the k-th row prototype of the normal basis prototype set R, with shape [D], and cos(·,·) be the cosine similarity operation. Let be the row-wise similarity vector of the i-th tile to the normal basis prototype set, with shape [M].

[0087] For a patch that falls near a certain normal sub-distribution The values ​​at the corresponding positions are relatively large, and the overall distribution shows a sharp peak; for leakage patches that deviate from all normal sub-distributions, The overall values ​​are small and the shape is flat.

[0088] The anchor attention calculation includes calculating the similarity vector between every two tiles. and Do the inner product and press Scaling, then Normalization yields the coupling weights for normal basis anchored attention. And use this to apply to the value vector We perform a weighted summation to obtain the final attention output.

[0089] In contrast, the coupling weights of the original attention... For query AND key inner product and according to After scaling The expressions for both are as follows:

[0090]

[0091]

[0092] In the formula, , , , These represent the deepest layers of the visual encoder. , The query, key, and value vectors for each map patch are defined, where d is the feature dimension of the query / key, corresponding to the scaling factor of the original attention, and M is the number of normal basis prototypes, corresponding to the scaling factor of the normal basis anchored attention. In this embodiment, M=64. For row-normalized functions, The first of the original attention Output vectors The first step in anchoring attention to the normal basal body output vectors, ( , )∈(1, H×W), where H×W is the spatial size of the deep feature map of the visual encoder.

[0093] It should be noted that the original basis Internal similarity replacement based on External similarity, that is, simultaneously... and The coupling relationship is projected onto a unified external dimension of "proximity to the normal substrate": the coupling weight between two patches is only large when they have similar membership patterns on the M normal sub-distributions; for leakage patches that deviate from all normal sub-distributions, their... The overall value is low, compared to any tile. The coupling weights obtained by the inner product are relatively small, thus appearing prominent in the attention map. Therefore, without introducing new trainable parameters, leakage patches in the shield tunnel scenario are naturally highlighted, while structural background patches are stably suppressed.

[0094] S5. Based on the cosine similarity between the multi-scale tile features and the three preset sets of text anchor points, and combined with the position-by-position clutter confidence, the leakage detection result is obtained.

[0095] In this embodiment, the detection network is trained using a two-stage, course-based training strategy to obtain the trained detection network. The training in this embodiment is conducted within the PyCharm integrated environment, Conda environment management, and the PyTorch deep learning framework. The experimental platform is based on Windows 11 Professional operating system, an NVIDIA GeForce RTX-3090 GPU with 24GB of video memory, and a 2.4GHz Intel(R) Xeon(R) Platinum 8351N CPU.

[0096] Before training, load the pre-trained OpenCLIP ViT-L-14-336px weights and store them in the model folder; when training the detection network starts, the command line will output the message "Loading pretrained ViT-L-14-336".

[0097] In this embodiment, the training iterations for the text encoding part are 5 times, and the learning rate is 1×10. -5 Batch size is 8, text-side clutter gating adapter layer number K T =3; the visual encoding part has 20 iterations and a learning rate of 5×10. -4 Batch size is 2, number of layers K of the visual clutter gating adapter L =8; Clutter template cluster center number K=32, template update momentum is 0.99; Normal basis prototype number M=64; Scale prior fusion weights =0.4、 =0.6; Weights of three types of anchor decoupling losses =0.1、 =0.05; Clutter confidence attenuation coefficient =0.7; the total weight η of the three types of anchor decoupling losses is 0.1.

[0098] The course-based two-stage training uses an auxiliary dataset and a pre-trained CLIP as input. First, before training begins, the text-end-domain differences are calculated offline once. Difference with visual end domain , and then and The relative size determination has two stages, and the order of these stages is as follows: If > If the text side is corrected first (as in this embodiment), then the visual side is corrected first.

[0099] The course-based two-stage training strategy designed in this embodiment aims to solve the problem of distribution differences between the industrial grayscale image domain and the natural image domain used for pre-training the visual language model, and the problem that simple fine-tuning can easily destroy the model's prior knowledge. Its core idea is: before training begins, the domain difference between the text branch and the visual branch relative to the original pre-training distribution is quantified offline. Then, the training courses are arranged according to the priority of "correcting the one with the larger domain difference first", so that the model gradually approaches the target industrial domain in the order of the largest deviation to the second largest deviation, avoiding the drift of the contrast anchor point and training instability caused by the simultaneous deviation of the visual and text ends.

[0100] The specific domain difference measurement process includes three sub-operations: "text-side embedding estimation", "visual-side embedding estimation", and "Euclidean distance quantization", which are calculated as follows:

[0101]

[0102] In the formula, This represents the average embedding vector obtained by passing all training text prompts (i.e., normal, noise, and leakage prompt templates) through the original unadapted text encoder, with the shape

[768] . represents the average embedding vector of the original model on the natural language prompt word template pool, with shape

[768] , reflecting the center of the text distribution during the pre-training stage; This represents the average embedding of global image features obtained by passing all images in the training set through the original unadapted visual encoder, with shape

[768] ; represents the average embedding of global image features of the original model on a natural image reference set, with shape

[768] ; , These represent the domain difference measures of the text branch and the visual branch relative to the original pre-trained distribution, respectively. This represents the L2 norm.

[0103] This embodiment was obtained through actual measurement on a self-built shield tunnel industrial grayscale map. Approximately 0.42 Approximately 0.18, Greater than The reason for this is that the text prompts related to shield tunnels (including terms such as "shield tunnel segments," "assembly joints," and "water leakage") deviate more significantly from the distribution of natural language templates during the pre-training stage; while the image branches, due to the large-scale pre-training of the visual encoder on natural images, still possess a certain robustness in the feature abstraction of industrial grayscale images, resulting in relatively smaller visual domain differences. Therefore, the training course is determined to first correct the text branches with larger domain differences, and then correct the visual branches with relatively smaller domain differences.

[0104] In this embodiment, the first stage is the text branch correction stage. In this stage, all parameters of the visual encoder (including the visual clutter gating adapter, multi-scale feature projector, and normal basis anchoring attention) are frozen, and only the text clutter gating adapter (containing trainable weights) is trained. Scaling factor The exponential moving average update of the text-side clutter template library; the visual branch uses a visual encoder with embedded normal basis anchored attention as the frozen teacher, and its output tile-level visual features. With global image features As a monitoring anchor.

[0105] To enhance the perception of global image information, this embodiment further superimposes the global image features (i.e., class token vectors) output by the visual encoder onto the tile-level features in an additive manner. Above this, the cosine similarity of three sets of text anchor points is calculated. Through this global-local fusion approach, the text side can perceive both fine-grained tile differences and coarse-grained global image context during the correction process, thereby obtaining a more stable text anchor point geometric structure.

[0106] The loss items in this stage are segmented losses. and the three types of anchor layer decoupling loss Weighted sum: A weighted combination of Dice loss and Focal loss is used to perform pixel-level supervision on the alignment of text anchors and tile features; right , , The geometric relationships of the three sets of text anchor points are constrained. This stage uses the Adam optimizer with a learning rate of 1×10⁻⁶. -5 The batch size is 8, the number of iterations is 5, the trainable parameters of the text are updated in reverse, and the weights of the text are saved in each iteration for loading in the subsequent inference stage.

[0107] The second stage is the visual branch correction stage. This stage fixes the text encoder, text-end clutter gating adapter, and three sets of text anchors obtained in the first stage. , , Input the constructed auxiliary dataset, and obtain patch-level features through the visual encoding part. Global image features And the clutter confidence level of the visual clutter gating adapter for each tile output. .

[0108] This stage involves two scale branches (small-scale branch) Large-scale branching ) Calculate and leak text anchor points respectively The cosine similarity is used to obtain the corresponding pixel-level leakage prediction. and Both are calculated based on clutter confidence levels. After positional attenuation, the segmentation loss is calculated by summing the results with the ground truth mask S; global image leakage prediction. Then based on global image features and The cosine similarity is obtained, and through The loss is compared with the image labels to serve as the image-level classification loss. .

[0109] This phase utilizes the Adam optimizer and a learning rate of 5×10⁻⁶. -4 Batch size 2, iteration count 20, reverse update of visual clutter gating adapter (including trainable weights) Scaling factor ) and multi-scale feature projectors (including projection heads) , Trainable parameters of ); normal basis prototype set The text parameters and three sets of text anchors remain frozen during this stage. Visual weights are saved in each iteration, and the optimal weights are selected as the final delivery parameters based on the pixel-level AUROC metric on the test set.

[0110] The expressions for the classification loss, segmentation loss, clutter confidence attenuation term, and corresponding cosine similarity calculation during the training process are as follows:

[0111]

[0112]

[0113]

[0114]

[0115]

[0116] In the formula, Represents classification loss. Represents the cross-entropy loss in binary classification (in binary classification problems) Cross-entropy is equivalent. For image-level leakage prediction, Image label (0 for normal, 1 for abnormal); Represents the original segmentation loss, by Loss and Loss-weighted composition, For pixel-level leakage prediction, A pixel-level ground truth mask; This is a pixel-level leakage prediction after the clutter confidence g is attenuated position by position; The clutter confidence attenuation coefficient is taken in this embodiment. =0.7; This represents the segmentation loss after adding clutter confidence attenuation; This indicates the calculation of cosine similarity. For global image visual features; The tile-level visual features are aggregated from a multi-scale feature projector. For leaky text anchor points.

[0117] The two-stage total loss strategy is as follows:

[0118]

[0119] In the formula, The total loss for the first stage is comprised of the segmented loss. Decoupling loss from three types of anchor layers Weighted combination, As the weighting coefficient, this embodiment takes... =0.1; The total loss for the second stage is comprised of categorized losses. Segmentation loss after clutter confidence attenuation of the two prediction branches of the multi-scale feature projector , The sum is obtained.

[0120] During training, the network parameters are optimized end-to-end using the total loss from the two stages mentioned above. The first stage uses the text-end clutter gating adapter and the text-end clutter template library as trainable objects. Backpropagation is performed for the total loss; the second stage freezes the text end, using the visual end clutter gating adapter and multi-scale feature projector as trainable objects. Backpropagation is performed to calculate the total loss.

[0121] Through a two-stage course guided by domain difference measurement, the text side first calculates the text domain difference. Perform thorough corrections to form a stable system. Three-anchor geometry; at the visual end, under the condition of fixed text anchor points, further based on visual domain difference. Fine-tuning of the clutter-gated adapter and the multi-scale feature projector is performed; their synergistic convergence enables the model to form a contrastive prior in the industrial grayscale domain that can suppress structural background clutter, avoiding contrast anchor point drift and training instability caused by simultaneous shifts at both the visual and text ends.

[0122] This embodiment also requires the design of three sets of text prompt word templates for the model's text encoding part: The first group is used to describe the clean and normal condition of the shield tunnel segments; The second group is used to describe the normal state of clutter, which includes structural backgrounds such as bolts, pipeline joints, engraved numbers, and surface undulations, but does not contain water leakage. The third group is used to describe abnormal water leakage conditions.

[0123] In the three sets of prompt word templates, "{}" are category placeholders. When implementing, they should be filled in with professional category terms in the field of shield tunnels, such as "shield tunnel segments" or "tunnel lining". In this embodiment, "shield tunnel segments" is used as the category description.

[0124] First set (clean and normal) text prompt template: {}; a {}; a {} without leakage; a {} without seepage; a dry {}; a {} without moisture; a {} without water stain; a intact {}; a {} without dampness; a watertight {}. Second group (normal noise) text prompt template: a {} with bolts, without leakage? a {} with pipe joints, without leakage; a {} with engraved markings, without leakage; a {} with surface unevenness, without leakage; a {} with stains, without leakage. Third group (leakage anomaly) text prompt template: a {} with water leakage; a {} with seepage; a {} with water stain; a damp {}; a wet {}; a leaking {}; a {} with water infiltration. The text encoding part segments the three sets of input text prompts into token sequences of shape [b, L] (where b is the batch size and L is the sequence length, defaulting to 77). The token_embedding layer then maps the token sequence into a 768-dimensional embedding vector and adds positional encoding, with a shape of [b, L, 68]. The embedding vector, after dimensionality rearrangement, is input into the Transformer module, passing through 12 Transformer blocks sequentially, and is then processed within the first K... T The output position of the Transformer block is inserted with a clutter-gated adapter to perform industrial domain adaptation of text features.

[0125] After completing network training and obtaining stable and converged model parameters, the detection network is used to perform forward inference on the shield tunnel segment images to be detected, so as to achieve pixel-level leakage location and image-level leakage probability output.

[0126] During the inference phase, the input shield tunnel segment images undergo the same preprocessing operations as in the training phase to ensure the consistency of the input data distribution: the original segment images are scaled to 518×518 pixels with short-side alignment; normalized according to a preset mean and variance; and organized into model input tensors according to a preset channel order. In engineering inspection scenarios with real-time requirements, the input resolution can be appropriately reduced or half-precision numerical representation can be used for inference calculations without reducing recognition accuracy.

[0127] Then, the three sets of text prompts designed are input, and normal text anchors are generated through the trained text encoder and the text-end clutter gating adapter. Clutter text anchor points With leaked text anchors The whole is denoted as a text anchor tensor with shape dimensions [b, 768, 3].

[0128] Input a 518×518 image of a shield tunnel segment to be detected, and obtain patch-level visual features through the trained visual encoding part. The shape dimensions are [b, N, 768]; at the same time, the visual clutter gating adapter outputs the clutter confidence g for each tile, with the shape being [b, N].

[0129] Then, the similarity scores of the two scale branches are output by the multi-scale feature projector, and then... =0.4、 =0.6 weighted fusion yields a three-channel score map of shape [b, N, 3] (corresponding to the normal channel, clutter channel, and leakage channel, respectively). The three-channel score map reflects the cosine similarity response of each patch to the three types of anchor points simultaneously, and is the basis for subsequent anomaly score fusion.

[0130] Furthermore, the three-channel score maps are merged into a single-channel anomaly score map using the following formula:

[0131] In the formula, , , Positions Normal anchor point Clutter anchor points Leakage anchor points The cosine similarity scores (all located in the interval [-1, 1], and mapped to [0, 1] after linear transformation); This represents the single-channel anomaly score obtained by fusion at this location; the denominator on the right is set to 2 to average the "leakage score" and the "reverse contribution of the non-defect score," making... The overall value falls within the [0, 1] interval, which facilitates subsequent joint attenuation with clutter confidence.

[0132] It is understandable that this refers to the location where "the normal channel and the clutter channel have high response at the same time, while the leakage channel has low response" (typical clutter area). The value is relatively low; it applies to locations with "high response in leakage channels and low response in normal and noise channels" (typical leakage areas). The value is relatively high, thus enabling initial semantic suppression of the structural background during the fusion stage.

[0133] Furthermore, the clutter confidence level g output by the visual clutter gating adapter is used to attenuate the anomaly score map position by position:

[0134] In the formula, =0.7 is the clutter confidence attenuation coefficient. This is the score for the attenuated anomalies. It should be understood that the closer the location is to the structural background clutter (…). The larger the value, the greater the attenuation of the leakage response, with a maximum attenuation of 70%; thus, false detection output in the clutter region is further suppressed at the pixel-level output end, and a "semantic-confidence" dual suppression mechanism is formed by integrating with the semantic level.

[0135] Please see Figure 2 , Figure 2 This presentation displays leakage detection results under normal, interfering, and dimly lit environments. The left group shows the original images, the middle group shows the abnormal thermal value maps, and the right group shows the binary masked images. The attenuated anomaly score images are then shown. Spatial smoothing is performed using Gaussian blur, followed by bilinear upsampling to the original image size, resulting in an anomalous thermal value image with shape [b, 1, 518, 518]. Figure 2 The darker the color in the middle group, the higher the probability of water leakage.

[0136] Global image visual features of shape [b, 768] With leaked text anchors Perform cosine similarity calculation to obtain global image leakage prediction. Then, based on the obtained abnormal thermal value images, the maximum abnormal score for each image is extracted. The image is shaped as [b]. The global image prediction and the maximum anomaly score are weighted and fused using equal weighting (each weighted at 0.5) to output the final image-level prediction result:

[0137] In the formula, Based on global image features With leaked text anchors Image-level leakage prediction is obtained by calculating cosine similarity. This represents the global maximum value of the pixel-level anomalous thermal value image; The final output image-level leakage probability is denoted as . The equal-weighted fusion strategy is suitable for industrial scenarios such as shield tunnels, which are characterized by "local small-scale wet stains and large-scale watermarks". By equal-weighted fusion of global semantic judgment and local peak response, it can take into account both the risks of "missing small-scale wet stains" and "falsely reporting large-scale noise".

[0138] To more clearly present the boundaries of abnormal regions and facilitate practical engineering applications, the output abnormal heatmap undergoes shape trimming and normalization. A fixed threshold segmentation method is used (in this embodiment, the threshold is set to 0.5): pixels in the normalized heatmap smaller than the threshold are marked as black (background), and pixels larger than the threshold are marked as white (abnormal regions), resulting in a black-and-white binary mask image, as shown below. Figure 2 As shown in the group on the right.

[0139] The final detection results after post-processing and screening are output in a structured form. Each detection result includes at least: image-level leakage probability, pixel-level abnormal heat map, binary mask image, and the storage path of the corresponding image, which facilitates subsequent engineering inspection record archiving and cross-period disease evolution analysis.

[0140] After completing the aforementioned leakage detection process, the collected shield tunnel segment images are processed according to the method of this invention to obtain detection results with good stability. The processed detection images show that even with structural background elements such as bolts, pipeline joints, markings, and surface undulations on the segment surface, the method of this invention can still output stable leakage probability responses and heatmaps. It can be observed that key leakage areas are continuously highlighted on the heatmap, and the binary mask map closely matches the manually labeled actual leakage areas. Meanwhile, structural background areas such as bolts and markings do not show significant responses on the heatmap, indicating that the clutter gating adapter, the three-type anchor layer decoupling loss, and the normal base anchoring attention have a certain ability to suppress structural background clutter.

[0141] Furthermore, for samples with different lighting conditions, different segment splice morphologies, and different leakage scales (including small-scale wet stains near bolt holes and large-scale watermarks along the splice joints), the predicted boundary of the method of this invention is closer to the actual target boundary. This indicates that the multi-scale feature projector's separate modeling of small-scale wet stains and large-scale watermarks, as well as the two-stage training guided by the domain difference metric, provide stable correction for the industrial grayscale domain, thereby improving the model's detection robustness and positioning accuracy in complex lining backgrounds.

[0142] The overall solution of this invention can be deployed on various hardware platforms such as cloud servers, industrial control PCs, or embedded neural network processors. The input interface receives images of tunnel segments collected by a tunnel boring machine (TBM) inspection vehicle, while the output interface provides pixel-level anomaly heatmaps, image-level leakage probabilities, and binary mask images, thereby supporting subsequent engineering inspections and defect record archiving. When deployed in industrial settings, the trained model can be exported to a model format suitable for inference acceleration (such as ONNX or TensorRT), and further improved inference efficiency can be achieved through optimization techniques such as quantization and tensor fusion. Normal basis prototype set R, clutter template library. With three sets of text anchors , , Each is saved independently in the form of a binary weight file. In the event of communication failure or limited computing power at the front end of the detection vehicle, only the visual encoding part and the three fixed sets of text anchors can be loaded to complete local inference, ensuring the continuous operation of the detection system.

[0143] Furthermore, the clutter-gated adapter, multi-scale feature projector, and weight files of the learnable projection layer obtained after the detection network of this invention completes two-stage training can be delivered independently; the already trained network parameters do not need to be retrained, and leakage detection can be performed on newly acquired shield tunnel segment images. For application scenarios with slight domain shifts, such as cross-working conditions, cross-subway lines, and cross-tunnel cross-sections, only the clutter sample library needs to be updated. Adaptation can be achieved by performing an offline update (i.e., K-means clustering on a small number of normal samples in the new scene), without the need to retrain the learnable weights of the visual clutter gating adapter and the multi-scale feature projector, which facilitates reducing the migration cost for engineering implementation.

[0144] It should be noted that the specific implementation methods described above, such as image processing, numerical simulation, and the construction and training of machine learning models, can all be accomplished by the processor by calling the corresponding computer program instructions stored in memory. Those skilled in the art can implement the above functions using algorithms and tools known in the prior art, according to actual needs.

[0145] Please see Figure 3 In an embodiment, to efficiently execute the visual language-based shield tunnel leakage identification method provided by this invention, this invention also provides a visual language-based shield tunnel leakage identification system, comprising: an input device 1, an output device 2, a processor 3, and a memory 4. The input device 1, output device 2, processor 3, and memory 4 are interconnected. The memory 4 stores program instructions used to execute the steps of the visual language-based shield tunnel leakage identification method. The visual language-based shield tunnel leakage identification system of this invention has a compact structure and stable performance, and can stably execute the visual language-based shield tunnel leakage identification method of this invention, further improving the overall applicability and practical application capability of this invention.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the present invention.

Claims

1. A shield tunnel water leakage identification method based on visual language, characterized in that, Includes the following steps: Acquire images of the shield tunnel segments to be detected and input them into a detection network built based on a contrastive language-image pre-trained model; The visual features extracted by the detection network are domain adapted by a clutter gating adapter to generate position-wise clutter confidence, so that visually significant but semantically normal clutter regions bypass the domain adaptation transformation. Multi-scale patch features are obtained by using a multi-scale feature projector to fuse shallow and deep visual features of different receptive fields according to the prior weighting of the training set leakage scale. Attention weighting is applied to the multi-scale patch features based on normal basis anchoring attention, and the normal basis prototype set obtained by offline clustering is used as a reference to highlight leakage areas that deviate from the normal distribution. Based on the cosine similarity between the multi-scale tile features and the three preset sets of text anchor points, and combined with the position-by-position clutter confidence, the leakage detection result is obtained.

2. The method according to claim 1, wherein, The clutter gating adapters are deployed in the shallow layers of the text encoder and the vision encoder, respectively, and internally maintain a clutter template library for performing the following steps: The maximum cosine similarity between the input features and the clutter template library is calculated as the position-by-position clutter confidence. The input features are adapted by performing an adaptation transformation to obtain the adapted features; Using the position-by-position clutter confidence as weight, the original input features and the adapted features are convexly combined and fused to obtain the output features; The clutter template library is obtained by K-means clustering of shallow patch features of normal samples in the training set, and is updated by exponential moving average during training.

3. The method according to claim 1, wherein, The multi-scale feature projector is deployed in the middle of the visual encoder to perform the following steps: Extract shallow group features and deep group features from the visual encoder, where shallow group features correspond to small effective receptive fields and deep group features correspond to large effective receptive fields. The shallow group features and deep group features are mapped to the shared semantic space through independent linear projection heads and then L2 normalized. The multi-scale patch features are obtained by weighting the two normalized features with fixed weights obtained from the scale distribution of the leakage labeled area in the offline statistical training set.

4. The method according to claim 1, wherein, The normal substrate anchored attention, deployed deep within the visual encoder, is used to perform the following steps: During the model initialization phase, the deep patch features obtained from the normal samples in the training set through the original visual encoder are subjected to offline K-means clustering to obtain a fixed normal basis prototype set. The query-key similarity calculation of the original multi-head attention is replaced by the inner product of the value vector of each tile and the row-wise cosine similarity vector of the normal basis prototype set to obtain the attention weight; The value vector is weighted and summed using the attention weights to obtain the attention-weighted features.

5. The method according to claim 1, wherein, The three sets of text anchor points are preset, including the following steps: Three sets of text prompt templates are preset, corresponding to the clean and normal state of shield tunnel segments, the normal state of clutter with structural background, and the abnormal state of water leakage, respectively. The three sets of text prompt word templates are input into the text encoder, and after encoding, the average value is taken to obtain the normal text anchor point, the noise text anchor point, and the leakage text anchor point. During the training phase, three types of anchor hierarchical decoupling losses are used to constrain the geometric relationships of the three sets of text anchor points. The constraint logic of the three types of anchor hierarchical decoupling losses includes: The first high-priority constraint is to make the cosine similarity between the leaked text anchor point and the normal text anchor point and the clutter text anchor point approach 0, thus separating the leaked semantics from the non-defect semantics. The second layer of low-priority constraints: make the cosine similarity between normal text anchors and clutter text anchors approach 1, and classify structural clutter semantics into the normal cluster.

6. The method according to claim 5, wherein, The three types of anchor layer decoupling losses satisfy: where, denotes three types of anchor decoupling losses, , , denote leakage text anchors, normal text anchors and noise text anchors, respectively, denotes vector inner product. Since the three groups of anchors are L2-normalized, the vector inner product is equivalent to the cosine similarity, denotes the weight coefficients of the two constraints.

7. The method for identifying water leakage in shield tunnels based on visual language according to claim 1, characterized in that, The detection network is trained using a two-stage, curriculum-based training strategy, including: Before training, calculate the domain difference between the text branch and the visual branch relative to the original pre-training distribution. If the text domain difference is greater than the visual domain difference, the visual branch parameters are frozen in the first stage, and only the text clutter gate adapter is trained. In the second stage, the text branch parameters are frozen, and the visual clutter gate adapter and the multi-scale feature projector are trained. If the visual domain difference is greater than the text domain difference, the visual branch should be corrected first, followed by the text branch. The text domain difference is the Euclidean distance between the average embedding of the training text prompt words obtained by the original text encoder and the average embedding of the pre-trained natural language prompt word template pool. The visual domain difference is the Euclidean distance between the average embedding of the training set images obtained by the original visual encoder and the average embedding of the pre-trained natural image reference set.

8. The method for identifying water leakage in shield tunnels based on visual language according to claim 1, characterized in that, The attenuation based on the position-by-position clutter confidence includes the following steps: The cosine similarity between the multi-scale patch features and the three sets of text anchors is calculated to obtain a three-channel score map. The three-channel score map is fused into a single-channel abnormal score map according to a preset fusion formula; The single-channel anomaly score map is attenuated position by position using the position-by-position clutter confidence to obtain the final anomaly score map.

9. The method for identifying water leakage in shield tunnels based on visual language according to claim 8, characterized in that, The preset fusion formula satisfies: in, For the decayed outlier scores, This represents the clutter confidence attenuation coefficient. Indicates position Clutter confidence level at the location This represents the single-channel anomaly score obtained through fusion. Indicates position The cosine similarity score for the leakage anchor points. Indicates position The cosine similarity score relative to normal anchor points. Indicates position The cosine similarity score of the clutter anchor point.

10. A shield tunnel leakage identification system based on visual language, characterized in that, The visual language-based shield tunnel leakage identification system includes: an input device, an output device, a processor, and a memory. The input device, output device, processor, and memory are interconnected. The memory stores program instructions, which are used to execute the visual language-based shield tunnel leakage identification method according to any one of claims 1-9.

Images