A shield tunnel structure health diagnosis method based on multi-modal image fusion

Abstract

This invention belongs to the field of shield tunnel structure diagnosis technology, and specifically to a method for health diagnosis of shield tunnel structures based on multimodal image fusion. The method includes the following steps: Step 1: Collaborative acquisition of multimodal images; Step 2: Multimodal image preprocessing and registration; Step 3: Layered multimodal image fusion; Step 4: Intelligent diagnosis and health assessment. This invention, through collaborative acquisition of multimodal images and feature-level and decision-level dual-layer fusion, combined with Gaussian filtering for noise reduction, histogram equalization enhancement, and unified coordinate system registration preprocessing, effectively compensates for the limitations of single-modal detection information. It integrates multi-dimensional data on appearance, deformation, temperature, and internal hidden damage, eliminates image noise and spatial bias, significantly improves the completeness and recognition accuracy of disease features, and fundamentally solves the problems of missed detections, false detections, and incomplete information in traditional detection methods, achieving full-dimensional and high-precision perception of shield tunnel structural diseases.

Description

Technical Field

[0001] This invention relates to the field of shield tunnel structure diagnosis technology, specifically a shield tunnel structure health diagnosis method based on multimodal image fusion. Background Technology

[0002] As the core underground structure of urban rail transit and highway-railway transportation, shield tunnels are susceptible to structural defects such as lining cracks, water leakage, segment misalignment, backfill voids, and steel corrosion during long-term operation. These defects continuously reduce the stability and durability of the tunnel structure, and in severe cases, can lead to safety accidents. Therefore, structural health diagnosis of shield tunnels is a crucial link in ensuring the safe operation of underground engineering projects. Traditional tunnel health inspections mainly rely on manual inspections, depending on visual observation and simple tools. This method suffers from low inspection efficiency, high labor intensity, strong subjectivity, and the tendency to miss minor defects, making it unsuitable for the routine inspection needs of long-distance, deep-buried shield tunnels.

[0003] Currently, shield tunnel structural inspection is increasingly employing single-modal image detection technologies, including visible light imaging, infrared thermal imaging, 3D laser scanning, and ground-penetrating radar. However, each of these single-modal detection methods has significant limitations. Visible light images can only identify surface defects in the lining and cannot detect internal damage; infrared thermal imaging can identify temperature anomalies and water leakage, but it is difficult to accurately quantify structural deformation; 3D laser point clouds can measure geometric deformation but cannot detect internal voids and corrosion; ground-penetrating radar can detect hidden damage, but its ability to identify minute surface cracks is insufficient. Single-modal data cannot comprehensively reflect the true health status of the tunnel structure.

[0004] Existing multimodal detection technologies lack systematic fusion solutions, often involving simple data overlay without feature-level and decision-level hierarchical fusion, failing to fully leverage the complementary advantages of multi-source data. Furthermore, these technologies lack standardized image preprocessing procedures, failing to employ precise Gaussian filtering for noise reduction and histogram equalization enhancement algorithms, resulting in significant image noise interference and unclear disease features. Moreover, the lack of a unified spatial coordinate system for precise registration of multimodal images leads to viewpoint and scale deviations, resulting in insufficient accuracy in disease localization and quantification.

[0005] Furthermore, existing shield tunnel health diagnosis technologies lack intelligent recognition models based on multimodal fusion features. This results in low accuracy in identifying minute defects such as crack widths ≤0.2mm and void thicknesses ≤5cm, and significant errors in quantifying defect parameters. The health assessment system is also incomplete, failing to comprehensively rate defects based on type, severity, and distribution, thus unable to output tiered early warning information. Seamless integration of detection data with the operation and maintenance platform is also lacking, hindering support for preventative tunnel maintenance and long-term health monitoring. Therefore, developing a shield tunnel structural health diagnosis method that features deep multimodal image fusion, comprehensive detection, high accuracy, high efficiency, and intelligent diagnosis has become a pressing technical challenge in this field. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a method for structural health diagnosis of shield tunnels based on multimodal image fusion, which solves the problems mentioned in the background section.

[0007] To achieve the above objectives, the present invention specifically adopts the following technical solution: A method for structural health diagnosis of shield tunnels based on multimodal image fusion includes the following steps: Step 1: Multimodal image collaborative acquisition, synchronously acquiring multi-source heterogeneous image data of the shield tunnel through an integrated sensing system; Step 2: Multimodal image preprocessing and registration. The acquired modal images are preprocessed and accurately registered based on a unified spatial coordinate system. Step 3: Layered multimodal image fusion. A two-layer fusion strategy of feature level and decision level is adopted. First, the disease features of each modality image are extracted and a fusion feature vector is constructed. Then, the independent diagnosis results of each modality are weighted and fused. Step 4: Intelligent diagnosis and health assessment. Based on the fusion feature training, the disease identification model is automatically quantified, and combined with the health assessment standards, the structural health level and early warning information are output.

[0008] Furthermore, in step 1, the multi-source heterogeneous image data includes high-resolution visible light images, infrared thermal imaging images, three-dimensional laser point cloud images, and ground-penetrating radar images; Among them, high-resolution visible light images are used to capture cracks and spalling defects on the surface of shield tunnel lining, infrared thermal imaging images are used to identify water leakage and internal temperature anomalies in the lining, three-dimensional laser point cloud images are used to measure segment misalignment and convergence deformation parameters, and ground penetrating radar images are used to detect internal damage such as voids behind the lining and steel corrosion.

[0009] Furthermore, in step 1, the integrated sensing system is mounted on a shield tunnel inspection vehicle or inspection robot. The integrated sensing system includes a visible light camera, an infrared thermal imager, a 3D laser scanner, and a ground-penetrating radar. All devices start up synchronously and work together, with a consistent acquisition frequency and an acquisition range covering the entire cross-section of the shield tunnel without any missed areas.

[0010] Further, in step 2, the preprocessing includes denoising, enhancement, and distortion correction; wherein, denoising employs a Gaussian filtering algorithm to remove image noise, and the specific expression of the Gaussian filtering algorithm is: ; in, The coordinates in the filter template are The Gaussian weight values ​​of the pixels are used to perform weighted averaging of image pixels to achieve noise reduction; The pixel coordinates are relative to the center point of the filter template, i.e., the center point coordinates are 0,0, representing the positional offset of the current pixel from the center point; The standard deviation of the Gaussian filter is used to determine the smoothness, and its value ranges from 0.5 to 2.0, taking into account the noise intensity of the tunnel image. Pi is the constant; the denominator is... The normalization coefficient ensures that the sum of all weight values ​​in the filter template is 1, thus avoiding image brightness shift. The image contrast is enhanced by employing a histogram equalization algorithm. The core expression and parameter definitions of the histogram equalization algorithm are as follows: First, the image grayscale histogram is calculated. Then calculate the cumulative distribution function (CDF) of the gray levels, which is the core expression for histogram equalization: ; The functions and their parameters are defined as follows: gray levels in the image The frequency of occurrence; This represents the kth gray level of the image, with a value range of 0-255, matching the gray level range of the tunnel image. For the image with gray levels of The number of pixels; N is the total number of pixels in the image; To equalize the corresponding gray levels The new grayscale value; This is a grayscale mapping function used to map the original grayscale levels. Mapped to new gray levels after equalization ; The cumulative distribution function represents the original gray levels. to The cumulative frequency of occurrence; distortion correction is based on camera intrinsic parameters and tunnel axis parameters to eliminate lens distortion and acquisition angle deviation; The unified spatial coordinate system is established based on the tunnel axis and combined with the position of the segment splicing joint, so as to achieve pixel-level and spatial-level double accurate registration of each modal image, with a registration error of no more than 0.5 pixels.

[0011] Furthermore, in step 3, the feature-level fusion specifically involves: using a deep learning network combining CNN convolutional neural network and Transformer to extract the disease features of each modality image, normalizing the extracted features, and then weighting and concatenating them through an attention mechanism to construct a multimodal fusion feature vector, thereby compensating for the lack of features in a single modality image and improving the completeness and recognizability of the disease features.

[0012] Further, in step 3, the decision-level fusion specifically involves: first, obtaining disease diagnosis results corresponding to each modality through disease recognition models trained independently for each modality image, including disease type, location, and severity; then, determining weighting coefficients based on the accuracy of each modality diagnosis results, performing weighted fusion of each modality diagnosis results, eliminating abnormal diagnosis results, and outputting a comprehensive and reliable disease diagnosis conclusion; wherein, the weighting coefficients are determined by cross-validation, and the sum of the weights of each modality is 1.

[0013] Furthermore, in step 4, the disease identification model takes a multimodal fusion feature vector as input and disease type, location, and size parameters as output. It is trained using a labeled multimodal image dataset. During training, the Adam optimizer is used, and the learning rate is adaptively adjusted. After the model is trained, the disease identification accuracy is not less than 95%. For minor diseases with crack width ≤ 0.2 mm and void thickness ≤ 5 cm, the identification accuracy is not less than 88%.

[0014] Furthermore, in step 4, the automatically quantified defect parameters include: the length, width, and direction of cracks; the area and range of water leakage; the height and length of segment misalignment; the range and thickness of voids behind the lining; and the area and degree of steel reinforcement corrosion; the quantification error of each parameter shall not exceed 5%.

[0015] Furthermore, in step 4, the health assessment standard refers to the relevant standards for shield tunnel structure health assessment in the prior art, and establishes a multi-index comprehensive evaluation system. The evaluation indicators include the type of defect, the severity of the defect, and the distribution range of the defect. The health level of the shield tunnel structure is divided into four levels: normal, minor, moderate, and severe, and corresponding early warning information is output for different levels. Among them, the severe level triggers an emergency early warning, prompting immediate maintenance and repair. Furthermore, it also includes data storage and transmission steps: the collected multimodal image data, preprocessed data, fused feature data, diagnostic results and health assessment reports are transmitted in real time to the tunnel operation and maintenance management platform via a wireless transmission module, and stored in encrypted form. The storage format is compatible with the existing operation and maintenance system, and supports data backtracking, querying and statistical analysis, providing data support for long-term health monitoring and preventive maintenance of shield tunnels.

[0016] Compared with existing technologies, this invention provides a method for structural health diagnosis of shield tunnels based on multimodal image fusion, which has the following beneficial effects: This invention effectively compensates for the shortcomings of single-modal detection information by combining multimodal image collaborative acquisition with feature-level and decision-level dual-layer fusion, along with Gaussian filtering for noise reduction, histogram equalization enhancement, and unified coordinate system registration preprocessing. It integrates multi-dimensional data on appearance, deformation, temperature, and internal hidden damage, eliminates image noise and spatial bias, and significantly improves the completeness and recognition accuracy of disease features. It fundamentally solves the problems of missed detection, false detection, and incomplete information in traditional detection methods, and achieves full-dimensional and high-precision perception of structural defects in shield tunnels.

[0017] This invention employs an intelligent recognition model based on multimodal fusion feature training, which can automatically complete the determination of disease type and quantification of size parameters. It has high accuracy in identifying minor diseases with small quantification errors, completely replacing manual judgment, significantly improving detection efficiency and objectivity, and reducing the intensity of manual inspection and subjective interference. This method is suitable for routine inspection of long-distance, deep-buried shield tunnels, breaking through the technical bottlenecks of low diagnostic efficiency, insufficient accuracy, and difficulty in automating existing technologies.

[0018] This invention constructs a multi-indicator comprehensive evaluation system based on existing technical standards. It classifies health levels into four levels according to disease type, severity, and distribution range, and outputs graded early warnings. Severe diseases can trigger emergency early warnings, quickly guiding maintenance and repair decisions. At the same time, it realizes encrypted data transmission and platform-based storage throughout the entire process, is compatible with existing operation and maintenance systems, and supports data backtracking analysis, forming a closed loop of "detection-diagnosis-assessment-early warning-operation and maintenance". This provides efficient and reliable technical support for long-term safety monitoring and preventive maintenance of shield tunnels. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall process of a shield tunnel structure health diagnosis method based on multimodal image fusion according to the present invention; Figure 2 This is a schematic flowchart of the shield tunnel structure health diagnosis method based on multimodal image fusion according to the present invention; Figure 3 This is a schematic diagram of the multimodal image collaborative acquisition system of the present invention. Figure 4This is a schematic diagram of the image preprocessing and multimodal fusion process of the present invention. Figure 5 This is a schematic diagram of the shield tunnel structural health assessment system of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] like Figure 1-5 As shown, the present invention proposes a method for structural health diagnosis of shield tunnels based on multimodal image fusion, comprising a detection carrier, a multimodal acquisition device, a preprocessing module, a hierarchical fusion module, a disease identification model, a health assessment module, and a data storage and transmission module; Among them, visible light cameras, infrared thermal imagers, 3D laser scanners, and ground-penetrating radars are integrated and mounted on shield tunnel inspection vehicles / inspection robots to form an integrated sensing system. All acquisition devices start up synchronously and maintain a consistent acquisition frequency. The output of the integrated sensing system is connected to the input of the preprocessing module. The output of the preprocessing module is connected to the input of the hierarchical fusion module. The output of the hierarchical fusion module is connected to the input of the disease identification model. The output of the disease identification model is connected to the input of the health assessment module. The output of the health assessment module is connected to the input of the data storage and transmission module. The output of the data storage and transmission module is connected to the tunnel operation and maintenance management platform.

[0022] Combined with appendix Figure 1 The overall process shown is as follows: Using a shield tunnel inspection vehicle / robot as a carrier, an integrated sensing system synchronously collects four types of multi-source heterogeneous image data of the shield tunnel: visible light, infrared thermal imaging, 3D laser point cloud, and ground-penetrating radar. The raw images are transmitted to a preprocessing module for Gaussian filtering denoising, histogram equalization enhancement, lens distortion correction, and unified spatial coordinate system registration. A layered fusion module then performs feature-level and decision-level dual-layer fusion to construct a multimodal fusion feature vector and output comprehensive disease diagnosis results. A disease identification model trained based on the fusion features completes automatic disease identification and parameter quantification. A health assessment module classifies structural health levels according to a multi-index comprehensive evaluation system and outputs corresponding early warning information. Finally, a data storage and transmission module encrypts and transmits the entire process data to the tunnel operation and maintenance management platform for data backtracking, querying, and statistical analysis.

[0023] like Figure 1-3As shown, Step 1: Multimodal image collaborative acquisition The integrated perception system mounted on the shield tunnel inspection vehicle / robot is activated. The visible light camera, infrared thermal imager, 3D laser scanner, and ground penetrating radar are activated simultaneously and work together, with consistent acquisition frequency and acquisition range covering the entire cross section of the shield tunnel without any missed areas. The system simultaneously acquired four types of multi-source heterogeneous image data: high-resolution visible light images, infrared thermal imaging images, three-dimensional laser point cloud images, and ground-penetrating radar images. Among them, high-resolution visible light images were used to capture cracks and spalling defects on the surface of the shield tunnel lining, infrared thermal imaging images were used to identify water leakage and internal temperature anomalies in the lining, three-dimensional laser point cloud images were used to measure segment misalignment and convergence deformation parameters, and ground-penetrating radar images were used to detect internal damage such as voids behind the lining and steel corrosion.

[0024] like Figure 4 As shown, step 2: multimodal image preprocessing and registration 1. The original multimodal images acquired in step 1 are sequentially processed by denoising, enhancement, and distortion correction, and accurate registration is completed based on a unified spatial coordinate system: Gaussian filtering for noise removal: The Gaussian filtering algorithm is used to remove image noise. The algorithm expression is as follows: ; in, The coordinates in the filter template are The Gaussian weight values ​​of the pixels are used to perform weighted averaging of image pixels to achieve noise reduction; The pixel coordinates are relative to the center point of the filter template, i.e., the center point coordinates are 0,0, representing the positional offset of the current pixel from the center point; The standard deviation of the Gaussian filter is used to determine the smoothness, and its value ranges from 0.5 to 2.0, taking into account the noise intensity of the tunnel image. Pi is the constant; the denominator is... The normalization coefficient ensures that the sum of all weight values ​​in the filter template is 1, thus avoiding image brightness shift. 2. Histogram Equalization Enhancement: Histogram equalization algorithm is used to improve image contrast. First, the image grayscale histogram is calculated. Next, calculate the cumulative distribution function (CDF) of the gray levels. The core expression is: ; gray levels in the image The frequency of occurrence; This represents the kth gray level of the image, with a value range of 0-255, matching the gray level range of the tunnel image. For the image with gray levels of The number of pixels; N is the total number of pixels in the image; To equalize the corresponding gray levels The new grayscale value; This is a grayscale mapping function used to map the original grayscale levels. Mapped to new gray levels after equalization ; The cumulative distribution function represents the original gray levels. to The cumulative frequency of occurrence; distortion correction is based on camera intrinsic parameters and tunnel axis parameters to eliminate lens distortion and acquisition angle deviation; 3. Distortion Correction and Spatial Registration: Lens distortion correction is completed based on camera intrinsic parameters and tunnel axis parameters to eliminate lens distortion and acquisition angle deviation; a unified spatial coordinate system is established with the tunnel axis as the reference and combined with the position of the segment splicing joint to achieve pixel-level and spatial-level double accurate registration of images of each modality, with a registration error of no more than 0.5 pixels.

[0025] like Figure 4 Step 3: Layered multimodal image fusion A two-layer fusion strategy, combining feature-level and decision-level techniques, is employed to process the preprocessed image. Feature-level fusion: A deep learning network combining CNN convolutional neural network and Transformer is used to extract the disease features of each modality of image. After normalization of the extracted features, they are weighted and concatenated through an attention mechanism to construct a multimodal fusion feature vector, which makes up for the lack of features in single modality images and improves the completeness and recognizability of disease features.

[0026] Decision-level fusion: First, disease identification models trained independently for each modality image are used to obtain disease diagnosis results for each modality, including disease type, location, and severity. Then, weighting coefficients are determined based on the accuracy of each modality diagnosis results, and the diagnosis results of each modality are weighted and fused. Abnormal diagnosis results are eliminated, and a comprehensive and reliable disease diagnosis conclusion is output. The weighting coefficients are determined by cross-validation, and the sum of the weights of each modality is 1.

[0027] like Figure 5 Step 4: Intelligent Diagnosis and Health Assessment Disease identification model inference: Using the multimodal fusion feature vector obtained in step 3 as input and the disease type, location, and size parameters as output, the disease identification model is used for diagnosis. The model is trained using a labeled multimodal image dataset. During training, the Adam optimizer is used and the learning rate is adaptively adjusted. The disease identification accuracy is not less than 95%, and the identification accuracy of minor diseases (crack width ≤ 0.2 mm, void thickness ≤ 5 cm) is not less than 88%.

[0028] Defect parameter quantification: Automatically quantify defect parameters, including the length, width, and direction of cracks, the area and range of water leakage, the height and length of segment misalignment, the range and thickness of voids behind the lining, and the area and degree of steel reinforcement corrosion. The quantification error of each parameter does not exceed 5%.

[0029] Health level assessment and early warning: Referring to the relevant standards for health assessment of shield tunnel structures in existing technologies, a comprehensive evaluation system with multiple indicators including the type of defect, the severity of the defect, and the distribution range of the defect is established. The health level of shield tunnel structures is divided into four levels: normal, minor, moderate, and severe. Early warning information is output for different levels, with the severe level triggering an emergency warning, prompting immediate maintenance and repair.

[0030] Furthermore, data storage and transmission The collected multimodal image data, preprocessed data, fused feature data, diagnostic results, and health assessment reports are transmitted in real time to the tunnel operation and maintenance management platform via a wireless transmission module and stored in encrypted form. The storage format is compatible with existing operation and maintenance systems and supports data backtracking, querying, and statistical analysis, providing data support for long-term health monitoring and preventive maintenance of shield tunnels.

[0031] Combined with appendix Figure 1 To be continued Figure 5 The structural health diagnosis method for shield tunnels based on multimodal image fusion, as described in this invention, was used to perform structural health diagnosis on a subway shield tunnel in a certain city. The complete workflow is as follows: 1. Multimodal image collaborative acquisition The shield tunnel inspection robot is activated, and its integrated sensing system works simultaneously: Visible light cameras acquire high-resolution images to identify lining cracks and spalling; Infrared thermal imagers acquire infrared images to identify water leaks and temperature anomalies; A 3D laser scanner acquires point cloud images for measuring segment misalignment and convergence deformation; Ground-penetrating radar collects radar images to detect voids behind the lining and corrosion of the reinforcing steel.

[0032] All devices collect data at the same frequency, covering the entire tunnel section with no blind spots, and simultaneously acquiring four types of multi-source heterogeneous image data.

[0033] 2. Multimodal image preprocessing and registration The acquired raw images are processed sequentially: 1) Gaussian filtering for noise reduction Formula used: ; in, The coordinates in the filter template are The Gaussian weight values ​​of the pixels, These are the pixel coordinates relative to the center point of the filter template. Take 1.0, The normalization coefficient is used to achieve image denoising without shifting brightness.

[0034] 2) Histogram equalization enhancement First calculate the grayscale histogram , Next, calculate the cumulative distribution function: The image grayscale is made uniform, improving the contrast of disease edges.

[0035] 3) Distortion correction and spatial registration Distortion correction is performed based on camera intrinsic parameters and tunnel axis; a unified coordinate system is established with tunnel axis and segment joint as reference to achieve pixel-level and spatial-level registration of multimodal images with a registration error ≤0.5 pixels.

[0036] 3. Hierarchical multimodal image fusion 1) Feature-level fusion Four types of image disease features were extracted using a CNN+Transformer network. After normalization, the features were weighted and concatenated using an attention mechanism to generate a multimodal fusion feature vector.

[0037] 2) Decision-level integration Each single-modal model outputs the disease type, location, and severity; weighting coefficients are determined using cross-validation, with a weight sum of 1; the results are weighted and fused, outliers are removed, and a comprehensive disease diagnosis result is output.

[0038] 4. Intelligent diagnosis and quantification of disease parameters The disease identification model is trained using the Adam optimizer with fused feature vectors as input. The disease identification accuracy is ≥95%, and the minor disease identification accuracy is ≥88%.

[0039] Automatic quantization output: Crack length, width, and direction Area and extent of water leakage Segment misalignment height and length Lining void range and thickness Area and extent of steel bar corrosion The quantization error of all parameters is ≤5%.

[0040] 5. Comprehensive assessment and early warning of structural health A multi-indicator evaluation system was established based on the type, severity, and distribution range of the defects, classifying tunnel health into: normal, minor, moderate, and severe.

[0041] In this embodiment, if the diagnosis is minor, a standard warning is output; if it is determined to be severe, an emergency warning is immediately triggered, prompting immediate maintenance and repair.

[0042] 6. Data storage and transmission The original images, preprocessed data, fused features, diagnostic results, and assessment reports are transmitted to the tunnel operation and maintenance management platform via a wireless module. The data is stored in encrypted form and is compatible with existing systems. It supports data backtracking, querying, and statistics, providing support for long-term monitoring and preventive maintenance.

[0043] This method can complete multi-dimensional defect detection and structural health assessment of tunnels in one go. It is comprehensive, accurate, and highly automated, fully meeting the engineering requirements for structural health diagnosis of shield tunnels.

[0044] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for structural health diagnosis of shield tunnels based on multimodal image fusion, characterized in that, Includes the following steps: Step 1: Multimodal image collaborative acquisition, synchronously acquiring multi-source heterogeneous image data of the shield tunnel through an integrated sensing system; Step 2: Multimodal image preprocessing and registration. The acquired modal images are preprocessed and accurately registered based on a unified spatial coordinate system. Step 3: Layered multimodal image fusion. A two-layer fusion strategy of feature level and decision level is adopted. First, the disease features of each modality image are extracted and a fusion feature vector is constructed. Then, the independent diagnosis results of each modality are weighted and fused. Step 4: Intelligent diagnosis and health assessment. Based on the fusion feature training, the disease identification model is automatically quantified, and combined with the health assessment standards, the structural health level and early warning information are output.

2. The method for structural health diagnosis of shield tunnels based on multimodal image fusion according to claim 1, characterized in that: In step 1, the multi-source heterogeneous image data includes high-resolution visible light images, infrared thermal imaging images, three-dimensional laser point cloud images, and ground-penetrating radar images. Among them, high-resolution visible light images are used to capture cracks and spalling defects on the surface of shield tunnel lining, infrared thermal imaging images are used to identify water leakage and internal temperature anomalies in the lining, three-dimensional laser point cloud images are used to measure segment misalignment and convergence deformation parameters, and ground penetrating radar images are used to detect internal damage such as voids behind the lining and steel corrosion.

3. The method for structural health diagnosis of shield tunnels based on multimodal image fusion according to claim 1, characterized in that: In step 1, the integrated sensing system is mounted on a shield tunnel inspection vehicle or inspection robot. The integrated sensing system includes a visible light camera, an infrared thermal imager, a 3D laser scanner, and a ground penetrating radar. All devices start up synchronously and work together, with a consistent acquisition frequency and an acquisition range covering the entire cross-section of the shield tunnel without any missed areas.

4. The method for structural health diagnosis of shield tunnels based on multimodal image fusion according to claim 1, characterized in that: In step 2, the preprocessing includes denoising, enhancement, and distortion correction; wherein, denoising employs a Gaussian filtering algorithm to remove image noise, and the specific expression of the Gaussian filtering algorithm is: ； in, The coordinates in the filter template are The Gaussian weight values ​​of the pixels are used to perform weighted averaging of image pixels to achieve noise reduction; The pixel coordinates are relative to the center point of the filter template, i.e., the center point coordinates are 0,0, representing the positional offset of the current pixel from the center point; The standard deviation of the Gaussian filter is used to determine the smoothness, and its value ranges from 0.5 to 2.0, taking into account the noise intensity of the tunnel image. Pi is the constant; the denominator is... The normalization coefficient ensures that the sum of all weight values ​​in the filter template is 1, thus avoiding image brightness shift. The image contrast is enhanced by employing a histogram equalization algorithm. The core expression and parameter definitions of the histogram equalization algorithm are as follows: First, the image grayscale histogram is calculated. Then calculate the cumulative distribution function (CDF) of the gray levels, which is the core expression for histogram equalization: ； The functions and their parameters are defined as follows: gray levels in the image The frequency of occurrence; This represents the kth gray level of the image, with a value range of 0-255, matching the gray level range of the tunnel image. For the image with gray levels of The number of pixels; N is the total number of pixels in the image; To equalize the corresponding gray levels The new grayscale value; This is a grayscale mapping function used to map the original grayscale levels. Mapped to new gray levels after equalization ; The cumulative distribution function represents the original gray levels. to The cumulative frequency of occurrence; distortion correction is based on camera intrinsic parameters and tunnel axis parameters to eliminate lens distortion and acquisition angle deviation; The unified spatial coordinate system is established based on the tunnel axis and combined with the position of the segment splicing joint, so as to achieve pixel-level and spatial-level double accurate registration of each modal image, with a registration error of no more than 0.5 pixels.

5. The method for structural health diagnosis of shield tunnels based on multimodal image fusion according to claim 1, characterized in that: In step 3, the feature-level fusion specifically involves: using a deep learning network combining CNN convolutional neural network and Transformer to extract the disease features of each modality image, normalizing the extracted features, and then weighting and concatenating them through an attention mechanism to construct a multimodal fusion feature vector, thereby compensating for the lack of features in a single modality image and improving the completeness and recognizability of the disease features.

6. The method for structural health diagnosis of shield tunnels based on multimodal image fusion according to claim 1, characterized in that: In step 3, the decision-level fusion specifically involves: first, obtaining disease diagnosis results corresponding to each modality through disease recognition models trained independently for each modality image, including disease type, location, and severity; then, determining weighting coefficients based on the accuracy of each modality diagnosis results, performing weighted fusion on each modality diagnosis results, eliminating abnormal diagnosis results, and outputting a comprehensive and reliable disease diagnosis conclusion; wherein, the weighting coefficients are determined by cross-validation, and the sum of the weights of each modality is 1.

7. The method for structural health diagnosis of shield tunnels based on multimodal image fusion according to claim 1, characterized in that: In step 4, the disease identification model takes a multimodal fusion feature vector as input and disease type, location, and size parameters as output. It is trained using a labeled multimodal image dataset. During training, the Adam optimizer is used and the learning rate is adaptively adjusted. After the model is trained, the disease identification accuracy is not less than 95%. For minor diseases with crack width ≤ 0.2 mm and void thickness ≤ 5 cm, the identification accuracy is not less than 88%.

8. The method for structural health diagnosis of shield tunnels based on multimodal image fusion according to claim 1, characterized in that: In step 4, the automatically quantified defect parameters include: the length, width, and direction of cracks; the area and range of water leakage; the height and length of segment misalignment; the range and thickness of voids behind the lining; and the area and degree of steel reinforcement corrosion. The quantification error of each parameter shall not exceed 5%.

9. The method for structural health diagnosis of shield tunnels based on multimodal image fusion according to claim 1, characterized in that: In step 4, the health assessment standard refers to the relevant standards for health assessment of shield tunnel structures in the prior art, and establishes a multi-index comprehensive evaluation system. The evaluation indicators include the type of disease, the severity of disease, and the distribution range of disease. The health level of the shield tunnel structure is divided into four levels: normal, minor, moderate, and severe. Correspondingly, different levels of early warning information are output. Among them, the severe level triggers an emergency early warning, prompting immediate maintenance and repair.

10. The method for structural health diagnosis of shield tunnels based on multimodal image fusion according to claim 1, characterized in that: It also includes data storage and transmission steps: the collected multimodal image data, preprocessed data, fused feature data, diagnostic results and health assessment reports are transmitted in real time to the tunnel operation and maintenance management platform through a wireless transmission module, and stored in encrypted form. The storage format is compatible with the existing operation and maintenance system, and supports data backtracking, querying and statistical analysis, providing data support for long-term health monitoring and preventive maintenance of shield tunnels.

Images