Tunnel Lining Compactness Defect Detection Method and System Based on Multimodal Data

Abstract

The present invention provides a method and system for detecting compactness defects of tunnel linings based on multimodal data, relating to the technical field of tunnel lining detection. The method includes performing preliminary quality detection on the tunnel lining through a ground penetrating radar to obtain the compactness defects to be detected; acquiring multimodal data of the detected and to-be-detected compactness defects; performing feature extraction and encoding processing on the multimodal data to obtain multimodal embedded representations; performing heterogeneous graph modeling and cross-modal contrast learning based on the multimodal embedded representations to obtain a graph structure; performing joint processing on the graph structure based on a sparse attention mechanism and multimodal fusion to obtain an embedded vector; performing defect category classification on the embedded vector based on diffusion mapping to obtain the classification result of the to-be-detected compactness defects, and the classification result includes concrete non-compactness defects, filling block stone defects, and tunnel slag defects. The present invention solves the problem that the existing single-modal detection method is difficult to identify the specific types of compactness defects.

Description

Technical Field

[0001] The present invention relates to the technical field of tunnel lining detection. Specifically, it relates to a method and system for detecting the compactness defects of tunnel linings based on multi-modal data. Background Art

[0002] For the quality detection of tunnel linings, ground penetrating radar is mostly used as a preliminary diagnostic tool. This method penetrates the lining with high-frequency electromagnetic waves and analyzes their echo responses to identify whether there are compactness problems inside. However, ground penetrating radar can only judge whether there are compactness defects in the lining. Due to the extremely similar electromagnetic wave responses for different types of defects (such as non-compact concrete, filled block stones and tunnel slag), it is difficult to effectively distinguish them. This makes it difficult to formulate targeted subsequent disease treatment strategies, affecting the practicality and accuracy of diagnosis.

[0003] Currently, the detection of lining defects mainly relies on single-modal detection means. The information dimensions obtained are limited, and there are significant ambiguities in the response characteristics for different types of compactness defects, resulting in insufficient classification accuracy. At the same time, existing research often ignores the potential time-delay correlation and causal mechanism between multi-modal data and lacks a systematic fusion modeling method, restricting the intelligent development of compactness defect identification.

[0004] Therefore, there is an urgent need for a new detection method that can fuse multi-modal perception data, fully explore the correlations between modalities, and improve the identification accuracy and classification ability of compactness defects. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and system for detecting the compactness defects of tunnel linings based on multi-modal data to improve the above problems. To achieve the above purpose, the technical solutions adopted by the present invention are as follows:

[0006] In a first aspect, the present application provides a method for detecting the compactness defects of tunnel linings based on multi-modal data, including:

[0007] Conduct a preliminary quality inspection of the tunnel lining through ground penetrating radar to obtain the compactness defects to be detected;

[0008] Obtain the multi-modal data of the detected and to-be-detected compactness defects, where the multi-modal data includes polarization response data, infrared active thermographic data, and acoustic shock response data;

[0009] Extract and encode the features of the multi-modal data to obtain a multi-modal embedded representation;

[0010] Based on the multi-modal embedded representation, perform heterogeneous graph modeling and cross-modal contrast learning to obtain a graph structure;

[0011] Jointly process the graph structure based on the sparse attention mechanism and multimodal fusion to obtain the embedding vector;

[0012] Based on diffusion mapping, classify the embedding vector for the defect category to obtain the classification result of the compactness defect to be detected, and the classification result includes concrete non-compactness defect, filling block stone defect, and hole slag defect.

[0013] In a second aspect, the present application also provides a tunnel lining compactness defect detection system based on multimodal data, including:

[0014] A preliminary detection unit for preliminarily detecting the quality of the tunnel lining through a geological radar to obtain the compactness defect to be detected;

[0015] An acquisition unit for acquiring multimodal data of the detected and to-be-detected compactness defects, and the multimodal data includes polarization response data, infrared active thermographic data, and acoustic shock response data;

[0016] An encoding unit for performing feature extraction and encoding processing on the multimodal data to obtain a multimodal embedding representation;

[0017] A modeling unit for performing heterogeneous graph modeling and cross-modal contrast learning based on the multimodal embedding representation to obtain a graph structure;

[0018] A joint processing unit for jointly processing the graph structure based on the sparse attention mechanism and multimodal fusion to obtain the embedding vector;

[0019] A classification unit for classifying the defect category of the embedding vector based on diffusion mapping to obtain the classification result of the compactness defect to be detected, and the classification result includes concrete non-compactness defect, filling block stone defect, and hole slag defect.

[0020] The beneficial effects of the present invention are as follows: The present invention preliminarily detects the compactness defect of the tunnel lining through a geological radar, and combines multimodal data to significantly improve the comprehensiveness and accuracy of compactness defect recognition. To solve the scale and semantic differences of multimodal data, a unified feature extraction and encoding strategy is proposed, and a three-way heterogeneous graph of modality-location-frequency band is constructed to effectively characterize the spatial, frequency domain, and modality differences of defect responses. In addition, a cross-modal causal learning mechanism is introduced, and the time-delay correlation between different modalities is jointly modeled through mutual information and Granger causal analysis to enhance the graph modeling ability and the discriminability and interpretability of features. The defect response distribution modeling based on kernel density estimation can accurately characterize the distribution characteristics of compactness defects, realize the adaptive setting of defect boundaries and discrimination thresholds, and further improve the reliability of compactness defect detection.

[0021] Other features and advantages of the present invention will be described in the following specification, and in part will be obvious from the specification, or can be understood by implementing the embodiments of the present invention. The objectives and other advantages of the present invention can be realized and obtained by the structures specifically pointed out in the written specification, claims, and drawings. Brief Description of the Drawings

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and thus should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0023] Figure 1 Schematic diagram of the process for detecting defects in the compactness of tunnel linings based on multimodal data described in the embodiments of the present invention;

[0024] Figure 2 Schematic diagram of the structure of the system for detecting defects in the compactness of tunnel linings based on multimodal data described in the embodiments of the present invention. Detailed Embodiments

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed present invention, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0026] It should be noted that: similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the present invention, terms such as "first", "second", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0027] Embodiment 1:

[0028] This embodiment provides a method for detecting defects in the compactness of tunnel linings based on multimodal data.

[0029] See Figure 1, the figure shows that the method includes step S1, step S2, step S3, step S4, step S5 and step S6.

[0030] Step S1: Conduct a preliminary quality inspection on the tunnel lining through a ground-penetrating radar to obtain the density defects to be detected;

[0031] In this embodiment, a preliminary quality inspection is carried out on the tunnel lining to be detected through a ground-penetrating radar, and the areas with abnormal echo characteristics are quickly obtained to obtain typical quality defect results, such as crack defects, cavities or voids, deformations, and density defects, etc. Since the abnormal echo characteristics of different quality defect results are different, the density defects of the tunnel lining to be detected and the position information of the density defects to be detected can be obtained.

[0032] Step S2: Obtain the multi-modal data of the detected and to-be-detected density defects, where the multi-modal data includes polarization response data, infrared active thermographic data, and acoustic shock response data;

[0033] In this embodiment, the multi-modal data of the detected density defects provides label references and feature priors for subsequent contrast learning, graph structure construction, and classification, etc., and the samples of the detected density defects can be obtained from the defect database of existing engineering projects.

[0034] Since a single modality has limitations and cannot fully distinguish the types of density defects, using multi-modal data can make up for their respective weaknesses and improve the ability to distinguish density defects. For example, the heat conduction characteristics, polarization response characteristics, and acoustic wave propagation characteristics of concrete non-dense defects, filled block stone defects, and cavity slag defects are not exactly the same.

[0035] Step S3: Perform feature extraction and encoding processing on the multi-modal data to obtain a multi-modal embedded representation;

[0036] In this embodiment, since the dimensions, scales, and distributions of the multi-modal data are different, directly splicing or comparing these data is likely to introduce inter-modal inconsistencies, resulting in chaotic features and unstable training. Therefore, feature extraction and encoding processing must be carried out to extract representative and dimensionally consistent feature vectors for each, reducing redundancy. The original multi-modal data with different sources and physical meanings is transformed into a representation form that can be uniformly analyzed and fused, that is, a multi-modal embedded representation.

[0037] In step S3, the obtaining of the multi-modal embedded representation includes:

[0038] Step S31: Conduct a polarization feature analysis on the polarization response data to obtain a polarization response vector;

[0039] In this embodiment, the echo amplitudes in different polarization states of the polarization response data are obtained, the polarization scattering matrix is constructed through the echo amplitudes, and then the polarization entropy, the uniformity, and the principal polarization direction angle are calculated through the polarization scattering matrix. The polarization response vector is constructed through the polarization entropy, the uniformity, and the principal polarization direction angle.

[0040] Step S32: Construct a temperature gradient tensor based on the infrared active thermography data, and extract the thermal anomaly response vector through the temperature gradient tensor.

[0041] In this embodiment, the infrared active thermography data is a sequence of multiple infrared thermography images. The temperature gradient tensor in the time series is calculated through the infrared active thermography data. A heat diffusion model is established through the heat conduction equation, and then the thermal conductivity and the thermal inertia are deduced inversely through the temperature gradient tensor.

[0042] According to the thermal conductivity and the thermal inertia, a thermal conductivity map and a thermal inertia distribution map are constructed, and the local anomalies (such as mutations, gradient extreme values, etc.) in the thermal conductivity map and the thermal inertia distribution map are used to extract the thermal anomaly response feature items (the thermal inertia distribution deviation value, the thermal conductivity gradient mutation index, the local heat diffusion coefficient estimation value, and the heat flow direction offset angle statistics), so as to obtain the thermal anomaly response vector. Therefore, the thermal anomaly response vector can reflect the thermal property anomaly characteristics inside the tunnel lining caused by factors such as the decrease in compactness, the embedding of foreign objects, or the replacement of materials.

[0043] Step S33: Construct an impact echo time delay spectrum based on the acoustic wave impact response data, and extract the acoustic wave impact response vector through the impact echo time delay spectrum. The acoustic wave impact response vector includes a reflection amplitude statistical matrix and an energy distribution texture feature.

[0044] In this embodiment, the acoustic wave impact response data is a time-domain signal, representing the acoustic wave signal intensity at different time points. The original acoustic wave impact response data is preprocessed to obtain the impact signal and the echo signal. The preprocessing includes denoising, filtering, and removing unnecessary background noise. The cross-correlation analysis or the matched filtering method is used to perform the correlation analysis on the impact signal and the echo signal to obtain the impact echo time delay spectrum.

[0045] The reflection amplitudes at different time points are extracted from the impact echo time delay spectrum, and a statistical matrix, that is, the reflection amplitude statistical matrix, is constructed to describe the statistical characteristics of the reflection intensity of the acoustic wave echo signal at different time delays. By calculating the local energy distribution of the impact echo time delay spectrum, the energy distribution texture feature is obtained to describe the energy distribution characteristics of the acoustic wave echo on the time axis. Finally, the complete acoustic wave impact response vector is constructed by combining the reflection amplitude statistical matrix and the energy distribution texture feature.

[0046] Step S34: Encode the polarization response vector, the thermal anomaly response vector, and the acoustic wave impact response vector through a multi-modal autoencoder to obtain a multi-modal embedding representation.

[0047] In this embodiment, the variational autoencoder is selected as the multi-modal autoencoder, and independent variational autoencoders are used to encode the response vectors of each modality (polarization response vector, thermal anomaly response vector, and acoustic shock response vector) to form a latent space distribution, that is, the embedding representation corresponding to the modality is obtained. Finally, the embedding representations of all modalities are combined into a multi-modal embedding representation. Among them, the modalities include polarization modality, thermal imaging modality, and acoustic modality.

[0048] Step S4: Perform heterogeneous graph modeling and cross-modal contrast learning based on the multi-modal embedding representation to obtain a graph structure;

[0049] In this embodiment, since the responses of the same defect in different modalities have information sharing and complementarity, a cross-modal contrast learning mechanism is introduced through the graph structure to improve the structural rationality and semantic discrimination of feature fusion between different modalities, and this cross-source dependence relationship can be more realistically modeled.

[0050] In step S4, the heterogeneous graph modeling and cross-modal contrast learning based on the multi-modal embedding representation to obtain a graph structure includes:

[0051] Step S41: Construct a three-way heterogeneous graph for each compactness defect with respect to modality-part-frequency band based on the multi-modal embedding representation;

[0052] In this embodiment, corresponding graph nodes are constructed according to each compactness defect. Each graph node is a three-way heterogeneous graph, that is, it includes modality information, part information, and frequency band information. The modality information is the multi-modal embedding representation, including the embedding representation of each modality. Among them, the perception mechanisms of each modality are different, and the sensitivities to different types of compactness defects are also different. There are differences in the material compactness of different parts (the crown, side walls, and shoulders) of the tunnel lining, and the damage mechanisms are also different. The frequency band controls the information penetration depth and resolution, compensating for the deficiencies of the modality itself. For example, the same type of compactness defect will have different manifestations in the modal response due to factors such as structural differences, force differences, and material property differences at different parts (such as the crown, side walls, shoulders, etc.). Through the distinction of part information, this spatial difference can be better understood and captured, so as to more accurately identify and classify defect types.

[0053] Therefore, through the three-way heterogeneous graph, these three types of information are explicitly incorporated into the graph modeling, which not only increases the integrity of the structural expression, but also improves the accuracy and interpretability of cross-modal defect recognition. The part information can be used for structural semantic mapping through the coordinates of the compactness defect, and the coordinates are mapped to the corresponding structural part labels.

[0054] Step S42: Extract the time-delay correlation between different modalities based on the improved multi-modal causal discovery algorithm, and generate the weight matrix of the tripartite heterogeneous graph based on the time-delay correlation;

[0055] In this embodiment, the weight matrix describes the relationship between the internal modal responses of each graph node, not only expressing the internal characteristics of the node itself, but also reflecting the multi-modal characteristics of the node in subsequent calculations.

[0056] In step S42, the generation steps of the weight matrix are as follows:

[0057] Step S421: Align the polarization response vector, the thermal anomaly response vector, and the acoustic shock response vector to obtain the alignment vectors of multiple modalities;

[0058] Step S422: Perform sliding window segmentation on the alignment vectors to obtain continuous vector segments;

[0059] Step S423: Calculate the mutual information between different modalities based on the continuous vector segments;

[0060] In this embodiment, the mutual information between continuous vector segments is calculated to measure the amount of information shared by different modalities within the same time period. The greater the mutual information, the stronger the correlation between the two modalities. Assume that modality and modality The vector segments on the window are and , then the calculation formula for mutual information is:

[0061] ;

[0062] ;

[0063] In the formula, represents the mutual information between and , and respectively represent the information entropy of and , represents the joint entropy of and , represents the vector segment of modality on the window , represents the vector segment of modality on the window , represents the mutual information between modality and modality , Indicates the total number of windows.

[0064] Step S424: Perform Granger causality analysis based on consecutive vector segments to obtain the causality strength between different modalities.

[0065] In this embodiment, two sets of models are established for each window, namely the non-causal model and the causal model. Specifically:

[0066] ;

[0067] ;

[0068] In the formula, represents modality at time in window of the first vector segment value, represents modality at time in window of the second vector segment value, represents the regression order, that is, how many time steps of information are used for modeling, represents modality at time in window at the delayed time of the first vector segment value, represents modality at time in window at the delayed time of the second vector segment value, represents modality at time in window at the delayed time of the second vector segment value, represents modality of the autoregressive coefficient, represents modality 's causal influence coefficient on modality , represents the residual of the non-causal model of modality , represents the residual of the causal model of modality affected by the existence of modality .

[0069] Among them, there is an influence of modality on modality in the causal model. Therefore, let the residual variances of the non-causal model and the causal model be and , the causal strength is obtained as:

[0070] ;

[0071] ;

[0072] In the formula, represents the causal strength of mode on mode with respect to mode , represents the causal strength of mode with respect to mode , represents the total number of windows.

[0073] Step S425: Construct the time-delay correlation between different modes through mutual information and causal strength, and use the time-delay correlation as the edge weight to construct the weight matrix of the tripartite heterogeneous graph.

[0074] In this embodiment, the formula for the time-delay correlation between two modes is:

[0075] ;

[0076] ;

[0077] In the formula, represents the time-delay correlation between mode and mode , represents the mutual information between mode and mode , and both represent weight parameters, represents the causal strength of mode with respect to mode , represents the causal strength of mode with respect to mode , represents the causal strength of mode with respect to mode .

[0078] Step S43: Based on the physical properties and structural design of the tunnel lining, perform physical constraint pruning and correction on the tripartite heterogeneous graph to obtain a multi-modal response graph;

[0079] In this embodiment, through the physical properties and structural design of the tunnel lining, unreasonable or redundant connection relationships in the tripartite heterogeneous graph can be eliminated. For example, edges with mutual information or causal strength less than the threshold are regarded as invalid.

[0080] Step S44: Optimize the representation consistency and structural contrast ability in the multi-modal response map through cross-modal contrast learning, and construct a graph structure with each density defect as a graph node and the similarity between defects as the edge weight.

[0081] In this embodiment, each graph node includes not only its embedded representations in multiple modalities, but also a weight matrix representing the relationship between the internal modal responses of the graph node. Calculate the embedding similarity of two graph nodes through the embedded representation and the weight matrix, calculate the spatial distance attenuation term and the frequency band similarity adjustment term through the part information and the frequency band information, and then construct the edge weight between two graph nodes, that is, the similarity between defects, through the embedding similarity, the spatial distance attenuation term, and the frequency band similarity adjustment term.

[0082] Based on whether the nodes come from the same true defect category, construct positive and negative sample pairs, and optimize the multi-modal embedded representation through a contrast learning loss function (such as the InfoNCE loss function), so that the edge weight between positive sample node pairs is enhanced and the edge weight of negative sample pairs is weakened, thereby improving the representation consistency and structural distinguishability in the multi-modal response map, and finally obtaining a graph structure with each density defect as a graph node and the similarity between defects as the edge weight. Therefore, the node feature vector of each graph node in the graph structure is composed of multi-modal embedded representation, weight matrix, part information, and frequency band information.

[0083] Step S5: Jointly process the graph structure based on the sparse attention mechanism and multi-modal fusion to obtain an embedded vector;

[0084] In step S5, the obtaining of the embedded vector includes:

[0085] Step S51: Perform normalization processing on the graph structure to generate a sparse graph representation. The normalization processing includes node feature normalization, edge weight matrix sparsification, and structural adjacency matrix standardization;

[0086] In this embodiment, normalize the node feature vector of each graph node in the graph structure to obtain a normalized node feature vector. When sparsifying the edge weight matrix, set the edges with edge weights lower than the preset threshold between graph nodes to zero, and only retain the strong similarity connections to form a sparse adjacency matrix. Calculate the degree matrix corresponding to the adjacency matrix, and use the symmetric normalization form to obtain a normalized sparse adjacency matrix. Update the graph structure through the normalized node feature vector and the normalized sparse adjacency matrix to obtain a sparse graph representation.

[0087] Step S52: Construct a modal channel for each modality based on the sparse graph representation;

[0088] In this embodiment, the normalized node feature vectors of each graph node in the sparse graph representation are split to obtain the feature sub-vectors in each modality. For each modality, the feature sub-vectors of the corresponding modality of all graph nodes are extracted to form the feature matrix of the modality channel. And each modality channel shares the same normalized sparse adjacency matrix, indicating the topological invariance of the sparse graph representation.

[0089] Step S53: Dynamically calculate and aggregate the neighbor information of each graph node in the sparse graph representation based on the sparse attention mechanism;

[0090] In this embodiment, first, the attention coefficients between each graph node and its corresponding neighbor nodes are calculated. Specifically, the attention coefficients are calculated by using the attention mechanism through the normalized node feature vectors and edge weights (determined by the normalized sparse adjacency matrix) of the graph node and its corresponding neighbor nodes.

[0091] After obtaining the attention coefficients between the graph node and each corresponding neighbor node, the normalized node feature vectors of each neighbor node are weighted and aggregated through an activation function to obtain the neighbor information of each graph node.

[0092] Step S54: For each modality channel, obtain the embedded feature representation of each graph node through the neighbor information of each graph node;

[0093] In this embodiment, the feature matrix of each modality channel is weighted and aggregated and updated through the neighbor information of each graph node to obtain the embedded feature representation of each graph node on each modality channel. This process is actually to update the features of the nodes through the neighbor information.

[0094] Step S55: Dynamically assign the weighting coefficients of each modality channel based on the global modality attention mechanism, and fuse the embedded feature representations of each modality channel based on the weighting coefficients to obtain the embedded vector of each graph node.

[0095] In this embodiment, when using the global modality attention mechanism, for each modality channel, the weighting coefficients are calculated according to the embedded feature representation of each graph node, and then the embedded feature representations of each modality channel are weighted and fused to obtain the embedded vector of each graph node. Therefore, this embedded vector fuses the embedded features of all modality channels and assigns the importance weights of each modality through the global modality attention mechanism, realizing the weighted integration of multi-modal information.

[0096] Step S6: Classify the embedded vector based on diffusion mapping to obtain the classification result of the density defect to be detected, and the classification result includes concrete non-compactness defect, filling block stone defect, and hole slag defect.

[0097] In step S6, the obtaining of the classification result of the density defect to be detected includes:

[0098] Step S61: Construct a diffusion operator based on the embedding vectors, perform eigen-decomposition based on the diffusion operator, and map each embedding vector to a manifold space point in the low-dimensional manifold space, where the manifold space points include the detected and to-be-detected manifold space points;

[0099] In this embodiment, based on the embedding vectors of each graph node, a similarity matrix between graph nodes is constructed using a Gaussian kernel function, and then a degree matrix is constructed through the similarity matrix. The diffusion operator is obtained based on the Markov diffusion process, specifically as follows:

[0100] ;

[0101] In the formula, represents the diffusion operator, represents the degree matrix, represents the similarity matrix.

[0102] Perform eigen-decomposition on the diffusion operator to obtain multiple eigenvectors and corresponding eigenvalues, which constitute the manifold space points in the low-dimensional manifold space, specifically as follows:

[0103] ;

[0104] In the formula, represents the coordinate of graph node in the low-dimensional manifold space under the hyperparameter , represents the th eigenvalue, represents the th component of the eigenvector corresponding to the th eigenvalue, where the hyperparameter is used to control the intensity of diffusion.

[0105] Step S62: Use kernel density estimation and the detected manifold space points to construct the probability density function of each defect category in the low-dimensional manifold space;

[0106] In this embodiment, for each defect category, the detected manifold space points (i.e., the manifold space points corresponding to the detected density defects) are used to estimate the probability density function of this defect category in the low-dimensional manifold space. In this step, a Gaussian kernel function is used for estimation, and the specific formula is:

[0107] ;

[0108] In the formula, represents the probability density function of defect category , represents defect category The number of detected manifold space points, represents the bandwidth parameter, which is used to control the width of the kernel function, represents the dimension of the low-dimensional manifold space, represents the defect category the coordinates of the detected manifold space points, represents the coordinates of the manifold space points to be detected, represents the Euclidean distance.

[0109] Step S63: Calculate the probability density of the manifold space points to be detected for each defect category based on the probability density function, and perform normalization processing on the probability density to obtain the normalized category probability;

[0110] In this embodiment, the probability density of the manifold space points to be detected for each defect category is calculated through the probability density function. In order to make the sum of the probability densities of all defect categories equal to 1, the probability densities of each defect category for each point to be detected are normalized to obtain the normalized category probability for each defect category.

[0111] Step S64: Select the defect category with the highest normalized category probability as the defect category of the manifold space points to be detected, and obtain the classification result of the compactness defect to be detected.

[0112] In summary, the present invention conducts preliminary detection on the tunnel lining through ground penetrating radar to quickly locate potential compactness defect areas, and then combines multi-modal sensing technologies such as polarization response, infrared active thermography, and acoustic impact to obtain multi-source response data of the lining, significantly improving the comprehensiveness and accuracy of defect identification. Aiming at the scale and semantic differences between multi-modal data, a unified feature extraction and coding strategy is proposed to obtain multi-modal embedding representations, and a three-way heterogeneous graph of modality - part - frequency band is constructed to effectively characterize the differences of defect responses in space, frequency domain, and modality.

[0113] In addition, the present invention further introduces a cross-modal causal learning mechanism, jointly models the time-delay correlation between different modalities through mutual information and Granger causal analysis, constructs a graph weight matrix with reasonable structure and accurate weights, enhances the discriminability and interpretability of features while improving the graph modeling expression ability. Through heterogeneous graph modeling and cross-modal contrast learning mechanism, not only can the complementary information of each modality be fused, but also the potential temporal causal structure can be fully explored. By introducing defect response distribution modeling based on kernel density estimation, the probability density distribution characteristics of different types of compactness defects in the embedding space can be accurately characterized, realizing boundary modeling of the defect area and adaptive setting of the discrimination threshold, and improving the refinement level of defect classification.

[0114] Embodiment 2:

[0115] AsFigure 2 As shown in Figure 2 , this embodiment provides a tunnel lining compactness defect detection system based on multi-modal data. The system includes:

[0116] A preliminary detection unit for preliminarily detecting the quality of the tunnel lining through a ground penetrating radar to obtain the compactness defects to be detected;

[0117] An acquisition unit for acquiring multi-modal data of the detected and to-be-detected compactness defects. The multi-modal data includes polarization response data, infrared active thermographic data, and acoustic shock response data;

[0118] An encoding unit for performing feature extraction and encoding processing on the multi-modal data to obtain a multi-modal embedding representation;

[0119] A modeling unit for performing heterogeneous graph modeling and cross-modal contrast learning based on the multi-modal embedding representation to obtain a graph structure;

[0120] A joint processing unit for jointly processing the graph structure based on a sparse attention mechanism and multi-modal fusion to obtain an embedding vector;

[0121] A classification unit for classifying the defect categories of the embedding vector based on diffusion mapping to obtain the classification results of the to-be-detected compactness defects. The classification results include concrete non-compactness defects, filling block stone defects, and tunnel slag defects.

[0122] The modeling unit includes:

[0123] A first construction subunit for constructing a ternary heterogeneous graph of each compactness defect with respect to modality - part - frequency band based on the multi-modal embedding representation;

[0124] An extraction subunit for extracting the time-delay correlation between different modalities based on an improved multi-modal causal discovery algorithm and generating a weight matrix for the ternary heterogeneous graph based on the time-delay correlation;

[0125] A correction subunit for performing physical constraint pruning and correction on the ternary heterogeneous graph based on the physical properties and structural design of the tunnel lining to obtain a multi-modal response graph;

[0126] A second construction subunit for optimizing the representation consistency and structural contrast ability in the multi-modal response graph through cross-modal contrast learning, and constructing a graph structure with each compactness defect as a graph node and the similarity between defects as the edge weight.

[0127] The joint processing unit includes:

[0128] A processing subunit for performing normalization processing on the graph structure to generate a sparse graph representation. The normalization processing includes node feature normalization, edge weight matrix sparsification, and structural adjacency matrix standardization;

[0129] A third construction subunit, configured to construct modal channels for each modality based on a sparse graph representation;

[0130] A first calculation subunit, configured to dynamically calculate and aggregate neighbor information of each graph node in the sparse graph representation based on a sparse attention mechanism;

[0131] A second calculation subunit, configured to, for each modal channel, obtain an embedded feature representation of each graph node through the neighbor information of each graph node;

[0132] A third calculation subunit, configured to dynamically allocate a weighting coefficient for each modal channel based on a global modal attention mechanism, and fuse the embedded feature representations of each modal channel based on the weighting coefficient to obtain an embedded vector of each graph node.

[0133] The classification unit includes:

[0134] A decomposition subunit, configured to construct a diffusion operator based on the embedded vector, perform eigen-decomposition based on the diffusion operator, and map each embedded vector to a manifold space point in a low-dimensional manifold space, where the manifold space points include detected and to-be-detected manifold space points;

[0135] A fourth calculation subunit, configured to construct a probability density function of each defect category on the low-dimensional manifold space using kernel density estimation and the detected manifold space points;

[0136] A fifth calculation subunit, configured to calculate the probability density of the to-be-detected manifold space points on each defect category based on the probability density function, and perform normalization processing on the probability density to obtain a normalized category probability;

[0137] A classification subunit, configured to select the defect category with the highest normalized category probability as the defect category of the to-be-detected manifold space point, and obtain a classification result of the to-be-detected compactness defect.

[0138] It should be noted that regarding the system in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0139] The above are only the preferred embodiments of the present invention, and are not used to limit the present invention. For those skilled in the art, the present invention may have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

[0140] As described above, it is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for detecting the compactness defect of tunnel lining based on multi-modal data, characterized in that, Including: Conduct preliminary quality inspection on the tunnel lining through ground penetrating radar to obtain the compactness defects to be inspected; Obtain multi-modal data of the inspected and to-be-inspected compactness defects, where the multi-modal data includes polarization response data, infrared active thermographic data, and acoustic shock response data; Extract features and perform encoding processing on the multi-modal data to obtain multi-modal embedding representations; Based on the multi-modal embedding representations, perform heterogeneous graph modeling and cross-modal contrast learning to obtain a graph structure; Based on the sparse attention mechanism and multi-modal fusion, jointly process the graph structure to obtain an embedding vector; Based on diffusion mapping, classify the embedding vector for defect categories to obtain the classification results of the to-be-inspected compactness defects, where the classification results include concrete non-compactness defects, filled blockstone defects, and tunnel slag defects; The step of performing heterogeneous graph modeling and cross-modal contrast learning based on the multi-modal embedding representations to obtain a graph structure includes: Based on the multi-modal embedding representations, construct a ternary heterogeneous graph for each compactness defect regarding modality - part - frequency band; Based on an improved multi-modal causal discovery algorithm, extract the time-delay correlation between different modalities, and generate a weight matrix for the ternary heterogeneous graph based on the time-delay correlation; Based on the physical properties and structural design of the tunnel lining, perform physical constraint pruning and correction on the ternary heterogeneous graph to obtain a multi-modal response graph; Through cross-modal contrast learning, optimize the representation consistency and structural contrast ability in the multi-modal response graph, and construct a graph structure with each compactness defect as a graph node and the similarity between defects as the edge weight.

2. The method for detecting the compactness defect of tunnel lining based on multimodal data according to claim 1, wherein The step of obtaining the multi-modal embedding representations includes: Perform polarization feature analysis on the polarization response data to obtain a polarization response vector; Based on the infrared active thermographic data, construct a temperature gradient tensor, and extract a thermal anomaly response vector through the temperature gradient tensor; Based on the acoustic shock response data, construct an impact echo time-delay spectrum, and extract an acoustic shock response vector through the impact echo time-delay spectrum, where the acoustic shock response vector includes a reflection amplitude statistical matrix and an energy distribution texture feature; Encode the polarization response vector, thermal anomaly response vector, and acoustic shock response vector through a multi-modal autoencoder to obtain multi-modal embedding representations.

3. The method for detecting the compactness defect of tunnel lining based on multimodal data according to claim 2, wherein The generation steps of the weight matrix are: Align the polarization response vector, thermal anomaly response vector, and acoustic shock response vector to obtain aligned vectors of multiple modalities; Perform sliding window segmentation on the aligned vectors to obtain continuous vector segments; Calculate the mutual information between different modalities based on the continuous vector segments; Perform Granger causality analysis based on the continuous vector segments to obtain the causality strength between different modalities; Construct the time-delay correlation between different modalities through the mutual information and causality strength, and use the time-delay correlation as the edge weight to construct the weight matrix of the ternary heterogeneous graph.

4. The method for detecting the compactness defect of tunnel lining based on multimodal data according to claim 1, wherein The step of obtaining the embedding vector includes: Perform normalization processing on the graph structure to generate a sparse graph representation, where the normalization processing includes node feature normalization, edge weight matrix sparsification, and structural adjacency matrix standardization; Based on the sparse graph representation, construct the modal channels of each modality; Based on the sparse attention mechanism, dynamically calculate and aggregate the neighbor information of each graph node in the sparse graph representation; For each modal channel, the embedded feature representation of each graph node is obtained through the neighbor information of each graph node; Based on the global modal attention mechanism, the weighted coefficients of each modal channel are dynamically assigned, and the embedded feature representations of each modal channel are fused based on the weighted coefficients to obtain the embedded vector of each graph node.

5. The method for detecting the compactness defect of tunnel lining based on multi-modal data according to claim 1, characterized in that , the obtaining of the classification result of the detected compactness defect includes: Based on the embedded vector, a diffusion operator is constructed, and eigenvalue decomposition is performed based on the diffusion operator to map each embedded vector into a manifold space point in the low-dimensional manifold space, where the manifold space points include the detected and to-be-detected manifold space points; Using kernel density estimation and the detected manifold space points to construct the probability density function of each defect category in the low-dimensional manifold space; Based on the probability density function, calculate the probability density of the to-be-detected manifold space points for each defect category, and perform normalization processing on the probability density to obtain the normalized category probability; Select the defect category with the highest normalized category probability as the defect category of the to-be-detected manifold space point to obtain the classification result of the detected compactness defect.

6. A tunnel lining compactness defect detection system based on multimodal data, characterized in that, including: A preliminary detection unit for performing preliminary quality detection on the tunnel lining through ground-penetrating radar to obtain the detected compactness defect; An acquisition unit for acquiring multi-modal data of the detected and to-be-detected compactness defects, where the multi-modal data includes polarization response data, infrared active thermal imaging data, and acoustic shock response data; An encoding unit for performing feature extraction and encoding processing on the multi-modal data to obtain a multi-modal embedded representation; A modeling unit for performing heterogeneous graph modeling and cross-modal contrast learning based on the multi-modal embedded representation to obtain a graph structure; A joint processing unit for jointly processing the graph structure based on the sparse attention mechanism and multi-modal fusion to obtain an embedded vector; A classification unit for classifying the defect categories of the embedded vector based on diffusion mapping to obtain the classification result of the detected compactness defect, where the classification result includes concrete non-compactness defects, filling blockstone defects, and cavity slag defects; The modeling unit includes: A first construction subunit for constructing a three-way heterogeneous graph of each compactness defect with respect to modality-site-frequency band based on the multi-modal embedded representation; An extraction subunit for extracting the time-delay correlation between different modalities based on an improved multi-modal causal discovery algorithm and generating the weight matrix of the three-way heterogeneous graph based on the time-delay correlation; A correction subunit for performing physical constraint pruning and correction on the three-way heterogeneous graph based on the physical properties and structural design of the tunnel lining to obtain a multi-modal response graph; A second construction subunit for optimizing the representation consistency and structural contrast ability in the multi-modal response graph through cross-modal contrast learning, and constructing a graph structure with each compactness defect as a graph node and the similarity between defects as the edge weight.

7. The tunnel lining compactness defect detection system based on multimodal data according to claim 6, characterized in that The joint processing unit includes: A processing subunit for performing normalization processing on the graph structure to generate a sparse graph representation, where the normalization processing includes node feature normalization, edge weight matrix sparsification, and structural adjacency matrix standardization; A third construction subunit for constructing the modal channels of each modality based on the sparse graph representation; The first computing subunit is configured to dynamically calculate and aggregate the neighbor information of each graph node in the sparse graph representation based on the sparse attention mechanism; The second computing subunit is configured to, for each modality channel, obtain the embedded feature representation of each graph node through the neighbor information of each graph node; The third computing subunit is configured to dynamically allocate the weighted coefficients of each modality channel based on the global modality attention mechanism, and fuse the embedded feature representations of each modality channel based on the weighted coefficients to obtain the embedded vector of each graph node.

8. The tunnel lining compactness defect detection system based on multimodal data according to claim 6, characterized in that, The classification unit includes: The decomposition subunit is configured to construct a diffusion operator based on the embedded vector, perform eigen-decomposition based on the diffusion operator, and map each embedded vector to a manifold space point in the low-dimensional manifold space, where the manifold space points include the detected and to-be-detected manifold space points; The fourth computing subunit is configured to construct the probability density function of each defect category on the low-dimensional manifold space using kernel density estimation and the detected manifold space points; The fifth computing subunit is configured to calculate the probability density of the to-be-detected manifold space point on each defect category based on the probability density function, and perform normalization processing on the probability density to obtain the normalized category probability; The classification subunit is configured to select the defect category with the highest normalized category probability as the defect category of the to-be-detected manifold space point, and obtain the classification result of the to-be-detected compactness defect.

Images