Bridge crack intelligent diagnosis system based on multi-modal data fusion

Abstract

The invention belongs to the technical field of bridge diagnosis, and discloses a bridge crack intelligent diagnosis system based on multi-modal data fusion. By fusing multi-source data such as visual images, sound wave detection and vibration signals, comprehensive perception and characterization of crack features are realized; constructing a bridge crack characteristic spectrum diagram by adopting a cross-modal feature extraction and heterogeneous feature coding technology; generating a crack evolution situation map based on space-time correlation analysis and knowledge graph construction; the robustness of the system in a complex environment is improved through environmental adaptability feature enhancement and multi-scale characterization; constructing a bridge safety risk hypergraph in combination with multi-dimensional risk analysis and multi-agent collaborative diagnosis; analyzing and revealing a crack evolution mechanism by applying a causal relationship; and finally, through dynamic fusion and uncertainty quantification, a crack intelligent diagnosis comprehensive report is generated. According to the system, the limitation of traditional single-mode diagnosis is broken through, and dynamic prediction and accurate risk assessment of fracture evolution are realized.

Description

Technical Field

[0001] This invention relates to the field of bridge diagnostic technology, and more specifically, to a bridge crack intelligent diagnostic system based on multimodal data fusion. Background Technology

[0002] As a critical node in transportation networks, the safety of bridge structures directly impacts national economic development and the safety of people's lives and property. With the rapid development of infrastructure construction in my country, the number of bridges continues to increase, and the problem of bridge structural cracks is becoming increasingly prominent. Cracks, as a typical manifestation of bridge structural damage, contain rich information about structural safety in their occurrence, development, and expansion, making them an important indicator for assessing bridge health.

[0003] Traditional diagnostic methods rely excessively on single-modal data, such as visual images from manual inspections or simple vibration monitoring, resulting in fragmented and partial information acquisition, akin to "the blind men and the elephant," failing to fully grasp the essential characteristics of cracks. In practical applications, the complex and variable environment of bridges (such as extreme weather, drastic temperature changes, and traffic load fluctuations) leads to poor diagnostic stability, often resulting in the embarrassing situation of "accurate in sunny weather, ineffective in rainy weather." Current technologies for understanding cracks mostly remain at the static and superficial level, lacking a deep understanding of the dynamic process of crack evolution, making it difficult to predict crack development trends, and causing maintenance decisions to lag behind damage development. The information barriers between multi-source heterogeneous data are prominent, with each monitoring system operating independently, resulting in severe data silos and preventing effective information fusion and complementary enhancement. Risk assessment methods are fragmented and experience-based, lacking a systematic multi-dimensional risk analysis framework, leading to the safety management dilemma of "small problems invisible, big problems too late to address." Insufficient understanding of the causal mechanisms of crack generation and development makes it difficult to fundamentally analyze the causes and evolution patterns, resulting in maintenance solutions that are often "treating the symptoms but not the root cause." The uncertainty of diagnostic results lacks effective quantification and expression, making it difficult for decision-makers to weigh risks and benefits, resulting in inappropriate allocation of maintenance resources. These problems are particularly prominent in important projects such as high-speed railway bridges and extra-large cross-river and cross-sea bridges, severely restricting the intelligent and precise development of bridge health monitoring and maintenance.

[0004] In view of this, the present invention proposes a bridge crack intelligent diagnosis system based on multimodal data fusion to solve the above problems. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art and to achieve the above objectives, the present invention provides the following technical solution: a bridge crack intelligent diagnosis system based on multimodal data fusion, comprising: The data acquisition and preprocessing module is used to acquire and preprocess multimodal data of the bridge structure to obtain the original multimodal feature set. The cross-modal feature extraction module is used to perform cross-modal feature extraction and heterogeneous feature encoding based on the original multimodal feature set to obtain a bridge crack feature spectrum map; The spatiotemporal correlation analysis module is used to perform spatiotemporal correlation analysis and construct a crack knowledge graph based on the bridge crack feature spectrum diagram to obtain a crack evolution status diagram. The environmental adaptability feature enhancement module is used to enhance the environmental adaptability features and perform multi-scale characterization of the crack state based on the crack evolution status map, obtain the crack multi-scale feature space, and construct a diagnostic engine. The safety risk analysis module is used to perform multi-dimensional safety risk analysis and multi-agent collaborative diagnosis of bridge structures, and to obtain a bridge safety risk hypermap. The causal relationship analysis module is used to perform causal relationship analysis on bridge cracks based on the bridge safety risk hypergraph to obtain the crack evolution causal network. The diagnostic result fusion module is used to dynamically fuse and quantify the uncertainty of the diagnostic results based on the crack evolution causal network and the diagnostic engine to obtain a comprehensive intelligent diagnostic report of the crack; the modules are connected to each other by wired and / or wireless means to realize data transmission between the modules.

[0006] Preferably, the multimodal data acquisition and preprocessing of the bridge structure to obtain a multimodal original feature set includes: Visual images, acoustic waves, and vibration signals of the bridge structure are acquired to obtain a multi-source heterogeneous dataset. Noise is filtered out and missing values ​​are repaired in the multi-source heterogeneous dataset to obtain a cleaned dataset. The cleaned dataset is subjected to time series synchronization processing to obtain aligned multimodal data, and outlier detection is performed based on the aligned multimodal data to obtain a standardized feature matrix; The standardized feature matrix is ​​subjected to a mode-specific transformation to obtain an enhanced feature set. Based on the enhanced feature set, mutual information is calculated for different modal features to obtain a modality correlation matrix. The modality correlation matrix is ​​evaluated for feature importance to obtain the feature weight distribution. Based on the feature weight distribution, features are adaptively selected to obtain an optimized feature set. The optimized feature set is grouped within each modality to obtain a modal feature clustering structure, and a multimodal original feature set is constructed based on the modal feature clustering structure.

[0007] Preferably, the step of performing cross-modal feature extraction and heterogeneous feature encoding based on the multimodal original feature set to obtain a bridge crack feature hierarchy map includes: Deep feature learning is performed on the original multimodal feature set to obtain intramodal deep feature representations, and a cross-modal attention mapping network is constructed based on the intramodal deep feature representations to obtain the correspondence of key features; Heterogeneous feature fusion is performed on the correspondence of the key features to obtain a unified feature space representation, and crack feature embedding is performed based on the unified feature space representation to obtain crack feature vectors; Spectral clustering analysis is performed on the crack feature vectors to obtain crack feature families, and based on the crack feature families, the similarity of features of different crack types is calculated to obtain a feature similarity matrix; The feature similarity matrix is ​​transformed into a graph structure to obtain an initial crack feature map. Based on the initial crack feature map, the relationships between nodes are adaptively weighted to obtain a weighted crack feature map. Multi-level community detection is performed on the weighted crack feature map to obtain the crack feature community structure. Based on the crack feature community structure, hierarchical evolution analysis is performed on the relationship between communities to obtain the bridge crack feature spectrum map.

[0008] Preferably, the step of performing spatiotemporal correlation analysis and constructing a crack knowledge graph based on the bridge crack feature spectrum to obtain a crack evolution status map includes: A topological analysis was performed on the bridge crack feature spectrum diagram to obtain a set of key nodes. Based on the set of key nodes, time series features of the bridge cracks were extracted to obtain crack evolution feature vectors. Spatiotemporal encoding is performed on the crack evolution feature vector to obtain a crack development identifier. Based on the crack development identifier and the bridge crack feature spectrum diagram, dynamic crack labeling is performed to obtain labeled data with evolution attributes. By combining the knowledge of experts in the engineering field, a crack type ontology is established to obtain a crack knowledge base. Based on the crack knowledge base and the labeled data, a crack knowledge graph is constructed to obtain an initial crack knowledge structure. Spatiotemporal association rule mining is performed on the initial crack knowledge structure to obtain a crack evolution pattern library. Based on the crack evolution pattern library, a graph structure model is performed on the crack development trend to obtain a crack evolution path graph. The spatiotemporal graph fusion processing of the crack evolution path map and the bridge crack feature spectrum map is performed to obtain a multi-dimensional crack situation representation. Based on the multi-dimensional crack situation representation, trend prediction analysis is performed to obtain a crack evolution situation map.

[0009] Preferably, the step of enhancing the environmental adaptability features and performing multi-scale characterization of the crack state based on the crack evolution status map to obtain a multi-scale feature space of the crack, and constructing a diagnostic engine, includes: The crack evolution situation map is segmented into multiple environment-specific sub-maps, and environmental influencing factors are extracted based on the multiple environment-specific sub-maps to obtain environmental feature vectors; The environmental feature vector is adaptively enhanced to obtain an environmental adaptability representation, and the environmental adaptability representation is decomposed into multiple scales to obtain crack feature representations at different scales. By fusing the crack feature representations at different scales, a multi-scale crack feature space is constructed, and boundary learning is performed in the multi-scale crack feature space to obtain the crack severity decision boundary. Based on the decision boundary of the severity of the crack, fuzzy reasoning is performed on the assessment rules under different environmental conditions to obtain an environmental adaptability diagnostic knowledge base; A diagnostic strategy network is created based on the environmental adaptability diagnostic knowledge base, and the diagnostic process is optimized by reinforcement learning based on the diagnostic strategy network and the crack multi-scale feature space to obtain an environmentally adaptive diagnostic engine.

[0010] Preferably, the step of performing multi-dimensional safety risk analysis and multi-agent collaborative diagnosis of the bridge structure to obtain a bridge safety risk hypermap includes: The safety risk factors of the bridge structure are decomposed hierarchically to obtain the risk factor hierarchy, and the structural importance analysis is performed based on the risk factor hierarchy to obtain the risk weight vector. The risk weight vector is subjected to safety assessment index association rule mining to obtain the assessment index association network, and a risk assessment function cluster is generated based on the assessment index association network; based on the risk assessment function cluster, a diagnostic agent specifically designed for different types of cracks is designed to obtain an initial multi-agent diagnostic model, and the initial multi-agent diagnostic model is collaboratively optimized to obtain an agent cooperation strategy set. Based on the set of agent cooperation strategies, federated learning is performed on the diagnostic results among agents to obtain an optimized multi-agent cooperative diagnostic model. Uncertainty quantification is performed on the optimized multi-agent collaborative diagnostic model to obtain the diagnostic confidence distribution. Based on the diagnostic confidence distribution, hypergraph modeling and hyperedge weight calculation are performed on the risk assessment results to obtain the bridge safety risk hypergraph.

[0011] Preferably, the step of performing causal relationship analysis on bridge cracks based on the bridge safety risk hypergraph to obtain a crack evolution causal network includes: The bridge safety risk hypergraph is decomposed into multiple risk subgraphs, and the evolution process of bridge cracks is analyzed by time-series correlation analysis based on the multiple risk subgraphs to obtain the crack evolution sequence. The crack evolution sequence is processed by a causal discovery algorithm to obtain a causal intensity matrix of crack state transitions. Based on the causal intensity matrix, the causal direction of the crack evolution path is determined to obtain a set of directed causal relationships. Structural learning is performed on the directed causal relationship set to obtain an initial crack causal graph model. Based on the initial crack causal graph model, physical constraints are introduced to perform causal inference to obtain a physically enhanced complete causal graph. The complete causal graph of the physical enhancement is verified and tested to obtain the crack evolution backbone network. Based on the crack evolution backbone network, the nonlinear relationship between nodes is fitted by deep learning to obtain a quantitative causal relationship function. Sensitivity analysis was performed on the quantitative causal relationship function to obtain key causal paths. Based on the key causal paths, the causal networks of various types of cracks were integrated to obtain the crack evolution causal network.

[0012] Preferably, the step of dynamically fusing and quantifying the uncertainty of the diagnostic results based on the crack evolution causal network and the diagnostic engine to obtain a comprehensive intelligent crack diagnosis report includes: The crack evolution causal network is topologically sorted to obtain a crack evolution stage sequence. Based on the crack evolution stage sequence, the output of the diagnostic engine is temporally aligned to obtain an aligned diagnostic result set. Multimodal evidence theory fusion is performed on the aligned diagnostic result set to obtain diagnostic features with different confidence levels. Based on the diagnostic features with different confidence levels, Bayesian uncertainty quantification is performed on the diagnostic results to obtain the diagnostic confidence distribution matrix. The diagnostic confidence distribution matrix is ​​subjected to deep probabilistic encoding to obtain the confidence interval of the diagnostic results, and the diagnostic results are dynamically weighted and fused according to the confidence interval to obtain a preliminary fusion report; The preliminary fusion report is optimized using reinforcement learning to obtain the optimal maintenance decision recommendation. Based on the optimal maintenance decision recommendation, a cost-benefit analysis is performed to obtain a priority list of maintenance solutions. The maintenance plan priority list and the preliminary fusion report are integrated to obtain a comprehensive intelligent crack diagnosis report, which includes crack type identification, severity assessment, evolution prediction, and maintenance recommendations.

[0013] A bridge crack intelligent diagnostic device based on multimodal data fusion is disclosed. The bridge crack intelligent diagnostic device based on multimodal data fusion includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor performs the implementation steps of the bridge crack intelligent diagnostic system based on multimodal data fusion.

[0014] A computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the implementation steps of the intelligent bridge crack diagnosis system based on multimodal data fusion.

[0015] The technical effects and advantages of the intelligent bridge crack diagnosis system based on multimodal data fusion of this invention are as follows: This invention achieves comprehensive perception and precise characterization of bridge crack features, breaking through the limitations of traditional single-modal diagnosis and significantly improving the accuracy and reliability of crack identification. By deeply exploring the spatiotemporal evolution of cracks, the system can proactively predict crack development trends, providing a scientific basis for bridge safety early warning. Under complex and changing environmental conditions, the system exhibits good environmental adaptability and robustness, ensuring the stability and consistency of diagnostic results. The innovative risk assessment mechanism achieves a leap from point-based monitoring to global assessment, making bridge safety status evaluation more systematic and refined. The deeply revealed causal mechanism of crack evolution provides theoretical support for precise governance and targeted maintenance, effectively reducing maintenance costs and extending the service life of bridges. The system's precise quantification of diagnostic uncertainty significantly improves the credibility of decision-making and risk control capabilities, providing bridge managers with a more reliable decision-making basis. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the intelligent bridge crack diagnosis system based on multimodal data fusion according to the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0019] This application provides a bridge crack intelligent diagnosis system based on multimodal data fusion. The executing entities of the bridge crack intelligent diagnosis system based on multimodal data fusion include, but are not limited to, the following: mechanical equipment, data processing platform, cloud server node, network upload device, etc., which can be considered as general computing nodes of this application. The data processing platform includes, but is not limited to, at least one of: image processing system, vibration analysis system, and cloud data management system.

[0020] Please see Figure 1This invention provides an intelligent bridge crack diagnosis system based on multimodal data fusion, comprising: The data acquisition and preprocessing module is used to acquire and preprocess multimodal data of the bridge structure to obtain the original multimodal feature set. The cross-modal feature extraction module is used to extract cross-modal features and encode heterogeneous features based on the original multimodal feature set to obtain a feature spectrum map of bridge cracks. The spatiotemporal correlation analysis module is used to perform spatiotemporal correlation analysis and construct a crack knowledge graph based on the bridge crack feature spectrum diagram to obtain a crack evolution status diagram. The environmental adaptability feature enhancement module is used to enhance the environmental adaptability features and multi-scale characterize the crack state based on the crack evolution status map, obtain the crack multi-scale feature space, and build a diagnostic engine. The safety risk analysis module is used to perform multi-dimensional safety risk analysis and multi-agent collaborative diagnosis of bridge structures, and to obtain a bridge safety risk hypermap. The causal relationship analysis module is used to perform causal relationship analysis on bridge cracks based on the bridge safety risk hypergraph, and obtain the causal network of crack evolution. The diagnostic results fusion module is used to dynamically fuse and quantify the uncertainty of diagnostic results based on the crack evolution causal network and diagnostic engine to obtain a comprehensive intelligent crack diagnostic report; the modules are connected to each other by wired and / or wireless means to realize data transmission between modules.

[0021] This invention acquires comprehensive information about bridge structures through multimodal data acquisition and preprocessing, and combines multi-source data such as visual images, acoustic wave detection, and vibration signals to provide a foundation for subsequent analysis. It utilizes cross-modal feature extraction and heterogeneous feature encoding to generate a bridge crack feature spectrum map, effectively integrating information from different modes to more comprehensively characterize crack features. Through spatiotemporal correlation analysis and knowledge graph construction, it obtains a crack evolution status map, gaining a deeper understanding of crack development patterns and spatiotemporal variation characteristics. It enhances the environmental adaptability of crack states and performs multi-scale characterization, constructing a diagnostic engine adapted to different environmental conditions to improve diagnostic robustness. It employs multi-dimensional safety risk analysis and multi-agent collaborative diagnosis to generate a safety risk hypergraph, comprehensively assessing the safety status of the bridge structure. Based on the safety risk hypergraph, it conducts causal relationship analysis of cracks, revealing the intrinsic mechanism and influencing factors of crack evolution. Finally, through dynamic fusion and uncertainty quantification, it generates an intelligent diagnostic comprehensive report, providing a scientific basis for bridge maintenance decisions.

[0022] In this embodiment of the invention, a bridge crack intelligent diagnosis system based on multimodal data fusion includes: The data acquisition and preprocessing module is used to acquire and preprocess multimodal data of the bridge structure to obtain the original multimodal feature set. In this embodiment, for the bridge structure, high-resolution camera equipment (such as 4K cameras and infrared thermal imagers) is deployed to acquire visual images of the bridge surface, capturing the morphological characteristics of visible cracks, including crack width, length, direction, and distribution location. Acoustic wave detection equipment (such as ultrasonic flaw detectors and acoustic emission sensors) is used to perform non-destructive testing on the bridge, obtaining acoustic characteristic data of the internal structure, including wave velocity, attenuation coefficient, and spectral characteristics. Vibration sensors (such as accelerometers and displacement sensors) are installed at key structural points of the bridge to collect vibration response signals of the bridge under natural excitation and traffic loads. This includes: amplitude, frequency, phase, etc. The collected multi-source heterogeneous datasets are initially organized to ensure a unified data format and the inclusion of necessary timestamps and spatial location information. Wavelet transform or median filtering methods are used to denoise the image data, removing noise interference caused by ambient lighting, weather conditions, etc. Frequency domain filters (such as bandpass filters) are applied to the acoustic data to remove environmental noise and interference signals generated by the equipment itself. Singular Spectrum Analysis (SSA) or Empirical Mode Decomposition (EMD) methods are used to filter noise from vibration signals. Missing values ​​that may occur during data acquisition are addressed using... Data restoration is performed using interpolation algorithms (such as linear interpolation and spline interpolation) or deep learning methods (such as GAN generative adversarial networks) to ensure the integrity of the dataset. Timestamp alignment techniques are used to synchronize data from different modalities according to their acquisition time, establishing a temporal correspondence between the multimodal data. Spatial registration techniques are applied to map data collected by different sensors to a unified spatial coordinate system, ensuring spatial consistency. Box plots or Local Outlier Factor (LOF) methods are used to detect outliers in the aligned multimodal data, identifying and marking data points that significantly deviate from the normal range. Outliers are processed (e.g., replaced, deleted, or marked) to ensure data reliability. Z-score or Min-Max standardization is performed on data from different modalities to ensure consistent scale across modalities, facilitating subsequent analysis. The standardized data is organized into a structured feature matrix, with each row representing a sampling point and each column representing a feature dimension. For image data, image enhancement techniques (e.g., histogram equalization, sharpening filtering) are applied to enhance the visual features of cracks. Time-frequency transformations (e.g., short-time Fourier transform, wavelet transform) are performed on acoustic data to extract richer frequency domain features.Modal analysis and spectral decomposition are performed on the vibration signal to extract modal parameters and frequency features. The enhanced modal features are organized into feature sets for subsequent analysis and processing. Mutual information calculation methods are used to evaluate the correlation between different modal features, constructing a modal correlation matrix to reflect the degree of information sharing among modes. Feature importance evaluation algorithms (such as random forest importance and Relief algorithm) are used to calculate the contribution of each feature to crack identification, generating a feature weight distribution. Based on the feature weight distribution, a threshold is set to filter important features, removing redundant and low-information features to achieve adaptive feature selection. Clustering algorithms such as K-means or DBSCAN are used to group the optimized feature set within modes, identifying feature subsets with similar characteristics to form a modal feature clustering structure. Based on the clustering results, the features of each mode are integrated to construct a multimodal original feature set, laying the foundation for subsequent cross-modal feature extraction.

[0023] The cross-modal feature extraction module is used to extract cross-modal features and encode heterogeneous features based on the original multimodal feature set to obtain a feature spectrum map of bridge cracks. In this embodiment, for the visual modality, a convolutional neural network (CNN) is applied to extract deep feature representations of the image, including the texture, shape, and contextual information of the crack. For the acoustic modality, a recurrent neural network (RNN) or a long short-term memory network (LSTM) is used to extract the temporal patterns and spectral features of the acoustic signal. For the vibration modality, an autoencoder or variational autoencoder (VAE) is used to learn the latent representation and dynamic features of the vibration signal. A cross-modal mapping network with an attention mechanism is constructed to realize the learning of the correspondence between features of different modalities. The attention weights reflect the importance of each modal feature in different contexts, identify the most critical feature elements for crack characterization in each modality, establish the correspondence between them, design a heterogeneous feature fusion module to map the features of different modalities to a unified feature space, and use tensor fusion or multi-view learning methods to retain the complementary information of each modality and reduce redundancy to generate a unified feature representation. In the unified feature space, dimensionality reduction techniques (such as t-SNE or UMAP) are used to embed the crack features, mapping high-dimensional features to a low-dimensional space. To form a more compact crack feature vector, a spectral clustering algorithm is applied to cluster the crack feature vector. Based on feature similarity, crack samples are grouped to form crack clusters with similar features. The similarity between features of different crack types is calculated, and a feature similarity matrix is ​​constructed using metrics such as cosine similarity or Mahalanobis distance. The feature similarity matrix is ​​then converted into a graph structure, where each node represents a type of crack feature, and the edges between nodes represent the similarity relationship between features. An initial crack feature map is constructed. Based on feature importance and similarity strength, adaptive weighting is applied to the relationships between nodes, with relationships between important features receiving higher weights. A weighted crack feature map is generated. A multi-level community detection algorithm (such as the Louvain method or Infomap algorithm) is applied to divide the weighted crack feature map into communities, identifying closely related feature groups and forming a crack feature community structure. The hierarchical relationship and evolution pattern between communities are analyzed to construct a phylogenetic structure reflecting the crack feature classification system. Finally, a bridge crack feature phylogenetic map is generated, which intuitively displays the classification system and correlation of crack features.

[0024] The spatiotemporal correlation analysis module is used to perform spatiotemporal correlation analysis and construct a crack knowledge graph based on the bridge crack feature spectrum diagram to obtain a crack evolution status diagram. In this embodiment, graph theory algorithms (such as centrality analysis and PageRank algorithm) are applied to perform topological structure analysis on the bridge crack feature spectrum diagram to identify key nodes (i.e., important crack features). These nodes have high connectivity or high influence in the network, forming a key node set. For crack features in the key node set, historical data on their changes over time are collected, and time series features such as trends, seasonality, periodicity, and abnormal patterns are extracted to form a crack evolution feature vector. A spatiotemporal encoder is designed to fuse and encode the spatial location information and temporal evolution features of the crack, generating a unique crack development identifier. This identifier contains the spatiotemporal information and evolutionary features of the crack. Based on the crack development identifier, nodes in the bridge crack feature spectrum diagram are dynamically marked, and attributes such as timestamps and evolution rates are added to generate labeled data with evolutionary attributes. In collaboration with experts in the field of bridge engineering, a crack type ontology is established, including knowledge of crack classification systems, causal mechanisms, development patterns, and severity, forming a structured crack ontology. A knowledge base is constructed by combining labeled data and a crack knowledge base. A crack knowledge graph is built, where nodes represent crack types, characteristics, and influencing factors, and edges represent the relationships between them, forming an initial crack knowledge structure. Association rule mining algorithms (such as Apriori or FP-Growth) are applied to analyze the association patterns of cracks in the temporal and spatial dimensions, discovering the laws and triggering conditions of crack evolution, and establishing a crack evolution pattern library. Based on the crack evolution pattern library, a directed graph model is used to model the development trend of cracks. Nodes represent crack states, and edges represent the probabilities and conditions of state transitions, constructing a crack evolution path graph. The crack evolution path graph is fused with the bridge crack feature spectrum graph to generate a multi-dimensional crack status representation, including crack characteristics, distribution, evolution laws, and influencing factors. Time series prediction models (such as ARIMA, Prophet, or deep learning models) are applied to predict and analyze the development trend of cracks, generating a crack evolution status map that intuitively displays the historical development, current state, and future trend of cracks.

[0025] The environmental adaptability feature enhancement module is used to enhance the environmental adaptability features and multi-scale characterize the crack state based on the crack evolution status map, obtain the crack multi-scale feature space, and build a diagnostic engine. In this embodiment, graph segmentation algorithms (such as spectral clustering and normalized segmentation) are used to segment the crack evolution map into subgraphs. Based on the similarity of environmental conditions, the graph is divided into multiple subgraphs, each representing the crack state under specific environmental conditions, forming multiple environment-specific subgraphs. Environmental influencing factors, such as temperature, humidity, load conditions, and material aging degree, are extracted from these subgraphs to construct environmental feature vectors, quantifying the impact of environmental factors on crack development. An adaptive feature enhancement network is designed to enhance crack features under different environmental conditions, improving the robustness and discriminative power of the features and generating an environmentally adaptive representation. Wavelet transform or multi-resolution analysis methods are applied to decompose the environmentally adaptive representation into multiple scales, capturing local details and global structural features of the crack at different scales, generating crack feature representations at different scales. A multi-scale feature fusion framework is designed to weightedly fuse crack features at different scales, retaining... Key information at each scale is used to construct a multi-scale feature space for cracks. Within this space, Support Vector Machines (SVM) or deep learning methods are applied to learn the decision boundaries for crack severity, distinguishing different crack states and forming crack severity decision boundaries. Based on fuzzy logic theory, a mapping relationship between environmental conditions and crack assessment rules is established, and a fuzzy inference system is designed to adaptively adjust assessment criteria according to different environmental conditions, generating an environmentally adaptive diagnostic knowledge base. Based on this knowledge base, a decision tree or Bayesian network is designed to construct a diagnostic strategy network, forming the decision process and inference mechanism for crack diagnosis. Reinforcement learning algorithms (such as Q-learning or deep Q-networks) are applied to optimize the diagnostic process, and a reward mechanism guides the system to learn the optimal diagnostic strategy, improving diagnostic accuracy. Ultimately, a diagnostic engine capable of adapting to different environmental conditions is formed, achieving accurate assessment and diagnosis of crack states.

[0026] The safety risk analysis module is used to perform multi-dimensional safety risk analysis and multi-agent collaborative diagnosis of bridge structures, and to obtain a bridge safety risk hypermap. In this embodiment, the Analytic Hierarchy Process (AHP) or structural decomposition technique is used to systematically decompose the safety risk factors of the bridge structure. Risk sources are identified from multiple dimensions such as materials, structure, environment, and load. A hierarchical structure of risk factors is constructed. Fault Tree Analysis (FTA) or Failure Mode and Effects Analysis (FMEA) methods are applied to assess the impact of each risk factor on bridge safety, calculate the importance weight of each risk factor, and form a risk weight vector. Association rule mining algorithms (such as Apriori or FP-Growth) are used to analyze the correlation between safety assessment indicators, discover the dependencies and synergistic effects between indicators, and construct an assessment indicator association network. Based on the assessment indicator association network, a community discovery algorithm is applied to identify functionally related indicator sets, forming risk assessment function clusters. Each function cluster is responsible for a specific type of risk assessment. For different types of cracks (such as fatigue cracks, temperature cracks, shrinkage cracks, etc.), specialized diagnostic agents are designed, each agent possessing specific... The knowledge base and reasoning mechanism constitute the initial multi-agent diagnostic model. A collaboration mechanism among agents is designed, including strategies for information sharing, task allocation, and result fusion. Communication protocols and collaboration rules between agents are established, forming a set of agent collaboration strategies. A federated learning framework is adopted to collaboratively train distributed agents while protecting data privacy. Each agent trains its model on local data and then shares model parameters instead of the original data. Through multiple rounds of iterative optimization, an optimized multi-agent collaborative diagnostic model is formed. Bayesian methods or ensemble learning techniques are applied to quantify the uncertainty of the multi-agent diagnostic results, calculate the confidence interval and reliability index of the diagnostic results, and generate a diagnostic credibility distribution. Based on hypergraph theory, a bridge safety risk hypergraph is constructed, where nodes represent risk factors, and hyperedges represent higher-order relationships between multiple factors. Hyperedge weights are calculated to reflect the severity of combinations of risk factors, ultimately forming a bridge safety risk hypergraph that comprehensively characterizes the safety risk status of the bridge structure.

[0027] The causal relationship analysis module is used to perform causal relationship analysis on bridge cracks based on the bridge safety risk hypergraph, and obtain the causal network of crack evolution. In this embodiment, a hypergraph decomposition algorithm is applied to decompose the bridge safety risk hypergraph into multiple subgraphs. Each subgraph represents a specific risk type or the risk state of a specific area, forming multiple risk subgraphs. Crack data at different time points are collected to construct a time series of crack states. Temporal correlation analysis methods (such as dynamic time warping and Granger causality test) are applied to analyze the temporal dependencies in the crack evolution process, generating a crack evolution sequence. Causal discovery algorithms (such as PC algorithm, FCI algorithm, or NOTEARS) are applied to process the crack evolution sequence, identifying causal relationships between variables, calculating causal strength, and constructing a causal strength matrix for crack state transitions. Based on the causal strength matrix, the causal direction of the crack evolution path is determined using the Directed Acyclic Graph (DAG) representation, identifying driving and response factors in the causal relationships, forming a directed causal relationship set. A structural learning algorithm (such as Bayesian network structure learning) is applied to the directed causal relationship set. Optimization is performed by learning the conditional independence and dependencies between variables, constructing an initial crack causal graph model, and introducing physical constraints for causal inference by combining the principles of bridge structural mechanics and materials science knowledge to ensure that causal relationships conform to physical laws, forming a physically enhanced complete causal graph. Through experimental verification or comparison with historical data, the predictive and explanatory power of the causal graph is tested, key causal paths are screened, and a crack evolution backbone network is formed. Deep learning methods (such as neural networks or Gaussian processes) are applied to fit the nonlinear causal relationships between nodes, quantify the intensity and form of causal influence, generate a quantitative causal relationship function, perform sensitivity analysis on the quantitative causal relationship function, identify the factors and paths that have the most significant impact on crack evolution, determine key causal paths, integrate the causal networks of different types of cracks (such as fatigue cracks, temperature cracks, shrinkage cracks, etc.), construct a unified crack evolution causal network, and comprehensively characterize the internal mechanism and influencing factors of crack development.

[0028] The diagnostic results fusion module is used to dynamically fuse and quantify the uncertainty of diagnostic results based on the crack evolution causal network and diagnostic engine, so as to obtain a comprehensive intelligent crack diagnosis report. In this embodiment, a topological sorting algorithm is applied to sort the causal network of crack evolution, determine the stage sequence of crack development, and form a crack evolution stage sequence. The diagnostic results output by the diagnostic engine are time-aligned with the crack evolution stage sequence to ensure consistency between the diagnostic results and the crack development stages, generating an aligned diagnostic result set. Dempster-Shafer evidence theory or fuzzy set theory is applied to fuse multimodal diagnostic results, handle the conflict and complementarity of different modal evidence, generate diagnostic features with different confidence levels, and use Bayesian methods to quantify the uncertainty of diagnostic results, calculate the posterior probability and confidence interval of the diagnostic conclusion, construct a diagnostic confidence distribution matrix, and apply deep probabilistic coding methods (such as variational autoencoders or Bayesian neural networks) to encode the diagnostic results. The system generates probability distributions and confidence intervals for diagnostic results. Based on the confidence interval information, it dynamically weights and fuses diagnostic results from different sources, assigning higher weights to results with higher confidence levels to form a preliminary fusion report. Reinforcement learning algorithms (such as policy gradient or Actor-Critic methods) are then applied to optimize the maintenance decision-making process, considering factors such as crack condition, safety risks, maintenance costs, and service life to generate optimal maintenance decision recommendations. Cost-benefit analyses are performed on different maintenance schemes to evaluate the input-output ratio and risk reduction effect of each scheme, resulting in a priority list of maintenance schemes. Finally, the system integrates diagnostic results, uncertainty analysis, and maintenance recommendations to generate a comprehensive intelligent crack diagnostic report, including crack type identification, severity assessment, evolution prediction, and maintenance recommendations, providing comprehensive support for bridge maintenance decisions.

[0029] In this embodiment of the invention, the data acquisition and preprocessing module is used to acquire and preprocess multimodal data of the bridge structure to obtain a multimodal raw feature set, specifically for: Visual images, acoustic waves, and vibration signals of the bridge structure were acquired to obtain a multi-source heterogeneous dataset. Noise was filtered out and missing values ​​were repaired from the multi-source heterogeneous dataset to obtain a cleaned dataset. The cleaned dataset is time-series synchronized to obtain aligned multimodal data, and outlier detection is performed based on the aligned multimodal data to obtain a standardized feature matrix. The standardized feature matrix is ​​subjected to mode-specific transformation to obtain an enhanced feature set. Based on the enhanced feature set, mutual information is calculated for different modal features to obtain the modality correlation matrix. The modality correlation matrix is ​​evaluated for feature importance to obtain the feature weight distribution. Based on the feature weight distribution, features are adaptively selected to obtain the optimized feature set. The optimized feature set is grouped within each modality to obtain the modal feature clustering structure, and then the multimodal original feature set is constructed based on the modal feature clustering structure.

[0030] In this embodiment, high-resolution cameras (such as 4K or 8K cameras) are used to acquire image data of the bridge surface under different angles and lighting conditions to ensure coverage of all possible crack areas. A drone equipped with a camera is used to acquire images of hard-to-reach areas, improving the comprehensiveness of data acquisition. Ultrasonic detection equipment is deployed to perform acoustic wave detection on the bridge. By analyzing the propagation characteristics of sound waves in the material, internal defects and cracks are identified. Acoustic emission sensors are used to monitor the acoustic signals generated by crack activity, capturing the dynamic development process of micro-cracks. Accelerometers, displacement sensors, and strain gauges are installed at key locations on the bridge to collect the bridge's vibration response under environmental excitation and traffic loads, recording the dynamic characteristics and deformation of the structure. The acquired image, acoustic wave, and vibration data are integrated into a multi-source heterogeneous dataset, ensuring that the data includes necessary timestamps and spatial location information. For image data, image processing algorithms (such as median filtering, Gaussian filtering, or wavelet transform) are applied to reduce noise and remove interference caused by uneven ambient lighting, shadows, and stains. For acoustic data, frequency domain filtering (such as bandpass filtering) is used to remove environmental noise and interference signals generated by the equipment itself, extracting effective acoustic features. For vibration signals, signal processing techniques (such as empirical mode decomposition and singular value decomposition) are used to filter out noise and separate structural response from environmental interference. For missing values ​​that appear during data acquisition, appropriate repair methods are selected according to the data type. For example, image data can use image inpainting algorithms (such as deep learning-based image inpainting), and acoustic and vibration data can use time series interpolation methods (such as linear interpolation, spline interpolation, or deep learning-based generative models) to generate a cleaned, high-quality dataset, providing a reliable foundation for subsequent analysis.

[0031] In this embodiment, the cross-modal feature extraction module is used to perform cross-modal feature extraction and heterogeneous feature encoding based on the original multimodal feature set to obtain a bridge crack feature hierarchy map, specifically used for: Deep feature learning is performed on the original multimodal feature set to obtain intramodal deep feature representations. Based on the intramodal deep feature representations, a cross-modal attention mapping network is constructed to obtain the correspondence of key features. Heterogeneous feature fusion is performed on the correspondence of key features to obtain a unified feature space representation, and crack feature embedding is performed based on the unified feature space representation to obtain crack feature vectors; Spectral clustering analysis was performed on the crack feature vectors to obtain crack feature clusters. Based on the crack feature clusters, the similarity of features for different crack types was calculated to obtain a feature similarity matrix. The feature similarity matrix is ​​transformed into a graph structure to obtain an initial crack feature map. Based on the initial crack feature map, the relationships between nodes are adaptively weighted to obtain a weighted crack feature map. Multi-level community detection is performed on the weighted crack feature map to obtain the crack feature community structure. Based on the crack feature community structure, hierarchical evolution analysis of the relationship between communities is performed to obtain the bridge crack feature spectrum map.

[0032] In this embodiment, for visual modal data, deep convolutional neural networks (such as ResNet, DenseNet, or EfficientNet) are used to extract deep features of the images, capturing the texture, shape, edges, and contextual information of the cracks. For acoustic modal data, recurrent neural networks (RNNs) or long short-term memory networks (LSTMs) are applied to extract temporal features, identifying patterns and anomalies in the acoustic signals. For vibration modal data, autoencoders or variational autoencoders (VAEs) are used to learn the latent representation of vibration signals, capturing structural dynamic characteristics. The original features of each modality are subjected to nonlinear transformation and dimensionality reduction to generate more compact and discriminative deep feature representations. A Transformer-based design is employed. The architecture employs a cross-modal attention mechanism to learn the correspondences between features from different modalities. Attention weights reflect the importance of each modal feature in different contexts. An attention mapping matrix is ​​constructed to quantify the interrelationships between features from different modalities, identifying the most critical feature elements for crack representation in each modality, establishing their correspondences, and forming a key feature correspondence graph. A multimodal fusion network is designed to map features from different modalities to a unified semantic space. Tensor fusion, multi-view learning, or graph neural network methods are used to preserve complementary information from each modality and reduce redundancy, generating a unified feature space representation. In this unified feature space, manifold learning methods (such as t-SNE, UMAP, or homotopy autoencoders) are applied to non-linearly represent crack features. Linear embedding maps high-dimensional features to a low-dimensional space, preserving the topological structure of the data and forming a more compact crack feature vector. Spectral clustering algorithms are applied to perform cluster analysis on the crack feature vectors, grouping crack samples based on feature similarity to identify crack types with similar features, forming crack feature clusters. The similarity between features of different crack types is calculated using metrics such as cosine similarity, Euclidean distance, or Mahalanobis distance. A feature similarity matrix is ​​constructed to quantify the similarity relationship between crack types. This feature similarity matrix is ​​then converted into a graph structure, where each node represents a type of crack feature, and edges between nodes represent the similarity relationship between features. The weight of each edge is determined by similarity. An initial crack feature graph is constructed, and based on feature importance and similarity strength, [further analysis is needed]. The relationships between nodes are adaptively weighted, and the importance of nodes is evaluated using algorithms such as PageRank or node centrality. Relationships between important features receive higher weights, generating a weighted crack feature map. Multi-level community detection algorithms (such as the Louvain method, Infomap algorithm, or spectral clustering) are applied to divide the weighted crack feature map into communities, identify closely related feature groups, form a crack feature community structure, analyze the hierarchical relationships and evolution patterns between communities, construct a phylogenetic structure reflecting the crack feature classification system, and apply hierarchical clustering or recursive community detection methods to identify the inclusion, derivation, and evolutionary relationships between communities. Finally, a bridge crack feature phylogenetic map is generated, which intuitively displays the classification system and correlation relationships of crack features.

[0033] In this embodiment, the spatiotemporal correlation analysis module is used to perform spatiotemporal correlation analysis and construct a crack knowledge graph based on the bridge crack feature spectrum diagram to obtain a crack evolution trend diagram. Specifically, it is used for: A topological analysis was performed on the feature spectrum diagram of bridge cracks to obtain a set of key nodes. Based on the set of key nodes, time series features of bridge cracks were extracted to obtain crack evolution feature vectors. Spatiotemporal encoding is performed on the crack evolution feature vector to obtain crack development identifiers. Based on the crack development identifiers and the bridge crack feature spectrum diagram, dynamic crack labeling is performed to obtain labeled data with evolution attributes. By combining the knowledge of experts in the engineering field, a crack type ontology is established to obtain a crack knowledge base. Based on the crack knowledge base and labeled data, a crack knowledge graph is constructed to obtain an initial crack knowledge structure. Spatiotemporal association rules are mined from the initial crack knowledge structure to obtain a crack evolution pattern library. Based on the crack evolution pattern library, a graph structure model is performed on the crack development trend to obtain a crack evolution path graph. Spatiotemporal graph fusion processing is performed on the crack evolution path map and the bridge crack feature spectrum map to obtain a multi-dimensional crack situation representation. Based on the multi-dimensional crack situation representation, trend prediction analysis is performed to obtain a crack evolution situation map.

[0034] In this embodiment, graph theory algorithms are applied to analyze the topological structure of the bridge crack feature spectrum graph, calculating indices such as degree centrality, betweenness centrality, and eigenvector centrality of nodes to evaluate the importance of nodes in the network. PageRank or HITS algorithms are used to identify key nodes in the network; these nodes represent the most important features for crack characterization, forming a set of key nodes. Historical data on crack features within this set are collected over time to construct a time-series dataset. Time-series analysis methods (such as Autoregressive Integral Moving Average (ARIMA) and Long Short-Term Memory (LSTM) networks) are used to extract features of crack changes over time, including trends, seasonality, periodicity, and anomalous patterns. Wavelet transform or Hilbert-Yellow Fiber transform is then applied. This paper employs a time-frequency analysis method to extract frequency domain features and nonlinear characteristics during crack evolution. By integrating these time and frequency domain features, a crack evolution feature vector is formed. A spatiotemporal encoder is designed to fuse and encode the spatial location information of the crack (e.g., coordinates on the bridge, structural unit type) with its temporal evolution features (e.g., development rate, periodic changes). Deep learning methods (e.g., spatiotemporal graph convolutional networks) are used to learn the spatiotemporal pattern of the crack, generating a unique crack development identifier. This identifier contains the spatiotemporal information and evolutionary features of the crack. Based on this identifier, nodes in the bridge crack feature spectrum diagram are dynamically labeled, adding attributes such as timestamps, evolution rates, and environmental conditions to each node. A time-series graph data structure is used to represent the dynamic development process of the crack, generating... Labeled data with evolutionary attributes, in collaboration with experts in bridge engineering, is used to establish a crack type ontology. This ontology defines concepts and relationships related to crack classification systems, causal mechanisms, development patterns, and severity levels. A formal crack knowledge model is constructed using ontology modeling languages ​​(such as OWL) or knowledge representation frameworks (such as RDF). Knowledge from engineering manuals, academic literature, and expert experience is integrated to form a structured crack knowledge base. Combining the labeled data and the crack knowledge base, a crack knowledge graph is constructed using knowledge graph technology. Nodes represent crack types, characteristics, influencing factors, and structural elements, while edges represent semantic relationships between them (such as "causes," "affects," and "belongs to"). Knowledge reasoning engines (such as rule-based reasoning or description logic-based reasoning) are then applied. This study enriches the knowledge graph, derives implicit relationships and knowledge, and forms an initial crack knowledge structure. It then applies association rule mining algorithms (such as Apriori, FP-Growth, or sequence pattern mining) to analyze the temporal and spatial association patterns of cracks, discovering the temporal patterns, spatial distribution characteristics, and triggering conditions of crack development. This identifies key stages and turning points in crack evolution, constructing a crack evolution pattern library. Based on this library, a directed graph model is used to model the development trend of cracks. Nodes represent crack states, edges represent the probabilities and conditions of state transitions, and edge weights reflect transition probabilities or rates. Markov decision processes or Bayesian networks are applied to simulate the crack evolution process, generating a crack evolution path graph. Finally, a spatiotemporal graph fusion algorithm is designed.This method integrates crack evolution path maps with bridge crack feature spectrum maps, combining static representations and dynamic evolution information of crack features. Tensor decomposition or multi-relation graph neural networks are used to process multi-dimensional data, generating a multi-dimensional crack situation representation that includes crack characteristics, distribution, evolution patterns, and influencing factors. Time series prediction models (such as ARIMA, Prophet, LSTM, or Transformer) are applied to predict and analyze crack development trends. Combined with factors such as environmental conditions, load changes, and material aging, the future state and development path of cracks are predicted, generating a crack evolution situation map that visually displays the historical development, current state, and future trends of cracks, providing a scientific basis for bridge maintenance decisions.

[0035] In this embodiment, the environmental adaptability feature enhancement module is used to enhance the environmental adaptability features and perform multi-scale characterization of the crack state based on the crack evolution status map, obtain the crack multi-scale feature space, and construct a diagnostic engine, specifically for: The crack evolution situation map is segmented into multiple environment-specific sub-maps. Based on these sub-maps, environmental influencing factors are extracted to obtain environmental feature vectors. An adaptive enhancement process is applied to the environmental feature vector to obtain an environmental adaptability representation. The environmental adaptability representation is then decomposed into multiple scales to obtain crack feature representations at different scales. By fusing crack feature representations at different scales, a multi-scale crack feature space is constructed, and boundary learning is performed in the multi-scale crack feature space to obtain the crack severity decision boundary. Based on the decision boundary of crack severity, fuzzy reasoning is performed on the assessment rules under different environmental conditions to obtain an environmental adaptability diagnosis knowledge base; A diagnostic strategy network is created based on an environmentally adaptive diagnostic knowledge base. The diagnostic process is then optimized through reinforcement learning based on the diagnostic strategy network and the multi-scale feature space of the cracks, resulting in an environmentally adaptive diagnostic engine.

[0036] In this embodiment, a graph segmentation algorithm (such as spectral clustering, Normalized Cut, or community detection-based segmentation) is used to segment the crack evolution map into subgraphs. Based on the similarity of environmental conditions (such as temperature range, humidity level, and load type), the map is divided into multiple subgraphs. Each subgraph represents the crack state and evolution pattern under specific environmental conditions, forming multiple environment-specific subgraphs. Environmental influencing factors, including temperature, humidity, load conditions, material aging degree, and exposure environment (such as acid rain and salt spray), are extracted from these subgraphs. A feature extractor (such as a convolutional neural network or graph neural network) is designed to learn environmental features from the subgraphs, constructing environmental feature vectors to quantify the influence of different environmental factors on crack development. An adaptive feature enhancement network is designed to target crack features under different environmental conditions. Enhancement processing is performed, employing contrastive learning or domain adaptation methods to improve the robustness and discriminative power of features, reducing the interference of environmental changes on crack feature representations. An environmental condition encoder is designed to encode environmental factors into the feature representation, generating an environmentally adaptive representation. Wavelet transform or multi-resolution analysis methods are applied to decompose the environmentally adaptive representation into multiple scales, capturing local details and global structural features of cracks at different scales. Small-scale representations capture local texture and micromorphology of cracks, medium-scale representations describe the shape and connectivity of cracks, and large-scale representations reflect the distribution patterns and overall trends of cracks, generating crack feature representations at different scales. A multi-scale feature fusion framework is designed, employing attention mechanisms or adaptive weighting methods to fuse features at different scales. The crack features at different scales are fused to retain key information while reducing redundancy and noise, constructing a multi-scale crack feature space. This comprehensively represents the characteristics of cracks under different scales and environmental conditions. Within this multi-scale feature space, Support Vector Machines (SVM), Random Forests, or deep learning methods are applied to learn the decision boundary for crack severity. Supervised or semi-supervised learning methods are used to distinguish different crack states (e.g., mild, moderate, severe, dangerous). Considering the influence of environmental factors, an environmental adaptability decision boundary is learned, forming the crack severity decision boundary. Based on fuzzy logic theory, a mapping relationship between environmental conditions and crack assessment rules is established, defining fuzzy sets and fuzzy rules to describe crack severity under different environmental conditions. To address changes in crack assessment criteria, a fuzzy inference system is designed to adaptively adjust the assessment criteria based on different environmental conditions, generating an environmentally adaptive diagnostic knowledge base. This base includes diagnostic rules and assessment criteria for different environmental conditions. Based on this knowledge base, a decision tree, Bayesian network, or rule system is designed to construct a diagnostic strategy network, forming the decision-making process and inference mechanism for crack diagnosis. Diagnostic steps, judgment conditions, and decision paths are defined. Reinforcement learning algorithms (such as Q-learning, deep Q-networks, or policy gradient methods) are applied to optimize the diagnostic process. A reward mechanism guides the system to learn the optimal diagnostic strategy. A reward function is designed to encourage accurate diagnosis and penalize misjudgments. The diagnostic strategy is continuously optimized through interaction with the environment.By combining a multi-scale feature space of cracks with a diagnostic strategy network, an environment-adaptive diagnostic engine is constructed. This engine can automatically select appropriate diagnostic strategies and evaluation criteria based on different environmental conditions and crack characteristics, achieving accurate assessment and diagnosis of crack conditions.

[0037] In this embodiment, the safety risk analysis module is used to perform multi-dimensional safety risk analysis and multi-agent collaborative diagnosis of the bridge structure, obtaining a bridge safety risk hypermap, specifically used for: The safety risk factors of the bridge structure are decomposed hierarchically to obtain the risk factor hierarchy, and the structural importance is analyzed based on the risk factor hierarchy to obtain the risk weight vector. The association rules of safety assessment indicators are mined on the risk weight vector to obtain the association network of assessment indicators, and a risk assessment function cluster is generated based on the association network of assessment indicators. Based on the risk assessment function cluster, diagnostic agents specifically designed for different types of cracks are obtained, resulting in an initial multi-agent diagnostic model. The initial multi-agent diagnostic model is then collaboratively optimized to obtain a set of agent cooperation strategies. Based on the agent cooperation strategy set, federated learning is performed on the diagnostic results between agents to obtain an optimized multi-agent cooperative diagnostic model. Uncertainty quantification is performed on the optimized multi-agent collaborative diagnostic model to obtain the diagnostic confidence distribution. Based on the diagnostic confidence distribution, hypergraph modeling and hyperedge weight calculation are performed on the risk assessment results to obtain the bridge safety risk hypergraph.

[0038] In this embodiment, the Analytic Hierarchy Process (AHP) or Structural Decomposition Technique is used to systematically decompose the safety risk factors of the bridge structure. Risk sources are identified from multiple dimensions, including materials (e.g., material aging, fatigue, corrosion), structure (e.g., design defects, construction quality, overload), environment (e.g., temperature changes, humidity, chemical corrosion), and loads (e.g., traffic loads, wind loads, earthquakes). A hierarchical structure of risk factors is constructed to represent the hierarchical relationships and mutual influences among these factors. Fault Tree Analysis (FTA) or Failure Mode and Effects Analysis (FMEA) methods are applied to assess the impact of each risk factor on bridge safety. The importance weight of each risk factor is calculated, considering its severity, probability of occurrence, and detectability. Combined with expert evaluation and historical data, the risk factors are quantified. The risk contribution of each factor is factored into a risk weight vector. Association rule mining algorithms (such as Apriori, FP-Growth, or association rule learning) are used to analyze the correlations between safety assessment indicators, discovering dependencies, synergistic effects, and causal relationships among indicators. Frequently occurring indicator combinations and correlation patterns are identified, and an assessment indicator correlation network is constructed to represent the mutual influence and information flow between indicators. Based on this network, community detection algorithms (such as the Louvain method, Infomap, or spectral clustering) are applied to identify functionally related indicator sets. Closely related indicators are grouped to form risk assessment functional clusters, each responsible for a specific type of risk assessment (such as structural safety, durability, and performance). For different types of cracks (such as fatigue cracks, temperature cracks, shrinkage cracks, shear cracks, etc.), specialized diagnostic agents are designed. Each agent possesses a specific knowledge base, reasoning mechanism, and diagnostic strategy. Optimization is performed based on the characteristics and development patterns of specific crack types to form an initial multi-agent diagnostic model. Collaboration mechanisms between agents are designed, including strategies for information sharing (such as sharing observation results and diagnostic information), task allocation (such as assigning diagnostic tasks based on expertise), and result fusion (such as integrating diagnostic conclusions from multiple agents). Communication protocols and collaboration rules between agents are established, defining the content, format, and timing of information exchange to form a set of agent collaboration strategies. A federated learning framework is adopted to manage the distributed agents while protecting data privacy. Collaborative training involves each agent training a diagnostic model on local data, then sharing model parameters instead of the original data. Parameter aggregation algorithms (such as FedAvg and FedProx) are designed to integrate the model parameters of multiple agents. Through multiple rounds of iterative optimization, an optimized multi-agent collaborative diagnostic model is formed. Bayesian methods (such as Bayesian neural networks and Gaussian processes) or ensemble learning techniques (such as random forests and boosting methods) are applied to quantify the uncertainty of the multi-agent diagnostic results. The confidence interval, variance, and reliability index of the diagnostic results are calculated. The stability and credibility of the diagnostic results under different conditions are analyzed, and a diagnostic credibility distribution is generated. A bridge safety risk hypergraph is constructed based on hypergraph theory, where nodes represent risk factors and structural elements.Hyperedges represent higher-order relationships between multiple factors. A hyperedge can connect more than two nodes, representing complex risk associations. Calculating hyperedge weights reflects the severity and probability of occurrence of combinations of risk factors. Considering the distribution of diagnostic confidence, uncertainty corrections are applied to the hyperedge weights, ultimately forming a bridge safety risk hypergraph that comprehensively characterizes the safety risk status of the bridge structure and the mutual influence of various risk factors.

[0039] In this embodiment, the causal relationship analysis module is used to perform causal relationship analysis on bridge cracks based on the bridge safety risk hypergraph, and obtain the crack evolution causal network. Specifically, it is used for: The bridge safety risk hypergraph is decomposed into multiple risk subgraphs. Based on these multiple risk subgraphs, the evolution process of bridge cracks is analyzed by time-series correlation analysis to obtain the crack evolution sequence. The causal discovery algorithm is applied to the crack evolution sequence to obtain the causal intensity matrix of crack state transition. Based on the causal intensity matrix, the causal direction of the crack evolution path is determined to obtain a set of directed causal relationships. Structural learning is performed on the directed causal relationship set to obtain an initial crack causal graph model. Based on the initial crack causal graph model, physical constraints are introduced to perform causal inference, resulting in a physically enhanced complete causal graph. The complete causal graph of physical enhancement was validated and tested to obtain the crack evolution backbone network. Based on the crack evolution backbone network, the nonlinear relationship between nodes was fitted by deep learning to obtain the quantitative causal relationship function. Sensitivity analysis was performed on the quantitative causal relationship function to obtain the key causal path. Based on the key causal path, the causal networks of various types of cracks were integrated to obtain the crack evolution causal network.

[0040] In this embodiment, a hypergraph decomposition algorithm (such as hyperedge projection, hypergraph decomposition, or hypergraph segmentation) is applied to decompose the bridge safety risk hypergraph into multiple subgraphs. Each subgraph represents a specific risk type or the risk status of a specific area, such as a material risk subgraph, a structural risk subgraph, and an environmental risk subgraph, forming multiple risk subgraphs. Crack monitoring data at different time points are collected to construct a time series of crack states, recording the changes in parameters such as crack width, length, and depth over time. Temporal correlation analysis methods (such as Dynamic Time Warping (DTW), Granger causality test, and cross-correlation analysis) are applied to analyze the temporal dependencies in the crack evolution process, identify the stage characteristics and turning points of crack development, and combine the risk subgraph information to analyze the relationship between risk factors and crack development. Sequence correlation is used to generate a crack evolution sequence, describing the development process of the crack from its initial state to its current state. Causal discovery algorithms (such as PC algorithm, FCI algorithm, NOTEARS, or causal acceleration methods) are applied to process the crack evolution sequence, inferring causal relationships between variables from observational data, identifying key factors and triggering conditions leading to crack development, calculating the influence intensity of each factor on crack state transitions, constructing a causal strength matrix for crack state transitions, quantifying the causal relationship strength between factors, and using the causal strength matrix, determining the causal direction of the crack evolution path using a directed acyclic graph (DAG) representation. Guiding information theory or conditional independence tests are applied to determine the direction of causal relationships, and driving factors in the causal relationships (such as loads) are identified. A directed causal relationship set is formed by considering factors such as temperature cycling and crack propagation (e.g., temperature changes) and response factors (e.g., crack propagation, structural deformation), representing the causal chain in the crack evolution process. Structural learning algorithms (e.g., Bayesian network structure learning, score optimization methods, or constraint optimization methods) are applied to optimize this set, learning the conditional independence and dependencies between variables. The sparsity and interpretability of causal relationships are considered, and an initial crack causal graph model is constructed to represent the causal network in the crack development process. Combining the principles of bridge structural mechanics, materials science knowledge, and fracture mechanics theory, physical constraints are introduced to the causal inference process, ensuring that the causal relationships conform to physical laws and engineering experience. Physical constraint conditions (e.g., force balance, energy conservation, material constitutive relations) are designed to guide the causal inference process. This process forms a physically enhanced complete causal graph, more accurately representing the mechanism of crack development. Through experimental verification or historical data comparison, the predictive and explanatory power of the causal graph is tested, and the generalization performance and stability of the causal model under different conditions are evaluated. Based on the verification results, key causal paths are selected, unstable or insignificant causal relationships are removed, and a crack evolution backbone network is formed, containing the most important causal chains and key influencing factors. Deep learning methods (such as neural networks, Gaussian process regression, or support vector regression) are applied to fit the nonlinear causal relationships between nodes, quantifying the strength, form, and dynamic characteristics of causal influences. The functional mapping relationship between causal variables is learned, generating a quantitative causal relationship function to accurately describe the influence mechanism of each factor on crack development.Sensitivity analysis was conducted on the quantitative causal relationship function to assess the sensitivity of each factor to crack evolution, identify the most significant factors and pathways affecting crack development, calculate sensitivity indices (such as partial derivatives and variance contribution rate), determine key causal pathways, and integrate causal networks of different types of cracks (such as fatigue cracks, temperature cracks, shrinkage cracks, and shear cracks) to construct a unified crack evolution causal network. This network comprehensively characterizes the intrinsic mechanisms and influencing factors of crack development, visualizes the entire process of crack formation and propagation, reveals the commonalities and characteristics of different types of cracks, and provides a theoretical basis for crack prevention and control.

[0041] In this embodiment, the diagnostic result fusion module is used to dynamically fuse and quantify the uncertainty of diagnostic results based on the crack evolution causal network and the diagnostic engine to obtain a comprehensive intelligent crack diagnosis report. Specifically, it is used for: The crack evolution causal network is topologically sorted to obtain the crack evolution stage sequence. Based on the crack evolution stage sequence, the output of the diagnostic engine is temporally aligned to obtain the aligned diagnostic result set. Multimodal evidence theory fusion is performed on the aligned diagnostic result set to obtain diagnostic features with different confidence levels. Based on the diagnostic features with different confidence levels, Bayesian uncertainty quantification is performed on the diagnostic results to obtain the diagnostic confidence distribution matrix. The diagnostic confidence distribution matrix is ​​subjected to deep probabilistic encoding to obtain the confidence interval of the diagnostic results. Based on the confidence interval, the diagnostic results are dynamically weighted and fused to obtain a preliminary fusion report. The preliminary fusion report is optimized using reinforcement learning to obtain the optimal maintenance decision recommendations. Based on the optimal maintenance decision recommendations, a cost-benefit analysis is performed to obtain a priority list of maintenance solutions. The maintenance plan priority list and preliminary fusion report are integrated to obtain a comprehensive intelligent crack diagnosis report, which includes crack type identification, severity assessment, evolution prediction and maintenance recommendations.

[0042] In this embodiment, a topological sorting algorithm (such as the Kahn algorithm or depth-first search) is applied to sort the causal network of crack evolution. Based on the chronological order of causal relationships, the stage sequence of crack development is determined, from the initial microcracks to their expansion, penetration, and eventual structural failure, forming a crack evolution stage sequence. The diagnostic results output by the diagnostic engine are time-aligned with this sequence to ensure consistency between the diagnostic results and the crack development stages. A mapping relationship between the diagnostic results and the crack evolution stages is established, generating an aligned diagnostic result set containing crack state assessments and diagnostic conclusions for different stages. Dempster-Shafer evidence theory or fuzzy set theory is then applied to fuse the multimodal diagnostic results. This approach addresses the conflict and complementarity of different modalities of evidence (such as visual image diagnosis, acoustic wave detection diagnosis, and vibration analysis diagnosis), designs evidence synthesis rules, integrates multi-source evidence, handles evidence conflicts, and generates diagnostic features with different confidence levels to represent the credibility and uncertainty of diagnostic conclusions. Bayesian methods (such as Bayesian networks and Monte Carlo methods) are used to quantify the uncertainty of diagnostic results, calculating the posterior probability distribution, confidence interval, and reliability index of diagnostic conclusions. Considering prior knowledge, observational evidence, and model uncertainty, a diagnostic credibility distribution matrix is ​​constructed to quantify the credibility and uncertainty of different diagnostic conclusions. Deep probabilistic coding methods (such as variational autoencoders, Bayesian neural networks, or deep set methods) are applied. The diagnostic results are encoded using a multi-source learning approach, transforming deterministic diagnoses into probability distributions. This generates probability distributions and confidence intervals for the diagnostic results, representing the uncertainty and reliability of the conclusions. Based on these confidence intervals, diagnostic results from different sources are dynamically weighted and fused, with higher-confidence results receiving higher weights and lower-confidence results receiving lower weights. An adaptive weighting strategy is designed to dynamically adjust weights based on the consistency and reliability of the diagnostic results, generating a preliminary fusion report. This comprehensive multi-source diagnostic report provides a more reliable crack assessment. Reinforcement learning algorithms (such as policy gradient, deep Q-networks, or the Actor-Critic method) are applied to optimize the maintenance decision-making process, considering crack status and safety risks. Considering factors such as maintenance costs, service life, and environmental impact, a reward function is designed to balance safety, economy, and sustainability. Through interactive learning with the environment, the optimal decision-making strategy is learned, generating optimal maintenance decision recommendations. Cost-benefit analyses are conducted on different maintenance options (such as monitoring and observation, crack grouting, structural reinforcement, and partial replacement), evaluating the input-output ratio, risk reduction effect, and long-term benefits of each option. Direct costs (such as materials and labor) and indirect costs (such as traffic disruption and service loss) are considered, calculating the cost-benefit ratio and return on investment. A priority list of maintenance options is generated, sorted by cost-benefit ratio. The diagnostic results, uncertainty analysis, maintenance recommendations, and priority list are integrated to generate a comprehensive intelligent crack diagnosis report.It includes crack type identification (e.g., fatigue cracks, temperature cracks, shear cracks, etc.), severity assessment (e.g., minor, moderate, severe, dangerous), evolution prediction (e.g., propagation rate, development trend), and maintenance recommendations (e.g., scheme selection, implementation timing, expected results), providing comprehensive support for bridge maintenance decisions.

[0043] This application also provides a bridge crack intelligent diagnosis device based on multimodal data fusion. The bridge crack intelligent diagnosis device based on multimodal data fusion includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor performs the implementation steps of the bridge crack intelligent diagnosis system based on multimodal data fusion in the above embodiments.

[0044] This application also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the implementation steps of the intelligent bridge crack diagnosis system based on multimodal data fusion.

[0045] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0046] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0047] In the description of this invention, it should be understood that the terms "first," "second," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0048] In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0049] In the description of this invention, "several" means one or more, and "a large number" means two or more.

[0050] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0051] All formulas in this manual are dimensionless and calculated numerically. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0052] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A bridge crack intelligent diagnosis system based on multimodal data fusion, characterized in that, include: The data acquisition and preprocessing module is used to acquire and preprocess multimodal data of the bridge structure to obtain the original multimodal feature set. The cross-modal feature extraction module is used to perform cross-modal feature extraction and heterogeneous feature encoding based on the original multimodal feature set to obtain a bridge crack feature spectrum map; The spatiotemporal correlation analysis module is used to perform spatiotemporal correlation analysis and construct a crack knowledge graph based on the bridge crack feature spectrum diagram to obtain a crack evolution status diagram. The environmental adaptability feature enhancement module is used to enhance the environmental adaptability features and perform multi-scale characterization of the crack state based on the crack evolution status map, obtain the crack multi-scale feature space, and construct a diagnostic engine. The safety risk analysis module is used to perform multi-dimensional safety risk analysis and multi-agent collaborative diagnosis of bridge structures, and to obtain a bridge safety risk hypermap. The causal relationship analysis module is used to perform causal relationship analysis on bridge cracks based on the bridge safety risk hypergraph to obtain the crack evolution causal network. The diagnostic result fusion module is used to dynamically fuse and quantify the uncertainty of the diagnostic results based on the crack evolution causal network and the diagnostic engine to obtain a comprehensive intelligent diagnostic report of the crack; the modules are connected to each other by wired and / or wireless means to realize data transmission between the modules.

2. The intelligent bridge crack diagnosis system based on multimodal data fusion according to claim 1, characterized in that, The process of acquiring and preprocessing multimodal data of the bridge structure yields a multimodal raw feature set, including: Visual images, acoustic waves, and vibration signals of the bridge structure are acquired to obtain a multi-source heterogeneous dataset. Noise is filtered out and missing values ​​are repaired in the multi-source heterogeneous dataset to obtain a cleaned dataset. The cleaned dataset is subjected to time series synchronization processing to obtain aligned multimodal data, and outlier detection is performed based on the aligned multimodal data to obtain a standardized feature matrix; The standardized feature matrix is ​​subjected to a mode-specific transformation to obtain an enhanced feature set. Based on the enhanced feature set, mutual information is calculated for different modal features to obtain a modality correlation matrix. The modality correlation matrix is ​​evaluated for feature importance to obtain the feature weight distribution. Based on the feature weight distribution, features are adaptively selected to obtain an optimized feature set. The optimized feature set is grouped within each modality to obtain a modal feature clustering structure, and a multimodal original feature set is constructed based on the modal feature clustering structure.

3. The intelligent bridge crack diagnosis system based on multimodal data fusion according to claim 2, characterized in that, The process of extracting cross-modal features and encoding heterogeneous features based on the multimodal original feature set to obtain a bridge crack feature hierarchy map includes: Deep feature learning is performed on the original multimodal feature set to obtain intramodal deep feature representations, and a cross-modal attention mapping network is constructed based on the intramodal deep feature representations to obtain the correspondence of key features; Heterogeneous feature fusion is performed on the correspondence of the key features to obtain a unified feature space representation, and crack feature embedding is performed based on the unified feature space representation to obtain crack feature vectors; Spectral clustering analysis is performed on the crack feature vectors to obtain crack feature families, and based on the crack feature families, the similarity of features of different crack types is calculated to obtain a feature similarity matrix; The feature similarity matrix is ​​transformed into a graph structure to obtain an initial crack feature map. Based on the initial crack feature map, the relationships between nodes are adaptively weighted to obtain a weighted crack feature map. Multi-level community detection is performed on the weighted crack feature map to obtain the crack feature community structure. Based on the crack feature community structure, hierarchical evolution analysis is performed on the relationship between communities to obtain the bridge crack feature spectrum map.

4. The intelligent bridge crack diagnosis system based on multimodal data fusion according to claim 3, characterized in that, The process of performing spatiotemporal correlation analysis and constructing a crack knowledge graph based on the bridge crack feature spectrum map to obtain a crack evolution trend map includes: A topological analysis was performed on the bridge crack feature spectrum diagram to obtain a set of key nodes. Based on the set of key nodes, time series features of the bridge cracks were extracted to obtain crack evolution feature vectors. Spatiotemporal encoding is performed on the crack evolution feature vector to obtain a crack development identifier. Based on the crack development identifier and the bridge crack feature spectrum diagram, dynamic crack labeling is performed to obtain labeled data with evolution attributes. By combining the knowledge of experts in the engineering field, a crack type ontology is established to obtain a crack knowledge base. Based on the crack knowledge base and the labeled data, a crack knowledge graph is constructed to obtain an initial crack knowledge structure. Spatiotemporal association rule mining is performed on the initial crack knowledge structure to obtain a crack evolution pattern library. Based on the crack evolution pattern library, a graph structure model is performed on the crack development trend to obtain a crack evolution path graph. The spatiotemporal graph fusion processing of the crack evolution path map and the bridge crack feature spectrum map is performed to obtain a multi-dimensional crack situation representation. Based on the multi-dimensional crack situation representation, trend prediction analysis is performed to obtain a crack evolution situation map.

5. The intelligent bridge crack diagnosis system based on multimodal data fusion according to claim 4, characterized in that, Based on the crack evolution status map, the crack state is enhanced with environmental adaptability features and multi-scale characterization to obtain a multi-scale feature space for the crack, and a diagnostic engine is constructed, including: The crack evolution situation map is segmented into multiple environment-specific sub-maps, and environmental influencing factors are extracted based on the multiple environment-specific sub-maps to obtain environmental feature vectors; The environmental feature vector is adaptively enhanced to obtain an environmental adaptability representation, and the environmental adaptability representation is decomposed into multiple scales to obtain crack feature representations at different scales. By fusing the crack feature representations at different scales, a multi-scale crack feature space is constructed, and boundary learning is performed in the multi-scale crack feature space to obtain the crack severity decision boundary. Based on the decision boundary of the severity of the crack, fuzzy reasoning is performed on the assessment rules under different environmental conditions to obtain an environmental adaptability diagnostic knowledge base; A diagnostic strategy network is created based on the environmental adaptability diagnostic knowledge base, and the diagnostic process is optimized by reinforcement learning based on the diagnostic strategy network and the multi-scale feature space of the crack, resulting in an environmentally adaptive diagnostic engine.

6. The intelligent bridge crack diagnosis system based on multimodal data fusion according to claim 5, characterized in that, The aforementioned multi-dimensional safety risk analysis and multi-agent collaborative diagnosis of bridge structures yields a bridge safety risk hypermap, including: The safety risk factors of the bridge structure are decomposed hierarchically to obtain the risk factor hierarchy, and the structural importance analysis is performed based on the risk factor hierarchy to obtain the risk weight vector. The risk weight vector is subjected to security assessment index association rule mining to obtain the assessment index association network, and a risk assessment function cluster is generated based on the assessment index association network. Based on the aforementioned risk assessment function cluster, diagnostic agents specifically designed for different types of cracks are obtained, resulting in an initial multi-agent diagnostic model. The initial multi-agent diagnostic model is then collaboratively optimized to obtain a set of agent cooperation strategies. Based on the set of agent cooperation strategies, federated learning is performed on the diagnostic results among agents to obtain an optimized multi-agent cooperative diagnostic model. Uncertainty quantification is performed on the optimized multi-agent collaborative diagnostic model to obtain the diagnostic confidence distribution. Based on the diagnostic confidence distribution, hypergraph modeling and hyperedge weight calculation are performed on the risk assessment results to obtain the bridge safety risk hypergraph.

7. The intelligent bridge crack diagnosis system based on multimodal data fusion according to claim 6, characterized in that, Based on the bridge safety risk hypergraph, a causal relationship analysis of bridge cracks is performed to obtain a crack evolution causal network, including: The bridge safety risk hypergraph is decomposed into multiple risk subgraphs, and the evolution process of bridge cracks is analyzed by time-series correlation analysis based on the multiple risk subgraphs to obtain the crack evolution sequence. The crack evolution sequence is processed by a causal discovery algorithm to obtain a causal intensity matrix of crack state transitions. Based on the causal intensity matrix, the causal direction of the crack evolution path is determined to obtain a set of directed causal relationships. Structural learning is performed on the directed causal relationship set to obtain an initial crack causal graph model. Based on the initial crack causal graph model, physical constraints are introduced to perform causal inference to obtain a physically enhanced complete causal graph. The complete causal graph of the physical enhancement is verified and tested to obtain the crack evolution backbone network. Based on the crack evolution backbone network, the nonlinear relationship between nodes is fitted by deep learning to obtain a quantitative causal relationship function. Sensitivity analysis was performed on the quantitative causal relationship function to obtain key causal paths. Based on the key causal paths, the causal networks of various types of cracks were integrated to obtain the crack evolution causal network.

8. The intelligent bridge crack diagnosis system based on multimodal data fusion according to claim 7, characterized in that, The diagnostic results are dynamically fused and uncertainty quantified based on the crack evolution causal network and the diagnostic engine to obtain a comprehensive intelligent crack diagnosis report, including: The crack evolution causal network is topologically sorted to obtain a crack evolution stage sequence. Based on the crack evolution stage sequence, the output of the diagnostic engine is temporally aligned to obtain an aligned diagnostic result set. Multimodal evidence theory fusion is performed on the aligned diagnostic result set to obtain diagnostic features with different confidence levels. Based on the diagnostic features with different confidence levels, Bayesian uncertainty quantification is performed on the diagnostic results to obtain the diagnostic confidence distribution matrix. The diagnostic confidence distribution matrix is ​​subjected to deep probabilistic encoding to obtain the confidence interval of the diagnostic results, and the diagnostic results are dynamically weighted and fused according to the confidence interval to obtain a preliminary fusion report; The preliminary fusion report is optimized using reinforcement learning to obtain the optimal maintenance decision recommendation. Based on the optimal maintenance decision recommendation, a cost-benefit analysis is performed to obtain a priority list of maintenance solutions. The maintenance plan priority list and the preliminary fusion report are integrated to obtain a comprehensive intelligent crack diagnosis report, which includes crack type identification, severity assessment, evolution prediction, and maintenance recommendations.

9. A smart bridge crack diagnosis device based on multimodal data fusion, characterized in that, The intelligent bridge crack diagnosis device based on multimodal data fusion includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor performs the implementation steps of the intelligent bridge crack diagnosis system based on multimodal data fusion as described in any one of claims 1-8.

10. A computer-readable storage medium storing instructions, characterized in that, When the instructions are executed on a computer, the computer performs the steps of the intelligent bridge crack diagnosis system based on multimodal data fusion as described in any one of claims 1-8.

Images