Bridge structure damage identification method combining multi-scale frequency domain modeling and large language model

Abstract

The invention relates to the field of structure health monitoring and artificial intelligence, in particular to a bridge structure damage identification method combining multi-scale frequency domain modeling and a large language model, which comprises the following steps: S1, extracting numerical statistical information of a vibration acceleration time sequence signal; s2, performing causal expansion convolution, local attention modeling and structural state classification based on the vibration acceleration time sequence signal to obtain a structural state feature tag; s3, multi-resolution time imaging is carried out on the vibration acceleration time sequence signal, time image characteristic decomposition is realized by using a three-axis attention mechanism, a short-term seasonal time characteristic, a long-term trend time characteristic and a cross-channel time characteristic are obtained, and a mixed characteristic is generated; s4, generating a structured text prompt template through the structure state feature tag and the numerical statistical information; and S5, inputting the structured text prompt template and the mixed features into a large language model for reasoning to obtain a bridge structure damage identification result. The bridge structure damage identification accuracy can be improved.

Description

Technical Field

[0001] This invention relates to the fields of structural health monitoring and artificial intelligence, specifically to a method for identifying bridge structural damage by combining multi-scale frequency domain modeling and large language models. Background Technology

[0002] With the rapid advancement of infrastructure construction in my country, the number of bridges continues to grow, and the long-term service status of bridge structures has attracted increasing attention. During long-term operation, bridges are inevitably affected by factors such as environmental corrosion, material aging, and changes in traffic loads, making them highly susceptible to fatigue damage, modal changes, and structural performance degradation, seriously impacting the safety, durability, and maintainability of bridge structures. Especially for critical bridges spanning rivers and seas, with long spans and heavy loads, structural anomalies or failures can trigger major safety accidents, causing severe economic losses and casualties. Therefore, research on bridge structural damage identification and health assessment technologies is of great significance for ensuring safe bridge operation and promoting intelligent maintenance.

[0003] Currently, bridge structural health monitoring systems widely deploy various sensors, such as those for acceleration, strain, and displacement, to collect real-time dynamic response data of the structure and conduct damage identification and condition assessment using data-driven methods. Existing methods mainly rely on the extraction and classification of single time-series features, failing to effectively integrate the spatial dependencies between multiple sensors and multimodal signals, and mostly neglecting the modeling capabilities of frequency domain features and periodic changes. Furthermore, existing models suffer from low identification accuracy and poor generalization ability when dealing with complex situations such as non-stationary signals, local abrupt changes, or multi-scale modal responses, making it difficult to meet the high-precision requirements for bridge damage location, severity assessment, and trend prediction in practical engineering applications.

[0004] Therefore, designing a method to improve the accuracy of bridge structural damage identification is an urgent technical problem that needs to be solved. Summary of the Invention

[0005] To address the shortcomings of the existing technologies, the technical problem to be solved by this invention is: how to provide a bridge structure damage identification method that combines multi-scale frequency domain modeling and large language modeling, which can comprehensively consider the features of time dimension, frequency dimension, and channel dimension, integrate multi-scale spectrum modeling and deep semantic reasoning mechanism, and fully explore the correlation, periodicity and abrupt change information in bridge structure state data, thereby improving the accuracy of bridge structure damage identification.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] A bridge structural damage identification method combining multi-scale frequency domain modeling and large language modeling includes:

[0008] S1: Extract numerical statistical information of vibration acceleration time-series signals at multiple locations of the bridge structure under test;

[0009] S2: Based on the vibration acceleration time-series signals at multiple locations of the bridge structure under test, the waveform transformation network performs causal dilation convolution, local attention modeling, and structural state classification to obtain structural state feature labels;

[0010] S3: Multi-resolution time imaging of vibration acceleration time-series signals at multiple locations of the bridge structure under test is performed through the mixer module. The time image feature decomposition is achieved using the three-axis attention mechanism to obtain short-term seasonal time features, long-term trend time features, and cross-channel time features. Then, mixed features are generated through feature mixing.

[0011] S4: Generate structured text prompt templates using structural state feature labels and numerical statistical information;

[0012] S5: Input the structured text prompt template and the hybrid features together into the large language model for inference to obtain the bridge structural damage identification results.

[0013] Preferably, in step S1, the numerical statistical information extracted from the vibration acceleration time-series signals at multiple locations of the bridge structure under test includes the minimum value, maximum value, mean value, standard deviation, and peak frequency.

[0014] Preferably, in step S2, the processing steps of the waveform transformation network include:

[0015] S201: Perform wavelet transform and mapping on the vibration acceleration time-series signals at multiple locations of the bridge structure under test to obtain wavelet transform features;

[0016] S202: Encode the wavelet transform features by relative position to obtain the initial features;

[0017] S203: The first feature is obtained by modeling the initial features using causal dilated convolution and local attention through a Waveformer encoder;

[0018] S204: The second feature is obtained by decoding the first feature after dimensionality reduction using the Waveformer decoder;

[0019] S205: Input the second feature into the auxiliary classification head to classify the structural state and obtain the structural state feature label.

[0020] Preferably, in step S201, after performing discrete wavelet transform on the signal of each channel in the vibration acceleration time series signal at multiple locations of the bridge structure under test, the high-frequency and low-frequency components generated at each level are expanded and spliced, and the spliced ​​result is projected to the required embedding dimension through a fully connected layer to obtain wavelet transform features.

[0021] The formula is expressed as:

[0022]

[0023] In the formula: Describes the wavelet transform feature of the i-th channel; x i This represents the signal of the i-th channel in the time-series signal of vibration acceleration at multiple locations on the bridge structure under test; DWT represents wavelet transform operation; Flatten represents flattening operation; FC represents fully connected layer.

[0024] Preferably, in step S302, the Waveformer encoder includes a cascaded feedforward neural network, a first Dropout regularization layer, a local window attention layer, a dilated causal convolutional layer, and a second regularization layer.

[0025] Specifically, the feature obtained by fusing the input of the Waveformer encoder with the output of the first Dropout regularization layer is used as the input of the local window attention layer; the feature obtained by fusing the input of the Waveformer encoder with the output of the first Dropout regularization layer is then fused with the output of the second regularization layer to obtain the first feature output by the Waveformer encoder.

[0026] Preferably, in step S205, the auxiliary classification head processing steps include:

[0027] S2051: After applying the second feature through a linear layer, perform global average pooling along the time dimension to obtain the average pooled feature.

[0028] The formula is expressed as:

[0029]

[0030] S2052: Based on average pooling features using a fully connected layer with a Sigmoid activation function. Perform multi-label classification to obtain structural state feature labels.

[0031] The formula is expressed as:

[0032]

[0033] In the formula: σ represents the Sigmoid activation function; W represents the weight matrix of the fully connected layer; b is the bias term;

[0034] Among them, structural state feature labels This includes normal signal strength, frequency shift, periodic enhancement, and intermittent abrupt changes.

[0035] Preferably, in step S3, the processing steps of the mixer module include:

[0036] S301: Perform R independent downsampling operations on the vibration acceleration time-series signal to obtain time series of R scales;

[0037] S302: Perform multi-resolution time imaging on the time series at R scales respectively to obtain two-dimensional time-frequency images with K resolutions;

[0038] S303: Encodes two-dimensional time-frequency images with K resolutions in R time series using convolutional layers to obtain R*K time images suitable for the three-axis attention mechanism;

[0039] S304: Through a three-axis attention mechanism, attention is extracted from R*K time images for short-term transformations within a period (time dimension), long-term transformations between periods (frequency dimension), and different channels (channel dimension), resulting in short-term seasonal time feature maps and long-term trend time feature maps at R scales, as well as 1×1×c channel-weighted features, where c represents the number of sensors, i.e., the number of channels.

[0040] S305: Combine short-term seasonal time feature maps and long-term trend time feature maps at R scales to obtain short-term-long-term fused features;

[0041] S306: Multiply the 1×1×c channel weighted feature with the short-term-long-term fusion feature to obtain the mixed feature.

[0042] Preferably, step S306 specifically includes the following steps:

[0043] S3061: Short-term seasonal temporal feature maps for R scales Perform bottom-up, coarse-to-fine depth-separable convolution processing to obtain local features H. season ;

[0044] The formula is expressed as:

[0045]

[0046] S3062: Long-term trend time feature plots at R scales using a transformer module A top-down interaction from fine to coarse scale is performed to obtain the global feature H.trend ;

[0047] The formula is expressed as:

[0048]

[0049] S3063: Local features H are transformed through a residual mixing layer season and global features H trend By performing fusion, the short-term and long-term fusion features H are obtained. mix ;

[0050] The formula is expressed as:

[0051] H mix =H season +H trend +FFN(H season ‖H trend );

[0052] In the formula: || represents a connection; FFN represents a feedforward neural network.

[0053] Preferably, in step S5, after embedding the structured text prompt template, the hybrid features are mapped to a dimension that matches the large language model space and appended to the embedding of the structured text prompt template to form a comprehensive embedding vector. The comprehensive embedding vector is then input into the large language model for inference to generate bridge structure damage identification results.

[0054] The bridge structural damage identification method combining multi-scale frequency domain modeling and large language model in this invention has the following advantages compared with existing technologies:

[0055] This invention extracts vibration acceleration time-series signals from multiple locations on the bridge structure under test and utilizes a waveform transformation network for causal dilation convolution and local attention modeling, addressing the problem of insufficient utilization of spatial dependencies between multi-sensor signals in existing methods. Causal dilation convolution, through its unique dilation structure, expands the receptive field during signal processing, enabling the network to capture the propagation patterns of signals at different locations in the time dimension, thereby perceiving the spatial propagation paths and mutual influences of signals. Simultaneously, local attention modeling further focuses on the signal features at key locations, enhancing the utilization of spatial correlations by calculating attention weights between signals at different locations. For example, when damage occurs in a certain part of the bridge, its vibration signal will propagate to sensors at other locations in a specific manner. Causal dilation convolution and local attention modeling can accurately capture this propagation pattern and spatial correlation, generating more accurate structural state feature labels, thereby improving the accuracy of bridge structural damage identification.

[0056] The mixer module in this invention effectively solves the problem that existing methods neglect the modeling ability of frequency domain features and periodic changes, making it difficult to accurately reflect the state changes of bridge structures at different time scales. This is achieved through multi-resolution temporal imaging and time image feature decomposition using a triaxial attention mechanism. Specifically, multi-resolution temporal imaging generates time-frequency images of vibration acceleration time-series signals at different time resolutions, extracting feature changes of the signal at different time scales and capturing the frequency domain information hidden within the signal. Simultaneously, the time image feature decomposition using the triaxial attention mechanism further decomposes the signal into seasonal temporal features, long-term trend temporal features, and cross-channel temporal features: seasonal temporal features accurately reflect the bridge vibration variation caused by periodic factors; long-term trend temporal features reveal the performance degradation trend of the bridge structure over a long period due to material aging and long-term loads; and cross-channel temporal features capture the coordinated change information between signals from sensors at different locations, reflecting the correlation between multi-sensor signals in the frequency domain and periodicity. By fusing these different types of features, the sensitivity and modeling ability of bridge structural state changes are improved, providing richer information for accurately identifying the state of the bridge structure.

[0057] This invention generates structured text prompt templates by combining structural state feature labels and numerical statistical information. These templates, along with hybrid features, are then input into a large language model for inference. This effectively overcomes the problems of low recognition accuracy and poor generalization ability of existing models when dealing with complex situations such as non-stationary signals, local abrupt changes, or multi-scale modal responses in bridge structures. The structural state feature labels contain key structural state information extracted by waveform transform networks, while the numerical statistical information provides the basic statistical characteristics of the signal. This multi-source information is presented in the form of structured text prompt templates, providing comprehensive and rich input for the large language model. Simultaneously, the large language model possesses powerful language understanding and reasoning capabilities, automatically learning the complex relationship between input information and the bridge structural state. Without the need for manually setting numerous rules and thresholds, it can quickly and accurately infer bridge structural damage identification results based on the multi-source input information when facing different types of complex signals, thereby significantly improving the accuracy and generalization ability of bridge structural damage identification. Attached Figure Description

[0058] To make the objectives, technical solutions, and advantages of the invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein:

[0059] Figure 1 A logical block diagram for a bridge structural damage identification method that combines multi-scale frequency domain modeling and a large language model.

[0060] Figure 2 This is a schematic diagram of multi-resolution temporal imaging.

[0061] Figure 3 This is a schematic diagram of temporal image feature decomposition.

[0062] Figure 4 A schematic diagram of layered blending of multi-scale seasonal and long-term trend time feature images.

[0063] Figure 5 It is a BSSI-LLM framework.

[0064] Figure 6 This is a template for prompts related to bridge monitoring data. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but only to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0066] The following detailed explanation illustrates the specific implementation methods:

[0067] Example:

[0068] This embodiment discloses a bridge structure damage identification method that combines multi-scale frequency domain modeling with a large language model.

[0069] like Figure 1 As shown, the bridge structural damage identification method combining multi-scale frequency domain modeling and large language model includes:

[0070] S1: Extract numerical statistical information of vibration acceleration time-series signals at multiple locations of the bridge structure under test;

[0071] S2: Based on the vibration acceleration time-series signals at multiple locations of the bridge structure under test, the waveform transformation network performs causal dilation convolution, local attention modeling, and structural state classification to obtain structural state feature labels;

[0072] S3: Multi-resolution time imaging of vibration acceleration time-series signals at multiple locations of the bridge structure under test is performed through the mixer module. The time image feature decomposition is achieved using the three-axis attention mechanism to obtain short-term seasonal time features, long-term trend time features, and cross-channel time features. Then, mixed features are generated through feature mixing.

[0073] S4: Generate structured text prompt templates using structural state feature labels and numerical statistical information;

[0074] S5: Input the structured text prompt template and hybrid features together into the pre-trained large language model for reasoning to obtain the bridge structural damage identification results.

[0075] This invention extracts vibration acceleration time-series signals from multiple locations on the bridge structure under test and utilizes a waveform transformation network for causal dilation convolution and local attention modeling, addressing the problem of insufficient utilization of spatial dependencies between multi-sensor signals in existing methods. Causal dilation convolution, through its unique dilation structure, expands the receptive field during signal processing, enabling the network to capture the propagation patterns of signals at different locations in the time dimension, thereby perceiving the spatial propagation paths and mutual influences of signals. Simultaneously, local attention modeling further focuses on the signal features at key locations, enhancing the utilization of spatial correlations by calculating attention weights between signals at different locations. For example, when damage occurs in a certain part of the bridge, its vibration signal will propagate to sensors at other locations in a specific manner. Causal dilation convolution and local attention modeling can accurately capture this propagation pattern and spatial correlation, generating more accurate structural state feature labels, thereby improving the accuracy of bridge structural damage identification.

[0076] The mixer module in this invention effectively solves the problem that existing methods neglect the modeling ability of frequency domain features and periodic changes, making it difficult to accurately reflect the state changes of bridge structures at different time scales. This is achieved through multi-resolution temporal imaging and time image feature decomposition using a triaxial attention mechanism. Specifically, multi-resolution temporal imaging generates time-frequency images of vibration acceleration time-series signals at different time resolutions, extracting feature changes of the signal at different time scales and capturing the frequency domain information hidden within the signal. Simultaneously, the time image feature decomposition using the triaxial attention mechanism further decomposes the signal into seasonal temporal features, long-term trend temporal features, and cross-channel temporal features: seasonal temporal features accurately reflect the bridge vibration variation caused by periodic factors; long-term trend temporal features reveal the performance degradation trend of the bridge structure over a long period due to material aging and long-term loads; and cross-channel temporal features capture the coordinated change information between signals from sensors at different locations, reflecting the correlation between multi-sensor signals in the frequency domain and periodicity. By fusing these different types of features, the sensitivity and modeling ability of bridge structural state changes are improved, providing richer information for accurately identifying the state of the bridge structure.

[0077] This invention generates structured text prompt templates by combining structural state feature labels and numerical statistical information. These templates, along with hybrid features, are then input into a large language model for inference. This effectively overcomes the problems of low recognition accuracy and poor generalization ability of existing models when dealing with complex situations such as non-stationary signals, local abrupt changes, or multi-scale modal responses in bridge structures. The structural state feature labels contain key structural state information extracted by waveform transform networks, while the numerical statistical information provides the basic statistical characteristics of the signal. This multi-source information is presented in the form of structured text prompt templates, providing comprehensive and rich input for the large language model. Simultaneously, the large language model possesses powerful language understanding and reasoning capabilities, automatically learning the complex relationship between input information and the bridge structural state. Without the need for manually setting numerous rules and thresholds, it can quickly and accurately infer bridge structural damage identification results based on the multi-source input information when facing different types of complex signals, thereby significantly improving the accuracy and generalization ability of bridge structural damage identification.

[0078] To better illustrate the technical solution of the present invention, this embodiment is described in the following parts.

[0079] I. Numerical Statistical Information

[0080] In this embodiment, the vibration acceleration time sequence signal The numerical statistics extracted (B is the batch size, T is the sequence length, and C is the number of sensor channels) include the minimum and maximum values, mean and standard deviation, and peak frequency.

[0081] The peak frequency is calculated using a Fast Fourier Transform (FFT):

[0082] f peak =argmax f |FFT(x)| 2 .

[0083] II. Waveformer Net

[0084] To generate meaningful structural state representations from raw acceleration signals, we propose a lightweight information model specifically designed for bridge structural health monitoring (SHM): WaveformerNet. Figure 1 As shown in the left half, the model contains three core components: Wavelet Transform, Waveformer encoder-decoder, and auxiliary classification head (including linear and classification layers).

[0085] First, wavelet decomposition can simultaneously capture both low-frequency and high-frequency components. Second, the relative position encoding mechanism can maintain the perception of the temporal order of events. Third, dilated causal convolution and local window attention models can capture local dynamic features such as temporal causality and short-term transients. These mechanisms enable the encoder to track subtle modal shifts under structural state changes. In the decoder, the GRU module optimizes long-short-term dependencies, followed by dimensionality reduction through distillation layers. Finally, an auxiliary classification head generates structural state feature labels, providing semantic cues for the downstream LLM-based classification module. Due to the non-stationarity and modal sparsity of vibration signals, Waveformer Net performs excellently in structural health monitoring tasks. Furthermore, damage often exhibits periodicity or frequency variations; therefore, this model significantly improves classification accuracy by generating rich features and semantic cues.

[0086] Specifically, the processing steps of the shape transformation network include:

[0087] S201: Perform wavelet transform and mapping on the vibration acceleration time-series signal to obtain wavelet transform features;

[0088] S202: Encode the wavelet transform features by relative position to obtain the initial features;

[0089] S203: The first feature is obtained by modeling the initial features using causal dilated convolution and local attention through a Waveformer encoder;

[0090] S204: After the first feature is reduced in dimensionality by the distillation layer, the second feature is obtained by the Waveformer decoder through gated recurrent unit structure processing and scaling point attention mechanism processing of the dimensionality-reduced first feature.

[0091] S205: Input the second feature into the linear layer and the classification layer to classify the structural state and obtain the structural state feature label.

[0092] 1. Wavelet Transform

[0093] To effectively extract hierarchical frequency features from raw bridge vibration signals, we introduced a wavelet-based coding module into WaveformerNet. Considering the characteristics of bridge structural health monitoring (SHM), vibration signals typically exhibit complex frequency patterns: high-frequency components may reflect localized cracks or loose joints, while low-frequency drift often corresponds to a decrease in overall stiffness. To capture these diverse frequency information that varies over time, we employed discrete wavelet transform as a frequency-selective feature encoder before modeling.

[0094] The formula for wavelet transform is expressed as follows:

[0095] X(a,b)=∫x(t)ψ a,b (t)dt;

[0096] In the formula: a represents the scaling parameter controlling the frequency resolution; b represents the translation parameter of the time series offset; ψ a,b (t) represents the wavelet function;

[0097] wavelet function ψ a,b (t) is derived from the mother wavelet ψ0(t), and the formula is expressed as:

[0098] ψ j,k (t)=2 -j / 2 ψ0(2 -j tk);

[0099] In this embodiment, the computationally efficient discrete wavelet transform is used, where the coefficients c j,k Calculated in the following way:

[0100]

[0101] In the formula: J represents the decomposition level (frequency scale); K represents the time position. These coefficients represent the local frequency components of the signal at different resolutions.

[0102] In this embodiment, after performing discrete wavelet transform on the signal of each channel in the vibration acceleration time series signal, the detail coefficients (representing the high-frequency part) and approximation coefficients (representing the low-frequency part) generated at each level are expanded and spliced, and the spliced ​​result is projected to the required embedding dimension through a fully connected layer to obtain the wavelet transform features.

[0103] The formula is expressed as:

[0104]

[0105] In the formula: Describes the wavelet transform feature of the i-th channel; x i This represents the signal of the i-th channel; DWT represents the wavelet transform operation; Flatten represents the flattening operation; and FC represents the fully connected layer. In this way, Waveformer Net gains the ability to identify periodic trends, intermittent abrupt changes, and local anomalies. These capabilities are crucial for accurately identifying bridge structural damage.

[0106] 2. Waveformer

[0107] To effectively model the time-series dependencies and periodic structures in bridge acceleration vibration signals, the Waveformer algorithm integrates dilated causal convolution, local window attention mechanism, and gated recurrent unit (GRU) structure. This combined design ensures both global perception capability and local sensitivity, which is crucial for detecting structural damage patterns at different time scales.

[0108] Dilated causal convolution is used to capture long-term dependencies for input sequences. The formula for calculating the output y(t) at time t is:

[0109]

[0110] Where w(k) are the learnable convolution weights, k represents the kernel size, and r is the dilation rate. This operation enables the model to expand its receptive field exponentially with depth, making it suitable for efficient analysis of long bridge vibration sequences.

[0111] To ensure that the local attention mechanism effectively captures short-term changes, we employ a local window attention mechanism, dividing the input sequence into overlapping windows of fixed length w. Within each window, a scaled dot product attention mechanism is applied:

[0112]

[0113] This allows Waveformer to focus on transient anomalies, such as responses to sudden vibrations or micro-damage. GRU is introduced to preserve sequence information and mitigate the vanishing gradient problem. Specifically, the update gate controls the degree of fusion between the current hidden state and historical states, while the reset gate adjusts the proportion of past information forgotten by the current input. This mechanism allows the model to highlight the modeling of response changes to anomalous signals without losing global context, thereby improving its ability to capture the non-stationarity of structural responses and temporal evolution patterns.

[0114] Specifically:

[0115] The Waveformer encoder effectively captures the dependencies between different positions within the input waveform sequence through a multi-layered structure, including a self-attention mechanism, and extracts a high-dimensional representation containing rich semantic and temporal features.

[0116] The Waveformer encoder consists of a cascaded feedforward neural network, a first Dropout regularization layer, a local window attention layer, a dilated causal convolutional layer, and a second regularization layer.

[0117] Specifically, the feature obtained by fusing the input of the Waveformer encoder with the output of the first Dropout regularization layer is used as the input of the local window attention layer; the feature obtained by fusing the input of the Waveformer encoder with the output of the first Dropout regularization layer is then fused with the output of the second regularization layer to obtain the first feature output by the Waveformer encoder.

[0118] The Waveformer decoder generates the target waveform sequence step by step based on the characteristics of the encoder output and the task requirements.

[0119] The Waveformer decoder decodes the first feature using GRU and a scaling point attention mechanism. Each GRU encoder processes a specific time segment, and they do not simply stack together, but rather work in an ordered, alternating manner through periodic cycles and forward / backward interactions. GRUs interact to handle forward / backward dependencies within the same sequence, employing a forward + backward + periodic modeling strategy. Scaling point attention introduces a scaling factor into the computation. Each GRU encoder's weight output is adjusted by the scaling point attention mechanism after or between each other, dynamically adjusting the "attention weights" of information across different time segments to enhance the ability to identify outliers and abrupt changes. The scaling point attention mechanism is an existing attention mechanism that introduces a scaling factor into the attention score calculation, typically the reciprocal of the square root of the feature dimension. This mechanism effectively adjusts the distribution of attention weights, avoiding the problem of vanishing gradients and training difficulties caused by excessively high feature dimensions, large dot product values, and small gradients in the Softmax function.

[0120] 3. Auxiliary classification head

[0121] The processing steps for the auxiliary classification head include:

[0122] S2051: Perform global average pooling on the second feature after linear layer processing to obtain the average pooled feature.

[0123] The formula is expressed as:

[0124]

[0125] S2052: Based on average pooling features using a fully connected layer with a Sigmoid activation function. Perform multi-label classification to obtain structural state feature labels.

[0126] The formula is expressed as:

[0127]

[0128] In the formula: σ represents the Sigmoid activation function, which is used to compress the output of the fully connected layer to the [0,1] interval, and is suitable for multi-label prediction tasks; W represents the weight matrix of the fully connected layer, and b is the bias term. Together, they represent the average pooling representation of the output of GRU or Waveformer. Projected onto the structural state label space to generate structural state feature labels. Structural state feature labels It is a vector consisting of multiple 0s and 1s (or probability values) representing the multiple label categories to which a sample belongs. It is the standard output in multi-label classification tasks.

[0129] Among them, structural state feature labels This includes normal signal strength, frequency shift, periodic enhancement, and intermittent abrupt changes.

[0130] III. MixerBlock (Mixer Module)

[0131] To capture the complex time-frequency dynamics and inter-channel correlations in bridge vibration signals, we introduced a dedicated multi-scale feature modeling module—the MixerBlock. For example... Figure 1 As shown in the right half, the process first downsamples the original vibration signal to generate multiple temporal resolutions, preserving structural features at different time scales while facilitating hierarchical analysis. After downsampling, the system performs multi-resolution temporal imaging. Temporal image feature decomposition accurately captures sudden environmental disturbances, progressive structural degradation, and inter-sensor correlations through attention mechanisms along the time, frequency, and channel axes. This integrated design significantly improves the model's recognition accuracy and adaptability under complex bridge acoustic health monitoring conditions. Finally, the multi-scale hybrid unit (MSM) integrates bottom-up hybridization based on depth convolution and top-down hybridization based on transformers. This dual-path fusion strategy ensures that the final representation retains both fine-grained local details and global high-order features. The hybrid block effectively integrates temporal, frequency, and spatial features across resolutions, enabling accurate identification of local damage patterns, long-term degradation, and inter-sensor correlations in bridge vibration data.

[0132] Specifically, the processing steps of the mixer module include:

[0133] S301: Perform R independent downsampling operations on the vibration acceleration time-series signal to obtain time series of R scales;

[0134] S302: Perform multi-resolution time imaging on the time series at R scales respectively to obtain two-dimensional time-frequency images with K resolutions;

[0135] S303: Encodes two-dimensional time-frequency images with K resolutions in R time series using convolutional layers to obtain R*K time images suitable for the three-axis attention mechanism;

[0136] S304: Through a three-axis attention mechanism, attention is extracted from R*K time images for short-term transformations within a period (time dimension), long-term transformations between periods (frequency dimension), and different channels (channel dimension), resulting in short-term seasonal time feature maps and long-term trend time feature maps at R scales, as well as 1×1×c channel-weighted features, where c represents the number of sensors, i.e., the number of channels.

[0137] Among them, the 1×1×c channel weighted feature is generated by weighting the channel weights of R*K time images;

[0138] S305: Combine short-term seasonal time feature maps and long-term trend time feature maps at R scales to obtain short-term-long-term fused features;

[0139] S306: Multiply the 1×1×c channel weighted feature with the short-term-long-term fusion feature to obtain the mixed feature.

[0140] 1. Multi-scale time series generation

[0141] To extract structural features at different time resolutions from bridge acceleration and vibration data, we first perform hierarchical downsampling on the original time-series data. This operation, when generating compressed sequences, preserves the overall trend and characteristics while effectively reducing the temporal granularity by systematically increasing the interval between data points. This decomposition method is crucial in structural health monitoring (SHM) scenarios—scenarios that require simultaneous analysis of short-term anomalies (such as impact events) and long-term degradation (such as stiffness loss).

[0142] For vibration acceleration time sequence signals The downsampling operation yields time series at R scales:

[0143] X (r) =Downsample r (X), r∈{1,2,...,R};

[0144] Each parameter The corresponding low-time-resolution version of the vibration acceleration time-series signal, where S r The step size or sampling factor represents the resolution r; average pooling is used as the downsampling method.

[0145]

[0146] This operation ensures that periodic patterns of different frequencies and the structure’s response under different loading conditions are preserved at the appropriate scale (e.g., short-term anomalous mutations and long-term fatigue damage).

[0147] 2. Multi-resolution temporal imaging

[0148] like Figure 2 As shown, this method transforms the input time series into a two-dimensional time-domain image set of (R=3)×(K=4) through frequency decomposition and spatial reshaping. A Fast Fourier Transform (FFT) is performed on the time series at each scale to extract the first K frequency components as one-dimensional time series at K resolutions.

[0149] The first K frequency components refer to the fact that the entire time series is considered as a sequence mainly composed of K periods, and each period corresponds to a p. For example, if the p of the first period corresponds to 3, then the time series is in a period of 3. If the p of the second period corresponds to 4, then it is in a period of 4. There are K p components in K periods.

[0150] The formula is expressed as:

[0151]

[0152] In the formula: f represents the FFT amplitude; k It is the kth clock frequency; Corresponding period length;

[0153] K resolution one-dimensional time series are reshaped by zero-filling and one-dimensional to two-dimensional transformation to obtain K resolution two-dimensional time-frequency images;

[0154] The formula is expressed as:

[0155]

[0156] In the formula: MRTI(X) represents the two-dimensional time-frequency image of the vibration acceleration time-series signal X; A two-dimensional frequency plot representing the k-th dominant frequency at scale r; x r This represents the time series at the r-th scale; Padding represents the padding of the time series x. r Fill with 0s to make it a length of p. k Integer multiples of; Reshape indicates reshaping a one-dimensional sequence according to period p. k Convert to shape A two-dimensional matrix, i.e., a two-dimensional time-frequency image.

[0157] Each two-dimensional time-frequency image Retain time periodicity p at scale r k and frequency f kLocalization structure; in order to focus on the most informative frequency band f k We employ a dynamic Top-k frequency selection strategy—instead of fixing the top k frequency bands, we retain the top 6% of the total frequency energy. This dynamic strategy is suitable for bridge monitoring systems because the effective frequency composition varies significantly depending on structure type, span length, support conditions, and operating environment. By adaptively selecting the frequency bands that contribute the most energy, MRTI provides compact and highly discriminative multi-resolution inputs for downstream decomposition and attention modeling.

[0158] 3. Temporal Image Feature Decomposition

[0159] like Figure 3 As shown, a time-image feature decomposition module is introduced to analyze the different patterns embedded in multi-resolution frequency images.

[0160] set up This indicates that at a given time scale, K represents the time image generated by the first K frequencies retained under multi-resolution time imaging, and C represents the number of channels, referring to different accelerometers.

[0161] Applying three projection operators Each function f(·) represents a one-dimensional convolution along its corresponding axis. S represents the transformation within a period; T represents the variation during the period; C characterizes the dependencies between different channels, such as synchronous modal responses of different spans or orientations. Indicates time resolution S i After FFT and Top-k frequency extraction, the reconstructed frequency-period two-dimensional time image is retained. Then, the images from different channels are stitched together to create a three-dimensional image. Three functions f... seasonal (·)f trend (·)f channel (·) is a one-dimensional convolution in three dimensions; its purpose is to generate feature representations suitable for attention mechanisms.

[0162] By setting independent attention heads along each axis, different features in bridge vibration data can be effectively extracted:

[0163] Z TID =Attn time (S)+Attn freq (T)+Attn channel (C);

[0164] In the formula: Attn time (S) indicates applying time-dimensional attention to short-term seasonal components; Attn freq (T) represents applying frequency-dimensional attention to the frequency components; Attn channel (C) indicates that channel dimension attention is applied to the channel components.

[0165] 4. Multi-scale hybrid unit

[0166] like Figure 4 As shown, the processing steps of the multi-scale mixing unit include:

[0167] S3051: Short-term seasonal temporal feature maps for R scales Perform bottom-up, coarse-to-fine depth-separable convolution processing to obtain local features H. season ;

[0168] The formula is expressed as:

[0169]

[0170] S3052: Long-term trend time feature plots at R scales using a transformer module A top-down interaction from fine to coarse scale is performed to obtain the global feature H. trend ;

[0171] The formula is expressed as:

[0172]

[0173] S3053: Local features H are transformed through a residual mixing layer. season and global features H trend By performing fusion, the short-term and long-term fusion features H are obtained. mix ;

[0174] The formula is expressed as:

[0175] H mix =H season +H trend +FFN(H season ‖H trend );

[0176] In the formula: || represents a connection; FFN represents a feedforward neural network.

[0177] IV. BSSI-LLM (Bridge Structural Damage Identification with Large Language Model)

[0178] To achieve high-level reasoning for bridging state classification, we specifically designed a prompting module and an input module. This design can simultaneously encode vibration signal features and structural state feature labels into a large language model. Based on this architecture, we propose the BSDI-LLM framework. This framework is as follows... Figure 5 As shown.

[0179] Semantic-aware reasoning is achieved by integrating a frozen large language model (LLM) and text prompts. This module utilizes structural state feature labels and numerical statistics extracted from preceding modules to generate descriptive prompts that encode key information about structural dynamics. Specifically, based on the periodic and transient patterns captured by WaveformerNet, we derive structural state feature labels. These structural state feature labels, together with statistical numerical information, constitute a structured text prompt template, such as... Figure 6 As shown. Specifically, first determine the format of the structured text prompt template, such as using key-value pairs, and then fill in the generated structural state feature labels and numerical statistics into the corresponding fields according to the template structure.

[0180] The prompt is directly embedded through a pre-trained large language model (LLaMA-7B) without fine-tuning. During inference, after embedding the structured text prompt template, the mixed features are mapped to a dimension that matches the large language model space and appended to the embedding of the structured text prompt template to form a comprehensive embedding vector. This comprehensive embedding vector is then input into the large language model for inference, ultimately generating the classification result.

[0181] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit the technical solutions. Those skilled in the art should understand that any modifications or equivalent substitutions to the technical solutions of the present invention without departing from the spirit and scope of the present invention should be covered within the scope of the claims of the present invention.

Claims

1. A bridge structural damage identification method combining multi-scale frequency domain modeling and large language model, characterized in that, include: S1: Extract numerical statistical information of vibration acceleration time-series signals at multiple locations of the bridge structure under test; S2: Based on the vibration acceleration time-series signals at multiple locations of the bridge structure under test, the waveform transformation network performs causal dilation convolution, local attention modeling, and structural state classification to obtain structural state feature labels; S3: Multi-resolution time imaging of vibration acceleration time-series signals at multiple locations of the bridge structure under test is performed through the mixer module. The time image feature decomposition is achieved using the three-axis attention mechanism to obtain short-term seasonal time features, long-term trend time features, and cross-channel time features. Then, mixed features are generated through feature mixing. S4: Generate structured text prompt templates using structural state feature labels and numerical statistical information; S5: Input the structured text prompt template and the hybrid features together into the large language model for inference to obtain the bridge structural damage identification results.

2. The bridge structural damage identification method based on joint multi-scale frequency domain modeling and large language model as described in claim 1, characterized in that: In step S1, the numerical statistical information extracted from the vibration acceleration time-series signals at multiple locations of the bridge structure under test includes the minimum value, maximum value, mean value, standard deviation, and peak frequency.

3. The bridge structural damage identification method based on joint multi-scale frequency domain modeling and large language model as described in claim 1, characterized in that: In step S2, the processing steps of the waveform transformation network include: S201: Perform wavelet transform and mapping on the vibration acceleration time-series signals at multiple locations of the bridge structure under test to obtain wavelet transform features; S202: Encode the wavelet transform features by relative position to obtain the initial features; S203: The first feature is obtained by modeling the initial features using causal dilated convolution and local attention through a Waveformer encoder; S204: The second feature is obtained by decoding the first feature after dimensionality reduction using the Waveformer decoder; S205: Input the second feature into the auxiliary classification head to classify the structural state and obtain the structural state feature label.

4. The bridge structural damage identification method based on combined multi-scale frequency domain modeling and large language model as described in claim 3, characterized in that: In step S201, after performing discrete wavelet transform on the signal of each channel in the vibration acceleration time series signal at multiple locations of the bridge structure under test, the high-frequency and low-frequency components generated at each level are expanded and spliced, and the spliced ​​result is projected to the required embedding dimension through a fully connected layer to obtain the wavelet transform features. The formula is expressed as: In the formula: Describes the wavelet transform feature of the i-th channel; x i This represents the signal of the i-th channel in the time-series signal of vibration acceleration at multiple locations on the bridge structure under test; DWT represents wavelet transform operation; Flatten represents flattening operation; FC represents fully connected layer.

5. The bridge structural damage identification method based on joint multi-scale frequency domain modeling and large language model as described in claim 3, characterized in that: In step S302, the Waveformer encoder includes a cascaded feedforward neural network, a first Dropout regularization layer, a local window attention layer, a dilated causal convolutional layer, and a second regularization layer. Specifically, the feature obtained by fusing the input of the Waveformer encoder with the output of the first Dropout regularization layer is used as the input of the local window attention layer; the feature obtained by fusing the input of the Waveformer encoder with the output of the first Dropout regularization layer is then fused with the output of the second regularization layer to obtain the first feature output by the Waveformer encoder.

6. The bridge structural damage identification method based on joint multi-scale frequency domain modeling and large language model as described in claim 1, characterized in that: In step S205, the processing steps for the auxiliary classification head include: S2051: After applying the second feature through a linear layer, perform global average pooling along the time dimension to obtain the average pooled feature. S2052: Based on average pooling features using a fully connected layer with a Sigmoid activation function. Perform multi-label classification to obtain structural state feature labels. The formula is expressed as: In the formula: σ represents the Sigmoid activation function; W represents the weight matrix of the fully connected layer; b is the bias term.

7. The bridge structural damage identification method based on joint multi-scale frequency domain modeling and large language model as described in claim 1, characterized in that: In step S3, the processing steps of the mixer module include: S301: Perform R independent downsampling operations on the vibration acceleration time-series signal to obtain time series of R scales; S302: Perform multi-resolution time imaging on the time series at R scales respectively to obtain two-dimensional time-frequency images with K resolutions; S303: Encodes two-dimensional time-frequency images with K resolutions in R time series using convolutional layers to obtain R*K time images suitable for the three-axis attention mechanism; S304: By using a three-axis attention mechanism, short-term transformations within periods, long-term transformations between periods, and attention extraction from different channels are performed on R*K time images to obtain short-term seasonal time feature maps and long-term trend time feature maps at R scales, as well as channel-weighted features. S305: Combine short-term seasonal time feature maps and long-term trend time feature maps at R scales to obtain short-term-long-term fused features; S306: Multiply the 1×1×c channel weighted feature with the short-term-long-term fusion feature to obtain the mixed feature.

8. The bridge structural damage identification method based on joint multi-scale frequency domain modeling and large language model as described in claim 7, characterized in that: Step S305 specifically includes the following steps: S3051: Short-term seasonal temporal feature maps for R scales Perform bottom-up, coarse-to-fine depth-separable convolution processing to obtain local features H. season ; The formula is expressed as: S3052: Long-term trend time feature plots at R scales using a transformer module A top-down interaction from fine to coarse scale is performed to obtain the global feature H. trend ; The formula is expressed as: S3053: Local features H are transformed through a residual mixing layer. season and global features H trend By performing fusion, the short-term and long-term fusion features H are obtained. mix ; The formula is expressed as: H mix =H season +H trend +FFN (H season ‖H trend ); In the formula: || represents a connection; FFN represents a feedforward neural network.

9. The bridge structural damage identification method based on joint multi-scale frequency domain modeling and large language model as described in claim 1, characterized in that: In step S5, after embedding the structured text prompt template, the hybrid features are mapped to a dimension that matches the large language model space and appended to the embedding of the structured text prompt template to form a comprehensive embedding vector. The comprehensive embedding vector is then input into the large language model for inference to generate the bridge structure damage identification result.

Images