DVS vibration signal semantic representation method based on prior knowledge enhancement

Abstract

The invention relates to a DVS vibration signal semantic representation method based on prior knowledge enhancement, and relates to the field of signal processing and artificial intelligence. The method comprises the following steps: (1) constructing a distributed vibration signal text enhancement data set; (2) cross-diffusing the standardized DVS vibration signal; (3) multi-band input alignment is carried out; (4) residual signal enhancement; (5) DVS vibration signal feature extraction; (6) aligning and mapping DVS vibration signal semantic features; and (7) fine adjustment of the DVSLLM model. According to the method, the technical problems that in the field of signal processing, sampling frequencies are not uniform, spatiotemporal features are difficult to extract, semantic information is difficult to represent and the like are solved, and a text enhancement data set is constructed by utilizing Deepseek-V3; a fine-grained interaction model from DVS vibration signals to natural language questions and answers is constructed through cross diffusion standardization, two-dimensional discrete transformation, multi-scale convolution, a CBAM attention mechanism, bilingual sense mapping alignment and DVSLLM model fine tuning, and an intelligent method is provided for road roadbed vibration event health monitoring.

Description

Technical Field

[0001] The present invention relates to the fields of signal processing and artificial intelligence, and mainly relates to a method for semantic representation and mapping of distributed vibration signals. Background Art

[0002] In the monitoring of urban road subgrades, distributed optical fiber monitoring technology is widely used in the field of infrastructure health monitoring due to its advantages of high sensitivity, real-time performance, and wide coverage. Distributed vibration sensing technology (DVS), represented by Rayleigh scattering, plays an important role in the monitoring of urban road subgrades. By real-time sensing of minute vibrations and strain changes inside the subgrade, continuous monitoring of key indicators such as vehicle loads, geological activities, and crack propagation can be achieved, and it can provide large-scale, all-weather, and high-precision assessment of the subgrade health status, timely warning of potential settlement, cracking, and uneven deformation problems, and providing a scientific basis for road maintenance and management.

[0003] Distributed vibration sensing technology faces many challenges in signal processing, including low signal-to-noise ratio, large amount of data, difficulty in event positioning and classification, and signal attenuation caused by fiber optic nonlinear effects. In terms of improving signal quality, scholars use methods such as wavelet transform and empirical mode decomposition (EMD) for noise reduction, and at the same time, deep learning methods are used to further optimize signal extraction and recognition. In terms of feature extraction, time-frequency analysis methods such as Fourier transform and short-time Fourier transform (STFT) are widely used, and machine learning and deep learning (such as CNN, LSTM) are combined for event detection and classification. To reduce computational complexity, methods such as principal component analysis (PCA) and autoencoder are used for data dimensionality reduction and compression. In terms of event positioning, researchers use cross-correlation method, time difference of arrival (TDOA), and end-to-end positioning technology of deep learning to improve event positioning accuracy. However, traditional DVS signal processing often relies on single-modal information extraction, is easily affected by noise interference, and has limited accuracy in event recognition in complex environments.

[0004] With the continuous development of artificial intelligence technology, especially the rise of multimodal semantic understanding, it provides new ideas and breakthroughs for the analysis of distributed vibration signals. Multimodal semantic understanding can integrate data from different sources. DVS signals can be fused with data such as images and texts to deeply explore the correlations between different modalities and enhance the detection ability of abnormal vibration events. Deep learning models, such as the Feature Transformer (Transformer) integrating self-attention mechanism, can learn the spatio-temporal features of different modalities and achieve semantic alignment, thus more accurately identifying event types, such as traffic accidents, geological disasters, construction vibrations, etc. In addition, the method based on Graph Neural Network (GNN) can model the spatial topology of the fiber optic sensing network to help explore the correlations between different monitoring points and optimize the ability to trace the source of abnormal events. Multimodal semantic understanding can also combine natural language processing technology to jointly analyze text data such as historical monitoring records and expert reports with DVS signals to form a knowledge-driven abnormal diagnosis system. Generally speaking, the introduction of multimodal semantic understanding provides higher accuracy, stronger robustness and wider adaptability for DVS signal analysis, promoting its popularization and application in fields such as urban road monitoring, pipeline safety and geological disaster warning. Summary of the Invention

[0005] Aiming at the problems existing in the above-mentioned prior art, the technical problem to be solved by the present invention is to provide a method for enhancing the semantic representation of DVS vibration signals based on prior knowledge, and its specific process is as Figure 1 shown.

[0006] The implementation steps of the technical solution are as follows:

[0007] (1) Construct a text-enhanced dataset for distributed vibration signals:

[0008] Aiming at the problems that the existing roadbed monitoring dataset has strong dependence on single working conditions, semantic disconnection between signals and texts, and lack of paired text descriptions to support the training of LLM models, a prior knowledge semantic enhancement framework is proposed. Through domain knowledge-driven text generation and multi-dimensional working condition coverage, the first text-enhanced dataset (DVSAG) supporting multimodal semantic understanding of distributed vibration signals is constructed.

[0009] The DVSAG dataset contains 266 pairs of distributed vibration signals and their fault types, that is, each group is "distributed vibration signal S r - fault label L r - working state ID - user prompt S t - text response R t"The five - element data chain includes all working conditions in the existing public datasets, specifically 27 working conditions such as dynamic traffic flow (peak / off - peak), extreme weather (rain / snow), construction disturbances (excavation / laying), and different geological conditions (soft soil / hard soil). Combining with the requirements of road health management, for each two - dimensional spatio - temporal distributed vibration signal, we have six different detection tasks, namely fault detection, reliability analysis, anomaly warning, fault diagnosis, maintenance suggestion, and life prediction. We use Deepseek - V3 to establish a signal - text response dataset. Each two - dimensional distributed vibration signal in the DVSAG dataset is paired with 6 user prompts and responses, with a total of 266 * 6 = 1596 text responses. Based on the text - enhanced signals, it supports the training and development of the LLM model, truly simulating the service environment of the existing road subgrade."

[0010] (2) Cross - diffusion normalization of DVS vibration signals:

[0011] The inconsistent dimensions of distributed vibration signals make it difficult to directly fuse information. Simple signal truncation or interpolation will lead to signal distortion. Therefore, we first construct spatio - temporal similarity matrices driven by dynamic multi - scale Gaussian kernels respectively to capture the local features in the time and space dimensions of the signals. For the time axis T, we use a locally adaptive dynamic bandwidth Gaussian kernel function to construct the time similarity matrix W time The elements in are defined as:

[0012]

[0013] where represents all spatial - position signals at the i - th time point. Similarly represents all spatial - position signals at the j - th time point, and there are T time points in total. σ time is the Gaussian kernel bandwidth parameter, controlling the similarity decay rate on the time axis.

[0014] Similarly, for the space axis D, we use a distance - weighted Gaussian kernel function to construct the space similarity matrix W space The elements in are defined as:

[0015]

[0016] where represents all time signals at the i - th spatial point. Similarly represents all time signals at the j - th spatial point, and there are D spatial points in total. σ space is the Gaussian kernel bandwidth parameter under distance - weighting, controlling the similarity decay rate on the space axis through distance proximity.

[0017] Using the time-axis similarity matrix alone is likely to overlook the physical space coupling topological constraints of adjacent sensors, while using the space-axis similarity matrix alone cannot reflect the dynamic time-delay diffusion propagation characteristics of distributed vibration signals. The risks of information islands and noise amplification lead to incomplete global signal features. Therefore, cross-diffusion is used next to fuse the time-axis and space-axis information to generate a robust consensus matrix and explore the spatio-temporal correlation of distributed vibration signals.

[0018] First, define the time similarity matrix W time and the transfer probability matrix P space of the space similarity matrix W time and P space , whose essence is the transfer probability of a Markov chain:

[0019]

[0020] Next, use the transfer probability matrix for alternating diffusion to achieve two-way enhancement of time and space information. The time matrix absorbs the space topology information, and the space matrix fuses the time-series evolution law. The state matrices at the t-th iteration are P time,t+1 and P space,t+1 as follows:

[0021]

[0022] When the convergence condition is reached, the consensus matrix P consensus is generated from the mean of the spatio-temporal diffusion matrices:

[0023]

[0024] where c is the number of iterations of the diffusion process.

[0025] Through the alternating diffusion process, the similarity matrices of the time axis and the space axis can enhance each other, effectively smoothing the noise and improving the robustness of the signal. The generated consensus matrix can be used in the subsequent standardization process to ensure that the time and space correlations of the signal are retained.

[0026] Finally, according to the consensus matrix, the signals with different sampling rates are aligned to a unified dimensional space, and the physical correlation of the vibration waveform is retained. The time axis and the space axis are standardized and aligned using the state matrix:

[0027]

[0028] is the signal at all spatial positions at t time points of the processed signal, is all the time signals at the d-th spatial point of the processed signal, and S v (t, d, :) is the signal value at the t-th time point and the d-th spatial position of the processed signal, and Sr (j, k, :) represents the signal value of the original signal at the j-th time point and the k-th spatial position.

[0029] (3) Multi-band input alignment:

[0030] Since the number of frequency components contained in the standardized two-dimensional signal is inconsistent, it is not suitable as the model input. We designed a two-dimensional discretization method 2D-DCT that can convert the distributed vibration signal into the frequency domain. By using the discrete cosine transform 2D-DCT, the vibration signal is converted into the frequency domain signal F v .

[0031] F v = 2D-DCT(S v )

[0032] Then, the frequency domain signal F v is subjected to discrete wavelet transform DWT to complete the frequency band decomposition. The first n f sub-bands are selected by arranging the band energies in descending order. The frequency components are unified into n f kinds by padding or cropping, and is used to normalize the band amplitudes. The normalized frequency is denoted as E v , and the specific calculation steps are as follows:

[0033]

[0034] Among them, the signals with a sampling rate s lower than n f are zero-padded, while the signals exceeding n f are truncated. To balance the computing resources and ensure the accuracy of fault classification, the amplitudes of the frequency sequence are normalized to [-1, 1]:

[0035]

[0036] Among them, β is a scaling factor, and its value is designed to be 0.02 according to experience.

[0037] (4) Residual signal enhancement:

[0038] At the same time, in order to eliminate the problems of distribution differences under different working conditions and the submergence of abnormal features, a fault-free reference signal is collected under the working conditions of normal service of the complex structure The collection conditions are:

[0039]

[0040] Indicates no fault, ID * = ID normal working condition, is the "five-element data chain" corresponding to the fault-free signal.

[0041] At the same time, the residual signal is calculated using the fault-free reference signal Quickly capture the abnormal components in the signal, enhance the sensitivity of the model to fault features, and finally the processed signal E v and the fault-free reference signal Residual signal E res are spliced together to construct a unified representation W of the distributed vibration signal,

[0042]

[0043] The unified representation of the signal can synthesize multi-dimensional information, eliminate distribution differences, lay a foundation for subsequent feature extraction, semantic mapping, and fault coupling analysis, and enhance the generalization ability and fault diagnosis accuracy of the model.

[0044] (5) Feature extraction of DVS vibration signal:

[0045] To extract the features of the two-dimensional distributed vibration signal, we designed a fault diagnosis network FCN (Fault Diagnosis Network). This network uses three independent two-dimensional convolutional kernels without shared weights to extract multi-scale features from the unified vibration signal W. Then, we fuse the multi-scale features through three CBAM (Convolutional Block Attention Module) dual attention modules, and then use two linear layers for fault classification.

[0046] 1) Multi-scale convolutional layer:

[0047] The two-dimensional distributed vibration signal usually contains various scale features such as time-frequency, spatial domain, and non-linearity. First, three independent two-dimensional convolutional kernels (without shared weights) are designed to extract initial features from the unified vibration signal . F1 mainly captures the spatial propagation attenuation characteristics (such as the exponential attenuation of vibration waves with the fiber optic distance), F2 extracts the temporal periodic characteristics (such as the pulse periodicity of vehicle loads), and F3 models the non-linear coupling effect (such as the chaotic vibration caused by geological activities), providing information for subsequent feature fusion and fault classification.

[0048] F1 = Conv2D(W, k1), F2 = Conv2D(W, k2), F3 = Conv2D(W, k3)

[0049] Conv2D represents the two-dimensional convolution operation, and k1 = k2 = k3 = (5, 5) represents the convolutional kernel size, which are the initially extracted multi-scale features, and the output convolutional kernel channel number is 3.

[0050] 2) CBAM attention mechanism:

[0051] There may be redundant information in the multi-scale features, and the features of different scales contribute differently to fault diagnosis. Through the CBAM attention mechanism, using the dual attention mechanisms of channels and space, the key fault information in the multi-scale features can be adaptively enhanced, and the interference of redundant features can be reduced.

[0052] For the preliminarily extracted multi-scale features F i (i = 1, 2, 3), the CBAM module is applied respectively, introducing the channel attention mechanism under vibration physical constraints and the spatial attention mechanism guided by the fault heat map:

[0053] F′ i = CBAM(F i )

[0054] In the multi-scale features, different channels may contain information of different importance. First, global average pooling AvgPool and global max pooling MaxPool are performed on the multi-scale features, and the pooling results are input into a shared multi-layer perceptron (MLP) to dynamically adjust the weight M of each channel c , enhancing the feature representation of key channels and suppressing unimportant channel features.

[0055]

[0056] MaxPool(F i ) = max F i (t, d, :)

[0057] M c (F i ) = σ(MLP(AvgPool(F i )) + MLP(MaxPool(F i )))

[0058] where σ is the Sigmoid function, used to normalize the weight to the range of [0, 1].

[0059] At the same time, in the multi-scale features, different spatial positions may contain information of different importance. Based on the output of the channel attention mechanism, average pooling ChannelAvgPool and global max pooling operations ChannelMaxPool are also performed on it, and the results of channel pooling are concatenated into M s , and the spatial attention weight is generated through a 5×5 convolutional layer.

[0060]

[0061] ChannelMaxPool(M c (Fi )) = maxM c (F i )

[0062] M s (F i ) = σ(f 5×5 (ChannelPool(F i )))

[0063] where f 5×5 is a 5×5 convolutional layer and σ is the Sigmoid function.

[0064] Finally, the channel attention weight, the spatial attention weight, and the feature F i are gradually multiplied to obtain the finally fused and enhanced feature F′ i :

[0065]

[0066] denotes element-wise multiplication, and the output is the feature after 3-channel fusion. F′ i is a two-dimensional feature matrix.

[0067] 3) Linear layer fault classification:

[0068] The feature F′ enhanced by CBAM i is still a two-dimensional feature map (T′×D′×C), which needs to be concatenated into a feature vector with a fixed length. Through GlobalAvgPool global average pooling, the two-dimensional feature map is compressed into a global feature vector F global :

[0069]

[0070] F i,global = GlobalAvgPool(F′ i )

[0071] F global = Contact(F 1,global , F 2,global , F 3,global )

[0072] Then, F global is input into the fully connected layer. Through two fully connected layers, a linear transformation is performed on the input features to map them to the fault category space and learn the combination relationship between the features.

[0073] H = ReLU(W1·F global + b1)

[0074] Z = W2·H + b2

[0075] Among them, W1 and W2 are weight matrices, h is the hidden layer dimension, b1 and b2 are bias terms, ReLU is the activation function, H is the hidden layer feature, and Z is the logits of the fault category.

[0076] Next, the logits Z need to be converted into a probability distribution for outputting the fault type, and it is normalized using the Softmax function:

[0077] P = softmax(Z)

[0078] The output P is the fault type, and the size of P is [1, γ], where γ represents the number of fault types.

[0079] Define the cross-entropy loss function during the entire model training process to measure the predicted fault result P i and the true fault label L r The difference between them:

[0080]

[0081] Among them, γ represents the number of fault types.

[0082] (6) DVS vibration signal semantic feature alignment and mapping:

[0083] To achieve the multi-task mapping of distributed vibration signal features to natural language for health monitoring, a semantic alignment framework is proposed. After fault feature extraction, the fault features are semanticized e fault , and the user text instruction token embedding is combined with the LLM e text to generate a multi-task natural language response, and double semantic mapping of the fault features and user prompts is performed.

[0084] The fault feature F is mapped to the LLM embedding space through a learnable projection matrix global e

[0085] = LayerNorm(W2·ReLU(DWPT(W1·F fault )) global ))

[0086] Among them aligned with the token dimension of the LLM, and the consistency between the features and text semantics is optimized through the contrastive learning loss :

[0087]

[0088] Among them, sim is the cosine similarity function, τ is the temperature coefficient, and B is the batch size.

[0089] Simultaneously, for the user's question S t perform dynamic word segmentation and embedding:

[0090] e text = Tokenizer(S t )·E LLM + PE(S t )

[0091] where Tokenizer is the word segmenter, E LLM is the word embedding matrix, and PE is the positional encoding, which retains the temporal information of the context statements.

[0092] Next, introduce the Gated Cross-Attention (GCA) mechanism to achieve fine-grained interaction between features and text.

[0093] Q = w qefault , K = w k e text , V = w v e text

[0094]

[0095] H fusion = α·Attention(Q, K, V)+(1 - α)e fault

[0096] where w q , w k , w v are the projection matrices for query, key, and value, W g is the gated weight matrix, α is the gated coefficient, and H fusion is the fused semantic feature, which enhances the robustness of the model by dynamically adjusting the information fusion ratio.

[0097] (7) DVSLLM model fine-tuning:

[0098] We used the pre-trained Qwen2-1.5B as the parameterized LLM to achieve basic human-computer interaction. Then, for the improvement of domain knowledge, we fine-tuned the LLM and the alignment layer using adaptive low-rank projection:

[0099]

[0100] where ΔW is the parameter increment update matrix, r is the rank of the low-rank projection, B i , A i are the low-rank matrix pairs, and f θA lightweight network based on a low-rank matrix dynamically generated from fused features, which only needs to train 0.03% of the parameters to adapt to multi-task requirements. In summary, a complete mapping chain from distributed vibration signals to natural semantics is established, providing decision support for multi-health management tasks under complex working conditions.

[0101] In summary, the advantages of the present invention over the prior art are as follows:

[0102] (1) For the first time, a distributed vibration signal text enhancement dataset (DVSAG) covering a variety of working conditions is constructed, including 266 groups of signal-text pairing data, providing data support for multi-task training of semantic understanding of distributed vibration signals. Aiming at the problem of inconsistent signal sampling rates under different working conditions, a cross-diffusion normalization method is proposed. A consensus matrix is generated through the alternating diffusion of the spatio-temporal similarity matrix. A multi-band input alignment mechanism is designed, combining 2D-DCT transform and discrete wavelet transform DWT to align the signal dimensions while ensuring the spatio-temporal correlation of the signal. A residual signal enhancement strategy is introduced, and the working condition difference is eliminated by using the fault-free reference signal, significantly improving the sensitivity of abnormal features.

[0103] (2) Use the fault-free reference signal and the residual signal to construct a unified representation of the distributed vibration signal. Then, a fault diagnosis network FCN is designed. The network uses three independent two-dimensional convolutional kernels without shared weights to extract multi-scale features from the unified vibration signal. Then, three CBAM spatio-temporal dual attention modules are used to fuse the multi-scale features, and two linear layers are used to classify the DVS signals for faults. A semantic alignment framework based on gated cross-attention is designed. The vibration signal features and user text questions are mapped to the LLM embedding space through a learnable projection matrix. An adaptive low-rank projection fine-tuning technology is adopted to quickly adapt to six types of health management tasks: fault detection, reliability analysis, anomaly warning, fault diagnosis, maintenance advice, and life prediction, realizing fine-grained interaction from DVS vibration signals to natural language questions and answers.

[0104] (3) Comprehensive experimental analysis shows that the method for enhancing the semantic representation of DVS vibration signals based on prior knowledge proposed in the present invention can effectively support multi-task health management of fault detection, reliability analysis, anomaly warning, fault diagnosis, maintenance advice, and life prediction, improving the reliability and stability of continuous detection and analysis of vibration events in actual roadbed monitoring. Description of the Drawings

[0105] To better understand the present invention, the following further description is made in conjunction with the drawings.

[0106] Figure 1 It is a flow chart of the steps for establishing a method for enhancing the semantic representation of DVS vibration signals based on prior knowledge;

[0107] Figure 2It is the algorithm flow chart implemented by the present invention;

[0108] Figure 3 It is the design diagram of the DVSLLM model structure in the implementation of the present invention;

[0109] Figures 4(a)-(d) are the experimental result diagrams in the implementation of the present invention, including the comparison of model accuracy with existing methods, the effect diagram of discriminant fault classification of the confusion matrix, the analysis result diagram of the ablation experiment, and the schematic diagram of DVSLLM interactive dialogue. Specific implementation manners

[0110] The present invention will be further described in detail below through implementation cases.

[0111] (1) Construct a distributed vibration signal text enhanced dataset:

[0112] First, collect roadbed vibration data under 27 different working conditions. The scenarios include different traffic flows, weather conditions, road construction activities, and geological conditions, etc. In terms of traffic flow, working day morning rush hour, evening rush hour, flat peak period, and different periods on weekends are selected to simulate the influence of different traffic volumes on roadbed vibration; weather conditions cover sunny days, rainy days, snowy days, etc., considering the effect of roadbed humidity and temperature changes on vibration signals under different weathers; road construction activities include nearby manual operations, road excavation, paving operations, etc.; geological conditions involve different geological conditions such as soft soil foundation and hard soil foundation. A total of 266 groups of different distributed vibration signals are collected, with the sampling rate varying from 500HZ to 4KHZ and the sampling time varying from 5min to 240min. The types of faults to be identified cover 6 categories and 18 types such as roadbed settlement, crack expansion, uneven deformation, pipeline leakage influence, and vibration anomalies caused by surrounding geological activities. At the same time, in order to complete 6 health management tasks of preset fault detection, reliability analysis, anomaly warning, fault diagnosis, maintenance suggestions, and life prediction, for the 266 groups of vibration data, use the Deepseek-V3 model to generate text responses for each group of data, that is, each group is a distributed vibration signal S r - Fault label L r - Working status ID - User prompt S t - Text response R t , a total of 266 * 6 = 1596 text-enhanced DVS vibration response signals are paired.

[0113] (2) Cross-diffusion standardize DVS vibration signals:

[0114] After completing the construction of the distributed vibration signal text enhanced dataset, due to the problem of inconsistent dimensions in the 266 groups of DVS vibration signals collected, simple signal truncation or interpolation will cause signal distortion. Therefore, we first construct spatio-temporal similarity matrices driven by dynamic multi-scale Gaussian kernels respectively to capture local features in the time and space dimensions of the signals. For the time axis T = 500, we use a locally adaptive dynamic bandwidth Gaussian kernel function to construct the time similarity matrix W time The elements in are defined as:

[0115]

[0116] where (D = 100, A is the corresponding amplitude) represents the signals at all spatial positions at the i-th time point. Similarly represents the signals at all spatial positions at the j-th time point. There are T time points in total. σ time = 0.5 is the Gaussian kernel bandwidth parameter, which controls the similarity decay rate on the time axis.

[0117] For the spatial axis D = 100, we use a distance-weighted Gaussian kernel function to construct the spatial similarity matrix W space The elements in are defined as:

[0118]

[0119] where (T = 500, A is the corresponding amplitude) represents the signals at all time points at the i-th spatial point. Similarly represents the signals at all time points at the j-th spatial point. There are D spatial points in total. σ space = 0.03 is the Gaussian kernel bandwidth parameter under distance weighting, which controls the similarity decay rate on the spatial axis.

[0120] Using only the time axis similarity matrix is likely to ignore the physical space coupling topology constraints of adjacent sensors, while using only the spatial axis similarity matrix cannot reflect the dynamic time-delay diffusion propagation characteristics of distributed vibration signals. The risks of information islands and noise amplification lead to incomplete global signal features. Therefore, next, we use cross-diffusion to fuse the time axis and spatial axis information to generate a robust consensus matrix and explore the spatio-temporal correlation of distributed vibration signals.

[0121] First, we define the transition probability matrices P time of the time similarity matrix W space and the spatial similarity matrix W time and P space :

[0122]

[0123] Then, alternating diffusion is carried out using the transition probability matrix to achieve two-way enhancement of time and space information. The time matrix absorbs the spatial topological information, and the space matrix integrates the temporal evolution law. The state matrix at the t-th iteration is P time,t+1 and P space,t+1 is:

[0124]

[0125] When the convergence condition is reached, the consensus matrix P is generated from the mean of the spatio-temporal diffusion matrix consensus :

[0126]

[0127] where c = 50 is the maximum number of iterations of the diffusion process

[0128] Through the alternating diffusion process, the similarity matrices of the time axis and the space axis can enhance each other, effectively smooth the noise, and improve the robustness of the signal. The generated consensus matrix can be used for the subsequent standardization process to ensure that the temporal and spatial correlations of the signal are retained

[0129] Finally, according to the consensus matrix, the signals with different sampling rates are aligned to the same dimension, and the correlation and feature distribution of the signal are retained. The time axis and the space axis are standardized and aligned using the state matrix:

[0130]

[0131] (T′ = 300, A is the corresponding amplitude) is the signal at all spatial positions at t time points of the processed signal (D′ = 60, A is the corresponding amplitude) is all the time signals at the d-th spatial point of the processed signal, S v (t, d, :) is the signal value at the t-th time point and the d-th spatial position of the processed signal, S r (j, k, :) is the signal value at the j-th time point and the k-th spatial position of the original signal

[0132] (3) Multi-band input alignment:

[0133] After the cross-diffusion standardization of the initial vibration data of DVS, the number of frequency components contained in the 266 groups of two-dimensional signals after standardization is inconsistent and cannot be directly used as the model input. Then, the multi-band input alignment operation is carried out. The signal is transformed into the frequency domain using the two-dimensional discrete cosine transform 2D-DCT, and then the discrete wavelet transform is performed on the frequency domain signal to complete the frequency decomposition. The appropriate frequency range is intercepted, and finally the band amplitude is normalized. The specific method is as follows:

[0134] Use the discrete cosine transform 2D-DCT to transform 266 groups of vibration signals S v (300×60) into the frequency domain signal E v .

[0135] F v = 2D-DCT(S v )

[0136] Then perform wavelet transform on F v to complete frequency decomposition. Select the first n f sub-bands by arranging the band energies in descending order, and unify the frequency components into n f = 24000 (hyperparameter training empirical value) by padding or cropping. And use to normalize the band amplitudes. The normalized frequency is denoted as E v . The specific calculation steps are as follows:

[0137]

[0138] Among them, the signals with a sampling rate s lower than n f are zero-padded, while the signals exceeding n f are truncated. To balance the computing resources and the accuracy of fault classification, and at the same time to enhance the stability of training, the amplitudes of the frequency sequences are normalized to [-1, 1].

[0139]

[0140] Among them, β is a scaling factor, and its value is designed to be 0.02 according to experience.

[0141] (4) Residual signal enhancement:

[0142] At the same time, in order to eliminate the problems of distribution differences under different working conditions and the submergence of abnormal features, collect fault-free reference signals under the working conditions of normal service of complex structures The collection conditions are:

[0143]

[0144] Indicates no fault, ID * = ID normal working condition. A total of 300 groups of fault-free reference signals are collected in the actual environment.

[0145] At the same time, use the fault-free reference signals to calculate the residual signals to quickly capture the abnormal components in the signals, enhance the sensitivity of the model to fault features. Finally, the processed signal E v , the fault-free reference signal and the residual signal F resStitch them together to construct a unified distributed vibration signal unified representation W.

[0146]

[0147] The dimension of the unified W is [24000, 3], laying a foundation for subsequent feature extraction, semantic mapping, and interactive question-and-answer analysis.

[0148] (5) Feature extraction of DVS vibration signals:

[0149] After completing the preprocessing of DVS vibration signals in the previous steps, a fault diagnosis network FCN is designed to extract features from the unified vibration signal W. This network uses three independent two-dimensional convolutional kernels without shared weights to extract multi-scale features from the unified vibration signal W. Then, we fuse the multi-scale features through three CBAM spatio-temporal dual attention modules, and then use two linear layers for fault classification.

[0150] 1) Multi-scale convolutional layer:

[0151] Two-dimensional distributed vibration signals usually contain various scale features such as time-frequency, spatial domain, and non-linearity. First, three independent two-dimensional convolutional kernels (without shared weights) are designed to extract initial features from the unified vibration signal Among them, F1 mainly captures the spatial propagation attenuation characteristics (such as the exponential attenuation of vibration waves with the fiber optic distance), F2 extracts the temporal periodicity characteristics (such as the pulse periodicity of vehicle loads), and F3 models the non-linear coupling effect (such as the chaotic vibration caused by geological activities), providing information for subsequent feature fusion and fault classification.

[0152] F1 = Conv2D(W, k1), F2 = Conv2D(W, k2), F3 = Conv2D(W, k3)

[0153] Conv2D represents the two-dimensional convolution operation, and k1 = k2 = k3 = (5, 5) represents the convolutional kernel size. They are the initially extracted multi-scale features, and the output convolutional kernel channels are 3.

[0154] 2) CBAM attention mechanism:

[0155] There may be redundant information in the multi-scale features, and the contributions of features at different scales to fault diagnosis are different. Through the CBAM attention mechanism, using the channel and spatial dual attention mechanisms, the key fault information in the multi-scale features can be adaptively enhanced, and the interference of redundant features can be reduced.

[0156] For the initially extracted multi-scale feature F i(i = 1, 2, 3), apply the CBAM module respectively, introducing the channel attention mechanism under vibration physical constraints and the spatial attention mechanism guided by the fault heat map:

[0157] F′ i = CBAM(F i )

[0158] In the multi-scale features, different channels may contain information of different importance. First, perform AvgPool global average pooling and MaxPool global maximum pooling operations on the multi-scale features, and input the pooling results into a shared multi-layer perceptron (MLP) to dynamically adjust the weight M of each channel c , enhancing the feature representation of key channels and suppressing unimportant channels.

[0159]

[0160] MaxPool(F i ) = maxF i (t, d, :)

[0161] M c (F i ) = σ(MLP(AvgPool(F i )) + MLP(MaxPool(F i )))

[0162] where σ is the Sigmoid function, used to normalize the weights to the range of [0, 1].

[0163] Meanwhile, in the multi-scale features, different spatial positions may contain information of different importance. Based on the output of the channel attention mechanism, perform ChannelAvgPool average pooling and ChannelMexPool global maximum pooling operations on it, and concatenate the results of channel pooling into M s , generating spatial attention weights through a 5×5 convolutional layer:

[0164]

[0165] ChannelMaxPool(M c (F i )) = max M c (F i )

[0166] M s (F i ) = σ(f 5×5 (ChannelPool(F i )))

[0167] where f 5×5 is a 5×5 convolutional layer, and σ is the Sigmoid function.

[0168] Finally, multiply the channel attention weight, the spatial attention weight, and the feature F i gradually to obtain the finally fused and enhanced feature F′ i :

[0169]

[0170] denotes element-wise multiplication, and the output 3-channel fused feature, F′ i is a two-dimensional feature matrix.

[0171] 3) Linear layer fault classification:

[0172] The feature F′ enhanced by CBAM i is still a two-dimensional feature map (24000×3×3), and it needs to be converted into a fixed-length feature vector. Through global average pooling, the two-dimensional feature map is compressed into a global feature vector F global :

[0173]

[0174] F i,global = GlobalAvgPool(F′ i )

[0175] F global = Contact(F 1,global ,F 2,global ,F 3,global )

[0176] Then input F global into the fully connected layer. Through two fully connected layers, perform a linear transformation on the input features, map them to the fault category space, and learn the combination relationship between the features,

[0177] H = ReLU(W1·F global +b1)

[0178] Z = W2·H + b2

[0179] where W1, W2 are weight matrices, h is the hidden layer dimension, b1, b2 are bias terms, ReLU is the activation function, H is the hidden layer feature, and Z is the logits of the fault category.

[0180] Then convert the logits Z into a probability distribution for outputting the fault type, and normalize it using the Softmax function:

[0181] P = softmax(Z)

[0182] The output P is the fault type, and the size of P is [1, γ], where γ = 18 represents the number of fault types.

[0183] During the entire model training process, the cross-entropy loss function L is defined to measure the fault result P i and the true fault label L r The difference between them is:

[0184]

[0185] where γ represents the number of fault types.

[0186] (6) DVS Vibration Signal Semantic Feature Alignment and Mapping:

[0187] After the feature extraction of the DVS vibration signal is completed, in order to achieve the multi-task mapping from vibration signal features to natural language for health monitoring, semantic feature alignment and mapping are required. A new semantic alignment framework is proposed, and the fault features are semantified after the fault feature extraction e fault , and the LLM is used to combine the user text instruction token embedding e text to generate a multi-task natural language response, and double semantic mapping of the fault features and user prompts is performed.

[0188] The fault feature F is mapped to the LLM embedding space through a learnable projection matrix global e

[0189] = LayerNorm(W2·ReLU(DWPT(W1·F fault )) global )

[0190] where it is aligned with the token dimension of the LLM, and the consistency between the features and text semantics is optimized through the contrastive learning loss :

[0191]

[0192] where sim is the cosine similarity function, τ is the temperature coefficient, and B is the batch size. According to the training empirical value, B = 32 is set.

[0193] At the same time, the user question S t is dynamically tokenized and embedded:

[0194] e text = Tokenizer(S t )·E LLM + PE(S t )

[0195] The tokenizer converts the input text into corresponding word vectors, and at the same time combines the positional encoding PE to retain the temporal information of the context sentences.

[0196] Then, the Gated Cross-Attention (GCA) mechanism is introduced to achieve fine-grained interaction between features and text.

[0197] Q = w q e fault ,K = w k e text ,V = w v e text

[0198]

[0199] H fusion = α·Attention(Q, K, V)+(1 - α)e fault

[0200] where w q 、w k 、w v are the projection matrices for queries, keys, and values, W g is the gated weight matrix, α is the gating coefficient, and H fusion is the fused semantic feature, which enhances the robustness of the model by dynamically adjusting the information fusion ratio.

[0201] (7) DVSLLM model fine-tuning:

[0202] We used the pre-trained Qwen2-1.5B as the parameterized LLM to achieve basic human-computer interaction. Then, for the improvement of domain knowledge, we fine-tuned the LLM and the alignment layer using adaptive low-rank projection:

[0203]

[0204] f θ is a lightweight network for dynamically generating low-rank matrices based on fused features, and only 0.03% of the parameters need to be trained to adapt to multi-task requirements. In summary, a complete mapping chain from distributed vibration signals to natural semantics is established, providing decision support for multi-health management tasks under complex working conditions.

[0205] To verify the effectiveness of the present invention, we conducted six different health monitoring task test experiments and compared them with other existing fault diagnosis and classification algorithms. Specifically, the DVSAG data was divided into a training set and a test set in a ratio of 7:3, and the accuracy rates on the test sets of different methods were compared. At the same time, the confusion matrix was used to judge the accuracy of fault classification, and ablation experiments were conducted on the key text enhancement module, attention mechanism feature fusion module, and signal-text semantic alignment module to prove their contribution degrees. Finally, the final effect of the model was demonstrated through interaction, and through the data visualization analysis in Figures 4(a)-(d), the effectiveness of the present invention was verified, indicating that the fine-tuned DVSLLM model in the present invention method has superiority in processing roadbed health monitoring, providing new possibilities for establishing an intelligent distributed optical fiber vibration signal semantic representation method, and being more suitable for popularization and use in practice.

Claims

1. A semantic representation method for DVS vibration signals enhanced by prior knowledge, characterized in that: (1)Construct a distributed vibration signal text-enhanced dataset; (2)Cross-diffusion standardize DVS vibration signals; (3)Multi-band input alignment; (4)Residual signal enhancement; (5)DVS vibration signal feature extraction; (6)DVS vibration signal semantic feature alignment and mapping; (7)DVSLLM model fine-tuning; specifically including the following seven steps: Step 1: Construct a distributed vibration signal text-enhanced dataset; Aiming at the problems of strong single-condition dependence in the existing roadbed monitoring dataset, signal-text semantic fragmentation, and lack of paired text descriptions to support LLM model training, a prior knowledge semantic enhancement framework is proposed. Through domain knowledge-driven text generation and multi-dimensional condition coverage, the first text-enhanced dataset (DVSAG) that supports multi-task semantic understanding of distributed vibration signals is constructed. The DVSAG dataset contains 266 pairs of distributed vibration signals and their fault types, that is, each group is "distributed vibration signal S r - fault label L r - working status ID - user prompt S t - text response R t " five - element data chain, which includes all working conditions in the existing public datasets. Specifically, there are 27 working conditions such as dynamic traffic flow (peak / off - peak), extreme weather (rain / snow), construction disturbances (excavation / laying), and different geological conditions (soft soil / hard soil), etc.; combined with the requirements of road health management, for each spatio - temporal two - dimensional distributed vibration signal, we have six different detection tasks, namely fault detection, reliability analysis, anomaly warning, fault diagnosis, maintenance suggestions, and life prediction. We use Deepseek - V3 to establish a signal text response dataset. Each two - dimensional distributed vibration signal in the DVSAG dataset is equipped with 6 user prompts and responses, with a total of 266 * 6 = 1596 text responses. Based on the text - enhanced signals, it supports the training and development of the LLM model, and truly simulates the service environment of the existing road subgrade; Step 2: Cross-diffusion standardize DVS vibration signals; The inconsistent dimensions of distributed vibration signals make it difficult to directly fuse information. Simple signal truncation or interpolation will lead to signal distortion. Therefore, we first construct spatio-temporal similarity matrices driven by dynamic multi-scale Gaussian kernels respectively to capture local features in the time and space dimensions of the signals. For the time axis T, a time similarity matrix is constructed using a locally adaptive dynamic bandwidth Gaussian kernel function W time The elements in are defined as: Among them represents all spatial position signals at the $i$-th time point. Similarly represents all spatial position signals at the $j$-th time point, and there are $T$ time points in total; $\sigma$ time is the Gaussian kernel bandwidth parameter, which controls the decay rate of similarity on the time axis; Similarly, for the spatial axis D, a spatial similarity matrix is constructed using a distance-weighted Gaussian kernel function W space The elements in are defined as: Among them represents all time signals at the i-th spatial point. Similarly represents all time signals at the j-th spatial point. There are D spatial points in total; σ space is the Gaussian kernel bandwidth parameter under distance weighting, which controls the similarity decay rate on the spatial axis through proximity in distance; Using only the time-axis similarity matrix is likely to ignore the physical space coupling topological constraints of adjacent sensors, while using only the space-axis similarity matrix cannot reflect the dynamic time-delay diffusion propagation characteristics of distributed vibration signals. The risk of information islands and noise amplification leads to incomplete global signal features. Therefore, cross-diffusion is used next to fuse time-axis and space-axis information to generate a robust consensus matrix and explore the spatio-temporal correlation of distributed vibration signals. First, define the time similarity matrix W time and the spatial similarity matrix W space of the transition probability matrix P time and P space , whose essence is the transition probability of a Markov chain: Then, alternating diffusion is performed using the transition probability matrix to achieve bidirectional enhancement of temporal and spatial information. The temporal matrix absorbs spatial topological information, and the spatial matrix integrates the temporal evolution law. The state matrix at the t-th iteration is P time,t+1 and P space,t+1 is as follows: When the convergence condition is reached, the consensus matrix P is generated from the mean of the spatio-temporal diffusion matrix consensus : Among them, c is the number of iterations of the diffusion process; Through the alternating diffusion process, the time-axis and space-axis similarity matrices can enhance each other, effectively smooth the noise, and improve the robustness of the signal; the generated consensus matrix can be used for the subsequent standardization process to ensure that the time and space correlations of the signal are retained. Finally, according to the consensus matrix, signals with different sampling rates are aligned to a unified dimensional space, and the physical correlation of the vibration waveform is retained. The time-axis and space-axis are standardized and aligned using the state matrix: It is the signals at all spatial positions of the processed signal at t time points, It is all the time signals of the processed signal at the d-th spatial point, S v (t, d, :) is the signal value of the processed signal at the t-th time point and the d-th spatial position, S r (j, k, :) is the signal value of the original signal at the j-th time point and the k-th spatial position; Step 3: Multi-band input alignment; Since the number of frequency components contained in the standardized two-dimensional signal is inconsistent and it is not suitable as the model input, we designed a two-dimensional discretization method 2D-DCT that can convert the distributed vibration signal into the frequency domain. By using the discrete cosine transform 2D-DCT, the vibration signal is converted into a frequency domain signal F v : F v = 2D-DCT(S v ); Next, perform a discrete wavelet transform (DWT) on the frequency-domain signal F v to complete the frequency-band decomposition. Select the first n f sub-bands by arranging the frequency-band energies in descending order, and unify the frequency components to n f types by padding or cropping. Then use to normalize the frequency-band amplitudes. The normalized frequencies are denoted as E v , and the specific calculation steps are as follows: Among them, signals with a sampling rate s lower than n f are zero-padded, while signals exceeding n f are truncated. To balance computing resources and ensure the accuracy of fault classification, the amplitudes of the frequency sequences are normalized to [-1, 1]: Among them, β is a scaling factor, and its value is designed to be 0.02 according to experience; Step 4: Residual signal enhancement; Meanwhile, to eliminate the problems of distribution differences and abnormal features being submerged under different working conditions, fault-free reference signals are collected under the working conditions of normal service of complex structures. The collection conditions are as follows: Indicates no fault, ID * = ID normal operating condition, is the "five - element data link" corresponding to the no - fault signal; At the same time, the residual signal is calculated using the fault-free reference signal Quickly capture the abnormal components in the signal, enhance the sensitivity of the model to fault features, and finally process the signal E v , the fault-free reference signal Residual signal E res Stitch them together to construct a unified distributed vibration signal unified representation W: The unified signal representation W can integrate multi-dimensional information, eliminate distribution differences, lay a foundation for subsequent feature extraction, semantic mapping, and fault coupling analysis, and enhance the generalization ability and fault diagnosis accuracy of the model. Step 5: DVS vibration signal feature extraction; To extract the features of two-dimensional distributed vibration signals, we designed a fault diagnosis network FCN (Fault Diagnosis Network). This network uses three independent two-dimensional convolutional kernels without shared weights to extract multi-scale features from the unified vibration signal W. Then, we fuse the multi-scale features through three CBAM (Convolutional Block Attention Module) dual attention modules, and then use two linear layers for fault classification. 1) Multi-scale convolutional layer: Two-dimensional distributed vibration signals usually contain various scale features such as time-frequency, spatial domain, and nonlinearity. First, three independent two-dimensional convolutional kernels (without shared weights) are designed to extract initial features from the unified vibration signal Among them, F1 mainly captures the spatial propagation attenuation characteristics (such as the exponential attenuation of vibration waves with the fiber optic distance), F2 extracts the temporal periodicity characteristics (such as the pulse periodicity of vehicle loads), and F3 models the nonlinear coupling effect (such as the chaotic vibration caused by geological activities), providing information for subsequent feature fusion and fault classification; F1 = Conv2D(W, k1), F2 = Conv2D(W, k2), F3 = Conv2D(W, k3); Conv2D represents a two-dimensional convolution operation, where k1 = k2 = k3 = (5, 5) represents the convolution kernel size, which are the preliminarily extracted multi-scale features, and the number of output convolution kernel channels is 3; 2) CBAM attention mechanism: There may be redundant information in multi-scale features, and features of different scales contribute differently to fault diagnosis. Through the CBAM attention mechanism, using dual channel and spatial attention mechanisms, the key fault information in multi-scale features can be adaptively enhanced, and the interference of redundant features can be reduced; For the initially extracted multi-scale features F i (i = 1, 2, 3), the CBAM module is applied respectively to introduce the channel attention mechanism under vibration physical constraints and the spatial attention mechanism guided by the fault heat map: F’ i = CBAM(F i ); In multi-scale features, different channels may contain information of different importance. First, perform AvgPool global average pooling and MaxPool global maximum pooling operations on the multi-scale features, and input the pooling results into a shared multi-layer perceptron (MLP) to dynamically adjust the weight M of each channel c , enhancing the feature representation of key channels and suppressing unimportant channel features; MaxPool(F i ) = maxF i (t, d, :); M c (F i ) = σ(MLP(AvgPool(F i )) + MLP(MaxPool(F i ))); where σ is the Sigmoid function, used to normalize the weights to the range [0, 1]; Meanwhile, in multi-scale features, different spatial positions may contain information of different importance. Based on the output of the channel attention mechanism, ChannelAvgPool average pooling and ChannelMaxPool global maximum pooling operations are also performed on it, and the results of channel pooling are concatenated into M s , and spatial attention weights are generated through a 5×5 convolutional layer; ChannelMaxPool(M c (F i )) = maxM c (F i ); M s (F i ) = σ(f 5×5 (ChannelPool(F i ))); where f 5×5 is a 5×5 convolutional layer, and σ is the Sigmoid function; Finally, multiply the channel attention weight, the spatial attention weight, and the feature F i step by step to obtain the finally fused and enhanced feature F' i : Indicates element-wise multiplication, and the output is the feature after 3-channel fusion, F’ i is a two-dimensional feature matrix; 3) Fault classification by linear layer: The enhanced feature F' after CBAM i It is still a two-dimensional feature map (T′×D′×C), which needs to be concatenated into a feature vector of a fixed length. Through GlobalAvgPool global average pooling, the two-dimensional feature map is compressed into a global feature vector F global : F i,global = GlobalAvgPool(F’ i ); F global = Contact(F 1,global , F 2,global , F 3,global ); Next, input F global into the fully connected layer. Perform a linear transformation on the input features through two fully connected layers, map them to the fault category space, and learn the combined relationship between the features. H = ReLU(W1·F global + b1); Z = W2·H + b2; where, W1, W2 are weight matrices, h is the hidden layer dimension, b1, b2 are bias terms, ReLU is the activation function, H is the hidden layer feature, and Z is the logits of the fault category; Next, the logits Z need to be converted into a probability distribution for outputting the fault type, and it is normalized using the Softmax function: P = softmax(Z); The output P is the fault type, and the size of P is [1, γ], where γ represents the number of fault types; Define the cross-entropy loss function throughout the model training process to measure the fault result P i and the true fault label L r The difference between them is: where, γ represents the number of fault types; Step Six: Alignment and mapping of semantic features of DVS vibration signals; To achieve the multi-task mapping of distributed vibration signal features to natural language for health monitoring, a semantic alignment framework is proposed. After fault feature extraction, the fault features are semanticized.e fault , using the LLM to combine user text instruction tokenization and embedding.e text Generate multi-task natural language responses to perform double semantic mapping of fault features and user prompts; Map the fault feature F through a learnable projection matrix global to the LLM embedding space e fault = LayerNorm(W2·ReLU(DWPT(W1·F global )); Among them Align with the token dimension of the LLM and optimize the consistency between features and text semantics through contrastive learning loss Optimize the consistency between features and text semantics: where sim is the cosine similarity function, τ is the temperature coefficient, and B is the batch size; At the same time, perform dynamic word segmentation and embedding on the user's question S t : e text = Tokenizer(S t )·E LLM + PE(S t ); Among them, Tokenizer is a tokenizer, and E LLM is a word embedding matrix, where PE is the positional encoding, which retains the temporal information of the context sentences; Next, the Gated Cross-Attention (GCA) mechanism is introduced to achieve fine-grained interaction between features and text; Q = w q e fault ,K = w k e text ,V = w v e text ; H fusion = α·Attention(Q, K, V)+(1 - α)e fault ; where w q , w k , w v are the projection matrices of query, key, and value, W g is the gating weight matrix, α is the gating coefficient, and H fusion is the fused semantic feature, enhancing the robustness of the model by dynamically adjusting the information fusion ratio; Step Seven: Fine-tuning of the DVSLLM model; We used the pre-trained Qwen2-1.5B as the parameterized LLM to achieve basic human-computer interaction. Then, for the improvement of domain knowledge, we fine-tuned the LLM and the alignment layer using adaptive low-rank projection: where ΔW is the parameter increment update matrix, r is the rank of the low-rank projection, and B i , A i is a pair of low-rank matrices, and f θ is a lightweight network for dynamically generating low-rank matrices based on fused features, which only needs to train 0.03% of the parameters to adapt to multi-task requirements; In summary, a complete mapping chain from distributed vibration signals to natural semantics is established, providing decision support for multi-health management tasks under complex working conditions.

Images