Method, apparatus and computer storage medium for feature extraction of fiber optic acoustic wave sensing data

Abstract

This invention relates to a method, apparatus, and computer storage medium for feature extraction from fiber optic acoustic wave sensing data. The method includes the following steps: Step 1, obtaining video training data; Step 2, obtaining visible video blocks; Step 3, obtaining the input tensor of the coding network; Step 4, obtaining visible video data features; Step 5, obtaining the reconstructed video; Step 6, obtaining the optimized autoencoder network parameters; Step 7, obtaining waterfall plot training data; Step 8, obtaining visible spatial time-frequency plot blocks; Step 9, obtaining the input tensor of the cross-modal coding network; Step 10, obtaining visible spatial time-frequency plot data features; Step 11, obtaining the reconstructed spatial time-frequency plot; Step 12, completing the overall model training; Step 13, obtaining new waterfall plot data features. This invention can effectively extract high-quality fiber optic sensing vibration signal features from waterfall plots, and can also effectively utilize data from different modalities to extract high-quality fiber optic sensing vibration signal features from waterfall plots.

Description

Technical Field

[0001] This invention relates to the field of distributed optical fiber sensing technology, specifically to a method, apparatus, and computer storage medium for extracting features from optical fiber acoustic wave sensing data. Background Technology

[0002] Distributed fiber optic sensing is a fiber optic sensing technology used to acquire vibration signals. Compared with electrical vibration sensors, it has advantages such as high resolution, small size, light weight, no electrical operation, and adaptability to harsh environments, and is widely used in fields such as earthquake monitoring, oil and gas pipeline monitoring, and railway monitoring. However, the signals generated by these sensors (waterfall plots) are two-dimensional signals, containing one time dimension and one spatial dimension. Since distributed fiber optic sensors are similar to microphone arrays, the data characteristics they generate are more like one-dimensional audio signals than easily understood two-dimensional image features. Therefore, waterfall plot data is more abstract to humans, posing a challenge to the subsequent design of feature extraction algorithms. In addition, distributed fiber optic sensors generate a large amount of data (due to their relatively high sampling rate and continuous measurement spatial resolution). The huge amount of data makes it more difficult to understand its data characteristics, further increasing the difficulty of designing feature extraction algorithms. More and more researchers are paying attention to the characteristics of waterfall plots and have begun to use deep learning techniques to extract waterfall plot features.

[0003] Deep learning methods for waterfall plot analysis have undergone several significant developmental stages. Early research reduced complexity by simplifying two-dimensional waterfall plots into one-dimensional temporal signals, employing architectures including one-dimensional convolutional neural networks and recurrent neural networks. While these methods incorporated advanced signal processing techniques such as Hidden Markov Models or Short-Time Fourier Transforms, their fundamental limitation lay in abandoning the inherent spatial correlations of distributed acoustic sensing. This simplification not only sacrificed the possibility of cross-channel verification but also led to a dual decline in algorithm robustness and classification accuracy. Alternative methods treated waterfall plots as regular two-dimensional images, typically employing standard two-dimensional convolutional architectures such as U-Net. However, these methods failed to consider the structural differences between natural images (dominated by spatial relationships) and waterfall plots (characterized by temporal evolution patterns), resulting in spatiotemporal representations that often failed to capture the dynamic essence of the underlying phenomena. More advanced hybrid methods attempted to address these limitations through a sequential processing flow: first extracting one-dimensional temporal features and then performing spatial fusion. While these methods improved performance through explicit spatiotemporal modeling, the sequential processing paradigm disrupted the symmetry of spatiotemporal information, leading to compromised quality of learned features. In their paper "DAS-MAE: a self-supervised framework for universal and high-performance representation learning of distributed acoustic sensing," researchers proposed an autoencoder employing a Transformer architecture. This autoencoder achieves synchronous processing of spatiotemporal information from waterfall plots and significantly outperforms semi-supervised baseline methods using unlabeled data, demonstrating superior generalization capabilities. However, a key challenge remains: the Transformer's requirement for massive amounts of training data limits its feature learning potential.

[0004] Recent advances in speech processing have demonstrated the significant effectiveness of applying the Transformer architecture to learn time-frequency representations from spectrograms. Inspired by this, we introduced video data into the field of distributed fiber optic sensing as additional training material for training our waterfall plot feature extraction method. After preprocessing the waterfall plot data and adjusting the autoencoder structure, we have achieved, to our knowledge, the first cross-modal training architecture in distributed fiber optic sensing. The improved model exhibits higher performance, learning higher-quality waterfall plot features, and significantly alleviates the problem of insufficient effective waterfall plot data. Summary of the Invention

[0005] In summary, to address the shortcomings of existing technologies, the technical problem to be solved by this invention is to provide a method, apparatus, and computer storage medium for extracting features from fiber optic acoustic wave sensing data.

[0006] The technical solution of this invention to solve the above-mentioned technical problems is as follows: A method for extracting features from fiber optic acoustic wave sensing data, comprising the following steps:

[0007] Step 1: Obtain video training data by collecting or acquiring publicly available data and preprocessing it;

[0008] Step 2: Based on the video training data obtained in Step 1, visible video blocks are obtained through cutting and masking processes;

[0009] Step 3: Based on the visible video blocks obtained in Step 2, tensor transformation is performed to obtain the input tensor of the encoding network;

[0010] Step 4: Based on the input tensor obtained in Step 3, the visible video data features are obtained through cross-modal coding network processing.

[0011] Step 5: Based on the visible video data features obtained in Step 4, the reconstructed video is obtained through cross-modal decoding network processing;

[0012] Step 6: Based on the reconstructed video obtained in Step 5 and the original video in Step 1, the differences are calculated and optimized to obtain the parameters of the autoencoder network trained and optimized by the video data.

[0013] Step 7: Obtain waterfall plot training data by collecting or acquiring publicly available data and preprocessing it;

[0014] Step 8: Based on the waterfall plot training data obtained in Step 7, obtain visible spatial time-frequency patches through short-time Fourier transform, cutting and masking processing.

[0015] Step 9: Based on the visible space time-frequency map obtained in Step 8, tensor transformation is performed to obtain the input tensor of the cross-modal coding network.

[0016] Step 10: Based on the input tensor obtained in Step 9, the visible space time-frequency map data features are obtained by processing the input tensor through a cross-modal coding network trained on video data.

[0017] Step 11: Based on the visible spatial time-frequency map data features obtained in Step 10, the reconstructed spatial time-frequency map is obtained through cross-modal decoding network processing;

[0018] Step 12: Based on the reconstructed spatial-time frequency map obtained in Step 11 and the original spatial-time frequency map before processing in Step 8, the differences are calculated and optimized to complete the overall training of the model.

[0019] Step 13: Based on the encoding network trained in Step 12, the new waterfall plot data is processed and features are extracted to finally obtain the new waterfall plot data features.

[0020] Based on the above technical solution, the present invention can be further improved as follows:

[0021] Furthermore, the video training data obtained in step 1 includes events, while background noise data that does not contain events does not need to be retained;

[0022] The specific cutting method in step 2 is as follows: the three-dimensional spatiotemporal video is cut into N cuboid video blocks with dimensions of T×S×S. The cuboid video blocks retain an image region of size S×S in the spatial dimension and retain T frames in the temporal dimension.

[0023] The masking method in step 2 is as follows: all video blocks are randomly masked according to a ratio R. R×N video blocks are randomly missing and are called masked video blocks. The remaining video blocks are called visible video blocks. The masking process for each video is independent of each other and is carried out in a uniform distribution.

[0024] Furthermore, step 3 specifically involves: stacking the visible video blocks obtained in step 2 to obtain a 4D tensor; flattening the three dimensions corresponding to the video blocks in this tensor to obtain a 2D tensor; since the video contains RGB channels, to facilitate network processing, an additional dimension is added to the 2D tensor to represent the RGB channels, expanding it into a 3D tensor with N dimensions. v ×(T×S×S)×3;

[0025] Step 4 specifically involves feeding the 3D tensor obtained in Step 3 into the encoding network: convolutional mapping expands the last dimension of the input tensor, outputting a tensor of the corresponding dimension; positional encoding adds trainable standard sine and cosine positional information to this output; the transformer encoder analyzes the similarity between visible video blocks, and after processing by linear layers, obtains the visible video data features, which have a dimension of N. v ×(T×S×S)×D1;

[0026] Where, N v This represents the number of visible video blocks.

[0027] Furthermore, step 5 specifically involves: supplementing the first dimension of the visible video data features obtained in step 4 to the number of video blocks N, and then using an attention mechanism network to reconstruct the video based on the features between all video blocks. The output size of the reconstructed video is N×(T×S×S)×3.

[0028] Step 6 involves identifying the differences between the reconstructed video and the original video, and then training the program, specifically as follows:

[0029] To obtain the absolute value error or mean square error between the values ​​of the reconstructed video and the original video, regularization processing can be added.

[0030] Optimize all trainable parameters in the initial autoencoder by selecting different optimizers, learning rates, and learning rate decay strategies until the best result is obtained.

[0031] Furthermore, the short-time Fourier transform in step 8 is specifically as follows: perform a short-time Fourier transform on the time axis of the two-dimensional spatiotemporal waterfall plot with latitude C×L, and take the absolute value to obtain the three-dimensional spatial time-frequency data with latitude C×F×F.

[0032] The cutting method in step 8 is as follows: the three-dimensional spatial time-frequency data is cut into M cuboid spatial time-frequency blocks with dimensions of T×S×S. The cuboid spatial time-frequency blocks are the same size as the video blocks, with T single channels in the spatial dimension and an S×S area in the time-frequency dimension.

[0033] The masking method in step 8 is as follows: randomly mask all spatial time-frequency patches according to a scale of R. Missing spatial time-frequency patches are masked spatial time-frequency patches, and the ones that are retained are visible spatial time-frequency patches. The masking process is independent and carried out in a uniform distribution. The number of masked spatial time-frequency patches is R×M.

[0034] Further, step 9 specifically involves: stacking the visible spatial time-frequency patches obtained in step 8 to obtain a 4D tensor; flattening the three dimensions corresponding to the patches in this tensor to obtain a 2D tensor; and adding a dimension to the 2D tensor to facilitate network processing, expanding it into a 3D tensor with dimension M. v ×(T×S×S)×1;

[0035] Step 10 specifically involves: feeding the 3D tensor obtained in step 9 into the autoencoder network parameters optimized in step 6; expanding the last dimension of the input tensor through convolutional mapping; adding trainable standard sine and cosine position information to the position encoding; and analyzing the similarity between visible spatial time-frequency map patches to obtain the visible spatial time-frequency map data features, the dimension of which is M. v ×(T×S×S)×D1;

[0036] Among them, M v This represents the number of visible spatial time-frequency patches.

[0037] Further, step 11 specifically involves: supplementing the first dimension of the visible spatial time-frequency map data features obtained in step 10 to the number of spatial time-frequency map patches M, and then using an attention mechanism network to reconstruct the spatial time-frequency map based on the features between all spatial time-frequency map patches. The output size of the reconstructed spatial time-frequency map is M×(T×S×S)×1.

[0038] Step 12 includes determining the differences between the reconstructed spatial time-frequency map and the original spatial time-frequency map, and then performing training, specifically:

[0039] To obtain the absolute value error or mean square error between the values ​​of the reconstructed spatial time-frequency plot and the original spatial time-frequency plot, regularization processing can be added;

[0040] Continue to optimize all trainable network parameters of the autoencoder after step 6, selecting different optimizers, learning rates, and learning rate decay strategies until the best results are achieved, and complete the training.

[0041] Further, step 13 specifically involves: performing short-time Fourier transform and segmentation on the new waterfall plot data according to the method in step 8 to obtain the corresponding spatial-time-frequency plots; stacking and flattening all spatial-time-frequency plots to obtain a 3D tensor, which is then fed into the encoding network trained in step 12 to output the features corresponding to the new waterfall plot data.

[0042] To address the aforementioned technical problems, this application also provides a fiber optic acoustic wave sensing data feature extraction device, comprising a processor and a memory, wherein the memory is coupled to the processor;

[0043] The processor may be one or more, and the memory is used to store computer programs; the processor is used to execute the computer programs in accordance with the above-described fiber optic acoustic wave sensing data feature extraction method.

[0044] To address the aforementioned technical problems, this application also provides a computer storage medium storing a computer program, which, when executed by a processor, implements the steps of the fiber optic acoustic wave sensing data feature extraction method described above.

[0045] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects:

[0046] This invention proposes a method that uses video data as feature data for waterfall plot feature extraction. A preprocessing method for the waterfall plot is presented to match the video data format, specifically by performing a short-time Fourier transform on the time axis of the waterfall plot to obtain a 3D spatial time-frequency plot, and then using the absolute value as the model input. The optimal segmentation mode for processing the spatiotemporal frequency information in the waterfall plot is given, namely, segmenting the waterfall plot into multiple spatial time-frequency blocks of equal volume (T×S×S), and using a powerful attention mechanism to analyze the similarity of spatial time-frequency sequences to obtain its spatiotemporal frequency information. Furthermore, a mask reconstruction task is used to pre-train the Transformer network with an appropriate task difficulty. This avoids simply learning a simple interpolation reconstruction algorithm, forcing the model to learn valuable features from limited information; it also reduces the amount of data in the waterfall plot or video while retaining important features, thus improving the model's training efficiency. Attached Figure Description

[0047] Figure 1 This is a flowchart of the method in Example 1;

[0048] Figure 2 This is a flowchart of the autoencoder model in Embodiment 1 of the present invention;

[0049] Figure 3 This is a flowchart of the pre-trained network extracting features from new data in Example 1;

[0050] Figure 4 A visual comparison chart of the features extracted by the method of this invention and principal component analysis in terms of implementation effect;

[0051] Figure 5 The confusion matrix of the pre-trained model (the method of this invention) and the initial model in the actual classification task is compared to demonstrate the implementation effect.

[0052] in;

[0053] Figure 2 The numbers 1, 16, 49, and 64 represent the spatial time-frequency plots obtained after cutting the 4×4×4 three-dimensional spatial time-frequency plot in the image. The numbering in the three-dimensional spatial time-frequency plot follows a default order, from left to right and from top to bottom. This numbering clarifies the stacking order of the spatial time-frequency plots.

[0054] Figure 3 The numbers 1, 16, 49, and 64 represent the spatial time-frequency plots obtained after cutting the 4×4×4 three-dimensional spatial time-frequency plot in the image. The numbering in the three-dimensional spatial time-frequency plot follows a default order, from left to right and from top to bottom. This numbering clarifies the stacking order of the spatial time-frequency plots. Detailed Implementation

[0055] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0056] A method for feature extraction from fiber optic acoustic wave sensing data includes the following steps:

[0057] Step 1: Obtain video training data by collecting or acquiring publicly available data and preprocessing it.

[0058] Collect existing video data or use public video datasets (such as Kinetics-400), normalize the data to obtain unlabeled video training data, and ensure that the data contains events; background noise data without events does not need to be retained.

[0059] Step 2: Based on the video training data obtained in Step 1, visible video blocks are obtained through cutting and masking.

[0060] The cutting method is as follows: the three-dimensional spatiotemporal video (one-dimensional time × two-dimensional space) is cut into N cuboid video blocks with dimensions T×S×S (the spatial dimension retains an image region of size S×S, and the temporal dimension retains T frames of images); the masking method is as follows: all video blocks are randomly masked according to a ratio R, R×N video blocks are randomly missing (called masked video blocks), and the remaining video blocks are retained (called visible video blocks, with a quantity of NR×N). The masking process for each video is independent of each other and is performed in a uniform distribution.

[0061] Step 3: Based on the visible video blocks obtained in Step 2, tensor transformation is performed to obtain the input tensor of the encoding network.

[0062] The visible video blocks obtained in step 2 are stacked to obtain a 4D tensor. The three dimensions corresponding to the video blocks in this tensor are flattened to obtain a 2D tensor. Since the video contains RGB channels, to facilitate network processing, an additional dimension is added to the 2D tensor to represent the RGB channels, expanding it into a 3D tensor (with N dimensions). v ×(T×S×S)×3,N v (Number of visible video blocks).

[0063] Step 4: Based on the input tensor obtained in Step 3, the visible video data features are obtained through cross-modal coding network processing.

[0064] The 3D tensor obtained in step 3 is fed into the encoding network (including convolutional mapping, positional encoding, and Transformer encoder modules): the convolutional mapping expands the last dimension of the input tensor and outputs a tensor of the corresponding dimension; the positional encoding adds trainable standard sine and cosine positional information to the output; the Transformer encoder analyzes the similarity between visible video blocks, and after processing by a linear layer, outputs visible video data features (with dimension N). v ×(T×S×S)×D 1， N v (Number of visible video blocks).

[0065] Step 5: Based on the visible video data features obtained in Step 4, the reconstructed video is obtained through cross-modal decoding network processing.

[0066] The first dimension of the visible video data features obtained in step 4 is padded to the number of video blocks N, and then the attention mechanism network is used to reconstruct the video based on the features between all video blocks. The output size of the reconstructed video is N×(T×S×S)×3.

[0067] Step 6: Based on the reconstructed video obtained in Step 5 and the original video in Step 1, the differences are calculated and optimized to obtain the parameters of the autoencoder network trained and optimized by the video data.

[0068] Step 6 involves identifying the differences between the reconstructed video and the original video, and then training the program, specifically as follows:

[0069] The absolute error or mean square error between the reconstructed video and the original video can be calculated, and regularization can be added.

[0070] Select an appropriate optimizer, learning rate, and learning rate decay strategy, and iteratively optimize all trainable parameters of the initial autoencoder until the best result is obtained.

[0071] Step 7: Obtain waterfall plot training data by collecting or acquiring publicly available data and preprocessing it.

[0072] Waterfall plot data can be collected using DAS devices, and waterfall plots containing events can be selected and retained, while waterfall plots with background noise but no events can be removed. The data can then be cropped to the same size and normalized. Alternatively, a publicly available distributed vibration sensor waterfall plot dataset can be used, and each data point can be normalized to obtain unlabeled waterfall plot training data.

[0073] Step 8: Based on the waterfall plot training data obtained in Step 7, obtain visible spatial time-frequency plots through short-time Fourier transform, cutting and masking.

[0074] The short-time Fourier transform is specifically performed by taking the absolute value of the time axis of the two-dimensional spatiotemporal waterfall plot (dimension C×L, one-dimensional space × one-dimensional time) to obtain the three-dimensional spatial time-frequency data (dimension C×F×F, one-dimensional space × two-dimensional time-frequency).

[0075] The specific cutting method is as follows: the three-dimensional spatial time-frequency data is cut into M cuboid spatial time-frequency patches with dimensions of T×S×S (the same size as the video patch, with T single channels in the spatial dimension and an S×S area in the time-frequency dimension).

[0076] The masking method is as follows: randomly mask all spatial time-frequency patches according to a scale of R. Missing patches are the masked spatial time-frequency patches (number R × M), and the retained patches are the visible spatial time-frequency patches (number M). v =MR×M), the masking process is independent and carried out in a uniform distribution.

[0077] Step 9: Based on the visible space time-frequency map obtained in Step 8, the input tensor of the cross-modal coding network is obtained through tensor transformation.

[0078] The visible spatial time-frequency patches obtained in step 8 are stacked to obtain a 4D tensor. The three dimensions corresponding to the patches in this tensor are flattened to obtain a 2D tensor. To facilitate network processing, an additional dimension is added to the 2D tensor (representing the network operation dimension, with a value of 1 in the waterfall plot), expanding it into a 3D tensor (with dimension M). v×(T×S×S)×1,M v (Number of visible spatial time-frequency patches).

[0079] Step 10: Based on the input tensor obtained in Step 9, the visible space time-frequency map data features are obtained by processing the input tensor through a cross-modal coding network trained on video data.

[0080] The 3D tensor obtained in step 9 is fed into the autoencoder network parameters optimized in step 6 (structure same as step 4): convolutional mapping expands the last dimension of the input tensor; position encoding adds trainable standard sine and cosine position information; the Transformer encoder (structure same as step 4) analyzes the similarity between visible spatial time-frequency map patches to obtain the visible spatial time-frequency map data features (dimensional M). v ×(T×S×S)×D 1， M v (Number of visible spatial time-frequency patches).

[0081] Step 11: Based on the visible spatial time-frequency map data features obtained in Step 10, the reconstructed spatial time-frequency map is obtained through cross-modal decoding network processing.

[0082] The first dimension of the visible spatial time-frequency map data features obtained in step 10 is padded to the number of spatial time-frequency map patches M. Then, the attention mechanism network is used to reconstruct the spatial time-frequency map based on the features between all spatial time-frequency map patches. The output size of the reconstructed spatial time-frequency map is M×(T×S×S)×1.

[0083] Step 12: Based on the reconstructed spatial time-frequency map obtained in Step 11 and the original spatial time-frequency map before processing in Step 8, the differences are calculated and optimized to complete the overall training of the model.

[0084] Step 12 includes determining the differences between the reconstructed spatial time-frequency map and the original spatial time-frequency map, and then performing training, specifically:

[0085] To obtain the absolute value error or mean square error between the values ​​of the reconstructed spatial time-frequency plot and the original spatial time-frequency plot, regularization processing can be added;

[0086] Continue to optimize all trainable network parameters of the autoencoder after step 6, selecting different optimizers, learning rates, and learning rate decay strategies until the best results are achieved, and complete the training.

[0087] Step 13: Based on the encoding network trained in Step 12, the new waterfall plot data is processed and features are extracted to finally obtain the new waterfall plot data features.

[0088] The new waterfall plot data is subjected to short-time Fourier transform (absolute value) and segmentation as described in step 8 to obtain the corresponding spatial-time frequency plots. All spatial-time frequency plots are stacked and flattened to obtain a 3D tensor (dimension M×(T×S×S)×1), which is fed into the encoding network trained in step 12 to output the features corresponding to the new waterfall plot data. These features can be mapped to different sizes according to the needs of subsequent tasks.

[0089] Example 1

[0090] Reference Figure 1 A method for feature extraction from distributed optical fiber acoustic wave sensing data, specifically including the following steps:

[0091] Step 1: Collect video data using existing equipment, or directly use existing public video datasets (Kinetics-400), and then normalize the data to obtain unlabeled data for pre-training.

[0092] Step 2, refer to Figure 1 The video data is divided into 2×16×16 video blocks, discarding redundant data, resulting in N video blocks of the same size. The divided video is then masked, retaining only 10% of the video blocks, called visible video blocks (0.1N in total), and discarding 90% of the video blocks, called masked video blocks (0.9N in total). Each video masking process is independent and does not interfere with others, proceeding completely uniformly.

[0093] Step 3, refer to Figure 1 The 0.1N visible video blocks are stacked to obtain a 4D tensor with a size of 0.1N×2×16×16. The three dimensions corresponding to the video blocks in the 4D tensor are further flattened to obtain a 2D tensor with a size of 0.1N×512. Each video contains three RGB channels. To facilitate subsequent network processing, a dimension is added to the 2D tensor to represent the RGB channels, expanding it into a 3D tensor with a size of 0.1N×512×3. The last dimension represents both the RGB channels and the dimension of network processing. For the original video, the value is 3.

[0094] Step 4, refer to Figure 2The 3D tensor obtained in the third step (corresponding to all visible video blocks in a video) is fed into the encoding network to extract features. The encoding network mainly consists of three modules: convolutional mapping, positional encoding, and Transformer encoder. Convolutional mapping expands the last dimension of the input 3D tensor from 3 to 384, enabling the extraction of more diverse features, with an output size of 0.1N×512×384. Positional encoding adds trainable standard sine and cosine positional information to the linear mapping output. After adding positional information, the attention mechanism of the Transformer encoder is used to analyze the similarity between visible video blocks and extract the corresponding video features. The Transformer encoder contains a 12-layer network, with each layer using a 6-head multi-head attention mechanism, specifically a self-attention mechanism. It calculates the input key and query matrices, multiplies the key matrix by the transpose of the query matrix, normalizes the result to obtain the weight matrix, and finally multiplies the weight matrix by the value matrix to obtain each self-attention output, with an output size of 0.1N×512×384. Six self-attention mechanism results are concatenated in dimension 0, resulting in an output of size 0.6N×512×384. A linear layer is used to map dimension 0 back to 0.1N, yielding a multi-head attention mechanism output of size 0.1N×512×384. Two more linear layers are then applied, first expanding the second dimension by a factor of 4 to obtain an output of 0.1N×512×1532, and then compressing the second dimension by a factor of 4. Finally, the Transformer encoder output, i.e., the extracted features, maintains the same size as the multi-head attention mechanism output, 0.1N×512×384.

[0095] Step 5, refer to Figure 2 The video features obtained in step four are then fed into the decoding network. The encoding network mainly consists of three modules: convolutional mapping, positional encoding, and a Transformer encoder. Convolutional mapping expands the last dimension of the input 3D tensor from 384 to 192, resulting in an output size of 0.1N×512×192. Before positional encoding, the location of each missing video block is reconstructed using a trainable tensor of size 1×512×192, changing the output size of the convolutional mapping to N×512×192. This is then fed into positional encoding, which adds trainable standard sine and cosine positional information, and the output is then fed into the Transformer encoder. The Transformer encoder follows the same data processing flow as in step three, containing a four-layer network. Each layer uses a three-head multi-head attention mechanism, resulting in a final output of a 3D tensor of size N×512×192. Finally, a linear mapping is used to map the last dimension of the output tensor to 3, yielding the reconstructed video of size N×512×3.

[0096] Step 6: Calculate the difference between the reconstructed video and the original video, and train the network. Here, the minimum mean squared error (MSE) is chosen as the reconstruction loss function:

[0097] ;

[0098] in, This represents the missing video block after the i-th video has passed through the mask. This represents the reconstruction estimate of the missing video block in the i-th video. Represents the L2 norm, This represents the average across the entire training dataset. When evaluating the difference between the reconstructed video and the original video, only the reconstruction loss of missing video patches is considered, and the impact of the model's computation on visible video patches is not taken into account. The optimization algorithm can be any gradient descent algorithm and its iterative optimization method.

[0099] Step 7 requires deploying fiber optic cables using existing DAS equipment to collect waterfall plot data. The collected waterfall plots are then filtered, retaining those with events in the time dimension and removing those without events within a given time period. The resulting waterfall plots are then cropped to the same size, such as 12×10000 (spatial × time), and the data is normalized to obtain unlabeled data for pre-training.

[0100] Alternatively, you can directly train using an existing publicly available distributed vibration sensor waterfall plot dataset (Cao X, Su Y, Jin Z, et al. An open dataset of φ-OTDR events with two classification models as baselines. Results in Optics, 2023, 10: 100372.), requiring only the normalization of each data point in the dataset. Each data point in the dataset is 12 × 10000 (spatial × temporal).

[0101] Step 8, refer to Figure 1 The waterfall plot data was Fourier transformed along the time axis to obtain a 3D spatial time-frequency map of size 12×96×96. This 3D spatial time-frequency map was then divided into 2×16×16 spatial time-frequency map patches, resulting in 216 spatial time-frequency map patches of the same size. The segmented spatial time-frequency map patches were then masked using the same method as the video masking: only 10% of the spatial time-frequency map patches were retained, called visible spatial time-frequency map patches (22 in total), while 90% were discarded, called masked spatial time-frequency map patches (194 in total). The masking process for each spatial time-frequency map was independent and did not interfere with each other, proceeding entirely according to a uniform distribution.

[0102] Step 9, refer to Figure 1 The 22 visible spatial time-frequency plots are stacked to obtain a 4D tensor with a size of 22×2×16×16. The spatial time-frequency plots in the 4D tensor are further flattened along the three dimensions to obtain a 2D tensor with a size of 22×512. To facilitate subsequent network processing, a dimension is added to the 2D tensor to expand it into a 3D tensor with a size of 22×512×1. The last dimension represents the dimension processed by the network, and its value is 1 for the original spatial time-frequency plot.

[0103] Step 10, refer to Figure 2 The 3D tensor obtained in step nine (corresponding to all visible spatial-time frequency patches in a waterfall plot) is fed into the encoding network to extract features. The encoding network mainly consists of three modules: convolutional mapping, positional encoding, and a Transformer encoder. Convolutional mapping expands the last dimension of the input 3D tensor from 1 to 384, enabling the extraction of more diverse features, with an output size of 22×512×384. Positional encoding adds trainable standard sine and cosine positional information to the linear mapping output. After adding positional information, the attention mechanism of the Transformer encoder is used to analyze the similarity between visible spatial-time frequency patches and extract the corresponding waterfall plot features. The Transformer encoder follows the same data processing flow as in step four, containing a 12-layer network, with each layer using a 6-head multi-head attention mechanism, and finally outputting a 3D tensor of size 22×512×384.

[0104] Step 11, refer to Figure 2 The waterfall plot features obtained in step 10 are then fed into the decoding network. The encoding network mainly consists of three modules: convolutional mapping, positional encoding, and a Transformer encoder. Convolutional mapping expands the last dimension of the input 3D tensor from 384 to 192, resulting in an output size of 22×512×192. Before positional encoding, the location of each missing waterfall plot is recovered using a trainable tensor of size 1×512×192, changing the output size of the convolutional mapping to 216×512×192. This is then fed into positional encoding, which adds trainable standard sine and cosine positional information, before the output is fed into the Transformer encoder. The Transformer encoder follows the same data processing flow as in step 4, containing a 4-layer network. Each layer uses a 3-head multi-head attention mechanism, resulting in a final output of a 216×512×192 3D tensor. Finally, a linear mapping is used to map the last dimension of the output tensor to 1, yielding a reconstructed spatial-temporal map of size 216×512×1.

[0105] Step 12: Calculate the difference between the reconstructed spatial-time-frequency map and the original spatial-time-frequency map, and train the network. Here, the minimum mean square error (MSE) is chosen as the reconstruction loss function:

[0106] ;

[0107] in, This represents the missing spatial-time frequency patch after compression of the i-th waterfall plot. This represents the reconstruction estimate of the missing spatial-time frequency patch in the i-th waterfall plot. Represents the L2 norm, This represents the average across the entire training dataset. When evaluating the difference between the reconstructed spatial-time-frequency map and the original spatial-time-frequency map, only the reconstruction loss of missing spatial-time-frequency map patches is considered, and the impact of the model computation on visible spatial-time-frequency map patches is not considered. The optimization algorithm can be any gradient descent algorithm and its iterative optimization method.

[0108] Step 13, refer to Figure 3 The trained encoding network is used to extract new waterfall plot data features. The new waterfall plot with extracted features needs to be cut according to the short-time Fourier transform and cutting method used during training, i.e., cut into waterfall plot tiles of dimension 2×16×16. After stacking and flattening all the waterfall plot tiles, a 3D tensor of size M×512×1 is obtained and fed into the trained encoding network to obtain the features corresponding to the new waterfall plot data, with a size of M×512×384. This feature can be mapped to different sizes according to the needs of subsequent tasks.

[0109] Example 2

[0110] This embodiment provides a fiber optic acoustic wave sensor data feature extraction device, including a processor and a memory, wherein the memory is coupled to the processor. There may be one or more processors, and the memory is used to store computer programs. The processor is used to execute the computer programs to implement the fiber optic acoustic wave sensor data feature extraction method described in Embodiment 1. It is worth noting that the computer device included in this embodiment can be shared with the computer device of a DAS demodulator, or it can be executed on two different computers. That is, after demodulation is completed on computer 1, the DAS data is copied to computer 2, and computer 2 is used to train the proposed model. In this example, since a publicly available DAS classification dataset is used, it is not shared with the computer device of the DAS demodulator; two independent computers are used.

[0111] Example 3

[0112] This embodiment provides a computer storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of the fiber optic acoustic wave sensing data feature extraction method described in Embodiment 1.

[0113] The computer storage medium can be a computer device, which may be a server. The computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores training data for pre-trained models. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. The computer program is executed by the processor to train the model.

[0114] Example 4

[0115] This embodiment provides a computer storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of the fiber optic acoustic wave sensing data feature extraction method described in Embodiment 1.

[0116] The computer storage medium can be another type of computer device, which can be a terminal. This computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor of this computer device provides computational and control capabilities. The memory of this computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface of this computer device is used for exchanging information between the processor and external devices. The communication interface of this computer device is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it is used to train the model. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0117] Implementation effect

[0118] To verify the effectiveness of this invention, this example uses model parameters trained on the publicly available video dataset Kinetics, and further trains on the publicly available DAS classification dataset (Cao X, et al. An open dataset of φ-OTDR events with two classification models as baselines. Results in Optics, 2023, 10: 100372.). This dataset contains six different event categories: background noise (0), digging (1), knocking (2), watering (3), shaking railings (4), and walking (5). The content in parentheses represents the label corresponding to the event. Each event contains multiple different signal sources to ensure the diversity of the dataset. This dataset contains approximately 15,000 data points, divided into a training set and a test set at a ratio of 4:1, with approximately 12,000 and 2,000 data points respectively. It is worth noting that only waterfall plot data was used during training, without utilizing their label information. The model was trained and optimized with a learning rate of 0.0001, which decayed with the number of iterations according to a cosine function. After 500 training iterations, a good result was achieved.

[0119] The trained encoding network was used to extract data features from the publicly available training dataset, resulting in a feature size of 216×512×384. The 3D features were averaged along the 0th dimension and then the 0th dimension was removed to obtain 2D averaged features. The t-distributed random neighborhood embedding (t-SNE) was used to reduce the dimensionality of the averaged features to a 2D 1×2 matrix, representing a point in the plane. The dimensionality reduction result does not depend on any label information, only preserving the proximity or similarity of features in the high-dimensional space. The data features of each waterfall plot in the publicly available training dataset were then dimensionality-reduced using the t-SNE method and plotted on... Figure 4 In order to achieve better visual effects, we use label information to extract features of different events, and then mark the corresponding two-dimensional points with different colors after dimensionality reduction. Figure 5 As can be clearly seen, features extracted from various types of waterfall plots exhibit a clear and obvious dispersion, while features from the same type of waterfall plot tend to cluster together. For ease of comparison, Principal Component Analysis (PCA) was used instead of the pre-trained model to extract features from the waterfall plots, and these features were visualized using the exact same t-SNE algorithm. Figure 5 In the results, after dimensionality reduction, the features generated by PCA showed no significant feature separation between different types of events, nor any significant clustering within the same event type. Its feature extraction capability was significantly lower than that of the method of this invention.

[0120] In the second embodiment, the pre-trained encoding network was used in a real-world anti-damage application in Zhengzhou, my country, to verify the model's generalization ability. In this experiment, the sensing fiber was buried in a U-shaped trench, and a DAS device with a sampling rate of 2 kSa / s and a spatial resolution of 10 meters was used to capture vibrations within a 70-meter range. We created a dataset containing eight different event types: background noise (0), road roller driving (1), road roller compaction (2), excavator digging (3), excavator driving (4), electric drill digging (5), fully loaded forklift driving (6), and empty forklift driving (7). The dataset contained approximately 1400 elements, with the number of elements for each type being 42 (0), 100 (1), 300 (2), 332 (3), 334 (4), 38 (5), 54 (6), and 186 (7), respectively. The size of each waterfall plot was set to 7×9984 (space×time). Each type of waterfall plot was divided into training and test data in a 4:1 ratio to create training and test sets. An additional trainable linear layer is added to the tail of the trained encoding network. This linear layer maps features to the probability of each class on the training set by optimizing the cross-entropy loss. It is then fine-tuned 150 times on the training set. After fine-tuning, a classification task is performed on the test set, and the observed classification accuracy is 95.3%. Since the accuracy exceeds 90%, this is valuable for practical applications. To demonstrate the advantages of the model in this invention, an identical classification model is built. This model uses initial parameters and has not been pre-trained; it is called the initial model. Supervised training of the initial model is performed on the same training set until convergence. Its classification accuracy on the test set is only 80.9%, a decrease of 14.4% compared to the pre-trained model of this invention. Figure 5 The confusion matrices for the pre-trained model (in this invention) and the initial model are given respectively. The horizontal axis of the confusion matrix represents the label predicted by the model, and the vertical axis represents the true label of the data (usually manually labeled). Therefore, the (i, j)th element in the matrix represents the data that the model predicts as the jth type of event, but the actual label is the ith type of event. The confusion matrix is ​​mostly a square matrix. The diagonal elements in the square matrix are the correctly predicted samples, and the off-diagonal elements represent the incorrectly predicted samples. The accuracy of the model prediction can be obtained by dividing the total number of data on the diagonal by the total number of data in the confusion matrix. It is not difficult to see from the confusion matrix that the prediction error of the pre-trained model (in this invention) in the operation of a fully loaded forklift (6) is much smaller than that of the initial model. This shows that the negative impact of the limited training samples can be mitigated by the pre-trained model (in this invention). In addition, the confusion between excavator digging (3) and excavator driving (4) in the initial model is improved in the results of the pre-trained model (in this invention).

[0121] The training of the above model can be achieved entirely or partially through software, hardware, or a combination thereof. The model can be embedded in or independent of the processor in a computer device in hardware form, or it can be stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0122] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for feature extraction from fiber optic acoustic wave sensing data, characterized in that, Includes the following steps: Step 1: Obtain video training data by collecting or acquiring publicly available data and preprocessing it; Step 2: Based on the video training data obtained in Step 1, visible video blocks are obtained through cutting and masking processes; Step 3: Based on the visible video blocks obtained in Step 2, tensor transformation is performed to obtain the input tensor of the encoding network; Step 4: Based on the input tensor obtained in Step 3, the visible video data features are obtained through cross-modal coding network processing. Step 5: Based on the visible video data features obtained in Step 4, the reconstructed video is obtained through cross-modal decoding network processing; Step 6: Based on the reconstructed video obtained in Step 5 and the original video in Step 1, the differences are calculated and optimized to obtain the parameters of the autoencoder network trained and optimized by the video data. Step 7: Obtain waterfall plot training data by collecting or acquiring publicly available data and preprocessing it; Step 8: Based on the waterfall plot training data obtained in Step 7, obtain visible spatial time-frequency patches through short-time Fourier transform, cutting and masking processing. Step 9: Based on the visible space time-frequency map obtained in Step 8, tensor transformation is performed to obtain the input tensor of the cross-modal coding network. Step 10: Based on the input tensor obtained in Step 9, the visible space time-frequency map data features are obtained by processing the input tensor through a cross-modal coding network trained on video data. Step 11: Based on the visible spatial time-frequency map data features obtained in Step 10, the reconstructed spatial time-frequency map is obtained through cross-modal decoding network processing; Step 12: Based on the reconstructed spatial-time frequency map obtained in Step 11 and the original spatial-time frequency map before processing in Step 8, the differences are calculated and optimized to complete the overall training of the model. Step 13: Based on the encoding network trained in Step 12, the new waterfall plot data is processed and features are extracted to finally obtain the new waterfall plot data features.

2. The fiber optic acoustic wave sensing data feature extraction method according to claim 1, characterized in that, The video training data obtained in step 1 includes events; background noise data that does not contain events does not need to be retained. The specific cutting method in step 2 is as follows: the three-dimensional spatiotemporal video is cut into N cuboid video blocks with dimensions of T×S×S. The cuboid video blocks retain an image region of size S×S in the spatial dimension and retain T frames in the temporal dimension. The masking method in step 2 is as follows: all video blocks are randomly masked according to a ratio R. R×N video blocks are randomly missing and are called masked video blocks. The remaining video blocks are called visible video blocks. The masking process for each video is independent of each other and is carried out in a uniform distribution.

3. The fiber optic acoustic wave sensing data feature extraction method according to claim 1, characterized in that, Step 3 specifically involves: stacking the visible video blocks obtained in Step 2 to obtain a 4D tensor; flattening the three dimensions corresponding to the video blocks in this tensor to obtain a 2D tensor; since the video contains RGB channels, to facilitate network processing, an additional dimension is added to the 2D tensor to represent the RGB channels, expanding it into a 3D tensor with dimension N. v ×(T×S×S)×3; Step 4 specifically involves feeding the 3D tensor obtained in Step 3 into the encoding network: convolutional mapping expands the last dimension of the input tensor, outputting a tensor of the corresponding dimension; positional encoding adds trainable standard sine and cosine positional information to this output; the transformer encoder analyzes the similarity between visible video blocks, and after processing by linear layers, obtains the visible video data features, which have a dimension of N. v ×(T×S×S)×D1; Where, N v This represents the number of visible video blocks.

4. The fiber optic acoustic wave sensing data feature extraction method according to claim 1, characterized in that, Step 5 specifically involves: completing the first dimension of the visible video data features obtained in Step 4 to the number of video blocks N, and then using an attention mechanism network to reconstruct the video based on the features between all video blocks. The output size of the reconstructed video is N×(T×S×S)×3. Step 6 involves identifying the differences between the reconstructed video and the original video, and then training the program, specifically as follows: To obtain the absolute value error or mean square error between the values ​​of the reconstructed video and the original video, regularization processing can be added. Optimize all trainable parameters in the initial autoencoder by selecting different optimizers, learning rates, and learning rate decay strategies until the best result is obtained.

5. The fiber optic acoustic wave sensing data feature extraction method according to claim 1, characterized in that, The short-time Fourier transform in step 8 is specifically as follows: perform a short-time Fourier transform on the time axis of the two-dimensional spatiotemporal waterfall plot with latitude C×L, and take the absolute value to obtain the three-dimensional spatial time-frequency data with latitude C×F×F. The cutting method in step 8 is as follows: the three-dimensional spatial time-frequency data is cut into M cuboid spatial time-frequency blocks with dimensions of T×S×S. The cuboid spatial time-frequency blocks are the same size as the video blocks, with T single channels in the spatial dimension and an S×S area in the time-frequency dimension. The masking method in step 8 is as follows: randomly mask all spatial time-frequency patches according to a scale of R. Missing spatial time-frequency patches are masked spatial time-frequency patches, and the ones that are retained are visible spatial time-frequency patches. The masking process is independent and carried out in a uniform distribution. The number of masked spatial time-frequency patches is R×M.

6. The fiber optic acoustic wave sensing data feature extraction method according to claim 1, characterized in that, Step 9 specifically involves: stacking the visible spatial time-frequency patches obtained in Step 8 to obtain a 4D tensor; flattening the three dimensions corresponding to the patches in this tensor to obtain a 2D tensor; and adding a dimension to the 2D tensor to facilitate network processing, expanding it into a 3D tensor with dimension M. v ×(T×S×S)×1; Step 10 specifically involves: feeding the 3D tensor obtained in step 9 into the autoencoder network parameters optimized in step 6; expanding the last dimension of the input tensor through convolutional mapping; adding trainable standard sine and cosine position information to the position encoding; and analyzing the similarity between visible spatial time-frequency map patches to obtain the visible spatial time-frequency map data features, the dimension of which is M. v ×(T×S×S)×D1; Among them, M v This represents the number of visible spatial time-frequency patches.

7. The fiber optic acoustic wave sensing data feature extraction method according to claim 1, characterized in that, Step 11 specifically involves: completing the first dimension of the visible spatial time-frequency map data features obtained in step 10 to the number of spatial time-frequency map patches M, and then using an attention mechanism network to reconstruct the spatial time-frequency map based on the features between all spatial time-frequency map patches. The output size of the reconstructed spatial time-frequency map is M×(T×S×S)×1. Step 12 includes determining the differences between the reconstructed spatial time-frequency map and the original spatial time-frequency map, and then performing training, specifically: To obtain the absolute value error or mean square error between the values ​​of the reconstructed spatial time-frequency plot and the original spatial time-frequency plot, regularization processing can be added; Continue to optimize all trainable network parameters of the autoencoder after step 6, selecting different optimizers, learning rates, and learning rate decay strategies until the best results are achieved, and complete the training.

8. The fiber optic acoustic wave sensing data feature extraction method according to claim 1, characterized in that, Step 13 specifically involves: performing a short-time Fourier transform and segmentation on the new waterfall plot data according to the method in step 8 to obtain the corresponding spatial-time frequency plots; stacking and flattening all spatial-time frequency plots to obtain a 3D tensor, which is then fed into the encoding network trained in step 12 to output the features corresponding to the new waterfall plot data.

9. A fiber optic acoustic wave sensing data feature extraction device, characterized in that, It includes a processor and a memory, wherein the memory is coupled to the processor; The processor may be one or more, and the memory is used to store a computer program; the processor is used to execute the computer program to implement the fiber optic acoustic wave sensing data feature extraction method according to any one of claims 1 to 8.

10. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which, when executed by a processor, implements the steps of the fiber optic acoustic wave sensing data feature extraction method as described in any one of claims 1 to 8.

Images