Distributed optical fiber acoustic wave sensing data lossless compression method and system based on probability prediction

Abstract

The application relates to a kind of distributed optical fiber acoustic wave sensing data lossless compression method and system based on probability prediction, by obtaining original sensing data and carrying out data block, difference processing, byte rearrangement and other pretreatment, obtain the data sequence to be encoded;Based on the encoded data and the construction of adjacent data, the space-time context information is constructed, the probability distribution of the current symbol to be encoded is predicted using a large model probability prediction module;According to the probability distribution, entropy coding is carried out to generate lossless compression code stream;The decoding end adopts consistent prediction process and entropy decoding to reconstruct the original data.The application uses a large model to learn the statistical dependence relationship of data in time dimension, space dimension and data block, to realize efficient lossless compression in the form of approximating information entropy lower bound, and greatly reduces the storage and transmission pressure of long-distance, high sampling rate distributed optical fiber acoustic wave sensing system under the premise of ensuring that data is completely consistent bit by bit.

Description

Technical Field

[0001] This invention relates to the field of fiber optic sensing technology, and in particular to a lossless compression method and system for distributed fiber optic acoustic wave sensing data based on probability prediction. Background Technology

[0002] Distributed fiber optic acoustic sensing technology utilizes optical fiber as a continuous sensing medium to achieve long-distance, high spatial resolution vibration and acoustic wave measurements by demodulating scattered light signals. It offers advantages such as resistance to electromagnetic interference, passive sensing, and adaptability to harsh environments, and has been widely applied in fields such as oil and gas pipeline monitoring, perimeter security, earthquake monitoring, and structural health monitoring.

[0003] With the continuous improvement of system detection distance, spatial resolution, and sampling rate, this technology generates extremely large-scale sensing data during operation. Taking a phase-sensitive optical time-domain reflectometer as an example, the sensing fiber is divided into a large number of spatial sampling points, and each point continuously collects vibration signals along the time direction, forming a two-dimensional waterfall plot data containing both spatial and temporal dimensions. This type of data has a high sampling frequency, long continuous acquisition time, multiple spatial channels, and rapid data updates. In long-term online monitoring, edge deployment, and multi-site collaborative scenarios, the direct storage and transmission of raw data puts enormous pressure on storage resources and communication bandwidth, severely restricting the system's real-time performance and engineering application capabilities.

[0004] Existing data compression methods are divided into two categories: lossy compression and lossless compression. Lossy compression can achieve high compression ratios through downsampling, transform coding, and other methods, but the reconstructed data suffers from irreversible distortion, and weak signals or key phase details may be lost, making it difficult to meet the high data integrity requirements of applications such as earthquake monitoring and pipeline leak early warning. Lossless compression can guarantee complete data consistency, but traditional methods are mostly based on general dictionary coding or entropy coding, which do not fully utilize the statistical laws of DAS data in terms of temporal continuity, spatial adjacency correlation, background noise stability, and the sparsity of vibration events, resulting in limited compression efficiency.

[0005] Deep learning models excel in sequence probability modeling. Theoretically, they can achieve efficient lossless compression by accurately predicting the probability distribution of current data symbols under contextual conditions, combined with arithmetic coding and isentropic coding methods, approximating the lower bound of information entropy. However, existing methods mostly focus on event recognition and lossy reconstruction, with few designs specifically for lossless compression of raw DAS data. Direct application faces the following challenges: DAS data has a unique spatiotemporal two-dimensional structure, making it difficult for general sequence models to adapt to the correlation between spatial channels and temporal sampling; data formats are diverse, with different formats imposing different requirements on lossless coding; model outputs often contain numerical errors, failing to directly meet lossless recovery requirements.

[0006] In summary, how to utilize large models for accurate probabilistic modeling and effectively combine it with entropy coding to improve compression efficiency while ensuring lossless bit-by-bit recovery, is an urgent problem to be solved in this field. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the existing technology by providing a lossless compression method and system for distributed fiber optic acoustic sensor data based on probabilistic prediction, thereby solving or partially solving the problem of low compression efficiency caused by the failure of traditional lossless compression methods to fully utilize the two-dimensional spatiotemporal redundancy of this type of data.

[0008] The objective of this invention can be achieved through the following technical solutions: One aspect of the present invention provides a lossless compression method for distributed fiber optic acoustic wave sensor data based on probabilistic prediction, comprising the following steps: Acquire raw sensing data collected by the distributed fiber optic acoustic wave sensing system; The original sensor data is preprocessed to obtain a data sequence to be encoded; Contextual information is constructed based on encoded data and neighborhood data, and a large model is used to predict the probability distribution of the current data symbol to be encoded. At the encoding end, entropy encoding is performed on the current data symbol to be encoded according to the probability distribution to generate a lossless compressed bitstream; At the decoding end, based on the large model probability prediction process and entropy decoding process consistent with the encoding end, data completely consistent with the original sensing data is reconstructed using the lossless compressed bitstream.

[0009] As a preferred technical solution, the preprocessing includes at least one of data segmentation, differential processing, and byte rearrangement, wherein: The data segmentation is used to divide the raw sensing data into data blocks to be compressed along the time direction and / or spatial direction; The differential processing is used to convert the original data into residual data by utilizing the temporal continuity and spatial correlation of the data. The byte rearrangement is used to rearrange different byte positions of multi-byte data according to the data storage format in order to enhance the statistical correlation between adjacent symbols.

[0010] As a preferred technical solution, the context information includes at least one of the following: The data symbols that have been encoded before the data symbol to be encoded, historical time-sampling data at the same optical fiber spatial location, adjacent spatial location data sampled at the same time, statistical information of adjacent data blocks, time location information and spatial location information.

[0011] As a preferred technical solution, the large model includes a probabilistic prediction model based on Transformer structure, state-space model structure, convolutional neural network structure, recurrent neural network structure, or a combination thereof.

[0012] As a preferred technical solution, the entropy coding includes at least one of arithmetic coding, interval coding, ANS coding, Huffman coding, or context-adaptive entropy coding.

[0013] As a preferred technical solution, the process of reconstructing data that is completely consistent with the original sensing data using the lossless compressed bitstream includes the following steps: Based on the decoding auxiliary information in the lossless compressed bitstream, the same preprocessing rules, context construction rules and large model probability prediction module as the encoder are used to obtain the probability distribution symbol by symbol and perform entropy decoding to recover the data sequence to be encoded. The inverse processing operation corresponding to the preprocessing at the encoding end is performed on the recovered data sequence to reconstruct the original sensing data.

[0014] As a preferred technical solution, the data symbol to be encoded corresponds to a preset symbol set, which includes binary bit symbols, 8-bit byte symbols, 16-bit integer symbols, 32-bit integer symbols, or symbols converted by reversible mapping. The reversible mapping is performed in the same way at the encoding and decoding ends, with forward conversion and reverse recovery performed according to the same rules.

[0015] Another aspect of the present invention provides a lossless compression system for distributed fiber optic acoustic wave sensor data based on probability prediction, used to implement the aforementioned lossless compression method for distributed fiber optic acoustic wave sensor data, the system comprising: The data acquisition module is used to acquire the raw sensing data collected by the distributed fiber optic acoustic wave sensing system. The data preprocessing module is used to preprocess the raw sensor data to obtain a data sequence to be encoded; The large model probability prediction module is used to predict the probability distribution of the current data symbol to be encoded based on context information constructed from encoded data and neighborhood data. The entropy coding module is used to entropy code the current data symbol to be encoded according to the probability distribution to generate a lossless compressed bitstream; The bitstream storage module is used to store or transmit the lossless compressed bitstream and decoding auxiliary information; The lossless decoding and reconstruction module is used to recover the data sequence to be encoded from the lossless compressed bitstream based on the large model probability prediction process and entropy decoding process consistent with the encoding end, and to reconstruct data that is completely consistent with the original sensing data.

[0016] In another aspect, the present invention provides a lossless compression terminal for distributed fiber optic acoustic wave sensor data based on large model probability prediction, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the aforementioned lossless compression method for distributed fiber optic acoustic wave sensor data.

[0017] In another aspect, the present invention provides a computer-readable storage medium including one or more programs executable by one or more processors of an electronic device, said one or more programs including methods for performing the aforementioned lossless compression method for distributed fiber optic acoustic sensing data.

[0018] Compared with the prior art, the present invention has at least one of the following beneficial effects: (1) Improve lossless compression ratio: Existing technologies use fixed discrete cosine transform and fixed second-order linear predictive coding as data decorrelation methods, which cannot fully capture the complex and non-stationary statistical dependencies of distributed optical fiber sensing data in the time and space dimensions, resulting in limited compression efficiency. Unlike existing methods, this invention introduces a large model, which is based on the spatiotemporal context information constructed from the encoded data and neighborhood data, and adaptively learns the long-range and short-range dependencies between data symbols. This breaks through the upper limit of the representation capability of fixed transform basis and linear prediction, and can more accurately model the data probability distribution, making the code stream output by entropy coding closer to the lower bound of information entropy, thereby significantly improving the lossless compression ratio.

[0019] (2) Improve the stability of compression performance under complex working conditions: Existing technologies extract the main coefficients of the frequency domain by energy percentage, which essentially relies on manually designed energy screening rules to separate the main body and details of the data. This has poor adaptability to non-stationary transient signals such as sudden vibrations and shocks. The difference components after screening may still have a large redundancy. This invention abandons the decomposition method of energy threshold truncation and directly uses a large model to predict the precise conditional probability of the current symbol based on the history and neighborhood context on the original data or the data after reversible difference / byte rearrangement. This avoids compression redundancy caused by unreasonable manual design transformation and truncation criteria. For DAS data containing non-stationary signals such as transient shocks and weak vibration precursors, the probability prediction is still accurate, ensuring the robustness of compression performance under different working conditions.

[0020] (3) Reduced bitstream output: Existing technologies mainly perform discrete cosine transform and linear prediction on one-dimensional raw data, without specifically designing for the unique two-dimensional spatiotemporal structure and multi-byte data type of distributed optical fiber sensing data. In the preprocessing process, this invention introduces data segmentation, differential processing and byte rearrangement, and learns the spatiotemporal context and correlation within multi-byte data by a large model. It can deeply mine data redundancy from three dimensions: spatial dimension, time dimension and data bit width dimension, and incorporate multi-dimensional correlation into the probability prediction model. Compared with the existing technology that only processes one-dimensional sequences, it further reduces bitstream output. Attached Figure Description

[0021] Figure 1 This is a flowchart of the lossless compression method for distributed fiber optic acoustic wave sensor data based on probability prediction in the embodiment. Figure 2 This is a schematic diagram of large model probability prediction and entropy coding in the embodiment; Figure 3 This is a schematic diagram of lossless decoding and reconstruction in the embodiment; Figure 4 This is a schematic diagram of the lossless compression system for distributed fiber optic acoustic wave sensor data in the embodiment. Figure 5 This is a schematic diagram of a lossless compression terminal for distributed fiber optic acoustic sensor data based on large model probability prediction in the embodiment. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0023] Example 1 To address the aforementioned problems in the existing technology, this embodiment provides a lossless compression method for distributed fiber optic acoustic wave sensor data based on probability prediction. This method aims to overcome the shortcomings of existing technologies, such as the large amount of distributed fiber optic acoustic wave sensor data, the difficulty of traditional lossless compression methods in fully utilizing the spatiotemporal correlation of sensor data, and the difficulty of deep learning reconstruction methods in ensuring complete lossless data recovery.

[0024] The method includes the following steps: Step 1: Acquire the raw sensor data (DAS data) collected by the distributed fiber optic acoustic wave sensing system.

[0025] The raw sensing data includes one-dimensional time-series data or two-dimensional waterfall plot data continuously acquired along the fiber optic distance and time directions. Distributed fiber optic acoustic wave sensing systems include, but are not limited to, phase-sensitive optical time-domain reflectometer systems, distributed acoustic wave sensing systems, and other distributed fiber optic sensing systems that detect vibration based on fiber optic backscattered signals.

[0026] Step 2: Preprocess the raw sensor data to obtain the data sequence to be encoded.

[0027] Preprocessing includes one or more of the following: data segmentation, format conversion, differential processing, quantization mapping, byte rearrangement, and context construction. Through the preprocessing process, the raw sensor data is transformed into a data sequence suitable for large-scale model probabilistic prediction and entropy coding.

[0028] Data segmentation is used to divide continuously acquired distributed fiber optic acoustic wave sensing data into several data blocks to be compressed. These data blocks can be arranged continuously along the time direction, along the fiber optic distance, or form a two-dimensional waterfall plot data block composed of both time and spatial directions. Different data blocks can be compressed independently or jointly using context information between adjacent data blocks.

[0029] Differential processing leverages the temporal continuity and spatial correlation of distributed fiber optic acoustic sensing data to transform raw data into residual data. Differential processing includes one or more of the following: temporal differential, spatial differential, two-dimensional joint differential, inter-block differential, or prediction residual calculation. By using differential processing, the dynamic range of the data sequence to be encoded can be reduced, improving the prediction accuracy of the large model probability prediction module for the current data symbol probability distribution.

[0030] Byte rearrangement is used to rearrange different byte bits according to the data storage format of the original sensor data. For multi-byte integer data, floating-point data, or phase demodulated data, the high-order byte, middle-order byte, low-order byte, or sign bit, exponent bit, and mantissa bit can be rearranged according to preset rules to make adjacent symbols have stronger statistical correlation, thereby improving the efficiency of subsequent probability prediction and entropy coding.

[0031] Step 3: Construct context information based on the encoded data and neighborhood data, and use the large model probability prediction module to predict the probability distribution of the current data symbol to be encoded.

[0032] Contextual information includes one or more of the following: data symbols already encoded before the current data symbol to be encoded, historical temporal sampling data at the same optical fiber spatial location, adjacent spatial location data sampled at the same time, statistical information of adjacent data blocks, temporal location information, and spatial location information. Based on the input contextual information, the large-model probability prediction module learns the correlation between distributed optical fiber acoustic sensing data in the temporal and spatial dimensions and outputs the probability distribution of the current data symbol to be encoded on a preset symbol set.

[0033] The large-scale probabilistic prediction module includes probabilistic prediction models based on Transformer architecture, state-space model architecture, convolutional neural network architecture, recurrent neural network architecture, hybrid expert model architecture, or combinations thereof. The input to the large-scale probabilistic prediction module is a sequence of symbols, residual sequences, byte sequences, data block feature sequences, or combinations thereof, composed of context information. The output is the conditional probability distribution of the current data symbol to be encoded on a preset symbol set.

[0034] The preset symbol set includes binary bit symbols, 8-bit byte symbols, 16-bit integer symbols, 24-bit integer symbols, 32-bit integer symbols, symbols obtained by reversibly mapping floating-point data, or symbols obtained by reversibly mapping residual data. Reversible mapping refers to a mapping method in which forward conversion and reverse recovery can be performed according to the same rules at both the encoding and decoding ends, so as to ensure that the decoding result is consistent with the original data.

[0035] During training, the large-scale probabilistic prediction module uses historical data from distributed fiber optic acoustic sensors as training samples and the true value of the current data symbol to be predicted as the supervision signal. It optimizes model parameters by minimizing negative log-likelihood loss, cross-entropy loss, or code length estimation loss, enabling the model to output a more accurate data symbol probability distribution based on contextual information. After training, the encoder and decoder use the same model parameters for probability prediction.

[0036] Step 4: Perform entropy encoding on the current data symbols to be encoded according to the probability distribution to generate a lossless compressed bitstream.

[0037] Entropy coding includes one or more of arithmetic coding, interval coding, ANS coding, Huffman coding, or context-adaptive entropy coding. The encoder sequentially encodes the data symbols in the data sequence to be encoded based on the probability distribution output by the large model probability prediction module, generating a lossless compressed bitstream. The lossless compressed bitstream also includes one or more decoding auxiliary information such as data block size, data type, preprocessing method, context construction rules, model identifier, encoding method, data block index, and checksum information.

[0038] The entropy coding process does not perform lossy replacement or approximate reconstruction of the current data symbols to be encoded. Instead, it reversibly encodes the real data symbols based on the probability distribution output by the large model probability prediction module. The large model probability prediction module provides the probability distribution, and the entropy coding module generates the reversible bitstream. Together, they achieve lossless compression of the original sensor data.

[0039] Step 5: At the decoding end, the original sensor data is reconstructed based on the large model probability prediction process and entropy decoding process consistent with those at the encoding end.

[0040] Based on the decoding auxiliary information in the lossless compressed bitstream, the decoding end employs the same preprocessing rules, context construction rules, and large model probability prediction module as the encoding end to obtain the probability distribution of the current data symbol to be decoded symbol by symbol. Then, based on the probability distribution, it performs entropy decoding on the lossless compressed bitstream to recover the data sequence to be encoded. Subsequently, the decoding end performs inverse byte rearrangement, inverse quantization mapping, inverse differential processing, and data block concatenation operations corresponding to the preprocessing process at the encoding end, thereby reconstructing a data result that is completely consistent with the original distributed fiber optic acoustic wave sensing data.

[0041] The large model probability prediction module, context construction rules, symbol encoding order, preprocessing parameters, and entropy encoding rules are kept consistent between the encoder and decoder. When recovering data symbol by symbol, the decoder updates the context information with the already decoded data symbols and uses it for probability prediction of subsequent data symbols to be decoded, thereby achieving an autoregressive decoding process synchronized with the encoder.

[0042] The following uses two-dimensional waterfall plot data acquired by a distributed fiber optic acoustic wave sensing system as an example to illustrate the specific implementation process of this invention. The two-dimensional waterfall plot data includes a time dimension and a spatial dimension. The time dimension represents continuous sampling moments, and the spatial dimension represents different sampling positions distributed along the sensing fiber. Each data point corresponds to the vibration response, phase change, amplitude information, or other demodulated sensing data at a specific time and spatial location.

[0043] See Figure 1 The lossless compression method in this embodiment mainly includes: S1, Acquisition of raw DAS data.

[0044] The raw DAS data is acquired from the distributed optical fiber acoustic wave sensing system. The raw DAS data can be one-dimensional continuous time-series data or two-dimensional waterfall plot data composed of time and spatial dimensions. For example, in one embodiment, the raw DAS data is represented as a matrix. ,in Indicates the number of time sampling points. The elements in the matrix represent the number of spatial sampling points along the fiber optic distance. Indicates the first The time sampling point, the first The sensor data values ​​are located at each spatial sampling point. The sensor data values ​​can be integer sample values, fixed-point data, floating-point data, phase demodulated data, intensity data, or data after reversible transformation.

[0045] S2, Data Preprocessing.

[0046] The raw DAS data is preprocessed to obtain the data sequence to be encoded. Specifically, the raw two-dimensional waterfall plot data can be divided into multiple data blocks according to a preset size. For example, the raw matrix can be divided into several time blocks according to the time direction, or into several spatial blocks according to the spatial direction, or it can be divided into two-dimensional data blocks simultaneously along both the time and spatial directions. Each data block can be compressed independently, or it can participate in the context construction together with adjacent data blocks.

[0047] In this embodiment, taking a two-dimensional data block as an example, the original waterfall plot data is divided into multiple blocks of size. Data blocks, in which This indicates the amount of time-sampled data contained in each data block. This indicates the number of spatial sampling points contained in each data block. For each data block, the format is first converted according to the data type, transforming the original sampled values ​​into an integer sequence or byte sequence suitable for symbol encoding. If the original data is floating-point data, it can be directly converted into a byte sequence according to the IEEE floating-point format, or a reversible mapping method can be used to map the floating-point data into an integer symbol sequence to ensure that the original data can be completely recovered later.

[0048] Furthermore, for multi-byte data, byte rearrangement can be performed. For example, for 16-bit, 24-bit, or 32-bit integer sample values, the high-order byte, middle-order byte, and low-order byte of each sample value can be arranged separately to form sequences to be encoded in different byte planes; alternatively, sequences to be encoded can be generated according to the original storage order, low-order priority, high-order priority, or interleaving order. Byte rearrangement can enhance the statistical correlation between adjacent symbols, making it easier for the large model's probability prediction module to learn patterns in the sequence to be encoded.

[0049] S3, Context Construction.

[0050] Reference Figure 2 This constructs the context input for the large model probability prediction module. For the current symbol to be encoded in the data sequence to be encoded... The encoding end selects one or more of the following as context information: previously encoded historical symbols, symbols at adjacent positions within the same data block, data from adjacent temporal sampling positions, data from adjacent spatial sampling positions, data block index, temporal location information, and spatial location information. The context information is concatenated into a context sequence according to a preset order and input into the large model probability prediction module.

[0051] In one embodiment, the large-model probability prediction module employs a Transformer-based autoregressive probability prediction network. The input sequence is first mapped to a high-dimensional feature representation through an embedding layer, and then temporal, spatial, and data block positional encodings are added. This is then input into a multi-layer Transformer structure for context modeling. The large-model probability prediction module outputs the conditional probability distribution of the current symbol to be encoded on a preset symbol set. For example, when the symbol to be encoded is an 8-bit byte, the preset symbol set contains 256 possible values ​​from 0 to 255. The large-model probability prediction module outputs a probability vector of length 256, representing the probability of the current byte to be encoded taking each value.

[0052] In other embodiments, the large model probability prediction module may also employ a state-space model, a recurrent neural network, a convolutional neural network, a hybrid expert model, or a combination of the above models. As long as the model can output the probability distribution of the current symbol to be encoded based on context information, it can serve as the large model probability prediction module in this invention.

[0053] S4, Large Model Probability Prediction.

[0054] Reference Figure 2 Based on the probability distribution output by the large model probability prediction module, entropy encoding is performed on the current real symbol to generate a lossless compressed bitstream. It should be noted that the large model in this invention does not directly generate replacement data, nor does it directly output reconstruction results; rather, it is only used to predict the probability distribution of the current symbol to be encoded. The actual value of the current symbol to be encoded is still fed into the entropy encoder, which performs reversible encoding on the real symbol according to the probability distribution.

[0055] Specifically, the encoding end can employ one or more of the following methods: arithmetic coding, interval coding, ANS coding, Huffman coding, or context-adaptive entropy coding. In one embodiment, an arithmetic encoder is used to encode the symbols to be encoded. For each symbol to be encoded, the large model probability prediction module outputs its probability distribution, the arithmetic encoder updates the encoding interval according to this probability distribution, and writes the encoding result into the compressed bitstream. For symbols with a high probability of occurrence, the arithmetic encoder assigns a shorter average code length; for symbols with a low probability of occurrence, it assigns a longer average code length, thereby achieving a compression effect close to information entropy.

[0056] During the encoding process, the encoder also writes necessary decoding auxiliary information into the lossless compressed bitstream or stores it in association with the bitstream. This decoding auxiliary information includes, but is not limited to, data block size, original data type, preprocessing method, differential rules, byte rearrangement method, context construction rules, symbol encoding order, large model identifier, model parameter version, entropy encoding method, data block index, and verification information. This decoding auxiliary information ensures that the decoder can perform probability prediction and entropy decoding in a manner consistent with the encoder.

[0057] S5 uses entropy coding to generate a lossless compressed bitstream.

[0058] Reference Figure 3 At the decoding end, the lossless compressed bitstream is decoded and reconstructed. The decoding end first reads the lossless compressed bitstream and its decoding auxiliary information, and determines the data block size, data type, preprocessing method, context construction rules, large model probability prediction module, and entropy decoding method based on the decoding auxiliary information. Subsequently, the decoding end performs symbol-by-symbol decoding in the same symbol order as the encoding end.

[0059] For the current symbol to be decoded, the decoder constructs the same context information as the encoder based on the previously decoded historical symbols and their corresponding time, spatial, and data block positions. This context information is then input into a large-model probability prediction module with parameters identical to those of the encoder to obtain the probability distribution of the current symbol. The entropy decoder decodes the current symbol from the compressed bitstream based on this probability distribution and adds it to the sequence of decoded symbols for context construction of subsequent symbols. This process is repeated until the sequence of data to be encoded for the current data block or all data blocks is completely recovered.

[0060] S6, lossless reconstruction at the decoding end.

[0061] The recovered data sequence to be encoded undergoes inverse preprocessing to reconstruct the original DAS data. Specifically, if the encoding end performs byte rearrangement, the decoding end performs inverse byte rearrangement according to the same rules, restoring multiple byte planes or rearranged byte sequences to the original byte order; if the encoding end performs data block segmentation, the decoding end splices the data blocks according to the data block index and data block size, ultimately reconstructing distributed fiber acoustic wave sensing data that is completely consistent with the original DAS data.

[0062] To verify the effectiveness of the lossless compression method for distributed fiber optic acoustic wave sensing data based on large model probability prediction provided in the above embodiments, this embodiment uses a publicly available distributed fiber optic acoustic wave sensing event dataset for experimentation. The dataset includes data from six typical events: background noise, digging, knocking, watering, shaking railings, and walking. All event data are stored in the form of a distributed fiber optic acoustic wave sensing waterfall plot, which reflects the data distribution characteristics of different vibration events in both time and space dimensions.

[0063] During the experiment, the traditional general lossless compression method gzip and the lossless compression method based on large model probability prediction in this embodiment were used to compress the data of the above six types of events. For each type of event, the original waterfall plot data was used as input, and the data size before compression and the bitstream size after compression were calculated respectively. The compression ratio was then calculated based on both. The compression ratio can be expressed as: in, Indicates the original data size before compression. This indicates the size of the compressed data. A higher compression ratio indicates better compression. After compression, the compressed bitstream is further decoded, and the decoded data is compared point-by-point with the original data before compression to verify the lossless nature of the compression process.

[0064] Experimental results show that the large-model lossless compression method proposed in this invention can achieve lossless compression and decoding reconstruction for six event categories: background noise, digging, knocking, watering, shaking railings, and walking. The decoding results are completely consistent with the original distributed fiber optic acoustic wave sensing data. Compared with the gzip method, the method of this invention achieves a higher compression ratio for most event categories, indicating that the large-model probability prediction module can effectively learn the correlation of distributed fiber optic acoustic wave sensing data in the temporal and spatial dimensions, and provide a more accurate symbol probability distribution for the entropy coding process. Specific experimental results are shown in Table 1: Table 1 Experimental Results As can be seen from Table 1, compared with the gzip method which mainly relies on general dictionary matching and repetitive pattern compression, the method in this embodiment can make probabilistic predictions on the current symbols to be encoded based on context information. Therefore, it is more suitable for handling the temporal correlation, spatial correlation and event local sparsity that are common in DAS data.

[0065] Example 2 Based on Example 1, this example provides a lossless compression system for distributed fiber optic acoustic wave sensor data based on probability prediction, used to implement the lossless compression method for distributed fiber optic acoustic wave sensor data in Example 1. (See also...) Figure 4 The system includes a data acquisition module, a data preprocessing module, a large model probability prediction module, an entropy coding module, a code stream storage module, and a lossless decoding and reconstruction module.

[0066] The data acquisition module acquires the raw sensor data collected by the distributed fiber optic acoustic sensing system; the data preprocessing module performs data segmentation, format conversion, differential processing, quantization mapping, byte rearrangement, or context construction on the raw sensor data to obtain the data sequence to be encoded; the large model probability prediction module outputs the probability distribution of the current data symbol to be encoded based on context information; the entropy encoding module performs entropy encoding on the current data symbol to be encoded based on the probability distribution to generate a lossless compressed bitstream; the bitstream storage module stores or transmits the lossless compressed bitstream and decoding auxiliary information; and the lossless decoding and reconstruction module recovers the data sequence to be encoded from the lossless compressed bitstream based on the large model probability prediction process and entropy decoding process consistent with the encoding end, and reconstructs a data result that is completely consistent with the original sensor data.

[0067] This embodiment provides an end-to-end joint framework combining large-model probabilistic prediction and entropy coding. The large-model probabilistic prediction module is pre-trained on a large amount of historical DAS data by minimizing the negative log-likelihood loss. The encoder and decoder use the learned model synchronously, realizing data-driven prediction and coding co-optimization. The model can learn statistical laws from massive amounts of data that cannot be exhaustively enumerated by human design, and has the ability to adapt and evolve to specific sensing scenarios. The compression performance is continuously improved as the model is optimized.

[0068] Example 3 Based on the foregoing embodiments, this embodiment provides a lossless compression terminal for distributed fiber optic acoustic sensor data based on large model probability prediction. See [link to relevant documentation]. Figure 5 It includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and a lossless compression method for distributed fiber optic acoustic wave sensing data based on large model probability prediction is used by the processor to execute the program.

[0069] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects: This invention utilizes a large-model probabilistic prediction module to learn the statistical correlations between distributed fiber optic acoustic wave sensor (DAS) data in the time, spatial, and data block dimensions. It then predicts the probability distribution of the current data symbol to be encoded based on contextual information, and combines this with entropy coding to generate a lossless compressed bitstream. Compared to traditional general lossless compression methods, this invention can more fully utilize the spatiotemporal correlations, background noise stability, and sparsity of local vibration events in DAS data, thereby improving compression efficiency.

[0070] This invention does not directly reconstruct the original data using a deep learning model. Instead, it utilizes a large model to provide a probability distribution and achieves reversible compression through entropy encoding and decoding. The encoding and decoding ends employ a consistent large model probability prediction process, context construction rules, and entropy encoding rules, ensuring that the decompressed data is completely consistent with the original distributed fiber optic acoustic wave sensing data before compression. This meets the requirements for the integrity of the original data in engineering monitoring, fault tracing, event identification, and subsequent precise analysis.

[0071] This invention is applicable to distributed fiber optic acoustic sensing systems with long distances, high sampling rates, and continuous operation. It can reduce the storage footprint and transmission bandwidth requirements of raw data, improve data management efficiency in edge deployment, remote transmission, and long-term online monitoring scenarios, and has good engineering application value.

[0072] Example 4 Based on the foregoing embodiments, this embodiment provides a computer-readable storage medium including one or more programs executable by one or more processors of an electronic device, the one or more programs including a method for performing lossless compression of distributed fiber optic acoustic wave sensing data according to embodiment 1.

[0073] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0074] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A lossless compression method for distributed fiber optic acoustic wave sensor data based on probabilistic prediction, characterized in that, Includes the following steps: Acquire raw sensing data collected by the distributed fiber optic acoustic wave sensing system; The original sensing data is preprocessed to obtain a data sequence to be encoded; Contextual information is constructed based on encoded data and neighborhood data, and a large model is used to predict the probability distribution of the current data symbol to be encoded. At the encoding end, entropy encoding is performed on the current data symbol to be encoded according to the probability distribution to generate a lossless compressed bitstream; At the decoding end, based on the large model probability prediction process and entropy decoding process consistent with the encoding end, data completely consistent with the original sensing data is reconstructed using the lossless compressed bitstream.

2. The lossless compression method for distributed fiber optic acoustic wave sensor data based on probabilistic prediction according to claim 1, characterized in that, The preprocessing includes at least one of data segmentation, differential processing, and byte rearrangement, wherein: The data block division is used to divide the raw sensing data into data blocks to be compressed along the time direction and / or spatial direction. The differential processing is used to convert the original data into residual data by utilizing the temporal continuity and spatial correlation of the data. The byte rearrangement is used to rearrange different byte positions of multi-byte data according to the data storage format in order to enhance the statistical correlation between adjacent symbols.

3. The lossless compression method for distributed fiber optic acoustic wave sensor data based on probabilistic prediction according to claim 1, characterized in that, The context information includes at least one of the following: The data symbols that have been encoded before the data symbol to be encoded, historical time-sampling data at the same optical fiber spatial location, adjacent spatial location data sampled at the same time, statistical information of adjacent data blocks, time location information and spatial location information.

4. The lossless compression method for distributed fiber optic acoustic wave sensor data based on probabilistic prediction according to claim 1, characterized in that, The large model includes probabilistic prediction models based on Transformer structures, state-space model structures, convolutional neural network structures, recurrent neural network structures, or combinations thereof.

5. The lossless compression method for distributed fiber optic acoustic wave sensor data based on probabilistic prediction according to claim 1, characterized in that, The entropy coding includes at least one of arithmetic coding, interval coding, ANS coding, Huffman coding, or context-adaptive entropy coding.

6. The lossless compression method for distributed fiber optic acoustic wave sensor data based on probabilistic prediction according to claim 1, characterized in that, The process of reconstructing data that is completely consistent with the original sensing data using the lossless compressed bitstream includes the following steps: Based on the decoding auxiliary information in the lossless compressed bitstream, the same preprocessing rules, context construction rules and large model probability prediction module as the encoder are used to obtain the probability distribution symbol by symbol and perform entropy decoding to recover the data sequence to be encoded. The inverse processing operation corresponding to the preprocessing at the encoding end is performed on the recovered data sequence to reconstruct the original sensing data.

7. The lossless compression method for distributed fiber optic acoustic wave sensor data based on probabilistic prediction according to claim 1, characterized in that, The data symbol to be encoded corresponds to a preset symbol set, which includes binary bit symbols, 8-bit byte symbols, 16-bit integer symbols, 32-bit integer symbols, or symbols converted by reversible mapping. The reversible mapping is performed by forward conversion and reverse recovery at the encoding and decoding ends according to the same rules.

8. A lossless compression system for distributed fiber optic acoustic sensor data based on probabilistic prediction, characterized in that, For implementing the lossless compression method for distributed fiber optic acoustic wave sensing data as described in any one of claims 1-7, the system comprises: The data acquisition module is used to acquire the raw sensing data collected by the distributed fiber optic acoustic wave sensing system. The data preprocessing module is used to preprocess the raw sensor data to obtain a data sequence to be encoded; The large model probability prediction module is used to predict the probability distribution of the current data symbol to be encoded based on context information constructed from encoded data and neighborhood data. The entropy coding module is used to entropy code the current data symbol to be encoded according to the probability distribution to generate a lossless compressed bitstream; The bitstream storage module is used to store or transmit the lossless compressed bitstream and decoding auxiliary information; The lossless decoding and reconstruction module is used to recover the data sequence to be encoded from the lossless compressed bitstream based on the large model probability prediction process and entropy decoding process consistent with the encoding end, and to reconstruct data that is completely consistent with the original sensing data.

9. A lossless compression terminal for distributed fiber optic acoustic sensor data based on large model probabilistic prediction, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the lossless compression method for distributed fiber optic acoustic wave sensing data as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, Includes one or more programs that are executed by one or more processors of an electronic device, the one or more programs including methods for performing lossless compression of distributed fiber optic acoustic sensing data as described in any one of claims 1-7.

Images