Structured storage method, query method and device of optical fiber sensing data

Abstract

The invention provides a structured storage method, query method and device for optical fiber sensing data. The structured storage method comprises the following steps: constructing an original two-dimensional sensing data matrix based on original optical fiber sensing data; optical fiber sensing data in the original two-dimensional sensing data matrix is floating point type data; grouping the original two-dimensional sensing data matrix according to a preset duration to obtain a plurality of original sensing data matrix blocks; scaling and rounding each original sensing data matrix block to obtain a first sensing data matrix block, the optical fiber sensing data in the first sensing data matrix block being integer data; binding the first sensing data matrix block with the metadata of the first sensing data matrix block to generate an independent data file; and classifying the plurality of independent data files based on the metadata to obtain a plurality of types of data file groups, storing the various types of data file groups in the same directory, and creating a main file for the directory. According to the invention, the pressure of storage hardware is reduced, the time consumption of full-amount decompression is avoided, and efficient data query is realized.

Description

Technical Field

[0001] This invention relates to the field of fiber optic sensing technology, and specifically to a structured storage method, query method, and apparatus for fiber optic sensing data. Background Technology

[0002] Fiber optic sensing technology, with its advantages of high sensitivity, resistance to electromagnetic interference, and distributed measurement, has been widely used in large-scale engineering fields such as infrastructure health monitoring, perimeter security, and oil and gas pipeline leak detection. Typical distributed fiber optic sensing systems (such as DAS and DTS) achieve real-time, continuous monitoring of physical fields (such as vibration, temperature, and strain) by implementing tens of thousands or even hundreds of thousands of virtual sensing points on continuous optical fibers and sampling at high frequencies (kHz to MHz). This high-density, high-frequency sampling mode leads to an explosive growth in data volume. For example, a system containing P145 sensing points with a sampling rate of 1kHz can generate terabytes of raw data per day. For a long time, the massive amounts of sensor data generated by these systems have typically been stored directly as continuous binary streams or simple segmented files, with data formats mostly high-precision floating-point (such as float32) to ensure the integrity of information such as phase. However, this storage method faces the following prominent bottlenecks: 1. High Storage Costs: Raw floating-point data occupies a large amount of space, requiring enormous storage resources over long periods, resulting in extremely high maintenance costs. While general compression algorithms such as ZIP and GZIP can be used for data compression, their compression efficiency is not ideal for high-precision floating-point sensor data. Furthermore, the compressed data format changes, requiring full decompression before any reading or analysis can be performed, compromising direct data accessibility and hindering efficient random queries. 2. Chaotic Data Management: Massive data files lack effective organizational structure and self-descriptive information. File naming is arbitrary, and metadata (such as device, channel, and time) is stored separately from or incompletely recorded with the sensor data itself, leading to difficulties in data traceability and easily causing management chaos. 3. Low Query and Analysis Efficiency: When analyzing data from a specific time period or a specific sensor within historical data, traditional methods often require traversing or decompressing the entire massive dataset, a time-consuming and labor-intensive process that cannot meet the demands for rapid response and interactive analysis.

[0003] Therefore, there is an urgent need to provide a structured storage method, query method, and device for fiber optic sensor data that can significantly reduce data storage volume while maintaining the self-descriptive nature and internal structure of the data, and support rapid location and access to sensor data, thus facilitating in-depth mining of long-term sensor monitoring data. Summary of the Invention

[0004] In view of this, it is necessary to provide a structured storage method, query method and device for fiber optic sensing data, so as to solve the technical problems of excessive data storage volume and inability to quickly locate and access sensing data in the prior art.

[0005] To address the aforementioned technical problems, in a first aspect, the present invention provides a structured storage method for fiber optic sensing data, comprising: A two-dimensional sensor data matrix is ​​constructed based on the raw fiber optic sensor data collected by the fiber optic sensor system, with sensor number as the row and frame number as the column; the fiber optic sensor data in the raw two-dimensional sensor data matrix is ​​floating-point data. The original two-dimensional sensor data matrix is ​​grouped according to a preset time period to obtain multiple original sensor data matrix blocks; Each of the original sensing data matrix blocks is scaled and rounded to obtain a first sensing data matrix block, wherein the fiber optic sensing data in the first sensing data matrix block is integer data. The first sensing data matrix block is bound to metadata describing the source and content of the first sensing data matrix block to generate an independent data file with a self-describing file name; Based on the metadata, multiple independent data files are categorized to obtain multiple data file groups, and each type of data file group is stored in the same directory, and a main file is created for the directory; The main file includes global metadata for all data file groups in the directory and a file index table. The file index table includes the identifier, time range, and storage location of each independent data file.

[0006] In one possible implementation, before scaling and rounding each of the original sensor data matrix blocks to obtain the first sensor data matrix block, the method further includes: Extract the reference matrix from each of the original sensor data matrix blocks; Subtract the fiber optic sensing data in the reference matrix from each row of fiber optic sensing data in the original sensing data matrix block to obtain the difference matrix block. The step of scaling and rounding each of the original sensor data matrix blocks to obtain the first sensor data matrix block includes: The scaling and rounding processes are applied to the difference matrix block to obtain the first sensing data matrix block.

[0007] In one possible implementation, the method further includes: A scaling factor is determined based on the phase error threshold allowed by the fiber optic sensing system, and the differential matrix block is scaled based on the scaling factor.

[0008] In one possible implementation, the metadata includes at least one of device identifier, channel identifier, number of sensors, and data timestamp range, and the self-describing file name is generated based on the metadata.

[0009] In one possible implementation, classifying multiple independent data files based on the metadata includes: Independent data files with the same device identifier and channel identifier are grouped into the same type of data file group.

[0010] In one possible implementation, the global metadata includes hardware configuration parameters and acquisition parameters, wherein the hardware configuration parameters include the laser wavelength and the effective refractive index of the optical fiber, and the acquisition parameters include the system sampling rate and the total number of sensing channels.

[0011] In one possible implementation, the storage location in the file index table is the relative or absolute path of the independent data file in the directory.

[0012] Secondly, the present invention also provides a method for querying fiber optic sensing data, comprising: In response to a data query request that includes a target time range, the file index table in the main file is parsed to determine the independent data file corresponding to the target time range; The independent data file is read based on the storage location in the file index table, and the independent data file is decompressed to obtain the original fiber optic sensing data. The file index table and the independent data file are generated based on the structured storage method for fiber optic sensing data described in any of the above possible implementations.

[0013] Thirdly, the present invention also provides a structured storage device for fiber optic sensing data, comprising: The raw two-dimensional sensor data matrix construction unit is used to construct a raw two-dimensional sensor data matrix based on the raw fiber optic sensor data collected by the fiber optic sensor system, with the sensor number as the row and the frame number as the column; the fiber optic sensor data in the raw two-dimensional sensor data matrix is ​​floating-point data. A two-dimensional sensor data matrix grouping unit is used to group the original two-dimensional sensor data matrix according to a preset time period to obtain multiple original sensor data matrix blocks. The scaling and rounding unit is used to scale and round each of the original sensing data matrix blocks to obtain a first sensing data matrix block, wherein the fiber optic sensing data in the first sensing data matrix block is integer data. An independent data file generation unit is used to bind the first sensing data matrix block with metadata describing the source and content of the first sensing data matrix block to generate an independent data file with a self-describing file name. The main file generation unit is used to classify multiple independent data files based on the metadata, obtain multiple data file groups, store the various data file groups in the same directory, and create a main file for the directory; The main file includes global metadata for all data file groups in the directory and a file index table. The file index table includes the identifier, time range, and storage location of each independent data file.

[0014] Fourthly, the present invention also provides a device for querying fiber optic sensor data, comprising: An independent data file location unit is used to respond to a data query request that includes a target time range, parse the file index table in the main file, and determine the independent data file corresponding to the target time range. An independent data file location and decompression unit is used to read the independent data file based on the storage location in the file index table, and decompress the independent data file to obtain the original fiber optic sensing data. The file index table and the independent data file are generated based on the structured storage method for fiber optic sensing data described in any of the above possible implementations.

[0015] The beneficial effects of this invention are: the structured storage method for fiber optic sensing data provided by this invention scales and rounds each original sensing data matrix block, converting floating-point fiber optic sensing data into integer data. Since the storage bit width of integer data is smaller than that of floating-point data, the scaling and rounding process compresses the fiber optic sensing data, reducing the hardware pressure on the storage of fiber optic sensing data.

[0016] Furthermore, this invention first divides the continuous raw two-dimensional sensor data matrix into data blocks along the time dimension according to a preset time interval. Then, each data block is bound to metadata describing its source and content. Finally, the independent data files are categorized, and a main file containing global metadata and a file index table is created for each, forming a structure of data block-independent file-main file index. This makes the massive amount of raw sensor data well-organized, greatly improving the scientific nature and convenience of data management. Moreover, when querying data, the system does not need to read or decompress the entire dataset. It only needs to parse the lightweight main file first, and then, like looking up a directory, it can instantly and accurately locate the specific independent data file. Subsequently, only that independent data file is decompressed and accessed. This completely solves the efficiency bottleneck of full-volume reading in traditional compression methods or raw data storage, making millisecond-level historical data retrieval possible and greatly improving the efficiency of subsequent data analysis. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A schematic flowchart of an embodiment of the structured storage method for fiber optic sensing data provided by the present invention; Figure 2 This is a schematic flowchart of an embodiment of the present invention for differential processing of raw sensor data matrix blocks; Figure 3 A schematic diagram illustrating an embodiment of the main file and independent data files provided by the present invention; Figure 4 A schematic flowchart of an embodiment of the fiber optic sensing data query method provided by the present invention; Figure 5 A schematic diagram of an embodiment of the structured storage device for fiber optic sensing data provided by the present invention; Figure 6 A schematic diagram of an embodiment of the fiber optic sensing data query device provided by the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0020] It should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this invention illustrate operations implemented according to some embodiments of the invention. It should be understood that the operations in the flowcharts may be implemented out of order, and steps without logical contextual relationships may be reversed or performed simultaneously. Furthermore, those skilled in the art, guided by the content of this invention, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0021] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0022] This invention provides a structured storage method, a query method, and a device for fiber optic sensing data, which will be described below.

[0023] Figure 1 This is a schematic flowchart of an embodiment of the structured storage method for fiber optic sensing data provided by the present invention, as shown below. Figure 1 As shown, the structured storage method for fiber optic sensing data includes: S101. Construct a raw two-dimensional sensing data matrix based on the raw fiber optic sensing data collected by the fiber optic sensing system, with the sensor number as the row and the frame number as the column; the fiber optic sensing data in the raw two-dimensional sensing data matrix is ​​floating-point data.

[0024] Specifically, if the fiber optic sensing system includes m sensors and the number of frames is n, then the original two-dimensional sensing data matrix is ​​an m×n matrix.

[0025] The number of bytes for floating-point data is 32 bits.

[0026] S102. Group the original two-dimensional sensor data matrix according to the preset time to obtain multiple original sensor data matrix blocks.

[0027] The preset duration can be set or adjusted according to actual needs or application scenarios. In a specific embodiment of the present invention, the preset duration is 1 minute.

[0028] S103. Scale and round each original sensor data matrix block to obtain the first sensor data matrix block. The fiber optic sensor data in the first sensor data matrix block is integer data.

[0029] Specifically, the number of bytes for integer data is 16, which means that it saves half the bit width, that is, half the storage space.

[0030] S104. Bind the first sensing data matrix block with metadata describing the source and content of the first sensing data matrix block to generate an independent data file with a self-describing file name.

[0031] In a specific embodiment of the present invention, the metadata includes at least one of device identifier, channel identifier, number of sensors, and data timestamp range, and the self-describing file name is generated based on the metadata.

[0032] Specifically, the self-describing file name can be obtained by concatenating various metadata.

[0033] For example, the self-describing file name can be D1_C2_P145_20231027_143045.wse, where D1 is the device identifier; C2 is the channel identifier; P145 is the number of sensors; 20231027143045 is the timestamp range, which represents the sensor data collected starting from 14:30:45 on October 27, 2023. The data timestamp range can be determined based on the preset duration, and wse is the file format.

[0034] S105. Based on metadata, classify multiple independent data files to obtain multiple data file groups, store all data file groups in the same directory, and create a main file for the directory; The main file includes global metadata for all data file groups in the directory and a file index table. The file index table includes the identifier, time range, and storage location of each independent data file.

[0035] In a specific embodiment of the present invention, classifying multiple independent data files based on metadata includes: Individual data files with the same device identifier and channel identifier are grouped into the same type of data file group.

[0036] For example, when the following six independent data files are included: D1_C1_P145_20231027080000.wse; D1_C1_P145_20231027080100.wse; D1_C1_P145_20231027080200.wse; D1_C2_P145_20231027080000.wse; D1_C2_P145_20231027080100.wse; D2_C1_P145_20231027080000.wse; The obtained data file group consists of three groups, namely: D1 / C1, D1 / C2, and D2 / C1, with D1 / C1 containing the following independent data files: D1_C1_P145_20231027080000.wse; D1_C1_P145_20231027080100.wse; D1_C1_P145_20231027080200.wse; The independent data files contained in D1 / C2 are: D1_C2_P145_20231027080000.wse; D1_C2_P145_20231027080100.wse; The independent data files contained in D2 / C1 are: D2_C1_P145_20231027080000.wse.

[0037] Based on this architecture of independent data files and data file groups, massive amounts of independent data files are automatically and clearly organized into a tree structure. Each directory naturally corresponds to a unique data source, enabling lightweight and highly readable data management of sensor data.

[0038] Because fiber optic sensing data is inherently relative and dimensionless, it must be combined with the hardware information that generates the signal to accurately convert it into physically meaningful quantities (such as strain, temperature, vibration acceleration, etc.) or to compare and reuse it across different systems.

[0039] Therefore, in some embodiments of the present invention, the global metadata includes: hardware configuration parameters and acquisition parameters, wherein the hardware configuration parameters include the laser wavelength and the effective refractive index of the optical fiber, and the acquisition parameters include the system sampling rate and the total number of sensing channels.

[0040] This invention, through setting global metadata including hardware configuration parameters, can convert meaningless parameters in the sensor data matrix into parameters with clear physical meaning, providing a data foundation for subsequent analysis.

[0041] It should be understood that the storage location in the file index table is the relative or absolute path of the independent data file in the directory.

[0042] It should be understood that the structured storage method for fiber optic sensor data in this embodiment of the invention can be implemented in any device based on the structured storage method for fiber optic sensor data, such as electronic devices like data storage devices or data reading devices. Specifically, the structured storage method for fiber optic sensor data is stored in the aforementioned device as a pre-programmed program. When the device is started, the program is invoked, and the structured storage method for fiber optic sensor data is implemented.

[0043] Compared with the prior art, the structured storage method for fiber optic sensing data provided in this embodiment of the invention performs scaling and rounding processing on each original sensing data matrix block, converting the floating-point fiber optic sensing data into integer data. Since the storage bit width of integer data is smaller than that of floating-point data, the scaling and rounding processing achieves compression of fiber optic sensing data, reducing the hardware pressure for storing fiber optic sensing data.

[0044] Furthermore, in this embodiment of the invention, the continuous raw two-dimensional sensor data matrix is ​​first divided into data blocks along the time dimension according to a preset time interval. Then, each data block is bound to metadata describing its source and content. Finally, the independent data files are categorized, and a main file containing global metadata and a file index table is created for each, forming a structure of data block-independent file-main file index. This makes the massive amount of raw sensor data well-organized, greatly improving the scientific nature and convenience of data management. Moreover, when performing data queries, the system does not need to read or decompress the entire dataset. It only needs to parse the lightweight main file first, and then, like looking up a directory, it can instantly and accurately locate the specific independent data file. Subsequently, only that independent data file is decompressed and accessed. This completely solves the efficiency bottleneck of full-volume reading in traditional compression methods or raw data storage, making millisecond-level historical data retrieval possible and greatly improving the efficiency of subsequent data analysis.

[0045] In practical applications, raw fiber optic sensing data is typically a floating-point number with a very large absolute value. For example, if the raw fiber optic sensing data is 10000000.123, converting it to an int16 integer would result in a value of 10000000123, far exceeding the maximum value of int16, since the maximum value of int16 is 32767. This means that a severe integer overflow can occur during the conversion from floating-point to integer. When this happens, the higher-order bits are truncated, the original signal information is completely lost, and the error is too large, making it impossible to obtain the correct raw fiber optic sensing data after subsequent data reconstruction.

[0046] To address this technical problem, in some embodiments of the present invention, such as... Figure 2 As shown, before step S103, the procedure further includes: S201. Extract the reference matrix from each original sensor data matrix block.

[0047] In a specific embodiment of the present invention, the reference matrix is ​​the first column of data in the original sensor data matrix block.

[0048] Specifically, if the original two-dimensional sensing data matrix is ​​an m×n matrix, then the reference matrix is ​​an m×1 matrix.

[0049] It should be understood that the reference matrix can also be the first row of data in the original sensor data matrix block.

[0050] S202. Subtract the fiber sensing data in the reference matrix from each row of fiber sensing data in the original sensing data matrix block to obtain the difference matrix block.

[0051] Then step S103 is specifically as follows: The difference matrix block is scaled and rounded to obtain the first sensing data matrix block.

[0052] For example, if the values ​​of the first three rows of fiber optic sensor data in a certain column are 10000000.123, 10000000.128, and 10000000.119 respectively, without differential processing, the integers would exceed the maximum value. However, after differential processing as proposed in this embodiment, the three values ​​in the first sensor data matrix block are 0.000, +0.005, and -0.004 respectively. After scaling and rounding, the resulting int16 values ​​are 0, 5, and -4, all falling within the safe range of int16 (-32768~32767). During subsequent restoration, dividing these integers by 1000 and adding the reference value allows for high-precision recovery of the original data.

[0053] In summary, the embodiments of the present invention obtain differential matrix blocks by performing differential processing on the original sensor data matrix blocks based on the reference matrix, which can ensure that no data overflow occurs during the conversion to integer data and ensure the accuracy of the subsequently restored sensor data.

[0054] To ensure the engineering usability of the subsequently restored sensor data, in some embodiments of the present invention, the structured storage method for fiber optic sensor data further includes: The scaling factor is determined based on the phase error threshold allowed by the fiber optic sensing system, and the differential matrix block is scaled based on the scaling factor.

[0055] The embodiments of the present invention determine the scaling factor based on the phase error threshold allowed by the fiber optic sensing system, which can ensure that the scaled data can meet the subsequent engineering error requirements and avoid excessive error, which would lead to poor accuracy or even unusable data after reconstruction.

[0056] In a specific embodiment of the present invention, the scaling factor is 1000.

[0057] To further illustrate the storage structure in the embodiments of the present invention, such as Figure 3 As shown, the main file is in wmdx format, and the independent data files are in wse format. Each independent data file includes multiple data blocks and corresponding self-describing filenames. Each data block consists of baseline data in float32 format and difference data in int16 format in the first line. The main file includes global metadata, a file index table, and hardware configuration parameters. The file index table includes a list of independent data files.

[0058] To verify the effectiveness of the structured storage method for fiber optic sensing data proposed in this embodiment of the invention, the storage size of the processed file in this embodiment of the invention is compared with the storage size of direct storage using binary BIN files. Specifically, using the phase signals of 145 sensors over 6 minutes as the original fiber optic sensing data, the overall space occupied by the method proposed in this embodiment of the invention is 104054 KB, while the storage size of direct storage using binary BIN files is 203910 KB, with a compression rate of 48%. This demonstrates the effectiveness of this invention in data compression and significantly saves data storage space.

[0059] Furthermore, after restoring the compressed data to the actual phase data, the difference between this data and the phase data directly stored using binary BIN files is calculated. The accuracy loss is 1‰, which is close to the calculation error and can be ignored in most application scenarios of fiber optic sensing systems, thus proving the feasibility of the structured storage method proposed in this embodiment of the invention.

[0060] In summary, the structured storage method for fiber optic sensor data proposed in this invention has the following advantages: First, it significantly improves compression efficiency. In typical vibration monitoring scenarios, compared to general compression algorithms such as ZIP and GZIP, the sensor data compression rate increases from 20% to 48%, greatly reducing storage overhead. Second, it solves the problems of chaotic dataset organization and environment-dependent parsing caused by the separation of metadata and sensor data in existing solutions. By using a file index table and independent data files, it achieves the binding storage of metadata and sensor data, ensuring the integrity of data format and layout. Simultaneously, the indexing function of the main file improves the convenience of data querying and cross-platform analysis. Third, the compression process is achieved through benchmark differential and integer quantization, resulting in only negligible loss of precision, fully meeting the needs of vibration signal phase feature analysis and avoiding the algorithm failure problem caused by precision loss in traditional compression methods.

[0061] This invention also provides a method for querying fiber optic sensing data, such as... Figure 4 As shown, the methods for querying fiber optic sensor data include: S401. Respond to a data query request that includes a target time range, parse the file index table in the main file, and determine the independent data file corresponding to the target time range; S402. Read the independent data file based on the storage location in the file index table, and decompress the independent data file to obtain the original fiber optic sensing data.

[0062] The file index table and independent data files are generated based on the structured storage method for fiber optic sensing data in any of the above embodiments.

[0063] This invention implements the binding and storage of metadata and sensor data through a file structure consisting of a file index table and independent data files, ensuring the integrity of data format and layout. At the same time, it enhances the convenience of data query and cross-platform analysis by leveraging the indexing function of the main file.

[0064] Furthermore, embodiments of the present invention also provide a structured storage device for fiber optic sensing data, such as... Figure 5 As shown, the structured storage device for fiber optic sensing data includes: The raw two-dimensional sensor data matrix construction unit 501 is used to construct a raw two-dimensional sensor data matrix based on the raw fiber optic sensor data collected by the fiber optic sensor system, with the sensor number as the row and the frame number as the column; the fiber optic sensor data in the raw two-dimensional sensor data matrix is ​​floating-point data. The two-dimensional sensor data matrix grouping unit 502 is used to group the original two-dimensional sensor data matrix according to a preset time to obtain multiple original sensor data matrix blocks. The scaling and rounding unit 503 is used to scale and round each original sensing data matrix block to obtain a first sensing data matrix block. The fiber optic sensing data in the first sensing data matrix block is integer data. Independent data file generation unit 504 is used to bind the first sensing data matrix block with metadata describing the source and content of the first sensing data matrix block to generate an independent data file with a self-describing file name; The main file generation unit 505 is used to classify multiple independent data files based on metadata, obtain multiple data file groups, store the various data file groups in the same directory, and create a main file for the directory; The main file includes global metadata for all data file groups in the directory and a file index table. The file index table includes the identifier, time range, and storage location of each independent data file.

[0065] The structured storage device 500 for optical fiber sensing data provided in the above embodiments can implement the technical solutions described in the above embodiments of the structured storage method for optical fiber sensing data. The specific implementation principles of each module or unit can be found in the corresponding content in the above embodiments of the structured storage method for optical fiber sensing data, and will not be repeated here.

[0066] Similarly, embodiments of the present invention also provide a device for querying fiber optic sensor data, such as... Figure 6 As shown, the fiber optic sensor data query device 600 includes: Independent data file location unit 601 is used to respond to data query requests including target time ranges, parse the file index table in the main file, and determine the independent data file corresponding to the target time range; The independent data file positioning and decompression unit 602 is used to read the independent data file based on the storage location in the file index table, and decompress the independent data file to obtain the original fiber optic sensing data. The file index table and independent data files are generated based on the structured storage method for fiber optic sensing data in any of the above possible implementations.

[0067] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0068] The above provides a detailed description of the structured storage method, query method, and apparatus for fiber optic sensing data provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method of structured storage of fiber optic sensor data, characterized by, The method comprises: constructing an original two-dimensional sensing data matrix with sensor number as row and frame number as column based on original fiber sensing data collected by a fiber sensing system; the fiber sensing data in the original two-dimensional sensing data matrix is floating point data; grouping the original two-dimensional sensing data matrix according to a preset time length to obtain a plurality of original sensing data matrix blocks; scaling and rounding processing the original sensing data matrix blocks to obtain a first sensing data matrix block, and the fiber sensing data in the first sensing data matrix block is integer type data; binding the first sensing data matrix block with metadata describing the source and content of the first sensing data matrix block to generate an independent data file with a self-describing file name; based on the metadata, classifying a plurality of independent data files to obtain a plurality of data file groups, and storing each data file group in the same directory, creating a master file for the directory; wherein the master file includes global metadata of all data file groups under the directory and a file index table, and the file index table includes identifiers, time ranges and storage locations of the independent data files.

2. The method of claim 1, wherein, Before the scaling and rounding processing of the original sensing data matrix blocks to obtain the first sensing data matrix block, the method further comprises: extracting a reference matrix from each original sensing data matrix block; subtracting the fiber sensing data in the reference matrix from each row of fiber sensing data in the original sensing data matrix block to obtain a difference matrix block; the scaling and rounding processing of the original sensing data matrix blocks to obtain the first sensing data matrix block comprises: scaling and rounding processing the difference matrix block to obtain the first sensing data matrix block.

3. The method of claim 1 or 2, wherein, The method further comprises: determining a scaling factor based on a phase error threshold allowed by the fiber sensing system, and scaling the difference matrix block based on the scaling factor.

4. The method of claim 1, wherein, The metadata includes at least one of device identification, channel identification, sensor quantity and data timestamp range, and the self-describing file name is generated based on the metadata.

5. The method of claim 4, wherein, The classification of a plurality of independent data files based on the metadata comprises: classifying independent data files with the same device identification and channel identification into the same data file group.

6. The method of claim 1, wherein, The global metadata includes hardware configuration parameters and acquisition parameters, the hardware configuration parameters include laser wavelength and effective refractive index of the fiber, and the acquisition parameters include system sampling rate and total number of sensing channels.

7. The method of claim 1, wherein, The storage location in the file index table is the relative path or absolute path of the independent data file in the directory.

8. A method of querying fiber optic sensor data, the method comprising: The method comprises: in response to a data query request including a target time range, analyzing the file index table in the master file to determine the independent data file corresponding to the target time range; reading the independent data file based on the storage location in the file index table, and decompressing the independent data file to obtain original fiber sensing data; the file index table and the independent data file are generated based on the structured storage method of fiber sensing data in any one of claims 1-7.

9. A structured storage device for fiber optic sensing data, characterized by, The method comprises: An original two-dimensional sensing data matrix construction unit is configured to construct an original two-dimensional sensing data matrix with sensor numbers as rows and frame numbers as columns based on original optical fiber sensing data collected by an optical fiber sensing system; The optical fiber sensing data in the original two-dimensional sensing data matrix is floating-point data; A two-dimensional sensing data matrix grouping unit is configured to group the original two-dimensional sensing data matrix according to a preset time length to obtain a plurality of original sensing data matrix blocks; A scaling and rounding unit is configured to scale and round each original sensing data matrix block to obtain a first sensing data matrix block, and the optical fiber sensing data in the first sensing data matrix block is integer data; An independent data file generation unit is configured to bind the first sensing data matrix block and metadata describing the source and content of the first sensing data matrix block to generate an independent data file with a self-describing file name; A main file generation unit is configured to classify a plurality of independent data files based on the metadata to obtain a plurality of data file groups, and store each data file group in a same directory, and create a main file for the directory; The main file includes global metadata of all data file groups under the directory and a file index table, and the file index table includes an identifier, a time range and a storage location of each independent data file.

10. An optical fiber sensing data query device, characterized by, It comprises: An independent data file positioning unit is configured to analyze the file index table in the main file in response to a data query request including a target time range, and determine an independent data file corresponding to the target time range; An independent data file positioning and decompression unit is configured to read the independent data file based on the storage location in the file index table, and decompress the independent data file to obtain original optical fiber sensing data; The file index table and the independent data file are generated based on the structured storage method of the optical fiber sensing data in any one of claims 1-7.

Images