A multi-dimensional time-series data lossy compression method based on dimension correlation grouping

Abstract

The application discloses a multi-dimensional time series data lossy compression method based on dimension correlation grouping. The method extracts dynamic trend characteristics of each dimension data, calculates dynamic error boundary of each dimension, constructs high correlation dimension grouping and determines primary and secondary dimensions in each grouping; introduces error feedback mechanism in time domain to perform differential processing on the data, generates quantized differences of each dimension data; calculates differential residuals of secondary dimensions in each group relative to the primary dimension, adjusts compression strategy according to the amplitude characteristics of the residuals, and generates a to-be-encoded set composed of a primary dimension sequence and a secondary dimension residual sequence; and performs compression on the to-be-encoded set through an entropy coding algorithm and encapsulates the to-be-encoded set. The compression strategy of the application adopts correlation grouping between dimensions, feedback quantization, residual construction between dimensions in a group and bit reorganization, and significantly improves compression density and reconstruction accuracy for large-scale sensor array data.

Description

Technical Field

[0001] This invention belongs to the field of data compression and industrial Internet of Things information processing technology, specifically relating to a technology for solving the problems of high redundancy and large storage load of large-scale multidimensional time-series data, and particularly to a lossy compression method for multidimensional time-series data based on dimensional correlation grouping. Background Technology

[0002] With the rapid development of industrial IoT, smart monitoring, and precision manufacturing technologies, sensor arrays have been widely used in the monitoring of complex systems. The large-scale, multi-dimensional time-series data generated in real time by these systems has significant characteristics such as high sampling frequency, strong dimensional coupling, and huge data volume, which places extremely high demands on the system's real-time transmission bandwidth, massive data storage space, and computing performance.

[0003] Existing multidimensional time-series data compression schemes typically employ compression algorithms based on temporal prediction or differential coding. These approaches primarily reduce the fluctuation range of time-domain values ​​by performing first-order or higher-order differential operations on a single-dimensional sequence, and then convert floating-point numbers into integer indices using linear quantization techniques. Finally, they are encapsulated using traditional Huffman coding or LZ series lossless compression algorithms. However, when processing large-scale multidimensional correlated data, existing technologies face the following serious bottlenecks in practical applications due to their failure to balance spatial coupling and temporal dynamics: First, spatial dimensional redundancy elimination is inefficient. Traditional compression methods rely too heavily on the evolution of a single dimension along the time axis, essentially breaking down multidimensional time-series data into several independent one-dimensional sequences for isolated processing, thus completely ignoring the strong coupling characteristics of sensor arrays in physical space. Due to the lack of scientific dimensional correlation analysis and grouping mechanisms, many dimensions with highly co-evolutionary patterns cannot achieve incremental spatial decoupling. This causes cross-dimensional common features that should be offset to be repeatedly stored as redundant data, becoming a core obstacle restricting the improvement of compression ratio.

[0004] Secondly, the error boundary is difficult to adapt to the dynamic characteristics of multidimensional time-series data. Existing lossy compression processes mostly adopt fixed error control strategies, which cannot adaptively adjust the error boundary according to the characteristic fluctuations of multidimensional data under different operating conditions. This rigid quantization mechanism not only limits the reconstruction fidelity in highly dynamic regions, but also makes the residual distribution prone to amplitude surges when processing across dimensions, making it difficult to ensure the robustness of data processing in complex industrial scenarios.

[0005] In summary, existing multidimensional time-series data compression technologies struggle to simultaneously achieve high compression ratios, high fidelity, and dynamic adaptability. There is an urgent need in this field for a novel technical solution that can deeply integrate dimensional spatial correlation analysis and possess error boundary control across different dimensions to meet the ever-increasing demand for massive data processing. Summary of the Invention

[0006] The present invention aims to at least partially solve one of the technical problems existing in the related art.

[0007] The purpose of this invention is to provide a lossy compression method for multidimensional time-series data based on dimensional correlation grouping, aiming to solve the problems of high spatial dimensional redundancy, fixed error boundaries, and the resulting limited compression ratio improvement and unstable data reconstruction fidelity in existing compression technologies. This method constructs dimensional mapping relationships by performing multidimensional data correlation analysis and employs a complete technical process from dimensional grouping, adaptive quantization with error feedback, spatial incremental decoupling to bit-level reassembly. This achieves high-efficiency compression of multidimensional time-series data under complex operating conditions, providing a robust and reliable data processing solution for the real-time transmission, efficient storage, and digital intelligent analysis of massive amounts of data in the Industrial Internet of Things (IIoT).

[0008] To achieve the above objectives, this invention provides a lossy compression method for multidimensional time-series data based on dimensional correlation grouping, comprising the following steps: S1. Obtain the raw multidimensional time series data, preprocess it, and extract the dynamic trend features of the data evolution over time in each dimension; S2. Based on the dynamic trend characteristics of data in each dimension, calculate the dynamic error boundary parameters of each dimension. At the same time, by calculating the correlation coefficient between the time difference sequences of each dimension, construct highly correlated dimension groups and determine the mapping topology relationship between primary and secondary dimensions within each group. S3. Perform differential processing on the data in the time domain and introduce an error feedback mechanism during the quantization process to compensate for quantization loss in real time and generate quantization differences of data in each dimension. S4. Calculate the difference residuals of the secondary dimension relative to the primary dimension within each group, and dynamically adjust the compression strategy according to the magnitude characteristics of the residual sequence to generate a set to be encoded consisting of the primary dimension sequence and the secondary dimension residual sequence. S5. Based on the generated set to be encoded, rearrange the bits in the bit plane to optimize the data distribution and reduce the sequence entropy. S6. The recombined set to be encoded is losslessly compressed using an entropy coding algorithm and encapsulated into standardized compressed data blocks.

[0009] A further preferred technical solution of the present invention is that the preprocessing method in step S1 is: to perform a first-order difference operator on the multidimensional time series matrix, to convert the values ​​of each dimension to the incremental space, and to extract the dynamic trend features of the data of each dimension as they evolve over time.

[0010] As a preferred method, the dynamic error boundary parameters of each dimension in step S2 are obtained by: analyzing the fluctuation amplitude of the data of each dimension and the application accuracy requirements in real time, and adaptively calculating the dynamic quantization step size parameters corresponding to each dimension as the dynamic error boundary for constraining the quantization accuracy.

[0011] Preferably, in step S2, highly correlated dimension groups are constructed and the mapping topology of primary and secondary dimensions is determined within each group, specifically as follows: Structural correlation construction: Based on the dynamic trend characteristics of data in each dimension, the correlation measure between each pair of dimensions is calculated using the Pearson correlation coefficient formula, a full-dimensional correlation matrix is ​​constructed, and a correlation model reflecting the coupling relationship between dimensions is generated according to the preset judgment rules. Grouping and Role Determination: The association model is divided into subgraphs using a clustering strategy to form multiple highly correlated dimension groups. Within each group, the primary and secondary dimension relationships are established according to the dimension index order, and a mapping topology containing the dimension index set and the correlation weight information within the group is output.

[0012] Preferably, step S3 performs differential processing on the data in the time domain and introduces an error feedback mechanism during the quantization process to compensate for quantization loss in real time, generating quantization differences for data in each dimension, specifically as follows: Compensated quantization and mapping reconstruction: Differential operations are performed in the time domain to extract data increments. An error feedback mechanism is introduced, and the input value at the next time step is dynamically compensated using the quantization reconstruction residual generated at the current time step to generate the corrected value to be quantized. Linear quantization is then performed based on the dynamic step size. Residual closed-loop iteration: Calculate the algebraic difference between the quantization correction value and the reconstructed representation, and feed the new residual back to the next time step.

[0013] Preferably, step S4 calculates the difference residuals of the secondary dimension relative to the primary dimension within each group, and dynamically adjusts the compression strategy based on the magnitude characteristics of the residual sequence to generate a set to be encoded consisting of the primary dimension sequence and the secondary dimension residual sequence, specifically: Spatial correlation incremental decoupling: Based on the quantization difference of the main dimension in each dimensional group, the algebraic offset of the secondary dimension relative to the main dimension in the same group is calculated to achieve decoupling of common features between dimensions and generate spatial domain residual sequences. Association validity assessment: Real-time monitoring of the amplitude characteristics of the spatial domain residual sequence, and dynamic decision on whether to retain the spatial residual result or fall back to the independent dimension encoding path based on its deviation from the preset threshold, generating a set to be encoded consisting of the main dimension sequence and the secondary dimension residual sequence.

[0014] Preferably, step S5 rearranges the bits of the generated set to be encoded in the bit plane to optimize the data distribution and reduce the sequence entropy value, specifically as follows: Bit-plane decomposition: Analyze the bit distribution of each value in the set to be encoded at different saliency levels and map it to a multidimensional bit matrix space; Bitstream serialization scan: Perform horizontal rearrangement and concatenation of the bit matrix according to the saliency order from high to low in the plane; Distribution feature optimization: By recombining, the data entropy value is concentrated on the bit sequence, generating a single-stream bit sequence with high statistical bias characteristics.

[0015] In another aspect, the present invention provides a non-transitory computer-readable storage medium having computer instructions stored thereon, the computer instructions causing a computer to execute the above-described multidimensional time-series data compression method.

[0016] In another aspect, the present invention provides an electronic device, comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus, and the processor calls logical instructions in the memory to execute the above-described multidimensional time-series data compression method.

[0017] In another aspect, the present invention provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer performs the above-described multidimensional time-series data compression method.

[0018] Beneficial Effects: This invention, by constructing a dimensional correlation grouping model and combining it with spatial incremental decoupling technology, deeply explores the strong spatiotemporal coupling characteristics of sensor arrays at both the physical and logical levels. This not only achieves accurate capture of the correlation features of multidimensional heterogeneous data but also effectively overcomes the limitations of independent compression of each dimension. This allows previously difficult-to-handle cross-dimensional redundant information to be fully offset in the incremental space, significantly improving the overall compression ratio of multidimensional data. By introducing an error feedback closed loop, dynamic compensation for quantization loss is achieved, enhancing the robustness of the algorithm under complex and fluctuating conditions while maintaining reconstruction fidelity. Furthermore, this method utilizes bit-level recombination technology to optimize the entropy distribution of the sequence, improving the matching degree between data features and statistical coding. This reduces the transmission bandwidth load and storage costs of industrial IoT systems, providing an efficient and reliable data foundation for the real-time processing and intelligent analysis of massive monitoring data. Attached Figure Description

[0019] Figure 1 The flowchart of a lossy compression method for multidimensional time-series data based on dimensional correlation grouping provided by the present invention is shown.

[0020] Figure 2 This is a schematic diagram of bit rearrangement in the set to be encoded in Embodiment 1 of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, embodiments of this invention, and should not be construed as limiting the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. In the description of this invention, it should be understood that the terminology used is for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0022] The following is combined Figure 1-2 This invention describes a lossy compression method for multidimensional time-series data based on dimensional correlation grouping.

[0023] Example 1: This example provides a lossy compression method for multidimensional time-series data based on dimensional correlation grouping. This example constructs a dimensional grouping model based on correlation analysis and combines it with dynamic quantization with error feedback, spatial incremental decoupling, and bit-level reassembly modules. By deeply mining the spatial correlation characteristics between multidimensional sensor sequences, the grouping mechanism identifies and eliminates redundant information between dimensions, effectively solving the storage efficiency bottleneck caused by insufficient utilization of spatial correlation in traditional methods when processing large-scale multidimensional data. This provides a reliable technical solution for the intensive storage of massive heterogeneous data in industrial monitoring scenarios.

[0024] The specific steps of the method in this embodiment are as follows: Figure 1 As shown, it includes: S1. Obtain the raw multidimensional time series data, preprocess it, and extract the dynamic trend features of the data evolution over time in each dimension; Analog signals are continuously acquired using sensors with multiple physical channels within a preset sampling period. Then, a multi-channel data acquisition card (DAQ) performs analog-to-digital conversion, converting the analog current or voltage signals into discrete digital sequences. The acquisition system aligns the sampling sequences of D physical dimensions along the time axis, forming a raw dataset with a unified time scale. Finally, this dataset is organized into a two-dimensional matrix, where the columns correspond to different measurement dimensions D, and the rows correspond to consecutive time sampling points N, thus constructing the initial multi-dimensional time series matrix to be processed.

[0025] The system receives a multidimensional time series matrix to be processed, containing D measurement dimensions and N time sampling points. By applying a first-order difference operator to each dimension, the system processes the sequence of any dimension in the matrix. Calculate the algebraic difference between adjacent sampling points, that is, let the increment value (where t=2,3,…,N), thus obtaining an increment matrix of size (N−1)×D. This process transforms the original data into the increment space by eliminating the static absolute value offset of each dimension, enabling subsequent correlation calculations to accurately capture the dynamic characteristics of the signal, such as the instantaneous fluctuation direction and rate of change. For example, for the ambient temperature and cabin oil temperature in a wind turbine, differential operations can eliminate the baseline temperature bias under different seasons, allowing subsequent correlation calculations to accurately capture the instantaneous temperature fluctuation characteristics caused by load changes.

[0026] S2. Based on the dynamic trend characteristics of data in each dimension, calculate the dynamic error boundary parameters of each dimension. At the same time, by calculating the correlation coefficient between the time difference sequences of each dimension, construct highly correlated dimension groups and determine the mapping topology relationship between primary and secondary dimensions within each group. Dynamic error boundary determination: Real-time analysis of the fluctuation amplitude and application accuracy requirements of data in each dimension. The system takes the first-order difference sequence of each dimension as input, statistically analyzes the distribution characteristics of the absolute difference values ​​between adjacent time sampling points, and obtains statistics characterizing the dynamic fluctuation level of that dimension, including the standard deviation of the difference amplitude, the maximum jump amplitude, and the minimum non-zero jump amplitude. These statistics are used to characterize the overall fluctuation intensity and instantaneous change characteristics of each dimension within the current time window. Based on the above statistical results, the system discriminates the change characteristics of the dimensions and uses a differentiated strategy to determine the quantization step size. For example, during the equipment startup phase, data fluctuations are severe with obvious instantaneous jumps; the quantization step size parameter is constrained based on the jump amplitude to ensure the retention of key change information. During the stable operation phase, the quantization step size is adaptively adjusted according to the difference fluctuation intensity to improve compression efficiency, and the dynamic quantization step size parameter corresponding to each dimension is adaptively calculated. This parameter serves as the boundary constraining quantization accuracy, ensuring the adaptability of the compression process under different operating conditions.

[0027] Adaptive Grouping and Role Establishment: First, structural association is constructed. Based on the generated incremental matrix, the mean, variance, and covariance of the incremental sequences between each pair of dimensions are calculated sequentially. Next, the correlation measure of each pair of dimensions is calculated using the Pearson correlation coefficient formula, constructing a symmetric D×D full-dimensional correlation matrix. Specifically, for any two incremental sequences corresponding to any two dimensions, the system first calculates their mean in the time dimension, and then calculates their respective variances and the covariance between them. Subsequently, the covariance is divided by the product of the corresponding standard deviations to obtain the normalized correlation measure. To avoid numerical instability caused by the denominator approaching zero, the normalization operation is skipped or the correlation result is set to zero when the variance calculation result is less than a preset minimum threshold ϵ, thus ensuring the numerical stability of the correlation measure. By repeating the above calculation process for all pair of dimensions, a symmetric D×D full-dimensional correlation matrix is ​​finally constructed. Then, the matrix is ​​binarized according to a preset correlation threshold, and an undirected adjacency list reflecting the strong coupling relationship between dimensions is constructed, converting continuous correlation values ​​into a discrete structured association model. Next, based on the undirected adjacency list, an adaptive subgraph partitioning strategy is performed using a greedy clustering approach: the algorithm traverses all dimensions, aggregating dimensions that meet the relevance threshold and their associated nodes into independent dimension groups. Finally, within each group, the original relevance matrix sub-blocks are retained through index mapping, and role assignments are established according to the index order of the sequence within that group: typically, the first dimension traversed is defined as the primary dimension, while the remaining dimensions within the same group are defined as secondary dimensions subordinate to this primary dimension. The final output is a mapping topology containing the set of dimension indices and the relevance weights within the group, providing a definite computational order for subsequent cross-dimensional residual extraction.

[0028] S3. Perform differential processing on the data in the time domain and introduce an error feedback mechanism during the quantization process to compensate for quantization loss in real time and generate quantization differences for data in each dimension; specifically including: compensated quantization and mapping reconstruction as well as residual closed-loop iteration.

[0029] S31. Compensation Quantization and Mapping Reconstruction: Perform differential operations in the time domain to extract data increments, introduce an error feedback mechanism, and use the quantization reconstruction residual generated at the current time to dynamically compensate the input value at the next time. For example, if the reconstructed value at the previous time is 0.05 smaller due to quantization, then 0.05 is added to the input value at the current time for correction, generating the corrected value to be quantized, and performing linear quantization according to the dynamic step size. S32. Residual Closed-Loop Iteration: The difference value at the current moment is superimposed with the accumulated residual from the previous moment and then quantized. Subsequently, the difference between the quantized reconstructed value and the residual correction difference is taken as the new residual, and this new residual is fed back to the next moment. Through this closed-loop absorption mechanism, the reconstruction accuracy is always kept under control, effectively suppressing the trend of quantization error shift.

[0030] S4. Calculate the difference residuals of the secondary dimension relative to the primary dimension within each group, and dynamically adjust the compression strategy according to the magnitude characteristics of the residual sequence to generate a set to be encoded consisting of the primary dimension sequence and the secondary dimension residual sequence. S41. Spatial correlation incremental decoupling: Based on the quantization difference of the main dimension in each dimension group, calculate the algebraic offset of the quantization difference of the secondary dimension in the same group relative to the benchmark, realize the decoupling of common features between dimensions and generate spatial domain residual sequence; for example, if the quantization value of the main temperature dimension is 10 and the quantization value of the secondary dimension is 11, then only its spatial increment 1 is recorded, thereby eliminating the redundancy of spatial dimensions within the group.

[0031] S42. Correlation Validity Assessment: The system monitors the amplitude characteristics of the spatial residual sequence in real time. Based on its deviation from a preset threshold, it dynamically decides whether to retain the spatial residual or revert to an independent dimension encoding path, ultimately generating a set to be encoded consisting of the main dimension sequence and the secondary dimension residuals. Taking a wind turbine monitoring scenario as an example, if a sudden local temperature rise causes the residual fluctuations in the secondary and main dimensions to exceed the preset threshold, the system dynamically decides to revert to an independent dimension encoding path, thus ensuring compression stability during abrupt changes in spatial correlation.

[0032] S5. Based on the generated set to be encoded, rearrange the bits in the bit plane to optimize the data distribution and reduce the sequence entropy value, such as... Figure 2 As shown; S51. Bit-plane decomposition: Analyze the bit distribution of each value in the set to be encoded at different saliency levels (such as sign bit, highest bit to lowest bit), map it to a multi-dimensional bit matrix space, and realize the conversion from decimal or floating-point representation to binary bit representation; S52. Bitstream Serialization Scan: Following the saliency order of the bit plane from high to low, the bit matrix is ​​rearranged and concatenated horizontally to transform numerical correlation into bit-level correlation. For example, the sign bits of all dimensions are concatenated first, followed by the highest-order bit, converting the implicit correlation of multidimensional values ​​into an explicit bitstream statistical bias. S53. Distribution feature optimization: By reorganizing the bit distribution, the data entropy value is concentrated on the bit sequence, generating a single-stream bit sequence with high statistical bias characteristics, which provides a good data feature foundation for the subsequent lossless coding stage to reach the theoretical compression ratio limit. S6. The recombined set to be encoded is losslessly compressed using an entropy coding algorithm and encapsulated into standardized compressed data blocks.

[0033] A high-efficiency entropy coding algorithm is used to perform lossless compression on the recombined bitstream, and combined with metadata such as dimension grouping mapping information, dynamic quantization step size and first-end reference value of each sequence, a standardized data block encapsulation is performed.

[0034] During the data block encapsulation phase, the system writes various metadata into the data block header in a predetermined order: First, it writes the dynamic quantization step size parameter dimension by dimension, encoding each quantization step size as a fixed-length bit in floating-point form; then, it writes the dimension grouping mapping information. Next, it writes the initial quantization bit width, and then sequentially writes the first quantization reference value for each dimension to support subsequent differential reconstruction; then, it writes the differential quantization bit width, the original sequence length, and the byte length of the compressed bitstream. Finally, it writes the compressed bitstream obtained through entropy encoding byte by byte to the end of the data block, forming a structured, self-describing data block output.

[0035] In this embodiment, by deeply integrating temporal domain differencing, spatial domain redundancy elimination, and bit recombination, deep stripping of redundancy from multidimensional time-series data is achieved. Unlike traditional independent-dimensional compression methods, this method fully utilizes the inter-dimensional correlation between sensors and ensures fidelity after lossy processing through a feedback loop.

[0036] Example 2: This example provides a non-transitory computer-readable storage medium storing computer instructions that cause a computer to execute a lossy compression method for multidimensional time-series data based on dimensional correlation grouping. The method includes the following steps: S1. Obtain the raw multidimensional time series data, preprocess it, and extract the dynamic trend features of the data evolution over time in each dimension; S2. Based on the dynamic trend characteristics of data in each dimension, calculate the dynamic error boundary parameters of each dimension. At the same time, by calculating the correlation coefficient between the time difference sequences of each dimension, construct highly correlated dimension groups and determine the mapping topology relationship between primary and secondary dimensions within each group. S3. Perform differential processing on the data in the time domain and introduce an error feedback mechanism during the quantization process to compensate for quantization loss in real time and generate quantization differences of data in each dimension. S4. Calculate the difference residuals of the secondary dimension relative to the primary dimension within each group, and dynamically adjust the compression strategy according to the magnitude characteristics of the residual sequence to generate a set to be encoded consisting of the primary dimension sequence and the secondary dimension residual sequence. S5. Based on the generated set to be encoded, rearrange the bits in the bit plane to optimize the data distribution and reduce the sequence entropy. S6. The recombined set to be encoded is losslessly compressed using an entropy coding algorithm and encapsulated into standardized compressed data blocks.

[0037] Example 3: This example provides an electronic device, including a processor, a communication interface, a memory, and a communication bus. The processor, communication interface, and memory communicate with each other via the communication bus. The processor calls logical instructions from the memory to execute a lossy compression method for multidimensional time-series data based on dimensional correlation grouping. This method includes the following steps: S1. Obtain the raw multidimensional time series data, preprocess it, and extract the dynamic trend features of the data evolution over time in each dimension; S2. Based on the dynamic trend characteristics of data in each dimension, calculate the dynamic error boundary parameters of each dimension. At the same time, by calculating the correlation coefficient between the time difference sequences of each dimension, construct highly correlated dimension groups and determine the mapping topology relationship between primary and secondary dimensions within each group. S3. Perform differential processing on the data in the time domain and introduce an error feedback mechanism during the quantization process to compensate for quantization loss in real time and generate quantization differences of data in each dimension. S4. Calculate the difference residuals of the secondary dimension relative to the primary dimension within each group, and dynamically adjust the compression strategy according to the magnitude characteristics of the residual sequence to generate a set to be encoded consisting of the primary dimension sequence and the secondary dimension residual sequence. S5. Based on the generated set to be encoded, rearrange the bits in the bit plane to optimize the data distribution and reduce the sequence entropy. S6. The recombined set to be encoded is losslessly compressed using an entropy coding algorithm and encapsulated into standardized compressed data blocks.

[0038] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0039] Example 4: This example provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer performs a lossy compression method for multidimensional time-series data based on dimensional correlation grouping. The method includes the following steps: S1. Obtain the raw multidimensional time series data, preprocess it, and extract the dynamic trend features of the data evolution over time in each dimension; S2. Based on the dynamic trend characteristics of data in each dimension, calculate the dynamic error boundary parameters of each dimension. At the same time, by calculating the correlation coefficient between the time difference sequences of each dimension, construct highly correlated dimension groups and determine the mapping topology relationship between primary and secondary dimensions within each group. S3. Perform differential processing on the data in the time domain and introduce an error feedback mechanism during the quantization process to compensate for quantization loss in real time and generate quantization differences of data in each dimension. S4. Calculate the difference residuals of the secondary dimension relative to the primary dimension within each group, and dynamically adjust the compression strategy according to the magnitude characteristics of the residual sequence to generate a set to be encoded consisting of the primary dimension sequence and the secondary dimension residual sequence. S5. Based on the generated set to be encoded, rearrange the bits in the bit plane to optimize the data distribution and reduce the sequence entropy. S6. The recombined set to be encoded is losslessly compressed using an entropy coding algorithm and encapsulated into standardized compressed data blocks.

[0040] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0041] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0042] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lossy compression method for multidimensional time-series data based on dimensional correlation grouping, characterized in that, Includes the following steps: S1. Obtain the raw multidimensional time series data, preprocess it, and extract the dynamic trend features of the data evolution over time in each dimension; S2. Based on the dynamic trend characteristics of data in each dimension, calculate the dynamic error boundary parameters of each dimension. At the same time, by calculating the correlation coefficient between the time difference sequences of each dimension, construct highly correlated dimension groups and determine the mapping topology relationship between primary and secondary dimensions within each group. S3. Perform differential processing on the data in the time domain and introduce an error feedback mechanism during the quantization process to compensate for quantization loss in real time and generate quantization differences of data in each dimension. S4. Calculate the difference residuals of the secondary dimension relative to the primary dimension within each group, and dynamically adjust the compression strategy according to the magnitude characteristics of the residual sequence to generate a set to be encoded consisting of the primary dimension sequence and the secondary dimension residual sequence. S5. Based on the generated set to be encoded, rearrange the bits in the bit plane to optimize the data distribution and reduce the sequence entropy. S6. The recombined set to be encoded is losslessly compressed using an entropy coding algorithm and encapsulated into standardized compressed data blocks.

2. The lossy compression method for multidimensional time-series data based on dimensional correlation grouping according to claim 1, characterized in that, The preprocessing method described in step S1 is as follows: perform a first-order difference operator on the multidimensional time series matrix to transform the values ​​of each dimension into the incremental space and extract the dynamic trend features of the data of each dimension as they evolve over time.

3. The lossy compression method for multidimensional time-series data based on dimensional correlation grouping according to claim 2, characterized in that, The method for obtaining the dynamic error boundary parameters of each dimension in step S2 is as follows: analyze the fluctuation amplitude of the data of each dimension and the application accuracy requirements in real time, and adaptively calculate the dynamic quantization step size parameters corresponding to each dimension as the dynamic error boundary that constrains the quantization accuracy.

4. The lossy compression method for multidimensional time-series data based on dimensional correlation grouping according to claim 3, characterized in that, Step S2 involves constructing highly relevant dimension groups and determining the mapping topology of primary and secondary dimensions within each group, specifically as follows: Structural correlation construction: Based on the dynamic trend characteristics of data in each dimension, the correlation measure between each pair of dimensions is calculated using the Pearson correlation coefficient formula, a full-dimensional correlation matrix is ​​constructed, and a correlation model reflecting the coupling relationship between dimensions is generated according to the preset judgment rules. Grouping and Role Determination: The association model is divided into subgraphs using a clustering strategy to form multiple highly correlated dimension groups. Within each group, the primary and secondary dimension relationships are established according to the dimension index order, and a mapping topology containing the dimension index set and the correlation weight information within the group is output.

5. The lossy compression method for multidimensional time-series data based on dimensional correlation grouping according to claim 4, characterized in that, Step S3 performs differential processing on the data in the time domain and introduces an error feedback mechanism during the quantization process to compensate for quantization loss in real time, generating quantization differences for each dimension of the data, specifically as follows: Compensated quantization and mapping reconstruction: Differential operations are performed in the time domain to extract data increments. An error feedback mechanism is introduced, and the input value at the next time step is dynamically compensated using the quantization reconstruction residual generated at the current time step to generate the corrected value to be quantized. Linear quantization is then performed based on the dynamic step size. Residual closed-loop iteration: Calculate the algebraic difference between the quantization correction value and the reconstructed representation, and feed the new residual back to the next time step.

6. The lossy compression method for multidimensional time-series data based on dimensional correlation grouping according to claim 5, characterized in that, Step S4 calculates the difference residuals of the secondary dimension relative to the primary dimension within each group, and dynamically adjusts the compression strategy based on the magnitude characteristics of the residual sequence to generate a set to be encoded consisting of the primary dimension sequence and the secondary dimension residual sequence, specifically: Spatial correlation incremental decoupling: Based on the quantization difference of the main dimension in each dimensional group, the algebraic offset of the secondary dimension relative to the main dimension in the same group is calculated to achieve decoupling of common features between dimensions and generate spatial domain residual sequences. Association validity assessment: Real-time monitoring of the amplitude characteristics of the spatial domain residual sequence, and dynamic decision on whether to retain the spatial residual result or fall back to the independent dimension encoding path based on its deviation from the preset threshold, generating a set to be encoded consisting of the main dimension sequence and the secondary dimension residual sequence.

7. The lossy compression method for multidimensional time-series data based on dimensional correlation grouping according to claim 6, characterized in that, Step S5, based on the generated set to be encoded, rearranges the bits in the bit plane to optimize the data distribution and reduce the sequence entropy, specifically as follows: Bit-plane decomposition: Analyze the bit distribution of each value in the set to be encoded at different saliency levels and map it to a multidimensional bit matrix space; Bitstream serialization scan: Perform horizontal rearrangement and concatenation of the bit matrix according to the saliency order from high to low in the plane; Distribution feature optimization: By recombining, the data entropy value is concentrated on the bit sequence, generating a single-stream bit sequence with high statistical bias characteristics.

8. A non-transitory computer-readable storage medium, characterized in that, It stores computer instructions that cause the computer to execute the multidimensional time-series data compression method according to any one of claims 1-7.

9. An electronic device, characterized in that, include: The system includes a processor, a communication interface, a memory, and a communication bus. The processor, communication interface, and memory communicate with each other via the communication bus. The processor calls logical instructions from the memory to execute the multidimensional time-series data compression method according to any one of claims 1-7.

10. A computer program product, characterized in that, The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer performs the multidimensional time-series data compression method according to any one of claims 1-7.

Images