A lossless compression method and system for smart internet of things table cycle waveform data

Abstract

The application discloses a lossless compression method and system for smart internet of things meter cycle waveform data, and belongs to the technical field of data compression and power signal processing. The method comprises the following steps: collecting original cycles of the smart internet of things meter, detecting cycle boundaries and dividing cycles for the collected original cycles, pre-aligning phases of adjacent cycles for the divided cycles to determine an amplitude scaling coefficient and a direct current offset, performing linear prediction based on the amplitude scaling coefficient and the direct current offset to obtain residual errors and parameter sequences; performing reversible transformation on the predicted residual errors and parameter sequences to obtain difference encoding and / or segmented reference encoding of the residual errors and the parameters, performing interval encoding based on a statistical model for the difference encoding and / or the segmented reference encoding, and performing self-describing transmission frame packaging based on the interval encoding. The implementation of the application significantly improves the encoding and decoding efficiency of cycle data.

Description

Technical Field

[0001] This invention relates to the fields of data compression and power signal processing technology, and more specifically, to a lossless compression method and system for the waveform data of smart IoT meters. Background Technology

[0002] In devices such as smart meters and edge sensing terminals, it is necessary to continuously collect and store waveform data such as voltage and current. Due to the high sampling frequency, large number of channels, and long recording time, the amount of raw waveform data is large, which puts a lot of pressure on the terminal's local storage, wireless transmission, background archiving, and real-time analysis.

[0003] Taking a waveform with 256 points per cycle, 3 bytes per point, and continuous recording for 75 cycles as an example, the raw data volume of a single channel is approximately 56.25KB. In a three-phase multi-channel scenario, such as simultaneous sampling of 3 voltage and 3 current channels, the raw data volume can be further increased to several hundred KB. For smart IoT devices using embedded processors and limited storage resources, there is an urgent need to efficiently compress waveform data while ensuring complete data recovery.

[0004] In existing technologies, common general-purpose lossless compression algorithms such as zlib and LZ4 mainly utilize statistical redundancy of byte strings and do not model the characteristics of power waveforms, such as obvious cycle characteristics, similarity between adjacent cycles, gradual amplitude variation, and gradual DC offset variation. Therefore, their compression ratio on cycle waveform data is usually limited. Another type of method uses differential sampling of adjacent samples or simple proportional prediction of adjacent cycles. Although it has a certain effect under steady-state conditions, the dynamic range of the residuals will increase significantly under non-stationary scenarios such as voltage sags, swells, harmonic disturbances, DC drift, and frequency fluctuations, leading to a decrease in compression ratio.

[0005] Furthermore, under asynchronous sampling or non-PLL sampling conditions, due to slight fluctuations in the system frequency, the phase positions of adjacent cycles on the sampling point sequence are often not perfectly aligned. Directly using the same sampling point number for prediction or differentiation will introduce additional phase mismatch errors, widening the residual statistical distribution and hindering subsequent entropy coding. Existing technologies generally lack a comprehensive solution that simultaneously addresses phase alignment, amplitude variation compensation, DC offset compensation, reversible residual coding, and the complexity of embedded real-time implementation. Summary of the Invention

[0006] To address the above problems, this invention proposes a lossless compression method for cycle waveform data of smart IoT meters, comprising: The system acquires raw frequency data from smart IoT meters, performs frequency boundary detection and frequency segmentation on the acquired raw frequency data, pre-aligns the phases of adjacent frequencies on the segmented frequencies to determine the amplitude scaling factor and DC offset, and performs linear prediction based on the amplitude scaling factor and DC offset to obtain the residual and parameter sequence. A reversible transformation is performed on the predicted residual and parameter sequence to obtain differential coding and / or segmented reference coding of the residual and parameters. The differential coding and / or segmented reference coding are then subjected to interval coding based on a statistical model. A self-describing transmission frame encapsulation is performed based on the interval coding. Synchronization word detection and frame synchronization processing are performed on the transmission frame to parse the frame header and side information. Based on the frame header and side information, the interval code is obtained. Based on the interval code, integer inverse mapping and parameter recovery are performed, and the parameter sequence is reconstructed by segment or by cycle. Based on linear prediction, the original cycle is recovered cycle by cycle according to the parameter sequence and residual.

[0007] Optionally, after performing cycle boundary detection and cycle segmentation on the acquired raw cycles, the first cycle can be obtained. The following is a sequence of one-cycle data: in, For the first Number of sampling points in a cycle data sequence is the original sampled value of the j-th sampling point in the i-th cycle, where j is the sampling point number within the cycle.

[0008] Optional phase pre-alignment includes: Regarding the first The cross-correlation function between the previous week's data sequence was calculated using the following method: in, For candidate offsets, To ensure a consistent number of sampling points per cycle during processing, The (i-1)th cycle, that is, the previous cycle or reference cycle of the current cycle. ; The offset corresponding to the maximum value of the cross-correlation function is taken as follows: According to the offset The phase-aligned reference sequence generated for the previous cycle is as follows: in, For interpolation operations; when When the value is an integer, the integer point is shifted; when... When the integer is not an integer, linear interpolation, spline interpolation, or polynomial interpolation is used.

[0009] Optionally, taking advantage of the high similarity in shape and the slow variation in amplitude and DC component between adjacent cycles, the first... A linear prediction model for linear prediction is established based on a series of cycle data, as follows: in: For the first The first cycle Predicted values ​​for each sampling point For the first The amplitude scaling factor of each cycle relative to the previous cycle. For the first The DC offset of each cycle relative to the previous cycle. This is the reference cycle after phase pre-alignment; The prediction residual is defined as: .

[0010] Optionally, the amplitude scaling factor and DC offset are jointly estimated using the least squares method, as shown in the following formula: Its closed-form solution is: in: in, For the first The amplitude scaling factor of each cycle relative to the previous cycle. For the first The DC offset of each cycle relative to the previous cycle. For the reference cycle after phase pre-alignment, The reference cycle mean after phase alignment. Let be the mean of the i-th cycle. This is the amplitude scaling factor. This is the DC offset. .

[0011] Optional, reversible transformations include: Differentiating the amplitude scaling factor, DC offset, and phase offset along the cycle number direction yields: For the residual sequence When the changes between adjacent sampling points are gradual, perform a first-order difference: Whether to perform differencing is adaptively selected based on the residual statistical characteristics, if the following conditions are met: in, If a preset threshold is used, then the differencing sequence is employed. Encode; otherwise, directly... Encode it.

[0012] Optionally, the interval coding compresses the non-negative integer sequence after invertible mapping using a statistical model-based interval coding, wherein the interval coding employs a static model and / or an adaptive model.

[0013] Optionally, the segmentation reference strategy before transmission frame encapsulation includes: dividing the total sequence into several independent segments, each segment containing K cycles, with the first cycle in each segment serving as the reference cycle for raw storage or low-compression encoding, and the remaining cycles within the segment using the previous cycle or the reference cycle of the segment as a reference for phase pre-alignment and linear prediction, and each segment being independently encoded and independently decoded.

[0014] Optionally, the transmission frame includes: a synchronization word, a version number, a mode identifier, a total length, a number of cycles, a number of points per cycle or a point description, segmentation information, a parameter representation format, phase offset information, bit width information or a maximum block value information, a compressed data stream, and a checksum.

[0015] Furthermore, this invention also proposes a lossless compression system for cycle waveform data of smart IoT meters, comprising: The first encoding unit is used to acquire the original frequency of the smart IoT meter, perform frequency boundary detection and frequency division on the acquired original frequency, perform phase pre-alignment of adjacent frequencies on the divided frequencies to determine the amplitude scaling factor and DC offset, and perform linear prediction based on the amplitude scaling factor and DC offset to obtain the residual and parameter sequence. The second coding unit is used to perform a reversible transformation on the predicted residual and parameter sequence to obtain differential coding and / or segmented reference coding of the residual and parameters, perform interval coding based on a statistical model on the differential coding and / or segmented reference coding, and perform self-describing transmission frame encapsulation based on the interval coding. The decoding unit is used to perform synchronization word detection and frame synchronization processing on the transmitted frame, parse the frame header and side information, obtain the interval code based on the frame header and side information, perform integer inverse mapping and parameter recovery based on the interval code, and reconstruct the parameter sequence by segment or by cycle. Based on linear prediction, the original cycle is recovered cycle by cycle according to the parameter sequence and residual.

[0016] Optionally, after performing cycle boundary detection and cycle segmentation on the acquired raw cycles, the first cycle can be obtained. The following is a sequence of one-cycle data: in, For the first The number of sampling points in a 1-cycle data sequence, where j is the sampling point number within a cycle.

[0017] Optional phase pre-alignment includes: Regarding the first The cross-correlation function between the previous week's data sequence was calculated using the following method: in, For candidate offsets, To ensure a consistent number of sampling points per cycle during processing, The (i-1)th cycle, that is, the previous cycle or reference cycle of the current cycle. ; The offset corresponding to the maximum value of the cross-correlation function is taken as follows: According to the offset The phase-aligned reference sequence generated for the previous cycle is as follows: in, For interpolation operations; when When the value is an integer, the integer point is shifted; when... When the integer is not an integer, linear interpolation, spline interpolation, or polynomial interpolation is used.

[0018] Optionally, taking advantage of the high similarity in shape and the slow variation in amplitude and DC component between adjacent cycles, the first... A linear prediction model for linear prediction is established based on a series of cycle data, as follows: in: For the first The first cycle Predicted values ​​for each sampling point For the first The amplitude scaling factor of each cycle relative to the previous cycle. For the first The DC offset of each cycle relative to the previous cycle. This is the reference cycle after phase pre-alignment; The prediction residual is defined as: .

[0019] Optionally, the amplitude scaling factor and DC offset are jointly estimated using the least squares method, as shown in the following formula: Its closed-form solution is: in: in, For the first The amplitude scaling factor of each cycle relative to the previous cycle. For the first The DC offset of each cycle relative to the previous cycle. For the reference cycle after phase pre-alignment, The reference cycle mean after phase alignment. Let be the mean of the i-th cycle. This is the amplitude scaling factor. This is the DC offset. Optional, reversible transformations include: Differentiating the amplitude scaling factor, DC offset, and phase offset along the cycle number direction yields: For the residual sequence When the changes between adjacent sampling points are gradual, perform a first-order difference: Whether to perform differencing is adaptively selected based on the residual statistical characteristics, if the following conditions are met: in, If a preset threshold is used, then the differencing sequence is employed. Encode; otherwise, directly... Encode it.

[0020] Optionally, the interval coding compresses the non-negative integer sequence after invertible mapping using a statistical model-based interval coding, wherein the interval coding employs a static model and / or an adaptive model.

[0021] Optionally, the segmentation reference strategy before transmission frame encapsulation includes: dividing the total sequence into several independent segments, each segment containing K cycles, with the first cycle in each segment serving as the reference cycle for raw storage or low-compression encoding, and the remaining cycles within the segment using the previous cycle or the reference cycle of the segment as a reference for phase pre-alignment and linear prediction, and each segment being independently encoded and independently decoded.

[0022] Optionally, the transmission frame includes: a synchronization word, a version number, a mode identifier, a total length, a number of cycles, a number of points per cycle or a point description, segmentation information, a parameter representation format, phase offset information, bit width information or a maximum block value information, a compressed data stream, and a checksum.

[0023] In another aspect, the present invention also provides a computing device, comprising: one or more processors; A processor is used to execute one or more programs; When the one or more programs are executed by the one or more processors, the method described above is implemented.

[0024] In another aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the method described above.

[0025] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention provides a lossless compression method for cycle waveform data of smart IoT meters, comprising: acquiring the original cycle of the smart IoT meter; performing cycle boundary detection and cycle segmentation on the acquired original cycle; performing phase pre-alignment of adjacent cycles on the segmented cycles to determine the amplitude scaling factor and DC offset; performing linear prediction based on the amplitude scaling factor and DC offset to obtain residuals and parameter sequences; performing a reversible transformation on the predicted residuals and parameter sequences to obtain differential coding and / or segmented reference coding of the residuals and parameters; performing interval coding based on a statistical model on the differential coding and / or segmented reference coding; performing self-describing transmission frame encapsulation based on the interval coding; performing synchronization word detection and frame synchronization processing on the transmission frame to parse the frame header and side information; obtaining interval coding based on the frame header and side information; performing integer inverse mapping and parameter recovery based on the interval coding; reconstructing the parameter sequence by segment or by cycle; and recovering the original cycle cycle by cycle based on linear prediction, according to the parameter sequence and residuals. The implementation of this invention has significantly improved the efficiency of encoding and decoding of cycle data. Attached Figure Description

[0026] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a flowchart of the encoding process at the sending end of the method of the present invention; Figure 3 This is a flowchart of the decoding process at the receiving end of the method of the present invention; Figure 4 This is a schematic diagram illustrating the parameter estimation of the linear prediction model in the method of this invention; Figure 5 This is a block diagram illustrating the adaptive quantization and arithmetic coding principle of the method of the present invention; Figure 6 This is a schematic diagram of waveform pre-alignment (phase compensation) in the method of the present invention; Figure 7 This is a schematic diagram of the segmentation benchmark strategy of the method of the present invention; Figure 8 A cycle diagram of an example of the method of the present invention; Figure 9 This is a structural diagram of the system of the present invention. Detailed Implementation

[0027] Exemplary embodiments of the invention will now be described with reference to the accompanying drawings. However, the invention may be embodied in many different forms and is not limited to the embodiments described herein. These embodiments are provided to fully and completely disclose the invention and to fully convey its scope to those skilled in the art. The terminology used in the exemplary embodiments illustrated in the drawings is not intended to limit the invention. In the drawings, the same units / elements are referred to by the same reference numerals.

[0028] Unless otherwise stated, the terms used herein (including technical terms) have their common meaning as understood by one of ordinary skill in the art. Furthermore, it is understood that terms defined in commonly used dictionaries should be understood to have a meaning consistent with the context of their relevant field, and not to be interpreted as having an idealized or overly formal meaning.

[0029] Example 1: This invention proposes a lossless compression method S100 for cycle waveform data of smart IoT meters, such as... Figure 1 As shown, it includes: S101: Raw frequency acquisition is performed on the smart IoT meter. The acquired raw frequency is then subjected to frequency boundary detection and frequency division. The phase pre-alignment of adjacent frequencies is performed on the divided frequencies to determine the amplitude scaling factor and DC offset. Linear prediction is then performed based on the amplitude scaling factor and DC offset to obtain the residual and parameter sequence. S102, perform a reversible transformation on the predicted residual and parameter sequence to obtain differential coding and / or segmented reference coding of the residual and parameters, perform interval coding based on a statistical model on the differential coding and / or segmented reference coding, and encapsulate a self-describing transmission frame based on the interval coding. S103, perform synchronization word detection and frame synchronization processing on the transmission frame, parse out the frame header and side information, obtain the interval code based on the frame header and side information, perform integer inverse mapping and parameter recovery based on the interval code, and reconstruct the parameter sequence by segment or by cycle. Based on linear prediction, recover the original cycle cycle by cycle according to the parameter sequence and residual.

[0030] Among them, after performing cycle boundary detection and cycle segmentation on the acquired raw cycles, the first cycle is obtained. The following is a sequence of one-cycle data: in, For the first Number of sampling points in a cycle data sequence is the original sampled value of the j-th sampling point in the i-th cycle, where j is the sampling point number within the cycle.

[0031] Phase pre-alignment includes: Regarding the first The cross-correlation function between the previous week's data sequence was calculated using the following method: in, For candidate offsets, To ensure a consistent number of sampling points per cycle during processing, The (i-1)th cycle, that is, the previous cycle or reference cycle of the current cycle. ; The offset corresponding to the maximum value of the cross-correlation function is taken as follows: According to the offset The phase-aligned reference sequence generated for the previous cycle is as follows: in, For interpolation operations; when When the value is an integer, the integer point is shifted; when... When the integer is not an integer, linear interpolation, spline interpolation, or polynomial interpolation is used.

[0032] Among them, taking advantage of the high similarity in shape and the slow change in amplitude and DC component between adjacent cycles, the first... A linear prediction model for linear prediction is established based on a series of cycle data, as follows: in: For the first The first cycle Predicted values ​​for each sampling point For the first The amplitude scaling factor of each cycle relative to the previous cycle. For the first The DC offset of each cycle relative to the previous cycle. This is the reference cycle after phase pre-alignment; The prediction residual is defined as: .

[0033] The amplitude scaling factor and DC offset are jointly estimated using the least squares method, as shown in the following formula: Its closed-form solution is: in: in, For the first The amplitude scaling factor of each cycle relative to the previous cycle. For the first The DC offset of each cycle relative to the previous cycle. For the reference cycle after phase pre-alignment, The reference cycle mean after phase alignment. Let be the mean of the i-th cycle. This is the amplitude scaling factor. This is the DC offset. .

[0034] Among them, reversible transformations include: Differentiating the amplitude scaling factor, DC offset, and phase offset along the cycle number direction yields: For the residual sequence When the changes between adjacent sampling points are gradual, perform a first-order difference: Whether to perform differencing is adaptively selected based on the residual statistical characteristics, if the following conditions are met: in, If a preset threshold is used, then the differencing sequence is employed. Encode; otherwise, directly... Encode it.

[0035] In this process, the interval coding compresses the non-negative integer sequence after invertible mapping using a statistical model-based interval coding method. The interval coding uses a static model and / or an adaptive model for encoding.

[0036] The segmentation reference strategy before encapsulation of the transmission frame includes: dividing the total sequence into several independent segments, each segment containing K cycles, the first cycle in each segment being used as the reference cycle for raw storage or low-compression encoding, and the remaining cycles in the segment being used as references to perform phase pre-alignment and linear prediction, with each segment being independently encoded and independently decoded.

[0037] The transmission frame includes: synchronization word, version number, mode identifier, total length, number of cycles, number of points per cycle or point description information, segmentation information, parameter representation format, phase offset information, bit width information or block maximum value information, compressed data stream, and checksum.

[0038] The invention will be further illustrated below with specific examples: The purpose of this invention is to address the problems in existing technologies, such as general compression algorithms not utilizing the structural characteristics of power cycle waveforms, the difficulty of adapting simple differential or proportional prediction to non-stationary operating conditions, and the reduction in compression efficiency due to phase misalignment of adjacent cycles under asynchronous sampling conditions. This invention proposes a lossless compression method for cycle waveform data of smart IoT meters.

[0039] This invention aims to achieve the following objectives: By leveraging the strong correlation between adjacent cycles of the power waveform, the current cycle can be predicted with high accuracy, thereby reducing the dynamic range of the residual. By jointly estimating the amplitude scaling factor and DC offset, the adaptability to operating conditions such as sag, gradual change, and DC drift is improved. Phase pre-alignment reduces phase mismatch errors caused by asynchronous sampling or frequency fluctuations; Improve entropy coding efficiency by adaptively selecting residual coding method, parameter difference method and segmentation benchmark strategy; By encapsulating self-describing frames, the receiving end can complete decoding independently without relying on external context; Implement low-complexity, real-time lossless compression and decompression on embedded hardware platforms.

[0040] Based on this, the present invention provides a lossless compression method for cycle waveform data of smart IoT meters, including a transmitting end encoding process and a receiving end decoding process. The transmitting end performs cycle division, phase pre-alignment, linear prediction, residual invertible transformation, entropy coding, and frame encapsulation on the original waveform; the receiving end performs entropy decoding, parameter recovery, and waveform reconstruction based on the complete side information carried in the frame, thereby achieving lossless recovery of the original cycle waveform data.

[0041] Its specific process includes: The encoding process at the sending end, such as Figure 2 As shown, it includes, in sequence: 1. Acquisition of raw waveforms; 2. Cycle boundary detection and cycle division; 3. Pre-alignment of adjacent cycles phase; 4. Linear prediction based on amplitude scaling factor and DC offset; 5. Predicting invertible integer transformations of residual and parameter sequences; 6. Differential coding and / or segmented baseline coding of residuals and parameters; 7. Interval coding based on statistical models; 8. Self-describing transport frame encapsulation.

[0042] The decoding process at the receiving end, such as Figure 3 As shown, it includes, in sequence: 1. Synchronization word detection and frame synchronization; 2. Parsing of frame header and side information; 3. Range decoding; 4. Integer inverse mapping and parameter recovery; 5. Reconstruct prediction parameters by segment or by cycle; 6. Reconstruct the original waveform using the prediction model and residual cycle-by-cycle wave analysis.

[0043] The following is combined with Figure 4-7 The specific process described above will be explained as follows: Among them, the Zhou Bo division includes: Cycle boundary detection is performed on the discrete voltage or current sequences obtained by continuous sampling to obtain the (i)th cycle data sequence: in, Let N be the number of sampling points for the (i)th cycle. In a preferred embodiment, the start and end positions of the cycle can be determined by zero-crossing detection, extreme value detection, phase estimation, or a combination thereof. For scenarios requiring uniform processing length, each cycle can also be rearranged or indexed to correspond to a fixed number of points (N).

[0044] Phase pre-alignment includes: Due to asynchronous sampling or power grid frequency fluctuations, there may be sampling phase misalignment between adjacent cycles. To reduce prediction error, for the (i)th cycle, the cross-correlation function between it and the previous cycle is calculated: in, Candidate offsets, within a preset search range Take the value from the inside.

[0045] Take the offset corresponding to the maximum value of the cross-correlation function: According to the offset Generate a phase-aligned reference sequence for the previous cycle: in, Indicates interpolation operation; when When the integer value is used, integer point translation can be used directly; when... When the integer is not an integer, linear interpolation, spline interpolation, or polynomial interpolation can be used.

[0046] It should be noted that phase pre-alignment is only used to construct the predictive reference sequence and does not change the original cycle data to be compressed, so it does not affect the final lossless recovery.

[0047] The linear prediction model includes: Taking advantage of the high similarity in shape and the slow changes in amplitude and DC component between adjacent cycles, the following linear prediction model is established for the (i)th cycle: in: This is the predicted value of the (i)th cycle and the (j)th sampling point; is the amplitude scaling factor of the (i)th cycle relative to the previous cycle; is the DC offset of the (i)th cycle relative to the previous cycle; This is the reference cycle after phase pre-alignment.

[0048] Accordingly, the prediction residual is defined as: Predictive parameter estimation, including: In a preferred embodiment, joint estimation using the least squares method is employed. and ,Right now: Its closed-form solution is: in: In resource-constrained embedded implementations, a simplified estimation method can also be used, first estimating the amplitude scaling factor: or Then calculate based on the mean: in, To prevent tiny positive numbers with a denominator of zero.

[0049] Among them, the parametric difference and residual invertible transformation include: To further reduce the dynamic range of the encoded object, the amplitude scaling factor, DC offset, and phase offset are differentially expressed along the cycle number direction, resulting in: In a preferred embodiment, to ensure complete reversibility, the above parameters can be represented in a fixed-point format and then differentially encoded.

[0050] For the residual sequence When the changes in adjacent sampling points are gradual, a first-order difference can be further performed: To avoid introducing a larger dynamic range in abrupt change scenarios, this invention can adaptively select whether to perform differencing based on the statistical characteristics of the residuals. Specifically, if the following conditions are met: in, If a preset threshold is used, preferably between 0.1 and 0.3, then the differencing sequence is used. Encode; otherwise, directly... Encode it.

[0051] The adaptive bit-width representation and the invertible integer mapping include: To balance lossless encoding and coding efficiency, the integer sequence to be encoded is divided into cycles or blocks, and the maximum absolute value within each block is calculated: Where (s(k)) represents the integer to be encoded within the block, including any one of residual, residual difference, and parameter difference.

[0052] The minimum bit width required for lossless representation of this block of data is determined based on the maximum absolute value: The additional bit is used to represent the symbol.

[0053] To improve the efficiency of subsequent entropy coding, the signed integer (s) can be invertibly mapped to a non-negative integer (u): The block width (w), block length, mode identifier, and necessary parameter information are written as side information into the frame header or mode-related header. The decoding end can completely recover the original integer sequence based on this side information, thereby ensuring lossless compression.

[0054] Among them, the interval encoding includes: The non-negative integer sequence after the above reversible mapping is compressed using interval coding based on a statistical model. The interval coding can be implemented using a static model, an adaptive model, or a combination of both. Preferably, an adaptive model is used to avoid explicitly transmitting the complete code table, reduce side information overhead, and enable the coding process to automatically adjust the probability estimation according to changes in the residual distribution and parameter difference distribution.

[0055] The segmented benchmark strategy includes: For long-term, continuously recorded cycle sequences, to avoid a decline in prediction performance due to excessively long propagation distance of a single reference, this invention divides the total sequence into several independent segments, each containing (K) cycles. The first cycle in each segment is used as the reference cycle for raw storage or low-compression encoding. The remaining cycles within the segment are pre-aligned and linearly predicted using the previous cycle or the reference cycle of that segment as references. Each segment is independently encoded and decoded.

[0056] By using a segmented baseline strategy, the increase in residuals caused by the accumulation of prediction mismatch in long sequences can be effectively suppressed, the compression rate stability can be improved, and random positioning and local decoding can be facilitated.

[0057] Frame encapsulation includes: The present invention preferably employs a self-describing frame structure, wherein the transmission frame includes at least: Synchronization word; Version number; Pattern identifier; Total length; Number of cycles; Weekly wave points or point description information; Segmentation information; Parameter representation format; Phase offset information; Bit width information or block maximum value information; Compress the data stream; Verification code.

[0058] With the above frame encapsulation method, the receiving end can complete independent decoding without relying on external context, which facilitates transmission and system integration between smart IoT meters, concentrators, edge gateways and main station systems.

[0059] The decoding process is as follows: After receiving the transmission frame, the receiving end first detects the synchronization word and parses the frame header and mode-related header to obtain segmentation information, bit width information, phase offset information, and compressed data stream. Then, it recovers the mapped integer sequence through interval decoding, and then recovers the residual, amplitude scaling factor, DC offset, and phase offset based on reversible integer inverse mapping and differential inverse transform.

[0060] For the (i)th cycle, according to the recovered... Generate reference sequence And based on the recovered and Constructing predicted values: Combined with the recovered residuals Reconstruct the original cycle data: If a segmented reference is used, each segment can independently complete waveform reconstruction in the manner described above, thereby achieving lossless recovery of the original waveform data.

[0061] Optional implementation methods and mode switching mechanisms are as follows: In different application scenarios, this invention can switch between multiple encoding modes based on device computing power, waveform stability, and compression targets. The encoding modes include at least: 1. Basic mode: Only performs cycle division, linear prediction, residual coding, and frame encapsulation; 2. Enhanced Mode: Adds phase pre-alignment to the basic mode; 3. High-compression mode: Further differential coding and interval coding are performed on the parameter sequence and residual sequence based on the enhanced mode; 4. Long sequence mode: Introduce a segmented baseline strategy on the high compression mode.

[0062] The mode selection can be determined based on at least one of the following conditions: Does the peak value of the cross-correlation exceed the preset threshold? Does the mean square error between the current cycle and the previous cycle exceed a preset threshold? Is the maximum absolute value of the residual after the first difference significantly smaller than the maximum absolute value of the original residual? Does the current sequence length exceed the preset segmentation threshold?

[0063] Through a mode switching mechanism, this invention can balance compression ratio, real-time performance, and implementation complexity under different hardware resources and operating conditions.

[0064] Compared with the prior art, the present invention has at least the following beneficial effects: 1. Significantly improved compression ratio: This invention fully utilizes the correlation between adjacent cycles of power frequency waveforms and significantly reduces the dynamic range of prediction residuals through phase pre-alignment, amplitude scaling compensation, and DC offset compensation. Compared with general lossless compression algorithms, it can achieve a higher compression ratio in frequency waveform data scenarios.

[0065] 2. Adaptable to asynchronous sampling and frequency fluctuation conditions: By determining the phase offset between adjacent cycles through cross-correlation and performing phase pre-alignment before prediction, the phase mismatch error under asynchronous sampling, non-phase-locked sampling, and frequency fluctuation conditions can be effectively reduced, thereby improving compression stability.

[0066] 3. Adaptable to various non-stationary scenarios: By jointly estimating the amplitude scaling factor and DC offset, this invention can simultaneously characterize the inter-cycle amplitude variation and DC drift. Compared with methods that only use proportional prediction or simple difference, it can still maintain better compression performance in sag, gradual variation, offset and disturbance scenarios.

[0067] 4. High encoding efficiency and completely reversible: By performing reversible integer mapping, differential coding, and interval coding on the residuals and parameter sequences, and by adaptively reducing representation redundancy through block-level bit width, coding efficiency is improved while ensuring that the original data can be fully recovered.

[0068] 5. Suitable for embedded real-time implementation: This invention supports both least-squares exact estimation and simplified estimation based on mean or RMS, and can be flexibly configured according to the processing capabilities of smart IoT meters or edge terminals to meet the requirements of real-time compression and real-time decoding.

[0069] 6. Self-describing frames facilitate project integration: By carrying all the side information required for decoding within the frame, this invention supports independent decoding, breakpoint resumption, segmented recovery, and transparent transmission between systems, making it highly applicable to engineering projects.

[0070] The following is a set of typical power frequency data, such as Figure 8 The following example illustrates the specific implementation process of the present invention.

[0071] (1) Test data conditions: Sampling method: Asynchronous equal-interval sampling, without phase-locked loop; Recorded data: Waveform data of a single-channel voltage cycle; Weekly wave points: 256; Number of cycles: 75 cycles; Signal characteristics: Amplitude sag, DC drift, Gaussian noise, and frequency fluctuations superimposed on the fundamental sinusoidal signal; During the sag, the amplitude drops to 0.5 times the rated value, the DC drift slope is about 0.002, the noise standard deviation is about 0.02, and the frequency fluctuation range is about ±0.05Hz.

[0072] (2) Encoding process: 1. Cyclic boundary detection: Zero-crossing detection is performed on the continuous sampling sequence to determine the starting position of each cycle and the corresponding sampling point sequence.

[0073] 2. Phase pre-alignment: For the (i)th cycle and the (i-1)th cycle, calculate the cross-correlation function, find the offset (δi) corresponding to the maximum correlation position within the preset search range, and construct the phase-aligned reference cycle (x~i-1(j)) accordingly.

[0074] 3. Determining linear prediction parameters: The amplitude scaling factor (ai) and DC offset (bi) are obtained using the least squares method, and a prediction model is established: x^i(j)=aix~i-1(j)+bi 4. Residual generation: Calculate the current cycle residual: ri(j)=xi(j)-aix~i-1(j)-bi 5. Adaptive processing of parameter differencing and residuals: Cyclic direction differences are performed on (ai), (bi), and (δi) respectively; the residual sequence is judged whether to perform first-order difference based on its smoothness of change, so as to reduce the coding dynamic range.

[0075] 6. Reversible integer mappings and range encoding: The maximum absolute value of the integer sequence to be encoded is counted by cycle or block to determine the minimum lossless bit width at the block level, and then the signed integers are mapped to non-negative integers before interval encoding is implemented.

[0076] 7. Frame encapsulation: The synchronization word, total length, version number, mode identifier, number of cycles, number of points per cycle, segmentation information, phase offset information, bit width information, and compressed data stream are encapsulated into a self-describing data frame.

[0077] (3) Decoding process: 1. Scan the synchronization word and complete frame synchronization; 2. Parse the frame header and mode-related headers to obtain the number of cycles, segment information, bit width information, and compressed bitstream; 3. Perform interval decoding on the compressed bitstream to recover the residual sequence, parameter difference sequence, and phase offset information; 4. Recover (ai), (bi), and (δi) corresponding to each cycle through inverse differential transform; 5. Generate a reference cycle (x~i-1(j)) based on (δi) and construct the predicted value: x^i(j)=aix~i-1(j)+bi 6. Recover the original cycle by combining the residuals: xi(j) = x^i(j) + ri(j) Under the aforementioned test data conditions, this invention compressed 75 consecutive cycles of data. After decoding, the recovered data was consistent with the original data point by point, verifying that this invention can achieve lossless compression. Compared with general compression methods that do not utilize the characteristics of cycle structure, this invention significantly improves the compression ratio for this type of data; it also maintains relatively stable compression performance in scenarios containing sags, DC drift, and frequency fluctuations.

[0078] Example 2: Furthermore, this invention also proposes a lossless compression system 200 for the weekly waveform data of smart IoT meters, such as... Figure 9 As shown, it includes: The first encoding unit 201 is used to collect the original frequency of the smart IoT meter, perform frequency boundary detection and frequency division on the collected original frequency, perform phase pre-alignment of adjacent frequencies on the divided frequencies to determine the amplitude scaling factor and DC offset, and perform linear prediction based on the amplitude scaling factor and DC offset to obtain the residual and parameter sequence. The second coding unit 202 is used to perform a reversible transformation on the predicted residual and parameter sequence to obtain differential coding and / or segmented reference coding of the residual and parameters, perform interval coding based on a statistical model on the differential coding and / or segmented reference coding, and perform self-describing transmission frame encapsulation based on the interval coding. The decoding unit 203 is used to perform synchronization word detection and frame synchronization processing on the transmission frame, parse the frame header and side information, obtain the interval code based on the frame header and side information, perform integer inverse mapping and parameter recovery based on the interval code, and reconstruct the parameter sequence by segment or by cycle. Based on linear prediction, the original cycle is recovered cycle by cycle according to the parameter sequence and residual.

[0079] Among them, after performing cycle boundary detection and cycle segmentation on the acquired raw cycles, the first cycle is obtained. The following is a sequence of one-cycle data: in, For the first Number of sampling points in a cycle data sequence is the original sampled value of the j-th sampling point in the i-th cycle, where j is the sampling point number within the cycle.

[0080] Phase pre-alignment includes: Regarding the first The cross-correlation function between the previous week's data sequence was calculated using the following method: in, For candidate offsets, To ensure a consistent number of sampling points per cycle during processing, The (i-1)th cycle, that is, the previous cycle or reference cycle of the current cycle. ; The offset corresponding to the maximum value of the cross-correlation function is taken as follows: According to the offset The phase-aligned reference sequence generated for the previous cycle is as follows: in, For interpolation operations; when When the value is an integer, the integer point is shifted; when... When the integer is not an integer, linear interpolation, spline interpolation, or polynomial interpolation is used.

[0081] Among them, taking advantage of the high similarity in shape and the slow change in amplitude and DC component between adjacent cycles, the first... A linear prediction model for linear prediction is established based on a series of cycle data, as follows: in: For the first The first cycle Predicted values ​​for each sampling point For the first The amplitude scaling factor of each cycle relative to the previous cycle. For the first The DC offset of each cycle relative to the previous cycle. This is the reference cycle after phase pre-alignment; The prediction residual is defined as: .

[0082] The amplitude scaling factor and DC offset are jointly estimated using the least squares method, as shown in the following formula: Its closed-form solution is: in: in, For the first The amplitude scaling factor of each cycle relative to the previous cycle. For the first The DC offset of each cycle relative to the previous cycle. For the reference cycle after phase pre-alignment, The reference cycle mean after phase alignment. Let be the mean of the i-th cycle. This is the amplitude scaling factor. This is the DC offset. .

[0083] Among them, reversible transformations include: Differentiating the amplitude scaling factor, DC offset, and phase offset along the cycle number direction yields: For the residual sequence When the changes between adjacent sampling points are gradual, perform a first-order difference: Whether to perform differencing is adaptively selected based on the residual statistical characteristics, if the following conditions are met: in, If a preset threshold is used, then the differencing sequence is employed. Encode; otherwise, directly... Encode it.

[0084] In this process, the interval coding compresses the non-negative integer sequence after invertible mapping using a statistical model-based interval coding method. The interval coding uses a static model and / or an adaptive model for encoding.

[0085] The segmentation reference strategy before encapsulation of the transmission frame includes: dividing the total sequence into several independent segments, each segment containing K cycles, the first cycle in each segment being used as the reference cycle for raw storage or low-compression encoding, and the remaining cycles in the segment being used as references to perform phase pre-alignment and linear prediction, with each segment being independently encoded and independently decoded.

[0086] The transmission frame includes: synchronization word, version number, mode identifier, total length, number of cycles, number of points per cycle or point description information, segmentation information, parameter representation format, phase offset information, bit width information or block maximum value information, compressed data stream, and checksum.

[0087] The implementation of this invention has significantly improved the efficiency of encoding and decoding of cycle data.

[0088] Example 3: Based on the same inventive concept, this invention also provides a computer device, which includes a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in the computer storage medium to implement corresponding method flows or corresponding functions, thereby implementing the steps of the methods in the above embodiments.

[0089] Example 4: Based on the same inventive concept, this invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the steps of the method in the above embodiments.

[0090] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0091] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0092] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0093] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0094] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0095] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A lossless compression method for cycle waveform data of smart IoT meters, characterized in that, include: The system acquires raw frequency data from smart IoT meters, performs frequency boundary detection and frequency segmentation on the acquired raw frequency data, pre-aligns the phases of adjacent frequencies on the segmented frequencies to determine the amplitude scaling factor and DC offset, and performs linear prediction based on the amplitude scaling factor and DC offset to obtain the residual and parameter sequence. A reversible transformation is performed on the predicted residual and parameter sequence to obtain differential coding and / or segmented reference coding of the residual and parameters. The differential coding and / or segmented reference coding are then subjected to interval coding based on a statistical model. A self-describing transmission frame encapsulation is performed based on the interval coding. Synchronization word detection and frame synchronization processing are performed on the transmission frame to parse the frame header and side information. Based on the frame header and side information, the interval code is obtained. Based on the interval code, integer inverse mapping and parameter recovery are performed, and the parameter sequence is reconstructed by segment or by cycle. Based on linear prediction, the original cycle is recovered cycle by cycle according to the parameter sequence and residual.

2. The lossless compression method according to claim 1, characterized in that, After performing cycle boundary detection and cycle segmentation on the acquired raw cycles, the first cycle is obtained. The following is a sequence of one-cycle data: in, For the first Number of sampling points in a cycle data sequence is the original sampled value of the j-th sampling point in the i-th cycle, where j is the sampling point number within the cycle.

3. The lossless compression method according to claim 1, characterized in that, The phase pre-alignment includes: Regarding the first The cross-correlation function between the previous week's data sequence was calculated using the following method: in, For candidate offsets, To ensure a consistent number of sampling points per cycle during processing, The (i-1)th cycle, that is, the previous cycle or reference cycle of the current cycle. ; The offset corresponding to the maximum value of the cross-correlation function is taken as follows: According to the offset The phase-aligned reference sequence generated for the previous cycle is as follows: in, For interpolation operations; when When the value is an integer, the integer point is shifted; when... When the integer is not an integer, linear interpolation, spline interpolation, or polynomial interpolation is used.

4. The lossless compression method according to claim 1, characterized in that, Taking advantage of the high similarity in shape and the slow changes in amplitude and DC component between adjacent cycles, the first... A linear prediction model for linear prediction is established based on a series of cycle data, as follows: in: For the first The first cycle Predicted values ​​for each sampling point For the first The amplitude scaling factor of each cycle relative to the previous cycle. For the first The DC offset of each cycle relative to the previous cycle. This is the reference cycle after phase pre-alignment; The prediction residual is defined as: 。 5. The lossless compression method according to claim 1, characterized in that, The amplitude scaling factor and DC offset are jointly estimated using the least squares method, as shown in the following formula: Its closed-form solution is: in: in, For the first The amplitude scaling factor of each cycle relative to the previous cycle. For the first The DC offset of each cycle relative to the previous cycle. For the reference cycle after phase pre-alignment, The reference cycle mean after phase alignment. Let be the mean of the i-th cycle. This is the amplitude scaling factor. This is the DC offset. .

6. The lossless compression method according to claim 1, characterized in that, The reversible transformation includes: Differentiating the amplitude scaling factor, DC offset, and phase offset along the cycle number direction yields: For residual sequence When the changes between adjacent sampling points are gradual, perform a first-order difference: Whether to perform differencing is adaptively selected based on the residual statistical characteristics, if the following conditions are met: in, If a preset threshold is used, then the differencing sequence is employed. Encode; otherwise, directly... Encode it.

7. The lossless compression method according to claim 1, characterized in that, The interval coding compresses the non-negative integer sequence after invertible mapping using a statistical model-based interval coding, which employs a static model and / or an adaptive model for encoding.

8. The lossless compression method according to claim 1, characterized in that, The segmentation reference strategy before encapsulation of the transmission frame includes: dividing the total sequence into several independent segments, each segment containing K cycles, the first cycle in each segment being used as the reference cycle for raw storage or low-compression encoding, and the remaining cycles in the segment being used as references to perform phase pre-alignment and linear prediction, with each segment being independently encoded and decoded.

9. The lossless compression method according to claim 1, characterized in that, The transmission frame includes: a synchronization word, a version number, a mode identifier, a total length, a number of cycles, a number of points per cycle or a point description, segmentation information, a parameter representation format, phase offset information, bit width information or a maximum block value information, a compressed data stream, and a checksum.

10. A lossless compression system for cycle waveform data of smart IoT meters, characterized in that, include: The first encoding unit is used to acquire the original frequency of the smart IoT meter, perform frequency boundary detection and frequency division on the acquired original frequency, perform phase pre-alignment of adjacent frequencies on the divided frequencies to determine the amplitude scaling factor and DC offset, and perform linear prediction based on the amplitude scaling factor and DC offset to obtain the residual and parameter sequence. The second coding unit is used to perform a reversible transformation on the predicted residual and parameter sequence to obtain differential coding and / or segmented reference coding of the residual and parameters, perform interval coding based on a statistical model on the differential coding and / or segmented reference coding, and perform self-describing transmission frame encapsulation based on the interval coding. The decoding unit is used to perform synchronization word detection and frame synchronization processing on the transmitted frame, parse the frame header and side information, obtain the interval code based on the frame header and side information, perform integer inverse mapping and parameter recovery based on the interval code, and reconstruct the parameter sequence by segment or by cycle. Based on linear prediction, the original cycle is recovered cycle by cycle according to the parameter sequence and residual.

11. The lossless compression system according to claim 10, characterized in that, After performing cycle boundary detection and cycle segmentation on the acquired raw cycles, the first cycle is obtained. The following is a sequence of one-cycle data: in, For the first The number of sampling points in a 1-cycle data sequence, where j is the sampling point number within a cycle.

12. The lossless compression system according to claim 10, characterized in that, The phase pre-alignment includes: Regarding the first The cross-correlation function between the previous week's data sequence was calculated using the following method: in, For candidate offsets, To ensure a consistent number of sampling points per cycle during processing, The (i-1)th cycle, that is, the previous cycle or reference cycle of the current cycle. ; The offset corresponding to the maximum value of the cross-correlation function is taken as follows: According to the offset The phase-aligned reference sequence generated for the previous cycle is as follows: in, For interpolation operations; when When the value is an integer, the integer point is shifted; when... When the integer is not an integer, linear interpolation, spline interpolation, or polynomial interpolation is used.

13. The lossless compression system according to claim 10, characterized in that, Taking advantage of the high similarity in shape and the slow changes in amplitude and DC component between adjacent cycles, the first... A linear prediction model for linear prediction is established based on a series of cycle data, as follows: in: For the first The first cycle Predicted values ​​for each sampling point For the first The amplitude scaling factor of each cycle relative to the previous cycle. For the first The DC offset of each cycle relative to the previous cycle. This is the reference cycle after phase pre-alignment; The prediction residual is defined as: 。 14. The lossless compression system according to claim 10, characterized in that, The amplitude scaling factor and DC offset are jointly estimated using the least squares method, as shown in the following formula: Its closed-form solution is: in: in, For the first The amplitude scaling factor of each cycle relative to the previous cycle. For the first The DC offset of each cycle relative to the previous cycle. For the reference cycle after phase pre-alignment, The reference cycle mean after phase alignment. Let be the mean of the i-th cycle. This is the amplitude scaling factor. This is the DC offset. .

15. The lossless compression system according to claim 10, characterized in that, The reversible transformation includes: Differentiating the amplitude scaling factor, DC offset, and phase offset along the cycle number direction yields: For residual sequence When the changes between adjacent sampling points are gradual, perform a first-order difference: Whether to perform differencing is adaptively selected based on the residual statistical characteristics, if the following conditions are met: in, If a preset threshold is used, then the differencing sequence is employed. Encode; otherwise, directly... Encode it.

16. The lossless compression system according to claim 10, characterized in that, The interval coding compresses the non-negative integer sequence after invertible mapping using a statistical model-based interval coding, which employs a static model and / or an adaptive model for encoding.

17. The lossless compression system according to claim 10, characterized in that, The segmentation reference strategy before encapsulation of the transmission frame includes: dividing the total sequence into several independent segments, each segment containing K cycles, the first cycle in each segment being used as the reference cycle for raw storage or low-compression encoding, and the remaining cycles in the segment being used as references to perform phase pre-alignment and linear prediction, with each segment being independently encoded and decoded.

18. The lossless compression system according to claim 10, characterized in that, The transmission frame includes: a synchronization word, a version number, a mode identifier, a total length, a number of cycles, a number of points per cycle or a point description, segmentation information, a parameter representation format, phase offset information, bit width information or a maximum block value information, a compressed data stream, and a checksum.

19. A computer device, characterized in that, include: One or more processors; A processor is used to execute one or more programs; When the one or more programs are executed by the one or more processors, the method described in any one of claims 1-9 is implemented.

20. A computer-readable storage medium, characterized in that, It contains a computer program, which, when executed, implements the method as described in any one of claims 1-9.

Images