An adaptive data compression method and system for intelligent converged terminals
By acquiring the periodic characteristics of power data and dynamically adjusting the lookup buffer length, combined with fluctuation adjustment factors and adjustment sensitivity, the efficiency limitation of the traditional LZ77 algorithm in power data compression is solved, and adaptive high-efficiency data compression is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- WUXI HENGTONG ELECTRIC CO LTD
- Filing Date
- 2026-05-11
- Publication Date
- 2026-06-05
AI Technical Summary
The traditional LZ77 algorithm, which uses a fixed-length lookup buffer, is difficult to adapt to the dynamic characteristics of power data, resulting in limited compression efficiency.
By acquiring the periodic characteristics of power data, the lookup buffer length is dynamically adjusted, and adaptive data compression is achieved by combining fluctuation adjustment factors and adjustment sensitivity.
It significantly improves the matching success rate and compression rate, optimizes the use of computing resources, and achieves a balance between compression efficiency and robustness under complex working conditions.
Smart Images
Figure CN122159889A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data compression. More specifically, this invention relates to an adaptive data compression method and system for intelligent converged terminals. Background Technology
[0002] In the process of power system automation and informatization, intelligent integrated terminals need to collect and upload massive amounts of power quality data. To alleviate storage and transmission pressure, data compression technology is often used. The LZ77 algorithm, as a classic lossless compression method, is considered for this scenario due to its fast decompression speed and predictable memory requirements. However, power data has unique dynamic characteristics: on the one hand, its load curve usually exhibits significant daily and weekly regular patterns; on the other hand, affected by random fluctuations in electricity load, the data also shows obvious non-stationarity in some areas.
[0003] The traditional LZ77 algorithm uses a fixed-length search buffer (sliding window). When compressing stable and periodic power data segments, the fixed window may not be able to cover the complete periodic pattern, resulting in insufficient search range and difficulty in finding longer matching strings, thus limiting the improvement of compression efficiency. Conversely, when processing data segments with drastic fluctuations, the patterns of historical data within the fixed window differ significantly from the current data to be compressed. An excessively large window can lead to a large number of invalid matching attempts, increasing computational overhead, but resulting in a low actual matching success rate and compression benefits. Therefore, the fixed window mechanism is difficult to adaptively adapt to the dynamic characteristics of power data, restricting further optimization of compression efficiency. Summary of the Invention
[0004] To address the technical problem that the fixed window mechanism in traditional compression methods is difficult to adaptively adapt to the dynamic characteristics of power data, thus hindering further optimization of compression efficiency, this invention provides solutions in the following aspects.
[0005] In a first aspect, an adaptive data compression method for a smart converged terminal includes: Obtain the original power data sequence to be compressed, and extract the periodic features of the original power data sequence to obtain the main period length; Based on the autocorrelation and main period length of the original power data sequence, the initial lookup buffer length and forward buffer length are determined. During compression, for the forward buffer constructed from the forward buffer length, a fluctuation adjustment factor and an adjustment sensitivity are calculated to adjust the initial lookup buffer length. The fluctuation adjustment factor is calculated based on the standard deviation of the data within the forward buffer and the standard deviation of the data within the corresponding lookup buffer of the initial lookup buffer length. The adjustment sensitivity is calculated based on the matching rate of each data match during historical compression processes. The matching rate is the ratio of the matching length of a single match to the corresponding lookup buffer length. Based on the initial lookup buffer length, fluctuation adjustment factor, and adjustment sensitivity, the actual length of the current lookup buffer is calculated and adjusted so that the adjusted lookup buffer can be used to perform data matching and compression on the current forward buffer. The data matching and compression is then repeated for the next forward buffer until the compression of the entire original power data sequence is completed.
[0006] Preferably, obtaining the main cycle length includes: Calculate the autocorrelation coefficient of the original power data sequence within a preset lag time range, and plot the autocorrelation coefficient curve; The autocorrelation coefficient curve is smoothed, and the lag time corresponding to the first significant peak of the smoothed autocorrelation coefficient curve is determined as the main period length.
[0007] Preferably, obtaining the initial lookup buffer length includes: For multiple integer multiples of the preset main period length, calculate the average value of the absolute value of the autocorrelation function of the original power data sequence over each integer multiple of the main period length; From all integer multiples, select the one that maximizes the average value, multiply the main period length by the selected integer multiple, and use the result as the initial lookup buffer length.
[0008] Preferably, the data to be compressed in the original power data sequence is taken as the starting point, the window size is set to the main cycle length, and the forward buffer to be compressed is constructed according to the starting point and the window size. The lookup buffer consists of compressed data located before the forward buffer, and its length is the initial lookup buffer length.
[0009] Preferably, obtaining the volatility adjustment factor includes: Calculate the standard deviation of the data in the forward buffer and the standard deviation of the data in the lookup buffer; calculate the absolute value of the difference between the two standard deviations, and then divide it by the larger of the two standard deviations to obtain the fluctuation difference ratio. The difference between the preset fluctuation threshold and the fluctuation difference ratio is used as the fluctuation adjustment factor.
[0010] Preferably, the acquisition of the sensitivity adjustment includes: Obtain the ratio of the matching length of each completed data match to the lookup buffer length when performing that match during the historical compression process, and calculate the mean and coefficient of variation of all ratios. The adjusted sensitivity is obtained by multiplying the exponential function value of the mean of all ratios by the normalized value of the coefficient of variation.
[0011] Preferably, obtaining the actual length includes: Multiplying the volatility adjustment factor and the adjustment sensitivity yields the composite adjustment coefficient; Add 1 to the composite adjustment factor to obtain the scaling factor; The initial lookup buffer length is multiplied by the scaling factor, and the result is rounded up to obtain the actual length.
[0012] In a second aspect, an adaptive data compression system for a smart converged terminal includes: a processor and a memory, wherein the memory stores computer program instructions that, when executed by the processor, implement any of the adaptive data compression methods for smart converged terminals described above.
[0013] The beneficial effects of this invention are: 1. This invention extracts the main period by analyzing the autocorrelation of power data and sets the initial lookup buffer length to an integer multiple of this period, so that the algorithm can cover the complete repetitive data pattern at the beginning of compression, which significantly improves the matching success rate and compression rate in the initial stage and avoids the inefficiency of setting the window size based on subjective experience.
[0014] 2. This invention calculates a fluctuation adjustment factor to evaluate the fluctuation similarity between the data segment to be compressed and historical data segments in real time. When the data is stable and the similarity is high, the search window is automatically increased to explore longer matches and improve the compression ratio; when the data fluctuates drastically and the similarity is low, the window is automatically decreased to avoid invalid matches and save computational resources. This mechanism achieves precise adaptation to local features of the data.
[0015] 3. This invention introduces an adjustment sensitivity calculated based on historical matching data (mean matching rate and coefficient of variation) to intelligently fine-tune the adjustment range determined by the fluctuation adjustment factor. When historical matching is good and stable, a larger adjustment range is allowed to optimize performance; when historical matching is poor or fluctuations are large, the adjustment range is suppressed to prevent drastic changes in window length due to a single misjudgment, thus ensuring the robustness of the compression process and a steady improvement in overall efficiency.
[0016] In summary, this invention combines periodic analysis, fluctuation similarity judgment, and historical performance feedback to achieve multi-dimensional and adaptive adjustment of the lookup buffer length. This effectively overcomes the inherent defects of fixed window size in dealing with the dynamic characteristics of power data and achieves a good balance between compression efficiency and computational cost under complex operating conditions. Attached Figure Description
[0017] Figure 1 This is a flowchart of steps S1-S4 in an adaptive data compression method for an intelligent fusion terminal according to an embodiment of the present invention.
[0018] Figure 2 This is a structural block diagram of an adaptive data compression system for an intelligent fusion terminal according to an embodiment of the present invention. Detailed Implementation
[0019] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.
[0020] Reference Figure 1 An adaptive data compression method for intelligent converged terminals includes steps S1-S4, as detailed below: S1: Obtain the original power data sequence to be compressed, and extract the periodic features of the original power data sequence to obtain the main period length.
[0021] Electricity data, especially load data, exhibits significant time-series characteristics, particularly regular fluctuations such as daily and weekly periods. This periodicity stems from user electricity consumption habits, production rhythms, and changes in the natural environment, resulting in similar load curve shapes within adjacent periods. Identifying and quantifying this periodicity before data compression provides structured guidance for the compression algorithm, avoiding reliance on experience to set the initial window size and thus improving initial matching efficiency. Ignoring periodicity and directly using fixed or random windows may result in the search buffer failing to cover the complete periodic pattern, reducing the matching success rate and affecting the overall compression ratio. Therefore, accurate extraction of periodic features is a prerequisite and foundation for adaptive compression.
[0022] In one embodiment, firstly, the time length is obtained as The raw power data sequence that needs to be compressed is set with a lag time range of [value missing]. For the original power data sequence, the autocorrelation coefficient between the original power data sequence and the power data sequence after each lag time is calculated (the specific calculation of the autocorrelation coefficient is based on existing technology and will not be elaborated here). The autocorrelation coefficient curve of the original power data sequence is plotted, and smoothing is performed using a filter such as the SG filter (Savitzky-Golay filter) to suppress noise interference. Finally, the first significant peak of the smoothed autocorrelation coefficient curve, i.e. the first local maximum, is identified. The lag time corresponding to the local maximum is the main period length.
[0023] The introduction of periodic features allows the lookup buffer length to cover at least one complete cycle, significantly improving the match hit rate.
[0024] It should be noted that this embodiment of the invention processes a one-time fixed data sequence; therefore, the aforementioned main cycle length remains the value initially calculated based on the original power data sequence throughout the compression process. In practical applications, especially in terminal systems that operate for extended periods and process continuous data streams, a periodic update mechanism can be enabled. For example, the main cycle length can be recalculated and updated every 24 hours based on the latest data window, allowing the algorithm to adapt to the long-term pattern evolution of the power data.
[0025] S2: Determine the initial lookup buffer length and forward buffer length based on the autocorrelation and main cycle length of the original power data sequence.
[0026] In the LZ77 algorithm framework, the matching operation is performed between two core data windows: the lookup buffer and the forward buffer. The lookup buffer contains historical power data that has been compressed or is yet to be matched, immediately preceding the current position. It is the "historical library" for the algorithm to search for matching patterns. The forward buffer contains the raw power data that is currently to be compressed, immediately following the current position. It is the "target block" to be matched.
[0027] Because the inherent correlations in power data are often not limited to a single master period, but may persist across multiple period multiples. For example, daily load curves are not only similar every day, but may also exhibit pattern reuse on the same day of the week. Therefore, the lookup buffer should cover a sufficiently long period of historical power data to include multiple potentially correlated periods, thereby improving the efficiency of finding long matches. Meanwhile, the size of the forward buffer, acting as a "current observation window" to be compressed, directly affects the granularity of a single matching operation.
[0028] In one embodiment, a set of candidate integer multiples (such as 1, 2, 3, 4, 5) of the principal period length are preset. For each integer multiple of the principal period length, the absolute value of the autocorrelation coefficient of the original power data sequence at each lag point is calculated, and the average value of the absolute values of the autocorrelation coefficients at all lag points is calculated. Then, from all integer multiples of the principal period length, the integer multiple that maximizes the average value of the absolute value of the above autocorrelation function is selected, and the principal period length is multiplied by the selected integer multiple to obtain the initial lookup buffer length.
[0029] In one embodiment, in order to align the target of each match with the periodic structure of the power data, the length of the forward buffer is directly set to the aforementioned main period length, that is, the size of the target data block for each match operation is fixed to a complete periodic unit.
[0030] In one embodiment, after determining the initial lookup buffer length and the forward buffer length, the sliding window required for the compression process is constructed as follows: Using the starting point of the current data to be compressed in the original power data sequence as the starting point of the window, and the main period length as the window size, continuous data starting from the starting point (the number is the same as the main period length) is extracted, thus constructing the forward buffer to be compressed.
[0031] The lookup buffer consists of compressed power data located before the aforementioned forward buffer. At the start of compression, the lookup buffer is initially empty. As compression progresses, compressed data continuously fills and updates the lookup buffer, whose initial length is set to the aforementioned initial lookup buffer length.
[0032] S3: During compression, for the forward buffer constructed from the forward buffer length, calculate the fluctuation adjustment factor and adjustment sensitivity used to adjust the initial lookup buffer length.
[0033] It is important to consider that power data, within a macro-cycle, can fluctuate locally due to various random factors. A fixed-size lookup buffer is insufficient to handle such dynamic changes: in sections where power data is stable and highly similar to historical data, an excessively small lookup buffer limits the possibility of finding longer matches; in sections where power data fluctuates drastically and differs significantly from historical data, an excessively large lookup buffer leads to numerous invalid match attempts, increasing the computational burden. Therefore, the size of the lookup buffer needs to be dynamically adjusted based on the local fluctuation characteristics of the data. However, adjusting solely based on current fluctuations may lack robustness; a feedback mechanism based on historical matching performance is also necessary to smooth and constrain the adjustment range, preventing drastic oscillations in the lookup buffer size due to local anomalies and ensuring the stability of the entire compression process.
[0034] In one embodiment, the standard deviation of the data in the forward buffer constructed by S2 is calculated, the standard deviation of the data in the lookup buffer constructed by S2 is calculated, and then the absolute value of the difference between the two standard deviations is calculated. The absolute value of the difference between the two standard deviations is divided by the larger of the two standard deviations to obtain the fluctuation difference ratio.
[0035] The fluctuation difference ratio calculated above is used to assess the similarity of fluctuation patterns between the lookup buffer and the forward buffer. The closer the fluctuation difference ratio is to 0, the more similar the fluctuation levels of the two data windows (i.e., the lookup buffer and the forward buffer) are, which may stem from similar operating conditions or load patterns, and the greater the likelihood that the historical power data contains high-value matching patterns. Conversely, the closer the fluctuation difference ratio is to 1, the greater the difference between the two data windows, such as a very stable period and a period of violent fluctuation, indicating a possible sudden change in operating conditions, and a lower match between historical power data and current power data.
[0036] Furthermore, a fluctuation threshold is set, for example, 0.5 (this value is at the midpoint of the fluctuation difference ratio range, providing a robust decision-making benchmark). The difference between this fluctuation threshold and the fluctuation difference ratio calculated above is used as the fluctuation adjustment factor. When the fluctuation adjustment factor is less than 0, it indicates that the current power data has a high degree of similarity to historical power data fluctuations, and the search buffer is tended to be increased to explore longer matches. When the fluctuation adjustment factor is greater than 0, the search buffer is tended to be decreased to avoid invalid searches.
[0037] In addition, when the volatility adjustment factor is equal to 0, it indicates that the volatility difference is in a critical state, and the system will maintain the current lookup buffer length as the baseline value and make no adjustment.
[0038] In one embodiment, all records that have been matched since the start of compression are retrieved. For each match, its match length and the actual length of the lookup buffer at the time of that match are recorded.
[0039] Then, the matching rate for each match is calculated, which is the ratio of the matching length to the actual length of the lookup buffer during the matching process.
[0040] The matching rate of all historical matches is statistically analyzed, and the mean (reflecting the overall matching efficiency) and the coefficient of variation (i.e., the ratio of the standard deviation to the mean) are calculated. Then, the adjustment sensitivity is calculated based on the mean and the coefficient of variation.
[0041] The above-mentioned sensitivity adjustment can be expressed by the following formula: In the formula, To adjust the sensitivity as described above, It is an exponential function with the natural number e as its base. For normalization processing, This represents the average matching rate across all historical occurrences. The coefficient of variation is the matching rate for all historical matches.
[0042] The above The higher, The smaller the value, the better the overall matching performance, allowing for a larger adjustment range; The smaller, The smaller the value, the more adjustments are allowed; conversely, the larger the value, the higher the value. The adjustment range was limited, tending towards a conservative approach.
[0043] In summary, by calculating the fluctuation adjustment factor, the system can keenly capture the local similarity between the current data segment to be compressed and historical data, providing an immediate and directional signal for adjusting the lookup buffer length. By calculating the adjustment sensitivity, the system analyzes the central tendency and stability of past matching results, intelligently modulating the magnitude of each adjustment. The combined effect of these two mechanisms enables the lookup buffer adjustment strategy to respond proactively and flexibly to instantaneous data changes while robustly and smoothly avoiding decision-making errors caused by accidental fluctuations, achieving a balance between accuracy and robustness in the adaptive process.
[0044] S4: Based on the initial lookup buffer length, fluctuation adjustment factor, and adjustment sensitivity, calculate and adjust the actual length of the current lookup buffer so as to use the adjusted lookup buffer to perform data matching and compression on the current forward buffer, and slide to repeat data matching and compression for the next forward buffer until the compression of the entire original power data sequence is completed.
[0045] After the above operations, we have obtained the initial lookup buffer length based on the overall cycle of power data, the adjustment direction reflecting the similarity of local fluctuations (i.e., the fluctuation adjustment factor), and the adjustment strength constraint based on historical experience (i.e., the adjustment sensitivity). We need to integrate this information to calculate the optimal actual length of the lookup buffer suitable for the current compression position.
[0046] In one embodiment, the fluctuation adjustment factor and the adjustment sensitivity are multiplied to obtain the composite adjustment coefficient; 1 is added to the composite adjustment coefficient to obtain the scaling coefficient; the initial lookup buffer length calculated by S2 above is multiplied by the scaling coefficient, and the result of the multiplication is rounded up to obtain the actual length of the lookup buffer.
[0047] After obtaining the actual length of the lookup buffer, perform the following operations: In the aforementioned search buffer, the longest matching string is searched in the forward buffer with a length equal to the main period length. If the length of the found match is not less than the preset minimum matching threshold, a triple (offset, matching length, next character) is output. Considering that power data is often multi-byte floating-point numbers and has encoding overhead, the matching threshold is preferably set to 3, requiring at least 3 consecutive data points to match successfully. This parameter can be configured according to the actual data format and compression level requirements.
[0048] If no match of sufficient length is found, the first character in the forward buffer is output as the unmatched literal.
[0049] Based on the matching result (match length or literal), slide the current compression position, i.e., the starting point of the forward buffer, forward by the number of data points. Update the historical matching records for subsequent sensitivity adjustments.
[0050] Furthermore, starting from the new compression position, repeat steps S3 and S4 above to perform the next round of adaptive compression.
[0051] Repeat the above "sensing-adjusting-compressing-sliding" cycle until the entire original power data sequence is completely compressed, and output the final compressed data stream. This completes the adaptive data compression of the power smart fusion terminal.
[0052] This invention also provides an adaptive data compression system for intelligent converged terminals. For example... Figure 2 As shown, the system includes a processor and a memory, the memory storing computer program instructions, which, when executed by the processor, implement the adaptive data compression method for a smart fusion terminal according to the first aspect of the present invention.
[0053] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and will not be described in detail here.
[0054] It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept, and these all fall within the scope of protection of this invention. Therefore, the scope of protection of this patent should be determined by the appended claims.
Claims
1. An adaptive data compression method for intelligent converged terminals, characterized in that, include: Obtain the original power data sequence to be compressed, and extract the periodic features of the original power data sequence to obtain the main period length; Based on the autocorrelation and main period length of the original power data sequence, the initial lookup buffer length and forward buffer length are determined. During compression, for the forward buffer constructed from the forward buffer length, a fluctuation adjustment factor and an adjustment sensitivity are calculated to adjust the initial lookup buffer length. The fluctuation adjustment factor is calculated based on the standard deviation of the data within the forward buffer and the standard deviation of the data within the corresponding lookup buffer of the initial lookup buffer length. The adjustment sensitivity is calculated based on the matching rate of each data match during historical compression processes. The matching rate is the ratio of the matching length of a single match to the corresponding lookup buffer length. Based on the initial lookup buffer length, fluctuation adjustment factor, and adjustment sensitivity, the actual length of the current lookup buffer is calculated and adjusted so that the adjusted lookup buffer can be used to perform data matching and compression on the current forward buffer. The data matching and compression is then repeated for the next forward buffer until the compression of the entire original power data sequence is completed.
2. The adaptive data compression method for intelligent fusion terminals according to claim 1, characterized in that, The acquisition of the main cycle length includes: Calculate the autocorrelation coefficient of the original power data sequence within a preset lag time range, and plot the autocorrelation coefficient curve; The autocorrelation coefficient curve is smoothed, and the lag time corresponding to the first significant peak of the smoothed autocorrelation coefficient curve is determined as the main period length.
3. The adaptive data compression method for intelligent converged terminals according to claim 1, characterized in that, Obtaining the initial lookup buffer length includes: For multiple integer multiples of the preset main period length, calculate the average value of the absolute value of the autocorrelation function of the original power data sequence over each integer multiple of the main period length; From all integer multiples, select the one that maximizes the average value, multiply the main period length by the selected integer multiple, and use the result as the initial lookup buffer length.
4. The adaptive data compression method for intelligent fusion terminals according to claim 1, characterized in that, Starting with the data to be compressed in the original power data sequence, and setting the window size to the main cycle length, a forward buffer to be compressed is constructed based on the starting point and the window size. The lookup buffer consists of compressed data located before the forward buffer, and its length is the initial lookup buffer length.
5. The adaptive data compression method for intelligent fusion terminals according to claim 1, characterized in that, The acquisition of the volatility adjustment factor includes: Calculate the standard deviation of the data in the forward buffer and the standard deviation of the data in the lookup buffer; calculate the absolute value of the difference between the two standard deviations, and then divide it by the larger of the two standard deviations to obtain the fluctuation difference ratio. The difference between the preset fluctuation threshold and the fluctuation difference ratio is used as the fluctuation adjustment factor.
6. The adaptive data compression method for intelligent fusion terminals according to claim 1, characterized in that, The acquisition of the sensitivity adjustment includes: Obtain the ratio of the matching length of each completed data match to the lookup buffer length when performing that match during the historical compression process, and calculate the mean and coefficient of variation of all ratios. The adjusted sensitivity is obtained by multiplying the exponential function value of the mean of all ratios by the normalized value of the coefficient of variation.
7. The adaptive data compression method for intelligent fusion terminals according to claim 1, characterized in that, The acquisition of the actual length includes: Multiplying the volatility adjustment factor and the adjustment sensitivity yields the composite adjustment coefficient; Add 1 to the composite adjustment factor to obtain the scaling factor; The initial lookup buffer length is multiplied by the scaling factor, and the result is rounded up to obtain the actual length.
8. An adaptive data compression system for intelligent converged terminals, characterized in that, include: A processor and a memory, the memory storing computer program instructions that, when executed by the processor, implement the adaptive data compression method for a smart converged terminal according to any one of claims 1-7.