Distributed photovoltaic data compression method and system based on entropy coding hybrid mode

Abstract

The present invention discloses a distributed photovoltaic data compression method and system based on an entropy coding hybrid mode, using Symlet wavelet transform to perform multi-scale decomposition on pre-processed data, applying a fan algorithm to dynamically compress data for generating wavelet coefficients, obtaining a first fan algorithm compressed data stream; decompressing the first fan algorithm compressed data stream through a fan algorithm to obtain a reconstructed signal; calculating the difference between the wavelet coefficient and the reconstructed signal to obtain a residual; applying Huffman coding to further compress the residual to obtain a residual data stream, integrating the first fan algorithm compressed data stream and the residual data stream to form a hybrid data stream, applying a lossless compression algorithm to the hybrid data stream, and completing efficient compression of photovoltaic data. The present invention achieves efficient compression of photovoltaic data while maintaining the accuracy and integrity of the data.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing in photovoltaic systems, and in particular relates to a distributed photovoltaic data intelligent compression method based on an entropy coding hybrid mode for improving data transmission and storage efficiency and related equipment. Background Art

[0002] As the world is rapidly moving towards intelligence and interconnection, IoT technology plays a key role in distributed photovoltaic power systems, supporting more efficient, reliable and sustainable power supply by collecting and transmitting key time series data. A major challenge is the huge amount of data generated by IoT devices, which places extremely high demands on storage and bandwidth resources. To this end, the use of onboard signal processing and data compression algorithms is an effective strategy to alleviate this problem.

[0003] In particular, by using a hybrid mode compression method based on entropy coding, the present invention significantly reduces the amount of data that needs to be transmitted and stored. This hybrid mode takes advantage of the efficiency of lossy compression and the information fidelity of lossless compression, optimizes communication energy consumption, and thus extends the battery life of the device. The key is that this method removes redundancy and noise while retaining the most important information in the data, achieving an excellent compression ratio and ensuring the availability and quality of the data.

[0004] In terms of compressed data processing, the hybrid mode based on entropy coding provides a new solution to the time series classification (TSC) problem. This compression technology makes the processed data more compact and reduces the need for preprocessing of advanced analysis techniques. At the same time, it creates conditions for subsequent data analysis and pattern recognition, making it more efficient and accurate to extract useful information from compressed data. Summary of the invention

[0005] Purpose of the invention: Aiming at the computing power, storage and energy limitations in distributed photovoltaic power systems, a distributed photovoltaic data compression method based on entropy coding hybrid mode is proposed.

[0006] Technical solution: To achieve the above purpose, the technical solution adopted by the present invention is:

[0007] A distributed photovoltaic data compression method based on entropy coding hybrid mode removes noise and extracts key features through Symlet wavelet transform. The fan algorithm is used to dynamically compress the data, and adaptive sampling is used to ensure that the reconstruction error of the intermediate samples does not exceed the preset threshold. The error threshold ε of the fan algorithm is optimized, and the optimal error threshold ε is determined through experimental analysis to balance the compression efficiency and signal fidelity. Huffman coding is applied for residual optimization compression to further reduce the data bit rate while ensuring the stability and reliability of the compression process. The specific steps include:

[0008] Step 1: preprocess the original photovoltaic data.

[0009] Step 2: Use Symlet wavelet transform to perform multi-scale decomposition on the preprocessed data to generate wavelet coefficients.

[0010] Step 3: Apply the fan algorithm to dynamically compress the data for generating wavelet coefficients to obtain the first fan algorithm compressed data stream. Apply the fan algorithm to dynamically compress the data for generating wavelet coefficients, where the first sample is permanently stored as the starting point, and the upper and lower error boundaries of subsequent samples are determined by the sample value ±ε, and the retained samples and slopes are updated by comparing the relationship between the new samples and the current slope boundary, where ε represents the optimal error threshold.

[0011] Step 4: Decompress the first sector algorithm compressed data stream using the sector algorithm to obtain a reconstructed signal. Calculate the difference between the wavelet coefficient and the reconstructed signal to obtain a residual.

[0012] Step 5: Apply Huffman coding to the residual to further compress it, and obtain a residual data stream, wherein the Huffman coding is based on a pre-constructed symbol and codeword association table, which reflects the occurrence frequency of different symbols.

[0013] Step 6: Integrate the first sector algorithm compressed data stream and the residual data stream to form a mixed data stream, apply a lossless compression algorithm to the mixed data stream, and complete the efficient compression of the photovoltaic data.

[0014] Preferably: the application of the fan algorithm includes the following steps: first, anchor the first data point S1 as the starting reference. Then, for the data point S2 that follows, calculate the two limit values ​​obtained by adding and subtracting it from ε, and use this as the boundary to draw two line segments with different slopes, recorded as U1 and L1. When the new data point S3 falls into the current slope interval, remove S2, update the slope to (U2, L2), and preferentially retain the slope pair (U1, L2) that covers a wider range of data points. If S4 exceeds this range, fix S3 and use it as a new starting point, and execute it in a loop until all samples are processed.

[0015] Preferably: Apply the fan algorithm using a recursive geometric method to implement the fan algorithm, including the following steps: Only estimate / store the most stringent slope and its projection, with the current starting sample As the starting point, The next sample is the sample currently being considered for deletion. is the end point, the current starting sample The projection point is point A. The projection point is point E, The projection point is point B, and AB is parallel to , E is on AB, maintaining the upper bound of the slope For AC, maintain the lower bound of the slope is AD, C is On the upper bound of the slope maintained The projection point on In maintaining the lower bound of the slope The projection points on the In a straight line, In maintaining the lower bound of the slope The projection point on is G, On the upper bound of the slope maintained The projection point on is F, and the calculation starts from the current starting point arrive exist The slope value at determines whether to keep .

[0016] ;

[0017] ;

[0018] ;

[0019] ;

[0020] in, is the current starting sample, is the sample currently being considered for deletion; is the next sample; and Represent the upper and lower bounds of the slope to be maintained, respectively, and are used to determine Whether it can be deleted without affecting the overall accuracy of the data; is the allowable error range; Indicates from the current starting sample to the current sample Duration of express The projection point B and On the upper bound of the slope maintained The distance between the projection points C on express The projection point B is In maintaining the lower bound of the slope The distance between the projection points D on Indicates the current starting sample The projection point A and The distance between the projection point B and Indicates the current starting sample The projection point A and The distance between the projection point E, express The projection point E and In maintaining the lower bound of the slope The distance between the projection points G on express The projection point E and On the upper bound of the slope maintained The distance between the projection points F on .

[0021] Preferred: The method of analyzing the optimal error threshold ε through experiments:

[0022] During initialization, the current starting sample S1 is permanently stored as the starting origin. Subsequently, for sample S2, its upper and lower error boundaries (S2±ε) are calculated, and two slopes (U1, L1) are determined to connect the origin S1 with the upper and lower boundary points respectively. If the amplitude of the subsequent sample S3 is within the boundary defined by the slope (U1, L1) at S3, sample S2 is eliminated. Next, the upper and lower error boundaries (S3±ε) of sample S3 are calculated, and new slopes (U2, L2) are drawn from the origin S1 to these boundary points. Compare the new and old slopes and retain the most convergent slope combination. When the amplitude of sample S4 exceeds the boundary defined by the slope (U1, L2), sample S3 is retained and used as the new origin, and the above steps are repeated for all remaining samples until the optimal error threshold ε is determined.

[0023] Preferably: the original photovoltaic data is subjected to Symlet wavelet transform to decompose it into wavelet functions of different scales and frequencies. Symlet wavelet transform is implemented through low-pass and high-pass filters. The low-pass filter is used to extract the trend component of the signal, while the high-pass filter is used to retain the details and noise information of the signal.

[0024] Preferably, the Symlet wavelet transform method:

[0025] Step 21: Select wavelet basis for transformation.

[0026] Step 22: determine the wavelet transform filter threshold, set all conversion values ​​below the wavelet transform filter threshold to zero, and retain important non-zero conversion values.

[0027] Step 23: Perform inverse discrete wavelet transform on the filtered data.

[0028] Preferably, the wavelet transform filter threshold is:

[0029]

[0030] In the formula, is the wavelet transform filter threshold, is the maximum number of decomposition layers, is the number of data sets.

[0031] Preferably: during Huffman coding, a shorter code word is assigned to a high-frequency symbol and a longer code word is assigned to a low-frequency symbol, thereby reducing the bit rate.

[0032] The present invention also provides a distributed photovoltaic data compression system based on an entropy coding hybrid mode, comprising a preprocessing unit, a Symlet wavelet transform unit, a fan algorithm compression unit, a fan algorithm decompression unit, a Huffman encoding unit, and a lossless compression unit, wherein:

[0033] The preprocessing unit is used to preprocess the original photovoltaic data.

[0034] The Symlet wavelet transform unit uses Symlet wavelet transform to perform multi-scale decomposition on the pre-processed data to generate wavelet coefficients.

[0035] The sector algorithm compression unit applies the sector algorithm to dynamically compress the data for generating wavelet coefficients to obtain a first sector algorithm compressed data stream.

[0036] The sector algorithm decompression unit is used to decompress the first sector algorithm compressed data stream through the sector algorithm to obtain a reconstructed signal. The difference between the wavelet coefficient and the reconstructed signal is calculated to obtain a residual.

[0037] The Huffman coding unit is used to apply Huffman coding to the residual to further compress it and obtain a residual data stream.

[0038] The lossless compression unit is used to integrate the first sector algorithm compressed data stream and the residual data stream to form a mixed data stream, and apply a lossless compression algorithm to the mixed data stream to complete efficient compression of photovoltaic data.

[0039] The present invention also provides an electronic device, comprising: at least one processor, at least one memory and a communication interface. The processor, memory and communication interface communicate with each other. The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the distributed photovoltaic data compression method based on the entropy coding hybrid mode according to any one of claims 1 to 8.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] The present invention combines lossy and lossless compression technologies to significantly improve data compression efficiency and reduce storage and transmission resource requirements. At the same time, the present invention effectively retains key information, removes redundancy, and optimizes data quality. At the same time, it reduces communication energy consumption, extends equipment life, simplifies the data analysis process, supports intelligent decision-making, and promotes the efficient operation and sustainable development of distributed photovoltaic systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a schematic diagram of Symlet 6 wavelet, where Figure 1 (a) is a schematic diagram of the Symlet 6 scaling function. Figure 1 (b) is a schematic diagram of Symlet 6 wavelet.

[0043] Figure 2 Schematic diagram of a fan algorithm according to an embodiment.

[0044] Figure 3 It is a schematic diagram of the fan algorithm implementation process of the embodiment.

[0045] Figure 4 It is a schematic diagram of the original data.

[0046] Figure 5 is a dynamically compressed data point with an error of 5%.

[0047] Figure 6 is a dynamically compressed data point with an error of 2%.

[0048] Figure 7 It is the residual probability histogram with ε=10 disclosed in the present invention.

[0049] Figure 8 It is a flow chart of the hybrid compression algorithm disclosed in the present invention. DETAILED DESCRIPTION

[0050] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these examples are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0051] A distributed photovoltaic data compression method based on entropy coding hybrid mode, such as Figure 8 As shown, from the coefficients obtained by wavelet transform, a path is selected for further processing. The coefficients on this path are compressed by the sector algorithm, which is an efficient lossy compression method that aims to achieve a high compression ratio while maintaining the recognizability of the signal. The sector algorithm quantizes and encodes the wavelet coefficients by analyzing the statistical characteristics of the signal to generate a compressed data stream. The wavelet coefficients on the other path are used to generate a residual signal. First, the wavelet transformed data is decompressed by the sector algorithm to obtain a reconstructed signal. Then, the difference between the original wavelet coefficients and the reconstructed signal is calculated, that is, the residual. The residual signal is then Huffman entropy coded, which is a lossless compression technology that reduces data redundancy by constructing an optimal coding table, thereby further compressing the residual data. Specifically, the following steps are included:

[0052] Step 1: Preprocess the raw photovoltaic data to remove noise and ensure signal quality.

[0053] Step 2: Use Symlet wavelet transform to perform multi-scale decomposition on the preprocessed data to generate wavelet coefficients.

[0054] This embodiment uses Symlet 6 wavelet transform for data preprocessing to capture the frequency domain characteristics of distributed photovoltaic time series data. The details are as follows:

[0055] The Symlet 6 wavelet is selected as the basic wavelet basis because of its excellent smoothness and detail capture ability, which can effectively process the complex frequency domain characteristics of distributed photovoltaic data. The Symlet wavelet is selected as the transformation basis function because of its good time-frequency locality and smoothness to retain the key characteristics of the signal.

[0056] Wavelet transform decomposition: Symlet wavelet transform is performed on the original photovoltaic data to decompose it into wavelet functions of different scales and frequencies to extract frequency domain information. The Symlet wavelet transform process is implemented through low-pass and high-pass filters. The low-pass filter is used to extract the trend component of the signal, while the high-pass filter is used to retain the details and noise information of the signal.

[0057] Knowing that for any ,but can be expressed as a series (converging to ), as shown in formula (1).

[0058] (1)

[0059] in, represents the original signal; is called the approximation coefficient; It is called detail coefficient; It is the scaling function (or parent wavelet) in the wavelet transform; it represents the low-frequency components of the signal; It is the wavelet function (or mother wavelet) in wavelet transform, which represents the high-frequency components of the signal; and are indices, indicating scale and position respectively.

[0060] In the wavelet transform process, low-pass and high-pass filters isolate the low-frequency and high-frequency components of the signal respectively. The low-pass filter extracts the trend component of the signal and filters out the high-frequency details and noise, while the high-pass filter does the opposite. It retains the details and noise information of the signal and eliminates the low-frequency trend. Symlet 6 wavelet, with its excellent smoothing and detail capture capabilities, is shown in Figure 1 The process of wavelet transform data compression is simplified into four key steps:

[0061] Step 21, select a wavelet basis for transformation: transform by selecting a suitable wavelet basis.

[0062] Step 22: determine the wavelet transform filter threshold, set all conversion values ​​below the wavelet transform filter threshold to zero, and retain important non-zero conversion values. Retain important non-zero conversion values ​​for subsequent processing.

[0063] By setting the wavelet transform filter threshold, insignificant wavelet transform values ​​are set to zero and important non-zero values ​​are retained.

[0064] The setting of wavelet transform filter threshold is crucial for data compression and must be determined in advance by appropriate methods. Hard thresholding is often used in data compression. It retains or discards wavelet coefficients by hard cutting, providing a clear boundary for the compression process. The global value of is as follows:

[0065]

[0066] In the formula, is the wavelet transform filter threshold, is the maximum number of decomposition layers, is the number of data sets.

[0067] Step 23: Perform inverse discrete wavelet transform (IDWT) on the filtered data.

[0068] Information entropy is a measure used to describe the degree of irregularity of information. The smaller the information entropy, the less information content and the better the compression effect. The information entropy under different thresholds is calculated using the coefficient signal after wavelet transformation, and the threshold with the smallest information entropy is selected as the optimal threshold.

[0069] Through the above steps, the present invention realizes efficient preprocessing of distributed photovoltaic time series data, provides high-quality input data for the subsequent hybrid compression algorithm, and thus ensures the high efficiency and reliability of the overall data compression method.

[0070] Step 3, applying the fan algorithm to dynamically compress the data for generating the wavelet coefficients, and obtaining the first fan algorithm compressed data stream.

[0071] This embodiment uses a fan-shaped algorithm to perform adaptive sub-sampling processing on the distributed photovoltaic system data after wavelet transformation to achieve efficient data compression. The details are as follows:

[0072] The fan-shaped algorithm is applied to dynamically compress the preprocessed data, where the first sample is permanently stored as the starting point, and the upper and lower error boundaries of subsequent samples are determined by the sample value ±ε. The retained samples and slopes are updated by comparing the relationship between the new sample and the current slope boundary. ε represents the optimal error threshold. The optimal error threshold ε is analyzed experimentally.

[0073] The fan algorithm is an adaptive subsampling algorithm for distributed photovoltaic system data. Its core is to build a straight line as long as possible between the initial data point and the terminal data point to ensure that the reconstruction error of all data points in between is strictly limited to below the preset maximum error threshold ε. Figure 2 As shown, applying the fan algorithm includes the following steps:

[0074] First, anchor the first data point S1 as the starting reference. Then, for the data point S2 that follows, calculate the two limit values ​​obtained by adding and subtracting it from the optimal error threshold ε, and use this as the boundary to draw two line segments with different slopes, recorded as U1 and L1 (draw two line segments with slopes of U1 and L1). When the new data point S3 falls into the current slope interval, remove S2, update the slope to (U2, L2), and retain the slope pair (U1, L2) that covers a wider range of data points. If S4 exceeds this range, fix S3 and use it as a new starting point, and execute it in a loop until all samples are processed.

[0075] With this strategy, storage requirements are significantly reduced, and only the time interval between adjacent retention points is stored as the field length.

[0076] Step 4: Decompress the first sector algorithm compressed data stream using the sector algorithm to obtain a reconstructed signal. Calculate the difference between the wavelet coefficient and the reconstructed signal to obtain a residual.

[0077] The decompression process relies on linear interpolation.

[0078] The direct implementation of the fan algorithm usually requires storing the slopes of all data points (from the last starting point to the current sample) and calculating their projections on the current sample. In order to simplify the calculation and reduce the implementation complexity, this embodiment adopts a recursive geometric method to implement the fan algorithm, and only estimates / stores the most stringent slope and its projection.

[0079] like Figure 3 As shown, is the current starting sample, is the sample currently being considered for deletion, is the next sample, and "Len" is the time from the current starting sample to the current sample duration. In order to decide whether to keep , you need to calculate from the current starting point arrive exist The slope value at and Therefore, the recursive geometric method to implement the fan algorithm includes the following steps:

[0080] Only estimate / store the strictest slope and its projection, starting with the current starting sample As the starting point, The next sample is the sample currently being considered for deletion. is the end point, the current starting sample The projection point is point A. The projection point is point E, The projection point is point B, and AB is parallel to , E is on AB, maintaining the upper bound of the slope For AC, maintain the lower bound of the slope is AD, C is On the upper bound of the slope maintained The projection point on In maintaining the lower bound of the slope The projection points on the In a straight line, In maintaining the lower bound of the slope The projection point on is G, On the upper bound of the slope maintained The projection point on is F, and the calculation starts from the current starting point arrive exist The slope value at determines whether to keep .

[0081] Using right triangles ABC and ABD, by calculating the distance from the current starting point S1 to exist The slope value at and Decide whether to keep , calculate the slope value according to the following formula.

[0082] ;

[0083] ;

[0084] ;

[0085] ;

[0086] in, is the current starting sample, is the sample currently being considered for deletion; is the next sample; and Represent the upper and lower bounds of the slope to be maintained, respectively, and are used to determine Whether it can be deleted without affecting the overall accuracy of the data; is the allowable error range; Indicates from the current starting sample to the current sample Duration of express The projection point B and On the upper bound of the slope maintained The distance between the projection points C on express The projection point B is In maintaining the lower bound of the slope The distance between the projection points D on Indicates the current starting sample The projection point A and The distance between the projection point B and Indicates the current starting sample The projection point A and The distance between the projection point E, express The projection point E and In maintaining the lower bound of the slope The distance between the projection points G on express The projection point E and On the upper bound of the slope maintained The distance between the projection points F on .

[0087] Through this series of steps, this embodiment achieves efficient compression and decompression of distributed photovoltaic system data while ensuring the accuracy and integrity of the data.

[0088] The fan algorithm is an adaptive sampling method. Its core is to draw a straight line as long as possible from the starting sample to the ending sample to ensure that the reconstruction error of the intermediate samples does not exceed the preset maximum error threshold ε. The method of analyzing the optimal error threshold ε through experiments:

[0089] In order to achieve a balance between efficient data compression and signal fidelity of the sector algorithm, this step aims to determine the optimal error threshold ε through experimental analysis. The details are as follows:

[0090] During initialization, the current starting sample S1 is permanently stored as the starting origin. Subsequently, for sample S2, its upper and lower error boundaries (S2±ε) are calculated, and two slopes (U1, L1) are determined to connect the origin S1 with the upper and lower boundary points. If the amplitude of the subsequent sample S3 is within the boundaries defined by the slope (U1, L1) at S3, sample S2 is eliminated. Next, the upper and lower error boundaries (S3±ε) of sample S3 are calculated, and new slopes (U2, L2) are drawn from the origin S1 to these boundary points. Compare the new and old slopes, and retain the most convergent slope combination, such as (U1, L2) in the example. When the amplitude of sample S4 exceeds the boundary defined by the slope (U1, L2), sample S3 is retained and used as the new origin, and the above steps are repeated for all remaining samples until the optimal error threshold ε is determined. The time interval between two consecutive samples retained by the algorithm is stored as the field length of each retained sample. The decompression process is implemented by simple linear interpolation.

[0091] In the Matlab environment, the fan-shaped algorithm is applied to the distributed photovoltaic data to perform compression and decompression experiments, and different ε values ​​are tested.

[0092] When initializing the algorithm, the first sample S1 is fixed as the starting point and stored permanently.

[0093] For each subsequent sample, calculate its upper and lower error boundaries (sample value ± ε), and determine the corresponding slope (U, L), connecting the origin and the boundary points.

[0094] If the amplitude of the new sample is within the boundaries defined by the current slope, the previous sample is discarded and the slope is updated.

[0095] When a new sample exceeds the current slope boundary, the previous sample is retained as the new origin and the above process is repeated.

[0096] The time interval between two consecutive retained samples is stored as the field length, which is used for linear interpolation during decompression.

[0097] Analyze the experimental results and observe the residual error distribution. Its maximum value should be limited by the selected ε value.

[0098] By comparing the compression efficiency and signal fidelity under different ε values, the ε value that provides the best performance is determined.

[0099] Through this step, an optimal error threshold ε can be selected for the sector algorithm to ensure that the storage requirements can be effectively reduced while maintaining the accuracy and integrity of the data during the data compression process.

[0100] like Figure 4-Figure 6 As shown in Figure 2. In Matlab, the fan-shaped algorithm is applied to distributed photovoltaic data. The experimental results show that the residual error is distributed around the zero point, and its maximum value is limited by the selected ε value (that is, the residual after compression and decompression is zero-mean distributed, and its peak value is regulated by the ε value). Therefore, this step aims to determine the optimal error threshold ε that can balance compression efficiency and signal fidelity through experimental analysis.

[0101] Step 5: Apply Huffman coding to the residual to further compress it, and obtain a residual data stream, wherein the Huffman coding is based on a pre-constructed symbol and codeword association table, which reflects the occurrence frequency of different symbols.

[0102] In order to further improve the data compression efficiency and ensure the stability of the compression process, this step uses Huffman coding to optimize the residual error. Huffman coding is a widely used data compression technology. Its core idea is to associate the most frequently occurring symbols with shorter codewords, and associate the less frequently occurring symbols with longer codewords, thereby achieving the purpose of reducing the total bit rate. Since most of the residual error samples are concentrated near the zero point (such as Figure 7 As shown in Figure 2, this method will further reduce the bit rate of the data. The symbol-codeword association table is pre-built based on the statistical data set. Considering the limited number of error symbols in this scenario (i.e., defined by ε), it can be simply implemented by looking up the table. The implementation of this step not only optimizes the efficiency of data compression, but also ensures the stability and reliability of the data compression process.

[0103] The details are as follows:

[0104] Huffman coding is a coding technology based on symbol frequency. It reduces the bit rate by assigning shorter code words to high-frequency symbols and longer code words to low-frequency symbols.

[0105] Given that the residual error samples are concentrated around zero (e.g. Figure 7 ), Huffman coding is particularly effective in this scenario and can significantly reduce the number of bits required for data.

[0106] A symbol-code word association table is pre-built. The table is generated based on a statistical data set and reflects the frequency of occurrence of different symbols.

[0107] In the current application scenario, since the number of symbols of the residual error is limited and is defined by the ε value, encoding can be quickly implemented by means of a lookup table.

[0108] The implementation of Huffman coding not only improves the efficiency of data compression, but also enhances the stability and reliability of the data compression process.

[0109] Through this step, Huffman coding is used to effectively optimize and compress the residual error, thereby achieving a higher data compression ratio while ensuring data accuracy.

[0110] Step 6: Integrate the first sector algorithm compressed data stream and the residual data stream to form a hybrid data stream, and apply a lossless compression algorithm to the hybrid data stream to achieve efficient compression of the photovoltaic data, so as to further reduce the data volume while ensuring the integrity and recoverability of the data.

[0111] The present invention realizes efficient compression of photovoltaic data while maintaining the accuracy and integrity of the data by comprehensively using wavelet transform, sector algorithm compression, error threshold optimization and Huffman coding technology.

[0112] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A distributed photovoltaic data compression method based on entropy coding hybrid mode, characterized in that: The following steps are involved: Step 1, preprocessing the original photovoltaic data; Step 2, using Symlet wavelet transform to perform multi-scale decomposition on the pre-processed data to generate wavelet coefficients; Step 3, applying the fan algorithm to dynamically compress the data for generating the wavelet coefficients, and obtaining the first fan algorithm compressed data stream; The fan algorithm is applied to dynamically compress the data of the generated wavelet coefficients, where the first sample is permanently stored as the starting point, and the upper and lower error boundaries of subsequent samples are determined by the sample value ±ε, and the retained samples and slopes are updated by comparing the relationship between the new samples and the current slope boundary, where ε represents the optimal error threshold; Applying the Sector Algorithm The recursive geometric method is used to implement the Sector Algorithm, which includes the following steps: Only the most stringent slope and its projection are estimated / stored, with the current starting sample As the starting point, The next sample is the sample currently being considered for deletion. is the end point, the current starting sample The projection point is point A. The projection point is point E, The projection point is point B, and AB is parallel to , E is on AB, maintaining the upper bound of the slope For AC, maintain the lower bound of the slope is AD, C is On the upper bound of the slope maintained The projection point on In maintaining the lower bound of the slope The projection points on the In a straight line, In maintaining the lower bound of the slope The projection point on is G, On the upper bound of the slope maintained The projection point on is F, and the calculation starts from the current starting point arrive exist The slope value at determines whether to keep ; ； ； ； ； in, is the current starting sample, is the sample currently being considered for deletion; is the next sample; represents the upper bound of the slope maintained, represents the lower bound of the slope maintained, is the allowable error range; Indicates from the current starting sample to the current sample duration, express The projection point B and On the upper bound of the slope maintained The distance between the projection points C on express The projection point B is In maintaining the lower bound of the slope The distance between the projection points D on Indicates the current starting sample The projection point A and The distance between the projection point B and Indicates the current starting sample The projection point A and The distance between the projection point E, express The projection point E and In maintaining the lower bound of the slope The distance between the projection points G on express The projection point E and On the upper bound of the slope maintained The distance between the projection points F on ; The method of analyzing the optimal error threshold ε through experiments: During initialization, the current starting sample S1 is permanently stored as the starting origin; then, for sample S2, its upper and lower error boundaries (S2±ε) are calculated, and two slopes (U1, L1) are determined to connect the origin S1 with the upper and lower boundary points respectively; if the amplitude of the subsequent sample S3 is within the boundary defined by the slope (U1, L1) at S3, sample S2 is eliminated; then, the upper and lower error boundaries (S3±ε) of sample S3 are calculated, and new slopes (U2, L2) are drawn from the origin S1 to these boundary points; the new and old slopes are compared, and the most convergent slope combination is retained; when the amplitude of sample S4 exceeds the boundary defined by the slope (U1, L2), sample S3 is retained and used as the new origin, and the above steps are repeated for all remaining samples until the optimal error threshold ε is determined; Step 4, decompressing the first sector algorithm compressed data stream by using the sector algorithm to obtain a reconstructed signal; calculating the difference between the wavelet coefficient and the reconstructed signal to obtain a residual; Step 5, applying Huffman coding to the residual to further compress it to obtain a residual data stream, wherein the Huffman coding is based on a pre-constructed symbol and codeword association table, which reflects the occurrence frequency of different symbols; Step 6: Integrate the first sector algorithm compressed data stream and the residual data stream to form a hybrid data stream, apply a lossless compression algorithm to the hybrid data stream, and complete the efficient compression of the photovoltaic data.

2. The distributed photovoltaic data compression method based on entropy coding hybrid mode according to claim 1 is characterized in that: The original photovoltaic data is subjected to Symlet wavelet transform to decompose it into wavelet functions of different scales and frequencies. The Symlet wavelet transform is implemented through low-pass and high-pass filters. The low-pass filter is used to extract the trend component of the signal, while the high-pass filter is used to retain the details and noise information of the signal.

3. The distributed photovoltaic data compression method based on entropy coding hybrid mode according to claim 2 is characterized in that: Symlet wavelet transform method: Step 21, select wavelet basis for conversion; Step 22, determine the wavelet transform filter threshold, set all conversion values ​​below the wavelet transform filter threshold to zero, and retain important non-zero conversion values; Step 23: Perform inverse discrete wavelet transform on the filtered data.

4. The distributed photovoltaic data compression method based on entropy coding hybrid mode according to claim 3 is characterized in that: Wavelet transform filter threshold: ； In the formula, is the wavelet transform filter threshold, is the maximum number of decomposition layers, is the number of data sets.

5. The distributed photovoltaic data compression method based on entropy coding hybrid mode according to claim 4 is characterized in that: When Huffman coding is used, a bit rate reduction is achieved by assigning shorter code words to high-frequency symbols and longer code words to low-frequency symbols.

6. A distributed photovoltaic data compression system based on entropy coding hybrid mode, characterized in that: It includes a preprocessing unit, a Symlet wavelet transform unit, a fan algorithm compression unit, a fan algorithm decompression unit, a Huffman coding unit, and a lossless compression unit, wherein: The preprocessing unit is used to preprocess the original photovoltaic data; The Symlet wavelet transform unit uses Symlet wavelet transform to perform multi-scale decomposition on the pre-processed data to generate wavelet coefficients; The sector algorithm compression unit applies the sector algorithm to dynamically compress the data for generating wavelet coefficients to obtain a first sector algorithm compressed data stream; The fan algorithm is applied to dynamically compress the data of the generated wavelet coefficients, where the first sample is permanently stored as the starting point, and the upper and lower error boundaries of subsequent samples are determined by the sample value ±ε, and the retained samples and slopes are updated by comparing the relationship between the new samples and the current slope boundary, where ε represents the optimal error threshold; Applying the Sector Algorithm The recursive geometric method is used to implement the Sector Algorithm, which includes the following steps: Only the most stringent slope and its projection are estimated / stored, with the current starting sample As the starting point, The next sample is the sample currently being considered for deletion. is the end point, the current starting sample The projection point is point A. The projection point is point E, The projection point is point B, and AB is parallel to , E is on AB, maintaining the upper bound of the slope For AC, maintain the lower bound of the slope is AD, C is On the upper bound of the slope maintained The projection point on In maintaining the lower bound of the slope The projection points on the In a straight line, In maintaining the lower bound of the slope The projection point on is G, On the upper bound of the slope maintained The projection point on is F, and the calculation starts from the current starting point arrive exist The slope value at determines whether to keep ; ； ； ； ； in, is the current starting sample, is the sample currently being considered for deletion; is the next sample; represents the upper bound of the slope maintained, represents the lower bound of the slope maintained, is the allowable error range; Indicates from the current starting sample to the current sample duration, express The projection point B and On the upper bound of the slope maintained The distance between the projection points C on express The projection point B is In maintaining the lower bound of the slope The distance between the projection points D on Indicates the current starting sample The projection point A and The distance between the projection point B and Indicates the current starting sample The projection point A and The distance between the projection point E, express The projection point E and In maintaining the lower bound of the slope The distance between the projection points G on express The projection point E and On the upper bound of the slope maintained The distance between the projection points F on ; The method of analyzing the optimal error threshold ε through experiments: During initialization, the current starting sample S1 is permanently stored as the starting origin; then, for sample S2, its upper and lower error boundaries (S2±ε) are calculated, and two slopes (U1, L1) are determined to connect the origin S1 with the upper and lower boundary points respectively; if the amplitude of the subsequent sample S3 is within the boundary defined by the slope (U1, L1) at S3, sample S2 is eliminated; then, the upper and lower error boundaries (S3±ε) of sample S3 are calculated, and new slopes (U2, L2) are drawn from the origin S1 to these boundary points; the new and old slopes are compared, and the most convergent slope combination is retained; when the amplitude of sample S4 exceeds the boundary defined by the slope (U1, L2), sample S3 is retained and used as the new origin, and the above steps are repeated for all remaining samples until the optimal error threshold ε is determined; The sector algorithm decompression unit is used to decompress the first sector algorithm compressed data stream through the sector algorithm to obtain a reconstructed signal; calculate the difference between the wavelet coefficient and the reconstructed signal to obtain a residual; The Huffman coding unit is used to apply Huffman coding to the residual to further compress it, so as to obtain a residual data stream; The lossless compression unit is used to integrate the first sector algorithm compressed data stream and the residual data stream to form a mixed data stream, and apply a lossless compression algorithm to the mixed data stream to complete efficient compression of photovoltaic data.

7. An electronic device, characterized in that: include: At least one processor, at least one memory and a communication interface; the processor, memory and communication interface communicate with each other; The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the distributed photovoltaic data compression method based on the entropy coding hybrid mode according to any one of claims 1 to 5.

Images