A trusted measurement data storage method based on blockchain technology
Through adaptive multi-order Hoffman coding, the encoding conflict problem caused by similar frequencies in trusted metrological data storage on the blockchain is solved, the compression efficiency and consistency are improved, and more efficient data storage is achieved.
Patent Information
- Application Number
- CN202411179172.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-27
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-27
AI Technical Summary
Trusted metered data on blockchain increases storage costs over time. Traditional Hoffman encodings are prone to coding conflicts when data frequencies are close, resulting in large differences in coding lengths and affecting compression efficiency.
Adaptive multi-order Hoffman coding is adopted to dynamically adjust the encoding order by analyzing the similarity of data characteristics and frequency, reducing encoding conflicts and improving compression efficiency.
It effectively avoids the problem of large differences in encoding lengths when data frequencies are similar, reduces the occurrence of encoding conflicts, and improves overall compression efficiency and coding consistency.
Smart Images

Figure CN118713680B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of electrical digital data processing, and in particular relates to a trusted metering data storage method based on blockchain technology. Background Art
[0002] Blockchain technology has broad application prospects in the field of trusted metering data storage, especially in scenarios involving multiple parties, data cannot be tampered with, and security requirements are high, such as the energy industry, medical industry, supply chain management, etc. As the importance of trusted metering data in various industries becomes increasingly prominent, the requirements of various industries for data storage are also becoming increasingly stringent.
[0003] Trusted metering data on blockchains typically grows over time, leading to increased storage costs. If long-term storage of large amounts of data poses a challenge to the economic feasibility of the application, and because trusted metering data needs to ensure its integrity and accuracy, it can be compressed through Huffman coding as a way to reduce storage costs. However, when traditional Huffman coding compresses data, when the frequencies of different symbols in a single Huffman table are very close, coding conflicts may occur, that is, some codes are longer than others. Summary of the invention
[0004] The purpose of the present invention is to solve the shortcomings existing in the prior art and propose a trusted metering data storage method based on blockchain technology. This method can effectively avoid the problem of large differences in coding lengths when data frequencies are similar, and reduce the occurrence of coding conflicts.
[0005] To achieve the above object, the present invention provides the following technical solutions:
[0006] A trusted measurement data storage method based on blockchain technology comprises the following steps:
[0007] S1. Obtaining trusted measurement data based on blockchain;
[0008] S2. Analyze the data characteristics and implement adaptive multi-order Huffman coding. Different characters in the data have different frequencies and different frequencies in different frequency ranges. The compression efficiency of multi-order Huffman coding is compared with traditional Huffman coding in different frequency ranges to achieve adaptive multi-order Huffman coding.
[0009] S3. Perform multi-order Huffman coding compression on the data and store it according to the adaptive result.
[0010] Preferably, the process of the order of the adaptive multi-order Huffman coding is as follows:
[0011] 1) Analyze the similar frequency trend and similarity within the neighborhood range through the similar frequency index of each character data and each character data, and obtain the similar frequency range of the data;
[0012] 2) Analyze the confidence of each similar frequency interval;
[0013] 3) Traverse the compression complexity of each optimal similar frequency interval and the order of adaptive multi-order Huffman coding.
[0014] Preferably, the similar frequency index of each character data is obtained in the following manner:
[0015] Count the frequency of character data of reliable measurement data and sort them from small to large;
[0016] Analyze each character data to see whether it is more inclined to the maximum frequency or the minimum frequency in the overall data, and determine the degree of its tendency;
[0017] Predict the similarity and analyze the actual similarity between the current character data and the character data frequency within a certain neighborhood range to obtain the similarity frequency index of each character data.
[0018] Preferably, the similar frequency index of each character data is specifically:
[0019] ;
[0020] Where: Represents the similar frequency index of the i-th character data; It represents the difference between the frequency of the i-th character data and the maximum frequency in the overall data;
[0021] Indicates the difference between the number of the i-th character data and the maximum code in the overall data, where the number is a label sorted from small to large according to frequency;
[0022] It represents the difference between the frequency of the i-th character data and the minimum frequency in the overall data;
[0023] The difference between the serial number of the i-th character data and the smallest code in the overall data, where the serial number is a number sorted from small to large in frequency;
[0024] o represents the sequence number of the character data in the left and right neighborhoods respectively judged in the i-th character data; It indicates the total number of character data in the left and right neighborhoods of the i-th character data; Represents the frequency of the i-th character data;
[0025] Represents the average frequency of the oth character data in the left and right neighborhoods of the i-th character data;
[0026] Represents the frequency ratio of the oth character data in the left and right neighborhoods of the i-th character data;
[0027] In the formula, Indicates the tendency to select the maximum value of the two as the current i-th character data.
[0028] Preferably, the confidence of analyzing each similar frequency interval is calculated as follows:
[0029] ;
[0030] In the formula, represents the confidence of the ath similar frequency interval; M represents that there are M types of character data in the ath similar frequency interval;
[0031] Represents the similar frequency index of the mth character data in the ath similar frequency interval;
[0032] Indicates the standard deviation of the proximity frequency index based on the character data in the current ath proximity frequency interval.
[0033] Preferably, the compression complexity of each optimal similar frequency interval is traversed, and each character data in the optimal similar frequency interval is allocated to different frequency categories.
[0034] Preferably, the order of the adaptive multi-order Huffman coding takes the character data types existing in each similar frequency interval as frequency types, and allocates the data in the remaining similar frequency intervals and the character data not in the optimal similar frequency interval to the frequency types. At the same time, the number of encodings is traversed to select the similar frequency interval corresponding to the least number of encodings, and the frequency type corresponding to the similar frequency interval is taken as the final frequency type, and the final frequency type is taken as the order of the adaptive multi-order Huffman coding.
[0035] Preferably, the compression complexity of the optimal similar frequency interval is calculated by encoding each optimal similar frequency interval as a coding test model, the compression complexity is calculated by the number of characters and the coding length, and the minimum value is selected.
[0036] Preferably, the compression complexity of the optimal close frequency interval is calculated as follows:
[0037]
[0038] Where: represents the best compression complexity in each optimal close frequency interval, which is determined by using each optimal close frequency interval as an ergodic test model;
[0039] G means there are G optimal similar frequency intervals;
[0040] K means that there are K types of character data in each optimal similar frequency interval;
[0041] Indicates the number of k-th character data in the g-th optimal similar frequency interval;
[0042] Indicates the encoding length corresponding to the k-th character data in the g-th optimal similar frequency interval;
[0043] is the number of similar frequency intervals.
[0044] Preferably, the data is compressed by multi-order Huffman coding according to the adaptive result, specifically:
[0045] The final frequency category is obtained according to the adaptive multi-order Huffman coding, and a Huffman coding table is constructed for each frequency category;
[0046] The final number of frequency categories is used as the order of multi-order Huffman coding;
[0047] Perform multi-order Huffman coding compression.
[0048] Technical effects and advantages of the present invention: The present invention provides a trusted metering data storage method based on blockchain technology. Compared with traditional technologies, the present invention determines the similar frequency index of each character data based on the similar conditions of the overall and neighborhood data in the overall data, and then obtains its similar frequency interval; due to the long-term accumulation of data and the huge amount of data, in order to improve the accuracy of the obtained traveling frequency interval, multiple optimal similar frequency intervals are screened by analyzing the confidence of the similar frequency interval. Finally, the compression complexity of each optimal similar frequency interval as the traversal test model is analyzed to complete the adaptive multi-order Huffman coding;
[0049] Secondly, this method solves the problem of large differences in code lengths when data frequencies are similar, reducing the occurrence of coding conflicts; at the same time, it completes adaptive multi-order Huffman coding to determine more efficient multi-order Huffman coding, which not only avoids fewer coding conflicts, but also takes compression efficiency into consideration, thereby improving the overall compression effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1 This is a flow chart of the trusted measurement data storage method based on blockchain technology of the present invention;
[0051] Figure 2 The flowchart is a flow chart of the order of adaptive multi-order Huffman coding of the present invention. DETAILED DESCRIPTION
[0052] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0053] The present invention provides a trusted metering data storage method based on blockchain technology. By analyzing data characteristics, according to the characteristics of similar frequencies of different characters and the degree of influence of different orders on compression efficiency, the order of adapting to multi-order Huffman coding is realized, thereby avoiding the problem of different coding lengths caused by similar data frequencies, and improving the overall data compression efficiency.
[0054] like Figure 1 As shown in the figure, the frequencies of different symbol data are similar, which may cause coding conflicts, that is, some codes are longer than other codes. By dividing the data into multiple stages and using different coding tables, such coding conflicts can be reduced, so that the data in each stage can be optimally compressed, which can improve the consistency of coding and the overall compression ratio. The specific steps are as follows:
[0055] S1. Obtain trusted measurement data based on blockchain; obtain the trusted measurement data that needs to be compressed from the blockchain by using the query interface of the blockchain platform;
[0056] S2. Analyze the data characteristics and implement adaptive multi-order Huffman coding. Different characters in the data have different similarities in frequency, and present different similarities in different frequency ranges. Analyze the compression efficiency of multi-order Huffman coding compared with traditional Huffman coding in different frequency ranges, and implement adaptation of the order of multi-order Huffman coding.
[0057] like Figure 2 As shown, the process of the order of adaptive multi-order Huffman coding is as follows:
[0058] 1) Obtain the similar frequency interval of the data through the similar frequency index of each character data;
[0059] 2) Analyze the confidence of each similar frequency interval;
[0060] 3) Traverse the compression complexity of each optimal similar frequency interval and the order of adaptive multi-order Huffman coding.
[0061] Specifically, the close frequency intervals for obtaining data are as follows:
[0062] Traditional Huffman coding assigns different code lengths according to the frequency of character data. The greater the frequency, the shorter the code length. If you want to obtain the various similar frequency intervals of the data, there are different degrees of similar frequency intervals in the overall data, which is determined by the frequency of the data. In this way, you can analyze the similar frequency trend of each character data and the degree of similarity within the neighborhood to obtain the similar frequency index of each character data.
[0063] From the above, we first count the character data frequencies of the credible measurement data and sort them from small to large; for each character data, we analyze whether it is more inclined to the maximum frequency or the minimum frequency in the overall data, and determine the degree of its tendency. The greater the tendency, the greater the predicted similarity between the current character data frequency and the frequency of the character data in the neighborhood; at the same time, we analyze the actual similarity between the current character data and the frequency of the character data in a certain neighborhood, and the greater the actual similarity. In this way, we can comprehensively obtain the similarity frequency index of each character data, as follows:
[0064]
[0065] Where: Represents the similar frequency index of the i-th character data;
[0066] It represents the difference between the frequency of the i-th character data and the maximum frequency in the overall data;
[0067] Indicates the difference between the number of the i-th character data and the maximum code in the overall data, where the number of the i-th character data is a number sorted in ascending order of frequency;
[0068] It represents the difference between the frequency of the i-th character data and the minimum frequency in the overall data;
[0069] Indicates the difference between the serial number of the i-th character data and the minimum code in the overall data, where the serial number of the i-th character data is a number sorted in ascending order of frequency;
[0070] o means judging the oth character data of the left and right neighborhoods of the i-th character data respectively; It indicates the total number of character data in the left and right neighborhoods of the i-th character data;
[0071] Represents the frequency of the i-th character data;
[0072] Represents the average frequency of the oth character data in the left and right neighborhoods of the i-th character data;
[0073] Represents the frequency ratio of the oth character data in the left and right neighborhoods of the i-th character data;
[0074] In the calculation formula, Indicates the tendency to select the maximum value of the two as the current i-th character data, which is based on the maximum and minimum value analysis of the overall data frequency; because when the difference between the i-th character data and the maximum or minimum frequency is smaller, and the number corresponding to the i-th character data and the maximum or minimum frequency is larger, it means that while ensuring the low frequency difference, the distance between the i-th character data and the character data corresponding to the maximum or minimum frequency is farther, and there is a similar frequency between the i-th character data and other characters in the neighborhood;
[0075] In the calculation formula, It only indicates the positive and negative signs. The above analysis can screen out the degree of tendency towards the maximum and minimum values of the overall data frequency. In order to indicate whether it tends to the maximum or minimum value, it is expressed in this form. -1 represents the tendency towards the minimum value; otherwise, it tends towards the maximum value.
[0076] In the calculation formula, It means that the degree of prediction of similar frequency is determined by the tendency degree of the i-th character data from the overall data;
[0077] In the calculation formula, To indicate the actual similarity frequency of the i-th character data; The actual similarity is reflected by the difference between the frequency of the i-th character data and the average frequency of the o-th character data on both sides of its neighborhood; however, the average frequency of the character data with the same sequence number on both sides of the neighborhood is used, and there may be two large and one small, and their average value happens to be close to the frequency of the i-th character data, so The more accurate the actual similarity frequency of the character data in the i-th character is, the more accurate the actual similarity frequency of the character data in the i-th character is.
[0078] The calculation formula comprehensively determines the similar frequency index of the i-th character data by predicting the whole and comparing the actual neighborhood.
[0079] From the above, the similar frequency index of various character data can be obtained. According to the frequency of the character data from large to small, the similar frequency index of each character data is fitted by the least squares method (the absolute value of the data is taken before fitting), and the maximum value points are selected as the core points of the similar frequency interval.
[0080] For each core point of the similar frequency interval, its character data frequency will be used to set a threshold of 0.05 to traverse both sides. The character data frequencies on both sides whose absolute value of the difference with the corresponding character data frequencies of the core points of the similar frequency interval is ≤0.05 will be included as the similar frequency interval determined by the current core point, and multiple similar frequency intervals are obtained as described above.
[0081] Among them, the confidence of each similar frequency interval is analyzed as follows:
[0082] The above process obtains multiple similar frequency intervals by setting thresholds. Although the threshold is set relatively small, due to the huge amount of time-accumulated data of trusted measurement data, there are too many types of corresponding character data, and the frequency of various characters may be too small. Therefore, it is necessary to perform similar frequency index consistency analysis and screening on the obtained similar frequency intervals to select the optimal similar frequency interval with higher confidence, specifically:
[0083]
[0084] Where: Indicates the confidence level of the ath similar frequency interval; represents the normalization function; Represents an exponential function with the natural constant e as the base.
[0085] M means that there are M types of character data in the ath similar frequency interval; Represents the similar frequency index of the mth character data in the ath similar frequency interval;
[0086] Represents the standard deviation of the close frequency index based on the character data of the current a-th close frequency interval;
[0087] In the formula, As the weight value, while obtaining the marching frequency index of various character data, the maximum or minimum value of the overall data frequency is also considered. Therefore, the more consistent the positive and negative values of the similar frequency indexes of the character data in the current a-th similar frequency interval are, the more consistent the tendency direction of the character data in the current similar frequency interval is, and the higher the corresponding weight value is;
[0088] In the formula, The standard deviation is used to reflect the consistency of the similar frequency index of the character data in the current running frequency interval. The smaller the standard deviation, the higher the confidence.
[0089] Here, the threshold is set to 0.6, the confidence of each similar frequency interval is judged, and the similar frequency interval with a confidence greater than or equal to the threshold is taken as the optimal similar frequency interval.
[0090] It should be noted that the compression complexity of each optimal similar frequency interval and the order of adaptive multi-order Huffman coding are specifically:
[0091] Since the frequencies of the character data in each optimal similar frequency interval are highly similar, the character data in the optimal similar frequency interval are allocated to different frequency categories to avoid the character data with similar frequencies appearing in the same coding table, causing the problem of large differences in coding lengths despite similar frequencies.
[0092] The frequency types referred to here are the number of character data types existing in the adjacent frequency interval, and the number of character data types existing in each adjacent frequency interval is different. The character data types existing in each adjacent frequency interval are respectively used as frequency types, and the data in the remaining adjacent frequency intervals and the character data that are not in the optimal adjacent frequency interval are assigned to the frequency types. At the same time, the number of encodings is traversed to select the adjacent frequency interval corresponding to the least number of encodings. The frequency type corresponding to the adjacent frequency interval is used as the final frequency type, and the final frequency type is used as the order of adaptive multi-order Huffman coding.
[0093] Here, each optimal similar frequency interval is used as a traversal test model, and the number of character data types in it is used as the number of frequency types, and it is allocated to each frequency type in turn according to the frequency of the character data. At the same time, for the character data in the remaining optimal similar frequency intervals except the current traversal test model, it is also allocated to each frequency type in turn according to its frequency. If it is not enough to be evenly allocated, it will be allocated periodically later. At the same time, for the remaining character data except the optimal similar frequency interval, it is also allocated to each frequency type in turn according to its frequency.
[0094] Thus, when each optimal close frequency interval is used as the ergodic test model, its corresponding frequency category and the character data contained in each category are obtained.
[0095]
[0096] Where: represents the best compression complexity in each optimal close frequency interval, which is determined by taking each optimal close frequency interval as an ergodic test model;
[0097] G means there are G optimal similar frequency intervals;
[0098] K means that there are K types of character data in each optimal similar frequency interval;
[0099] Indicates the number of k-th character data in the g-th optimal similar frequency interval;
[0100] Indicates the encoding length corresponding to the k-th character data in the g-th optimal similar frequency interval;
[0101] is the number of similar frequency intervals;
[0102] The calculation formula as a whole encodes each optimal similar frequency interval as a coding test model, calculates the compression complexity by the number of characters and the coding length, and selects the minimum value. The lower the compression complexity of each optimal similar frequency interval, the more efficient it is when the character data type existing in the optimal similar frequency interval is used as the frequency type to allocate the remaining character data for multi-order Huffman compression.
[0103] In this way, the optimal compression complexity in each optimal adjacent frequency interval can be screened out, and the number of character data types corresponding to the optimal adjacent frequency interval is used as the final frequency type, and the character data in the remaining optimal adjacent frequency intervals and non-optimal adjacent frequency intervals are allocated to the final frequency type.
[0104] S3, constructing a Huffman coding table for each frequency type according to the adaptive result, taking the final number of frequency types as the order of multi-order Huffman coding, and realizing multi-order Huffman coding compression;
[0105] This reduces the problem of large differences in code lengths due to similar frequencies.
[0106] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A trusted measurement data storage method based on blockchain technology, It is characterized in that The steps include: S1. Obtaining trusted measurement data based on blockchain; S2. Analyze the data characteristics and implement adaptive multi-order Huffman coding. Different characters in the data have different frequencies and different frequencies in different frequency ranges. The compression efficiency of multi-order Huffman coding is compared with traditional Huffman coding in different frequency ranges to achieve adaptive multi-order Huffman coding. S3, compressing and storing the data by multi-order Huffman coding according to the adaptive result; The process of the order of the adaptive multi-order Huffman coding is as follows: 1) Analyze the similar frequency trend and similarity within the neighborhood range through the similar frequency index of each character data and each character data, and obtain the similar frequency range of the data; 2) analyzing the confidence of each similar frequency interval; and selecting the optimal similar frequency interval according to the confidence; 3) Traverse the compression complexity of each optimal similar frequency interval and the order of adaptive multi-order Huffman coding; The close frequency index of each character data is obtained in the following manner: Count the frequency of character data of reliable measurement data and sort them from small to large; Analyze each character data to see whether it is more inclined to the maximum frequency or the minimum frequency in the overall data, and determine the degree of its tendency; Predict the similarity, and analyze the actual similarity between the current character data and the character data frequency within a certain neighborhood, and obtain the similarity frequency index of each character data; The similar frequency index of each character data is specifically: ; Where: Represents the similar frequency index of the i-th character data; It represents the difference between the frequency of the i-th character data and the maximum frequency in the overall data; The difference between the number of the i-th character data and the maximum code in the overall data, where the number is a label sorted from small to large according to frequency; It represents the difference between the frequency of the i-th character data and the minimum frequency in the overall data; The difference between the serial number of the i-th character data and the smallest code in the overall data, where the serial number is a number sorted from small to large in frequency; o represents the sequence number of the character data in the left and right neighborhoods respectively judged in the i-th character data; It indicates the total number of character data in the left and right neighborhoods of the i-th character data; Represents the frequency of the i-th character data; Represents the average frequency of the oth character data in the left and right neighborhoods of the i-th character data; Represents the frequency ratio of the oth character data in the left and right neighborhoods of the i-th character data; In the formula, Indicates the tendency to select the maximum value of the two as the current i-th character data; The confidence level of each similar frequency interval is calculated as follows: ; In the formula, represents the confidence of the ath similar frequency interval; M represents that there are M types of character data in the ath similar frequency interval; Represents the similar frequency index of the mth character data in the ath similar frequency interval; Represents the standard deviation of the close frequency index based on the character data of the current a-th close frequency interval; The compression complexity of each optimal similar frequency interval is traversed, and each character data in the optimal similar frequency interval is allocated to different frequency categories; The order of the adaptive multi-order Huffman coding is to use the character data types existing in each similar frequency interval as frequency types, and to allocate the data in the other similar frequency intervals and the character data not in the optimal similar frequency interval to the frequency types, and to traverse the encoding times, select the similar frequency interval corresponding to the minimum encoding times, and use the frequency type corresponding to the similar frequency interval as the final frequency type, and use the final frequency type as the order of the adaptive multi-order Huffman coding; The compression complexity of the optimal similar frequency interval is calculated by encoding each optimal similar frequency interval as a coding test model, the compression complexity is calculated by the number of characters and the coding length, and the minimum value is selected; The compression complexity of the optimal close frequency interval is calculated as follows: Where: represents the best compression complexity in each optimal close frequency interval, which is determined by using each optimal close frequency interval as an ergodic test model; G means there are G optimal similar frequency intervals; K means that there are K types of character data in each optimal similar frequency interval; Indicates the number of k-th character data in the g-th optimal similar frequency interval; Indicates the encoding length corresponding to the k-th character data in the g-th optimal similar frequency interval; is the number of similar frequency intervals.
2. According to the method for storing trusted measurement data based on blockchain technology in claim 1, It is characterized in that The data is compressed by multi-order Huffman coding according to the adaptive result, specifically: The final frequency category is obtained according to the adaptive multi-order Huffman coding, and a Huffman coding table is constructed for each frequency category; The final number of frequency categories is used as the order of multi-order Huffman coding; Perform multi-order Huffman coding compression.
