Entropy estimation method of entropy source

Abstract

The invention discloses an entropy estimation method of an entropy source, and belongs to the technical field of random number security. Obtaining a random sequence generated by a to-be-detected entropy source; the random sequence is segmented according to a set length to obtain a plurality of feature sequences, the length is equal to correlation length + 1, the number of the correlation length before each feature sequence serves as a feature vector, and the last number serves as a label corresponding to the feature vector; dividing the plurality of feature sequences into a training set and a test set according to a set proportion; performing tree training based on the training set to obtain a decision tree; and inputting the test set into the decision tree to obtain a prediction result, and obtaining the minimum entropy of the entropy source to be tested according to the prediction result. According to the method, prediction is completed under the condition of using less resources and time while prediction is accurate, so that random sequence prediction is more accurate and efficient.

Description

Technical Field

[0001] The invention belongs to the field of random number security technology, and in particular relates to an entropy estimation method for an entropy source. Background Art

[0002] Random numbers are widely used in cryptography and information security, such as generating encryption keys, initialization vectors, and one-time random numbers in challenge-response mechanisms. Among many usage scenarios, the random numbers used in the present invention are mainly generated by two types of random number generators (RNG): deterministic random number generators (DRNG) and true random number generators (TRNG), where TRNG is further divided into physical and non-physical types according to the entropy source. DRNG is usually faster and more efficient in generating random numbers, while TRNG often has more ideal unpredictability. Since the two have different advantages and disadvantages, the present invention often combines them together and uses TRNG to generate a seed sequence as the output of DRNG, which not only guarantees unpredictability to a certain extent, but also can generate random numbers quickly and efficiently.

[0003] Since the present invention uses two random number generators in combination, the unpredictability of TRNG as a seed entropy source requires the present invention to make some guarantees to ensure that its security meets the standards. The security of random numbers also significantly affects the security of cryptographic implementation, so high-quality RNG is essential. Entropy is a key indicator for evaluating the quality of RNG. Entropy estimation can be based on random models or statistical methods. Entropy estimation based on random models is performed by modeling the random number generation process of the entropy source, such as the common AIS 31 and ISO / IEC 18031 standards. By analyzing and strictly regulating its internal structure and mechanism in detail, it is theoretically determined whether the generated random numbers meet the entropy value requirements, thereby ensuring the quality of RNG. Recent articles on RNG focus on using random models for entropy estimation, optimizing and improving the quality and structure of TRNG. However, the problem with entropy estimation based on random models is that it is highly dependent on the prior knowledge of the target RNG, so it is less efficient and less versatile. Of course, its rigorous analysis also ensures the accuracy of the entropy value, making it a common testing method.

[0004] Entropy estimation based on statistical methods obtains results by checking the output random numbers from unknown entropy sources. The principle is to identify defects in the statistical properties of the output sequence and associate these defects with the entropy value to establish a minimum entropy value, which can be regarded as the lower limit of the quality of the target RNG. The most famous statistical entropy estimation method is the NIST SP800-90B standard, which contains detection methods based mainly on statistical techniques. This method is essentially suitable for measuring different RNGs efficiently and portable. Although its accuracy cannot be compared with the test method of random models, the widely portable test RNG is an important factor that cannot be ignored in current real-world usage.

[0005] The statistical methods of 90B have been updated and optimized. Kim et al. proposed improvements in the efficiency of the compression estimator and designed an online estimator. Woo et al. solved the overestimation problem of the longest repeating subsequence estimator and proposed a more stable estimator. In addition to statistical entropy estimation, 90B also includes prediction model entropy estimation. Kelsey et al. proposed a general method for prediction model entropy estimation in 2015. This method complements statistical entropy estimation, can correctly estimate entropy values ​​for some complex entropy sources, and is integrated into the 90B standard. Subsequently, Kim et al. proposed an efficient algorithm for MultiMCW in 2017 and fully optimized four estimators in 2018. However, their optimization method is relatively simple and failed to significantly improve efficiency. Due to the difficulty in improving the 90B prediction estimator, recent research has gradually turned to other prediction models. Kelsey et al. also suggested using more complex, specially designed machine models to replace prediction models for sequence prediction in subsequent work. In addition to the 90B standard, new methods based on other prediction models have also attracted attention, among which neural network (NN) entropy estimation is the most popular. This method uses various NNs to predict random sequences, and shows excellent results for some complex entropy sources, even for situations that the 90B prediction model method has difficulty coping with. In 2018, Yang et al. first proposed a prediction estimator based on a feedforward neural network (FNN) and a recurrent neural network (RNN), an innovation that marked the beginning of using NNs for prediction estimation. Subsequently, Lv et al. expanded this work in 2020 and conducted detailed tests on different random sequences. NNs, with their complex models, show strong learning ability and high prediction accuracy. However, NNs require multiple training iterations to learn parameters, resulting in long training times. Although NNs can outperform 90B in prediction time for high-bit and long sequences, the training time on commonly used bit widths and 1MB of data is still long. In addition, these models have difficulty in effectively predicting second-order Markov models, often performing worse than 90B, and even almost unable to make accurate predictions. To improve the accuracy of NNs, Zhang et al. proposed the TPA-LSTM model in 2020. This model combines LSTM (suitable for time series prediction) and the attention mechanism (TPA) from Transformer to significantly improve prediction accuracy. Their paper showed that this model for predicting white noise randomness can be applied as a general prediction estimator model. Subsequently, in 2023, Zhang et al. further optimized the TPA-LSTM model and significantly reduced the size of the model through quantization and pruning. They also introduced one-hot encoding to avoid prediction inaccuracies caused by learning the numerical size of the data. Despite these improvements, the TPA-LSTM model still has difficulties in predicting second-order Markov models, and its execution time is longer than FNN and RNN.However, due to its extremely high prediction accuracy, current NN prediction models still regard it as the most accurate prediction estimator, but its time and resource consumption are worrying.

[0006] 90B often cannot correctly measure complex entropy sources, and NN often requires a lot of time and resource consumption. Summary of the invention

[0007] Entropy estimation method for entropy source:

[0008] Obtain a random sequence generated by the entropy source to be detected;

[0009] The random sequence is divided according to a set length to obtain multiple feature sequences, wherein the length is equal to the correlation length + 1, the first digit of the correlation length of each feature sequence is used as a feature vector, and the last digit is used as a label corresponding to the feature vector;

[0010] Divide the above multiple feature sequences into training sets and test sets according to a set ratio;

[0011] Performing tree training based on the training set to obtain a decision tree;

[0012] The test set is input into the decision tree to obtain a prediction result, and the minimum entropy of the entropy source to be tested is obtained according to the prediction result.

[0013] Furthermore, a correlation test is performed on the random sequence to obtain the correlation length.

[0014] Further, the method for determining the correlation length includes:

[0015] Calculate the number of occurrences of consecutive set length numbers, and calculate the upper bound of the number of consecutive occurrences of numbers under each set length;

[0016] Traversing and recording the number of consecutive occurrences of numbers of different set lengths in the random sequence;

[0017] When the number of times is greater than the upper limit of the number of consecutive occurrences, the longest length is selected as the correlation length.

[0018] Furthermore, the segmentation method includes:

[0019] The random sequence is intercepted with the correlation length + 1 as window n, and the i-th characteristic sequence is {x i, ...,x i+n}, where {x i, ...,x i+n-1} is a feature, x i+n For label.

[0020] Furthermore, the tree training is:

[0021] The tree nodes are divided from the root node based on the training set, and based on the splitting criterion gain, the splitting point with the best splitting effect and the highest gain is selected as the node of the decision tree.

[0022] Furthermore, the nodes with the same eigenvalues ​​are tested for uniform distribution to prune the decision tree.

[0023] Furthermore, the uniformity distribution test adopts a chi-square test or a continuity-corrected chi-square test.

[0024] Furthermore, the prediction result includes prediction accuracy and the longest number of consecutive predictions.

[0025] The beneficial effects of the present invention are as follows:

[0026] The present invention takes into account the fact that random sequence prediction needs to be insensitive to the size of numbers, and uses a new prediction model decision tree to solve the problem that 90B prediction accuracy for complex entropy sources is insufficient and NN consumes too much time and resources. Using a decision tree as an entropy estimation method allows the present invention to complete the prediction while using less resources and time, making random sequence prediction more accurate and efficient. The correlation length test and pruning method in the present invention ensure the accuracy and efficiency of the decision tree. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a flow chart of an embodiment of the present invention.

[0028] Figure 2 Schematic diagram of data processing in an embodiment of the present invention.

[0029] Figure 3 Schematic diagram of decision tree training results in an embodiment of the present invention.

[0030] Figure 4 The figure is a comparison chart of the accuracy between the method of the embodiment of the present invention and the prior art.

[0031] Figure 5 The figure is a comparison chart of the execution time between the method of the embodiment of the present invention and the prior art.

[0032] Figure 6 The figure is a comparison chart of the comprehensive effects of the method of the embodiment of the present invention and the prior art. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0034] The present invention predicts random sequences in accordance with the 90B process, so the most basic process is to give a random sequence file containing 1MB of data and the number of bits contained in each byte, and then get the estimated minimum entropy after running the program of the present invention.

[0035] The program of the technical solution of the present invention is constructed using the decision tree code of the mlpack library in C++ language and the NIST SP800-90B code. The specific process is as follows: Figure 1 shown.

[0036] When using decision trees directly for entropy estimation, significant problems are encountered. Different entropy sources have different dependencies. For example, the first-level Markov entropy source depends on the previous sample, while the second-level Markov entropy source depends on the combination of the first two samples. If these dependencies cannot be clearly determined, it is impossible to accurately predict the minimum entropy, and the result may lead to a conclusion close to the maximum entropy. Therefore, it is necessary to determine which previous samples are related to the current sample. The first is to perform a correlation length test. The code is written in C language to calculate the number of occurrences of 2 to 17 consecutive numbers (corresponding to 1 to 16 calculated by the 90B standard, and one result data is added for each calculation). The correlation between samples is determined by the frequency of repetition of certain sequences. For example, in order to determine whether a sequence sample is related to the previous n-1 samples, the present invention needs to count the number of occurrences of any consecutive n samples. If the number exceeds the expected frequency in a random situation, it indicates that there is a correlation. Specifically, in a random situation, the probability of any consecutive n d-bit samples repeating is 2 -dn , follows uniform distribution. According to the central limit theorem, the maximum number of repetitions follows normal distribution. The present invention can pre-calculate the mean and standard deviation of the maximum number of repetitions by conducting 100 random experiments, and establish a confidence interval. Let the mean be μ, the variance be σ, and the upper limit of the confidence interval be

[0037] The 90B standard directly uses 1 to 16 samples as features to predict the next sample, balancing the actual correlation and efficiency. Therefore, the embodiment of the present invention also measures the correlation of 1 to 16 samples. The embodiment of the present invention traverses and calculates the number of consecutive appearances of 2 to 17 numbers under uniform distribution. The present invention pre-calculates the number of consecutive appearances of 100 groups of 2-5 numbers. The probability of 6 or more appearing in 1MB of data is 0, so there is no need to calculate (the upper limit of the number of appearances is 0). As long as it appears in the correlation test, it is considered relevant. The variance and mean of the 100 groups of data calculated are calculated, and the upper limit of the number of consecutive appearances is calculated based on the normal distribution. Each time a random sequence is predicted, as long as it exceeds the upper limit according to this result, it is considered that the correlation is sufficient, and the longest one is taken as the correlation length. In the embodiment of the present invention, the longest number of 5 digits exceeds the upper limit, then the first 4 are used as decision tree training features, and the fifth is used as a label, and its correlation length is 4.

[0038] After obtaining the correlation length step, the present invention divides 1MB of data into 0.8MB of training data and 0.2MB of prediction data, and performs data processing. The present invention performs data processing as follows Figure 2 Data processing mainly includes segmenting and labeling the random numbers generated from the entropy source to be detected. For a random sequence {x1…,x m}, where 0<n<m, all of which are natural numbers, the present invention divides them into features and labels according to the following method: For the qth feature, it is divided into {x q ,...,x q+n-2}, where q≥1 is a natural number, and the qth label is x q+n-1 , that is, use n-1 numbers to predict the next number.

[0039] Since the present invention uses the mlpack library, it needs to correspond to the data type it uses. Therefore, the present invention converts all features of the training data and the predicted data into feature vectors, and sets them to the mat type required by the decision tree of mlpack, and sets the label vectors of the training data and the predicted data to the aram::Row type. The training data operates step+1 data each time, and the first step data is used as the feature vector, and the last data is used as the label corresponding to this feature vector. Each time a group of data is processed, it moves back one number and continues to be processed as the next group of data. The same operation is performed on the predicted data to obtain the training set and data set of the embodiment of the present invention.

[0040] The tree training process of the embodiment of the present invention Tree training: This part includes dividing the tree nodes from the root node based on the training set. The specific steps are as follows: Input the training set. For a feature vector i-th component x i , calculate the eigenvector for each x in xi The gain of the splitting criterion of >z, where z represents the value of all possible random numbers except the maximum value. After traversing all components, the present invention selects the node with the best splitting effect and the highest gain as the node. It is determined whether all nodes have been divided; if not, continue to add new nodes; if the division is completed, enter the next stage. Figure 3 A simple training result is given, which is a trained tree with a correlation length of 3. For the test data, the final prediction result, that is, the value in the leaf node, can be obtained by accessing the leaf node from the root node of the tree as required according to the size of the component.

[0041] Next, we train the decision tree and input the training set into the decision tree instance of mlpack. <ginigain>Initialize a decision tree dt. The reason for using the Gini coefficient here is that the accuracy of other methods is not much different from it, but the efficiency is not as good as the Gini coefficient. Then use the dt.train method to train the tree. The parameters are selected as default. Input the training set to the train method of the decision tree. Note that the number of labels that can be selected is 2. d , d is the bit width of the random sequence.

[0042] The present invention finds that allowing decision trees to grow without restriction can lead to overfitting of certain data sets, and the accuracy decreases as the tree grows. The overfitting problem occurs when the growth of the tree causes the accuracy to decrease. The main reason is that data sets with different entropy sources may require different degrees of pruning to prevent the decision tree from overfitting. Specifically, when the tree reaches a certain size and the data is unevenly distributed, further splitting is unnecessary. At this time, the maximum value should be used as the label of the node. Continuing to grow the decision tree may cause rare data to be assigned as leaf node labels, thereby reducing the overall accuracy.

[0043] In order to solve the above problems, judgment is required during the training process. When a node is uneven, that is, it does not meet the uniform distribution, it is judged that it should be pruned. The pruning method involves a uniform distribution test. If the node passes the 99% confidence uniform distribution test, it is regarded as a leaf node and its corresponding label is set to the most frequent label. The reason for this method is that when the numbers in the same position of all feature vectors are the same, it indicates that the data relationship along the node path has been fully defined. If the label distribution within the node is highly concentrated, pruning is required. On the contrary, if the distribution is uniform enough, it means that there are still patterns that have not been discovered and the node should continue to split. For the uniform distribution test, when the average number of occurrences of the label is greater than or equal to 5, the chi-square test is used; otherwise, the continuity-corrected chi-square test is used. The test statistic for uniform distribution is as follows.

[0044]

[0045] where x i represents the number of occurrences of the i-th label, represents the average number of occurrences of a label, and k is the number of label categories. For these statistics, the present invention can compare them with the values ​​of the 0.01 quantile with k-1 degrees of freedom in the chi-square table. If they are greater than the values ​​in the table, it indicates that the uniform distribution test has not passed, and pruning is required, that is, the node is directly recorded as a leaf node.

[0046] The specific method is that the present invention needs to find the best_binary_numeric_split_impl.hpp file, which is a subfile of the decision tree training process in mlpack, and the BestBinaryNumericSplit in the mlpack namespace <fitnessfunction>In the ::SplitIfBetter method, the present invention adds new code at the place where it is initially determined whether to return this function (that is, to stop splitting as a leaf node): calculate the mean and variance of the number of occurrences of all values ​​of the label corresponding to the current function, that is, the parameter labels, and calculate the χ value for this number of occurrences. 2 The value of the distribution needs to satisfy whether the leaf node is less than 1000. Less than 1000 means the number of labels with the same value for all values ​​of a certain feature. After testing, the specific value of the decision tree split degree that basically meets the judgment condition is about 1000. Slightly increasing or decreasing this value will not affect the final result. 2 The distribution can be considered to satisfy the uniform distribution, and further testing is needed. Otherwise, the termination function is directly returned to prevent overfitting.

[0047] Then, the Classify method of the decision tree is used to input the test set into the decision tree in groups. Unlike the training set, when training data, only the training set needs to be input into the train method as a whole, while when predicting data, the feature vectors of the test set need to be input sequentially to obtain a prediction result at a time. While obtaining the prediction result, the present invention needs to record the overall prediction accuracy, that is, the number of correct predictions divided by the total number, and the longest continuous number of correct predictions plus 1, in preparation for the subsequent calculation of the minimum entropy.

[0048] Entropy estimation: Finally, the present invention inputs the test set into the decision tree and determines the corresponding nodes in turn to obtain the prediction results. At the same time, P is calculated based on the prediction results. global and P local , the formula is as follows. Finally, these results are used to calculate the minimum entropy.

[0049]

[0050] Among them, P global Refers to the global prediction accuracy, P local It refers to the local prediction accuracy, P refers to the number of correct predictions divided by the total number of predictions, r refers to the longest number of consecutive predictions + 1, and α is the confidence level.

[0051] The last step is to calculate the minimum entropy. The present invention needs to use the calculation function in 90B. Here you can query the C code of 90B in github, directly borrow the predictionEstimate function therein, and directly input the obtained data into the function to get the minimum entropy of the entropy source.

[0052] The method of the embodiment of the present invention is experimentally compared with the prior art.

[0053] In all data comparisons, 90B represents the US standard NIST SP800-90B, TPA-LSTM is the most advanced neural network, and DT is the method of the present invention. First of all, in terms of prediction accuracy, the present invention has the most accurate prediction results for the second-order Markov entropy source, such as Figure 4 and Table 1, Figure 4 The correct line in the middle is the theoretical entropy value. The closer it fits the correct line, the more accurate it is. It can be found that DT basically fits the theoretical entropy, while other methods will overestimate the entropy value.

[0054] Table 1

[0055] CPU Jitter Entropy Source Periodic Entropy Source Normal and Sine XOR Entropy Sources Normal and sine plus entropy source 90B 2.978 1.748 6.135 4.789 DT 2.596 1.536 5.083 4.135 TPA-LSTM 2.370 1.760 5.059 4.127

[0056] Table 1 shows the entropy values ​​of different entropy sources predicted by various methods. The entropy sources of unknown theoretical entropy include: CPU jitter entropy source generates random sequences through CPU clock cycle jitter, periodic entropy source is a cycle such as 50 numbers, there are some numbers that remain unchanged in all subsequent cycles, normal and sine XOR, plus entropy source is the XOR and addition of the random sequence of the normal distribution entropy source and the sine function entropy source. The lower the predicted entropy value, the higher the accuracy. It can be found that the method of the embodiment of the present invention can basically reach the lowest entropy value. Although the neural network still has some advantages, in the time comparison described later, the neural network lags behind too much.

[0057] Figure 5 The time dimension is used to compare the methods. The logarithmic coordinates are used to represent the time comparison, the horizontal axis is the bit width of the random sequence, and the vertical axis is the execution time. TPA-LSTM (CPU) represents the running time when the CPU is directly used instead of the GPU. In this way, the time difference is compared. The method of the embodiment of the present invention is basically the same as 90B, and both are significantly ahead of the neural network.

[0058] Finally, a comprehensive comparison is made, such as Figure 6 As shown. Time and accuracy are used as horizontal and vertical coordinates respectively, so the method closer to the origin has the best comprehensive effect. It can be found from the figure that DT is always in the lower left corner, and the comprehensive effect of the method in the embodiment of the present invention is the best.

[0059] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. A person skilled in the art may modify or make equivalent substitutions for the technical solutions of the present invention without departing from the spirit and scope of the present invention. The protection scope of the present invention shall be subject to the claims.< / fitnessfunction> < / ginigain>

Claims

1. Entropy estimation method of entropy source: Obtain a random sequence generated by the entropy source to be detected; The random sequence is divided according to a set length to obtain multiple feature sequences, wherein the length is equal to the correlation length + 1, the first digit of the correlation length of each feature sequence is used as a feature vector, and the last digit is used as a label corresponding to the feature vector; Divide the above multiple feature sequences into training sets and test sets according to a set ratio; Performing tree training based on the training set to obtain a decision tree; The test set is input into the decision tree to obtain a prediction result, and the minimum entropy of the entropy source to be tested is obtained according to the prediction result.

2. The method according to claim 1, characterized in that: A correlation test is performed on the random sequence to obtain the correlation length.

3. The method according to claim 2, characterized in that The method of determining the correlation length includes: Calculate the number of occurrences of consecutive set length numbers, and calculate the upper bound of the number of consecutive occurrences of numbers under each set length; Traversing and recording the number of consecutive occurrences of numbers of different set lengths in the random sequence; When the number of times is greater than the upper limit of the number of consecutive occurrences, the longest length is selected as the correlation length.

4. The method according to claim 1, characterized in that: The segmentation method includes: The random sequence is intercepted with the correlation length + 1 as window n, and the i-th characteristic sequence is {x i, ...,x i+n }, where {x i, ...,x i+n-1 } is a feature, x i+n For label.

5. The method according to claim 1, characterized in that The tree training is: The tree nodes are divided from the root node based on the training set, and based on the splitting criterion gain, the splitting point with the best splitting effect and the highest gain is selected as the node of the decision tree.

6. The method according to claim 1, characterized in that Decision trees are pruned by testing the uniformity of distribution of nodes with all the same eigenvalues.

7. The method according to claim 6, characterized in that The uniformity distribution test adopts chi-square test or continuity-corrected chi-square test.

8. The method according to claim 1, characterized in that The prediction results include prediction accuracy and the longest continuous prediction number.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 8.

10. A storage medium storing a computer program, wherein when the computer program is executed by a computer, the method according to any one of claims 1 to 8 is implemented.

Images