Data integrity verification system and method based on Hash algorithm

Abstract

The invention discloses a data integrity verification system and method based on a Hash algorithm, and relates to the technical field of data verification, and the method comprises the following steps: carrying out the Hash algorithm processing on original input data to generate a Hash value with a fixed length, collecting the generated Hash value, and constructing an analysis set, the hash values comprise hash values corresponding to all input data; after the analysis set is constructed, features related to hash value changes are extracted from the analysis set, and the extracted features are quantified. According to the method, the analysis set is constructed, the Hash value change features are extracted, and the mode that the Hash value change is not significant is intelligently recognized in combination with machine learning model prediction. On this basis, a random salt value is introduced, the number of Hash iterations is dynamically adjusted, the randomness and complexity of a Hash algorithm are enhanced, and insufficient change of a Hash value caused by data regularity is avoided, so that the Hash collision risk is reduced, it is ensured that tiny data change is accurately reflected, and the safety and defense capability of data integrity verification are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data verification, and in particular to a data integrity verification system and method based on a hash algorithm. Background Art

[0002] The data integrity verification system based on the hash algorithm is a technology that verifies whether the data has been changed during transmission or storage by using a hash function. In this system, the original content of the data is processed by the hash algorithm to generate a hash value (also called a summary) of fixed length. This hash value is the unique "fingerprint" of the data and has strong collision resistance and sensitivity. The transmitted or stored data is sent or stored together with its hash value. When the data receiver or user needs to verify the integrity of the data, the received data is hashed again and the calculated hash value is compared with the original hash value. If the two are consistent, it means that the data has not been tampered with during transmission or storage. Otherwise, it means that the data may have been tampered with or lost. Common hash algorithms include MD5, SHA-1, SHA-256, etc.

[0003] The prior art has the following deficiencies: When existing hash algorithms process input changes, if the output hash value changes slightly or insignificantly, current technology is generally unable to intelligently identify this situation. When the input data is highly structured or regular, the algorithm may not be able to produce sufficiently diverse hash values, resulting in different inputs generating the same hash value. If the change in the input data does not cause a significant change in the hash value, multiple different data may generate the same hash value, which in turn causes verification failure and even allows forged data to pass verification.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0005] The purpose of the present invention is to provide a data integrity verification system and method based on a hash algorithm. By constructing an analysis set and extracting features related to hash value changes, combined with the prediction of a machine learning model, it is possible to intelligently identify potential patterns in which hash value changes are not significant. When the hash value change fails to effectively respond to data changes, by introducing a random salt value and dynamically adjusting the number of hash iterations of the salt value, the complexity and randomness of the hash calculation are increased to avoid insufficient hash value changes due to low entropy or repeated patterns of the data. By enhancing the randomness and adaptability of the hash algorithm, the risk of hash collisions is effectively reduced, and it is ensured that the hash value can accurately reflect these changes in the face of minor data changes, thereby enhancing the reliability and security of data integrity verification and improving the system's defense capabilities against potential attacks to solve the problems in the above-mentioned background technology.

[0006] In order to achieve the above object, the present invention provides the following technical solution: a data integrity verification method based on a hash algorithm, comprising the following steps: The original input data is processed by a hash algorithm to generate a hash value of a fixed length. The generated hash values ​​are collected and an analysis set is constructed, which contains the hash values ​​corresponding to all the input data. After the analysis set is constructed, the features related to the hash value changes are extracted from it, and the extracted features are quantified to preliminarily identify potential patterns in which the hash value changes are not significant; The quantized features are input as feature vectors into the trained machine learning model, and the model is used to predict the changes in the hash value output results. When it is identified that the hash value change is not significant, the generated random salt value is combined with the original data to form new input data, which is then processed through the hash algorithm to generate a new hash value, thereby improving the randomness and complexity of the hash result. The number of hash iterations of the salt value is dynamically adjusted according to the insignificant degree of hash value change to reduce the risk of hash collision.

[0007] Preferably, the original input data is processed by a hash algorithm to generate a hash value of a fixed length, specifically: The hash algorithm maps the input data to a fixed-length hash value. The hash value is a string of numbers, and the same input always produces the same hash value.

[0008] Preferably, features related to hash value changes are extracted from the analysis set, wherein the extracted features include the bit flip rate between adjacent hash values ​​and the similarity between adjacent hash values, the bit flip rate between adjacent hash values ​​and the similarity between adjacent hash values ​​are quantified to generate bit flip rate quantization values ​​and adjacent hash similarity quantization values, respectively, and potential patterns in which hash value changes are not significant are preliminarily identified through the bit flip rate quantization values ​​and adjacent hash similarity quantization values.

[0009] Preferably, the quantized value of the bit flip rate and the quantized value of the adjacent hash similarity after quantization are input into the trained machine learning model as feature vectors, and a change response index is generated based on the prediction of the machine learning model, and the change of the hash value output result is predicted by the change response index.

[0010] Preferably, the change response index generated when the trained machine learning model predicts the change of the hash value output result is compared and analyzed with a pre-set change response index reference threshold to predict the insignificant change of the hash value. The specific steps are as follows: if the change response index is less than the change response index reference threshold, the hash value change is determined to be insignificant; if the change response index is greater than or equal to the change response index reference threshold, the hash value change is determined to be significant.

[0011] Preferably, when it is identified that the hash value does not change significantly, the generated random salt value is combined with the original data to form new input data, which is then processed by the hash algorithm to generate a new hash value. The specific steps are as follows: After identifying that the hash value changes are not significant, first generate a random salt value and compare it with the original data D Combine to form new input data. The specific expression is: ,in: is an internal parameter used to control the source of randomness, Indicated in the parameter The high entropy random sequence generated under the hood ensures the unpredictability of the salt value. Indicates that after performing a hash operation on a random sequence, the result is taken The power further increases the complexity of the salt value. Indicates that and Cascade, D is the original data, Enter the new hash. After getting the new hash input Afterwards, multiple iterations are performed through the hash algorithm, and the number of iterations is determined by the following formula: ,in: The number of salt hash iterations determines the number of new hash inputs. The number of rounds of repeated hashing, is the change response index, which is a measure of the degree to which the hash value changes are not significant. is the reference threshold of the change response index, is the basic multiplication factor, used to control the overall iteration intensity, is the exponential amplification parameter, which has no significant effect on the change Perform nonlinear transformations, Indicates rounding down to ensure that the number of iterations is an integer. is a logarithmic adjustment factor that amplifies the effect of insignificant changes on the number of iterations.

[0012] Preferably, the specific steps of quantizing the bit flip rate between adjacent Hash values ​​to generate a bit flip rate quantization value are as follows: For the hash value of a set of input data, a bit flip matrix is ​​constructed to record the bit-by-bit changes between adjacent hash values. The constructed expression is: ,in: Indicates i The hash value is j The bit flip condition of the bit, and Respectively represent i and i +1 hash value in j The binary value of the bit, It represents XOR operation. If the two bits are the same, the result is 0. If they are different, the result is 1, indicating that the bit is flipped. is the weight function, defined as: ,in: M is the length of the hash value, mod It is the abbreviation of modulus operation, which means the operation of taking the remainder. The bit index is calculated j Hash length M The result after modulo, that is, the index j In the hash value range The loop index within, if the index j Exceed M , then by taking the modulus M The range is cycled without exceeding the predetermined bit range, which is used to enhance the influence of different bits; The bit flip rate quantization value is calculated based on the bit flip matrix BFM to quantify the insignificance of the hash value change. The calculation expression is: ,in: is the bit flip rate quantized value, N is the total number of hash samples, that is, the number of hash values ​​in the data set, The sine function is used to nonlinearly scale the bit index, so that the bit flip in the middle of the hash value contributes more to the quantized value of the bit flip rate, while the bit flip at the beginning and end has less impact, avoiding the patterned hash collision from being ignored. E It is an exponential adjustment factor used to amplify the impact of low flip rate and increase the sensitivity of the bit flip rate quantization value to hash collision.

[0013] Preferably, the specific steps of quantifying the similarity between adjacent hash values ​​to generate adjacent hash similarity quantization values ​​are as follows: Normalized weighted Hamming similarity is used to measure the overall similarity of hash values. The calculation expression of weighted Hamming similarity is: ,in: represents the normalized weighted Hamming similarity, and Respectively represent the hash values ​​calculated from adjacent input data in the k Binary number on bit, For the k The dynamic weight of a bit is calculated using the entropy weight method and is defined as follows: ,in, Indicates the probability of the bit appearing as 0 or 1 in multiple hash values; According to the normalized weighted Hamming similarity To calculate the adjacent hash similarity quantization value, the adjacent hash similarity quantization value reflects the similarity of adjacent hash values. The specific calculation expression is: ,in, Represents the quantized value of adjacent hash similarity, The maximum value of the normalized weighted Hamming similarity among all adjacent hash value calculations.

[0014] A data integrity verification system based on a hash algorithm, including a hash value generation module, a feature extraction and quantification module, a machine learning prediction module, a salt value generation and hash enhancement module, and a dynamic adjustment and collision protection module; The hash value generation module processes the original input data through a hash algorithm to generate a hash value of a fixed length, collects the generated hash values ​​and constructs an analysis set, which contains the hash values ​​corresponding to all the input data; The feature extraction and quantification module extracts features related to hash value changes from the analysis set after it is built, quantifies the extracted features, and preliminarily identifies potential patterns in which hash value changes are not significant; The machine learning prediction module inputs the quantized features as feature vectors into the trained machine learning model, and uses the model to predict the changes in the hash value output results; The salt value generation and hash enhancement module, when it is recognized that the hash value change is not significant, combines the generated random salt value with the original data to form new input data, and then processes it through the hash algorithm to generate a new hash value, thereby improving the randomness and complexity of the hash result; The dynamic adjustment and collision protection module dynamically adjusts the number of hash iterations of the salt value according to the insignificant degree of hash value change to reduce the risk of hash collision.

[0015] In the above technical solution, the technical effects and advantages provided by the present invention are: The present invention constructs an analysis set and extracts features related to hash value changes, combined with the prediction of a machine learning model, to intelligently identify potential patterns where hash value changes are not significant. When hash value changes fail to effectively respond to data changes, the complexity and randomness of the hash calculation are increased by introducing a random salt value and dynamically adjusting the number of hash iterations of the salt value, thereby avoiding situations where insufficient hash value changes are caused by low entropy or repeated patterns of the data. By enhancing the randomness and adaptability of the hash algorithm, the risk of hash collisions is effectively reduced, and it is ensured that the hash value can accurately reflect these changes when facing small data changes, thereby enhancing the reliability and security of data integrity verification and improving the system's defense capabilities against potential attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0017] Figure 1 The present invention is a method flow chart of a data integrity verification method based on a hash algorithm.

[0018] Figure 2 The present invention is a module schematic diagram of a data integrity verification system based on a hash algorithm. DETAILED DESCRIPTION

[0019] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that the description of the present disclosure will be more comprehensive and complete, and the concept of the example embodiments will be fully conveyed to those skilled in the art.

[0020] The present invention provides Figure 1 A data integrity verification method based on a hash algorithm is shown, comprising the following steps: The original input data is processed by a hash algorithm to generate a hash value of a fixed length. The generated hash values ​​are collected and an analysis set is constructed, which contains the hash values ​​corresponding to all the input data. The hash algorithm maps input data (whether it is text, files, or data in other formats) to a fixed-length hash value, usually a string of numbers, and the same input always produces the same hash value. The role of this step is to provide a standardized data identifier for subsequent analysis, and to quickly generate a "fingerprint" of the data, laying the foundation for data integrity verification and other subsequent processing. The generated hash value is the unique identifier of the data and is therefore the core of the entire solution.

[0021] Collect the generated hash values ​​and construct an analysis set, which contains the hash values ​​corresponding to all input data. This is to form a benchmark and comparison set for data changes by collecting and organizing a large amount of data hash values. The analysis set can help identify the law of hash value changes, especially when processing structured data, to discover potential regularities and patterns, and thus provide a rich data foundation for subsequent feature extraction and model prediction. Through this step, you can understand the distribution characteristics of hash values ​​under different inputs.

[0022] After the analysis set is constructed, the features related to the hash value changes are extracted from it, and the extracted features are quantified to preliminarily identify potential patterns in which the hash value changes are not significant; Features related to hash value changes are extracted from the analysis set, where the extracted features include the bit flip rate between adjacent hash values ​​(i.e., the ratio of bits that change in the hash value under different inputs) and the similarity between adjacent hash values ​​(i.e., the hash values ​​generated for similar input data). The bit flip rate between adjacent hash values ​​and the similarity between adjacent hash values ​​are quantified to generate a bit flip rate quantization value and an adjacent hash similarity quantization value, respectively. The bit flip rate quantization value and the adjacent hash similarity quantization value are used to preliminarily identify potential patterns in which hash value changes are not significant.

[0023] A low bit flip rate between adjacent hash values ​​does indicate that the hash value changes are not significant. One of the ideal properties of a hash algorithm is the avalanche effect, that is, when the input data changes slightly, the output hash value should undergo a large range of bit flips, thereby ensuring the unpredictability and uniform distribution of the hash value. If the bit flip rate is low, it means that even if the input data changes, the different bits of the hash value remain relatively stable, which means that the hash algorithm fails to effectively capture the changes in the input data. This phenomenon may be caused by multiple factors, such as the hash algorithm's lack of sensitivity to specific input patterns, the input data is highly structured, the entropy value is low, or there are defects in the algorithm design. A low bit flip rate may lead to excessive similarity between hash values, thereby increasing the risk of hash collisions, so that different data may generate the same or highly similar hash values, thereby reducing the reliability of the hash algorithm in security applications such as data integrity verification, digital signatures, and encrypted storage.

[0024] The specific steps of quantizing the bit flip rate between adjacent hash values ​​to generate the bit flip rate quantization value are as follows: For the hash value of a set of input data, a bit flip matrix (BFM) is constructed to record the bit-by-bit changes between adjacent hash values. The constructed expression is: ,in: Indicates i The hash value is j The bit flip condition of the bit, and Respectively represent i and i +1 hash value in j The binary value of the bit (0 or 1), Indicates the exclusive OR operation (XOR). If the two bits are the same, the result is 0. If they are different, the result is 1, indicating that the bit is flipped. is the weight function, defined as: ,in: M is the length of the hash value (e.g. 256 bits), mod It is the abbreviation of Modulo Operation, which means the operation of taking the remainder. The bit index is calculated j Hash length M The result after modulo, that is, the index j In the hash value range The loop index within, if the index j Exceed M , then by taking the modulus MThe weight function is used to enhance the influence of different bits, so that the bits with less flipping contribute more to the final result, thereby amplifying the influence of low flipping rate and improving sensitivity. This step captures the bit-by-bit changes of adjacent hash values ​​by constructing a BFM matrix and introduces nonlinear weights to make the impact of some low-frequency flipped bits more prominent, thereby avoiding the smoothing problem that may be caused by simple mean processing. Through the XOR operation, the flipping of each bit can be accurately quantified, and the introduction of the weight function can enhance the impact of certain key bits and increase the sensitivity of the calculation results to insignificant change patterns.

[0025] The bit flip rate quantization value is calculated based on the bit flip matrix BFM to quantify the insignificance of the hash value change. The calculation expression is: ,in: is the bit flip rate quantized value, N is the total number of hash samples, that is, the number of hash values ​​in the data set, The sine function is used to nonlinearly scale the bit index, so that the bit flip in the middle of the hash value contributes more to the quantized value of the bit flip rate, while the bit flip at the beginning and end has less impact, thus avoiding some patterned hash collisions from being ignored. E is an exponential adjustment factor (such as 1.5) used to amplify the impact of lower flip rates and increase the sensitivity of the bit flip rate quantization value to hash collisions.

[0026] It can be seen from the bit flip rate quantization value that the smaller the performance value of the bit flip rate quantization value generated by quantizing the bit flip rate between adjacent hash values, the lower the bit flip rate of the hash value, that is, the similarity between adjacent hash values ​​is high and the change is not significant, indicating that the hash algorithm has a weak ability to distinguish input data, which may lead to an increased risk of hash collision. On the contrary, the larger the performance value of the bit flip rate quantization value, the higher the bit flip ratio of the hash value, indicating that a small change in the input data leads to a large-scale flip of the hash value, indicating that the hash algorithm has a good avalanche effect, can effectively distinguish different input data, and improve the resistance to hash collision. Therefore, by monitoring the numerical changes of the bit flip rate quantization value, the robustness of the hash algorithm can be evaluated.

[0027] A high degree of similarity between adjacent hash values ​​usually indicates that the hash value changes are not significant, which means that the hash algorithm lacks sufficient sensitivity to small changes in the input data. Ideally, the hash algorithm should have an avalanche effect, that is, when the input data changes slightly, the hash value it generates should undergo a large range of bit flips to ensure the randomness and unpredictability of the output. However, when the hash values ​​generated by adjacent input data (that is, there are only small differences between the input data) are highly similar, it means that the hash algorithm does not produce enough changes when processing these data, which may lead to an increased risk of hash collisions. For example, if the Hamming distance of the hash values ​​of two input data that only differ by one character or one byte is too small, it means that the distribution characteristics of the hash algorithm are insufficient and may not be sensitive enough to structured data, low entropy data, or patterned data. This situation may be exploited by attackers to construct forged data and deceive hash-based integrity verification.

[0028] The specific steps of quantifying the similarity between adjacent hash values ​​to generate adjacent hash similarity quantization values ​​are as follows: When evaluating the similarity of adjacent hash values, it is necessary to consider the changes in each bit and assign different weights to different bits to highlight stable patterns. To this end, the normalized weighted Hamming similarity is used to measure the overall similarity of hash values. The calculation expression of the weighted Hamming similarity is: ,in: represents the normalized weighted Hamming similarity, and Respectively represent the hash values ​​calculated from adjacent input data in the k The binary number on the bit (0 or 1), For the k The dynamic weight of a bit is calculated using the entropy weight method and is defined as follows: ,in, Indicates the probability of the bit appearing as 0 or 1 in multiple hash values. A lower entropy value means that the bit changes less, so it is given a higher weight. The purpose of this step is to quantify the bit-level similarity between adjacent hash values ​​through the normalized weighted Hamming similarity, and focus on the bits with less changes to detect whether the hash value is sensitive enough to small changes in the input data. If the normalized weighted Hamming similarity is close to 1, it means that the adjacent hash values ​​have little change, and the hash algorithm may not have fully broken up the input data, resulting in hash collisions or security risks.

[0029] According to the normalized weighted Hamming similarity To calculate the adjacent hash similarity quantization value, the adjacent hash similarity quantization value reflects the similarity of adjacent hash values. The specific calculation expression is: ,in, Represents the quantized value of adjacent hash similarity, The maximum value of the normalized weighted Hamming similarity among all adjacent hash value calculations; The closer the value of the adjacent hash similarity quantization value is to 1, the higher the similarity of the adjacent hash values ​​is, and the hash value changes insignificantly; conversely, the closer the value is to 0, the hash value changes significantly; By calculating the quantified values ​​of adjacent hash similarities, we can further quantify the degree of change in the hash value and help analyze the stability and adaptability of the hash algorithm, especially when facing similar or highly structured input data. This also provides valuable indicators for further optimizing the hash algorithm to ensure that the hash algorithm can maintain good performance under different data change conditions.

[0030] It can be seen from the adjacent hash similarity quantization value that the larger the performance value of the adjacent hash similarity quantization value generated by quantifying the similarity between adjacent hash values, the higher the similarity between adjacent hash values, which means that the change in hash value is not significant. In other words, when the adjacent hash similarity quantization value is close to 1, it means that even if there are slight differences between the input data, the generated hash value changes very little, and the hash algorithm is not sensitive enough to these slight input changes, which may lead to an increased risk of collision. When the adjacent hash similarity quantization value is close to 0, it means that the difference between adjacent hash values ​​is large, and the hash value can significantly respond to changes in the input, indicating that the hash algorithm has good anti-collision and adaptability. Therefore, the adjacent hash similarity quantization value is an important indicator to measure the degree of response of the hash value to input changes. The smaller the value, the more significant the hash value change, and vice versa.

[0031] The quantized features are input as feature vectors into the trained machine learning model, and the model is used to predict the changes in the hash value output results. The quantized value of the bit flip rate and the quantized value of the adjacent hash similarity after quantization are input into the trained machine learning model as feature vectors, and a change response index is generated based on the prediction of the machine learning model. The change of the hash value output result is predicted by the change response index.

[0032] A trained machine learning model is a model that has been trained on a large amount of data and can effectively make accurate predictions based on the input features. In this scenario, the training process refers to the input of the quantized bit flip rate quantization value and the adjacent hash similarity quantization value as input features into the machine learning model for training. During the training process, the model repeatedly learns the relationship between different features in the data and the target output, and continuously optimizes its internal parameters (such as weights and biases) to reduce the error between the predicted results and the actual values. After training, the model can infer on new input data and predict the changes in the hash value output results.

[0033] The main function of the trained machine learning model in this solution is to predict the changes in the hash value output by analyzing the characteristics of hash value changes. These changes usually refer to whether the hash value changes significantly due to slight changes in the input data. The machine learning model uses known sample data (i.e., input features and corresponding output results) during the training process. Based on these data, the model will gradually learn how to extract useful information from the bit flip rate quantization value and the adjacent hash similarity quantization value features, and finally form a rule or model that can predict the change of the hash value. In this way, the model can not only capture the rules of significant changes, but also respond sensitively to slight changes. By predicting the generated change response index, the machine learning model can accurately determine whether the output result will change slightly after the input data is processed by the hash algorithm, thereby improving the responsiveness and reliability of the hash algorithm to changes in input data.

[0034] In the specific implementation, the trained machine learning model works through the following steps. First, sample data containing known hash value change patterns are used for training. Each sample data includes input features (such as bit flip rate quantization values ​​and adjacent hash similarity quantization values) and actual hash value change results (i.e., the degree of change in the output of the hash value). Through these data, the machine learning model can learn the relationship between the bit flip rate quantization value and the adjacent hash similarity quantization value and the hash value change, and adjust its internal parameters to achieve prediction accuracy. Common machine learning algorithms, such as support vector machines (SVM), random forests, neural networks, etc., can be used for this type of prediction problem. The advantage of each algorithm is that it processes data differently, which may focus on capturing nonlinear relationships, dimensionality reduction, or improving the ability to classify features.

[0035] The trained machine learning model has a strong generalization ability. It can make reasonable predictions based on previously learned rules when facing unseen data. For example, if the change response index predicted by the model is high, it means that the hash value responds weakly to small changes in the input data, and there may be a risk of hash collision or data tampering. When the change response index is low, it means that the hash value can respond well to small changes in the input data, thereby increasing data integrity and security. Therefore, the role of the trained machine learning model is very important. It can not only make effective predictions on the current data, but also make optimization suggestions based on the actual situation of the hash algorithm to enhance the performance of the hash algorithm when dealing with small changes. The effectiveness of this prediction process depends on the model's accurate learning of the input features during the training process and its reasonable inference of future data.

[0036] The machine learning model is not specifically limited here, and can quantify the bit flip rate Quantified value of similarity with adjacent hashes Conduct comprehensive analysis to generate a change response index In order to realize the technical solution of the present invention, the present invention provides a specific implementation method; the change response index The resulting calculation formula is: , where , are the bit flip rate quantization values Quantified value of similarity with adjacent hashes The weight factor of , All are greater than 0. Weight factor and It is used to adjust the influence of the bit flip rate quantization value and the adjacent hash similarity quantization value on the final result. Specifically, the weight factor and The weights of the two input features are determined respectively in the calculation of the change response index. By setting appropriate weights, the model can adjust the influence of the bit flip rate quantization value and the adjacent hash similarity quantization value according to their different sensitivities to hash value changes, thereby more accurately predicting the performance of hash values ​​under small changes. For example, if the bit flip rate quantization value can better reflect the response of the hash value to small changes, Can be set to a larger value; if the adjacent hash similarity quantization value is more important to the prediction result, adjust These weight factors are usually learned during the machine learning model training process to ensure that the final change response index can effectively reflect the degree of change in the input data.

[0037] It can be seen from the change response index that the smaller the performance value of the bit flip rate quantization value generated by quantizing the bit flip rate between adjacent hash values, the larger the performance value of the adjacent hash similarity quantization value generated by quantizing the similarity between adjacent hash values. That is, the smaller the performance value of the change response index generated when the trained machine learning model predicts the change of the hash value output result, the less significant the change of the hash value, and vice versa.

[0038] The change response index generated by the trained machine learning model when predicting the change of the hash value output result is compared with the pre-set change response index reference threshold to predict the insignificant change of the hash value. The specific steps are as follows: If the change response index is less than the change response index reference threshold, it is determined that the hash value change is not significant; if the change response index is greater than or equal to the change response index reference threshold, it is determined that the hash value change is significant.

[0039] When it is identified that the hash value change is not significant, the generated random salt value is combined with the original data to form new input data, which is then processed through the hash algorithm to generate a new hash value, thereby improving the randomness and complexity of the hash result, avoiding insufficient hash value changes due to data regularity, and dynamically adjusting the number of hash iterations of the salt value according to the insignificant degree of hash value change to reduce the risk of hash collision; When it is recognized that the hash value does not change significantly, the generated random salt value is combined with the original data to form new input data, which is then processed by the hash algorithm to generate a new hash value. The specific steps are as follows: After identifying that the hash value changes are not significant, first generate a random salt value and compare it with the original data D Combine to form new input data. The specific expression is: ,in: is an internal parameter used to control the source of randomness (which can be provided by the system clock, environmental noise, or a hardware random source), Indicated in the parameter The high entropy random sequence generated under the hood ensures the unpredictability of the salt value. Indicates that after performing a hash operation on a random sequence, the result is taken The power (which can be achieved by truncation or bit flipping) further increases the complexity of the salt value. Indicates that and Cascade, D is the original data, Enter the new hash. The purpose of this step is to ensure the randomness and uniqueness of the salt value, and synthesize it with the original data as a new input, thereby breaking the existing regularity of the data, avoiding the problem of insignificant changes in hash values, and providing a higher degree of randomness for subsequent iterations.

[0040] After getting the new hash input Afterwards, multiple iterations are performed through the hash algorithm, and the number of iterations is determined by the following formula: ,in: The number of salt hash iterations determines the number of new hash inputs. The number of rounds of repeated hashing, It is the change response index, which is a measure of the degree to which the hash value changes insignificantly. The larger the value, the less obvious the change. is the reference threshold of the change response index, is the basic multiplication factor, used to control the overall iteration intensity, is the exponential amplification parameter, which has no significant effect on the change Perform nonlinear transformations, Indicates rounding down to ensure that the number of iterations is an integer. is a logarithmic adjustment factor that amplifies the impact of insignificant changes on the number of iterations; The purpose of this step is: if the hash value does not change significantly Larger, The corresponding increase will increase the number of iterations, making the hash calculation more complicated and significantly increasing the difficulty of hash collision; If the hash value changes significantly If the value is smaller, the number of iterations will be relatively reduced, saving computing resources and maintaining high security. Through this dynamic adjustment, a balance can be achieved between security and performance, reducing the risk of hash collisions and enhancing data integrity.

[0041] The above-mentioned data integrity verification method based on hash algorithm can effectively solve the problem of insignificant hash value changes and improve the anti-collision ability and data integrity of hash algorithm. By constructing an analysis set and extracting features related to hash value changes, combined with the prediction of the machine learning model, it is possible to intelligently identify potential patterns of insignificant hash value changes. When the hash value change fails to effectively respond to data changes, by introducing a random salt value and dynamically adjusting the number of hash iterations of the salt value, the complexity and randomness of the hash calculation can be significantly increased to avoid insufficient hash value changes due to low entropy or repeated patterns of data. This solution effectively reduces the risk of hash collisions by enhancing the randomness and adaptability of the hash algorithm, and ensures that the hash value can accurately reflect these changes when facing small data changes, thereby enhancing the reliability and security of data integrity verification and improving the system's defense capabilities against potential attacks.

[0042] The present invention provides Figure 2 The data integrity verification system based on the hash algorithm shown includes a hash value generation module, a feature extraction and quantification module, a machine learning prediction module, a salt value generation and hash enhancement module, and a dynamic adjustment and collision protection module; The hash value generation module processes the original input data through a hash algorithm to generate a hash value of a fixed length, collects the generated hash values ​​and constructs an analysis set, which contains the hash values ​​corresponding to all the input data; The feature extraction and quantification module extracts features related to hash value changes from the analysis set after it is built, quantifies the extracted features, and preliminarily identifies potential patterns in which hash value changes are not significant; The machine learning prediction module inputs the quantized features as feature vectors into the trained machine learning model, and uses the model to predict the changes in the hash value output results; The salt value generation and hash enhancement module, when it is recognized that the hash value change is not significant, combines the generated random salt value with the original data to form new input data, and then processes it through the hash algorithm to generate a new hash value, thereby improving the randomness and complexity of the hash result; The dynamic adjustment and collision protection module dynamically adjusts the number of hash iterations of the salt value according to the insignificant degree of hash value change to reduce the risk of hash collision.

[0043] The data integrity verification method based on hash algorithm provided in an embodiment of the present invention is implemented by the above-mentioned data integrity verification system based on hash algorithm. The specific method and process of the data integrity verification system based on hash algorithm are detailed in the embodiment of the data integrity verification method based on hash algorithm, which will not be repeated here.

[0044] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0045] The above description is only by way of illustration of certain exemplary embodiments of the present invention. It is undoubted that those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

[0046] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0047] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0048] In the several embodiments provided in the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0049] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0050] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0051] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0052] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data integrity verification method based on a hash algorithm, characterized in that: The following steps are involved: The original input data is processed by a hash algorithm to generate a hash value of a fixed length. The generated hash values ​​are collected and an analysis set is constructed, which contains the hash values ​​corresponding to all the input data. After the analysis set is constructed, the features related to the hash value changes are extracted from it, and the extracted features are quantified to preliminarily identify potential patterns in which the hash value changes are not significant; The quantized features are input as feature vectors into the trained machine learning model, and the model is used to predict the changes in the hash value output results. When it is identified that the hash value change is not significant, the generated random salt value is combined with the original data to form new input data, which is then processed through the hash algorithm to generate a new hash value, thereby improving the randomness and complexity of the hash result. The number of hash iterations of the salt value is dynamically adjusted according to the insignificant degree of hash value change to reduce the risk of hash collision.

2. A data integrity verification method based on a hash algorithm according to claim 1, characterized in that: The original input data is processed by a hash algorithm to generate a hash value of fixed length, specifically: The hash algorithm maps the input data to a fixed-length hash value. The hash value is a string of numbers, and the same input always produces the same hash value.

3. The data integrity verification method based on the hash algorithm according to claim 1 is characterized in that: Features related to hash value changes are extracted from the analysis set, wherein the extracted features include the bit flip rate between adjacent hash values ​​and the similarity between adjacent hash values. The bit flip rate between adjacent hash values ​​and the similarity between adjacent hash values ​​are quantified to generate a bit flip rate quantization value and an adjacent hash similarity quantization value, respectively. Potential patterns in which hash value changes are not significant are preliminarily identified through the bit flip rate quantization value and the adjacent hash similarity quantization value.

4. The data integrity verification method based on the hash algorithm according to claim 3 is characterized in that: The quantized bit flip rate quantization value and the adjacent hash similarity quantization value after quantization are input into the trained machine learning model as feature vectors, and a change response index is generated based on the prediction of the machine learning model. The change of the hash value output result is predicted by the change response index.

5. A data integrity verification method based on a hash algorithm according to claim 4, characterized in that: The change response index generated when the trained machine learning model predicts the change of the hash value output result is compared and analyzed with the pre-set change response index reference threshold to predict the insignificant change of the hash value. The specific steps are as follows: if the change response index is less than the change response index reference threshold, the hash value change is determined to be insignificant; if the change response index is greater than or equal to the change response index reference threshold, the hash value change is determined to be significant.

6. A data integrity verification method based on a hash algorithm according to claim 5, characterized in that: When it is recognized that the hash value does not change significantly, the generated random salt value is combined with the original data to form new input data, which is then processed by the hash algorithm to generate a new hash value. The specific steps are as follows: After identifying that the hash value changes are not significant, first generate a random salt value and compare it with the original data D Combine to form new input data. The specific expression is: ,in: is an internal parameter used to control the source of randomness, Indicated in the parameter The high entropy random sequence generated under the hood ensures the unpredictability of the salt value. Indicates that after performing a hash operation on a random sequence, the result is taken The power further increases the complexity of the salt value. Indicates that and Cascade, D is the original data, Enter the new hash. After getting the new hash input Afterwards, multiple iterations are performed through the hash algorithm, and the number of iterations is determined by the following formula: ,in: The number of salt hash iterations determines the number of new hash inputs. The number of rounds of repeated hashing, is the change response index, which is a measure of the degree to which the hash value changes are not significant. is the reference threshold of the change response index, is the basic multiplication factor, used to control the overall iteration intensity, is the exponential amplification parameter, which has no significant effect on the change Perform nonlinear transformations, Indicates rounding down to ensure that the number of iterations is an integer. is a logarithmic adjustment factor that amplifies the effect of insignificant changes on the number of iterations.

7. The data integrity verification method based on the hash algorithm according to claim 3 is characterized in that: The specific steps of quantizing the bit flip rate between adjacent hash values ​​to generate the bit flip rate quantization value are as follows: For the hash value of a set of input data, a bit flip matrix is ​​constructed to record the bit-by-bit changes between adjacent hash values. The constructed expression is: ,in: Indicates i The hash value is j The bit flip condition of the bit, and Respectively represent i and i +1 hash value in j The binary value of the bit, It represents XOR operation. If the two bits are the same, the result is 0. If they are different, the result is 1, indicating that the bit is flipped. is the weight function, defined as: ,in: M is the length of the hash value, mod It is the abbreviation of modulus operation, which means the operation of taking the remainder. The bit index is calculated j Hash length M The result after modulo, that is, the index j In the hash value range The loop index within, if the index j Exceed M , then by taking the modulus M The range is cycled without exceeding the predetermined bit range, which is used to enhance the influence of different bits; The bit flip rate quantization value is calculated based on the bit flip matrix BFM to quantify the insignificance of the hash value change. The calculation expression is: ,in: is the bit flip rate quantized value, N is the total number of hash samples, that is, the number of hash values ​​in the data set, The sine function is used to nonlinearly scale the bit index, so that the bit flip in the middle of the hash value contributes more to the quantized value of the bit flip rate, while the bit flip at the beginning and end has less impact, avoiding the patterned hash collision from being ignored. E It is an exponential adjustment factor used to amplify the impact of low flip rate and increase the sensitivity of the bit flip rate quantization value to hash collision.

8. The data integrity verification method based on the hash algorithm according to claim 3 is characterized in that: The specific steps of quantifying the similarity between adjacent hash values ​​to generate adjacent hash similarity quantization values ​​are as follows: Normalized weighted Hamming similarity is used to measure the overall similarity of hash values. The calculation expression of weighted Hamming similarity is: ,in: represents the normalized weighted Hamming similarity, and Respectively represent the hash values ​​calculated from adjacent input data in the k The binary number on the bit, For the k The dynamic weight of a bit is calculated using the entropy weight method and is defined as follows: ,in, Indicates the probability of the bit appearing as 0 or 1 in multiple hash values; According to the normalized weighted Hamming similarity To calculate the adjacent hash similarity quantization value, the adjacent hash similarity quantization value reflects the similarity of adjacent hash values. The specific calculation expression is: ,in, Represents the quantized value of adjacent hash similarity, The maximum value of the normalized weighted Hamming similarity among all adjacent hash value calculations.

9. A data integrity verification system based on a hash algorithm, used to implement the data integrity verification method based on a hash algorithm as described in any one of claims 1 to 8, characterized in that: It includes hash value generation module, feature extraction and quantification module, machine learning prediction module, salt value generation and hash enhancement module, and dynamic adjustment and collision protection module; The hash value generation module processes the original input data through a hash algorithm to generate a hash value of a fixed length, collects the generated hash values ​​and constructs an analysis set, which contains the hash values ​​corresponding to all the input data; The feature extraction and quantification module extracts features related to hash value changes from the analysis set after it is built, quantifies the extracted features, and preliminarily identifies potential patterns in which hash value changes are not significant; The machine learning prediction module inputs the quantized features as feature vectors into the trained machine learning model, and uses the model to predict the changes in the hash value output results; The salt value generation and hash enhancement module, when it is recognized that the hash value change is not significant, combines the generated random salt value with the original data to form new input data, and then processes it through the hash algorithm to generate a new hash value, thereby improving the randomness and complexity of the hash result; The dynamic adjustment and collision protection module dynamically adjusts the number of hash iterations of the salt value according to the insignificant degree of hash value change to reduce the risk of hash collision.

Images