File integrity checking method, device, equipment, medium and product

Abstract

The application discloses a file integrity verification method, device, equipment, medium and product. The method comprises the following steps: acquiring a structure parameter of a file tree and historical performance data of a verification system, wherein the file tree is generated according to a plurality of data blocks constituting a to-be-verified file, and the verification system is used for performing an integrity verification task of the to-be-verified file; determining a verification mode according to the structure parameter and a file size critical value, wherein the file size critical value is determined according to the historical performance data, and the verification mode comprises serial verification and parallel verification; performing the integrity verification task on the to-be-verified file according to the verification mode; and determining a verification result output by the verification mode. The application solves the technical problem of low verification efficiency and long time consumption caused by the fact that the related art adopts a serial verification mode to implement file integrity verification.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a method, apparatus, device, medium, and product for verifying document integrity. Background Technology

[0002] In related technologies, file integrity verification is implemented using a single-threaded serial method. This method requires sequentially comparing the hash values ​​of each data block of the file to be verified to ensure that the data has not been tampered with. Therefore, it suffers from low verification efficiency.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides a method, apparatus, device, medium, and product for verifying file integrity, in order to at least solve the technical problems of low verification efficiency and long time consumption caused by the use of serial verification in related technologies to verify file integrity.

[0005] According to one aspect of the embodiments of this application, a file integrity verification method is provided, comprising: acquiring structural parameters of a file tree and historical performance data of a verification system, wherein the file tree is generated based on multiple data blocks constituting a file to be verified, and the verification system is used to perform an integrity verification task on the file to be verified; determining a verification method based on the structural parameters and a file size threshold, wherein the file size threshold is determined based on historical performance data, and when the data volume of the file to be verified is the file size threshold, the time consumed by performing the integrity verification task using different verification methods is the same, the verification methods include: serial verification and parallel verification; performing the integrity verification task on the file to be verified according to the verification method, wherein when the verification method is serial verification, the verification system uses one processor to perform the integrity verification task, and when the verification method is parallel verification, the verification system uses multiple processors to perform the integrity verification task; and determining the verification result output by the executed verification method.

[0006] Optionally, the verification method is determined based on the structural parameters and the file size threshold, including: comparing the file size threshold with the number of nodes recorded in the structural parameters to obtain a first comparison result, wherein the number of nodes indicates the total number of leaf nodes in the file tree; if the first comparison result indicates that the number of nodes is greater than or equal to the file size threshold, the verification method is determined to be parallel verification; if the first comparison result indicates that the number of nodes is less than the file size threshold, the verification method is determined to be serial verification.

[0007] Optionally, the file size threshold is determined by the following methods: The effective parallelism of the verification system is determined based on multiple performance scores recorded in historical performance data, where each performance score quantifies the performance of a processor, and the performance score of each processor is determined based on the throughput generated by the processor performing file integrity verification; the effective parallelism is the number of processors performing parallel verification. The baseline time of the verification system is determined based on multiple parallel verification times, multiple serial verification times, and the effective parallelism recorded in historical performance data, where each parallel verification time is the historical time taken by the verification system to perform parallel verification, each serial verification time is the historical time taken by the verification system to perform serial verification, and the baseline time is the time generated by performing parallel verification. Systematic time overhead, with the baseline time unaffected by the file to be verified; the single data block hash calculation time of the verification system is determined based on the verification time of multiple subtrees and the number of nodes in multiple subtrees recorded in historical performance data. Here, the verification time of each subtree is the time consumed by the verification system to process each subtree, the number of subtree nodes is the number of nodes contained in each subtree, and the average hash calculation time of a single data block is the average time spent calculating the hash value of all data blocks during the execution of the file integrity verification task; the file size threshold is determined based on the effective parallelism, the baseline time, the hash calculation time of a single data block, and the data caching time recorded in historical performance data. Here, the data caching time is the time taken for the verification system to read data blocks.

[0008] Optionally, the performance score of each processor is determined by the following method: when the verification system performs the file integrity verification task for the first time, the performance score of each processor is a preset score value; when the verification system performs the file integrity verification task for the second time, the performance score of each processor is an updated score value to be updated. Each time the score value to be updated is updated, the score result of the score value to be updated is determined based on the throughput generated by the processor in the current integrity verification task and the score value to be updated. When the performance score is updated for the first time, the score value to be updated is the preset score value; when the performance score is updated for the second time, the score value to be updated is the updated result of the previous score value.

[0009] Optionally, before determining the verification method based on structural parameters and file size thresholds, the process includes: determining a second comparison result between the effective parallelism and a preset number of processors; and determining the maximum and minimum performance scores among multiple performance scores, determining the ratio of the maximum to the minimum performance score, and determining a third comparison result between the ratio and a preset value; if the second comparison result indicates that the effective parallelism is less than or equal to the preset number of processors, or if the third comparison result indicates that the ratio is greater than the preset value, determining the verification method as serial verification; and if the second comparison result indicates that the effective parallelism is greater than or equal to the preset number of processors, and the third comparison result indicates that the ratio is less than or equal to the preset value, determining the verification method based on structural parameters and file size thresholds.

[0010] Optionally, the integrity verification task of the file to be verified is performed according to the verification method, including: when the verification method is serial verification, the file tree is allocated to a processor in the verification system, wherein one processor completes the integrity verification task of the file to be verified; when the verification method is parallel verification, the allocation scheme of the file tree is determined according to historical performance data and structural parameters, the file tree is split into multiple subtrees according to the allocation scheme, and each subtree is allocated to a corresponding processor, wherein multiple processors complete the integrity verification task of the file to be verified in parallel.

[0011] Optionally, determining the file tree allocation scheme based on historical performance data and structural parameters includes: determining a first number of subtrees based on the effective parallelism recorded in the historical performance data; and, for each processor, determining a second number of leaf nodes contained in the subtree allocated to the processor based on the effective parallelism and the performance score corresponding to the processor, wherein the performance score is recorded in the historical performance data; and, determining the depth of the subtree allocated to each processor based on the performance score corresponding to each processor, the lowest performance score, a preset subtree depth, and a preset fan-out, wherein the lowest performance score is the performance score with the smallest value among multiple performance scores; determining the subtree allocated to each processor in the file tree based on the second number and the depth of the subtree allocated to each processor, and generating an allocation scheme based on the subtree allocated to each processor and the first number, wherein different subtrees contain different nodes.

[0012] Optionally, determining the verification result output by the verification method includes: when the verification method is serial verification, obtaining a first hash value output by a processor, wherein the first hash value is calculated by a processor on the file tree in the order from leaf node to root node; if the first hash value is consistent with the target hash value pre-stored in the verification system, determining the verification result as verification passed; if the first hash value is inconsistent with the target hash value, determining the verification result as verification failed.

[0013] Optionally, determining the verification result output by the verification method further includes: when the verification method is parallel verification, obtaining the subtree verification result output by each processor, and aggregating multiple subtree verification results according to the level corresponding to the subtree verification result to obtain a second hash value, wherein each subtree verification result is the hash value of the root node of a subtree, and the second hash value is the hash value of the root node of the file tree; if the second hash value is consistent with the target hash value, determining the verification result as verification passed; if the second hash value is inconsistent with the target hash value, determining the verification result as verification failed.

[0014] Optionally, after determining the verification result output by the verification method, the method further includes: if the verification result is a successful verification, obtaining the performance data generated by the verification system during the integrity verification task of the file to be verified; determining a new file size threshold based on the new performance score corresponding to each performance score recorded in the performance data and historical performance data, and based on multiple new performance scores and performance data; updating the file size threshold to the new file size threshold if the difference between the new file size threshold and the file size threshold is greater than a preset difference; and stopping the update of the file size threshold if the difference between the new file size threshold and the file size threshold is greater than a preset difference.

[0015] Optionally, after determining the verification result output by the verification method, the method further includes: if the verification result is a verification failure, outputting verification failure information, wherein the verification failure information includes: the subtree where the hash value verification failed, and the failure to read the subtree.

[0016] According to another aspect of the embodiments of this application, a file integrity verification device is also provided, comprising: an acquisition module, configured to acquire structural parameters of a file tree and historical performance data of a verification system, wherein the file tree is generated based on multiple data blocks constituting the file to be verified, and the verification system is used to perform an integrity verification task on the file to be verified; a determination module, configured to determine a verification method based on the structural parameters and a file size threshold, wherein the file size threshold is determined based on historical performance data, and when the data volume of the file to be verified is the file size threshold, the time consumed by performing the integrity verification task using different verification methods is the same, the verification methods include: serial verification and parallel verification; an execution module, configured to execute the integrity verification task on the file to be verified according to the verification method, wherein when the verification method is serial verification, the verification system uses one processor to execute the integrity verification task, and when the verification method is parallel verification, the verification system uses multiple processors to execute the integrity verification task; and an output module, configured to determine the verification result output by executing the verification method.

[0017] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, which stores a computer program, wherein the above-described file integrity verification method is executed by running the computer program on the device where the non-volatile storage medium is located.

[0018] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-described file integrity verification method through the computer program.

[0019] According to another aspect of the embodiments of this application, a computer program product is also provided, including computer instructions, which, when executed by a processor, implement the steps of the above-described file integrity verification method.

[0020] In this embodiment, the structural parameters of the file tree and historical performance data of the verification system are obtained. The file tree is generated from multiple data blocks that make up the file to be verified, and the verification system is used to perform the integrity verification task of the file to be verified. The verification method is determined according to the structural parameters and the file size threshold. The file size threshold is determined based on historical performance data. When the data volume of the file to be verified is the file size threshold, the time taken to perform the integrity verification task is the same for different verification methods. The verification methods include serial verification and parallel verification. The integrity verification task is performed on the file to be verified according to the verification method. When the verification method is serial verification, the verification system uses one processor to perform the integrity verification task. When the verification method is parallel verification, the verification system uses multiple processors to perform the integrity verification task. The method of outputting the verification result of the execution verification method is determined. By dynamically evaluating the matching relationship between the data volume of the file to be verified and the file size threshold, it is determined whether to allocate the file integrity verification task of the file to be verified to multiple processors for parallel collaborative execution. This achieves the goal of maximizing the utilization of multi-core computing resources, thereby improving the efficiency of file integrity verification and shortening the verification time. This solves the technical problem of low verification efficiency and long verification time caused by the serial verification method used in related technologies. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0022] Figure 1 This is a hardware structure block diagram of a computer terminal for implementing a file integrity verification method according to an embodiment of this application;

[0023] Figure 2 This is a flowchart of the steps of a document integrity verification method according to an embodiment of this application;

[0024] Figure 3 This is a schematic diagram of a file tree according to an embodiment of this application;

[0025] Figure 4 This is a schematic diagram of a subtree allocation according to an embodiment of this application;

[0026] Figure 5 This is a timing flowchart of a parallel verification according to an embodiment of this application;

[0027] Figure 6 This is a structural diagram of a document integrity verification device according to an embodiment of this application;

[0028] Figure 7 This is a structural diagram of a component for performing parallel verification according to an embodiment of this application. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] To better understand the embodiments of this application, the technical terms involved in the embodiments of this application are explained below:

[0032] File tree: The file tree in this embodiment is also called a Merkle tree, which is a tree-shaped data structure. The value of each non-leaf node in the Merkle tree (file tree) is calculated by hashing the values ​​of its child nodes.

[0033] Filesystem Verity (fsverity): A file integrity verification mechanism provided by the Linux kernel, which uses a Merkle tree to perform block hash verification of file content.

[0034] Workqueue: A mechanism provided by the Linux kernel to delay the execution of tasks, which is scheduled and executed by the kernel's worker thread pool.

[0035] Worker Thread: An execution unit in a work queue, the number of which is usually related to the number of processor (CPU) cores.

[0036] Fanout: The maximum number of child nodes that each node in a tree can have.

[0037] In related technologies, the Linux kernel's Filesystem Verity mechanism calculates hash values ​​by dividing files into fixed-size blocks and constructing multi-level Merkle trees. Using the root hash as the sole trust anchor, it serially verifies the complete path from leaf nodes to the root node during file access. This single-threaded serial verification method suffers from high initial file load latency, low multi-core CPU resource utilization, and low verification efficiency. Additionally, related technologies attempt to pre-calculate and cache the Merkle tree verification results in user space to avoid the overhead of online verification. For example, during file deployment, the `fsveritymeasure` command pre-calculates and caches the root node's hash value; at runtime, the pre-calculated root hash is used for fast comparison, skipping the actual node-by-node verification. This approach achieves low runtime latency by moving computationally intensive work to the deployment phase; however, it is unsuitable for files with dynamic content. Furthermore, even with pre-computed root hashes, the Merkle tree nodes still need to be loaded from disk into memory and verified during the initial check, and the user-space pre-computation tool itself is executed serially, making the pre-computation process time-consuming. Therefore, the problems of low verification efficiency and long processing time persist. To address these issues, this application provides relevant solutions, which are detailed below.

[0038] According to an embodiment of this application, a method embodiment for verifying file integrity is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0039] The methods and embodiments provided in this application can be executed on mobile terminals, computer terminals, or similar computing devices. Figure 1 A hardware block diagram of a computer terminal for implementing a file integrity verification method is shown. Figure 1 As shown, the computer terminal 10 may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0040] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10. As involved in the embodiments of this application, the data processing circuits serve as processor control (e.g., selection of a variable resistor termination path connected to an interface).

[0041] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the file integrity verification method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the aforementioned file integrity verification method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0042] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0043] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10.

[0044] This application provides a file integrity verification method that can run in the above-described operating environment. Figure 2 This is a flowchart of the steps of the file integrity verification method provided in the embodiments of this application, such as... Figure 2 As shown, the method includes the following steps:

[0045] Step S202: Obtain the structural parameters of the file tree and the historical performance data of the verification system. The file tree is generated based on multiple data blocks that make up the file to be verified, and the verification system is used to perform the integrity verification task of the file to be verified.

[0046] This application provides a file integrity verification method with selectable verification methods. In the file integrity verification method provided in this application, the specific verification method to be used for the file integrity verification of the file to be verified is determined according to the file size of the file to be verified and the performance of the verification system that performs the integrity verification task on the file to be verified. Therefore, in step S202, before starting the file integrity verification, the structural parameters of the file tree generated by the file to be verified and the historical performance data of the verification system are first obtained. The file tree is a Merkle tree generated by dividing the file to be verified into blocks. The structural parameters of the file tree include: the total number of leaf nodes, the tree depth, the fan-out degree, etc. The historical performance data of the verification system is used to describe the performance of the verification system in the past execution of file integrity verification, including: the performance score of each CPU core in the verification system, the execution time of past tasks, and the file reading time (or IO wait time).

[0047] Step S204: Determine the verification method based on the structural parameters and the file size threshold. The file size threshold is determined based on historical performance data. When the data volume of the file to be verified is the file size threshold, the time taken to perform the integrity verification task using different verification methods is the same. The verification methods include serial verification and parallel verification.

[0048] In step S204, the specific verification method for performing the file integrity verification task on the file to be verified is determined based on the file size of the file to be verified and the performance of the verification system. The file size of the file to be verified specifies the amount of data in the file to be verified, which is determined based on the structural parameters obtained in step S202; the performance of the verification system is determined based on historical performance data. In the method provided in this application embodiment, an adaptive convergence threshold (i.e., a file size critical value) is determined based on the historical performance data of the verification system. This file size critical value represents the file size (the amount of data in the file) for which serial verification and parallel verification take equal time. That is, when the amount of data in the file to be verified is the file size critical value, the verification time generated by using multiple processors to execute the file integrity verification task in parallel is equal to the verification time generated by using one processor to execute the file integrity verification task. Therefore, in step S204, it can be determined whether the verification method used this time is serial verification or parallel verification based on the structural parameters and the aforementioned file size critical value.

[0049] Optionally, the verification method is determined based on the structural parameters and the file size threshold, including: comparing the file size threshold with the number of nodes recorded in the structural parameters to obtain a first comparison result, wherein the number of nodes indicates the total number of leaf nodes in the file tree; if the first comparison result indicates that the number of nodes is greater than or equal to the file size threshold, the verification method is determined to be parallel verification; if the first comparison result indicates that the number of nodes is less than the file size threshold, the verification method is determined to be serial verification.

[0050] In step S204, when determining the verification method based on the structural parameters of the file tree and the file size threshold, the number of nodes (L) is extracted from the structural parameters of the Merkle tree (i.e., the file tree). Specifically, the number of nodes (L) refers to the number of data blocks that make up the file tree, which is the number of nodes contained in the bottom layer of the file tree (the layer furthest from the root node). Figure 3 This is a diagram of the file tree. Figure 3 The Merkle tree shown has three levels. The level containing data blocks D0, D1, D2, and D3 is the bottom level of the file tree. Therefore, Figure 3 The number of nodes (L) in the file tree shown is 4; in Figure 3 In the file tree shown, H1 is the hash value calculated based on the hash values ​​of data block D0 and data block D1, H2 is the hash value calculated based on the hash values ​​of data block D2 and data block D3, and the root hash is... Figure 3 The hash value of the root node of the file tree shown, such as Figure 3As shown, the root hash is calculated from H1 and H2. According to the above description of the file tree, the hash value of each parent node in the file tree implicitly contains the integrity information of the contents of all its child nodes. Therefore, tampering with the content of any leaf node will result in a mismatch in the root hash. However, as long as the root hash is reliable, the entire tree can be verified by checking the root hash. Therefore, the method provided in this embodiment can verify the integrity of the file represented by the file tree by checking the root hash of the file tree. In this embodiment, in order to verify the file to be verified by checking the root hash of the file tree, the file tree obtained in step S202 is generated after dividing the file to be verified into multiple data blocks. Assuming the file to be verified is divided into N data blocks (leaf nodes), the first level of the file tree contains N hash nodes. Each node in the second level of the file tree is calculated by pairwise hashing of the hash nodes in the first level, and so on, until the root node has only one hash value (RootHash).

[0051] Next, the number of nodes (L) extracted from the structural parameters above will be compared with the file size threshold ( A comparison is performed, and the verification method is determined based on the comparison result (i.e., the first comparison result). Where L ≥ If the file size is deemed suitable for parallel processing (i.e., parallel verification consumes fewer resources and takes less time compared to serial verification), then parallel verification mode is initiated, dividing the tree into multiple subtrees and distributing them to multi-core worker threads. If L < If the resource overhead generated by parallel verification exceeds the resource overhead saved, the system will revert to serial verification mode and traverse the entire tree sequentially by a single thread.

[0052] The method provided in this application determines the verification method based on the dynamic threshold (i.e., the file size threshold) for switching verification modes determined according to real system performance data and the actual size of the verification file. This enables intelligent switching between serial and parallel verification methods. Serial verification is used for small files with data volume less than the file size threshold, while parallel verification is used for large files with data volume greater than the file size threshold. This ensures that the system can perform file integrity verification with optimal verification efficiency regardless of file size.

[0053] According to some optional embodiments of this application, the file size threshold is determined by the following method: The effective parallelism of the verification system is determined based on multiple performance scores recorded in historical performance data, wherein each performance score quantifies the performance of a processor, the performance score of each processor is determined based on the throughput generated by the processor performing file integrity verification, and the effective parallelism is the number of processors performing parallel verification; a baseline time for the verification system is determined based on multiple parallel verification times, multiple serial verification times, and the effective parallelism recorded in historical performance data, wherein each parallel verification time is the historical time taken for the verification system to perform parallel verification, each serial verification time is the historical time taken for the verification system to perform serial verification, and the baseline time is the time taken for the parallel verification system to perform serial verification. The systemic time overhead of verification is not affected by the file to be verified. The single data block hash calculation time of the verification system is determined based on the verification time of multiple subtrees and the number of nodes in multiple subtrees recorded in historical performance data. Here, the verification time of each subtree is the time consumed by the verification system to process each subtree, the number of subtree nodes is the number of nodes contained in each subtree, and the average hash calculation time of a single data block is the average time spent calculating the hash value of all data blocks during the execution of the file integrity verification task. The file size threshold is determined based on the effective parallelism, the baseline time, the hash calculation time of a single data block, and the data caching time recorded in historical performance data. Here, the data caching time is the time taken for the verification system to read data blocks.

[0054] As mentioned in the above embodiments, the file size threshold is determined based on the historical performance data of the verification system. Specifically, the file size threshold is determined as follows: The method can be described as a formula: In the above formula, The effective parallelism of the verification system refers to the number of processors in the system that can perform file integrity verification in parallel. In this embodiment, the effective parallelism can be based on the performance score of each processor recorded in historical performance data. For example, the method for determining the effective degree of parallelism can be described by a formula: .

[0055] The above determines the critical value of file size ( In the formula, This refers to the fixed time overhead (i.e., baseline time) of the verification system using a parallel verification method to perform file integrity verification tasks. This time overhead is independent of the amount of data in the file to be verified. This is because parallel verification requires simultaneously starting multiple processors. Therefore, while parallel verification can reduce verification time, it also introduces additional time overhead. For example, starting multiple processors introduces additional time overhead compared to starting a single processor. Generally, the systemic additional time overhead of performing parallel verification is greater than that of performing serial verification. ,in, It is the total additional overhead time introduced by parallelization. This is the baseline duration for serial verification, and the aforementioned systemic additional time overhead ( It mainly consists of the following parts: the time overhead of subtree creation and task descriptor initialization ( The time overhead of work queue scheduling and context switching ( The time overhead of waiting for synchronization between subtrees () ), and the time overhead of result aggregation and root hash calculation ( ) Determined, that is Since the aforementioned overheads cannot be directly measured, this embodiment of the application indirectly estimates the time overhead by using the actual execution time. In this embodiment, the time overhead is estimated based on the actual time taken for the verification system to perform one parallel verification (…). ) and the duration of a single serial check based on CPU performance estimation ( Determine the fixed time overhead (i.e., baseline time) for the verification system to implement file integrity verification tasks using a parallel verification method. Specifically, the baseline time ( The process of determining ) can be described by the formula: In this embodiment of the application, in order to improve the estimation accuracy, the calculation file size threshold ( When using ) The value is the average of multiple measurements. ),in, K represents the number of times the measurement and verification system performs parallel verification. This is the fixed overhead incurred by the system performing parallel verification during the Kth measurement and verification cycle. , It is the actual time for the verification system to perform one parallel verification during the k-th measurement. ), It is the serial verification time estimated based on CPU performance during the k-th measurement. ).

[0056] The above determines the critical value of file size ( In the formula, This refers to the time required for hash calculation on a single data block. The method for determining it can be described by the formula: ,in, This refers to the time consumed by the processor to process the i-th subtree during parallel verification, specifically the time consumed to calculate the hash value of the root node of the i-th subtree. This represents the number of nodes contained in the i-th subtree (i.e., the number of subtree nodes). In this embodiment, when counting the number of subtree nodes for each subtree, it is necessary to simultaneously count the number of leaf nodes and the number of intermediate nodes in the subtree. That is, the number of subtree nodes is the sum of the number of leaf nodes and the number of intermediate nodes in the subtree. It represents seeking a settlement.

[0057] The above determines the critical value of file size ( In the formula, It is the proportion of I / O wait time to hash calculation time. I / O wait time is the time it takes to read data blocks of the file to be verified through the input / output interface (I / O) (i.e., data cache duration).

[0058] If the verification system has performed both serial and parallel verification, the historical performance data records the latest performance scores of each processor in the verification system (updated performance scores based on the performance data from the most recent parallel verification), baseline latency, single-block hash calculation time, and data caching duration. Therefore, the file size threshold can be determined based on the above data recorded in the historical performance data. If the verification system has not performed file integrity verification during initialization, the performance score will be the preset score value (e.g., 1), and the baseline time, effective parallelism, single data block hash calculation time, and data caching time will be recorded in the manually entered input data.

[0059] The method provided in this embodiment determines the file size threshold based on the performance data of actual file integrity verification, realizes the adaptive update of the file size threshold, improves the adaptability of the file size threshold to actual application scenarios, improves the accuracy of verification mode switching, and improves verification efficiency.

[0060] According to some alternative embodiments of this application, the performance score of each processor is determined by the following method: when the verification system performs the file integrity verification task for the first time, the performance score of each processor is a preset score value; when the verification system performs the file integrity verification task for the second time, the performance score of each processor is an updated score value to be updated, wherein each time the score value to be updated is updated, the score result of the score value to be updated is determined based on the throughput generated by the processor in the current execution of the integrity verification task and the score value to be updated. When the performance score is updated for the first time, the score value to be updated is the preset score value; when the performance score is updated for the second time, the score value to be updated is the update result of the previous score value.

[0061] As mentioned in the above embodiments, the processor's performance score ( The performance data generated by the verification system performing file integrity checks is updated. Specifically, the method for updating the performance score of each processor can be described by the formula: ,in, This refers to the performance score of processor j recorded in the historical performance data (i.e., the score to be updated). Since the performance score is updated after each file integrity check, the historical performance data actually records the initial performance score that has been updated M times. M times is the number of times the verification system has performed file integrity checks. Each performance score update is based on the performance score after the previous update, and it updates the result of the previous score update. The initial performance score is the performance score value preset for each processor when the verification system first performs the file integrity check task (i.e., the preset score value). The initial performance score value is the same for each processor. It is the actual throughput generated by processor j performing hash calculations on the subtree assigned to it. ,in, It is the number of leaf nodes (the number of bottom-level nodes) of the subtree allocated to processor j. It is the actual time consumed by the processor to perform hash calculations on the subtree. This represents the average throughput of processors performing file integrity checks in parallel during parallel verification. N is the number of processor cores. A processor core is the smallest processing unit in a processor that can independently execute a task. Smoothness coefficient ( This is used to control the weighting of new and old scores. In this embodiment, The value range is 0.3 to 0.5.

[0062] The method provided in this application embodiment adaptively updates its performance score based on the actual throughput of the processor performing file integrity verification, so as to dynamically identify and quantify the real-time computing power differences of heterogeneous CPU cores during actual verification, realize the accurate allocation of subtree load, and thereby achieve the technical effect of continuously optimizing the balance and efficiency of parallel verification and improving resource utilization.

[0063] Optionally, before determining the verification method based on structural parameters and file size thresholds, the process includes: determining a second comparison result between the effective parallelism and a preset number of processors; and determining the maximum and minimum performance scores among multiple performance scores, determining the ratio of the maximum to the minimum performance score, and determining a third comparison result between the ratio and a preset value; if the second comparison result indicates that the effective parallelism is less than or equal to the preset number of processors, or if the third comparison result indicates that the ratio is greater than the preset value, determining the verification method as serial verification; and if the second comparison result indicates that the effective parallelism is greater than or equal to the preset number of processors, and the third comparison result indicates that the ratio is less than or equal to the preset value, determining the verification method based on structural parameters and file size thresholds.

[0064] In this embodiment, based on the total number of leaf nodes (L) of the Merkle tree and the file size threshold ( Before determining the verification method, first determine the effective parallelism of the verification system. Compare with the preset number of processors, according to The comparison result with the preset number of processors (i.e., the second comparison result) determines whether the parallel verification performed by the verification system will generate benefits; at the same time, it determines the maximum performance score among all CPU performance scores. ) and minimum performance score ( The ratio of ) The ratio ( ) is compared with the preset value (10), and the ratio is determined according to the ratio ( The comparison result between 1 and 10 (i.e., the third comparison result) determines the performance difference of heterogeneous CPUs in the verification system; if A value ≤2 (preset processor number threshold) indicates limited benefits from parallelization; if A value >10 indicates extreme heterogeneity in the system, where the subtree balancing algorithm has limited effectiveness. Therefore, when any of the above conditions occur, the system reverts to serial verification mode, only applying the algorithm when... >2 and Only when the value is ≤10 will step S204 be executed, which involves determining the threshold based on the structural parameters and file size. The process for determining the verification method.

[0065] The reason for implementing the above solution is that the solution provided in this application embodiment ensures the benefits of parallelization ( Parallel verification is only used when the value is greater than 0. The benefits of parallelization ( This refers to the time consumed when performing file integrity checks using a parallel verification method for the same file to be verified. Compared to the time consumed by performing file integrity checks using the serial verification method. Time saved, and ≤2 (preset processor number threshold) means that the number of processors capable of performing file integrity checks is limited, thus limiting the benefits of parallelization. When the value is greater than 10, the performance differences between processors are significant. The time consumed by waiting for multiple processors to execute synchronously may outweigh the time saved by parallel verification, leading to an increase in overall execution time. In other words, the benefits of parallelization cannot be guaranteed in either of these scenarios. Since the value is greater than 0, falling back to serial parity mode is a better choice.

[0066] The method provided in this embodiment introduces a dual security verification mechanism based on the effective parallelism and the ratio of core performance heterogeneity. This avoids blindly starting parallel verification in extreme scenarios where CPU heterogeneity is too high or parallelism is insufficient, thus avoiding situations such as task allocation imbalance and a surge in synchronization latency, and improving the stability of file integrity verification.

[0067] Step S206: Perform integrity verification task on the file to be verified according to the verification method. Wherein, when the verification method is serial verification, the verification system uses one processor to perform integrity verification task. When the verification method is parallel verification, the verification system uses multiple processors to perform integrity verification task.

[0068] In step S206, the verification system performs file integrity verification on the file to be verified using the verification method determined in step S204. Specifically, when the verification method is serial verification, a processor in the verification system is started to perform file integrity verification on the file to be verified. In the serial verification method, a CPU in the verification system traverses the nodes of the Merkle tree (i.e., the file tree) from the leaf node to the root node in a bottom-up order, and calculates the hash value of each node in turn until the root node of the Merkle tree is obtained.

[0069] When the verification method is parallel verification, multiple processors in the verification system are started to execute the integrity verification task of the file to be verified in parallel. Specifically, in the parallel verification method, the file tree is divided into multiple subtrees according to the performance of the verification system and the structural parameters of the file tree. The subtrees are divided into multiple subtrees from bottom to top (from the leaf node to the root node). Each subtree is assigned an independent verification task and bound to the corresponding processor. Multiple worker threads concurrently execute the hash calculation and verification within the subtree. Finally, the root hash of each subtree is aggregated and the root node of the file tree is verified.

[0070] Optionally, the integrity verification task of the file to be verified is performed according to the verification method, including: when the verification method is serial verification, the file tree is allocated to a processor in the verification system, wherein one processor completes the integrity verification task of the file to be verified; when the verification method is parallel verification, the allocation scheme of the file tree is determined according to historical performance data and structural parameters, the file tree is split into multiple subtrees according to the allocation scheme, and each subtree is allocated to a corresponding processor, wherein multiple processors complete the integrity verification task of the file to be verified in parallel.

[0071] In this embodiment, when the verification method is serial verification, the system assigns the root node verification task of the entire Merkle tree to a single processor. This processor traverses all nodes layer by layer from the lowest level to the top level (the level where the root node is located) of the file tree, calculating the hash value of each node until the hash value of the root node of the file tree is obtained. When the verification method is parallel verification, the dynamic performance score of each CPU core recorded in the historical performance data of the verification system is used. The allocation scheme is determined based on the structural parameters of the Merkle tree (including the total number of leaf nodes L, fan-out degree, number of levels, etc.). The allocation scheme records the number of subtrees to be split into in the file tree, the boundary of each subtree, and the subtree allocated to each processor. According to the above allocation scheme, each subtree obtained from splitting the file tree is assigned to the corresponding processor. These multiple processors execute the integrity verification task in parallel. When multiple processors execute the integrity verification task in parallel, each processor performs file integrity verification on the subtree allocated to it.

[0072] The method provided in this embodiment dynamically optimizes the subtree partitioning scheme of the file tree based on the actual performance of the processor during parallel verification, realizing optimal resource adaptation of the verification task under files of different sizes and heterogeneous hardware, and improving the verification efficiency of file integrity verification.

[0073] According to some optional embodiments of this application, determining the allocation scheme of the file tree based on historical performance data and structural parameters includes: determining a first number of subtrees based on the effective parallelism recorded in the historical performance data; and, for each processor, determining a second number of leaf nodes contained in the subtree allocated to the processor based on the effective parallelism and the performance score corresponding to the processor, wherein the performance score is recorded in the historical performance data; and, determining the depth of the subtree allocated to each processor based on the performance score corresponding to each processor, the lowest performance score, a preset subtree depth, and a preset fan-out, wherein the lowest performance score is the performance score with the smallest value among multiple performance scores; determining the subtree allocated to each processor in the file tree based on the second number and the depth of the subtree allocated to each processor, and generating an allocation scheme based on the subtree allocated to each processor and the first number, wherein different subtrees contain different nodes.

[0074] In parallel verification mode, determining the subtree allocation scheme requires determining the number of subtrees (i.e., the first quantity, representing how many subtrees the file tree will be divided into), the number of leaf nodes in each subtree (i.e., the second quantity), and the depth of the subtrees. In this embodiment, the effective parallelism is calculated based on historical performance data. Determine the number of subtrees (i.e., the first number), where, will be related to... The integer with the smallest difference that does not exceed the total number of CPU cores is determined as the number of subtrees.

[0075] The number of leaf nodes contained in the subtree allocated to each processor (i.e., the second number) is determined by: for processor j, based on its performance score. Sum of all core performance scores ( ), proportional Calculate the number of leaf nodes contained in the subtree assigned to it (i.e., the second number). This ensures that high-performance cores can handle more computing loads.

[0076] The depth of the subtree allocated to each processor is according to... The method described is determined, wherein, It is the floor function. It is the depth of the subtree allocated to processor j. It is the preset subtree depth, and F is the fan-out degree of the file tree. In this embodiment, when the performance score of processor i ( ) and the performance score of processor j When the ratio of ) is greater than F (fan-out of the file tree), simple adjustment within the same level cannot meet the balance requirements. In this case, the subtree depth of the high-scoring CPU is allowed to increase, so that it covers more levels. The root node of the cross-level subtree can be located at different levels of the original Merkle tree. It is the lowest recorded performance score from the historical performance data used to determine the depth of the subtree.

[0077] After determining the above information, based on the number of leaf nodes allocated to each processor Depth of subtree In the Merkle tree, starting from the bottom leaf node, the subtree boundaries are divided from bottom to top (from the bottom node to the root node) to ensure that the subtree nodes do not overlap and are completely covered. Finally, a complete allocation scheme containing subtree ID, root node position, leaf range and bound CPU is generated.

[0078] Figure 4 This is a diagram illustrating subtree allocation, such as... Figure 4 As shown, the method provided in the embodiments of this application is used to score processor performance. Subtrees are assigned to them, and the allocation results are as follows: Subtree 0, which contains more leaf nodes (D0-D3 and D_extra1, D_extra2) and has a greater computational load, is assigned to the high-performance CPU with a higher performance score, and subtree 1, which contains fewer leaf nodes (D4-D7) and has a smaller computational load, is assigned to the low-performance CPU.

[0079] The method provided in this application achieves fine-grained load balancing of subtrees under heterogeneous CPU architecture by integrating a multi-dimensional collaborative allocation mechanism that combines effective parallelism, performance scoring ratio, and dynamic subtree depth. This unbalanced subtree allocation enables large cores and small cores to complete their respective verification tasks in similar timeframes, avoiding situations where large cores are idle while waiting for small cores. This avoids the long-tail latency problem present in traditional equal-distribution strategies and shortens the execution time of parallel verification.

[0080] Optionally, determining the verification result output by the verification method includes: when the verification method is serial verification, obtaining a first hash value output by a processor, wherein the first hash value is calculated by a processor on the file tree in the order from leaf node to root node; if the first hash value is consistent with the target hash value pre-stored in the verification system, determining the verification result as verification passed; if the first hash value is inconsistent with the target hash value, determining the verification result as verification failed.

[0081] When the verification method performed in step S206 is serial verification, the system reads the corresponding data block or child node hash sequentially from the bottom leaf node along the Merkle tree by a single CPU core, and calculates the SHA-256 hash value of each node in turn until a unique root node hash value (i.e., the first hash value) is finally generated. This process strictly follows the native verification logic of fsverity and does not skip any intermediate nodes. Then, the first hash value is compared byte by byte with the root hash value (i.e., the target hash value) that is pre-generated and stored by the trusted signature key in the file metadata. If they are completely consistent, it is determined that the file content has not been tampered with and the verification result is passed. If there is any bit difference, it is determined that the file integrity has been damaged and the verification result is failed.

[0082] The method provided in this application embodiment performs file integrity verification based on the security semantics of fsverity, ensuring the absolute consistency and traceability of the verification results in serial mode, and guaranteeing the accuracy of file integrity verification performed in low-performance or small file scenarios.

[0083] According to some optional embodiments of this application, determining the verification result output by the verification method further includes: when the verification method is parallel verification, obtaining the subtree verification result output by each processor, and aggregating multiple subtree verification results according to the level corresponding to the subtree verification result to obtain a second hash value, wherein each subtree verification result is the hash value of the root node of a subtree, and the second hash value is the hash value of the root node of the file tree; if the second hash value is consistent with the target hash value, determining the verification result as verification passed; if the second hash value is inconsistent with the target hash value, determining the verification result as verification failed.

[0084] Figure 5 This is the timing flowchart for parallel verification, such as... Figure 5 As shown, when the verification method executed in step S206 is parallel verification, after all subtree verification tasks are completed, the inter-subtree synchronization controller waits for all worker threads to return results through the wait_for_completion mechanism, collects the subtree root node hash value (i.e., subtree verification result) output by each CPU, and uses these subtree root hash values ​​as input according to the hierarchical structure of the subtree in the original Merkle tree, performing hash aggregation layer by layer upwards according to the fan-out degree of the original tree. For example, if there are 4 subtrees, and the hash values ​​of these four subtree roots correspond to the four nodes of the second level of the original tree, then they are combined in pairs to calculate the hash of the parent node, obtaining two intermediate hashes of the first level, and then further merged to generate the hash value of the root node of the file tree (i.e., the second hash value). This aggregation process strictly reuses the original hash combination logic of fsverity, only changing the calculation order without changing the algorithm, ensuring complete equivalence with the serial verification result. Finally, the second hash value is compared with the pre-stored, signed and authenticated target root hash value (RootHash) in the file extension attributes. If they are completely identical, the verification is considered successful; otherwise, it is considered a failure.

[0085] Still Figure 5As shown, the decision to perform parallel verification is triggered by the following steps: The Virtual File System (VFS) layer initiates the first fsverity verification request to the subtree partitioning manager; the subtree partitioning manager initializes the verification context to allocate CPU status arrays. Specifically, the subtree partitioning manager reads the structural parameters of the file tree (i.e., the Merkle tree) (including the number of levels, leaf nodes, fan-out, etc.) and reads the performance scores of each CPU from the performance evaluation module to determine the convergence threshold (i.e., the file size critical value); if the result of the above convergence threshold (i.e., the file size critical value) determination is that the current file size is less than the convergence threshold, then the serial verification method is used; a processor is selected to complete the integrity verification of the current file, and then the verification result is returned to the Virtual File System (VFS) layer. Otherwise, if the result of the above convergence threshold (i.e., the file size critical value) determination is that the current file size is greater than or equal to the convergence threshold, then the integrity verification of the current file is determined to be performed using parallel verification.

[0086] Figure 5 The document describes the steps performed by each component when using parallel verification. Specifically, the subtree partitioning manager constructs subtrees from the bottom up from the file tree (i.e., the Merkle tree), based on the above... Calculate the number of leaf nodes contained in the subtree assigned to CPUj (i.e., the second number). The process creates k subtree verification tasks (the number of CPUs performing parallel verification) and sends these tasks to the work queue adaptation layer. The work queue adaptation layer submits all subtree verification tasks to the kernel worker thread pool and assigns each task to a target CPU core for execution. When worker threads execute subtree verification in parallel, multiple worker threads in the kernel worker thread pool retrieve subtree verification tasks from the work queue and execute them, such as... Figure 5As shown, each worker thread independently completes the full verification process (loop) of a subtree: Each worker thread first records the start time, then traverses the subtree according to the range of leaf nodes in the task descriptor. Each thread starts from a leaf node and traverses all nodes of its assigned subtree level by level upwards. During the traversal, for each node in the subtree (whether a leaf node or an intermediate node), a data block is read from the page cache or disk, and its hash value is calculated. The hash values ​​are aggregated level by level in the subtree from leaf to root until the hash value of the root node of the subtree is calculated. Finally, the thread records the end time and writes the subtree root hash value to the per-CPU result cache. The inter-subtree synchronization controller and the result aggregator main thread wait for all subtree tasks to complete by calling a function (wait_for_completion) or by polling. Once all subtrees have been verified, the inter-subtree synchronization controller triggers the aggregation process, which is the hash aggregation and comparison process described in this embodiment: read the root node hash values ​​of all subtrees from the result cache, and calculate the parent node hashes layer by layer upwards according to the original hierarchical structure of the Merkle tree, using these subtree root hash values ​​as input, until the root hash value of the file tree (i.e., the second hash value) is calculated.

[0087] The method provided in this application improves the speed of file integrity verification and ensures the accuracy of file integrity verification results by verifying each subtree in parallel and strictly aggregating the hash values ​​of the subtree roots according to the Merkle tree hierarchy, without changing the security semantics of fsverity.

[0088] Step S208: Determine the verification result output by the verification method.

[0089] In step S208, the verification result generated after executing the verification method in step S206 is determined. For serial verification, the output verification result is determined by comparing the root hash (the hash value of the root node of the file tree) output by the processor performing the integrity verification with the pre-stored signature root hash. For parallel verification, the results output by multiple processors are aggregated to obtain an aggregated result, and then the aggregated result is compared with the pre-stored signature root hash to determine the output verification result. The aggregated result is the hash value of the root node of the file tree determined by the parallel verification method. The above verification results include two types: verification failure and verification success. The verification result is considered successful only if the hash value output by the execution of the verification method (including the hash value output by serial verification and the aggregated result output by parallel verification) matches the pre-stored signature root hash.

[0090] In addition, in the method provided in this application embodiment, if the verification result is successful, but the file size threshold has not reached a convergence state, the file size threshold used in step S204 will be updated according to the performance data generated by the verification system in this file integrity verification. If the file size threshold has reached a convergence state, there is no need to update the file size threshold. However, if the file size threshold has been updated, the updated file size threshold will be used in the next file integrity verification.

[0091] According to some embodiments of this application, after determining the verification result output by the verification method, the method further includes: if the verification result is a successful verification, obtaining the performance data generated by the verification system during the execution of the integrity verification task on the file to be verified; determining a new file size threshold based on the new performance score corresponding to each performance score recorded in the performance data and historical performance data, and based on multiple new performance scores and performance data; updating the file size threshold to the new file size threshold if the difference between the new file size threshold and the file size threshold is greater than a preset difference; and stopping the updating of the file size threshold if the difference between the new file size threshold and the file size threshold is greater than a preset difference.

[0092] Still Figure 5 As shown in the embodiments of this application, the method also supports updating the convergence threshold (i.e., the file size threshold) after executing the verification method. In this embodiment, when the result of the parallel verification is passed, performance data such as the actual execution time of each CPU core, the number of allocated leaf nodes, and the task creation and synchronization overhead during this verification process are collected. Combined with historical performance scores (such as the performance scores saved after the last parallel verification), the performance score is updated according to the exponentially weighted moving average (EWMA) method. The method for updating the performance score can be described as the formula... ,in, This is the updated result of the performance score of processor i (i.e., the new performance score). It is the throughput of processor i in this parallel verification operation. It is the performance score of processor i recorded in historical performance data. It is the average throughput. It is a smoothing coefficient. The effective parallelism is recalculated after updating the performance score for each processor. ,Will Substitute into the convergence threshold formula The system obtains a new file size threshold. Then, it compares the absolute difference between the old and new thresholds. Only when the difference exceeds a preset threshold (e.g., 10%) will the original file size threshold be updated to the new value. Once the update is complete, subsequent updates are paused to avoid frequent fluctuations.

[0093] The method provided in this embodiment updates the file size threshold based on the processor's real-time performance data, realizing intelligent switching of the parallel verification strategy under different file sizes and hardware loads. This avoids ineffective parallel overhead when performing parallel verification on small files and improves verification efficiency.

[0094] In another embodiment, the system environment can be taken into account when updating the file size threshold. When the system environment is considered, the file size threshold is not only dynamically updated based on CPU performance scores, but also further integrated with real-time system environment factors for adaptive correction. Specifically, the system obtains the current average load (load_avg) by reading / proc / loadavg, and uses the number of CPU cores as the baseline load (load_base). The system load factor f_load is determined according to the rule defined by the formula f_load=1+λ×(load_avg-load_base) (λ is 0.1~0.3). When the system is under high load, f_load>1, and the threshold is appropriately increased to suppress parallelization overhead. Simultaneously, the system continuously calculates the ratio of current IO wait time to hash calculation time. (r_io_current) is compared with the historical benchmark r_io_base (recorded in historical performance data). The storage performance factor (f_io) is determined according to the rules defined by the formula f_io=r_io_current / r_io_base. When storage latency increases, f_io>1, and the threshold is actively reduced to enhance parallel benefits and mask IO bottlenecks. Finally, the file size threshold (L_threshold) used in this judgment and verification method is multiplied by the preset coefficient β (recommended 1.2~1.5), the load factor f_load, and the storage factor f_io to obtain the final environment-aware threshold L_threshold_final, that is, L_threshold_final=L_threshold×β×f_load×f_io. In this embodiment, an update is performed only when the relative change between this value and the original threshold (L_threshold) exceeds a preset difference (e.g., 10%); otherwise, the original value is maintained. This mechanism enables dynamic perception and intelligent threshold optimization of multi-dimensional operating environments (CPU load, storage media performance), allowing the parallel verification strategy to accurately grasp performance inflection points under complex operating conditions such as high server concurrency and heterogeneous storage, significantly improving the stability, adaptability, and energy efficiency of the system in real deployment scenarios.

[0095] Optionally, after determining the verification result output by the verification method, the method further includes: if the verification result is a verification failure, outputting verification failure information, wherein the verification failure information includes: the subtree where the hash value verification failed, and the failure to read the subtree.

[0096] In this embodiment, when any subtree verification task fails during parallel verification due to data block read failure (such as disk I / O error, page cache invalidation) or hash value mismatch (such as content tampering), the subtree verification executor will report the failure reason (such as error code EIO or hash comparison inconsistency) along with the corresponding subtree ID, start and end leaf node range, and target hash value to the verification context manager. After summarizing all subtree results, if the verification context manager detects any task failure, it will terminate the remaining unfinished tasks and, while returning the verification failure result, output precise failure information to the user or log system, including: the subtree number of the hash verification failure and the file byte range it covers, as well as the disk block address or page cache index corresponding to the subtree that could not be read due to I / O abnormality, thereby achieving fine-grained and traceable fault location.

[0097] The method provided in this embodiment ensures security while providing accurate fault diagnosis basis for system operation and maintenance, which can shorten the anomaly investigation time and improve the maintainability of file integrity verification.

[0098] Through the above steps, the system intelligently selects the verification method by dynamically sensing the system load and hardware heterogeneity characteristics. Without changing the fsverity security model, the system automatically selects the optimal verification method based on the file size and system capabilities, thereby reducing the latency of the first file access, improving the multi-core utilization of the verification system, and increasing the verification speed.

[0099] Figure 6 This is a structural diagram of the document integrity verification device provided in the embodiments of this application, such as... Figure 6 As shown, the file integrity verification device includes: an acquisition module 60, used to acquire the structural parameters of the file tree and the historical performance data of the verification system, wherein the file tree is generated based on multiple data blocks that make up the file to be verified, and the verification system is used to perform the integrity verification task of the file to be verified; a determination module 62, used to determine the verification method based on the structural parameters and the file size threshold, wherein the file size threshold is determined based on the historical performance data, and when the data volume of the file to be verified is the file size threshold, the time taken to perform the integrity verification task using different verification methods is the same, and the verification methods include: serial verification and parallel verification; an execution module 64, used to execute the integrity verification task of the file to be verified according to the verification method, wherein when the verification method is serial verification, the verification system uses one processor to execute the integrity verification task, and when the verification method is parallel verification, the verification system uses multiple processors to execute the integrity verification task; and an output module 66, used to determine the verification result output by the executed verification method.

[0100] Figure 7 This is an architecture diagram of the components that perform parallel verification, such as... Figure 7As shown, when execution module 64 performs file integrity verification tasks using a parallel verification method, execution module 64 and output module 66 jointly execute parallel verification by calling components such as the subtree partition manager, inter-subtree synchronization controller, verification context manager, work queue adapter, Merkle tree traverser, task result aggregator, memory allocator, subtree verification executor, subtree result cache, and subtree hash aggregator. Specifically, the subtree partition manager is responsible for the full management of the subtree's lifecycle and state. Its execution method is to split the Merkle tree into multiple independent subtrees and generate independent verification tasks for each subtree. The work queue adapter layer follows the logic of handling task failure retries. Its execution method is to distribute these subtree verification tasks to the kernel worker thread pool for parallel execution through kernel work queue mechanisms (such as alloc_workqueue and queue_work_on). The inter-subtree synchronization controller acts as a synchronization hub, ensuring that subsequent steps are triggered only after all subtree verification tasks are completed, and coordinating concurrency and synchronization points. The verification context manager centrally manages the shared state during concurrent processes, using per-CPU variables or locking mechanisms to protect the intermediate hash results and error states of each subtree, ensuring thread safety. During parallel verification, multiple subtree verification executors run in parallel, each independently executing the complete verification process from the subtree root node down to the leaf node, calculating the hash of all nodes within the subtree and returning the subtree root hash (i.e., the subtree verification result). The subtree result cache temporarily stores the root hashes of each subtree to avoid duplicate calculations and assists in managing the page cache of nodes within the subtree. The subtree hash aggregator collects the root node hash values ​​returned by all subtrees, aggregating them upwards to calculate the parent node hashes until the final root hash is verified. The Merkle tree traverser provides underlying support, traversing the Merkle tree structure by subtree range and returning information about the nodes to be verified. The task result aggregator summarizes the execution results of all subtree tasks, terminating the entire verification process once a failed subtree is detected. Furthermore, the memory allocator pre-allocates the necessary memory buffers (including the subtree hash result cache and task descriptions) before verification begins, significantly reducing runtime dynamic allocation overhead.

[0101] It should be noted that, Figure 6 Preferred embodiments of the shown examples can be found in [reference needed]. Figure 2 The relevant descriptions of the embodiments shown will not be repeated here.

[0102] This application also provides a non-volatile storage medium storing a computer program, wherein the above-mentioned file integrity verification method is executed by running the computer program on the device where the non-volatile storage medium is located.

[0103] The aforementioned non-volatile storage medium is used to store a program that performs the following functions: acquiring the structural parameters of a file tree and historical performance data of a verification system, wherein the file tree is generated from multiple data blocks that make up the file to be verified, and the verification system is used to perform integrity verification tasks on the file to be verified; determining the verification method based on the structural parameters and a file size threshold, wherein the file size threshold is determined based on historical performance data, and when the data volume of the file to be verified is equal to the file size threshold, the time consumed by performing integrity verification tasks using different verification methods is the same, including serial verification and parallel verification; performing integrity verification tasks on the file to be verified according to the verification method, wherein when the verification method is serial verification, the verification system uses one processor to perform the integrity verification task, and when the verification method is parallel verification, the verification system uses multiple processors to perform the integrity verification task; and determining the verification result output by the executed verification method.

[0104] This application also provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor is configured to execute the above-mentioned file integrity verification method through the computer program.

[0105] The processor in the aforementioned electronic device is used to run a program that performs the following functions: acquiring the structural parameters of the file tree and historical performance data of the verification system, wherein the file tree is generated based on multiple data blocks that make up the file to be verified, and the verification system is used to perform the integrity verification task of the file to be verified; determining the verification method based on the structural parameters and the file size threshold, wherein the file size threshold is determined based on historical performance data, and when the data volume of the file to be verified is the file size threshold, the time taken to perform the integrity verification task using different verification methods is the same, the verification methods include: serial verification and parallel verification; performing the integrity verification task of the file to be verified according to the verification method, wherein when the verification method is serial verification, the verification system uses one processor to perform the integrity verification task, and when the verification method is parallel verification, the verification system uses multiple processors to perform the integrity verification task; and determining the verification result output by the executed verification method.

[0106] This application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the above-described file integrity verification method.

[0107] It should be noted that each module in the above-mentioned file integrity verification device can be a program module (for example, a set of program instructions that implement a certain function) or a hardware module. For the latter, it can be manifested in the following forms, but is not limited to them: each of the above modules is manifested as a processor, or the functions of each of the above modules are implemented by a processor.

[0108] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0109] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0110] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0111] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0112] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0113] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to related technologies, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0114] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for verifying file integrity, characterized in that, include: The structural parameters of the file tree and the historical performance data of the verification system are obtained, wherein the file tree is generated based on multiple data blocks that make up the file to be verified, and the verification system is used to perform the integrity verification task of the file to be verified. Determining the verification method based on the structural parameters and file size threshold includes: comparing the file size threshold with the number of nodes recorded in the structural parameters to obtain a first comparison result, wherein the number of nodes indicates the total number of leaf nodes in the file tree; if the first comparison result indicates that the number of nodes is greater than or equal to the file size threshold, determining the verification method as parallel verification; if the first comparison result indicates that at least the number of nodes is less than the file size threshold, determining the verification method as serial verification; wherein the file size threshold is determined based on the historical performance data, and when the data volume of the file to be verified is the file size threshold, the time consumed to perform the integrity verification task using different verification methods is the same, and the verification methods include: serial verification and parallel verification; The integrity verification task is performed on the file to be verified according to the verification method. In the case that the verification method is serial verification, the verification system uses one processor to perform the integrity verification task. In the case that the verification method is parallel verification, the verification system uses multiple processors to perform the integrity verification task. Determine the verification result output by executing the verification method.

2. The method according to claim 1, characterized in that, The file size threshold is determined using the following method: The effective parallelism of the verification system is determined based on multiple performance scores recorded in the historical performance data, wherein each performance score is used to quantify the performance of a processor, and the performance score of each processor is determined based on the throughput generated by the processor performing file integrity verification. The effective parallelism is the number of processors performing the parallel verification. The baseline time of the verification system is determined based on the multiple parallel verification times, multiple serial verification times, and the effective parallelism recorded in the historical performance data. Each parallel verification time is the time taken by the verification system to perform the parallel verification in the past, and each serial verification time is the time taken by the verification system to perform the serial verification in the past. The baseline time is the systematic time overhead generated by performing the parallel verification, and the baseline time is not affected by the file to be verified. The single data block hash calculation time of the verification system is determined based on the verification duration of multiple subtrees and the number of multiple subtree nodes recorded in the historical performance data. Here, the verification duration of each subtree is the time consumed by the verification system to process each subtree, the number of subtree nodes is the number of nodes contained in each subtree, and the average hash calculation time of a single data block is the average time consumed to calculate the hash value of all data blocks during the execution of the file integrity verification task. The file size threshold is determined based on the effective parallelism, the baseline time, the hash calculation time of a single data block, and the data cache duration recorded in the historical performance data, wherein the data cache duration is the time it takes for the verification system to read the data block.

3. The method according to claim 2, characterized in that, The performance score for each processor is determined by the following method: When the verification system performs the file integrity verification task for the first time, the performance score of each processor is a preset score value; When the verification system is not performing the file integrity verification task for the first time, the performance score of each processor is an updated score to be updated. Each time the score to be updated is updated, the score result of the score to be updated is determined based on the throughput generated by the processor in the current execution of the integrity verification task and the score to be updated. When the performance score is updated for the first time, the score to be updated is the preset score value. When the performance score is not updated for the first time, the score to be updated is the update result of the previous score value.

4. The method according to claim 2, characterized in that, Before determining the verification method based on the structural parameters and file size threshold, the process includes: Determine the second comparison result between the effective parallelism and the preset number of processors; and, Among the multiple performance scores, determine the maximum performance score and the minimum performance score, determine the ratio of the maximum performance score to the minimum performance score, and determine a third comparison result of the ratio with a preset value; If the second comparison result indicates that the effective parallelism is less than or equal to the preset number of processors, or if the third comparison result indicates that the ratio is greater than the preset value, the verification method is determined to be the serial verification. If the second comparison result indicates that the effective parallelism is greater than or equal to the preset number of processors, and the third comparison result indicates that the ratio is less than or equal to the preset value, the verification method is determined based on the structural parameters and the file size threshold.

5. The method according to claim 1, characterized in that, Perform an integrity verification task on the file to be verified according to the verification method, including: When the verification method is serial verification, the file tree is assigned to a processor in the verification system, wherein the processor completes the integrity verification task of the file to be verified. When the verification method is parallel verification, the allocation scheme of the file tree is determined according to the historical performance data and the structural parameters. The file tree is split into multiple subtrees according to the allocation scheme, and each subtree is assigned to the corresponding processor. The multiple processors complete the integrity verification task of the file to be verified in parallel.

6. The method according to claim 5, characterized in that, Determining the file tree allocation scheme based on the historical performance data and the structural parameters includes: The first number of subtrees is determined based on the effective parallelism recorded in the historical performance data; and... For each processor, a second number of leaf nodes in the subtree allocated to that processor is determined based on the effective parallelism and the performance score corresponding to that processor, wherein the performance score is recorded in the historical performance data; and, The depth of the subtree allocated to each processor is determined based on the performance score, the lowest performance score, the preset subtree depth, and the preset fan-out degree corresponding to each processor, wherein the lowest performance score is the performance score with the smallest value among a plurality of performance scores; The subtrees allocated to each processor are determined in the file tree based on the second quantity and the depth of the subtrees allocated to each processor, and the allocation scheme is generated based on the subtrees allocated to each processor and the first quantity, wherein different subtrees contain different nodes.

7. The method according to claim 1, characterized in that, Determine the verification result output by executing the verification method, including: When the verification method is the serial verification, a first hash value output by the processor is obtained, wherein the first hash value is calculated by the processor on the file tree in the order from leaf node to root node; If the first hash value matches the target hash value pre-stored in the verification system, the verification result is determined to be a successful verification. If the first hash value is inconsistent with the target hash value, the verification result is determined to be a verification failure.

8. The method according to claim 7, characterized in that, Determining the verification result output by the verification method also includes: When the verification method is parallel verification, the subtree verification result output by each processor is obtained, and multiple subtree verification results are aggregated according to the level corresponding to the subtree verification result to obtain a second hash value. Each subtree verification result is the hash value of the root node of a subtree, and the second hash value is the hash value of the root node of the file tree. If the second hash value matches the target hash value, the verification result is determined to be a successful verification. If the second hash value is inconsistent with the target hash value, the verification result is determined to be a verification failure.

9. The method according to claim 1, characterized in that, After determining the verification result output by the verification method, the process also includes: If the verification result is that the verification passes, obtain the performance data generated by the verification system during the process of performing the integrity verification task on the file to be verified; A new performance score is determined based on each performance score recorded in the performance data and the historical performance data, and a new file size threshold is determined based on multiple new performance scores and the performance data. If the difference between the new file size threshold and the file size threshold is greater than a preset difference, the file size threshold will be updated to the new file size threshold. If the difference between the new file size threshold and the file size threshold is greater than a preset difference, the update of the file size threshold will stop.

10. The method according to claim 1, characterized in that, After determining the verification result output by the verification method, the method further includes: if the verification result is a verification failure, outputting verification failure information, wherein the verification failure information includes: the subtree where the hash value verification failed, and the failure to read the subtree.

11. A document integrity verification device, characterized in that, include: The acquisition module is used to acquire the structural parameters of the file tree and the historical performance data of the verification system. The file tree is generated based on multiple data blocks that make up the file to be verified, and the verification system is used to perform the integrity verification task of the file to be verified. A determination module is used to determine a verification method based on the structural parameters and a file size threshold, including: comparing the file size threshold with the number of nodes recorded in the structural parameters to obtain a first comparison result, wherein the number of nodes indicates the total number of leaf nodes in the file tree; if the first comparison result indicates that the number of nodes is greater than or equal to the file size threshold, determining that the verification method is parallel verification; if the first comparison result indicates that at least the number of nodes is less than the file size threshold, determining that the verification method is serial verification; wherein the file size threshold is determined based on the historical performance data, and when the data volume of the file to be verified is the file size threshold, the time consumed to perform the integrity verification task using different verification methods is the same, and the verification methods include: serial verification and parallel verification; An execution module is configured to perform an integrity verification task on the file to be verified according to the verification method, wherein, when the verification method is serial verification, the verification system uses one processor to perform the integrity verification task, and when the verification method is parallel verification, the verification system uses multiple processors to perform the integrity verification task. The output module is used to determine the verification result output by performing the verification method.

12. A non-volatile storage medium, characterized in that, The non-volatile storage medium stores a computer program, wherein the device containing the non-volatile storage medium executes the file integrity verification method according to any one of claims 1 to 10 by running the computer program.

13. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the file integrity verification method according to any one of claims 1 to 10 through the computer program.

14. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the file integrity verification method according to any one of claims 1 to 10.

Images