A method and system for virtual disk space management based on block reconstruction

Abstract

The present application belongs to the technical field of space management, and discloses a virtual disk space management method and system based on block reconstruction, which comprises the following steps: acquiring real-time block state data of a qcow2 image in a virtual machine running environment, extracting candidate blocks and constructing a bidirectional state comparison matrix according to the real-time block state data; calculating the hollow confidence score of each candidate block according to the bidirectional state comparison matrix and in combination with the real-time block state data; performing hierarchical sorting and continuous block address segment screening according to the hollow confidence score to obtain a to-be-reconstructed hollow segment; dividing the qcow2 image into multiple reconstruction units, applying a micro-granularity read-write lock to the reconstruction unit containing the to-be-reconstructed hollow segment, performing a hollow punching operation after acquiring the read-write lock, and synchronously updating L2 mapping table entries; and monitoring the I / O request delay change rate of the virtual machine in real time, and dynamically adjusting the execution rate of the block reconstruction operation based thereon; the present application improves the virtual machine storage resource utilization rate and service stability.

Description

Technical Field

[0001] This invention relates to the field of space management technology, and more specifically, to a method and system for virtual disk space management based on block reconstruction. Background Technology

[0002] Virtual disk images, as a core storage abstraction and management component in modern cloud computing platforms, virtualized data centers, containerized deployments, and desktop cloud systems, are widely used in various scenarios requiring flexible storage configuration and data security due to their elastic resource allocation, multi-tenant isolation, and snapshot backup capabilities. With the rapid evolution of cloud-native architectures, the requirements for virtual disk storage efficiency and space utilization are constantly increasing, especially in public cloud platforms, virtual desktop infrastructure (VDI), software-defined storage (SDS), and container persistent storage. Storage cost optimization and fine-grained space management have become core competitive indicators in the industry.

[0003] However, in practical applications of qcow2 images, the "only increase, no decrease" space expansion phenomenon caused by the host physical layer's inability to synchronously reclaim space after the guest file system releases it is a key bottleneck restricting storage resource utilization. Existing technologies generally use offline tools or periodic full scans, ignoring the three-dimensional dynamic differences between the guest logical release and the host physical allocation state at the virtualization layer, file system layer, and block device layer. This leads to a dilemma in space reclamation strategies between business continuity requirements and data security assurance. Especially in high-availability scenarios such as public cloud platforms, online transaction systems, and critical business databases, when virtual machines frequently perform file creation and deletion operations, offline reclamation tools require downtime maintenance windows, severely impacting business continuity; while the online TRIM mechanism only marks release at the guest level and cannot penetrate the virtualization layer to reach the host file system, causing the disk image to continuously expand and resulting in a high degree of space waste.

[0004] Traditional fstrim commands and periodic image compression methods cannot establish a dynamic mapping between client logical release and host physical occupancy. Under conditions of high-frequency data changes and complex snapshot chains, the accuracy of space reclamation is severely insufficient, and the risk of erroneously reclamating active data is high. Furthermore, due to the lack of a consistency verification model between client TRIM intent and host physical block state, the system cannot distinguish between the intermediate states of "logically released but not physically reclaimed" and "logically released and physically reclaimed." This leads to significant deviations in storage space calculations and metadata inconsistencies in dynamic scenarios such as virtual machine migration and snapshot recovery.

[0005] Furthermore, existing technologies fail to comprehensively consider the coupling effect between snapshot dependencies and space reclamation. Especially in multi-layered snapshot chain environments, the data corruption risk caused by the asynchrony between physical cluster reference counting and logical release marking makes reclamation operations difficult to implement. A single-strategy brute-force reclamation cannot simultaneously address the different security requirements of independent blocks without snapshot dependencies and shared blocks, making it difficult for the system to balance storage efficiency and data integrity. Moreover, when performing space reclamation operations during virtual machine service operation, existing methods lack real-time perception and dynamic adjustment mechanisms for the impact on I / O performance. Full-speed reclamation leads to I / O bandwidth contention and latency jitter, severely impacting normal virtual machine operations and causing excessive performance degradation. Meanwhile, an overly conservative reclamation rate results in low space release efficiency, failing to meet the timeliness requirements of scenarios with scarce storage resources.

[0006] In view of this, the present invention proposes a virtual disk space management method and system based on block reconstruction to solve the above problems. Summary of the Invention

[0007] To overcome the aforementioned deficiencies of the prior art and to achieve the above objectives, the present invention provides the following technical solution:

[0008] A method for managing virtual disk space based on block reconstruction, comprising:

[0009] Obtain real-time block status data of the qcow2 image in the virtual machine runtime environment. The real-time block status data includes L1 / L2 mapping table entries, trim instruction sequence, block-level dirty data bitmap, and physical block allocation record.

[0010] A host physical mapping set is constructed based on the L1 / L2 mapping table entries and physical block allocation records; and a client logic release set is constructed synchronously based on the trim / discard instruction sequence.

[0011] The host physical mapping set and the client logic release set are cross-compared, and candidate blocks are extracted based on the comparison results, and a bidirectional state comparison matrix is ​​constructed.

[0012] Based on the bidirectional state comparison matrix and combined with the block-level dirty data bitmap, the hole confidence score of each candidate block is calculated.

[0013] Candidate blocks are sorted and ranked according to the hole confidence score, and continuous block address segments with confidence scores greater than a preset security threshold are selected as hole segments to be reconstructed.

[0014] The qcow2 image is divided into multiple reconstruction units, and a fine-grained read-write lock is applied for the reconstruction unit containing the hole segment to be reconstructed. After the read-write lock is acquired, the hole punching operation is performed, and the L2 mapping table entries are updated synchronously.

[0015] During the hole punching operation, the I / O request latency change rate of the virtual machine is monitored in real time, and the execution rate of the block reconstruction operation is dynamically adjusted according to the I / O request latency change rate.

[0016] Furthermore, the process of constructing the host physical mapping set includes:

[0017] Read the header metadata area in the image file corresponding to the qcow2 image to obtain the starting offset address, total number of entries, and cluster size parameter of the L1 mapping table entry;

[0018] Traverse each L1 entry in the L1 mapping table based on the starting offset address, and extract the physical offset address pointing to the L2 mapping table entry for each non-empty L1 entry.

[0019] Perform an entry-by-entry scan of each allocated L2 mapping entry, and read the physical cluster offset address and reference count field in each L2 entry;

[0020] Based on the physical block allocation record, verify the allocation status of the physical cluster offset address in each L2 table entry in the host file system, and filter out residual mapping entries that have been released by the physical layer but have not been updated by the L2 table entry.

[0021] The verified physical cluster offset address is associated with the corresponding virtual block address to construct the host physical mapping set. Each entry in the host physical mapping set contains a virtual block address, a physical cluster offset address, and a cluster allocation status identifier.

[0022] Furthermore, the process of constructing the customer logic release set includes:

[0023] Register hook functions for intercepting TRIM commands; capture all block device requests sent by the virtual machine guest through the registered hook functions, parse the command type field of each request, and identify space release requests that conform to trim or discard semantics;

[0024] Extract the starting address of the target logical block, the number of blocks to be released, and the timestamp of the request from the identified space release request, encapsulate them as a trim event record and append them to the circular buffer queue;

[0025] The trim event records in the circular buffer queue are sorted by logical block address, and a range merging operation is performed on multiple trim event records with consecutive addresses or overlapping ranges.

[0026] The client logic release set is constructed based on the merged trim event records. Each entry in the client logic release set contains a logic block address range, a cumulative number of releases, and a recent release timestamp.

[0027] Furthermore, the construction process of the bidirectional state alignment matrix includes:

[0028] Using the virtual block address space of the qcow2 image as the row index range, a two-dimensional state matrix structure is created, where each row of the two-dimensional state matrix corresponds to a virtual block address;

[0029] Traverse each entry in the host physical mapping set, locate the row corresponding to the virtual block address of the corresponding entry in the two-dimensional state matrix, and set the physical allocation status flag of the row to the allocated status.

[0030] Traverse each entry in the client logic release set, locate multiple rows corresponding to the logical block address range of the corresponding entry in the two-dimensional state matrix, and set the logical release status flag of the corresponding row to the released state.

[0031] Read the block-level dirty data bitmap and mark the block addresses set in the block-level dirty data bitmap with additional active write status flag bits in the two-dimensional state matrix;

[0032] Scan each row of the two-dimensional state matrix, filter out rows where the physical allocation status flag is allocated, the logical release status flag is released, and the active write status flag is inactive, take the virtual block address in the corresponding row as candidate block, and extract it as a set of candidate empty block addresses, which is recorded as the output result of the bidirectional state comparison matrix.

[0033] Furthermore, the process of calculating the hole confidence score includes:

[0034] Read the last modified timestamp of the candidate block from the block-level dirty data bitmap, obtain the difference between it and the current timestamp, and construct a time decay factor based on it;

[0035] The access frequency value of the candidate block is read from the access frequency counter array pre-deployed in the virtual machine, the ratio of the access frequency value to the global average access frequency within a preset time window is calculated, and a cold data confidence factor is constructed based on it.

[0036] The number of trim instructions covering the current candidate block is counted from the trim instruction sequence, and the ratio of the cumulative number of sectors covered by trim instructions in the candidate block to the total number of sectors in the candidate block is calculated based on the ratio result. A release deterministic factor is then constructed based on the ratio result.

[0037] Read the L2 table entry corresponding to the candidate block, and check whether the L2 table entry has a snapshot dependency mark or a copy-on-write mark. If a dependency mark exists, construct the metadata security factor with a reduced weight value; if no dependency mark exists, construct the metadata security factor with a normal weight value.

[0038] The time decay factor, release determinism factor, cold data confidence factor, and metadata security factor are weighted and fused to obtain the hole confidence score of the corresponding candidate block.

[0039] Furthermore, the process of obtaining the void segment to be reconstructed includes:

[0040] The candidate empty block address set is sorted in ascending order by virtual block address, and candidate blocks with consecutive addresses are merged into candidate block segments. The starting address and number of blocks in each candidate block segment are recorded.

[0041] Obtain the minimum and mean values ​​of the hole confidence scores of all candidate blocks within each candidate block segment, and use the weighted combination of the minimum and mean values ​​as the segment-level confidence score of that candidate block segment.

[0042] Based on the comparison between the segment-level confidence score and the preset multi-level security threshold, the candidate block segments are divided into three categories: high confidence level, medium confidence level, and low confidence level.

[0043] Read the snapshot chain metadata of the qcow2 image, traverse the L2 mapping table entry of each snapshot node in the snapshot chain metadata, check whether the physical cluster address of the candidate block segment is referenced by other snapshot nodes, and set the snapshot lock flag for the candidate block segment whose physical cluster address is referenced by other snapshot nodes.

[0044] Candidate blocks with high confidence levels and no snapshot lock flag are identified as void segments to be reconstructed; candidate blocks with medium confidence levels are added to the observation queue, and a decision is made after recalculating the void confidence score in the next evaluation cycle.

[0045] Furthermore, the process of performing the hole drilling operation includes:

[0046] The virtual address space of the qcow2 image is evenly divided according to the preset chunk size to obtain several non-overlapping reconstruction units. A unique chunk identifier is assigned to each reconstruction unit, and a mapping table from chunk identifier to virtual address range is established.

[0047] Traverse all the empty segments to be reconstructed, map the start cluster address and end cluster address of each empty segment to the corresponding chunk identifier, and build a chunk queue to be processed;

[0048] A chunk-level lock manager is established for the qcow2 image. The lock manager maintains the lock state of the read and write locks allocated to each reconstruction unit. The lock state includes three states: idle, read locked, and write locked.

[0049] Chunk identifiers are retrieved sequentially from the chunk queue to be processed. A write lock for the corresponding chunk is requested from the lock manager. After successfully acquiring the chunk write lock, the corresponding virtual address range is calculated based on the chunk identifier. All cluster addresses of the hole segment to be reconstructed within the corresponding range are traversed. The cluster addresses are converted into file offsets of the corresponding image files of the qcow2 image. The file offsets corresponding to all cluster addresses within the reconstruction unit are collected and organized, and continuous offset intervals are merged to form several continuous file offset ranges.

[0050] A fallocate system call is made to the file offset range, and a hole punching operation is performed by specifying the FALLOC_FL_PUNCH_HOLE and FALLOC_FL_KEEP_SIZE flags, so that the physical storage space corresponding to the file offset range is returned to the host file system.

[0051] Furthermore, the process of updating L2 mapping table entries includes:

[0052] Before performing the hole punching operation, the L2 mapping table entry to be modified is read from the corresponding image file of the qcow2 image and saved to the memory buffer, and a working copy of the L2 mapping table entry is created.

[0053] Locate the L2 entry corresponding to the void segment to be reconstructed in the working copy, reset the physical cluster offset address field of the corresponding L2 entry to zero, and modify the allocation status bit of the L2 entry to the unallocated status.

[0054] Calculate the CRC check value for the modified working copy, apply for a new free cluster area in the qcow2 image, write the modified working copy and the CRC check value into the free cluster area, and record the writing position as the offset address of the new L2 mapping table.

[0055] Read the L1 table entry that points to the original L2 table entry from the L1 mapping table, replace the pointer value of the L1 table entry with the offset address of the new L2 mapping table through an atomic write operation, persist the update of the L1 mapping table to the storage medium, and simultaneously add the physical cluster offset address occupied by the original L2 table entry to the free cluster pool with a reference count of zero.

[0056] Furthermore, the process of adjusting the execution rate of block refactoring operations includes:

[0057] Before and after each hole punching operation, the I / O performance metrics of the virtual machine are collected. The I / O performance metrics include the average response time of read requests, the average response time of write requests, and the I / O request queue depth.

[0058] Using the I / O performance metrics collected under virtual machine idle running conditions as the benchmark value, the ratio of the collected I / O performance metrics to the benchmark value is calculated to obtain the I / O request latency change rate.

[0059] When the rate of change of I / O request latency is greater than the preset first warning threshold but not greater than the preset second warning threshold, the execution interval of the reconstruction operation will be increased to a preset multiple of the current execution interval.

[0060] When the rate of change of I / O request latency exceeds the preset second warning threshold, all reconstruction operations are immediately suspended and a cooling timer is started. After the cooling timer expires, the reconstruction operation is resumed at the lowest execution rate.

[0061] When the rate of change of I / O request latency is less than the preset safety recovery threshold for multiple consecutive sampling periods, the execution interval of the reconstruction operation is gradually shortened until it is restored to the default execution rate.

[0062] A block-based virtual disk space management system, comprising:

[0063] The spatial state collection module is used to acquire real-time block state data of the qcow2 image in the virtual machine runtime environment, construct host physical mapping set and client logic release set based on the real-time block state data, cross-compare them, extract candidate blocks based on the comparison results, and construct a bidirectional state comparison matrix.

[0064] The dynamic hole identification module is used to calculate the hole confidence score of each candidate block based on the bidirectional state comparison matrix and the real-time block state data; and to sort the candidate blocks by level and filter the continuous block address segments based on the hole confidence score to obtain the hole segments to be reconstructed.

[0065] The online reconstruction module is used to divide the qcow2 image into multiple reconstruction units, apply for a fine-grained read-write lock for the reconstruction unit containing the hole segment to be reconstructed, perform hole punching operation after acquiring the read-write lock, and update the L2 mapping table entries synchronously.

[0066] The monitoring and feedback module is used to monitor the I / O request latency change rate of the virtual machine in real time during the hole punching operation, and dynamically adjust the execution rate of the block reconstruction operation based on the I / O request latency change rate.

[0067] The technical effects and advantages of the virtual disk space management method and system based on block reconstruction of the present invention are as follows:

[0068] This invention maintains efficient and secure storage space management performance throughout the entire lifecycle of a virtual machine, significantly improving the space utilization and data security of the qcow2 virtual disk. Through the synergistic effect of a three-dimensional state fusion mechanism and a four-factor confidence assessment model, this invention substantially improves the accuracy of void block identification and the coverage of space reclamation, significantly enhancing efficiency compared to traditional offline tools. Simultaneously, it reduces the risk of data corruption due to erroneous reclamation to an extremely low level, achieving an optimal balance between reclamation efficiency and data security. Attached Figure Description

[0069] Figure 1 This is a schematic diagram of a virtual disk space management method based on block reconstruction according to the present invention;

[0070] Figure 2 This is a schematic diagram of a block-based virtual disk space management system according to the present invention. Detailed Implementation

[0071] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0072] Example 1, please refer to Figure 1 As shown in this embodiment, a virtual disk space management method based on block reconstruction includes:

[0073] First, a dedicated virtualization layer status monitoring network is used to acquire real-time block status data sets of the qcow2 image in the virtual machine runtime environment. This block status data set covers multi-level status information from virtualization layer metadata to the host file system physical layer. The monitoring network employs a multi-source data synchronous acquisition mechanism, continuously collecting four types of key status data throughout the virtual machine runtime cycle: L1 / L2 mapping entries of the qcow2 image, TRIM instruction sequences issued by the virtual machine guest, block-level dirty data bitmaps, and physical block allocation records of the host file system. The L1 / L2 mapping entries reflect the mapping relationship between virtual addresses and physical clusters; the TRIM instruction sequences record space release operations of the guest file system; the dirty data bitmap identifies the modification status of data blocks; and the physical block allocation records characterize the actual storage space occupancy of the host file system. These four types of data together form a status dataset with a three-dimensional perspective encompassing the virtual layer, logical layer, and physical layer. qcow2 refers to the copy-on-write (COW) disk image format commonly used in QEMU virtualization technology, supporting snapshots and dynamic expansion.

[0074] Based on the mapping relationship between virtual blocks and physical storage in the L1 / L2 mapping table entries and the physical block allocation record, a set of physical block mapping relationships reflecting the perspective of the host file system is constructed, denoted as the host physical mapping set. The host physical mapping set characterizes the actual physical storage space distribution of virtual disks within the host file system. By traversing the hierarchical structure of the L1 / L2 mapping table, the physical cluster offset address corresponding to each allocated virtual block is extracted and cross-validated with the block allocation bitmap of the host file system, filtering out residual invalid mapping entries to form an accurate representation of the true allocation state at the physical layer. Simultaneously, based on the logical block address range recorded in the TRIM instruction sequence and... Release timestamp information to construct a set of logical block release relationships reflecting the perspective of the guest file system, denoted as the guest logical release set. The guest logical release set represents the distribution status of space that the virtual machine's internal file system considers to be released but has not yet been reclaimed at the physical layer. By registering the TRIM instruction interception hook function at the virtualization layer, all DISCARD / TRIM semantic requests sent by the guest are captured, and continuous or overlapping release intervals are merged and deduplicated to form a complete record of the logical layer's release intention. Among them, DISCARD / TRIM refers to the mechanism of the virtual machine's internal file system marking "unused blocks" to notify the storage backend that space can be reclaimed.

[0075] Based on the state differences between the host physical mapping set and the client logical release set, a dual-perspective cross-comparison analysis is performed to extract the range of block addresses marked as allocated in the host physical mapping set but released in the client logical release set, and to construct a bidirectional state comparison matrix. The bidirectional state comparison matrix is ​​a spatial representation of the contradictions in the virtual disk block state. It uses the virtual block address as the index dimension and includes three state dimensions: physical allocation status flag, logical release status flag, and active write status flag. In the construction process, a two-dimensional matrix structure covering the entire virtual address space is first created. The host physical mapping set and the client logical release set are traversed sequentially, and the physical allocation and logical release status flags are filled into the corresponding rows of the matrix. Combined with the block-level dirty data bitmap, data blocks with recent active writes are marked. Finally, a set of candidate empty block addresses that are physically allocated, logically released, and have no active writes are selected. These candidate blocks have the potential to be safely reclaimed.

[0076] By analyzing the correlation between the temporal characteristics, access characteristics, and metadata characteristics of candidate hollow blocks in the bidirectional state comparison matrix, a hollow confidence score is calculated for each candidate block. The hollow confidence score quantifies the likelihood of a data block being safely reclaimed, comprehensively reflecting the characteristics of four dimensions: block cooldown duration, release certainty, access activity, and metadata security. During the calculation, the last modification timestamp of the candidate block is extracted from the block-level dirty data bitmap, and the difference between the timestamp and the current time is calculated as the block cooldown duration, constructing a time decay factor to reflect the temporal stability of the data block. The proportion of sectors covered by TRIM instructions in the candidate block is statistically analyzed from the TRIM event queue, constructing a release certainty factor to reflect the clarity of the client's release intention. The access frequency value of the candidate block is read from the access frequency counter, and its ratio with the global average access frequency is calculated, constructing a cold data confidence factor to reflect the access popularity of the data block. The snapshot dependency flag of the L2 table entry corresponding to the candidate block is checked from the metadata mapping index, constructing a metadata security factor to reflect the metadata risk of the reclamation operation. The four factors are fused through a weighted product. When any factor is a penalty value, the hole confidence score is significantly reduced, ensuring the safety of the reclamation operation. Among them, the access frequency counter is pre-deployed on the I / O path of the virtual machine block device to count the number of times the data block is read and written within a specific time window.

[0077] Based on the distribution characteristics of the hole confidence scores and the comparison with the preset multi-level security thresholds, candidate hole blocks are sorted and filtered in a hierarchical manner to determine the set of hole segments to be reconstructed. Hollow segments to be reconstructed refer to continuous candidate block intervals with high confidence and no snapshot dependency, which can safely perform physical space reclamation operations. During the filtering process, candidate blocks are first arranged in ascending order of virtual address and block segments with consecutive addresses are merged. The segment-level confidence score of each block segment is calculated. Based on the comparison results of the segment-level confidence and the security threshold, the blocks are divided into three levels: high confidence, medium confidence, and low confidence. The snapshot chain metadata of the qcow2 image is further traversed to check whether the physical cluster address of the block segment is referenced by other snapshot nodes. Locking flags are set for blocks with snapshot dependencies. Finally, blocks with high confidence and no snapshot locks are identified as hole segments to be reconstructed, and blocks with medium confidence are added to the observation queue for re-evaluation in subsequent cycles. This hierarchical processing strategy balances reclamation efficiency and security.

[0078] The virtual address space of the qcow2 image is evenly divided according to a preset chunk size to form multiple reconstruction units. Fine-grained locking and hole punching operations are implemented on reconstruction units containing empty segments to be reconstructed. The reconstruction unit is the basic processing granularity for virtual disk space reconstruction, and its size is set to an integer multiple of the qcow2 cluster size, enabling data reading and metadata updates to be completed within a single I / O operation cycle. During execution, a chunk-level lock manager maintains the lock state of each reconstruction unit. Chunk identifiers of reconstruction units are sequentially retrieved from the pending chunk queue, and the corresponding fine-grained read / write lock is requested from the lock manager. Upon detecting no lock conflict, a write lock state is set and the lock holder information is recorded. After successfully acquiring the lock, the empty segments to be reclaimed within the reconstruction unit are... The cluster address is converted to the file offset of the corresponding image file of the qcow2 image. The host operating system makes a fallocate system call, sets the FALLOC_FL_PUNCH_HOLE and FALLOC_FL_KEEP_SIZE flags, and performs a hole punching operation to release the corresponding physical storage space. Simultaneously, the L2 mapping table is atomically updated through a copy-on-write mechanism: the L2 mapping table to be modified is read into memory to create a working copy. In the copy, the physical cluster offset address of the reclaimed block is reset to zero and the allocation status bit is modified. The modified copy is written to the newly allocated free cluster area. The L1 table entry pointer is updated to point to the new L2 mapping table position through atomic operations, and metadata is continuously updated to ensure the atomicity and consistency of space reclamation and metadata updates.

[0079] During the hole punching and metadata update operations, the I / O request latency change rate of the virtual machine is collected in real time through the virtualization layer performance monitoring interface. The execution rate of the block reconstruction operation is dynamically adjusted based on the comparison between the latency change rate and a preset latency threshold. The I / O request latency change rate reflects the impact of the reconstruction operation on the virtual machine's service performance. Performance indicators such as the average response time of read and write requests and the I / O queue depth are collected, and the change rate of the current performance indicators is calculated with the baseline value of the virtual machine's idle state as a reference. The adjustment strategy adopts a three-level response mechanism: when the latency change rate is greater than the preset first warning threshold but not greater than the preset second warning threshold, [the following will occur]. The execution interval of the refactoring operation is increased by a preset multiple to reduce the impact on business operations; when the latency change rate exceeds the preset second warning threshold, all refactoring operations are immediately suspended and a cooling timer is started, and the operation is resumed at the lowest rate after the timeout; when the latency change rate is less than the preset safe recovery threshold for multiple consecutive sampling periods, the execution interval is gradually shortened until the default rate is restored; at the same time, by maintaining the historical sequence of the latency change rate through a sliding window, the business load status of subsequent periods is predicted through time-series trend analysis, and the refactoring rate is proactively reduced before the predicted business peak period, thus realizing an adaptive performance guarantee mechanism to ensure that the space reclamation operation does not affect the normal business operation of the virtual machine.

[0080] In embodiments of the present invention, the detailed implementation steps for constructing the host physical mapping set include:

[0081] The header metadata region of the qcow2 image file is read, the hierarchical structure information of the L1 mapping table is parsed, and the physical location index of the L2 mapping table is extracted. Parsing the header metadata region effectively obtains the logical structure layout and mapping hierarchy information of the image file. From the header metadata region, structural descriptors such as the starting offset address of L1 mapping table entries, the total number of entries, cluster size parameters, and cluster bitmasks are extracted to form an L1 hierarchical structure information vector. This information effectively describes the address space partitioning pattern of the virtual disk and reflects the storage organization method of the qcow2 image. In specific implementation, the starting address of the L1 table is first read based on the header metadata region of the qcow2 image file, and the virtual disk size and cluster size are read simultaneously. These parameters allow the acquisition of the number of L1 table entries. The calculation of the number of L1 table entries is based on the ratio between the virtual disk size and the address space range covered by each L1 table entry; this ratio reflects the address space segmentation granularity of the image.

[0082] The process involves traversing each entry in the L1 mapping table, parsing the L2 mapping table pointer, and constructing a physical cluster allocation state diagram. The L2 mapping table pointer is a key field in the L1 table that points to the next level of mapping structure. By reading each entry sequentially, a segmented mapping relationship of the virtual address space can be established. Within the entire L1 mapping table, the 8-byte data content of each L1 table entry is read sequentially. For non-empty L1 table entries, the L2 mapping table physical offset address field is extracted. This physical offset address field occupies the lower 62 bits of the table entry. The effective address bits are extracted through bitmasking operations to form the file offset pointing to the L2 mapping table. The physical cluster allocation state diagram is represented by a two-dimensional array. The horizontal axis corresponds to the L1 table entry index, and the vertical axis corresponds to the virtual address segment governed by the L1 table entry. The array elements store the location information of the L2 mapping table. The physical cluster allocation state diagram can be used to visually display the address segments allocated to the L2 mapping table and the uninitialized address segments in the virtual disk.

[0083] Each valid L2 mapping table is scanned entry by entry, and cross-validation is performed using the physical block allocation records of the host file system to identify residual mapping entries and valid mapping regions. Cross-validation is used to discover and filter abnormal entries where the L2 table entries are inconsistent with the actual physical allocation status. The complete L2 table content is read based on the L2 mapping table location information. The L2 table contains several 8-byte entries, each recording the physical location of a data cluster. The physical cluster offset address field and flag field are extracted from each L2 table entry. The physical cluster offset address indicates the storage location of the data cluster in the image file, and the flag field indicates the allocation status, compression status, and copy-on-write status of the cluster. The extracted physical cluster offset address is converted into a block address of the host file system, and the actual allocation status of the corresponding block address is queried. If an L2 table entry is marked as allocated but the host file system shows that the block is not allocated, it is determined to be a residual mapping entry. Residual mapping entries are usually caused by abnormal interruption or incomplete metadata updates. Regions with high physical verification failure rates usually correspond to metadata inconsistencies and require mapping repair. The host file system refers to the underlying physical file system on the running host machine that directly stores the qcow2 image.

[0084] The verified L2 table entries, physical block allocation records, reference count metadata, and cluster allocation status identifiers are correlated in multiple dimensions to generate a host physical mapping set. This multi-dimensional correlation employs structured data organization and indexing methods to integrate mapping information from different sources into a unified mapping representation. The reference count metadata records the number of times each physical cluster is referenced, used to determine whether the cluster is shared by snapshots. Cluster allocation status identifiers include status types such as allocated, unallocated, compressed storage, and zero cluster. During the correlation process, mapping entries are established using the virtual block address as the primary key. Each mapping entry contains the virtual block address, physical cluster offset address, reference count value, and allocation status identifier. The system includes fields such as verification timestamps; it assigns priorities based on the reliability of various information types, with physical block allocation records having the highest priority. When L2 entries conflict with physical records, the physical record takes precedence. To improve query efficiency, the system constructs two index structures for the host physical mapping set: hash indexes and B-tree indexes. Hash indexes support fast and accurate virtual address lookups, while B-tree indexes support efficient address range scanning. The final generated host physical mapping set is a multi-index relational data structure, with each dimension corresponding to a mapping attribute field. This structure comprehensively represents the physical storage occupancy status of the virtual disk within the host file system, providing an accurate physical allocation information basis for subsequent space reclamation operations.

[0085] In this embodiment of the invention, the detailed implementation steps for constructing the client logic release set include:

[0086] A TRIM instruction interception mechanism is registered in the block device driver unit of the virtual machine, and request listening hook functions are deployed to establish a capture channel for guest space release behavior. Hook function registration is the fundamental means of obtaining the release intent of the guest file system and can effectively capture all block device space management requests issued by the virtual machine. During the block device driver initialization phase of the virtualization software, pre-interception hooks and post-processing hooks are registered to the critical path of the request processing queue. The pre-interception hook is triggered before the request enters the processing flow, and the post-processing hook is triggered after the request is completed. The triggering condition of the hook function is set to activate when the virtual block device receives any I / O request. When triggered, the complete context information of the I / O request is passed to the hook function, including the request type identifier, target logic, etc. Parameters such as block address, number of sectors operated on, and request priority effectively describe the client's operational intent on the virtual disk and reflect the client's file system space management behavior. In specific implementation, callback function pointers are registered at the block device layer of QEMU (Virtual Machine Monitor), and the addresses of custom TRIM interception processing functions are registered to the hook list of the block device request dispatcher. The hook list adopts a priority queue structure, and the TRIM interception hook is set to high priority to ensure execution before other processing logic. After registration, a dedicated TRIM event circular buffer queue is created. The queue capacity is dynamically adjusted according to the I / O load characteristics of the virtual machine. When the queue is close to full, the persistence mechanism is automatically triggered to write the historical records to the disk.

[0087] Command type identification and semantic parsing are performed on virtual block device requests captured by hook functions to filter out TRIM and DISCARD requests that conform to space release semantics. Command type identification accurately determines the semantic type of the operation by parsing the command field of the request. Inside the hook function, the command type field of the request data structure is read first. This field uses standard block device command encoding, which is a special instruction sent by the client in the virtualization environment to send a space release request. For the identified space release request, the parameter segment of the request is further parsed to extract the target logical block start address (LBA), the number of consecutive sectors to be released, and the attribute flags of the release operation. These are then encapsulated as a TRIM event record and appended to the circular buffer queue. The target logical block start address refers to the starting point of the range of virtual disk addresses that the client file system expects to release. The number of consecutive sectors to be released defines the length of the consecutive release interval starting from the start address. The attribute flags include semantic information such as whether to execute immediately and whether to guarantee data erasure. During the parsing process, the timestamp of the request arrival is recorded synchronously for subsequent analysis of the temporal characteristics of the space release behavior. The semantic parsing results intuitively show the virtual disk areas that are no longer needed in the client file system and the virtual disk areas that can be safely reclaimed.

[0088] The TRIM event records in the circular buffer queue are sorted in ascending order by the starting address of the logical block. For multiple TRIM event records with consecutive addresses or overlapping intervals, an interval merging operation is performed to eliminate redundant duplicate release records. The interval merging operation integrates fragmented release requests to form a compact logical release region representation. TRIM event records to be processed are periodically read in batches from the circular buffer queue. The read operation uses an atomic snapshot mechanism to ensure that new TRIM event records are not lost during the read process. The read TRIM event records are first sorted according to the starting address of the logical block using merge sort or quicksort. In the sorted record sequence, adjacent or overlapping release intervals can be identified by linear scan, and interval merging is performed using a scanline algorithm: the current merge interval is initialized to the address range of the first record, and the sorted sequence is traversed starting from the second record; for each record, its starting address is calculated to be no greater than the ending address of the current merge interval plus 1. (Determine if the intervals are continuous or overlapping). If they are continuous or overlapping, update the end address of the current merge interval to the maximum of the two end addresses, indicating that the interval merge is successful. If they are not continuous, output the current merge interval to the result set and initialize the next merge interval with the new record. For completely overlapping records, that is, the address range of the new record is completely contained within the current merge interval, they are identified as redundant duplicate release records. These records are usually generated by periodic TRIM tasks or batch optimization operations of the client file system. During the merging process, the number of times each merge interval is covered by the original TRIM request is synchronously counted. This number reflects the intensity of duplicate release of the address range by the client.

[0089] A client logical release set is constructed based on the merged TRIM event records. Each entry in the client logical release set contains three core attributes: logical block address range, cumulative release count, and most recent release timestamp. The client logical release set is a persistent representation of the client file system space release status, organized using an indexed data structure to support efficient address range queries and status update operations. In the specific implementation, a relational data table is created for the release set. Each entry in the table corresponds to a continuous logical release range. The fields of the entry are defined as follows: the logical block address range field contains two subfields: the starting logical block address and the ending logical block address, representing the release address range in the form of a closed interval; the cumulative release count field records the cumulative number of times the address range has been covered by TRIM commands within the observation period. The initial value is the coverage count count counted in the merge operation, and the count is incremented if the range is TRIMed again subsequently; the most recent release timestamp field records the time of the last TRIM operation within the range, used to evaluate the timeliness of the release status and calculate the time decay factor in the hole confidence score; an interval tree index is constructed for the logical block address range field to improve query efficiency. The final client logical release set contains thousands to tens of thousands of release records, which can comprehensively reflect the virtual disk address space that the client file system considers to have been released but has not yet been reclaimed by the host physical layer. It provides accurate logical release status data for subsequent cross-comparison with the host physical mapping set and is a key data source for identifying reclaimable void blocks.

[0090] In this embodiment of the invention, the detailed implementation steps for constructing the bidirectional state alignment matrix include:

[0091] Using the virtual block address space of the qcow2 image as the row index range, a two-dimensional state matrix structure is created, and a mapping table is established from virtual block addresses to multi-dimensional state attributes. The two-dimensional state matrix is ​​the core data structure for realizing dual-view state fusion. It can allocate an independent state storage unit for each virtual block and support parallel marking and fast querying of multi-dimensional state information. The matrix creation process first reads the virtual disk size and cluster size parameters from the header metadata area of ​​the qcow2 image, and calculates the total number of virtual blocks based on these parameters (calculated by dividing the virtual disk size by the cluster size and rounding up). The number of rows in the matrix is ​​set to the total number of virtual blocks, with each row corresponding to a unique virtual block address. The number of columns in the matrix is ​​set to the number of dimensions of the status attributes, including three core columns: the first column is the physical allocation status flag, the second column is the logical release status flag, and the third column is the active write status flag. The three flags in the core columns are compressed into different bits of a single byte; for example, bit 0 represents the physical allocation status (0 for unallocated, 1 for allocated), bit 1 represents the logical release status (0 for not released, 1 for released), and bit 2 represents the active write status (0 for inactive, 1 for active). This matrix representation unifies the scattered mapping relationships and release records into the same address space, providing an efficient data access foundation for subsequent status comparison and conflict identification.

[0092] The process iterates through each entry in the host physical mapping set, locates the corresponding virtual block address row in the two-dimensional state matrix, and sets the physical allocation status flag of that row to the allocated state. Physical allocation status filling is the first stage of synchronizing physical occupancy information from the host file system's perspective to the state matrix. This process sequentially reads each mapping entry from the host physical mapping set's data structure. Each mapping entry contains fields such as virtual block address, physical cluster offset address, reference count, and allocation status flag. For each read mapping entry, the virtual block address field is extracted first, directly corresponding to the row index of the two-dimensional state matrix. The memory offset of the matrix row is calculated using the virtual block address, calculated as the base address (starting storage location) plus the virtual block address multiplied by the number of bytes per row, locating the storage location of the target row. The status byte of the target row is read, and a bitwise OR operation is used to set bit 0 to 1, indicating that the physical allocation status flag is allocated. The setting operation is executed atomically to ensure consistency of state updates in a multi-threaded environment. For entries with a reference count greater than 1, a shared identifier is recorded in the extended bits of the status byte. This shared identifier is used to determine whether the block is referenced by the snapshot, preventing accidental reclamation of shared data blocks. During the traversal, the system maintains a processing progress counter, outputting a log every 1000 entries processed to facilitate monitoring the processing status and diagnosing performance bottlenecks. After the traversal is complete, the rows in the two-dimensional status matrix where all physical allocation status flags are 1 accurately reflect the set of virtual blocks actually occupied by the qcow2 image in the host file system. Once this stage is complete, the first dimension of the two-dimensional status matrix is ​​filled, preparing for the subsequent overlay of logical release statuses.

[0093] The process iterates through each entry in the client's logical release set, locating multiple rows corresponding to the logical block address range of that entry in the two-dimensional state matrix. The logical release status flags of these rows are then batch-set to the released state. Logical release status filling is the second stage of synchronizing space release information from the client's file system perspective to the state matrix. This process sequentially reads each release entry from the client's logical release set data structure. Each release entry contains fields such as the logical block address range (starting and ending logical block addresses), cumulative release count, and most recent release timestamp. For each read release entry, the starting and ending logical block addresses are extracted, and the number of virtual blocks contained in the address range is calculated. The number equals the ending address minus the starting address plus 1. A batch update strategy is used to process continuous address intervals: first, the memory offset of the two-dimensional state matrix row corresponding to the starting logical block address is calculated; then, each row within the interval is processed iteratively. For each virtual block address within the interval, the corresponding matrix row is located, the status byte of that row is read, and a bitwise OR operation is used to set the first bit to 1, indicating that the logical release status flag is released. During the traversal, the cumulative number of releases is recorded in the extended field of the state matrix. This count is used to calculate the release certainty factor in the hole confidence score. After the traversal is completed, all rows in the two-dimensional state matrix with the logical release status flag set to 1 completely represent the set of virtual blocks released by the client file system by sending the TRIM command.

[0094] The current snapshot of the block-level dirty data bitmap is read, and the block addresses set in the bitmap are marked with additional active write status flags in the two-dimensional state matrix. Active write status marking is the third stage of processing, overlaying real-time access information of data blocks. This process filters data blocks with recent write operations, preventing the reclamation of still-in-use active data. The block-level dirty data bitmap records whether any write operations have occurred to each virtual block since the last checkpoint. Each bit in the block-level dirty data bitmap corresponds to a virtual block; a bit of 1 indicates that the block has been modified (dirty data), and a bit of 0 indicates that the block has not been modified (clean data). First, the shared memory region is read through a pre-deployed API interface to obtain a current snapshot copy of the dirty data bitmap; the current snapshot copy is scanned byte by byte; when a bit is found to be 1, the virtual block address corresponding to that bit is obtained. The corresponding row in the state matrix is ​​located based on the virtual block address, the status byte of the corresponding row is read, and a bitwise OR operation is used to set the second bit to 1, indicating that the active write status flag is active. Simultaneously, the last modification timestamp of each dirty block is read from the virtualization layer and appended to an extended field of the state matrix. This timestamp is used to calculate the time decay factor in the hole confidence score. After this stage is completed, a multi-dimensional state view is formed that integrates the host physical layer, the client logic layer, and the real-time access layer.

[0095] The process involves scanning each row of the two-dimensional state matrix, selecting rows where the physical allocation status flag is "allocated," the logical release status flag is "released," and the active write status flag is "inactive," and extracting the virtual block addresses of these rows into a candidate empty block address set. This process involves sequentially scanning the status bytes of each row starting from row 0 of the two-dimensional state matrix. For each row, the status byte is read and a triple condition check is performed: the first check checks if bit 0 (physical allocation status) is 1; the second check checks if bit 1 (logical release status) is 1; and the third check checks if bit 2 (active write status) is 0. These triple conditions are determined using a bitmask combination. When a status byte meets the conditions, the corresponding virtual block address is selected as a candidate block and added to the candidate empty block address set. The candidate set is stored using a dynamic array structure. Furthermore, during the corresponding scanning process, additional information for each candidate block is extracted from the extended fields of the two-dimensional state matrix, including the cumulative number of releases, the most recent release timestamp, the last modification timestamp, and the sharing identifier, and stored as metadata of the candidate block in the set. After the scan is completed, the candidate empty block address set is sorted by virtual block address. The sorted set facilitates subsequent continuous block merging and hierarchical processing operations. The final output candidate empty block address set is the core result of the pre-constructed bidirectional state comparison matrix, which clearly identifies virtual blocks that have been allocated at the host physical layer, released at the client logical layer, and currently have no active access. The corresponding virtual blocks have the basic conditions for safe reclamation, providing an accurate candidate data source for subsequent hole confidence scoring and hierarchical screening.

[0096] In this embodiment of the invention, the detailed implementation steps for calculating the hole confidence score for each candidate block include:

[0097] The current timestamp is obtained and the cooling duration of candidate blocks is calculated. A time decay factor is constructed to quantify the temporal stability characteristics of data blocks. The time decay factor is a temporal dimension indicator for evaluating the possibility of safe reclamation of data blocks, which can effectively capture the idle time characteristics and access time intervals of blocks. First, the current timestamp is obtained. For each candidate hole block selected in the bidirectional state comparison matrix, the last write timestamp of the corresponding block is extracted from the extended field of the block-level dirty data bitmap. The difference between the evaluation benchmark time and the last write timestamp is calculated to obtain the block cooling duration, and a time decay factor is constructed based on it. The time decay factor is calculated by comparing the block cooling duration with a preset time decay constant. When the block cooling duration is longer, the time decay factor approaches the saturation value, and when the block cooling duration is shorter, the time decay factor approaches zero. The numerical range of the time decay factor is [0, 1], which comprehensively describes the temporal aging degree of the data block and provides a quantitative basis for the temporal dimension evaluation of hole confidence.

[0098] The proportion of sectors covered by TRIM instructions in candidate blocks is statistically analyzed to construct a release deterministic factor to quantify the clarity of the client space release intention. The release deterministic factor is a key parameter for measuring the integrity of logical releases. By analyzing the coverage of block addresses by TRIM instructions, the thoroughness of the client file system release operation can be revealed. The calculation process is based on TRIM event records in the client logical release set, obtaining all TRIM operations overlapping with candidate block addresses; querying all release records covering the current candidate block address from the client logical release set; for each retrieved TRIM record, extracting the start and end addresses of the release interval, calculating the intersection of this interval with the candidate block address range, and accumulating the number of sectors in all intersections to obtain the cumulative number of sectors covered by TRIM instructions; simultaneously reading the total number of sectors in the candidate blocks, calculating the TRIM coverage rate, which is the cumulative number of covered sectors divided by the total number of sectors; constructing the release deterministic factor based on the TRIM coverage rate, and calculating the release deterministic factor... The formula is as follows: the TRIM coverage is segmented and mapped. When the TRIM coverage reaches full coverage, the release deterministic factor is assigned the highest confidence weight. When the TRIM coverage is partial coverage, linear interpolation is performed based on the coverage percentage. For block addresses with multiple repeated TRIMs, the cumulative number of releases is read from the client's logical release set, and the number of releases is used as the confidence weighting coefficient. The more releases, the higher the additional gain of the release deterministic factor (up to a preset upper limit threshold, usually 1.2 times). The accurate calculation of the release deterministic factor reflects the strength of the client's intention to release the block, providing reliable data support for the logical dimension evaluation of hole confidence.

[0099] The system reads access frequency statistics for candidate blocks and queries metadata dependencies to construct a cold data confidence factor and a metadata security factor to assess access popularity and snapshot dependency risk. The cold data confidence factor and metadata security factor are dual expressions of block activity and metadata consistency, respectively describing the block's access behavior characteristics and snapshot sharing status. The cold data confidence factor is calculated based on an access frequency counter array, which records the number of I / O accesses for each virtual block within the observation time window. By obtaining the access frequency value corresponding to the candidate block and simultaneously calculating the global average access frequency of all virtual blocks within a preset time window as the benchmark value for activity judgment, the access activity of the candidate block is calculated by dividing the candidate block access frequency by the global average access frequency. A cold data confidence factor is constructed based on the access activity. The calculation method for the cold data confidence factor is: a reverse mapping of access activity; the lower the access activity, the higher the cold data confidence factor; when the access activity exceeds the activity threshold, the cold data confidence factor is set as a penalty value. The metadata security factor is calculated by querying the metadata mapping index of the qcow2 image, reading the L2 mapping table entry status corresponding to the candidate block, and checking the flag field of the table entry; the table entry flag includes snapshot dependency markers. The L2 table includes status bits such as copy-on-write (COW) and compression, and reads two entry flags: snapshot dependency and copy-on-write. When an L2 entry is a snapshot dependency, it indicates that the physical cluster is shared by multiple snapshots and the reference count is greater than 1. In this case, the metadata security factor is set to a reduced weight value (e.g., 0.3) to prevent accidental reclamation from corrupting snapshot data. When it is a copy-on-write (COW) entry, it indicates that the block is in a copy-on-write (COW) state, and the metadata security factor is also set to a reduced weight value. For independent blocks without any dependency flags, the metadata security factor is set to a normal weight (usually 1.0), indicating safe reclamation at the metadata level. These two factors together reflect the access frequency and metadata constraints of the data block and are the security dimension guarantee in the void confidence assessment.

[0100] The void confidence score is obtained by weighted fusion of time decay factor, release determinism factor, cold data confidence factor and metadata security factor. The void confidence score is a comprehensive quantitative expression of the probability of candidate block reclamation, which can comprehensively weigh four key characteristics: time stability, release determinism, access popularity and metadata security. The weighted fusion adopts a weighted geometric mean formula. The void confidence score can comprehensively characterize the reclamation security and necessity of candidate blocks, and provide an accurate decision basis for subsequent hierarchical screening and void punching operations, so as to ensure the security and effectiveness of virtual disk space reclamation.

[0101] In this embodiment of the invention, the detailed implementation steps for screening the void segments to be reconstructed include:

[0102] The candidate hole block address set is sorted in ascending order by virtual block address. Candidate blocks with consecutive addresses are merged into candidate block segments, and the starting address and number of blocks in each segment are recorded. Ascending sorting reorganizes the scattered candidate blocks in the address space according to their virtual address order, effectively identifying spatially adjacent candidate blocks. Based on the sorting result, the candidate block addresses are traversed to determine if the virtual addresses of adjacent candidate blocks are consecutive. If they are consecutive, they are grouped into the same candidate block segment. The originally discrete candidate blocks are aggregated into a continuous segment structure, forming a candidate block segment set. A descriptor is created for each candidate block segment, recording the starting virtual block address and the number of blocks it contains. This reduces the granularity of subsequent processing, improves the efficiency of the reconstruction operation, and reflects the true distribution characteristics of mirrored holes in the virtual address space.

[0103] The minimum and arithmetic mean of the hole confidence scores for all candidate blocks within each candidate block segment are calculated. The weighted combination of the minimum and arithmetic mean is then used as the segment-level confidence score for that candidate block segment. The segment-level confidence score is a comprehensive measure of the overall reliability of the candidate block segment. This process first extracts the hole confidence scores for each candidate block within the segment and calculates the minimum and arithmetic mean of these scores. The minimum and arithmetic mean are then weighted according to preset weighting coefficients, which are pre-defined by those skilled in the art. This weighting strategy avoids completely rejecting an entire segment due to low scores in individual candidate blocks and prevents the arithmetic mean from masking potential risks, thus forming a balanced segment-level confidence score.

[0104] Based on a comparison between the segment-level confidence score and preset multi-level security thresholds, candidate blocks are categorized into three levels: high confidence, medium confidence, and low confidence. The multi-level security threshold system employs a dual-threshold grading mechanism, setting two key parameters: a high confidence threshold and a medium confidence threshold. The specific values ​​corresponding to the threshold ranges are dynamically adjusted based on the image type and application scenario. For each candidate block, its segment-level confidence score is compared with the high confidence threshold. If the score is greater than or equal to the high confidence threshold, it is classified as high confidence; if the score is less than the high confidence threshold but greater than or equal to the medium confidence threshold, it is classified as medium confidence; if the score is less than the medium confidence threshold, it is classified as low confidence. Candidate blocks with a high confidence level have high reconstruction security, candidate blocks with a medium confidence level require further observation, and candidate blocks with a low confidence level are not reconstructed for the time being.

[0105] The process reads the snapshot chain metadata of the qcow2 image, traverses the L2 mapping table of each snapshot node in the snapshot chain, checks whether the physical cluster address of the candidate block segment is referenced by other snapshot nodes, and sets a snapshot lock flag for candidate block segments with snapshot dependencies. The snapshot dependency check is used to identify whether the candidate block segment participates in the inter-snap-write replication mechanism. This process reads the location and number of entries of the snapshot table from the header metadata area of ​​the corresponding qcow2 image file, parses the topology of the snapshot chain, and obtains the L1 table address of each snapshot node. It then sequentially accesses the L1 table of each snapshot node to locate the corresponding L2 mapping table, which records the mapping relationship from virtual block addresses to physical cluster addresses in that snapshot. For each candidate block segment, it extracts the range of virtual block addresses it contains and searches for the physical cluster address corresponding to the virtual address in the L2 mapping table of each snapshot node. If the physical cluster address of the candidate block segment is referenced in other snapshot nodes outside the current snapshot, it indicates that the block segment is shared across snapshots, and a snapshot lock flag is set for the candidate block segment. The snapshot lock flag indicates that the block involves snapshot dependencies. Direct reconstruction may disrupt the data consistency of the snapshot and should be excluded from the scope of this round of reconstruction.

[0106] Candidate blocks with high confidence levels and no snapshot locking flags are identified as void segments to be reconstructed. Candidate blocks with medium confidence levels are added to an observation queue, and their void confidence scores are recalculated in the next evaluation cycle before a final decision is made. This process iterates through all candidate blocks, selecting those that possess both high confidence levels and no snapshot locking flags. These blocks offer sufficient reconstruction safety without affecting the integrity of the snapshot chain and are added to the set of void segments to be reconstructed. Blocks in this set will undergo actual void reconstruction operations in subsequent steps. For candidate blocks with medium confidence levels, a delayed decision strategy is adopted: they are added to an observation queue and timestamped. In the next evaluation cycle, the access patterns and I / O statistics of the corresponding blocks are reread, and the void confidence scores are updated. If the score rises to a high confidence level and there is no snapshot dependency, it is added to the set of void segments to be reconstructed; if the score drops to a low confidence level, it is removed from the observation queue. Candidate blocks with low confidence levels or those marked with snapshot lock are directly excluded from the scope of this round of reconstruction, ensuring the safety and reliability of the reconstruction process. Through this hierarchical screening mechanism, the candidate blocks are accurately classified and differentiated, maximizing the reconstruction effect while minimizing data risks.

[0107] In this embodiment of the invention, the detailed implementation steps for performing the hole drilling operation include:

[0108] The chunk size is calculated based on the cluster size of the qcow2 image and the preset reconstruction concurrency parameters. The chunk size directly affects reconstruction efficiency and concurrency control precision. During the calculation process, the chunk size is set to an integer multiple of the cluster size according to the file system's preset block alignment requirements and corresponding I / O throughput capabilities, ensuring strict alignment between chunk boundaries and cluster boundaries to avoid fragmented storage layouts. Simultaneously, the number of clusters contained in the chunk is adjusted according to the reconstruction concurrency parameters, enabling the data volume of a single chunk to be read in a single I / O operation. The reconstruction concurrency parameters refer to the maximum number of task threads or parallel processing units allowed to concurrently perform hole punching operations during the qcow2 image hole reconstruction process. In this application, the chunk represents the basic structure of the qcow2 image being divided into fixed-granularity units (i.e., reconstruction units) for online processing.

[0109] The virtual address space of the qcow2 mirror is uniformly divided according to the obtained chunk size. The virtual address space division begins at the start of the mirror's virtual address space and continues at equal intervals according to the chunk size until the entire virtual address space is covered, resulting in several non-overlapping reconstructable units. Each reconstructable unit is assigned a unique chunk identifier, which is a consecutive integer sequence starting from zero. This facilitates array index access and allows for quick location of the target reconstructable unit through simple numerical operations. A bidirectional mapping table is established between chunk identifiers and virtual address ranges. This bidirectional mapping table is stored in key-value pairs, where the key is the chunk identifier and the value is an address range structure containing the start and end virtual addresses. This unified partitioning management mechanism transforms the continuous virtual address space into discrete, manageable units, providing a clear granular division for subsequent concurrent scheduling and lock control.

[0110] The process iterates through all the void segments to be reconstructed, precisely mapping each void segment to its corresponding reconstructing unit, and constructing a queue of chunks to be processed. The void segment mapping process requires processing each entry in the list of void segments to be reconstructed one by one. The starting and ending cluster addresses of each void segment are read, and the starting and ending positions of the void segment in the virtual address space are calculated based on the conversion relationship between cluster addresses and virtual addresses. A reverse lookup is performed using the established bidirectional mapping table (where the reverse lookup quickly locates the corresponding chunk identifier based on any virtual address) to determine the chunk identifiers covered by the corresponding virtual address range. When the void segment to be reconstructed is entirely within a single reconstructing unit, it is directly associated with the corresponding chunk identifier. When the void segment crosses the boundaries of multiple reconstructing units, it is split into multiple sub-segments according to the reconstructing unit boundaries, and each sub-segment is associated with its corresponding chunk identifier, ensuring that each reconstructing unit only processes the void portion within its address range during subsequent processing. All chunk identifiers containing void segments to be reconstructed are collected and aggregated. After removing duplicate identifiers, a queue of chunks to be processed is formed. Each entry in the queue not only records the chunk identifier, but also associates with detailed information of all void segments to be processed within the reconstructing unit, including the precise cluster address range and confidence score of each void segment. Through this mapping mechanism, the originally disordered void segments to be reconstructed are organized into various reconstructing units in an orderly manner, laying the data foundation for subsequent batch processing and concurrent execution.

[0111] A chunk-level lock manager is established for the qcow2 image to implement concurrent access control at the refactoring unit level. The lock manager is the core mechanism to ensure the safety of concurrent refactoring operations. It adopts a fine-grained chunk-level locking strategy to maintain an independent lock state record for each refactoring unit in the image. The lock manager internally maintains a lock state array equal to the number of refactoring units. Array indices correspond to chunk identifiers, and each array element records the currently allocated read-write lock state for that refactoring unit. Lock states are divided into three types: idle, read-locked, and write-locked. The idle state indicates that the refactoring unit is not currently occupied by any thread or process and can accept new lock requests. The read-locked state allows multiple concurrent operations to access the refactoring unit in read-only mode, suitable for query and verification scenarios. The write-locked state ensures exclusive access to the refactoring unit; any other operation must wait for the write lock to be released before accessing it. Lock state transitions are implemented using atomic operation instructions, ensuring consistency and mutual exclusion of state transitions in a multi-threaded concurrent environment. Simultaneously, the lock manager also records metadata such as the lock holder identifier, lock acquisition timestamp, and lock type for each lock, used for deadlock detection, timeout handling, and lock contention statistics. Compared to traditional global locks or file-level locks, the chunk-level lock manager significantly reduces the scope of lock contention, enabling puncturing operations of different refactoring units to be executed in true parallel, fully leveraging the parallel computing capabilities of multi-core processors, and significantly improving the overall throughput of refactoring operations.

[0112] The process involves retrieving chunk identifiers sequentially from the queue of chunks to be processed, requesting a write lock from the lock manager, and performing an address translation operation. The process follows the queue order, retrieving chunk identifiers one by one, and initiating a write lock request for each chunk retrieved. The write lock request process includes two phases: lock state check and lock allocation. First, the current lock state of the target reconstructing unit is checked to verify if it is idle and not held by any other thread. If the check passes, the lock state of the reconstructing unit is set to write-locked using an atomic operation, and the current thread's lock information is recorded. If the target reconstructing unit is already locked, the concurrency strategy determines whether to wait for or skip that chunk and proceed to process the next chunk in the queue. Identifier; After successfully acquiring the chunk write lock, the address translation stage begins. The virtual address range corresponding to the reconstruction unit is obtained by querying the mapping table based on the chunk identifier, clarifying the virtual address boundaries of this puncturing operation. The cluster address list of all void segments to be reconstructed within this virtual address range is traversed. For each cluster address, the metadata structure of the qcow2 image is queried, including the mapping relationship between the L1 and L2 tables, resolving the correspondence between the virtual cluster address and its physical location within the image file. The qcow2 format employs a two-level indirect mapping mechanism: the virtual address first needs to find the corresponding L2 table location through the L1 table index, and then find the offset of the actual data cluster in the image file through the L2 table index, finally calculating the file offset of the cluster in the qcow2 image. The file offsets corresponding to all cluster addresses within the reconstruction unit are collected and organized, merging consecutive offset intervals to form several consecutive file offset ranges.

[0113] The calculated file offset range is used to make a fallocate system call to perform a hole punching operation to reclaim storage space. When performing the hole punching operation, the FALLOC_FL_PUNCH_HOLE flag and the FALLOC_FL_KEEP_SIZE flag must be specified at the same time. The `FALLOC_FL_PUNCH_HOLE` flag indicates that puncturing mode will be executed. Punching mode means that allocated physical storage blocks within a specified file offset range will be released and returned to the file system. In the file's logical structure, this range is marked as a hole, and the system will return zero data when reading this range subsequently. The `FALLOC_FL_KEEP_SIZE` flag ensures that the puncturing operation does not change the file's logical size; the file size attribute remains unchanged, only the internal physical storage allocation state changes. During the system call execution, the file system layer marks the physical disk blocks corresponding to the specified offset range as unallocated, updates the file system's block allocation bitmap and metadata, and returns the corresponding storage space to the host file system's free space pool, which can be reallocated and used by other files in the file system. After the operation is complete, the internal metadata of the qcow2 image needs to be updated synchronously, modifying the allocation flags of the corresponding L2 table entries, marking the punctured clusters as unallocated, and maintaining the consistency between the qcow2 metadata and the underlying file storage state. Finally, the chunk write lock of this reconstruction unit is released, restoring the lock state to an idle state, allowing other operations to access this reconstruction unit. By using this system-level hole punching mechanism in conjunction with qcow2 metadata synchronization updates, precise storage space reclamation is achieved, realizing the goals of image reconstruction and space optimization.

[0114] In this embodiment of the invention, the detailed implementation steps for updating L2 mapping table entries include:

[0115] Before performing the hole punching operation, the L2 mapping table to be modified is read from the qcow2 image into a memory buffer, creating a working copy of the L2 mapping table. This process first obtains the file offset address of the target L2 mapping table by querying the L1 mapping table before modifying the metadata. Then, a file read operation is performed based on this file offset address, loading the complete L2 mapping table content from the image file into the memory buffer. The memory buffer is pre-allocated with sufficient space to hold the entire L2 mapping table. After loading, a working copy of the L2 mapping table is created in the memory buffer. The working copy is a complete copy of the original table data, and all subsequent modification operations are performed on the working copy, without directly modifying the original L2 mapping table in the image file. This approach protects the integrity of the original metadata, ensuring that existing mapping relationships are not damaged even if exceptions occur during modification. Furthermore, it lays the foundation for a subsequent copy-on-write update mechanism, supporting atomic metadata switching operations.

[0116] In the working copy, locate the L2 entry corresponding to the hole segment to be reconstructed, and modify the key fields of the entry to remove the mapping relationship. The location process requires determining the entry index position of the corresponding virtual cluster in the L2 mapping table based on the virtual cluster address range of the hole segment to be reconstructed. Based on the entry index position, quickly locate the corresponding L2 entry position in the working copy. For each virtual cluster covered by the hole segment to be reconstructed, the content of its corresponding L2 entry needs to be modified. First, reset the physical cluster offset address field of the L2 entry to zero. Zero is a special marker agreed upon in the qcow2 format, indicating that the virtual cluster has not been allocated any physical storage space, and reading the virtual cluster should return all zero data. At the same time, modify the allocation status bit in the entry, changing it from allocated to unallocated. This allocation status bit is a flag field in the metadata that clearly identifies the cluster allocation status, and together with the physical cluster offset address, it constitutes complete allocation information. For holes spanning multiple virtual clusters, multiple consecutive L2 entries need to be processed in batches, traversing the entire virtual cluster address range and modifying the corresponding entries one by one; while keeping other entry fields unchanged during the modification process, including auxiliary information such as compression flags and write flags during copying, to ensure that only fields related to hole reconstruction are updated.

[0117] Calculate the CRC checksum for the modified working copy, request a new free cluster area, and write it to the updated L2 mapping table. The checksum calculation is done by traversing the binary data of all L2 entries in the working copy, calculating the cumulative checksum byte by byte according to the CRC checksum algorithm, and obtaining a CRC checksum that uniquely identifies the content of the working copy. After the checksum calculation is completed, storage space needs to be allocated in the qcow2 image for the updated L2 mapping table. A free cluster request is initiated to the image's space allocation manager. The allocation manager searches for consecutive free clusters in the unallocated area of ​​the image file. The number of free clusters is determined according to the size of the L2 mapping table. After the request is successful, the physical cluster offset address of the new free cluster area is obtained. This physical cluster offset address is a byte offset relative to the start position of the image file. The L2 mapping table data in the working copy and the calculated CRC checksum are written to the corresponding free cluster area. The file offset position of this write operation is recorded simultaneously. The file offset position is the offset address of the new L2 mapping table, which is a key parameter needed for subsequent updates to the L1 mapping table.

[0118] Read the L1 table entry that points to the original L2 mapping table from the L1 mapping table, and replace the pointer value through an atomic write operation to complete the mapping switch; the L1 mapping table is the top-level index structure of the qcow2 image metadata, where each L1 table entry stores a pointer to the L2 mapping table, and the pointer value is the physical offset address of the L2 mapping table in the image file. First, the location of the L1 entry corresponding to the L2 mapping table to be updated needs to be determined. The entry number of the corresponding L2 mapping table in the L1 mapping table is obtained based on the high-order index of the virtual address. The corresponding entry of the L1 mapping table is read from the image file, and its currently stored pointer value is obtained. This pointer value points to the storage location of the original L2 mapping table, which is the old address to be replaced. Next, the offset address of the new L2 mapping table is used as the new pointer value, and an atomic write operation is performed to replace the pointer value in the L1 entry from the address of the original L2 mapping table to the offset address of the new L2 mapping table. The key to the atomic write operation is ensuring the indivisibility of the write. After the pointer value replacement is completed, the L1 mapping table immediately points to the new L2 mapping table containing the updated content. Although the original L2 mapping table still exists in the image file, it is no longer referenced by the L1 mapping table. The update of the L1 mapping table is then forcibly persisted to the storage medium using the fsync system call, ensuring the persistence of metadata updates. fsync is a predefined system call interface specifically used to synchronize all modified data and metadata of a file from the memory cache to the underlying storage medium.

[0119] The physical cluster offset addresses occupied by the original L2 mapping table are added to the free cluster pool with a reference count of zero, completing the reclamation and reuse of storage space. This process first determines the physical cluster offset addresses occupied by the original L2 mapping table and the number of clusters it spans; then it checks the reference count status of these physical clusters. The reference count is an internal counter used by the qcow2 mirror to track the number of times each physical cluster is referenced. When the reference count of a physical cluster drops to zero, it indicates that no virtual address or metadata structure points to that virtual cluster, and the virtual cluster can be safely released. Since the original L2 mapping table is no longer referenced by the L1 mapping table, the reference count of the physical clusters it occupies should be zero. The physical cluster offset addresses are then added to the free cluster pool data structure maintained by the mirror manager. The free cluster pool is a dedicated management pool that records reusable storage space, using data structures such as linked lists, bitmaps, or red-black trees to organize free cluster information, supporting fast space allocation and release operations. After being added to the free cluster pool, these physical clusters can be reused for subsequent data write operations or metadata update operations in the next reconstruction cycle. When a new L2 mapping table needs to allocate storage space, it is preferentially obtained from the idle cluster pool to achieve the recycling of storage space.

[0120] In this embodiment of the invention, the detailed implementation steps for adjusting the execution rate of block reconstruction operations include:

[0121] Before and after each hole punching operation, I / O performance metrics of the virtual machine are collected through the performance monitoring interface of the virtualization layer. The performance monitoring interface is a standardized monitoring method provided by the virtualization platform, capable of acquiring underlying I / O operational status data without interfering with the normal operation of the virtual machine. This process begins by sending a query request to the performance monitoring interface of the virtualization layer before each hole punching operation begins, collecting a snapshot of the virtual machine's I / O performance metrics at the current moment, and recording it as the pre-operation performance baseline. I / O performance metrics include average read request response time (representing the average time span from request submission to data return after the virtual machine initiates a read operation), average write request response time (representing the average time span from request submission to write completion confirmation after the virtual machine initiates a write operation), and I / O request queue depth (representing the number of incomplete I / O requests waiting to be processed at the block device layer). After the hole punching operation is completed, another snapshot of I / O performance metrics is collected through the same monitoring interface and recorded as the post-operation performance status. By comparing the changes in I / O performance metrics before and after the operation, the immediate impact of this hole punching operation on the virtual machine's business I / O performance can be accurately assessed.

[0122] Using I / O performance metrics collected during virtual machine idle operation as a baseline, the ratio of the current collected metrics to the baseline value is calculated as the I / O request latency change rate. Idle operation refers to the ideal state where the virtual machine has started but is not running any business load; in this state, the virtual machine's I / O system is at its lowest load level. Metrics are collected during the initial startup of the virtual machine. The ratio of the current collected I / O performance metrics to the corresponding baseline value is calculated to obtain the change factor for each metric. The ratios of the current average response time for read requests, the average response time for write requests, and the I / O request queue depth to the baseline value are calculated separately. These three ratios are then weighted and averaged or the maximum value is taken to obtain the comprehensive I / O request latency change rate. This relative ratio calculation method eliminates the absolute performance differences caused by variations in hardware platforms and configurations, making the adjustment strategy universally applicable.

[0123] When the I / O request latency change rate exceeds both the preset first warning threshold and the preset second warning threshold, the execution interval of the refactoring operation is increased to a preset multiple of the current execution interval, implementing a gradual rate reduction strategy. The first warning threshold is the initial warning line that triggers the adjustment mechanism, typically set between 1.2 and 1.5. When the I / O request latency change rate is detected to be greater than the preset first warning threshold, it indicates that the refactoring operation has begun to have a perceptible impact on the normal business of the virtual machine, requiring a proactive reduction in the execution speed of the refactoring operation to alleviate resource contention. The execution interval refers to the time interval between two adjacent hole punching operations. The current execution interval is increased to a preset multiple, the specific multiple value of which is set by those skilled in the art based on the actual scenario. This gradual adjustment strategy avoids drastic fluctuations in rate, responding promptly to performance degradation signals without causing the refactoring task to be unable to complete for an extended period due to excessive conservatism. After adjustment, the trend of the I / O request latency change rate is continuously monitored. If the latency change rate falls back, the current execution interval is maintained; if it continues to rise, more stringent limiting measures are prepared to be triggered.

[0124] When the I / O request latency change rate exceeds the preset second warning threshold, all refactoring operations are immediately suspended and a cooldown timer is started to execute the emergency protection mechanism. The second warning threshold is a strict red line for performance protection, typically set between 1.8 and 2.5, indicating that the virtual machine's service performance has been severely affected. When the I / O request latency change rate is detected to be greater than the preset second warning threshold, it indicates that the refactoring operation has caused a serious resource conflict with the virtual machine's service load. Continuing to execute the refactoring operation may lead to the unavailability of the virtual machine's services or a significant deterioration in user experience. Therefore, all ongoing and pending refactoring operations are immediately suspended, including hole punching operations that have been initiated but not yet completed. The suspension is achieved by setting a global suspension flag. All refactoring threads immediately stop processing new chunks and release the resource locks they hold upon detecting this flag. Simultaneously, a cooldown timer is initiated. This timer is a countdown timer with an expiration duration set according to system load characteristics. During the cooldown period, reconstruction operations are completely halted, and all I / O bandwidth and storage resources are allocated to the virtual machine's workload, allowing the virtual machine I / O system sufficient time to recover to normal. After the cooldown timer expires, the original execution rate is not immediately restored. Instead, reconstruction operations are cautiously resumed at the lowest possible execution rate, typically referring to the maximum execution interval or minimum concurrency setting, ensuring that the reconstruction operation restarts with minimal impact. After recovery, the I / O request latency rate is continuously and closely monitored, and decisions are made based on subsequent changes to determine whether to further reduce or gradually increase the speed.

[0125] When the rate of change of I / O request latency is less than the preset safe recovery threshold for multiple consecutive sampling periods, the execution interval of the reconstruction operation is gradually shortened until it is restored to the default execution rate, thereby achieving adaptive rate improvement. The safe recovery threshold is the standard line for judging the stable recovery of virtual machine I / O performance. It is usually set to a value between 1.1 and 1.2, which means that the current I / O latency is only slightly higher than that of the idle state, the virtual machine load is light, and there is sufficient I / O resource margin. A single sample value below the safe recovery threshold might be an occasional fluctuation and cannot be used as sufficient evidence for speedup. Therefore, it is required that the value remain below the safe recovery threshold for multiple consecutive sampling periods. When the value remains below the safe recovery threshold for multiple consecutive sampling periods, it indicates that the virtual machine is in a stable low-load state and has the ability to withstand a higher reconstruction rate. At this point, the execution rate of the reconstruction operation can be appropriately increased by gradually shortening the execution interval. The shortening strategy adopts a graded and gradual approach, shortening the current execution interval by a fixed percentage each time to avoid performance shocks caused by sudden rate changes. After each shortening of the execution interval, the I / O request latency change rate is monitored for several subsequent sampling periods. If it remains below the safe recovery threshold, the interval is shortened further; if it exceeds the threshold, the shortening stops and the current interval is maintained until the execution interval returns to the default execution rate, i.e., the standard reconstruction rate initially set by the system. Through this adaptive rate increase mechanism, the reconstruction task can automatically accelerate its execution progress when the virtual machine load decreases, improving overall reconstruction efficiency and making full use of available system resources.

[0126] Example 2, please refer to Figure 2 As shown, parts not described in detail in this embodiment are described in Embodiment 1. A block-based virtual disk space management system is provided, including:

[0127] The spatial state collection module is used to acquire real-time block state data of the qcow2 image in the virtual machine runtime environment, construct host physical mapping set and client logic release set based on the real-time block state data, cross-compare them, extract candidate blocks based on the comparison results, and construct a bidirectional state comparison matrix.

[0128] The dynamic hole identification module is used to calculate the hole confidence score of each candidate block based on the bidirectional state comparison matrix and the real-time block state data; and to sort the candidate blocks by level and filter the continuous block address segments based on the hole confidence score to obtain the hole segments to be reconstructed.

[0129] The online reconstruction module is used to divide the qcow2 image into multiple reconstruction units, apply for a fine-grained read-write lock for the reconstruction unit containing the hole segment to be reconstructed, perform hole punching operation after acquiring the read-write lock, and update the L2 mapping table entries synchronously.

[0130] The monitoring and feedback module is used to monitor the I / O request latency change rate of the virtual machine in real time during the hole punching operation, and dynamically adjust the execution rate of the block reconstruction operation based on the I / O request latency change rate.

[0131] The modules are connected via wired and / or wireless means to enable data transmission between them.

[0132] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0133] All formulas in this manual are dimensionless and calculated numerically. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0134] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A virtual disk space management method based on block reconstruction, characterized in that, include: Obtain real-time block status data of the qcow2 image in the virtual machine runtime environment. The real-time block status data includes L1 / L2 mapping table entries, trim instruction sequence, block-level dirty data bitmap, and physical block allocation record. A host physical mapping set is constructed based on the L1 / L2 mapping table entries and physical block allocation records; and a client logic release set is constructed synchronously based on the trim / discard instruction sequence. The host physical mapping set and the client logic release set are cross-compared, and candidate blocks are extracted based on the comparison results, and a bidirectional state comparison matrix is ​​constructed. Based on the bidirectional state comparison matrix and combined with the block-level dirty data bitmap, the hole confidence score of each candidate block is calculated. Candidate blocks are sorted and ranked according to the hole confidence score, and continuous block address segments with confidence scores greater than a preset security threshold are selected as hole segments to be reconstructed. The qcow2 image is divided into multiple reconstruction units, and a fine-grained read-write lock is applied for the reconstruction unit containing the hole segment to be reconstructed. After the read-write lock is acquired, the hole punching operation is performed, and the L2 mapping table entries are updated synchronously. During the hole punching operation, the I / O request latency change rate of the virtual machine is monitored in real time, and the execution rate of the block reconstruction operation is dynamically adjusted according to the I / O request latency change rate.

2. The virtual disk space management method based on block reconstruction according to claim 1, characterized in that, The process of constructing the host physical mapping set includes: Read the header metadata area in the image file corresponding to the qcow2 image to obtain the starting offset address, total number of entries, and cluster size parameter of the L1 mapping table entry; Based on the starting offset address, traverse each L1 entry in the L1 mapping table and extract the physical offset address pointing to the L2 mapping table entry for each non-empty L1 entry. Perform an entry-by-entry scan of each allocated L2 mapping entry, and read the physical cluster offset address and reference count field in each L2 entry; Based on the physical block allocation record, verify the allocation status of the physical cluster offset address in each L2 table entry in the host file system, and filter out residual mapping entries that have been released by the physical layer but have not been updated by the L2 table entry. The verified physical cluster offset address is associated with the corresponding virtual block address to construct the host physical mapping set. Each entry in the host physical mapping set contains a virtual block address, a physical cluster offset address, and a cluster allocation status identifier.

3. The virtual disk space management method based on block reconstruction according to claim 1, characterized in that, The process of constructing the customer logic release set includes: Register hook functions for intercepting TRIM commands; capture all block device requests sent by the virtual machine guest through the registered hook functions, parse the command type field of each request, and identify space release requests that conform to trim or discard semantics; Extract the starting address of the target logical block, the number of blocks to be released, and the timestamp of the request from the identified space release request, encapsulate them as a trim event record and append them to the circular buffer queue; The trim event records in the circular buffer queue are sorted by logical block address, and a range merging operation is performed on multiple trim event records with consecutive addresses or overlapping ranges. The client logic release set is constructed based on the merged trim event records. Each entry in the client logic release set contains a logic block address range, a cumulative number of releases, and a recent release timestamp.

4. The virtual disk space management method based on block reconstruction according to claim 1, characterized in that, The process of constructing the bidirectional state alignment matrix includes: Using the virtual block address space of the qcow2 image as the row index range, a two-dimensional state matrix structure is created, where each row of the two-dimensional state matrix corresponds to a virtual block address; Traverse each entry in the host physical mapping set, locate the row corresponding to the virtual block address of the corresponding entry in the two-dimensional state matrix, and set the physical allocation status flag of the row to the allocated status. Traverse each entry in the client logic release set, locate multiple rows corresponding to the logical block address range of the corresponding entry in the two-dimensional state matrix, and set the logical release status flag of the corresponding row to the released state. Read the block-level dirty data bitmap and mark the block addresses set in the block-level dirty data bitmap with additional active write status flag bits in the two-dimensional state matrix; Scan each row of the two-dimensional state matrix, filter out rows where the physical allocation status flag is allocated, the logical release status flag is released, and the active write status flag is inactive, take the virtual block address in the corresponding row as candidate block, extract it into a candidate empty block address set, and record it as the output result of the bidirectional state comparison matrix.

5. The virtual disk space management method based on block reconstruction according to claim 1, characterized in that, The process of calculating the hole confidence score includes: Read the last modified timestamp of the candidate block from the block-level dirty data bitmap, obtain the difference between it and the current timestamp, and construct a time decay factor based on it; The access frequency value of the candidate block is read from the access frequency counter array pre-deployed in the virtual machine, the ratio of the access frequency value to the global average access frequency within a preset time window is calculated, and a cold data confidence factor is constructed based on it. The number of trim instructions covering the current candidate block is counted from the trim instruction sequence, and the ratio of the cumulative number of sectors covered by trim instructions in the candidate block to the total number of sectors in the candidate block is calculated based on the ratio result. A release deterministic factor is then constructed based on the ratio result. Read the L2 table entry corresponding to the candidate block, and check whether the L2 table entry has a snapshot dependency mark or a copy-on-write mark. If a dependency mark exists, construct the metadata security factor with a reduced weight value; if no dependency mark exists, construct the metadata security factor with a normal weight value. The time decay factor, release determinism factor, cold data confidence factor, and metadata security factor are weighted and fused to obtain the hole confidence score of the corresponding candidate block.

6. The virtual disk space management method based on block reconstruction according to claim 4, characterized in that, The process of obtaining the void segment to be reconstructed includes: The candidate empty block address set is sorted in ascending order by virtual block address, and candidate blocks with consecutive addresses are merged into candidate block segments. The starting address and number of blocks in each candidate block segment are recorded. Obtain the minimum and mean values ​​of the hole confidence scores of all candidate blocks within each candidate block segment, and use the weighted combination of the minimum and mean values ​​as the segment-level confidence score of that candidate block segment. Based on the comparison between the segment-level confidence score and the preset multi-level security threshold, the candidate block segments are divided into three categories: high confidence level, medium confidence level, and low confidence level. Read the snapshot chain metadata of the qcow2 image, traverse the L2 mapping table entry of each snapshot node in the snapshot chain metadata, check whether the physical cluster address of the candidate block segment is referenced by other snapshot nodes, and set the snapshot lock flag for the candidate block segment whose physical cluster address is referenced by other snapshot nodes. Candidate blocks with high confidence levels and no snapshot lock flag are identified as void segments to be reconstructed; candidate blocks with medium confidence levels are added to the observation queue, and a decision is made after recalculating the void confidence score in the next evaluation cycle.

7. The virtual disk space management method based on block reconstruction according to claim 1, characterized in that, The process of performing a hole drilling operation includes: The virtual address space of the qcow2 image is evenly divided according to the preset chunk size to obtain several non-overlapping reconstruction units. A unique chunk identifier is assigned to each reconstruction unit, and a mapping table from chunk identifier to virtual address range is established. Traverse all the empty segments to be reconstructed, map the start cluster address and end cluster address of each empty segment to the corresponding chunk identifier, and build a chunk queue to be processed; A chunk-level lock manager is established for the qcow2 image. The lock manager maintains the lock state of the read and write locks allocated to each reconstruction unit. The lock state includes three states: idle, read locked, and write locked. Chunk identifiers are retrieved sequentially from the chunk queue to be processed. A write lock for the corresponding chunk is requested from the lock manager. After successfully acquiring the chunk write lock, the corresponding virtual address range is calculated based on the chunk identifier. All cluster addresses of the hole segment to be reconstructed within the corresponding range are traversed. The cluster addresses are converted into file offsets of the corresponding image files of the qcow2 image. The file offsets corresponding to all cluster addresses within the reconstruction unit are collected and organized, and continuous offset intervals are merged to form several continuous file offset ranges. A fallocate system call is made to the file offset range, and a hole punching operation is performed by specifying the FALLOC_FL_PUNCH_HOLE and FALLOC_FL_KEEP_SIZE flags, so that the physical storage space corresponding to the file offset range is returned to the host file system.

8. The virtual disk space management method based on block reconstruction according to claim 1, characterized in that, The process of updating L2 mapping table entries includes: Before performing the hole punching operation, the L2 mapping table entry to be modified is read from the corresponding image file of the qcow2 image and saved to the memory buffer, and a working copy of the L2 mapping table entry is created. Locate the L2 entry corresponding to the void segment to be reconstructed in the working copy, reset the physical cluster offset address field of the corresponding L2 entry to zero, and modify the allocation status bit of the L2 entry to the unallocated status. Calculate the CRC check value for the modified working copy, apply for a new free cluster area in the qcow2 image, write the modified working copy and the CRC check value into the free cluster area, and record the writing position as the offset address of the new L2 mapping table. Read the L1 table entry that points to the original L2 table entry from the L1 mapping table, replace the pointer value of the L1 table entry with the offset address of the new L2 mapping table through an atomic write operation, persist the update of the L1 mapping table to the storage medium, and simultaneously add the physical cluster offset address occupied by the original L2 table entry to the free cluster pool with a reference count of zero.

9. The virtual disk space management method based on block reconstruction according to claim 1, characterized in that, The process of adjusting the execution rate of block refactoring operations includes: Before and after each hole punching operation, the I / O performance metrics of the virtual machine are collected. The I / O performance metrics include the average response time of read requests, the average response time of write requests, and the I / O request queue depth. Using the I / O performance metrics collected under virtual machine idle running conditions as the benchmark value, the ratio of the collected I / O performance metrics to the benchmark value is calculated to obtain the I / O request latency change rate. When the rate of change of I / O request latency is greater than the preset first warning threshold but not greater than the preset second warning threshold, the execution interval of the reconstruction operation will be increased to a preset multiple of the current execution interval. When the rate of change of I / O request latency exceeds the preset second warning threshold, all reconstruction operations are immediately suspended and a cooling timer is started. After the cooling timer expires, the reconstruction operation is resumed at the lowest execution rate. When the rate of change of I / O request latency is less than the preset safety recovery threshold for multiple consecutive sampling periods, the execution interval of the reconstruction operation is gradually shortened until it is restored to the default execution rate.

10. A block-based virtual disk space management system, used to implement the block-based virtual disk space management method according to any one of claims 1 to 9, characterized in that, include: The space state collection module is used to obtain real-time block state data of the qcow2 image in the virtual machine runtime environment, and construct the host physical mapping set and the client logical release set based on the real-time block state data. They are then cross-aligned, candidate blocks are extracted based on the alignment results, and a bidirectional state alignment matrix is ​​constructed. The dynamic hole identification module is used to calculate the hole confidence score of each candidate block based on the bidirectional state comparison matrix and the real-time block state data. The candidate blocks are then sorted and continuous block address segments are filtered based on the hole confidence score to obtain the hole segments to be reconstructed. The online reconstruction module is used to divide the qcow2 image into multiple reconstruction units, apply for a fine-grained read-write lock for the reconstruction unit containing the hole segment to be reconstructed, perform hole punching operation after acquiring the read-write lock, and update the L2 mapping table entries synchronously. The monitoring and feedback module is used to monitor the I / O request latency change rate of the virtual machine in real time during the hole punching operation, and dynamically adjust the execution rate of the block reconstruction operation based on the I / O request latency change rate.

Images