Cross-disk block-level recombination migration method supporting capacity self-adaption

Abstract

The invention discloses a cross-disk block-level recombination migration method supporting capacity self-adaption, and relates to the technical field of computer storage management, and the method specifically comprises the following steps: collecting and preprocessing source disk information; planning a target environment and constructing a mapping table; transmitting a core structure and initializing a target partition; block-level data migration and log synchronization are carried out; according to the method, a partition table structure, a file system structure and an initial bitmap are constructed by scanning a source disk partition table and file system metadata to lock effective data, and a target new bitmap and a source-target data block mapping table are generated based on a target disk capacity planning mapping rule; according to the method, capacity adaptation and defragmentation are achieved, a core structure is transmitted, a target partition table and a file system are initialized, effective data blocks and log blocks are migrated according to a mapping table, finally, the migration effect is ensured through multi-layer verification, partition intelligent expansion and contraction, cross-disk multi-target mapping and fragment optimization are achieved, and migration flexibility and efficiency are improved.

Description

Technical Field

[0001] This invention relates to the field of computer storage management technology, specifically to a method for cross-disk partition block-level reorganization migration that supports capacity adaptation. Background Technology

[0002] Block-level migration, as an important method of data migration, operates on data blocks in storage devices. Each block is typically tens or hundreds of KB in size. This technology directly reads and writes data blocks on the underlying storage system without processing the file system, thus enabling efficient migration of large amounts of data while ensuring data integrity and consistency. In cross-disk partition reorganization scenarios, the capacity adaptation potential of block-level migration can be achieved by dynamically adjusting the data block allocation strategy. With the surge in demand for server virtualization, cloud disaster recovery, and terminal upgrades, it has become commonplace to quickly migrate the operating system or data partitions on the source disk to the target disk with different capacities. Traditional block-level copying technology has significant limitations, including poor capacity adaptability, insufficient support for multi-target scenarios, and severe fragmentation. Traditional block-level copying tools (such as dd) require strict matching of the physical capacity of the source and target partitions, requiring the target partition capacity to be greater than or equal to the total capacity of the source partition. If the target capacity is less than the source partition but greater than the effective data volume of the source (e.g., source partition 100GB / effective 30GB → target 80GB), or if the target partition needs to be expanded (e.g., 50GB → 100GB), the file system must be manually adjusted; otherwise, the copying or migration cannot be completed. This operation is complex and error-prone, resulting in poor capacity adaptability. Furthermore, existing technologies cannot automatically distribute and migrate multiple partitions from a single source disk to different physical disks (e.g., partition 1 → disk A, partition 2 → disk B), and lack cross-disk consistency guarantee mechanisms. This can easily disrupt file system associations (e.g., directory structure, index pointing), leading to incomplete data and insufficient support for multi-target scenarios. Traditional solutions only skip free blocks and still write data in the original LBA (Logical Block Address) order. If the source partition is fragmented, the target partition will inherit the fragmentation problem, leading to decreased IO performance and severe fragmentation. Existing technologies support basic disk partition copying, but they still fail to solve core issues such as dynamic capacity adaptation, multi-target mapping consistency, and fragmentation optimization, making it difficult to meet the needs of flexible storage applications. Summary of the Invention

[0003] This invention provides a cross-disk partition block-level reorganization migration method that supports capacity adaptation. It can effectively solve the significant limitations of traditional block-level replication technology mentioned in the background, such as poor capacity adaptability, insufficient support for multi-target scenarios, and severe fragmentation. Existing technologies support basic disk partition replication, but still fail to solve the core problems of dynamic capacity adaptation, multi-target mapping consistency, and fragmentation optimization, making it difficult to meet the needs of flexible storage applications.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a cross-disk partition block-level reorganization and migration method that supports capacity adaptation, supports multi-target and optimizes fragmentation during block-level partition copying. This method involves block-level disk partition copying, migration, and optimization, and is suitable for data backup and recovery and storage migration scenarios. Specifically, it includes the following steps: S1: Source disk information collection and preprocessing; S2: Target environment planning and mapping table construction; S3: Core structure transmission and target partition initialization; S4: Block-level data migration and log synchronization; S5: Migration verification and consistency check; S1 includes the following steps: S11: Source disk partition table scan and partition table structure construction; S12: Source file system metadata scanning and file system structure construction; S13: Initial bitmap construction of source disk partitions; S2 includes the following steps: S21: Target disk information scanning and capacity verification; S22: Multi-objective mapping rule; S23: Construction of new bitmap for target disk partition and source-target mapping table; S24: Update the file system structure; S3 includes the following steps: S31: Core structure is transferred to the target disk; S32: Target disk partition table construction; S33: Target disk file system construction; S4 includes the following steps: S41: Valid data block migration; S42: File log block migration; S5 includes the following steps: S51: Data integrity verification; S52: File system availability verification; S53: Capacity and fragmentation verification; S54: Exception rollback and report generation.

[0005] According to the above technical solution, in step S1, by scanning the source disk partition table and file system metadata, a partition table structure, a file system structure and an initial bitmap are constructed to lock the valid data to be migrated, providing basic data support for subsequent mapping and migration; S2, based on the target disk capacity and distribution, plans mapping rules, constructs a new bitmap of the target disk partitions and a source-target data block mapping table, and updates the file system structure; S3 involves transferring the partition table structure, updated file system structure, and source-target mapping table of the source disk to the target disk, and initializing the target partition table and file system based on these structures to prepare for data migration. S4, based on the initial bitmap and mapping table, migrates the source valid data blocks to the target partition and synchronizes log blocks to ensure data consistency; The S5 process involves multiple layers of verification, including block-level comparison, file system verification, and performance testing, to ensure that the migration results meet expectations.

[0006] According to the above technical solution, S11 specifically involves scanning the partition table of the source disk to extract key partition information and record it in the partition table structure. During migration, the partition table structure needs to be transferred to the target disk. In step S12, key information is extracted and recorded in the file system structure by scanning the metadata of the source partition file system. During migration, the file system structure needs to be transferred to the target disk. S13 is to create a bitmap based on the partition size and store it in the form of a binary file. Each bit of the bitmap corresponds to a continuous data block area of ​​the source partition, and the initial value is 0. Traverse all files in the file system, parse the file data block positions, and set the corresponding position in the bitmap to 1, indicating that the data block corresponding to this bit is valid data; The size of the data block corresponding to the bitmap is set according to different file systems.

[0007] According to the above technical solution, in step S21, the physical parameters of the target disk are scanned, the total physical capacity, number of available blocks, and size of a single block of each target disk are recorded, and it is verified whether the capacity meets the migration requirements. The capacity verification logic includes the following: Single-target scenario: Total number of available blocks on the target disk ≥ Total number of valid blocks on the source partition; Multi-target scenario: The number of available blocks on each target disk is greater than or equal to the number of valid blocks on the corresponding source partition.

[0008] According to the above technical solution, as described in S23, the specific construction process includes the following steps: S2301, traverse the files in the source file system, traversing all files in directory hierarchy order; S2302, Data block rearrangement and mapping, specifically for each file, mapping its scattered source data blocks to target contiguous blocks in logical order; S2303, record the mapping relationship, wherein the mapping table content includes the start and end of the source logical block address, the start and end of the target logical block address, the file ID to which it belongs, and the block type; S2304, Generate a new target bitmap. Specifically, based on the target block range of the mapping table, mark the corresponding position in the new bitmap as 1 and the rest as 0, and verify the continuity of the bitmap. S2305, persistently saves bitmaps and mapping tables.

[0009] According to the above technical solution, step S24 specifically includes the following implementation steps: S2401 creates a metadata structure for each file and directory in the target disk file system, sets basic attributes such as filename, path, size, and creation / modification / access time, sets security attributes such as file permissions, owner, group, and access control list, and sets file type and file flags; For special files, their metadata must be set correctly according to the file system specification, and file system-specific extended attributes or custom metadata must be written. S2402, Update the global metadata of the file system to ensure that the file system structure is complete and usable; S2403, verify the correctness of metadata writing, which can be done by traversing the file system structure and comparing it with the source disk; S2404 persists all metadata to the metadata area of ​​the target disk's file system.

[0010] According to the above technical solution, step S31 involves transmitting the partition table structure, the updated file system structure, and the source-target mapping table to the target disk. During the transmission process, a checksum is calculated to ensure integrity. The data is temporarily stored in the boot partition or memory buffer of the target disk and written to the corresponding metadata area after the target partition is initialized. S32 is based on the partition table structure, creating a partition table on the target disk, defining the physical layout of the target partition, ensuring that the partition logic of the target disk is consistent with that of the source disk, and providing physical space for file system initialization; In step S33, based on the updated file system structure, the file system structure information is parsed, a file system of the same type as the source is created on the target partition, the target file system metadata is initialized, the basic framework of the target file system is constructed, and the file system structure construction log is recorded.

[0011] According to the above technical solution, step S32 specifically includes the following implementation steps: S3201 scans the target environment, records the total physical capacity, number of available blocks, and size of each target disk, confirms that the target disk is unpartitioned or uninitialized, and ensures that there are no existing partition tables or partitions on the disk. S3202, verify capacity feasibility, including single-target scenario and multi-target scenario. In single-target scenario: the total number of available blocks on the target disk is greater than or equal to the total number of valid blocks on the source partition. In multi-target scenario: the number of available blocks on each target disk is greater than or equal to the number of valid blocks on the corresponding source partition. S3203 creates a new partition table structure on the target disk based on the source disk's partition table type, and allocates the starting and ending sectors of each partition sequentially to ensure partition alignment for improved performance. This includes the following specific scenarios: Target disk capacity = source partition total capacity: Create a partition table on the target disk with the same physical layout as the source partition according to the source partition structure, and match the total capacity of the source partition without adding or reducing space. Target disk capacity > source partition total capacity: When creating the partition table on the target disk, the starting LBA remains unchanged, the extended partition extends to the range corresponding to the target capacity from the ending LBA, and the extended space is reserved in the target partition as free blocks. It is then enabled through the file system extension mechanism to achieve adaptive capacity expansion. Source partition total capacity > target disk capacity > source partition effective data volume: When creating the partition table on the target disk, adjust the end LBA of the partition to the range corresponding to the target capacity. The shrinking process only removes free blocks in the source partition that exceed the target capacity, ensuring that the target partition capacity completely covers the effective data of the source partition without affecting data integrity. S3204 sets the partition type identifier, sets the partition unique identifier, writes to the partition table, and completes the creation of all partitions.

[0012] According to the above technical solution, in step S41, the valid data block marked as 1 in the initial bitmap of the source partition is read and written to the corresponding position of the target according to the mapping table, including migration order and multi-threading optimization; The migration order ensures that the basic structure of the file system is in place first during the migration process. In the event of an unexpected interruption, the recovery process can better locate and recover data based on the migrated metadata, maximizing the recoverability of the target file system. For single-target scenarios, multi-threading optimization enables multi-threading, and the migration is performed in segments according to the target block range of the mapping table. Block conflicts are avoided by using distributed locks. For multi-target scenarios across disks, multiple threads are started to handle the migration of different partitions to different target disks respectively. The threads synchronize their progress through shared memory. In step S42, the log blocks of the file system are migrated. The valid data blocks of the file log are read according to the initial bitmap, the location of the mapped file log data blocks is obtained from the mapping table, and the file log data blocks are migrated to the target disk partition to ensure data consistency. Log blocks record incomplete file operations in the source partition; synchronization after migration can prevent data loss or inconsistency.

[0013] According to the above technical solution, S51 specifically includes block-level comparison and file-level sampling verification; S52 specifically includes verification through testing, including mount testing, read / write testing, and startup testing. Mounting tests involve attempting to mount the target partition to the test environment to check for any file system corruption warnings. If a corruption warning appears, it is repaired using the file system's built-in repair tools. The read / write test involves creating new files, writing data, and deleting files on the target partition to verify that the file system can perform block-level allocation and release operations normally, and that the metadata is updated synchronously after the operation. The boot test mainly targets the boot partition. The target partition is loaded through a virtual machine to verify whether the operating system can boot normally to the login screen and whether the core programs can run normally. The S53 includes capacity adaptation and fragmentation rate verification. Capacity adaptation is achieved by using df and disk properties to check the target partition capacity and confirm that it is consistent with the plan. Fragmentation rate verification is achieved by using a dedicated tool to detect the fragmentation rate of the target partition and storing the fragmentation rate result as evidence for performance optimization. S54 includes a rollback mechanism and a migration report generation. The rollback mechanism automatically triggers a rollback when verification fails, deletes the target partition data, restores the target partition snapshot before migration, or re-migrates based on the list of valid blocks and the mapping table.

[0014] Compared with existing technologies, the beneficial effects of this invention on the capacity-adaptive cross-disk partition block-level reorganization migration method are as follows: 1. By scanning the source disk partition table and file system metadata, the system constructs the partition table structure, file system structure, and initial bitmap to lock valid data. Based on the target disk capacity, it plans mapping rules, generates a new target bitmap and a source-target data block mapping table, and achieves capacity adaptation and defragmentation. It transmits the core structure and initializes the target partition table and file system. It migrates valid data blocks and log blocks according to the mapping table. Finally, it uses multi-layer verification to ensure the migration effect. This solves the problems of poor capacity adaptability, insufficient support for multi-target scenarios, and severe fragmentation in traditional block-level replication. It realizes intelligent partition scaling, cross-disk multi-target mapping, and fragmentation optimization, improving migration flexibility and efficiency.

[0015] 2. By parsing the source file system metadata, a new metadata template adapted to the target capacity is generated. The block writing sequence is reorganized according to the file continuity. It supports cross-disk parallel mapping and realizes unified processing of intelligent partition scaling, defragmentation and multi-target migration. It is a block-level partition copying technology that can adapt to capacity, support multiple targets and optimize fragmentation, providing efficient and reliable technical application support for cloud migration, storage upgrade and downgrade, disaster recovery and other scenarios. Regardless of whether the target partition is larger or smaller than the source partition, as long as its capacity is greater than or equal to the amount of data used, the file system can be automatically expanded or shrunk. This breaks through the limitation of traditional block-level copying that "capacity must be greater than or equal to the source partition", enabling elastic capacity migration and more flexibly handling various complex disk copying and migration scenarios, thus improving the migration success rate.

[0016] 3. Supports writing each partition of the source disk to different physical disks, ensuring file system consistency through metadata reorganization, ensuring data integrity and accessibility, supporting flexible mapping across multiple disks, and discarding the original LBA order during the copying stage, rearranging the block sequence according to the file logical order, eliminating target disk fragmentation, significantly improving random read and write performance, while reducing subsequent disk defragmentation overhead and improving defragmentation and performance; By addressing the block-level characteristics of different file systems and combining the management of effective block lists, fragmentation rates, and mapping tables, capacity adaptation, cross-disk multi-target mapping, and fragmentation optimization are achieved, solving the core pain points of traditional block-level replication. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0018] In the attached diagram: Figure 1 This is a schematic diagram of the overall process of the present invention; Figure 2 This is a schematic diagram illustrating the application scenario of cross-disk partition migration according to the present invention. Figure 3 This is the initial bitmap of the source disk partition of the present invention. Figure 4 This is a schematic diagram of the source-target mapping table of the present invention; Figure 5 This is the new bitmap of the target disk partition in this invention. Detailed Implementation

[0019] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0020] Example: Figure 1-5 As shown, this invention provides a technical solution that supports capacity-adaptive cross-disk partition block-level reorganization and migration method. It can adaptively handle capacity, support multiple targets, and optimize fragmentation in block-level partition copying. It involves block-level disk partition copying, migration, and optimization, and is suitable for data backup, recovery, and storage migration scenarios. Specifically, it includes the following steps: S1: Source disk information collection and preprocessing; S2: Target environment planning and mapping table construction; S3: Core structure transmission and target partition initialization; S4: Block-level data migration and log synchronization; S5: Migration verification and consistency check; S1 includes the following steps: S11: Source disk partition table scan and partition table structure construction; S12: Source file system metadata scanning and file system structure construction; S13: Initial bitmap construction of source disk partitions; S2 includes the following steps: S21: Target disk information scanning and capacity verification; S22: Multi-objective mapping rule; S23: Construction of new bitmap for target disk partition and source-target mapping table; S24: Update the file system structure; S3 includes the following steps: S31: Core structure is transferred to the target disk; S32: Target disk partition table construction; S33: Target disk file system construction; S4 includes the following steps: S41: Valid data block migration; S42: File log block migration; S5 includes the following steps: S51: Data integrity verification; S52: File system availability verification; S53: Capacity and fragmentation verification; S54: Exception rollback and report generation.

[0021] Based on the above technical solution, S1, by scanning the source disk partition table and file system metadata, constructs the partition table structure, file system structure and initial bitmap, so as to accurately locate the valid data to be migrated, and provide basic data support for subsequent mapping and migration; S2, based on the target disk capacity and distribution, plans mapping rules, constructs a new bitmap of the target disk partition and a source-target data block mapping table, and updates the file system structure to achieve the dual goals of capacity adaptation and defragmentation; S3 transfers the partition table structure, updated file system structure, and source-target mapping table of the source disk to the target disk, and initializes the target partition table and file system based on these structures to prepare for data migration. S4, based on the initial bitmap and mapping table, where the initial bitmap is used to filter valid blocks and the mapping table is the target address, migrates the source valid data blocks to the target partition and synchronizes log blocks to ensure data consistency; S5 uses multi-layered verification, including block-level comparison, file system verification, and performance testing, to ensure that the migration results meet expectations.

[0022] Based on the above technical solution, S11 specifically extracts key partition information and records it in the partition table structure by scanning the partition table of the source disk, including MBR and GPT. During migration, the partition table structure needs to be transferred to the target disk. The purpose of the partition table structure is to record the physical layout of each partition on the source disk, including its location, capacity, and type, serving as a blueprint for the creation of partitions on the target disk, and ensuring the logical relationship between the target partition and the source partition, including the distinction between the boot partition and the data partition. The partition table structure includes: identifying the partition table type, including GPT and MBR; identifying the disk's unique identifier, specifically the GUID of the GPT partition table and the disk serial number of the MBR partition table; identifying the partition type, specifically primary partition, extended partition, and logical partition; identifying partition information, specifically the partition start position, partition end position, and partition size; and identifying the file system type in the partition, including NTFS, EXT4, and XFS. S12: By scanning the metadata of the source partition file system, namely the index that records the association between files and data blocks, key information is extracted and recorded in the file system structure. During migration, the file system structure needs to be transferred to the target disk. The purpose of the file system structure is to record the core parameters of the file system and the file-data block mapping relationship. The core parameters include block size and metadata location, which are the basis for subsequent construction of mapping table and implementation of defragmentation, ensuring that the target file system can correctly identify data. The file system structure includes: identifying the file system type, block size, total number of blocks and number of valid data blocks; identifying the file name, path and size; identifying the file creation time, modification time and access time; identifying file permissions and access control; identifying the file owner and group; identifying the file type, including regular files, directories, symbolic links, pipes and sockets; identifying file flags; and identifying some file system-specific metadata, including inodes and inodes in XFS and EXT4, and the MFT and Master File Table in NTFS. S13 is a bitmap created based on the partition size and stored in binary file format. Each bit of the bitmap corresponds to a contiguous data block area of ​​the source partition, with an initial value of 0. Traverse all files in the file system, parse the file data block positions, and set the corresponding position in the bitmap to 1, indicating that the data block corresponding to this bit is valid data and needs to be migrated; The size of the data block corresponding to the bitmap varies depending on the file system. Specifically, the NTFS file system uses clusters as the smallest unit of storage and obtains the cluster size of NTFS as the smallest unit of bitmap. XFS and EXT4 use blocks as the basic unit of storage and obtain the block size from the superblock as the smallest unit of bitmap. The purpose of the initial bitmap of the source disk partition is to: intuitively distinguish between valid data blocks that need to be migrated and free blocks that can be discarded in the source partition, avoid redundant transmission, improve migration efficiency, and also serve as a filter for reading data blocks during migration transmission, that is, only reading blocks marked as 1.

[0023] Based on the above technical solution, S21, scan the physical parameters of the target disk. The target disk includes single disk and multiple disks. Record the total physical capacity, number of available blocks and single block size of each target disk to ensure block-level compatibility and verify whether its capacity meets the migration requirements. The capacity verification logic includes the following: Single-target scenario: The total number of available blocks on the target disk is greater than or equal to the total number of valid blocks on the source partition. Specifically, the number of available blocks on the target disk B (30GB) is greater than or equal to the number of valid blocks on the source disk A-partition 1 (20GB). Multi-target scenario: The number of available blocks on each target disk is greater than or equal to the number of valid blocks on the corresponding source partition. Specifically: Disk B 30GB ≥ 20GB of valid blocks in source disk A - partition 1, Disk C 100GB ≥ 50GB of valid blocks in source disk A - partition 2; S22 includes scenarios with multiple partitions on the same disk and scenarios across disks. In the scenario with multiple partitions on the same disk, the target space is allocated according to the order of the source partitions. This scenario includes partition 1 and partition 2 of source disk A to target disk B. The system partition is preferentially allocated to the front of the disk. The system partition includes NTFS to reduce seek time. The data partition is allocated to the back of the disk. The data partition includes EXT4. In practical applications, the scenario across disks includes partition 1 of source disk A to target disk B and partition 2 of source disk A to target disk C. The system partition can be allocated to SSD to improve boot speed, and large-capacity data partitions can be allocated to HDD to reduce costs. S23, based on the mapping rules and the order of the source files, construct a new target bitmap and a source-target data block mapping table to achieve fragmentation and capacity adaptation; The purpose of the new bitmap of the target disk partition is to mark the block 1 and free block 0 that have been allocated to the source valid blocks in the target partition, to ensure the reasonable use of the target space, and to provide a basis for the expansion and contraction of the file system. The purpose of the source-target mapping table is to record the correspondence between the source valid blocks and the target blocks, which is a roadmap for block-level migration. By rearranging the data blocks according to the file order, the target partition is defragmented, and the problem of fragment inheritance of the source partition is solved. S24. Based on the mapping table, update the file system structure in step S12, record the new data block location, file log data block location, file data block size, and file log data block size of the target file system, and ensure that the target file system can recognize the new association between the file and the data block, so as to provide an updated index for subsequent file system initialization.

[0024] Based on the above technical solution, S23, the specific construction process includes the following steps: S2301, Traverse the files in the source file system, traversing all files in directory hierarchy order, in the order of root directory → subdirectory → file; S2302, Data block rearrangement and mapping, specifically for each file, mapping its scattered source data blocks to target contiguous blocks in logical order, specifically: the data blocks of source file 1 are 5, 6, 7, 11 → target contiguous blocks 1, 2, 3, 4, the data blocks of source file 2 are 1, 2, 3, 4, 8 → target contiguous blocks 5, 6, 7, 8, 9; S2303, records the mapping relationship, wherein the mapping table content includes the start and end of the source logical block address, the start and end of the target logical block address, the file ID to which it belongs, and the block type, which includes data block, metadata block, and log block; S2304, Generate a new target bitmap. Specifically, based on the target block range of the mapping table, mark the corresponding position in the new bitmap as 1 and the rest as 0, and verify the continuity of the bitmap. S2305, persistently saves bitmaps and mapping tables.

[0025] Based on the above technical solution, S24 specifically includes the following implementation steps: S2401 creates a metadata structure for each file and directory in the target disk file system, sets basic attributes such as filename, path, size, and creation / modification / access time, sets security attributes such as file permissions, owner, group, and access control list (ACL), and sets file type and file flags. File types include regular files, directories, and symbolic links. For special files, their metadata must be set correctly according to the file system specification. Special files include symbolic links, pipes, and sockets. File system-specific extended attributes or custom metadata are written, such as NTFS extended attributes and EXT4 xattr. S2402, update the global metadata of the file system, including the superblock, root directory pointer, and free block bitmap, to ensure that the file system structure is complete and usable; S2403 verifies the correctness of metadata writing. This can be done by traversing the file system structure and comparing it with the source disk to ensure that there are no omissions or errors. S2404 persists all metadata to the metadata area of ​​the target disk's file system, ensuring data consistency in the event of power failure or other abnormal situations.

[0026] Based on the above technical solution, S31 is to transfer the partition table structure, the updated file system structure, and the source-target mapping table to the target disk. During the transfer process, the checksum is calculated to ensure integrity. The data is temporarily stored in the boot partition or memory buffer of the target disk. The boot partition is a temporary partition created by the LiveCD. After the target partition is initialized, the corresponding metadata area is written to it. S32 is based on a partition table structure. It creates a partition table on the target disk, defines the physical layout of the target partition, and ensures that the partition logic of the target disk is consistent with that of the source disk, including the distinction between the boot partition and the data partition, and provides physical space for file system initialization. S33, based on the updated file system structure, parses the file system structure information, including file name, path, size, timestamp, permissions, owner, file type, file flag, and specific metadata. It creates a file system of the same type as the source on the target partition, initializes the target file system metadata, and builds the basic framework of the target file system, including the superblock and inode table. At the same time, it records the file system structure construction log for easy traceability and troubleshooting.

[0027] Based on the above technical solution, S32 specifically includes the following implementation steps: S3201 scans the target environment, specifically single disk or multiple disks, and records the total physical capacity, number of available blocks, and size of each target disk. The size of each block is consistent with the source disk to ensure block-level compatibility. It confirms that the target disk is unpartitioned or uninitialized and ensures that there are no existing partition tables or partitions on the disk. S3202, verify capacity feasibility, including single-target scenario and multi-target scenario. In single-target scenario: the total number of available blocks on the target disk is greater than or equal to the total number of valid blocks on the source partition. Specifically, the number of available blocks on target disk B (30GB) is greater than or equal to the number of valid blocks on source disk A (partition 1) (20GB). In multi-target scenario: the number of available blocks on each target disk is greater than or equal to the number of valid blocks on the corresponding source partition. Specifically, disk B (30GB) is greater than or equal to the number of valid blocks on source disk A (partition 1) (20GB), and disk C (100GB) is greater than or equal to the number of valid blocks on source disk A (partition 2) (50GB). The S3203, based on the source disk's partition table type (including GPT and MBR), creates a new partition table structure on the target disk, sequentially allocating the starting and ending sectors of each partition to ensure partition alignment for performance improvement, including 4K alignment. Specifically, this includes the following: Target disk capacity = total capacity of source partition, i.e. no expansion or shrinkage scenario: create a partition table on the target disk with the same physical layout as the source partition according to the source partition structure, and match the total capacity of the source partition without adding or reducing space. When the target disk capacity is greater than the total capacity of the source partition, i.e., the extended space scenario, the starting LBA remains unchanged when creating the partition table on the target disk. The extended partition extends from the ending LBA to the range corresponding to the target capacity. The extended space is reserved in the target partition as free blocks, without changing the location of the valid data in the source partition. It is then enabled through the file system extension mechanism to achieve adaptive capacity expansion. The total capacity of the source partition > the capacity of the target disk > the amount of valid data in the source partition. In the shrinking scenario, when creating the partition table on the target disk, the range of the end LBA of the partition to the target capacity must be greater than the total LBA of the valid data in the source partition. The shrinking process only removes free blocks in the source partition that exceed the target capacity, ensuring that the capacity of the target partition completely covers the valid data in the source partition and does not affect data integrity. S3204 sets the partition type identifier, including EFI, Linux filesystem, and Windows basic data; sets the partition unique identifier, specifically a GUID; writes it to the partition table; and completes the creation of all partitions. Verify the partition table and partition structure to ensure that all partition information is correct, and record the partition creation process and result logs for easy traceability and troubleshooting.

[0028] Based on the above technical solution, S41, read the valid data block marked as 1 in the initial bitmap of the source partition, and write it to the corresponding position of the target according to the mapping table, including migration order and multi-threading optimization; The migration order ensures file system availability during the migration process. The basic structure of the file system is in place first, so in the event of an unexpected interruption, the recovery process can better locate and recover data based on the migrated metadata, maximizing the recoverability of the target file system. Among them, the metadata block, including the superblock, MFT, and inode table, ensures the priority availability of the file system structure; the boot block, sector 0 of the system partition, ensures boot functionality; the directory file block ensures the integrity of the directory structure; and the ordinary file block is migrated in file order, consistent with the mapping table. For single-target scenarios, multi-threading optimization enables multi-threading and migration is performed in segments according to the target block range of the mapping table. Specifically, thread 1 processes target blocks 1-10,000, and thread 2 processes blocks 10,000+10,000-20,000. Block conflicts are avoided by using a distributed lock, which is a hash lock based on the target block address. For multi-target scenarios across disks, multiple threads are started to handle the migration of different partitions to different target disks. The threads synchronize their progress through shared memory. S42, migrate the file system log blocks, record uncommitted transactions, read the valid file log data blocks according to the initial bitmap, obtain the mapped file log data block positions from the mapping table, and migrate the file log data blocks to the target disk partition to ensure data consistency; Log blocks record incomplete file operations on the source partition, including cached data that has not been written to disk. Synchronization after migration can prevent data loss or inconsistency, specifically transaction rollback during power outage recovery.

[0029] Based on the above technical solution, S51 specifically includes block-level comparison and file-level sampling verification; Block-level comparison involves traversing all blocks in the target partition, calculating their hash values, and comparing them with the hash values ​​of the valid block list in the source partition to ensure that all valid blocks are migrated completely without omissions or tampering. File-level sampling verification involves randomly selecting a certain percentage of files, prioritizing large files and system files (20%), comparing the target file with the source file in terms of size, modification time, and content, and ensuring content consistency through comparison of the complete files concatenated at the block level. S52 specifically includes verification through testing, including mount testing, read / write testing, and boot testing; Mounting tests involve attempting to mount the target partition to the test environment to check for file system corruption warnings, specifically metadata errors. If such errors occur, they are repaired using the file system's built-in repair tools. The read / write test involves creating new files, writing data, and deleting files on the target partition to verify that the file system can perform block-level allocation and release operations normally, and that the metadata is updated synchronously after the operation, specifically that the number of free blocks is reduced correctly. The boot test mainly targets the boot partition. The target partition is loaded through a virtual machine to verify whether the operating system can boot normally to the login screen and whether the core programs can run normally. The core programs include browsers and office software. S53 includes capacity adaptation and fragmentation rate verification. Capacity adaptation involves checking the target partition capacity using df and disk properties to confirm consistency with the plan, specifically shrinking to 30GB or expanding to 170GB. Fragmentation rate verification uses dedicated tools to detect the fragmentation rate of the target partition. Specifically, contig is used for NTFS, e4defrag-c is used for EXT4, and xfs_db is used for XFS. The fragmentation rate results are stored as evidence for performance optimization. S54 includes a rollback mechanism and a migration report generation. The rollback mechanism is automatically triggered when verification fails, including when critical blocks are lost or the system fails to start. It deletes the target partition data, restores the target partition snapshot before migration, automatically creates it before migration, or re-migrates based on the list of valid blocks and the mapping table. The generated migration report mainly records key indicators such as total migration time, number of migrated blocks, data volume, fragmentation rate optimization, and IO performance improvement, providing a reference for subsequent migrations.

[0030] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for cross-disk partition block level reorganization migration with capacity adaptation, characterized in that: The block-level partition replication capable of self-adapting capacity, supporting multiple targets and optimizing fragmentation involves block-level disk partition replication, migration and optimization, and is suitable for use scenarios of data backup recovery and storage migration, and specifically comprises the following steps: S1: source disk information acquisition and preprocessing; S2: target environment planning and mapping table construction; S3: core structure transmission and target partition initialization; S4: block-level data migration and log synchronization; S5: migration verification and consistency check; The S1 comprises the following steps: S11: source disk partition table scanning and partition table structure construction; S12: source file system metadata scanning and file system structure construction; S13: source disk partition initial bitmap construction; The S2 comprises the following steps: S21: target disk information scanning and capacity verification; S22: multiple target mapping rules; S23: target disk partition new bitmap and source-target mapping table construction; S24: update file system structure; The S3 comprises the following steps: S31: core structure transmission to target disk; S32: target disk partition table construction; S33: target disk file system construction; The S4 comprises the following steps: S41: effective data block migration; S42: file log block migration; The S5 comprises the following steps: S51: data integrity verification; S52: file system availability verification; S53: capacity and fragmentation verification; S54: exception rollback and report generation.

2. The method of claim 1, wherein: The S1, by scanning the source disk partition table and file system metadata, constructs the partition table structure, file system structure and initial bitmap to lock the effective data to be migrated, and provides basic data support for subsequent mapping and migration; The S2, based on the target disk capacity and distribution, plans the mapping rules, constructs the target disk partition new bitmap, source-target data block mapping table and updates the file system structure; The S3 is to transmit the partition table structure of the source disk, the updated file system structure and the source-target mapping table to the target disk, initialize the target partition table and file system based on these structures, and prepare for data migration; The S4, based on the initial bitmap and the mapping table, migrates the source effective data block to the target partition, and synchronizes the log block to ensure data consistency; The S5 is to ensure that the migration effect meets the expectation through multi-layer verification of block-level comparison, file system check and performance test.

3. The method of claim 1, wherein: The S11 specifically extracts the partition key information by scanning the partition table of the source disk and records it in the partition table structure, and the partition table structure needs to be transmitted to the target disk during migration; The S12 extracts the key information by scanning the metadata of the source partition file system and records it in the file system structure, and the file system structure needs to be transmitted to the target disk during migration; The S13 is to create a bitmap according to the partition size, which is stored in the form of a binary file, and each bit of the bitmap corresponds to a continuous data block area of the source partition, and the initial value is 0; All files in the file system are traversed, the file data block position is parsed, and the corresponding position in the bitmap is set to 1, indicating that the data block corresponding to this bit is valid data; The bitmap corresponding data block size is set according to different file systems.

4. The method of claim 3, wherein: The S21 scans the physical parameters of the target disk, records the total physical capacity, the number of available blocks, and the size of a single block of each target disk, and verifies whether the capacity meets the migration requirements; The capacity verification logic includes the following: Single-target scenario: the total number of available blocks of the target disk is greater than the total number of effective blocks of the source partition; Multi-target scenario: the number of available blocks of each target disk is greater than the number of effective blocks of the corresponding source partition.

5. The method of claim 4, wherein: The S23 includes the following steps in the specific construction process: S2301, traverse the files of the source file system in directory hierarchy order; S2302, data block rearrangement and mapping, specifically for each file, map the dispersed source data blocks to target contiguous blocks in logical order; S2303, record the mapping relationship, where the mapping table content includes the source logical block address start and end, the target logical block address start and end, the file ID, and the block type; S2304, generate a new target bitmap, specifically mark the corresponding position as 1 in the new bitmap according to the target block range of the mapping table, and verify the bitmap continuity; S2305, persistently save the bitmap and mapping table.

6. The method of claim 4, wherein: The S24 includes the following implementation steps: S2401, create metadata structures for each file and directory in the target disk file system one by one, set the file name, path, size, creation / modification / access, and basic attributes of the target file system construction time, set the security attributes of file permissions, owner, group, access control list, set the file type and file flag; For special files, the metadata should be correctly set according to the file system specification, and the file system specific extended attributes or custom metadata should be written; S2402, update the global metadata of the file system to ensure the integrity and availability of the file system structure; S2403, verify the correctness of the metadata writing, which can be compared with the source disk by traversing the file system structure; S2404, persistently save all metadata to the metadata area of the target disk file system.

7. The method of claim 1, wherein: The S31 is to transmit the partition table structure, the updated file system structure, and the source-target mapping table to the target disk, calculate the checksum to ensure integrity during transmission, and temporarily store it in the boot partition or memory buffer of the target disk. After the target partition is initialized, it is written to the corresponding metadata area; The S32 is to create a partition table based on the partition table structure, define the physical layout of the target partition, and ensure that the partition logic of the target disk is consistent with that of the source disk, providing physical space for file system initialization; The S33 is based on the updated file system structure to analyze the file system structure information, create the same type of file system as the source end in the target partition, initialize the target file system metadata, and build the basic framework of the target file system, while recording the file system structure construction log.

8. The method of claim 7, wherein: The S32 includes the following implementation steps: S3201, scan the target environment, record the total physical capacity, the number of available blocks, and the size of a single block of each target disk, confirm that the target disk is in an unpartitioned or uninitialized state, and ensure that there is no existing partition table and partition on the disk; S3202, verify capacity feasibility, including single-target scenario and multi-target scenario. In single-target scenario: the total number of available blocks on the target disk is greater than or equal to the total number of valid blocks on the source partition. In multi-target scenario: the number of available blocks on each target disk is greater than or equal to the number of valid blocks on the corresponding source partition. S3203 creates a new partition table structure on the target disk based on the source disk's partition table type, and allocates the starting and ending sectors of each partition sequentially to ensure partition alignment for improved performance. This includes the following specific scenarios: Target disk capacity = source partition total capacity: Create a partition table on the target disk with the same physical layout as the source partition according to the source partition structure, and match the total capacity of the source partition without adding or reducing space. Target disk capacity > source partition total capacity: When creating the partition table on the target disk, the starting LBA remains unchanged, the extended partition extends to the range corresponding to the target capacity from the ending LBA, and the extended space is reserved in the target partition as free blocks. It is then enabled through the file system extension mechanism to achieve adaptive capacity expansion. Source partition total capacity > target disk capacity > source partition effective data volume: When creating the partition table on the target disk, adjust the end LBA of the partition to the range corresponding to the target capacity. The shrinking process only removes free blocks in the source partition that exceed the target capacity, ensuring that the target partition capacity completely covers the effective data of the source partition without affecting data integrity. S3204 sets the partition type identifier, sets the partition unique identifier, writes to the partition table, and completes the creation of all partitions.

9. The method of claim 1, wherein: In step S41, the valid data blocks marked as 1 in the initial bitmap of the source partition are read and written to the corresponding positions of the target according to the mapping table, including migration order and multi-threading optimization. The migration order ensures that the basic structure of the file system is in place first during the migration process. In the event of an unexpected interruption, the recovery process can better locate and recover data based on the migrated metadata, maximizing the recoverability of the target file system. For single-target scenarios, multi-threading optimization enables multi-threading, and the migration is performed in segments according to the target block range of the mapping table. Block conflicts are avoided by using distributed locks. For multi-target scenarios across disks, multiple threads are started to handle the migration of different partitions to different target disks respectively. The threads synchronize their progress through shared memory. In step S42, the log blocks of the file system are migrated. The valid data blocks of the file log are read according to the initial bitmap, the location of the mapped file log data blocks is obtained from the mapping table, and the file log data blocks are migrated to the target disk partition to ensure data consistency. Log blocks record incomplete file operations in the source partition; synchronization after migration can prevent data loss or inconsistency.

10. The method of claim 1, wherein: S51 specifically includes block-level comparison and file-level sampling verification; S52 specifically includes verification through testing, including mount testing, read / write testing, and startup testing. Mounting tests involve attempting to mount the target partition to the test environment to check for any file system corruption warnings. If a corruption warning appears, it is repaired using the file system's built-in repair tools. The read / write test involves creating new files, writing data, and deleting files on the target partition to verify that the file system can perform block-level allocation and release operations normally, and that the metadata is updated synchronously after the operation. The boot test mainly targets the boot partition. The target partition is loaded through a virtual machine to verify whether the operating system can boot normally to the login screen and whether the core programs can run normally. The S53 includes capacity adaptation and fragmentation rate verification. Capacity adaptation is achieved by using df and disk properties to check the target partition capacity and confirm that it is consistent with the plan. Fragmentation rate verification is achieved by using a dedicated tool to detect the fragmentation rate of the target partition and storing the fragmentation rate result as evidence for performance optimization. S54 includes a rollback mechanism and a migration report generation. The rollback mechanism automatically triggers a rollback when verification fails, deletes the target partition data, restores the target partition snapshot before migration, or re-migrates based on the list of valid blocks and the mapping table.

Images