A method and system for migrating logical units in heterogeneous storage environments

By sending low-priority probe requests to obtain device status in a heterogeneous storage environment, dynamically scheduling the migration of cold data and performing incremental synchronization, the problems of business latency and data consistency during the migration process are solved, and the migration of logical units is achieved without the user's awareness.

CN123114378APending Publication Date: 2026-11-17ZIGUANG HENGYUE TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611408667.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-11
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing logical unit migration methods cannot effectively reduce business input/output latency during migration in heterogeneous storage environments, and it is difficult to guarantee data consistency and business continuity. In particular, resource contention intensifies latency when background tasks are executed, and existing technologies cannot adapt to storage devices with different interface protocols.

Method used

By sending low-priority probe requests to heterogeneous storage devices, the internal operating status characteristics of the devices are obtained. The migration is dynamically scheduled based on the access frequency of the source end and the resource status of the target end. Cold data is migrated first, and incremental synchronization is performed by tracking host write operations during the migration. Finally, the path is switched to reduce the interference of migration I/O on business I/O and ensure data consistency.

Benefits of technology

It enables seamless migration in heterogeneous storage environments, reduces interference with business input and output during migration, ensures data consistency and business continuity, and improves the reliability and efficiency of migration.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123114378A_ABST
    Figure CN123114378A_ABST
Patent Text Reader

Abstract

This invention discloses a method and system for migrating logical units in heterogeneous storage environments, belonging to the field of data storage technology. It includes: dividing the logical unit to be migrated into multiple data blocks and establishing status records; acquiring the operating status information of the source and target storage devices and the access information of the input / output paths for the data blocks, and determining the migration priority of each data block; determining whether the data migration conditions are met based on the operating status information; if the conditions are met, selecting data blocks according to priority to migrate from the source storage device to the target storage device and updating the status records; during the migration, tracking data modifications to the source storage device and synchronizing them to the target storage device, keeping the logical access interface unchanged; and updating the logical address mapping after confirming data consistency, switching the input / output paths to the target storage device. This application can reduce the interference of migrating input / output on business input / output.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of data storage technology, and specifically discloses a method and system for migrating logical units in heterogeneous storage environments. Background Technology

[0002] The existing logical unit migration process generally includes three stages: full copy, incremental synchronization, and mapping switch. During the full copy and incremental synchronization stages, the migration task and the host's business input and output share the bandwidth and processing capacity of the source and target storage devices. If the migration rate is not properly controlled, it will lead to increased business input and output latency, especially worsening tail latency.

[0003] Existing technologies typically employ fixed time windows, fixed rate limits, or migration sequence control based on hot / cold data identification. Fixed time windows or fixed rate limits do not consider the actual running status of background tasks within the storage device. When background tasks such as garbage collection and wear leveling are performed within the device, fixed-rate migration input / output will compete with these background tasks for device resources, further exacerbating latency in business input / output. Migration sequence control based on hot / cold data identification usually only uses business access statistics from the source end, failing to utilize idle resource information within the target device or the correlation between the latency characteristics of the source and target storage devices. This can lead to potential conflicts between source-side read migration and business read operations, and between target-side write migration and business write operations, preventing truly seamless migration.

[0004] Furthermore, in heterogeneous storage environments, source and target storage devices differ in physical media type, interface protocol type, or control method. Existing migration mechanisms struggle to adapt to the detection methods and status characteristics of different interface protocols simultaneously, and also find it difficult to guarantee data consistency and business continuity during the migration process. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides a logical unit migration method and system for heterogeneous storage environments. By sending probe requests to heterogeneous storage devices to obtain externally observable response characteristics, identifying the idle state of the devices, and dynamically scheduling migration based on the source end access frequency and the target end resource status, cold data is prioritized for migration. During migration, host write operations are tracked for incremental synchronization, and finally, the path is switched to reduce the interference of migration I / O on business I / O and ensure data consistency.

[0006] To achieve the above-mentioned objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a method for migrating logical cells in heterogeneous storage environments, the method comprising: Divide the logical unit to be migrated into multiple data blocks and establish a status record to record the migration status of each data block; Obtain the running status information of the source storage object and the target storage object; obtain the access information of the input and output paths for the data block, and determine the migration priority of each data block based on the access information; Based on the running status information of the source storage object and the target storage object, determine whether the data migration conditions are currently met; When the data migration conditions are met, the data blocks to be migrated are selected according to the migration priority of the data blocks, the selected data blocks are migrated from the source storage object to the target storage object, and the corresponding status records are updated according to the migration results. During data block migration, data modifications to the source storage object are tracked, and data synchronization is performed on the target storage object based on the tracking results; wherein, during the data block migration and data synchronization, the logical access interface to the logical unit remains unchanged; After confirming that the data corresponding to the logical unit to be migrated is consistent in the target storage object, the logical address mapping in the input / output path is updated, so that the input / output path is switched from the source storage object to the target storage object.

[0007] Preferably, obtaining the operating status information of the source storage device and the target storage device includes: Send probe requests to the source storage device and the target storage device respectively; Receive response information from the source storage device and the target storage device in response to the probe request; The operating status characteristics of the corresponding storage device are determined based on the response information; The probe request uses a command that matches the interface protocol of the corresponding storage device, and sets the priority of the probe request to be lower than that of the business input / output request through the priority field, service level field or background task identifier in the interface protocol.

[0008] Preferably, sending probe requests to the source storage device and the target storage device respectively includes: A read probe request sent to the source storage device; and, A write probe request is sent to the target storage device; The write probe request is written to the probe area of ​​the target storage device, and the written probe data is released or discarded after the write probe request is completed. The probe area does not belong to the business data mapping area. The response information includes a response delay, which is determined based on the time difference between the completion time and the submission time of the probe request.

[0009] Preferably, determining the migration priority of each data block based on the access information includes: The activity characteristics of the data block are determined based on the most recent access time, historical access interval, and current access continuity. The data blocks are divided into multiple migration priority categories based on the activity characteristics; The lower the access activity of a data block, the lower the migration impact on host service input and output, and the higher the migration priority of the data block.

[0010] Preferably, determining whether the data migration conditions are currently met includes: Based on the respective operating status characteristics of the source storage device and the target storage device, determine whether the source storage device and the target storage device are in a state suitable for performing the migration operation; When both the source storage device and the target storage device are in a state suitable for performing a migration operation, multiple data blocks of migration priority categories are allowed to be migrated. When only one of the source storage device and the target storage device is in a state suitable for performing the migration operation, data blocks of a predetermined migration priority category are allowed to be migrated. When neither the source storage device nor the target storage device is in a state suitable for performing the migration operation, the data block migration is paused, and data modifications generated by the host continue to be tracked.

[0011] Preferably, migrating the selected data block from the source storage device to the target storage device includes: Obtain target storage resources from the target storage device for storing the selected data blocks; Establish the correspondence between the source logical address and the target logical address of the selected data block; Read the selected data block from the source storage device and write the read data block to the target storage resource corresponding to the target logical address; After the data block migration is successful and passes the consistency check, the target logical address is made valid as a migration result.

[0012] Preferably, performing a consistency check on the data block includes: First verification information is determined based on data read from the source storage device; The second verification information is determined based on the data written to the target storage device, the data read from the target storage device, or the verification information returned by the target storage device; Compare the first verification information and the second verification information; When the first verification information and the second verification information are consistent, the corresponding data block is determined to have been successfully migrated; when the first verification information and the second verification information are inconsistent, the migration of the corresponding data block is re-executed or the corresponding data block is marked as a migration anomaly.

[0013] Preferably, the data modification generated by the tracking host on the source storage device includes: Determine the data blocks involved in the host write request; Set a change flag for the data block modified by the host write request, and record the write version or write order of the data block; After the data block completes its initial migration, the corresponding data is read from the source storage device and written to the target storage device based on the data block with the change identifier. Once the data in the target storage device matches the corresponding data in the source storage device, the change flag of the data block is cleared. If the data block is modified again during data synchronization, the change identifier is retained or reset, and the latest version of the data block is synchronized in subsequent synchronization processes.

[0014] Preferably, the processing prior to updating the logical address mapping in the host input / output path includes: Suspend receiving new host write requests or freeze host write requests; Waiting for the submitted host write request to complete; Perform the final round of data synchronization; Confirm that there are no out-of-sync data modifications; Refresh the write caches associated with the source storage device and the target storage device; Update the source logical address mapping to the target logical address mapping, and restore host input / output requests after the target logical address mapping takes effect.

[0015] Secondly, the present invention provides a logic cell migration system for heterogeneous storage environments, comprising: The status management module is used to divide the logical units to be migrated into multiple data blocks and establish a status record to record the migration status of each data block. The running status acquisition module is used to acquire running status information of the source storage object and the target storage object; acquire access information of the input and output paths for the data block; and determine the migration priority of each data block based on the access information. The migration condition determination module is used to determine whether the data migration conditions are met based on the running status information of the source storage object and the target storage object. The data migration module is used to select data blocks to be migrated according to the migration priority of the data blocks when the data migration conditions are met, migrate the selected data blocks from the source storage object to the target storage object, and update the corresponding status records according to the migration results. The data synchronization module is used to track data modifications made to the source storage object during data block migration and to perform data synchronization on the target storage object based on the tracking results; wherein, during the data block migration and data synchronization, the logical access interface to the logical unit remains unchanged; The mapping switching module is used to update the logical address mapping in the input / output path after confirming that the data corresponding to the logical unit to be migrated is consistent in the target storage object, so that the input / output path is switched from the source storage object to the target storage object.

[0016] Compared with the closest existing technology, the present invention has the following advantages: This application proposes a logical unit migration method and system for heterogeneous storage environments. By sending low-priority probe requests to the source and target storage devices respectively, and determining the operating status characteristics based on the response delay of the probe requests, it can identify the real busy state caused by background tasks inside the storage device without accessing the internal firmware of the storage device, thereby avoiding performing migration operations when the device is busy.

[0017] This invention forms tiered migration conditions by jointly judging the operating status of the source storage device and the target storage device, avoiding misjudgments caused by rate control based solely on host service load or the status of a single device, and helping to reduce the interference of migration input / output on service input / output.

[0018] This invention extracts the access activity characteristics of data blocks and divides the migration priority accordingly, so that data blocks with low access activity are migrated first, which helps to reduce the conflict between migration input / output and business input / output in address space during the migration process.

[0019] This invention uses an online write tracking and incremental synchronization mechanism to continuously track data modifications made by the host to the source storage device during data block migration and synchronize them to the target storage device, thus ensuring data consistency during the migration process.

[0020] This invention ensures data consistency between the source and target storage devices during logical address mapping switching by pausing host write requests, performing the final round of synchronization, and refreshing the write cache before mapping switching, which is beneficial for achieving uninterrupted online migration of services.

[0021] This invention, through data block-level consistency verification, status recording, and retry mechanisms, enables the migration process to be recovered based on status records after a migration failure, which helps to improve the reliability of migration. Attached Figure Description

[0022] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0023] Figure 1 This is a flowchart of a logical unit migration method for heterogeneous storage environments provided by the present invention; Figure 2 This is a schematic diagram of the process for constructing a traceability graph provided by the present invention; Figure 3 This is a flowchart of the detection command acquisition and operation status information acquisition provided by the present invention; Figure 4 This is a flowchart of the logical address mapping switching process provided by the present invention; Figure 5 This is a schematic diagram of the logical unit migration system architecture for heterogeneous storage environments provided by the present invention; Figure 6 This is a structural diagram of a logic unit migration system for heterogeneous storage environments provided by the present invention; Figure 7 This is a diagram of the internal structure of the electronic device provided by the present invention. Detailed Implementation

[0024] The embodiments of the technical solution of the present invention will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of the present invention and are therefore merely examples, and should not be construed as limiting the scope of protection of the present invention.

[0025] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application should have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0026] This invention provides a method and system for migrating logical units in heterogeneous storage environments. The embodiments of this invention will be described below with reference to the accompanying drawings.

[0027] Example 1: As Figure 1 As shown, Embodiment 1 of this application provides a logical cell migration method for heterogeneous storage environments. This method specifically includes the following steps: S101, the logic unit to be migrated is divided into multiple data blocks, and a status record is established to record the migration status of each data block.

[0028] The migration scheduler obtains the capacity, block size, and alignment requirements of the logical units to be migrated, and divides them into multiple data blocks according to the minimum write granularity supported by the target storage device. The block size can be configured from 64 KiB to 8 MiB, with a default value of 1 MiB, and is aligned according to the logical block boundaries of the source and target devices. The selection of block size is based on factors including the minimum input / output granularity of the source and target storage devices, the average size of business input / output, the host cache capacity, the storage overhead of the dirty block bitmap, and the migration read / write command queue depth.

[0029] Each data block has a unique block identifier. The migration scheduler establishes a block status table as a status record, which includes at least: block identifier, source logical address, destination logical address or allocable resource range, data block length, migration status, dirty block status, checksum, number of retries, and last update time. The initial value of the migration status is "not migrated".

[0030] S102, obtain the operating status information of the source storage device and the target storage device; obtain the access information of the input / output path for the data block, and determine the migration priority of each data block according to the access information.

[0031] like Figure 2 As shown, the migration scheduler initiates probe command collection on both the source and target storage devices simultaneously with the start of the migration task. The probe commands are low-priority input / output commands conforming to their respective interface protocol specifications. The priority of the probe requests is set lower than the priority of business input / output requests through priority fields, service level fields, or background task identifiers in the interface protocol. The probe command collection period can be configured from 50 milliseconds to 1 second, with a default value of 100 milliseconds.

[0032] During each acquisition cycle, the migration scheduler sends a read probe request to the source storage device to read 4KiB of data; it also sends a write probe request to the target storage device to write 4KiB of data to the probe area of ​​the target storage device, and discards the data after completion. The probe area is isolated from the business data area and the migration target data area, and does not belong to the business data mapping area.

[0033] For each probe request, the migration scheduler records the command submission time, command completion time, device command queue depth at the submission time, command success status, and probe access address and type. The response latency equals the command completion time minus the command submission time. Based on the response information, the migration scheduler determines the corresponding storage device's operational status characteristics, including response latency statistics and queue depth changes.

[0034] When a probe request times out or fails, a retry is performed on the first failure; two consecutive failures mark the corresponding storage device as "unstable"; failed samples are not included in the latency percentile calculation, but the number of failures is recorded; migration is paused when the number of consecutive failures reaches a preset number; migration condition judgment is resumed after the device recovers and obtains sufficient valid samples again.

[0035] The migration scheduler maintains a sliding window, each containing response latency samples for the N most recent valid probe requests, with N defaulting to 50. When the number of valid samples is less than 30, a fully safe window check is not performed; only a conservative pause or low-speed state is allowed. Each time a new sample is collected, the oldest sample is removed, and statistical characteristics are recalculated.

[0036] Operational characteristics include response latency, right tail slope, and the number of events that increase queue depth.

[0037] The right-tail slope of the response delay is calculated as follows: The most recent N valid samples are sorted in ascending order, and the 90th percentile delay value P90, the 99th percentile delay value P99, and the 99.9th percentile delay value P99.9 are taken. The right-tail slope S is then calculated. tail =(P99.9 P99) / (P99 P90+ε); where N is the number of valid probe request response latency samples included in the sliding window, P90 is the 90th percentile of the response latency, P99 is the 99th percentile of the response latency, P99.9 is the 99.9th percentile of the response latency, and S tail The right tail slope of the response delay, ε is a positive number to prevent division by zero, with a default value of 1 microsecond. S tail The larger the value, the more severe the tail latency expansion caused by background tasks within the device.

[0038] The number of queue depth increase events is calculated as follows: Maintain the queue depth time series and record the queue depth value Q for each sampling point. i Perform first-order difference ΔQ on the queue depth sequence i =Q i Q i 1, where i is the index of the sampling point number, Q iLet ΔQ be the queue depth value for the i-th sampling point. i The first-order difference of the queue depth sequence is used; when the current first-order difference is positive and the first-order difference of the previous sampling point is also positive, a queue depth increase event is determined to have occurred, and the number N of increase events in the last 10 sampling periods is recorded. inc N inc This represents the number of events that caused the queue depth to increase within the last 10 sampling periods.

[0039] To avoid frequent start-stop cycles caused by a single abnormal sample, a hysteresis mechanism is adopted: a stricter threshold is used to enter the safety window, while a slightly more lenient threshold is used to maintain the safety window; if a serious timeout, device disconnection, or continuous queue growth occurs, the system immediately enters the prohibition window.

[0040] The migration scheduler collects access records for each data block along the host input / output path of the source storage device using a block-level tracking mechanism. Access records include access type, access start address, access length, access timestamp, and corresponding data block identifier. Migration input / output and probe input / output are not included in the statistics.

[0041] For each data block, the migration scheduler maintains the following statistics: recent access timestamp, historical access interval sequence, historical address step sequence, current consecutive access count, and dirty block status. The statistical period can be configured from 1 second to 30 seconds, with a default value of 5 seconds. Historical access records are stored in a fixed-length queue to limit memory usage.

[0042] Activity characteristics include the most recent access interval t last Historical access interval coefficient of variation C Vinterval and the current consecutive access count c run Recent visit interval t last The data block t that has never been accessed is the current time minus the most recent access timestamp. last Denoteed as infinity. The coefficient of variation C for historical access intervals. Vinterval The historical visit interval sequence is calculated by dividing the standard deviation by the mean; if the sample size is less than two, it is recorded as 0. The current consecutive visit count c. run This represents the number of consecutive accesses to the same data block. If other data blocks are accessed in between, the count is reset to zero.

[0043] The migration scheduler categorizes data blocks into multiple migration priority classes based on activity characteristics. The classification rule is: if t last <1 second and c run If C ≥ 2, it is considered an active block with a migration priority of 3; otherwise, if C VintervalIf the address value is greater than 1.2 and the median address step size is greater than 4 MiB, it is determined to be a wandering block with a migration priority of 2; otherwise, it is determined to be a quiet block with a migration priority of 1. Data blocks that have never been accessed are conservatively classified, with a priority no higher than that of confirmed quiet blocks. The smaller the priority value, the higher the migration priority, meaning the lower the access activity of the data block and the lower the migration impact on host service input and output, the higher the migration priority of the corresponding data block.

[0044] S103, determine whether the data migration conditions are met based on the operating status information of the source storage device and the target storage device.

[0045] The migration scheduler jointly determines whether the data migration conditions are met based on the operating status characteristics of the source and target storage devices.

[0046] The target storage device also needs to collect the proportion of available reserved space R. reserve And the current write amplification factor WAF. R reserve The WAF is obtained through the SMART attribute or equivalent interface of the target storage device; the WAF is calculated through the SMART attribute or the number of write commands recorded by the target storage device; WAFavg is the average write amplification factor of the target storage device since startup.

[0047] The first latency condition for the source storage device is: S tail ,source<2 andNinc,source≤1.

[0048] The second latency condition for the target storage device is: S tail ,target<3 andNinc,target≤1 andR reserve >20% and WAF <1.2·WAFavg.

[0049] When the source storage device meets the first latency condition and the target storage device meets the second latency condition, it is determined to be a fully safe window, allowing the migration of data blocks of multiple migration priority categories, that is, selecting the data blocks to be migrated in the order of quiet blocks, wandering blocks, and active blocks.

[0050] When only the source storage device or the target storage device meets the corresponding latency condition, and the absolute value of P99.9 of the source storage device does not exceed 2 milliseconds and the absolute value of P99.9 of the target storage device does not exceed 50 milliseconds, it is determined to be a conditional safety window, allowing the migration of data blocks of the predetermined migration priority category, that is, only quiet blocks are selected for migration.

[0051] Any situation that does not meet the full security window and conditional security window requirements, including P99.9 of the source storage device exceeding 2 milliseconds, P99.9 of the target storage device exceeding 50 milliseconds, Ninc of the source or target storage device being greater than or equal to 3, Rreserve of the target storage device being less than 5%, device failure, continuous probe failure, or severe timeout, is considered a prohibited window, data block migration is suspended, and data modifications generated by the host continue to be tracked.

[0052] The above thresholds can be configured through the device baseline. Before the migration begins, baseline latency and queue depth are collected under low load conditions, and then the upper limit of service tail latency is observed under service load to determine the migration thresholds.

[0053] S104, when the data migration conditions are met, select the data block to be migrated according to the migration priority of the data block, migrate the selected data block from the source storage device to the target storage device, and update the corresponding status record according to the migration result.

[0054] When the data migration conditions are met, the migration scheduler selects data blocks to be migrated based on their migration priority. Within the fully safe window, data blocks are selected in the order of quiet blocks, wandering blocks, and active blocks; within the conditionally safe window, only quiet blocks are selected.

[0055] After determining the migration window, the migration scheduler obtains allocatable resource information from the target storage device. This resource can be an allocatable logical address within the target logical unit, a free logical address range in the storage pool, a free physical disk range provided by the array controller, a resource bitmap provided by the target storage device driver or virtualization layer, or a target address range isolated from the business write area. For solid-state drives using a flash translation layer, the migration scheduler does not require direct access to flash physical page addresses; the target address resources are mapped to the underlying physical media by the target storage device controller or storage virtualization layer.

[0056] The migration scheduler reserves migration resources in the target resource bitmap and establishes a mapping between data block identifiers and target logical addresses to avoid multiple migration tasks being allocated the same resource. If no available resources exist on the target side, the migration is paused and periodically re-queried; if the target resources are insufficient for an extended period, the migration is reported as unable to continue, and the status of completed data blocks is retained.

[0057] For each selected data block, the migration scheduler performs the following operations: First, check that the data block has not been migrated and has not been marked as a current dirty block.

[0058] Second, read all the data of the data block from the source storage device into the host memory buffer, and set the migration read command to low input / output priority.

[0059] Third, calculate the verification value of the read data as the first verification information.

[0060] Fourth, data is written to the address resources allocated to the target end with low input / output priority.

[0061] Fifth, recalculate or read the checksum of the data written to the target device, and use it as the second checksum. The second checksum can be determined based on the data written to the target storage device, the data read from the target storage device, or the checksum returned by the target storage device.

[0062] Sixth, compare the first and second verification information. If they match, the data block migration is considered successful, making the target logical address valid as a migration result. Update the migration status of the data block in the block status table to "copied and awaiting synchronization" or "completed," and record the target address. If they do not match, release the target resource and re-migrate. After reaching the retry limit, mark the data block as a migration anomaly and pause the task.

[0063] Migration rate is controlled using a token bucket or equivalent rate limiting mechanism. The maximum rate for the fully secure window is determined by comprehensively considering the smaller of the available bandwidth at the source and destination ends, the tail latency of the service input / output destination, the current device queue depth, the host memory buffer capacity, and the write amplification factor at the destination end. When there is no dedicated bandwidth configuration for the device, the migration maximum rate can be set to a preset ratio of the smaller of the measured sequential write bandwidth at the destination end and the measured read bandwidth at the source end.

[0064] S105, during data block migration, data modifications generated for the source storage device are tracked, and data synchronization is performed on the target storage device based on the tracking results; wherein, during the data block migration and data synchronization, the logical access interface to the logical unit remains unchanged.

[0065] like Figure 3 As shown, to ensure data consistency during online migration, write tracking is initiated before full replication begins. When the host performs a write operation on the source logical unit, the migration scheduler marks the corresponding data block as a dirty block based on the write logical address and length, and records it in the dirty block bitmap or migration log.

[0066] Write tracking can be performed in any of the following ways: copying host write requests at the storage virtualization layer so that write requests are written to both the source and target ends simultaneously; continuing to provide business writes at the source end and recording the corresponding logical address, and subsequently rereading dirty blocks from the source end and synchronizing them to the target end; or using a snapshot or copy-on-write mechanism to save the source end data view at the start of full replication.

[0067] The migration scheduler determines the data blocks involved in the host write request, sets a change flag for the data blocks modified by the host write request, and records the write version or write order of the data blocks.

[0068] After the initial migration of data blocks, the migration scheduler reads the corresponding data from the source storage device and writes it to the target storage device based on the data blocks with change flags. Once the data in the target storage device matches the corresponding data in the source storage device, the change flag for that data block is cleared.

[0069] If the data block is modified again during data synchronization, the change flag is retained or reset, and the latest version of the data block is synchronized in subsequent synchronization processes.

[0070] After full replication is complete, the migration scheduler performs incremental synchronization based on the dirty block bitmap. New dirty blocks are continuously recorded during incremental synchronization. Mapping switching is only allowed when the number of dirty blocks falls below a preset threshold in at least two consecutive rounds of incremental scans, or when business writes are briefly frozen and the last round of synchronization is completed.

[0071] During data block migration and data synchronization, the logical access interface to the logical unit remains unchanged. The host's business input and output continue to access the source storage device through the logical access interface, and the migration operation is transparent to the host's business.

[0072] S106, after confirming that the data corresponding to the logical unit to be migrated is consistent in the target storage device, update the logical address mapping in the input / output path so that the input / output path is switched from the source storage device to the target storage device.

[0073] like Figure 4 As shown, when all data blocks to be synchronized have been verified and no new dirty blocks have been generated in two consecutive rounds of synchronization or the number of dirty blocks is less than the threshold, and when neither the source storage device nor the target storage device is in the forbidden window and the target cache is successfully refreshed, the migration scheduler enters the switching preparation phase.

[0074] Before updating the logical address mappings in the input / output paths, the migration scheduler performs the following processing: First, suspend receiving new host write requests or freeze host write requests.

[0075] Second, wait for the submitted host write request to complete.

[0076] Third, perform the final round of data synchronization.

[0077] Fourth, confirm that there are no unsynchronized data modifications.

[0078] Fifth, send cache refresh commands to the source and target storage devices to refresh the write caches associated with them.

[0079] After completing the above processing, the migration scheduler updates the source logical address mapping to the target logical address mapping, switching the input / output path from the source storage device to the target storage device, and performs readability verification on the target storage device. Host input / output requests are restored after the target logical address mapping takes effect.

[0080] After a successful switchover, the source data is protected for a configurable period of time. Once the target business input and output are confirmed to be normal, the source logical unit resources are released. In the event of a switchover failure, the source mapping is restored, and incremental synchronization continues according to the block status table.

[0081] Example 2: Based on the same technical concept, Example 2 provides a logical unit migration system for heterogeneous storage environments. like Figure 5 As shown, in terms of hardware, the logical unit migration system for heterogeneous storage environments provided in this embodiment includes a source storage device 1, a target storage device 2, a migration scheduler 3, a block status table 4, a write tracking module 5, and a storage virtualization layer 6. The host 7 accesses the logical units through the logical access interface provided by the storage virtualization layer 6. The source storage device 1 and the target storage device 2 differ in physical media type, interface protocol type, or control method, constituting a heterogeneous storage system. The migration scheduler 3 is deployed on the host 7, the storage virtualization layer 6, or the control node where the multipath software resides.

[0082] The system provided above is used to execute the logic unit migration method described in Embodiment 1. In terms of software, such as... Figure 6 As shown, the system includes a status management module 301, a running status acquisition module 302, a migration condition determination module 303, a data migration module 304, a data synchronization module 305, and a mapping switching module 306. All these modules are integrated into a migration scheduler 3, which is deployed on the host 7, the storage virtualization layer 6, or the control node where the multipath software resides. The modules interact with each other via an internal bus or application programming interface. The block status table 4 is shared and accessed by the data migration module 304, the data synchronization module 305, and the mapping switching module 306. Each module updates the status field of the corresponding data block in the block status table 4 based on its own operation results.

[0083] The state management module 301 receives capacity information, logical block size information, and alignment requirement information of the logical unit to be migrated at its input end, and its output end is connected to the block status table 4. The state management module 301 is used to divide the logical unit to be migrated into multiple data blocks and establish a status record to record the migration status of each data block. Specifically, the state management module 301 divides the data blocks according to the smallest write granularity supported by the target storage device 2, assigns a unique block identifier to each data block, and establishes the block status table 4. The block status table 4 includes at least the block identifier, source logical address, target logical address or allocable resource range, data block length, migration status, dirty block status, checksum, number of retries, and last update time. The state management module 301 sets the initial value of the migration status to "not migrated".

[0084] The first input terminal of the operation status acquisition module 302 is connected to the source storage device 1, the second input terminal is connected to the target storage device 2, the third input terminal is connected to the input / output path of the host 7, and the output terminal is connected to the migration condition determination module 303 and the data migration module 304. The operation status acquisition module 302 is used to acquire the operation status information of the source storage device 1 and the target storage device 2, acquire the access information of the input / output path for data blocks, and determine the migration priority of each data block based on the access information. Specifically, the operation status acquisition module 302 sends probe requests to the source storage device 1 and the target storage device 2 respectively, receives the response information returned by both in response to the probe requests, and determines the operation status characteristics of the corresponding storage devices based on the response information. The probe requests use commands that match the interface protocol of the corresponding storage devices, and the priority of the probe requests is set to be lower than the priority of the business input / output requests through the priority field, service level field, or background task identifier in the interface protocol. The probe request sent to source storage device 1 is a read probe request, and the probe request sent to target storage device 2 is a write probe request. The write probe request writes to the probe area of ​​target storage device 2 and releases or discards the probe data after completion. The probe area does not belong to the business data mapping area. The response information includes the response latency, which is determined based on the time difference between the completion time and the submission time of the probe request. The running status acquisition module 302 also collects the access records of each data block on the host input / output path of source storage device 1. Based on the most recent access time, historical access interval, and current access continuity of the data block, it determines the activity characteristics of the data block and then divides the data block into multiple migration priority categories based on the activity characteristics. The lower the access activity of the data block, the higher its migration priority.

[0085] The input of the migration condition determination module 303 is connected to the output of the running status acquisition module 302, and the output is connected to the data migration module 304. The migration condition determination module 303 is used to determine whether the data migration conditions are met based on the running status information of the source storage device 1 and the target storage device 2. Specifically, the migration condition determination module 303 determines whether the source storage device 1 and the target storage device 2 are in a suitable state for performing the migration operation based on their respective running status characteristics. When both are in a suitable state for performing the migration operation, multiple data blocks of different migration priority categories are allowed to be migrated; when only one storage device is in a suitable state for performing the migration operation, data blocks of a predetermined migration priority category are allowed to be migrated; when neither is in a suitable state for performing the migration operation, data block migration is suspended, and the data synchronization module 305 is notified to continue tracking data modifications generated by the host.

[0086] The data migration module 304 has its first input connected to the output of the migration condition determination module 303, its second input connected to the output of the running status acquisition module 302, its third input connected to the source storage device 1, and its output connected to the target storage device 2 and the block status table 4. When the data migration conditions are met, the data migration module 304 selects the data blocks to be migrated based on their migration priority, migrates the selected data blocks from the source storage device 1 to the target storage device 2, and updates the corresponding status records based on the migration results. Specifically, the data migration module 304 obtains the target storage resources from the target storage device 2 to store the selected data blocks, establishes a correspondence between the source logical address and the target logical address of the selected data blocks, reads the selected data blocks from the source storage device 1 and writes them to the target storage resources corresponding to the target logical address, and after the data block migration is successful and passes the consistency check, makes the target logical address valid as a migration result and updates the migration status of the corresponding data block in the block status table 4. Consistency verification is performed by the data migration module 304: First verification information is determined based on the data read from the source storage device 1, and second verification information is determined based on the data written to the target storage device 2, the data read from the target storage device 2, or the verification information returned by the target storage device 2. The first verification information and the second verification information are compared. If they are consistent, the corresponding data block is determined to have been successfully migrated. If they are inconsistent, the migration of the corresponding data block is re-executed or the corresponding data block is marked as a migration anomaly.

[0087] The data synchronization module 305 has its first input connected to the source storage device 1, its second input connected to the write request path of the host 7, and its output connected to the target storage device 2 and the block status table 4. The data synchronization module 305 tracks data modifications made to the source storage device 1 during data block migration and performs data synchronization on the target storage device 2 based on the tracking results. During data block migration and data synchronization, the data synchronization module 305 maintains the logical access interface to the logical unit unchanged. Specifically, the data synchronization module 305 identifies the data blocks involved in the host write request, sets a change flag for the data blocks modified by the host write request, and records the write version or write order of the data blocks. After the initial migration of the data blocks, it reads the corresponding data from the source storage device 1 and writes it to the target storage device 2 based on the data blocks with change flags. After the data in the target storage device 2 matches the corresponding data in the source storage device 1, the change flag of the data block is cleared. If the data block is modified again during data synchronization, the change flag is retained or reset, and the latest version of the data block is synchronized in subsequent synchronization processes.

[0088] The mapping switching module 306 has its first input connected to the output of the data synchronization module 305, its second input connected to the source storage device 1, its third input connected to the target storage device 2, and its output connected to the logical address mapping table of the storage virtualization layer 6. The mapping switching module 306 updates the logical address mapping in the input / output path after confirming that the data corresponding to the logical unit to be migrated is consistent in the target storage device 2, thus switching the input / output path from the source storage device 1 to the target storage device 2. Specifically, before updating the logical address mapping, the mapping switching module 306 suspends receiving new host write requests or freezes host write requests, waits for submitted host write requests to complete, performs the last round of data synchronization, confirms that there are no unsynchronized data modifications, refreshes the write caches related to the source storage device 1 and the target storage device 2, then updates the source logical address mapping to the target logical address mapping, and resumes host input / output requests after the target logical address mapping takes effect.

[0089] In one embodiment, the implementation method in a storage array scenario is as follows: When the source storage device 1 or the target storage device 2 is a storage array, the operating status acquisition module 302 can collect probe samples according to the controller, member devices, or input / output paths, and calculate the Stail, Ninc, and P99.9 of each member device. The array security status adopts the worst-case device status, or is calculated according to the weighted status of the path where the current logical unit is located. If any critical path goes offline, has insufficient resources, or experiences a severe timeout, the system enters a prohibited window.

[0090] Specific application examples: Taking the migration of a logical unit where source storage device 1 is an NVMe solid-state drive and target storage device 2 is a SATA solid-state drive as an example, the migration scheduler 3 is deployed on the storage virtualization layer 6 of host 7.

[0091] Before the migration begins, the data synchronization module 305 initiates write tracing, fixing the full copy view through copy-on-write or snapshot. The status management module 301 divides the logical unit into 1MiB data blocks and establishes the block status table 4.

[0092] The running status acquisition module 302 sends a probe command every 100 milliseconds, maintains the 50 most recent valid delay samples, and counts the queue depth increase events in the last 10 sampling periods. Every 5 seconds, it calculates and classifies the data block activity inertia characteristics.

[0093] The migration condition determination module 303 jointly determines the safe migration window based on the delay conditions of the source and target ends.

[0094] Within the fully secure window, the data migration module 304 reserves an idle logical address from the target end resource bitmap, prioritizes quiet blocks to perform low-priority migration reads and writes, and performs verification.

[0095] The host writes generated during full replication are recorded as dirty blocks by the data synchronization module 305. After full replication is completed, the data synchronization module 305 repeatedly performs incremental synchronization until the number of dirty blocks reaches the switching threshold.

[0096] The mapping switching module 306 then briefly pauses host input and output, completes final synchronization and cache refresh, updates logical unit mapping, and releases source resources after successful verification at the target end.

[0097] Example 3: In one embodiment, Example 3 of the present invention also provides an electronic device; the electronic device may be a terminal, and its internal structure diagram may be as follows. Figure 7As shown. The electronic device includes a processor, memory, communication interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements the logic unit migration method for heterogeneous storage environments as described in any one of steps S101 to S106. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.

[0098] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0099] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0100] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0101] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0102] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0103] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of the claims of the present invention pending approval.

Claims

1. A method for migrating logical units in heterogeneous storage environments, characterized in that, The method includes: Divide the logical unit to be migrated into multiple data blocks and establish a status record to record the migration status of each data block; Obtain the running status information of the source storage object and the target storage object; obtain the access information of the input and output paths for the data block, and determine the migration priority of each data block based on the access information; Based on the running status information of the source storage object and the target storage object, determine whether the data migration conditions are currently met; When the data migration conditions are met, the data blocks to be migrated are selected according to the migration priority of the data blocks, the selected data blocks are migrated from the source storage object to the target storage object, and the corresponding status records are updated according to the migration results. During data block migration, data modifications to the source storage object are tracked, and data synchronization is performed on the target storage object based on the tracking results; wherein, during the data block migration and data synchronization, the logical access interface to the logical unit remains unchanged; After confirming that the data corresponding to the logical unit to be migrated is consistent in the target storage object, the logical address mapping in the input / output path is updated, so that the input / output path is switched from the source storage object to the target storage object.

2. The method according to claim 1, characterized in that, The step of obtaining the operating status information of the source storage device and the target storage device includes: Send probe requests to the source storage device and the target storage device respectively; Receive response information from the source storage device and the target storage device in response to the probe request; The operating status characteristics of the corresponding storage device are determined based on the response information; The probe request uses a command that matches the interface protocol of the corresponding storage device, and sets the priority of the probe request to be lower than that of the business input / output request through the priority field, service level field or background task identifier in the interface protocol.

3. The method according to claim 2, characterized in that, Sending probe requests to the source storage device and the target storage device respectively includes: A read probe request sent to the source storage device; and, A write probe request is sent to the target storage device; The write probe request is written to the probe area of ​​the target storage device, and the written probe data is released or discarded after the write probe request is completed. The probe area does not belong to the business data mapping area. The response information includes a response delay, which is determined based on the time difference between the completion time and the submission time of the probe request.

4. The method according to claim 1, characterized in that, The step of determining the migration priority of each data block based on the access information includes: The activity characteristics of the data block are determined based on the most recent access time, historical access interval, and current access continuity. The data blocks are divided into multiple migration priority categories based on the activity characteristics; The lower the access activity of a data block, the lower the migration impact on host service input and output, and the higher the migration priority of the data block.

5. The method according to claim 1, characterized in that, Determining whether the data migration conditions are currently met includes: Based on the respective operating status characteristics of the source storage device and the target storage device, determine whether the source storage device and the target storage device are in a state suitable for performing the migration operation; When both the source storage device and the target storage device are in a state suitable for performing a migration operation, multiple data blocks of migration priority categories are allowed to be migrated. When only one of the source storage device and the target storage device is in a state suitable for performing the migration operation, data blocks of a predetermined migration priority category are allowed to be migrated. When neither the source storage device nor the target storage device is in a state suitable for performing the migration operation, the data block migration is paused, and data modifications generated by the host continue to be tracked.

6. The method according to claim 1, characterized in that, The step of migrating the selected data block from the source storage device to the target storage device includes: Obtain target storage resources from the target storage device for storing the selected data blocks; Establish the correspondence between the source logical address and the target logical address of the selected data block; Read the selected data block from the source storage device and write the read data block to the target storage resource corresponding to the target logical address; After the data block migration is successful and passes the consistency check, the target logical address is made valid as a migration result.

7. The method according to claim 6, characterized in that, The process of performing a consistency check on the data block includes: First verification information is determined based on data read from the source storage device; The second verification information is determined based on the data written to the target storage device, the data read from the target storage device, or the verification information returned by the target storage device; Compare the first verification information and the second verification information; When the first verification information and the second verification information are consistent, the corresponding data block is determined to have been successfully migrated; when the first verification information and the second verification information are inconsistent, the migration of the corresponding data block is re-executed or the corresponding data block is marked as a migration anomaly.

8. The method according to claim 1, characterized in that, The data modifications made by the tracking host to the source storage device include: Determine the data blocks involved in the host write request; Set a change flag for the data block modified by the host write request, and record the write version or write order of the data block; After the data block completes its initial migration, the corresponding data is read from the source storage device and written to the target storage device based on the data block with the change identifier. Once the data in the target storage device matches the corresponding data in the source storage device, the change flag of the data block is cleared. If the data block is modified again during data synchronization, the change identifier is retained or reset, and the latest version of the data block is synchronized in subsequent synchronization processes.

9. The method according to claim 1, characterized in that, The processing prior to updating the logical address mapping in the host input / output path includes: Suspend receiving new host write requests or freeze host write requests; Waiting for the submitted host write request to complete; Perform the final round of data synchronization; Confirm that there are no out-of-sync data modifications; Refresh the write caches associated with the source storage device and the target storage device; Update the source logical address mapping to the target logical address mapping, and restore host input / output requests after the target logical address mapping takes effect.

10. A logic cell migration system for heterogeneous storage environments, characterized in that, The system includes: The status management module is used to divide the logical units to be migrated into multiple data blocks and establish a status record to record the migration status of each data block. The running status acquisition module is used to acquire running status information of the source storage object and the target storage object; acquire access information of the input and output paths for the data block; and determine the migration priority of each data block based on the access information. The migration condition determination module is used to determine whether the data migration conditions are met based on the running status information of the source storage object and the target storage object. The data migration module is used to select data blocks to be migrated according to the migration priority of the data blocks when the data migration conditions are met, migrate the selected data blocks from the source storage object to the target storage object, and update the corresponding status records according to the migration results. The data synchronization module is used to track data modifications made to the source storage object during data block migration and to perform data synchronization on the target storage object based on the tracking results; wherein, during the data block migration and data synchronization, the logical access interface to the logical unit remains unchanged; The mapping switching module is used to update the logical address mapping in the input / output path after confirming that the data corresponding to the logical unit to be migrated is consistent in the target storage object, so that the input / output path is switched from the source storage object to the target storage object.