A data migration method and device, electronic equipment and storage medium

Abstract

The application provides a data migration method and device, electronic equipment and storage medium. The method comprises the following steps: determining a target data block to be migrated; copying data content of the target data block stored on a source storage medium to a destination storage medium, and recording data block information of the target data block, wherein the data block information comprises storage location information and data content; adding a read-write lock to the target data block to pause reading and writing of the target data block; migrating the storage location information and the data content to the destination storage medium, and releasing the read-write lock to complete migration of the target data block. In the process of copying the data block to be migrated on the destination storage medium, the business is normally executed, and the reading and writing of the data block by the business within the period are recorded. When the recorded content is migrated, the read-write lock of the data block is set again to pause the reading and writing of the data block by the business, thereby shortening the duration of adding the read-write lock to the data block.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a data migration method, apparatus, electronic device, and storage medium. Background Technology

[0002] As the amount of data from internet users continues to grow, storage costs will account for an increasingly larger proportion of the total cost of a database.

[0003] Among the many solutions, one approach is to implement tiered storage for hot and cold data. That is, "hot data" is stored on expensive, small-capacity, high-speed solid-state drives (SSDs), while "cold data" is stored on inexpensive, large-capacity, slower-speed hard disk drives (HDDs). Since the hotness or coldness of data changes with user access, data whose hotness or coldness has changed needs to be migrated in real-time or periodically. Currently, migrating data requires interrupting the corresponding business operations, and restarting the business operations after the migration is complete, resulting in a significant business downtime. Summary of the Invention

[0004] The purpose of this application is to provide a data migration method, apparatus, electronic device, and storage medium to reduce the duration of service interruption during data migration.

[0005] In a first aspect, embodiments of this application provide a data migration method, including:

[0006] Identify the target data block to be migrated;

[0007] Copy the data content of the target data block stored on the source storage medium to the destination storage medium, and record the data block information of the target data block, including storage location information and updated data;

[0008] Add a read-write lock to the target data block to suspend reading and writing to the target data block;

[0009] The storage location information and updated data are migrated to the destination storage medium, and the read-write lock is released to complete the migration of the target data block.

[0010] In this embodiment of the application, during the process of copying the data block to be migrated to the destination storage medium, the business is executed normally, and the read and write status of the data block during this period is recorded. When the recorded content is migrated, a read and write lock is set for the data block, suspending the business's read and write operations on the data block. After the migration of the recorded data is completed, the read and write lock is released, thereby shortening the time for adding a read and write lock to the data block.

[0011] In any embodiment, the method for determining the target data block to be migrated includes:

[0012] Count the number of accesses to each data block and sort the data blocks according to the number of accesses;

[0013] The target data blocks to be migrated are determined based on the data block size, data block order, and storage medium capacity.

[0014] This application embodiment divides data blocks into "cold data" or "hot data" by dividing the data block size, sorting the number of accesses of the data blocks, and the capacity of each storage medium, and determines the target data block to be migrated according to the current storage location of the data block. The amount of calculation required to determine cold and hot data is relatively small.

[0015] In any embodiment, the method further includes:

[0016] Obtain the UUID of the snapshot corresponding to each data block and the data block number ObjiectID. Obtain the key value of the corresponding data block based on the UUID and ObjiectID. The ObjiectID is obtained by rounding down the size of the data block.

[0017] Obtain the sequence number bitmapID of each data block stored in the storage medium, and obtain the value of the corresponding data block based on the bitmapID and the type of storage medium;

[0018] Generate index information for the data block based on its key and value.

[0019] This application embodiment generates corresponding index information for each data block in advance, which makes it easier for the business to quickly find the corresponding data block, and at the same time reduces the probability of data block errors during migration.

[0020] In any embodiment, before acquiring a read-write lock on the target data block, the method further includes:

[0021] It receives business requests, determines the target data block based on pre-built database index information, and performs read and write operations on the target data block.

[0022] In this embodiment of the application, a read-write lock is only applied to the target data block during the migration of the data block information. Before that, the business can be executed normally, thus shortening the time required to lock the target data block.

[0023] In any embodiment, copying the data content of a data block stored on the source storage medium to the destination storage medium includes:

[0024] An empty new data block is generated on the destination storage medium, and the data content of the target data block is copied into the new data block.

[0025] In this embodiment of the application, during the process of copying the data block to be migrated to the destination storage medium, the business is executed normally, and the read and write status of the data block during this period is recorded. When the recorded content is migrated, a read and write lock is set for the data block, suspending the business's read and write operations on the data block. After the migration of the recorded data is completed, the read and write lock is released, thereby shortening the time for adding a read and write lock to the data block.

[0026] In any embodiment, the key value corresponding to the new data block is generated based on the UUID, ObjectID, and suffix identifier of the target data block; the value value corresponding to the new data block is generated based on the bitmapID of the target data block and the type of the destination storage medium.

[0027] After completing the migration of the target data block, the method also includes:

[0028] Remove the suffix identifier from the key value of the new data block.

[0029] In this embodiment of the application, after the migration of the target data block is completed, the suffix identifier on the key value in the new data block is deleted so that the new data block can be accurately located when a business request arrives.

[0030] In any embodiment, after the migration of the target data block is completed, the method further includes:

[0031] Delete the target data block from the source storage medium to free up storage space in the source storage medium.

[0032] Secondly, embodiments of this application provide a data migration apparatus, comprising:

[0033] The data determination module is used to determine the target data block to be migrated;

[0034] The data copy module is used to copy the data content of the target data block stored on the source storage medium to the destination storage medium, and record the data block information of the target data block, including storage location information and updated data;

[0035] The locking module is used to acquire read-write locks on the target data block to suspend reading and writing of the target data block;

[0036] The data update module is used to migrate the storage location information and updated data to the destination storage medium and release the read-write lock to complete the migration of the target data block.

[0037] Thirdly, embodiments of this application provide an electronic device, including: a processor, a memory, and a bus, wherein,

[0038] The processor and the memory communicate with each other via the bus;

[0039] The memory stores program instructions that can be executed by the processor, and the processor can execute the method of the first aspect by calling the program instructions.

[0040] Fourthly, embodiments of this application provide a non-transitory computer-readable storage medium, comprising:

[0041] The non-transitory computer-readable storage medium stores computer instructions that cause the computer to perform the method of the first aspect.

[0042] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0043] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a schematic diagram of a data migration method provided in an embodiment of this application;

[0045] Figure 2 This is a flowchart illustrating a method for determining a target data block, provided in an embodiment of this application.

[0046] Figure 3 This application provides a schematic diagram of the virtual disk relationship of a virtual machine as an embodiment of the present application;

[0047] Figure 4 This is a schematic diagram of data block key value splicing provided in an embodiment of this application;

[0048] Figure 5 A data block index diagram provided for embodiments of this application;

[0049] Figure 6 This application provides a schematic diagram of data block value splicing in an embodiment;

[0050] Figure 7 This is a schematic diagram of another data migration method provided in an embodiment of this application;

[0051] Figure 8A schematic flowchart of a data migration method provided in this application embodiment;

[0052] Figure 9 This is a schematic diagram of a data migration device provided in an embodiment of this application;

[0053] Figure 10 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0054] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0056] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0057] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0058] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0059] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0060] In the description of the embodiments of this application, the technical terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of this application and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.

[0061] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0062] Currently, before migrating a data block, reading and writing to that data block need to be paused. After the data block is migrated, the read and write locks on that data block are released. During this period, the business cannot be executed for a relatively long time, causing users to experience lag when making business requests.

[0063] To address the aforementioned issues, the inventors of this application, after extensive research, have proposed a data processing method. This method allows normal business operations while copying data content from a data block to be migrated. Only when new data is generated during the migration process is a read-write lock applied to the data block. The read-write lock is released after the newly generated data has been copied. Therefore, the timing of locking the data block using a read-write lock is postponed, and the amount of data copied during the locking period is reduced, resulting in a shorter locking time.

[0064] It is understood that the data migration method provided in this application embodiment can be applied to electronic devices, which may include terminals and servers; wherein the terminal may specifically be a smartphone, tablet computer, computer, personal digital assistant (PDA), etc.; and the server may specifically be an application server or a web server.

[0065] Figure 1 This is a schematic diagram of a data migration method provided in an embodiment of this application, such as... Figure 1 As shown, the method includes:

[0066] Step 101: Determine the target data block to be migrated.

[0067] In practical implementation, electronic devices include various types of storage media, each containing data blocks. A data block can be understood as a partition of the storage space of the storage media according to a preset size. To reduce the impact on business operations during data block migration, the data blocks can be divided into smaller blocks, for example, each data block could be 1MB in size. Of course, other sizes are also possible, and this application does not specifically limit this.

[0068] Storage media can include solid-state drives (SSDs) and hard disk drives (HDDs), with different types used to store data blocks of varying accessibility. The accessibility of a data block can be determined by the frequency with which the data within it is accessed; frequently accessed data blocks are called "hot data," while infrequently accessed data blocks are called "cold data." Hot data can be stored on SSDs with high read / write speeds, while cold data can be stored on HDDs with lower read / write speeds.

[0069] It should be noted that the hotness / coldness level of data blocks can be further subdivided, and the types of storage media contained in electronic devices can also be more diverse. The hotness / coldness level of data blocks can be correlated with the types of storage media contained in the electronic device, with different types of storage media used to store data blocks of different hotness / coldness levels. For ease of description, this application embodiment uses solid-state drives and hard disk drives as examples, and the hotness / coldness level of data blocks includes cold data and hot data.

[0070] Because solid-state drives (SSDs) have limited capacity, it's impossible to store all the data from electronic devices on them. To optimize user experience, hot data is stored in SSD blocks, while cold data is stored in HDD blocks. However, in practice, the hotness or coldness of data is not constant. A data block that is hot in one time period may become cold in the next; similarly, cold data in one time period may become hot data in the next.

[0071] Therefore, electronic devices can periodically perform hot and cold data scans on the storage medium to determine if there are any target data blocks that need to be migrated. It is understood that if hot data is stored on a mechanical hard drive, or cold data is stored on a solid-state drive, then that data block is the target data block that needs to be migrated.

[0072] Step 102: Copy the data content of the target data block stored on the source storage medium to the destination storage medium, and record the data block information of the target data block, including storage location information and updated data.

[0073] In the specific implementation process, the source storage medium refers to the storage medium on which the target data block is stored before migration. The destination storage medium refers to the storage medium to which the target data is to be migrated. For example, if the target data block was originally stored on a mechanical hard disk and needs to be migrated to a solid-state disk, then the mechanical hard disk is the source storage medium and the solid-state disk is the destination storage medium.

[0074] After identifying the target data block to be migrated, the electronic device copies the data stored in the target data block to the destination storage medium. It is understandable that since copying the content of the target data block to the storage medium takes time, and services can still operate normally during this period, it is possible that some data in the data block already copied to the destination storage medium has been modified. For example, if the target data block contains data ABCDE, and during the copying process, B is changed to B', to ensure data integrity and allow services to operate normally, the electronic device can record the target data block's data block information on the source storage medium. This data block information includes storage location information and updated data. The storage location information refers to the offset of the modified data within the target data block, i.e., the starting address of the modified data within the target data block, for example, the starting address of B in the target data block. The updated data refers to the specific content of the modified data, such as B'.

[0075] Understandably, a data block may also include a data length, which refers to the number of bytes occupied by the modified data, such as the number of bytes occupied by B. A data block may include the storage location information of one or more pieces of modified data and updated data, or it may be empty. When a data block is empty, it indicates that the data content in the target data block did not change during the copying process to the destination storage medium.

[0076] Step 103: Add a read-write lock to the target data block to suspend reading and writing to the target data block.

[0077] Before copying data block information to the destination storage medium, the electronic device applies a read-write lock to the target data block on the source storage medium. It can be understood that after applying a read-write lock to the target data block, the target data block cannot be read or written. Therefore, the services that need to access the target data block will also be suspended.

[0078] Step 104: Migrate the storage location information and updated data to the destination storage medium and release the read-write lock to complete the migration of the target data block.

[0079] After acquiring a read-write lock on a target data block, the electronic device migrates the storage location information contained in the data block information to the destination storage medium. The purpose of migrating the data block information is to use the storage location information in the data block information to locate the data that needs to be updated in the destination storage medium, and then replace the located data with the updated data from the data block information. This completes the migration of the target data block.

[0080] After the migration of the target data block is complete, release the read-write lock on the target data block.

[0081] In this embodiment of the application, during the process of copying the data block to be migrated to the destination storage medium, the business is executed normally, and the read and write status of the data block during this period is recorded. When the recorded content is migrated, a read and write lock is set for the data block, suspending the business's read and write operations on the data block. After the migration of the recorded data is completed, the read and write lock is released, thereby shortening the time for adding a read and write lock to the data block.

[0082] Based on the above embodiments, the target data block to be migrated is determined, including:

[0083] Count the number of accesses to each data block and sort the data blocks according to the number of accesses;

[0084] The target data blocks to be migrated are determined based on the data block size, data block order, and storage medium capacity.

[0085] In the specific implementation process, the number of times each data block in the electronic device is accessed within a preset time period (e.g., 24 hours, 40 hours, etc.) can be counted, i.e., the number of times the data is read and written. The electronic device can sort the data blocks according to the number of accesses, for example, sorting them from most to least accessed. The frequency of access to a data block determines whether it should reside on a solid-state drive (SSD) or a mechanical hard disk (HDD). After hot and cold data interaction, data with high access frequency resides on the SSD, and data with low access frequency resides on the HDD.

[0086] Figure 2 This is a flowchart illustrating a method for determining a target data block, as provided in an embodiment of this application. Figure 2As shown in the figure. After sorting according to the access times of data blocks, each data block has a serial number, which is defined as X. The capacity of the mechanical disk is CH, the capacity of the solid-state disk is CS, and the size of the data block is Q. If X < CS / Q, the data block with the serial number X is hot data. Then, it is judged whether the data block is located on the solid-state disk. If it is on the solid-state disk, no migration is required. If it is not on the solid-state disk, data interaction is required, and this data block is the target data block. On the contrary, if X ≥ CS / Q, the data block with the serial number X is cold data. Then, it is judged whether the data block is located on the mechanical disk. If it is on the mechanical disk, no migration is required. If it is not on the mechanical disk, data interaction is required, and this data block is the target data block

[0087] In the embodiment of the present application, the data blocks are divided into "cold data" or "hot data" by the size of the divided data blocks, the sorting of the access times of the data blocks, and the capacities of each storage medium, and the target data block to be migrated is determined according to the current storage location of the data blocks, and the calculation amount for determining the cold and hot data is small

[0088] Based on the above embodiment, the method further includes

[0089] Obtain the UUID of the snapshot corresponding to each data block and the data block label ObjiectID, and obtain the key value of the corresponding data block according to the UUID and ObjiectID; where, ObjiectID is obtained according to the address of the data block in the snapshot and the size of the data block

[0090] Obtain the serial number bitmapID of each data block stored in the storage medium, and obtain the value value of the corresponding data block according to the bitmapID and the type of the storage medium

[0091] Generate the index information of the data block according to the key value and value value corresponding to the data block

[0092] In the specific implementation process, in order to accurately find the accessed data block according to the service request and ensure the correctness of the data block during the migration process, an index can be established in advance for the data blocks in the storage medium Figure 3 A schematic diagram of the virtual disk relationship of a virtual machine provided by the embodiment of the present application is as Figure 3 shown. In order to establish an index, the rocksdb database can be used to store the index of the data block, and this database is an embeddable and persistent key-value storage Figure 3 It includes a solid-state disk SSD and two mechanical disks HDD, and each disk contains data blocks

[0093] A component is a logical concept that maps a segment of space on a virtual disk to a physical disk. A component contains snapshots, which are composed of data blocks. To locate these data blocks, an index relationship between snapshots and data blocks can be established. Each snapshot has a unique identifier (Universally Unique Identifier, UUID) (occupying 16 bytes). The data block number of a snapshot is calculated based on the address of the storage space corresponding to the snapshot and the size of the data block. For example, if the data block size is 1MB, the address corresponding to the data block is 011. Dividing this address by the data block size and rounding the result gives the ObserverID of the data block. Concatenating the snapshot's UUID and the data block's ObserverID yields the key value of the data block. Specifically... Figure 4 As shown.

[0094] Figure 5 A data block index diagram provided for embodiments of this application, such as Figure 5 As shown. Each data block has a unique serial number bitmapID in its corresponding disk (the numbers in the SSD and HDD diagrams). The bitmapID is stored in 8 bytes, with the highest bit (bit 63) using 0 and 1 to distinguish between HDD and SSD, as shown in the diagram. Figure 6 The value of the data block is these 8 bytes.

[0095] This application embodiment generates corresponding index information for each data block in advance, which makes it easier for the business to quickly find the corresponding data block, and at the same time reduces the probability of data block errors during migration.

[0096] Based on the above embodiments, before acquiring a read-write lock on the target data block, the method further includes:

[0097] It receives business requests, determines the target data block based on pre-built database index information, and performs read and write operations on the target data block.

[0098] In the actual implementation process, the electronic device can still receive and process business requests normally before copying the database block information. After receiving a business request, the electronic device determines the target data block based on the index information of each data block in the database. It can be understood that this target data block may be the data block being migrated. After determining the target data block, read and write operations are performed on the target data block according to the specific business request.

[0099] In this embodiment of the application, a read-write lock is only applied to the target data block during the migration of the data block information. Before that, the business can be executed normally, thus shortening the time required to lock the target data block.

[0100] Based on the above embodiments, copying the data content of a data block stored on the source storage medium to the destination storage medium includes:

[0101] An empty new data block is generated on the destination storage medium, and the data content of the target data block is copied into the new data block.

[0102] In practice, when an electronic device copies the data content of a target data block to the destination storage medium, it can create a new empty data block on the destination storage medium. This new data block is the same size as the target data block. After creating the new data block, the data content is copied from the source storage medium into the new data block.

[0103] Taking the migration of a target data block from a mechanical hard disk (HDD) to a solid-state drive (SSD) as an example: A data block can only have one unique key value in an electronic device. When creating a new data block, a suffix needs to be added to the key value, such as "temp". The value of the key with the "temp" suffix represents the bitmapID of the newly created data block stored on the destination storage medium. Finally, using the atomicity of database operations, the original key value of "snapshot uuid + ObjectId" is updated to "Bit 63 is 1, bits 0-62 are the SSD's bitmapId", and the key-value pair with the "temp" suffix is ​​deleted from the database.

[0104]

[0105] In this embodiment of the application, during the process of copying the data block to be migrated to the destination storage medium, the business is executed normally, and the read and write status of the data block during this period is recorded. When the recorded content is migrated, a read and write lock is set for the data block, suspending the business's read and write operations on the data block. After the migration of the recorded data is completed, the read and write lock is released, thereby shortening the time for adding a read and write lock to the data block.

[0106] Figure 7 This is a schematic diagram of another data migration method provided in an embodiment of this application. Figure 8 A schematic flowchart of a data migration method provided in this application embodiment is shown below. Figure 7 and Figure 8 As shown, the method includes:

[0107] Step A: The virtual machine running in the electronic device receives the service request, finds the corresponding data block through the index established in the database, and performs read and write operations on the data in the data block;

[0108] Step B: The electronic device counts the access frequency of each data block and determines the target data block to be migrated. Figure 7 The stripe with a filling pattern 1) takes the migration of the target data block on the mechanical hard disk to the solid-state disk as an example. The business in step A is not interrupted. The business is still written on the data block on the mechanical hard disk, and the write storage location information and update data are recorded in memory. An empty new data block is created on the solid-state disk.

[0109] Step C: The electronic device reads the contents of the target data block to be migrated from the mechanical hard disk and copies it to an empty new data block on the solid-state disk (the business I / O does not stop here; if new data is written just before the data content is copied, this new data will also be copied to the new data block on the solid-state disk). If new data is written during the copying process, the data block information of the target data block is recorded.

[0110] Step D: Lock the target data block and suspend business I / O. Previously, locking needed to begin in step B, which prevented data reading and writing, causing business interruption and significant impact. This embodiment adds a read-write lock in step D, only affecting the copied data block information (see...). Figure 7 When another filling strip (2) is added, locking is applied, which significantly reduces the locking force and reduces the impact on business during hot and cold data interaction.

[0111] Step E: Read the storage location information and update data recorded in Step D from memory, and then write the update data into the data block in the solid-state disk.

[0112] Step F: After the data copy is complete, delete the target data block that was originally on the mechanical hard drive to free up space on the mechanical hard drive.

[0113] The data migration method provided in this application can be applied to distributed storage and hyperconverged products, reducing the impact on business I / O during cold and hot data migration. During the first migration, data blocks are not locked, allowing normal read and write operations. The data is copied completely, and the offset and length of the newly written data are recorded in memory. When copying the records in memory, read-write locks are required. Businesses can read normally, but writes are paused. The 1MB data block is locked by a write lock, which does not affect the read and write operations of other data blocks. After copying the data recorded in memory, the index is finally set to point to the new data block. Because only the write I / O of the 1MB data record is locked during the copy process, the impact on normal business read and write operations is greatly reduced, minimizing client-side lag during cold and hot data interaction.

[0114] Figure 9 This is a schematic diagram of a data migration device provided in an embodiment of this application. The device can be a module, program segment, or code on an electronic device. It should be understood that this device is similar to the one described above. Figure 1 The method implementation corresponds to this and can be executed. Figure 1The specific functions of the device involved in each step of the method embodiment can be found in the description above; to avoid repetition, detailed descriptions are omitted here. The device includes: a data determination module 901, a data copying module 902, a locking module 903, and a data update module 904, wherein:

[0115] Data determination module 901 is used to determine the target data block to be migrated;

[0116] The data copy module 902 is used to copy the data content of the target data block stored on the source storage medium to the destination storage medium, and record the data block information of the target data block, including storage location information and update data;

[0117] The locking module 903 is used to add a read-write lock to the target data block to suspend reading and writing of the target data block;

[0118] The data update module 904 is used to migrate the storage location information and the updated data to the destination storage medium and release the read-write lock to complete the migration of the target data block.

[0119] Based on the above embodiments, the data determination module 901 is specifically used for:

[0120] The number of accesses to each data block is counted, and the data blocks are sorted according to the number of accesses.

[0121] The target data blocks to be migrated are determined based on the data block size, data block order, and storage medium capacity.

[0122] Based on the above embodiments, the device further includes an index building module, used for:

[0123] Obtain the UUID and data block identifier ObjectID of the snapshot corresponding to each data block, and obtain the key value of the corresponding data block based on the UUID and ObjectID; wherein, the ObjectID is obtained based on the address of the data block in the snapshot and the size of the data block;

[0124] Obtain the sequence number bitmapID of each data block stored in the storage medium, and obtain the value of the corresponding data block according to the bitmapID and the type of the storage medium;

[0125] The index information of the data block is generated based on the key and value values ​​corresponding to the data block.

[0126] Based on the above embodiments, the device further includes a service processing module, used for:

[0127] The system receives business requests, determines the target data block based on pre-built database index information, and performs read and write operations on the target data block.

[0128] Based on the above embodiments, the data update module 904 is specifically used for:

[0129] An empty new data block is generated on the target storage medium, and the data content of the target data block is copied into the new data block.

[0130] Based on the above embodiments, the key value corresponding to the new data block is generated according to the UUID, ObjectID, and suffix identifier of the target data block; the value value corresponding to the new data block is generated according to the bitmapID of the target data block and the type of the destination storage medium; the device further includes: a suffix deletion module, used for:

[0131] Remove the suffix identifier from the key value of the new data block.

[0132] Based on the above embodiments, the device further includes a data block deletion module, used for:

[0133] Delete the target data block from the source storage medium.

[0134] Figure 10 This is a schematic diagram of the physical structure of the electronic device provided in the embodiments of this application, such as... Figure 10 As shown, the electronic device includes: a processor 1001, a memory 1002, and a bus 1003; wherein,

[0135] The processor 1001 and the memory 1002 communicate with each other through the bus 1003;

[0136] The processor 1001 is used to call program instructions in the memory 1002 to execute the methods provided in the above-described method embodiments, including, for example, determining a target data block to be migrated; copying the data content of the target data block stored on the source storage medium to the destination storage medium, and recording the data block information of the target data block, the data block information including storage location information and update data; acquiring a read-write lock on the target data block to suspend reading and writing to the target data block; migrating the storage location information and the update data to the destination storage medium, and releasing the read-write lock to complete the migration of the target data block.

[0137] The processor 1001 can be an integrated circuit chip with signal processing capabilities. The processor 1001 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor.

[0138] The memory 1002 may include, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0139] This embodiment discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer can execute the methods provided in the above-described method embodiments, such as: determining a target data block to be migrated; copying the data content of the target data block stored on the source storage medium to the destination storage medium, and recording the data block information of the target data block, the data block information including storage location information and updated data; acquiring a read-write lock on the target data block to suspend reading and writing to the target data block; migrating the storage location information and the updated data to the destination storage medium, and releasing the read-write lock to complete the migration of the target data block.

[0140] This embodiment provides a non-transitory computer-readable storage medium storing computer instructions that cause the computer to execute the methods provided in the above-described method embodiments. These instructions include, for example,: determining a target data block to be migrated; copying the data content of the target data block stored on a source storage medium to a destination storage medium, and recording data block information of the target data block, including storage location information and updated data; acquiring a read-write lock on the target data block to suspend reading and writing to the target data block; migrating the storage location information and the updated data to the destination storage medium, and releasing the read-write lock to complete the migration of the target data block.

[0141] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0142] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0143] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0144] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0145] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A data migration method, characterized in that, include: Identify the target data block to be migrated; Copy the data content of the target data block stored on the source storage medium to the destination storage medium, and record the data block information of the target data block, including storage location information and updated data; A read-write lock is applied to the target data block to suspend reading and writing of the target data block; The storage location information and the updated data are migrated to the destination storage medium, and the read-write lock is released to complete the migration of the target data block. The step of determining the target data block to be migrated includes: obtaining the UUID and data block identifier ObjectID of the snapshot corresponding to each data block; obtaining the key value of the corresponding data block based on the UUID and the ObjectID; wherein, the ObjectID is obtained based on the address of the data block in the snapshot and the size of the data block; obtaining the sequence number bitmapID of the storage medium in which each data block is stored; obtaining the value value of the corresponding data block based on the bitmapID and the type of the storage medium; and generating index information of the data block based on the key value and value corresponding to the data block. The step of copying the data content of the data block stored on the source storage medium to the destination storage medium includes: generating an empty new data block on the destination storage medium, and copying the data content of the target data block into the new data block; the key value corresponding to the new data block is generated based on the UUID, ObjectID, and suffix identifier of the target data block; the value value corresponding to the new data block is generated based on the bitmapID of the target data block and the type of the destination storage medium. After completing the migration of the target data block, the method further includes: using atomic operations of the database to update the value of the key corresponding to the target data block to the bitmapID and type of the destination storage medium, and deleting the suffix identifier in the key value of the new data block; The step of determining the target data block to be migrated includes: counting the number of accesses to each data block and sorting the data blocks according to the number of accesses; and determining the target data block to be migrated according to the data block size, the data block sorting and the capacity of the storage medium. Before acquiring a read-write lock on the target data block, the method further includes: receiving a business request, determining the target data block based on pre-built database index information, and performing read-write operations on the target data block.

2. The method according to claim 1, characterized in that, After completing the migration of the target data block, the method further includes: Delete the target data block from the source storage medium.

3. A data migration device, characterized in that, include: The data determination module is used to determine the target data block to be migrated; The data copy module is used to copy the data content of the target data block stored on the source storage medium to the destination storage medium, and record the data block information of the target data block, including storage location information and updated data; A locking module is used to acquire a read-write lock on the target data block to suspend reading and writing of the target data block; The data update module is used to migrate the storage location information and the updated data to the destination storage medium and release the read-write lock to complete the migration of the target data block; An indexing module is used to obtain the UUID and ObjectID of the snapshot corresponding to each data block, and to obtain the key value of the corresponding data block based on the UUID and ObjectID; wherein, the ObjectID is obtained based on the address of the data block in the snapshot and the size of the data block; to obtain the bitmapID of the sequence number stored in the storage medium of each data block, and to obtain the value value of the corresponding data block based on the bitmapID and the type of the storage medium; and to generate the index information of the data block based on the key value and value of the data block. The data update module is specifically used for: generating an empty new data block on the destination storage medium, and copying the data content of the target data block into the new data block; the key value corresponding to the new data block is generated based on the UUID, ObjectID, and suffix identifier of the target data block; the value value corresponding to the new data block is generated based on the bitmapID of the target data block and the type of the destination storage medium. The suffix deletion module is used to update the value of the key corresponding to the target data block to the bitmapID and type of the destination storage medium by using atomic operations of the database after the migration of the target data block is completed, and delete the target data block in the source storage medium. The data determination module is specifically used for: counting the number of accesses to each data block and sorting each data block according to the number of accesses; and determining the target data block to be migrated according to the data block size, data block sorting and storage medium capacity. Also includes: The business processing module is used to receive a business request before acquiring a read-write lock on the target data block, determine the target data block based on pre-built database index information, and perform read and write operations on the target data block.

4. An electronic device, characterized in that, include: Processor, memory, and bus, among which, The processor and the memory communicate with each other via the bus; The memory stores program instructions that can be executed by the processor, and the processor can invoke the program instructions to perform the method as described in any one of claims 1-2.

5. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions, which, when executed by a computer, cause the computer to perform the method as described in any one of claims 1-2.

Images