Data migration method, device, electronic device and readable storage medium

Abstract

The present application discloses a data migration method, device, electronic device and readable storage medium. The method includes pre-building a first storage pool and a second storage pool with different storage performances, wherein the storage performance of the first storage pool is better than that of the second storage pool; dividing the target data into first sub-data and second sub-data, storing the first sub-data in the first storage pool and the second sub-data in the second storage pool; when receiving a migration instruction for the target data, writing the first sub-data read from the first storage pool to the second storage pool, and sending a request to update the storage location information of the target data to a metadata server; when the metadata server responds that the update is completed, deleting the first sub-data from the first storage pool to complete the migration operation of the target data, thereby improving the efficiency of file migration and reducing the probability of file migration errors.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data migration method, device, electronic device and readable storage medium. Background Art

[0002] With the rapid development of computer technology and its wide application in daily work and life, various computer data have shown explosive growth, and storage technology has also developed accordingly. In order to adapt to the fast and secure storage of data, distributed file systems are applied. Distributed file systems are clusters composed of multiple file storage node servers. Files are stored in blocks, with objects as the basic unit. They support storing a copy of data on multiple nodes. Each node can obtain complete data through inter-node communication. When a node goes down, complete data can be restored according to the configured strategy. It has the characteristics of high availability, high performance, and high scalability. Each node provides metadata services, namely MDS, for various metadata access operations to balance business pressure.

[0003] For distributed file systems, file migration, also known as file tiering, is inevitable. The process of performing file migration or file tiering in related technologies is as follows: the migration client reads all file data from the advanced pool; the migration client writes the data into the slow pool; the metadata is updated to the metadata server MDS, that is, the data pool of the file is changed from the advanced pool to the slow pool, and the MDS responds to the client after the update; after receiving the response, the client deletes the data in the advanced pool; the migration succeeds / failed.

[0004] However, this type of file tiering is only applicable to scenarios where file data is stored in a single data pool. When files are stored in multiple data pools, the data that needs to be read, written, and deleted during file tiering migration is different. At the same time, exceptions may occur in each step, namely read errors, write errors, metadata modification errors, deletion errors, etc.

[0005] In view of this, how to improve the efficiency of file migration and reduce the probability of file migration errors is a technical problem that technical personnel in the relevant field need to solve. Summary of the invention

[0006] The present application provides a data migration method, device, electronic device and readable storage medium, which can improve the efficiency of file migration and reduce the probability of file migration errors.

[0007] To solve the above technical problems, the embodiments of the present invention provide the following technical solutions:

[0008] An embodiment of the present invention provides a data migration method, including:

[0009] Pre-building a first storage pool and a second storage pool with different storage performances, wherein the storage performance of the first storage pool is better than that of the second storage pool;

[0010] The target data is divided into first sub-data and second sub-data; the first sub-data is stored in the first storage pool, and the second sub-data is stored in the second storage pool;

[0011] When receiving the migration instruction of the target data, writing the first sub-data read from the first storage pool to the second storage pool, and sending a request to update the storage location information of the target data to the metadata server;

[0012] When the metadata server responds that the update is complete, the first sub-data is deleted from the first storage pool to complete the migration operation of the target data.

[0013] Optionally, writing the first sub-data read from the first storage pool to the second storage pool includes:

[0014] Reading the first sub-data from the first storage pool;

[0015] Determining whether the first sub-data is successfully read from the first storage pool;

[0016] If the first sub-data is successfully read from the first storage pool, writing the first sub-data to the second storage pool;

[0017] If the first sub-data is not successfully read from the first storage pool, the junk data in the second storage pool is deleted, and migration failure information is sent at the same time.

[0018] Optionally, after writing the first sub-data read from the first storage pool to the second storage pool, the step includes:

[0019] determining whether the first sub-data is successfully written into the second storage pool;

[0020] If the first sub-data is not successfully written into the second storage pool, the junk data in the second storage pool is deleted, and migration failure information is sent at the same time.

[0021] Optionally, after sending the request to the metadata server to update the storage location information of the target data, the method further includes:

[0022] Determining whether the metadata server successfully updates the metadata;

[0023] If the metadata server fails to update the metadata successfully, the junk data in the second storage pool is deleted and a migration failure message is sent;

[0024] If the metadata server successfully updates the metadata, the junk data in the first storage pool is deleted.

[0025] Optionally, after deleting the garbage data in the first storage pool, the method further includes:

[0026] If the data deletion operation in the first storage pool is successfully completed, the data migration is successful;

[0027] If the data deletion operation in the first storage pool is not successfully completed, the data migration fails, and the garbage data in the first storage pool will continue to be deleted when the data migration operation is performed next time.

[0028] Optionally, dividing the target data into the first sub-data and the second sub-data includes:

[0029] When receiving target data to be written, determining whether the target data is greater than a preset capacity threshold;

[0030] If the target data is larger than a preset capacity threshold, the target data is divided into a first sub-data and a second sub-data; the space occupied capacity value of the first sub-data is smaller than that of the second sub-data;

[0031] If the target data is smaller than or equal to a preset capacity threshold, the target data is stored in the first storage pool.

[0032] Optionally, the sending a request to the metadata server to update the storage location information of the target data includes:

[0033] An instruction to update the data pool to the second storage pool is sent to the metadata server.

[0034] Another aspect of the present invention provides a data migration device, including:

[0035] A storage pool construction module, used to pre-construct a first storage pool and a second storage pool with different storage performances, wherein the storage performance of the first storage pool is better than that of the second storage pool;

[0036] A data segmentation and storage module, used for segmenting target data into first sub-data and second sub-data; the first sub-data is stored in the first storage pool, and the second sub-data is stored in the second storage pool;

[0037] a data migration module, configured to, upon receiving a migration instruction for the target data, write the first sub-data read from the first storage pool to the second storage pool, and send a request to the metadata server to update the storage location information of the target data;

[0038] The data deletion module is used to delete the first sub-data from the first storage pool when the metadata server responds that the update is completed, so as to complete the migration operation of the target data.

[0039] An embodiment of the present invention further provides an electronic device, comprising a processor, wherein the processor is configured to implement the steps of the data migration method as described in any of the preceding items when executing a computer program stored in a memory.

[0040] Finally, an embodiment of the present invention further provides a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the data migration method as described in any of the preceding items are implemented.

[0041] The advantage of the technical solution provided by the present application is that, based on file multi-data pool storage, the target data to be written is divided into two parts, each of which is stored in a first storage pool and a second storage pool with different storage performances. When migrating file tiered data, only the data of the first storage pool with better performance is read and written to the second storage pool, and the metadata is modified from two storage pools to one storage pool. When deleting, only the data in the first storage pool is deleted. In this way, the data for tiered reading, writing, metadata modification, and old data deletion operations only have the data of the first storage pool, rather than all the data of the file, thereby effectively improving the efficiency of file migration and reducing the probability of file migration errors, further enriching the file tiered migration scenarios, improving the stability of the storage system, and improving the competitiveness of the product.

[0042] In addition, the embodiments of the present invention also provide corresponding implementation devices, electronic devices and readable storage media for the data migration method, which further make the method more practical, and the devices, electronic devices and readable storage media have corresponding advantages.

[0043] It is to be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A flowchart of a data migration method provided by an embodiment of the present invention;

[0046] Figure 2 A schematic diagram of a flow chart of another data migration method provided by an embodiment of the present invention;

[0047] Figure 3 A structural diagram of a specific implementation of a data migration device provided by an embodiment of the present invention;

[0048] Figure 4 A structural diagram of a specific implementation of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0050] The terms "first", "second", "third", "fourth", etc. in the specification and claims of this application and the above drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may include steps or units that are not listed.

[0051] After introducing the technical solutions of the embodiments of the present invention, various non-limiting implementation methods of the present application are described in detail below.

[0052] See first Figure 1 , Figure 1 A flowchart of a data migration method provided by an embodiment of the present invention is provided. The embodiment of the present invention may include the following contents:

[0053] S101: pre-build a first storage pool and a second storage pool with different storage performances.

[0054] The storage performance of the first storage pool in this embodiment is better than that of the second storage pool. The first storage pool can be, for example, a high-level pool, that is, a data pool composed of solid-state drives (SSDs); the second storage pool can be a slow pool, that is, a data pool composed of mechanical hard drives (HDDs). This embodiment constructs two data pools with different performances to implement multi-data pool storage of files, that is, the data of the files to be written are stored in these two data pools. For example, the first 4M of the files to be written can be stored in the high-level pool, and the rest can be stored in the slow pool.

[0055] S102: Divide the target data into first sub-data and second sub-data.

[0056] Since the file size cannot be identified when the file is created, writing a large file to the first storage pool, such as the advanced pool, will waste SSD high-cost storage media resources. In order to take into account both storage performance and storage cost, the target data can be divided into two parts. The target data is stored in two storage pools, the first sub-data, such as the first half, is stored in the first storage pool, and the second sub-data, i.e. the remaining part of the data, is stored in the second storage pool.

[0057] S103: When receiving the migration instruction of the target data, writing the first sub-data read from the first storage pool to the second storage pool, and sending a request to update the storage location information of the target data to the metadata server.

[0058] The migration instruction of this embodiment refers to file stratification, that is, to achieve data flow between multiple storage media without user perception, that is, data migration. Data migration may need to support multiple user strategies. For data that meets the migration requirements, the system performs imperceptible file data relocation processing without affecting normal business, which can meet the user's storage solution needs for performance stratification and hot and cold data stratification. The metadata server of this embodiment is used to cache distributed file system metadata and provide various metadata access operations. The metadata server of this embodiment is also used to load hierarchical index tables and migrate files. After the files are migrated in this embodiment, the data storage location information in the metadata server changes from 2 storage pools to 1 storage pool. In order to facilitate metadata changes, an instruction to update the data pool to the second storage pool can be directly sent to the metadata server to implement the metadata update operation.

[0059] S104: When the metadata server responds that the update is complete, the first sub-data is deleted from the first storage pool to complete the migration operation of the target data.

[0060] When migrating large files with file tiering, first determine whether it is stored in multiple data pools. For example, the first 4M data is stored in the first storage pool, and the rest is stored in the second storage pool. If so, only the first 4M data of the file in the first storage pool will be migrated. The entire process only has these 4M data. When there are exceptions at each stage of the migration, only the 4M data will be cleared. That is, the data in the first storage pool is valid before the metadata is modified, and the data in the second storage pool is valid after the metadata is modified. Therefore, after the data migration is completed, that is, after the metadata successfully modifies the data, the data in the first storage pool can be cleared.

[0061] In the technical solution provided by the embodiment of the present invention, based on file multi-data pool storage, the target data to be written is divided into two parts, and each is stored in a first storage pool and a second storage pool with different storage performances. When the file tiered data is migrated, only the data of the first storage pool with better performance is read and written to the second storage pool, and the metadata is modified from the two storage pools to one storage pool. When deleting, only the data in the first storage pool is deleted. In this way, the data for tiered read, write, metadata modification, and old data deletion operations only have the data of the first storage pool, rather than all the data of the file, thereby effectively improving the file migration efficiency and reducing the error probability of file migration, further enriching the file tiered migration scenarios, improving the stability of the storage system, and improving the competitiveness of the product.

[0062] In the above embodiment, if the file is recorded in two data pools, the migration client only reads the corresponding first sub-data from the first storage pool, and then writes the data into the second storage pool, and no longer reads and writes all the file data. After the first sub-data is written into the second storage pool, the metadata of the data pool is notified to be updated to the second storage pool. At this time, the two data pools are changed to one, and the data are all in the second storage pool. After the MDS updates the metadata, it responds to the migration client, and the migration client only deletes the first sub-data from the first storage pool at this time; during the migration process, when the read fails, the garbage data in the slow pool is cleaned up, and the migration fails; when the write fails, the garbage data in the slow pool is cleaned up, and the migration fails; when the metadata update fails, the garbage data in the slow pool is cleaned up, and when the deletion fails, the migration fails, and the garbage data in the advanced pool is cleaned up again in the next migration, so as to further enrich the file tiering scenario and improve the system stability, which may specifically include:

[0063] As an optional implementation manner, the process of writing the first sub-data read from the first storage pool to the second storage pool may include:

[0064] Read the first sub-data from the first storage pool; determine whether the first sub-data is successfully read from the first storage pool; if the first sub-data is successfully read from the first storage pool, write the first sub-data to the second storage pool; if the first sub-data is not successfully read from the first storage pool, delete the garbage data in the second storage pool and send a migration failure message.

[0065] As another optional implementation, after writing the first sub-data read from the first storage pool to the second storage pool, the method may further include:

[0066] Determine whether the first sub-data is successfully written into the second storage pool; if the first sub-data is not successfully written into the second storage pool, delete the garbage data in the second storage pool and send a migration failure message.

[0067] As another optional implementation, after sending a request to the metadata server to update the storage location information of the target data, the following may also be included:

[0068] Determine whether the metadata server successfully updates the metadata; if the metadata server fails to update the metadata, delete the garbage data in the second storage pool and send a migration failure message; if the metadata server successfully updates the metadata, delete the garbage data in the first storage pool.

[0069] If the data deletion operation in the first storage pool is successfully completed, the data migration is successful;

[0070] If the data deletion operation in the first storage pool is not successfully completed, the data migration fails, and the garbage data in the first storage pool will continue to be deleted when the data migration operation is performed next time.

[0071] In this embodiment, in order to improve the efficiency of the entire file migration, the space occupied capacity value of the first sub-data is smaller than that of the second sub-data; for example, the first sub-data is the first 4M data of the target data. When the file is migrated, only the first 4M data of the file is read from the first storage pool, and the data is written to the second storage pool. When the metadata is updated, the data pool is changed to the second pool, and then only the first 4M data of the file in the first storage pool is deleted. For the exceptions in each stage, the data before the metadata is changed is valid in the first storage pool. At this time, the first 4M data of the file in the second storage pool is cleared, and this part of the data is garbage data. If it is after the metadata is changed, the valid data is the second storage pool. At this time, the first 4M data of the file in the first storage pool is cleared, and this part of the data is garbage data, thereby effectively improving the efficiency of file migration and reducing the probability of errors during file migration.

[0072] Based on the above embodiment, in order to improve the flexibility of the entire file migration and enhance the user experience, based on the above embodiment, the process of dividing the target data into the first sub-data and the second sub-data may include:

[0073] When receiving the target data to be written, determining whether the target data is larger than a preset capacity threshold;

[0074] If the target data is larger than a preset capacity threshold, the target data is divided into a first sub-data and a second sub-data;

[0075] If the target data is less than or equal to the preset capacity threshold, the target data is stored in the first storage pool.

[0076] In this embodiment, large files are stored in multiple data pools, and the data stored in the first storage pool is smaller than the data in the second storage pool, and small files are directly stored in the first storage pool, so that the data of file migration is very small, the migration rate is very fast, and it is not easy to make mistakes. As for the preset capacity threshold, it can be determined according to the distributed file storage system and the running business, and the value is used to determine whether the target data is a large file or a small file.

[0077] In order to make the technical personnel in the relevant field more clearly understand the technical solution of the present application, the present application is also combined with Figure 2 An illustrative example is given, where the first storage pool is a high-speed pool, the second storage pool is a slow-speed pool, the first 4 MB of data of a file is written to the high-speed pool, and the rest of the data is written to the slow-speed pool, which may include the following:

[0078] 1. If the file has only one data pool, the migration client reads all the file data from the first storage pool and writes the read data into the second storage pool; updates the metadata to the metadata server MDS, that is, changes the file's data pool from the first storage pool to the second storage pool. After the MDS completes the update, it responds to the client; after receiving the response, the client deletes the data in the first storage pool.

[0079] 2. If the file records two data pools: read the first 4M data of the file from the advanced pool, and determine whether the reading is abnormal based on the data reading response information. If the reading is abnormal, clean up the first 4M garbage data of the file in the slow pool; if the reading is successful, write the 4M data to the slow pool, and determine whether the writing is abnormal based on the data writing response information. If the writing is abnormal, clean up the first 4M garbage data of the file in the slow pool; if the writing is normal, update the metadata. After the MDS is updated, respond to the migration client. If the metadata modification fails, clean up the first 4M garbage data of the file in the slow pool; if the metadata modification is successful, delete the 4M data in the advanced pool. Migrate the client and delete the first 4M data of the file in the advanced pool. If the deletion fails, wait for the next time to clean up the first 4M garbage data of the file in the advanced pool.

[0080] From the above, it can be seen that this embodiment implements a file tiered large file data migration and exception handling method based on file multi-data pool storage. The first 4M data of the large file only exists in the advanced pool, and the rest is stored in the slow pool. The four steps of tiered reading, writing, metadata modification, and old data deletion are adapted separately, that is, the data operated in each step is only 4M, not all the data of the file. In the event of an exception in each stage, only 4M of garbage data is cleaned up, further enriching the file tiered migration scenarios, improving system stability, and enhancing product competitiveness.

[0081] It should be noted that there is no strict order of execution between the steps in this application. As long as they comply with the logical order, these steps can be executed simultaneously or in a certain preset order. Figure 1-Figure 2 This is just a schematic and does not mean that this is the only execution order.

[0082] The embodiment of the present invention also provides a corresponding device for the data migration method, which further makes the method more practical. The device can be described from the perspective of functional modules and hardware. The data migration device provided by the embodiment of the present invention is introduced below. The data migration device described below and the data migration method described above can be referred to each other.

[0083] From the perspective of functional modules, see Figure 3 , Figure 3 A structural diagram of a data migration device provided in an embodiment of the present invention in a specific implementation manner, the device may include:

[0084] The storage pool construction module 301 is used to pre-construct a first storage pool and a second storage pool with different storage performances, where the storage performance of the first storage pool is better than that of the second storage pool.

[0085] The data segmentation and storage module 302 is used to segment the target data into first sub-data and second sub-data, the first sub-data is stored in the first storage pool, and the second sub-data is stored in the second storage pool;

[0086] The data migration module 303 is used to write the first sub-data read from the first storage pool to the second storage pool when receiving the migration instruction of the target data, and send a request to the metadata server to update the storage location information of the target data;

[0087] The data deletion module 304 is configured to delete the first sub-data from the first storage pool when the metadata server responds that the update is completed, so as to complete the migration operation of the target data.

[0088] Optionally, in some implementations of the present embodiment, the above-mentioned data migration module 303 can be used to: read the first sub-data from the first storage pool; determine whether the first sub-data is successfully read from the first storage pool; if the first sub-data is successfully read from the first storage pool, write the first sub-data to the second storage pool; if the first sub-data is not successfully read from the first storage pool, delete the garbage data in the second storage pool and send a migration failure message.

[0089] As an optional implementation of this embodiment, the above-mentioned device may also include a deletion module, which is used to delete the garbage data in the second storage pool and send a migration failure message if the first sub-data is not successfully written to the second storage pool; if the metadata server fails to successfully update the metadata, the garbage data in the second storage pool is deleted and a migration failure message is sent; if the metadata server successfully updates the metadata, the garbage data in the first storage pool is deleted.

[0090] As another optional implementation of this embodiment, the above-mentioned device may also include a post-migration processing module, which is used to, if the data deletion operation in the first storage pool is successfully completed, the data migration is successful; if the data deletion operation in the first storage pool is not successfully completed, the data migration fails, and the garbage data in the first storage pool continues to be deleted when the data migration operation is performed next time.

[0091] Optionally, in some other implementations of the present embodiment, the above-mentioned data segmentation storage module 302 can be further used for: when receiving target data to be written, if the target data is larger than a preset capacity threshold, the target data is segmented into first sub-data and second sub-data; the space occupied capacity value of the first sub-data is smaller than that of the second sub-data; if the target data is less than or equal to the preset capacity threshold, the target data is stored in the first storage pool.

[0092] As an optional implementation of this embodiment, the data migration module 303 may also be used to send an instruction to update the data pool to the second storage pool to the metadata server.

[0093] The functions of the functional modules of the data migration device according to the embodiment of the present invention can be specifically implemented according to the method in the above method embodiment. The specific implementation process can refer to the relevant description of the above method embodiment, which will not be repeated here.

[0094] It can be seen from the above that the embodiments of the present invention can reduce the error probability of file migration while improving the efficiency of file migration.

[0095] The data migration device mentioned above is described from the perspective of functional modules. Furthermore, the present application also provides an electronic device, which is described from the perspective of hardware. Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application in one implementation manner. Figure 4 As shown, the electronic device includes a memory 40 for storing a computer program; and a processor 41 for implementing the steps of the data migration method mentioned in any of the above embodiments when executing the computer program.

[0096] Among them, the processor 41 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the processor 41 may also be a controller, a microcontroller, a microprocessor or other data processing chip. The processor 41 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 41 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 41 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 41 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0097] The memory 40 may include one or more computer-readable storage media, which may be non-transitory. The memory 40 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash storage devices. The memory 40 may be an internal storage unit of an electronic device in some embodiments, such as a hard disk of a server. The memory 40 may also be an external storage device of an electronic device in other embodiments, such as a plug-in hard disk equipped on a server, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. Further, the memory 40 may also include both an internal storage unit of an electronic device and an external storage device. The memory 40 may not only be used to store application software and various types of data installed in the electronic device, such as: the code of the program for executing the vulnerability processing method, etc., but may also be used to temporarily store data that has been output or is to be output. In this embodiment, the memory 40 is at least used to store the following computer program 401, wherein, after the computer program is loaded and executed by the processor 41, the relevant steps of the data migration method disclosed in any of the aforementioned embodiments can be implemented. In addition, the resources stored in the memory 40 may also include an operating system 402 and data 403, and the storage method may be temporary storage or permanent storage. The operating system 402 may include Windows, Unix, Linux, etc. The data 403 may include but is not limited to data corresponding to the data migration result.

[0098] In some embodiments, the electronic device may further include a display screen 42, an input / output interface 43, a communication interface 44 or a network interface, a power supply 45 and a communication bus 46. Among them, the display screen 42 and the input / output interface 43 such as a keyboard belong to the user interface, and the optional user interface may also include a standard wired interface, a wireless interface, etc. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device, etc. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface. The communication interface 44 may optionally include a wired interface and / or a wireless interface, such as a WI-FI interface, a Bluetooth interface, etc., which is usually used to establish a communication connection between the electronic device and other electronic devices. The communication bus 46 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0099] Those skilled in the art will understand that Figure 4 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than those shown in the figure, for example, it may also include a sensor 47 for realizing various functions.

[0100] The functions of the functional modules of the electronic device described in the embodiment of the present invention can be specifically implemented according to the method in the above method embodiment. The specific implementation process can refer to the relevant description of the above method embodiment, which will not be repeated here.

[0101] It can be seen from the above that the embodiments of the present invention can reduce the error probability of file migration while improving the efficiency of file migration.

[0102] It is understandable that if the data migration method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), electrically erasable programmable ROM, register, hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), magnetic memory, removable disk, CD-ROM, disk or optical disk, etc. Various media that can store program codes.

[0103] Based on this, an embodiment of the present invention further provides a readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps of the data migration method described in any of the above embodiments are performed.

[0104] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the hardware disclosed in the embodiments, including devices and electronic devices, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part.

[0105] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0106] The above is a detailed introduction to a data migration method, device, electronic device and readable storage medium provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A data migration method, characterized in that: include: Pre-building a first storage pool and a second storage pool with different storage performances, wherein the storage performance of the first storage pool is better than that of the second storage pool; dividing the target data into first sub-data and second sub-data; The first sub-data is stored in the first storage pool, and the second sub-data is stored in the second storage pool; When receiving the migration instruction of the target data, the first sub-data read from the first storage pool is written to the second storage pool, and a request for updating the storage location information of the target data is sent to the metadata server; wherein the migration instruction refers to the file layering; the metadata server is used to cache the distributed file system metadata, load the hierarchical index table, and migrate files; When the metadata server responds that the update is completed, deleting the first sub-data from the first storage pool to complete the migration operation of the target data; The step of writing the first sub-data read from the first storage pool to the second storage pool includes: Reading the first sub-data from the first storage pool; Determining whether the first sub-data is successfully read from the first storage pool; If the first sub-data is successfully read from the first storage pool, writing the first sub-data to the second storage pool; If the first sub-data is not successfully read from the first storage pool, the junk data in the second storage pool is deleted, and migration failure information is sent at the same time.

2. The data migration method according to claim 1, characterized in that: After writing the first sub-data read from the first storage pool to the second storage pool, the method includes: determining whether the first sub-data is successfully written into the second storage pool; If the first sub-data is not successfully written into the second storage pool, the junk data in the second storage pool is deleted, and migration failure information is sent at the same time.

3. The data migration method according to claim 1, characterized in that: After sending the request to the metadata server to update the storage location information of the target data, the method further includes: Determining whether the metadata server successfully updates the metadata; If the metadata server fails to update the metadata successfully, the junk data in the second storage pool is deleted and a migration failure message is sent; If the metadata server successfully updates the metadata, the junk data in the first storage pool is deleted.

4. The data migration method according to claim 3, characterized in that: After deleting the garbage data in the first storage pool, the method further includes: If the data deletion operation in the first storage pool is successfully completed, the data migration is successful; If the data deletion operation in the first storage pool is not successfully completed, the data migration fails, and the garbage data in the first storage pool will continue to be deleted when the data migration operation is performed next time.

5. The data migration method according to any one of claims 1 to 4, characterized in that: The step of dividing the target data into first sub-data and second sub-data includes: When receiving target data to be written, determining whether the target data is greater than a preset capacity threshold; If the target data is larger than a preset capacity threshold, the target data is divided into a first sub-data and a second sub-data; the space occupied capacity value of the first sub-data is smaller than that of the second sub-data; If the target data is smaller than or equal to a preset capacity threshold, the target data is stored in the first storage pool.

6. The data migration method according to claim 5, characterized in that: The sending of a request to the metadata server to update the storage location information of the target data includes: An instruction to update the data pool to the second storage pool is sent to the metadata server.

7. A data migration device, characterized in that: include: A storage pool construction module, used to pre-construct a first storage pool and a second storage pool with different storage performances, wherein the storage performance of the first storage pool is better than that of the second storage pool; A data segmentation storage module, used for segmenting the target data into first sub-data and second sub-data; The first sub-data is stored in the first storage pool, and the second sub-data is stored in the second storage pool; a data migration module, configured to, upon receiving a migration instruction for the target data, write the first sub-data read from the first storage pool to the second storage pool, and simultaneously send a request to a metadata server to update the storage location information of the target data; wherein the migration instruction refers to file stratification; and the metadata server is configured to cache distributed file system metadata, load a hierarchical index table, and migrate files; a data deletion module, configured to delete the first sub-data from the first storage pool when the metadata server responds that the update is completed, so as to complete the migration operation of the target data; Wherein, the data migration module is further used for: reading the first sub-data from the first storage pool; Determining whether the first sub-data is successfully read from the first storage pool; If the first sub-data is successfully read from the first storage pool, writing the first sub-data to the second storage pool; If the first sub-data is not successfully read from the first storage pool, the junk data in the second storage pool is deleted, and migration failure information is sent at the same time.

8. An electronic device, characterized in that: It comprises a processor and a memory, wherein the processor is used to implement the steps of the data migration method according to any one of claims 1 to 6 when executing a computer program stored in the memory.

9. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data migration method according to any one of claims 1 to 6 are implemented.

Images