Data migration method, apparatus, device, and medium

Abstract

Embodiments of the present application disclose a data migration method, device, equipment and medium, and relate to data processing technology. The method comprises: determining a candidate storage medium from candidate medium categories corresponding to a medium category to which a current storage medium of to-be-migrated data belongs according to the medium category; calculating migration benefits and migration losses of migration from the current storage medium to the candidate storage medium respectively, wherein the migration benefits and migration losses are used to quantify changes in data reading performance before and after migration; determining a target storage medium from the candidate storage medium according to the migration benefits and migration losses; and migrating the to-be-migrated data to the target storage medium. Embodiments of the present application provide a data migration method, device, equipment and medium, and realize automatic and accurate migration of data.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to data processing technology, and in particular, to a data migration method, device, equipment and medium. BACKGROUND

[0002] In a distributed storage system, there is a great difference in access mode of different types of data. For hot data, it is frequently read after being written, and the throughput and latency of writing and reading are both very high, and it is more suitable to be stored in SSD, NVM and other fast media. For cold data, it is basically not read after being written, and the requirement of throughput and latency of writing and reading is not very high, and it is more suitable to be stored in cold backup HDD, even magnetic tape. By realizing the differentiation of hot and cold data, and storing them using different media, the data storage cost can be greatly reduced without affecting the user access.

[0003] Currently, the following several schemes are generally used to solve the automatic transfer of hot and cold data:

[0004] (1) Different storage resource pools are established for hot and cold data, and different storage media are used for storage. In the same storage pool, mainly one kind of medium is used to store data. The user decides to write data to which resource pool, and uses file copy to complete the migration of hot and cold data.

[0005] (2) A unified resource pool is established for hot and cold data, and the user specifies the nature of the data when writing. The storage system stores the data in different media according to the user's indication.

[0006] (3) A unified resource pool is established for hot and cold data, and the access frequency of the user data is counted in the storage system after the user data is written, and the data is automatically migrated according to the access frequency in the recent period of time.

[0007] However, the above three schemes have the following problems:

[0008] Scheme (1) transfers the logic of hot and cold data to the user, which greatly increases the complexity of the user's logic. Moreover, when the user processes data transfer and migration, the storage system cannot distinguish between normal read and write and read and write generated by migration, so it cannot distinguish between the two scenarios, such as priority.

[0009] Scheme (2) can only be applied to scenarios where data hotness is determined, such as data backup business. Since the nature of the data is specified when writing, the storage system cannot automatically transfer.

[0010] Scheme (3) can solve the problem of transfer. However, it only considers the access frequency as a factor for the hotness of data. SUMMARY

[0011] Embodiments of the present application provide a data migration method, device, equipment and medium to realize automatic and accurate data migration.

[0012] In a first aspect, embodiments of the present application provide a data migration method, comprising:

[0013] determining a candidate storage medium from candidate medium categories corresponding to a medium category to which a current storage medium of the data to be migrated belongs;

[0014] calculating migration benefits and migration losses of migrating from the current storage medium to the candidate storage medium, respectively, wherein the migration benefits and migration losses are used to quantify changes in data reading performance before and after migration;

[0015] if it is determined that the candidate storage medium is a target storage medium according to the migration benefits and migration losses, migrating the data to be migrated to the target storage medium.

[0016] In a second aspect, embodiments of the present application further provide a data migration device, comprising:

[0017] a candidate storage determination module configured to determine a candidate storage medium from candidate medium categories corresponding to a medium category to which a current storage medium of the data to be migrated belongs;

[0018] a performance change calculation module configured to calculate migration benefits and migration losses of migrating from the current storage medium to the candidate storage medium, respectively, wherein the migration benefits and migration losses are used to quantify changes in data reading performance before and after migration;

[0019] a data migration module configured to migrate the data to be migrated to a target storage medium if it is determined that the candidate storage medium is the target storage medium according to the migration benefits and migration losses.

[0020] In a third aspect, embodiments of the present application further provide an equipment, comprising:

[0021] one or more processors;

[0022] a storage device configured to store one or more programs,

[0023] when the one or more programs are executed by the one or more processors, the one or more processors implement the data migration method as described in any of the embodiments of the present application.

[0024] In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the data migration method according to any one of the embodiments of the present application.

[0025] The embodiment of the present application calculates the migration benefit and the migration loss of migrating from the current storage medium to the candidate storage medium, respectively, wherein the migration benefit and the migration loss are used to quantify the change of the data reading performance before and after the migration; determines the target storage medium from the candidate storage media according to the migration benefit and the migration loss; and migrates the to-be-migrated data to the target storage medium to complete the automatic flow of the data. The benefit and the loss brought by the data migration are accurately calculated and compared, and the data migration can be more reasonably determined. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 is a flowchart of a data migration method provided by the embodiment one of the present application;

[0027] Figure 2 is a flowchart of a data migration method provided by the embodiment two of the present application;

[0028] Figure 3 is an execution schematic diagram of a data migration method provided by the embodiment three of the present application;

[0029] Figure 4 is a structural schematic diagram of a data migration device provided by the embodiment four of the present application;

[0030] Figure 5 is a structural schematic diagram of a device provided by the embodiment five of the present application. DETAILED DESCRIPTION

[0031] The present application will be further described below in conjunction with the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. In addition, it should be noted that, for the convenience of description, only the parts related to the present application are shown in the drawings, but not all the structures.

[0032] Embodiment one

[0033] Figure 1 is a flowchart of a data migration method provided by the embodiment one of the present application. The embodiment can be applicable to the case of accurately migrating and storing data. The method can be executed by a data migration device, which can be realized by software and / or hardware. Referring to Figure 1 , the data migration method provided by the embodiment includes:

[0034] S110, determining a candidate storage medium from candidate medium categories corresponding to a medium category to which a current storage medium of the data to be migrated belongs.

[0035] Specifically, if the current storage medium is a memory, the medium category to which the current storage medium belongs is a fast reading medium category; if the current storage medium is a disk, the medium category to which the current storage medium belongs is a delay reading medium category.

[0036] The fast reading medium category corresponds to the delay reading medium category.

[0037] Specifically, the determining of the candidate storage medium from the candidate medium categories corresponding to the medium category to which the current storage medium of the data to be migrated belongs comprises:

[0038] If the current storage medium of the data to be migrated belongs to the fast reading medium category, it is determined that the target storage medium belongs to the delay reading medium category, and a candidate storage medium is determined from the storage medium of the delay reading medium category; or

[0039] If the current storage medium of the data to be migrated belongs to the delay reading medium category, it is determined that the target storage medium belongs to the fast reading medium category, and a candidate storage medium is determined from the storage medium of the fast reading medium category.

[0040] S120, respectively calculating a migration benefit and a migration loss of migrating the data to be migrated from the current storage medium to the candidate storage medium.

[0041] The migration benefit and the migration loss are used to quantify the change of data reading performance before and after migration.

[0042] Specifically, the migration benefit and the migration loss of migrating the data to be migrated from the current storage medium to the candidate storage medium are calculated according to a change value of storage cost of migrating the data to be migrated from the current storage medium to the candidate storage medium, a change value of delay of migrating the data to be migrated from the current storage medium to the candidate storage medium, a data size of the data to be migrated, and an access frequency of the data to be migrated.

[0043] Typically, a product of the data size of the data to be migrated and the change value of migration storage cost from the current storage medium to the candidate storage medium is taken as the migration benefit.

[0044] S130, if the candidate storage medium is determined to be the target storage medium according to the migration benefit and the migration loss, migrating the data to be migrated to the target storage medium.

[0045] Specifically, the determining of the target storage medium from the candidate storage medium according to the migration benefit and the migration loss comprises:

[0046] If the migration benefit is greater than the migration loss, the candidate storage medium is determined as the target storage medium; or

[0047] If the migration benefit is greater than the migration loss, and the difference between the migration benefit and the migration loss is greater than a set threshold, the candidate storage medium is determined as the target storage medium

[0048] The technical scheme of the embodiment of the application, by calculating the migration benefit and the migration loss of migrating from the current storage medium to the candidate storage medium, wherein the migration benefit and the migration loss are used to quantify the change of data reading performance before and after migration; determining the target storage medium from the candidate storage medium according to the migration benefit and the migration loss; migrating the to-be-migrated data to the target storage medium to complete the automatic flow of data. Through the accurate calculation and comparison of the benefit and loss brought by data migration, the migration of data can be more reasonably determined.

[0049] To improve the loading speed of cold data, after the to-be-migrated data is migrated to the target storage medium, the method further comprises:

[0050] In response to a preloading request for the to-be-migrated data, determining whether the to-be-migrated data is cold data according to the medium category to which the target storage medium of the to-be-migrated data belongs;

[0051] If yes, reading the to-be-migrated data from the target storage medium and storing it in a fast access storage medium;

[0052] In response to a final loading request for the to-be-migrated data, reading the to-be-migrated data from the fast access storage medium and feeding back to the request end.

[0053] Embodiment two

[0054] Figure 2 is a flowchart of a data migration method provided by the second embodiment of the application. The present embodiment is an optional solution based on the above-mentioned embodiments. Referring to Figure 2 , the data migration method provided by the present embodiment comprises:

[0055] S210, determining a candidate storage medium from the candidate medium category corresponding to the medium category to which the current storage medium of the to-be-migrated data belongs according to the medium category to which the current storage medium of the to-be-migrated data belongs.

[0056] S220, determining a migration benefit according to the migration storage cost change value from the current storage medium to the candidate storage medium, the data size of the to-be-migrated data and the access mode of the to-be-migrated data.

[0057] The migration storage cost change value from the current storage medium to the candidate storage medium refers to a remaining value of a storage cost of unit data in the current storage medium minus a storage cost of unit data in the candidate storage medium.

[0058] Specifically, the access mode of the data to be migrated includes a log file access mode, an SST file access mode, and a distributed computing intermediate file access mode.

[0059] Specifically, the determining of the migration benefit according to the migration storage cost change value from the current storage medium to the candidate storage medium, the data size of the data to be migrated, and the access mode of the data to be migrated includes:

[0060] determining a benefit weight according to the access mode of the data to be migrated;

[0061] calculating a product of the benefit weight, the data size of the data to be migrated, and the migration storage cost change value from the current storage medium to the candidate storage medium;

[0062] determining the migration benefit according to the calculated product.

[0063] S230, determining a migration loss according to a migration latency change value from the current storage medium to the candidate storage medium, a data size of the data to be migrated, an access mode of the data to be migrated, and an access frequency of the data to be migrated.

[0064] The migration latency change value from the current storage medium to the candidate storage medium refers to a remaining value of an access duration of the data to be migrated in the current storage medium minus an access duration of the data to be migrated in the candidate storage medium.

[0065] Specifically, the determining of the migration loss according to the migration latency change value from the current storage medium to the candidate storage medium, the data size of the data to be migrated, the access mode of the data to be migrated, and the access frequency of the data to be migrated includes:

[0066] determining a loss weight according to the access mode of the data to be migrated;

[0067] calculating a product of the loss weight, the data size of the data to be migrated, the access frequency of the data to be migrated, and the latency change value from the current storage medium to the candidate storage medium;

[0068] determining the migration loss according to the calculated product.

[0069] Because in the access to the to-be-migrated data, in addition to including user-initiated access, system-automatically-initiated access is also included.

[0070] Therefore, to improve the determination accuracy of the data hotness, the determination of the access frequency comprises:

[0071] Obtaining access request information of the to-be-migrated data;

[0072] According to the request source parameter in the access request information, determining whether the access request is from a user;

[0073] Counting the access requests from the user;

[0074] Determining the access frequency according to the counting result.

[0075] S240, if it is determined that the candidate storage medium is the target storage medium according to the migration benefit and the migration loss, migrating the to-be-migrated data to the target storage medium.

[0076] The execution sequence of the steps in the embodiment is not limited, and S230 can be executed before S220.

[0077] The technical solution of the embodiment of the application can more accurately calculate and compare the benefit and loss brought by data migration by considering the access information of multiple dimensions of data, including the size of data, the access frequency of data, and the access mode of data, so as to perform reasonable data migration. In the determination of data migration based on the access information of multiple dimensions, the judgment of the hotness of data is implied. The hotness of data can be more accurately judged according to the access information of multiple dimensions. Furthermore, the storage medium of data can be changed in real time or quasi-real time along with the change of the hotness of data.

[0078] Embodiment three

[0079] Figure 3 is an execution schematic diagram of a data migration method provided by the third embodiment of the application. The embodiment is an optional solution based on the above-mentioned embodiments. Referring to Figure 3 , the data migration method provided by the embodiment comprises:

[0080] (1) Judgment of hot and cold data.

[0081] Through the analysis system independent of the single machine IO (read and write) engine, the IO information in block (data block) units is extracted, including: block size, access frequency, access pattern. The migration loss is mainly the increase of access delay. The migration benefit comes from the reduction of storage cost. The migration loss and migration benefit are calculated by combining the access information of the block, so as to reasonably judge whether the data needs to be migrated.

[0082] needMigrate=gain(size,accessFreq,accessPattern,storeCost)–lost(size,accessFreq,accessPattern,transCost)>threshold

[0083] Wherein, size: the size of the block, accessFreq: the access frequency of the block, accessPattern: the access pattern of the block, storeCost: representing the storage cost of the block, transCost: representing the transmission cost of the block access, threshold: representing the set threshold.

[0084] Specifically, if needMigrate is greater than threshold, it is determined that the hotness of the data to be migrated has changed, and the storage medium of the data to be migrated needs to be migrated.

[0085] (2) Data migration.

[0086] After obtaining the data that should be migrated, the analysis system will give feedback to the storage system, and the storage system will determine the specific data migration method and path. Possible scenarios include:

[0087] a. Multiple storage media exist on a single storage node: the single storage node completes the migration of data between different storage media.

[0088] b. There is only one storage medium on each storage node: multiple storage nodes cooperate to complete the data migration between different storage nodes through the network.

[0089] c. Each storage pool has only one storage medium: the separate scheduling module uses file-level reading and rewriting to complete the migration of data between different storage pools.

[0090] (3) Reading of cold data.

[0091] For cold data, since it is often stored in media with large capacity, low cost and high access delay, direct reading will have relatively high delay. Therefore, for this scenario, the following optimization scheme is proposed in this embodiment:

[0092] Providing a preloading interface: the upper layer calls the preloading interface before data access, informing the corresponding data storage node. After receiving the request, the storage node will immediately read the corresponding data into a medium with higher access speed, such as memory or cache in SSD. After receiving the real read request, it can be read from these media, thereby ensuring lower access latency.

[0093] In a distributed storage system, not all reads are initiated by end users. For such non-user-initiated reads, they should not be considered in the calculation of data hotness. Therefore, to filter out interfering access information and improve the accuracy of data hotness judgment, by providing a Flag mark of access directory in the read interface, the calling party can explicitly specify whether a read is initiated by an end user and whether it should be calculated in the data hotness, thereby improving the accuracy of data migration.

[0094] The technical solution of the embodiment of the application can better solve some problems of the original scheme, including: completing the automatic flow of data, realizing the real-time or quasi-real-time change of the storage medium of data with the change of the hotness of data; by considering the access information of multiple dimensions of data, including the size of data, the access frequency of data, and the access mode of data, the hotness of data can be more accurately judged. By accurately calculating and comparing the benefits and losses brought by data migration, the migration plan can be more reasonably determined. By providing an access destination mark, interfering access information can be filtered out, so that the judgment of data hotness is more accurate.

[0095] It should be noted that, after the technical teaching of the embodiment, those skilled in the art have the motivation to combine any one of the embodiments described in the above embodiments to realize the automatic and accurate migration of data.

[0096] Embodiment four

[0097] Figure 4 is a structural schematic diagram of a data migration device provided by the fourth embodiment of the application. Referring to Figure 4 The data migration device provided by the embodiment comprises a candidate storage determination module 10, a performance change calculation module 20, and a data migration module 30.

[0098] The candidate storage determination module 10 is configured to determine a candidate storage medium from a candidate medium category corresponding to a medium category to which a current storage medium of the data to be migrated belongs.

[0099] The performance change calculation module 20 is configured to calculate a migration benefit and a migration loss of migrating the to-be-migrated data from the current storage medium to the candidate storage medium, respectively, wherein the migration benefit and the migration loss are used to quantify the change of the data read performance before and after the migration.

[0100] The data migration module 30 is configured to migrate the to-be-migrated data to the target storage medium if it is determined that the candidate storage medium is the target storage medium according to the migration benefit and the migration loss.

[0101] The embodiment of the present application calculates a migration benefit and a migration loss of migrating the to-be-migrated data from the current storage medium to the candidate storage medium, respectively, wherein the migration benefit and the migration loss are used to quantify the change of the data read performance before and after the migration, determines a target storage medium from the candidate storage medium according to the migration benefit and the migration loss, and migrates the to-be-migrated data to the target storage medium, so as to complete the automatic flow of the data. The benefit and the loss brought by the data migration are accurately calculated and compared, so that the migration of the data can be more reasonably determined.

[0102] Further, the performance change calculation module comprises a benefit determination unit and a loss determination unit.

[0103] The benefit determination unit is configured to determine the migration benefit according to a migration storage cost change value from the current storage medium to the candidate storage medium, a data size of the to-be-migrated data, and an access mode of the to-be-migrated data.

[0104] The loss determination unit is configured to determine the migration loss according to a migration delay change value from the current storage medium to the candidate storage medium, the data size of the to-be-migrated data, the access mode of the to-be-migrated data, and an access frequency of the to-be-migrated data.

[0105] Further, the benefit determination unit is specifically configured to:

[0106] determine a benefit weight according to the access mode of the to-be-migrated data;

[0107] calculate a product of the benefit weight, the data size of the to-be-migrated data, and the migration storage cost change value from the current storage medium to the candidate storage medium;

[0108] determine the migration benefit according to the calculated product.

[0109] Further, the determination of the access frequency comprises:

[0110] obtaining access request information of the to-be-migrated data;

[0111] According to a request source parameter in the access request information, it is determined whether the access request is from a user or not;

[0112] The access requests from the user are counted.

[0113] According to the counting result, the access frequency is determined.

[0114] Further, the candidate storage determination module comprises a candidate storage determination unit.

[0115] The candidate storage determination unit is configured to, if the current storage medium of the to-be-migrated data belongs to the fast reading medium category, determine that the target storage medium belongs to the delay reading medium category, and determine the candidate storage medium from the storage medium of the delay reading medium category.

[0116] If the current storage medium of the to-be-migrated data belongs to the delay reading medium category, it is determined that the target storage medium belongs to the fast reading medium category, and the candidate storage medium is determined from the storage medium of the fast reading medium category.

[0117] Further, the device further comprises a cold data determination module, a data preloading module and a data feedback module.

[0118] The cold data determination module is configured to, in response to a preloading request of the to-be-migrated data, determine whether the to-be-migrated data is cold data according to a medium category to which the current storage medium of the to-be-migrated data belongs.

[0119] The data preloading module is configured to, if yes, read the to-be-migrated data from the current storage medium and store the to-be-migrated data in the fast access storage medium.

[0120] The data feedback module is configured to, in response to a final loading request of the to-be-migrated data, read the to-be-migrated data from the fast access storage medium and feed back to the request end.

[0121] The data migration device provided by the embodiment of the application can execute the data migration method provided by any embodiment of the application, and has the corresponding function modules and beneficial effects of the execution method.

[0122] Embodiment five

[0123] Figure 5 The structural schematic diagram of a device provided by the embodiment five of the application is shown. Figure 5 A block diagram of an exemplary device 12 suitable for use in implementing embodiments of the application is shown. Figure 5 The device 12 shown is merely an example and should not be taken as limiting the functionality or applicability of embodiments of the application.

[0124] As Figure 5As shown, device 12 is represented as a general-purpose computing device. Components of device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and a bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0125] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0126] Device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by device 12, including volatile and non-volatile media, removable and non-removable media.

[0127] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 5 Not shown; usually referred to as a "hard drive"). Although Figure 5 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0128] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0129] Device 12 can also communicate with one or more external devices 14 such as a keyboard, a pointing device, a display 24, etc.; other devices that enable a user to interact with device 12; and / or any devices (e.g., net cards, modems, etc.) that enable device 12 to communicate with one or more other computing devices. Such communication can occur via input / output (I / O) interface(s) 22. Still yet, device 12 can communicate with one or more networks, such as a local area network (LAN), a wide area network (WAN), and / or the Internet, through network adapter 20. As depicted, network adapter 20 communicates with the other components of device 12 via bus 18. It should be appreciated that although not shown, other hardware and / or software modules could be used in conjunction with device 12. Examples, include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0130] Processing unit 16 can execute various functions applications and data processing by running programs stored in system memory 28, such as implementing the data migration method provided by embodiments of the present application.

[0131] Embodiment six

[0132] Embodiment six of the present application also provides a computer readable storage medium, having stored thereon a computer program, which when executed by a processor implements the data migration method as claimed in any one of the embodiments of the present application, the method comprising:

[0133] determining a candidate storage medium from candidate medium categories corresponding to a medium category to which the current storage medium of the data to be migrated belongs, according to the medium category to which the current storage medium of the data to be migrated belongs;

[0134] calculating a migration benefit and a migration loss of migrating from the current storage medium to the candidate storage medium, respectively, wherein the migration benefit and the migration loss are used to quantify the change of data read performance before and after migration;

[0135] if the candidate storage medium is determined as the target storage medium according to the migration benefit and the migration loss, migrating the data to be migrated into the target storage medium.

[0136] The computer storage medium of the embodiments of the present application can adopt any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples (non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device.

[0137] The computer readable signal medium can include a propagated data signal with computer readable program code embodied therein. The propagated data signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium can be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport program code for use by or in connection with an instruction execution system, apparatus, or device.

[0138] Program code embodied on a computer readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0139] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In an embodiment of the present application, the remote computer can be a server or another desktop computer.

[0140] Note that the above merely describes preferred embodiments of the present application and the principles of the technology applied. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, modifications and substitutions can be made without departing from the scope of the present application. Therefore, although the present application has been described in detail through the above embodiments, the present application is not limited to the above embodiments, and can include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A data migration method, characterized by, The method comprises the following steps: determining a candidate storage medium from a candidate medium category corresponding to a medium category to which a current storage medium of the data to be migrated belongs; calculating a migration benefit and a migration loss of migrating the data to be migrated from the current storage medium to the candidate storage medium, wherein the migration benefit and the migration loss are used to quantify the change of data reading performance before and after migration; if the candidate storage medium is determined to be a target storage medium according to the migration benefit and the migration loss, migrating the data to be migrated to the target storage medium; wherein the calculating of the migration benefit and the migration loss of migrating the data to be migrated from the current storage medium to the candidate storage medium comprises: determining the migration benefit according to a migration storage cost change value from the current storage medium to the candidate storage medium, a data size of the data to be migrated and an access mode of the data to be migrated; determining the migration loss according to a migration delay change value from the current storage medium to the candidate storage medium, the data size of the data to be migrated, the access mode of the data to be migrated and an access frequency of the data to be migrated.

2. The method of claim 1, wherein, The determining of the migration benefit according to the migration storage cost change value from the current storage medium to the candidate storage medium, the data size of the data to be migrated and the access mode of the data to be migrated comprises: determining a benefit weight according to the access mode of the data to be migrated; calculating a product of the benefit weight, the data size of the data to be migrated and the migration storage cost change value from the current storage medium to the candidate storage medium; determining the migration benefit according to the calculated product.

3. The method of claim 1, wherein, The determination of the access frequency comprises: obtaining access request information of the data to be migrated; determining whether the access request is from a user according to a request source parameter in the access request information; counting the access request from the user; determining the access frequency according to the counting result.

4. The method of claim 1, wherein, The determining of the candidate storage medium from the candidate medium category corresponding to the medium category to which the current storage medium of the data to be migrated belongs comprises: if the current storage medium of the data to be migrated belongs to a fast reading medium category, determining that a target storage medium belongs to a delay reading medium category, and determining the candidate storage medium from the storage medium of the delay reading medium category; or if the current storage medium of the data to be migrated belongs to a delay reading medium category, determining that a target storage medium belongs to a fast reading medium category, and determining the candidate storage medium from the storage medium of the fast reading medium category.

5. The method of claim 1, wherein, The method further comprises: in response to a preloading request for the data to be migrated, determining whether the data to be migrated is cold data according to a medium category to which a current storage medium of the data to be migrated belongs; if yes, reading the data to be migrated from the current storage medium and storing the data to be migrated in a fast access storage medium; in response to a final loading request for the data to be migrated, reading the data to be migrated from the fast access storage medium and feeding back to a request end.

6. A data migration apparatus, characterized by comprising: The method comprises the following steps: The candidate storage determination module is configured to determine a candidate storage medium from candidate medium categories corresponding to a medium category to which a current storage medium of the to-be-migrated data belongs, according to the medium category to which the current storage medium of the to-be-migrated data belongs. The performance change calculation module is configured to calculate a migration benefit and a migration loss of migrating from the current storage medium to the candidate storage medium, respectively, wherein the migration benefit and the migration loss are used to quantify changes in data reading performance before and after migration. The data migration module is configured to migrate the to-be-migrated data to a target storage medium if the candidate storage medium is determined to be the target storage medium according to the migration benefit and the migration loss. The performance change calculation module includes: The benefit determination unit is configured to determine a migration benefit according to a migration storage cost change value from the current storage medium to the candidate storage medium, a data size of the to-be-migrated data, and an access mode of the to-be-migrated data. The loss determination unit is configured to determine a migration loss according to a migration delay change value from the current storage medium to the candidate storage medium, a data size of the to-be-migrated data, an access mode of the to-be-migrated data, and an access frequency of the to-be-migrated data.

7. The apparatus of claim 6, wherein, The benefit determination unit is specifically configured to: determine a benefit weight according to the access mode of the to-be-migrated data; calculate a product of the benefit weight, the data size of the to-be-migrated data, and the migration storage cost change value from the current storage medium to the candidate storage medium; and determine the migration benefit according to the calculated product.

8. The apparatus of claim 6, wherein, The determination of the access frequency includes: obtaining access request information of the to-be-migrated data; determining whether the access request is from a user according to a request source parameter in the access request information; counting the access request from the user; and determining the access frequency according to a statistical result.

9. The apparatus of claim 6, wherein, The candidate storage determination module includes: The candidate storage determination unit is configured to determine a target storage medium to belong to a delay reading medium category, and determine a candidate storage medium from storage media of the delay reading medium category, if the current storage medium of the to-be-migrated data belongs to a fast reading medium category; or The candidate storage determination unit is configured to determine a target storage medium to belong to a fast reading medium category, and determine a candidate storage medium from storage media of the fast reading medium category, if the current storage medium of the to-be-migrated data belongs to a delay reading medium category.

10. The apparatus of claim 6, wherein, The device further includes: The cold data determination module is configured to determine whether the to-be-migrated data is cold data according to a medium category to which a current storage medium of the to-be-migrated data belongs, in response to a preloading request for the to-be-migrated data. The data preloading module is configured to read the to-be-migrated data from the current storage medium and store the to-be-migrated data in a fast access storage medium if the to-be-migrated data is cold data. The data feedback module is configured to read the to-be-migrated data from the fast access storage medium and feed back the to-be-migrated data to a request end in response to a final loading request for the to-be-migrated data.

11. An apparatus, comprising: The device includes: one or more processors; a storage device configured to store one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement a data migration method as claimed in any one of claims 1-5.

12. A computer readable storage medium having stored thereon a computer program, characterized in that, The program, when executed by a processor, implements a data migration method as claimed in any one of claims 1-5.

Images