A caching load balancing method and apparatus

Abstract

A cache load balancing method and device provided by an embodiment of the present invention obtain a first load parameter of a first cache disk group and a second load parameter of a second cache disk group; when the first load parameter is greater than the second load parameter, determine a first quantity of file block groups to be migrated according to the difference between the first load parameter and the second load parameter; and migrate at least a part of the file block groups stored in the first cache disk group to the second cache disk group according to the first quantity. The present invention can realize data migration between different cache devices, so that the loads borne by different cache devices during actual operation are relatively balanced, and further the overall performance of the cache system can be fully exerted.

Description

Technical Field

[0001] The present invention relates to the technical field of data storage, and in particular, to a cache load balancing method and apparatus. Background Art

[0002] The current cache system is characterized in that each cache device (such as a solid state disk) serves externally through a fixed IP and port and occupies an exclusive thread. It can be considered that each cache device has an independent service instance. If a client needs to obtain data from a cache device, it needs to read in units of large file blocks, resulting in the read and write access of the entire large file block being located on this one service instance. In addition, the access to the large file block by different clients must be serial, and the access to the large file block by the same client is also serial.

[0003] In addition, the current cache system does not have a method for data migration between different cache devices, resulting in a situation where some cache devices store a large amount of hot data, while some cache devices store a small amount of hot data. This further causes the cache device storing a large amount of hot data to be frequently hit by the data read requests of the client. Since the access to the data is serial, the load on the cache device is large, even exceeding the maximum load of the cache device, resulting in a serious decline in the performance of the cache device and further reducing the overall performance of the cache system where the cache device is located. For the cache device storing a small amount of hot data, it is in an "idle" state most of the time, that is, the probability of the cache device being hit by the data read request of the client is extremely small, seriously wasting the resources of the cache device. It can be seen from this that the loads borne by different cache devices in the existing cache system during actual operation are unbalanced, resulting in a serious decline in the performance of the cache device with a high load and serious waste of the resources of the cache device with a low load, so that the overall performance of the cache system is not only not fully exerted, but also the performance will decline. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a cache load balancing method and apparatus to enable data migration between different cache devices, so that the loads borne by different cache devices during actual operation are relatively balanced, and further enable the overall performance of the cache system to be fully exerted. The specific technical solutions are as follows:

[0005] In a first aspect, a cache load balancing method includes:

[0006] Obtaining a first load parameter of a first cache disk group and a second load parameter of a second cache disk group;

[0007] When the first load parameter is greater than the second load parameter, determine a first quantity of file block groups to be migrated according to the difference between the first load parameter and the second load parameter;

[0008] According to the first quantity, migrate at least part of the file block groups stored in the first cache disk group to the second cache disk group.

[0009] Combined with the first aspect, in some optional embodiments, both the first load parameter and the second load parameter are storage capacity parameters, or both the first load parameter and the second load parameter are bandwidth parameters.

[0010] Combined with the first aspect, in some optional embodiments, the cache device includes a plurality of cache disk groups, the plurality of cache disk groups include the first cache disk group and the second cache disk group, the first cache disk group is the cache disk group with the largest load parameter among the plurality of cache disk groups, and the second disk cache group is the cache disk group with the smallest load parameter among the plurality of cache disk groups.

[0011] Optionally, in some optional embodiments, the step of when the first load parameter is greater than the second load parameter, determining a first quantity of file block groups to be migrated according to the difference between the first load parameter and the second load parameter includes:

[0012] When the first load parameter is greater than the second load parameter and the difference between the first load parameter and the second load parameter is greater than a preset difference, determine a first quantity of file block groups to be migrated according to the difference between the first load parameter and the second load parameter.

[0013] Combined with the first aspect, in some optional embodiments, the step of determining a first quantity of file block groups to be migrated according to the difference between the first load parameter and the second load parameter includes:

[0014] Determine a first quantity of file block groups to be migrated according to the difference between the first load parameter and the second load parameter, and according to the first total load capacity of the first cache disk group and the second total load capacity of the second cache disk group.

[0015] Combined with the previous embodiment, in some optional embodiments, the step of determining a first quantity of file block groups to be migrated according to the difference between the first load parameter and the second load parameter, and according to the first total load capacity of the first cache disk group and the second total load capacity of the second cache disk group includes:

[0016] According to the formula:

[0017] Calculate the first quantity x of the file block groups to be migrated, where the n 1 is the first total load capacity, and the u 1 is the used amount of the first total load capacity, and the n 2 is the second total load capacity, and the u 2 is the used amount of the second total load capacity.

[0018] Optionally, in some alternative embodiments, determining the first quantity of the file block groups to be migrated according to the gap between the first load parameter and the second load parameter, and according to the first total load capacity of the first cache disk group and the second total load capacity of the second cache disk group, includes:

[0019] According to the formula:

[0020] Calculate the first quantity x of the file block groups to be migrated, where the n 1 is the first total load capacity, and the u 1 is the used amount of the first total load capacity, and the n 2 is the second total load capacity, and the u 2 is the used amount of the second total load capacity, and p is a preset ratio.

[0021] Combined with the first aspect, in some alternative embodiments, migrating at least some of the file block groups stored in the first cache disk group to the second cache disk group according to the first quantity, includes:

[0022] Determine whether the first quantity is greater than a preset quantity. If so, migrate the preset quantity of file block groups stored in the first cache disk group to the second cache disk group. Otherwise, migrate the first quantity of file block groups stored in the first cache disk group to the second cache disk group.

[0023] Combined with the previous embodiment, in some alternative embodiments, migrating the preset quantity of file block groups stored in the first cache disk group to the second cache disk group includes: searching for the preset quantity of file block groups stored in the first cache disk group in a non-cache disk group; saving the found preset quantity of file block groups to the second cache disk group; deleting the preset quantity of file block groups stored in the first cache disk group;

[0024] and / or,

[0025] Said migrating the first quantity of file block groups stored in the first cache disk group to the second cache disk group includes: searching for the first quantity of file block groups stored in the first cache disk group in a non-cache disk group; saving the found first quantity of file block groups to the second cache disk group; and deleting the first quantity of file block groups stored in the first cache disk group.

[0026] In a second aspect, a cache load balancing device includes: a load parameter obtaining unit, a first quantity determining unit, and a migrating unit.

[0027] The load parameter obtaining unit is configured to obtain a first load parameter of a first cache disk group and a second load parameter of a second cache disk group.

[0028] The first quantity determining unit is configured to, when the first load parameter is greater than the second load parameter, determine a first quantity of file block groups to be migrated according to the difference between the first load parameter and the second load parameter.

[0029] The migrating unit is configured to migrate at least part of the file block groups stored in the first cache disk group to the second cache disk group according to the first quantity.

[0030] A cache load balancing method and device provided by an embodiment of the present invention obtain a first load parameter of a first cache disk group and a second load parameter of a second cache disk group; when the first load parameter is greater than the second load parameter, determine a first quantity of file block groups to be migrated according to the difference between the first load parameter and the second load parameter; and migrate at least part of the file block groups stored in the first cache disk group to the second cache disk group according to the first quantity. In combination with the first aspect, in some optional embodiments, both the first load parameter and the second load parameter are storage capacity parameters, or both the first load parameter and the second load parameter are bandwidth parameters. It can be seen from this that the present invention can realize data migration between different cache devices, so that the loads borne by different cache devices during actual operation are relatively balanced, and further the overall performance of the cache system can be fully exerted. Of course, any product or method implementing the present invention does not necessarily need to achieve all the above-mentioned advantages at the same time. Description of the Drawings

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0032] Figure 1 It is a flowchart of a client requesting hot data from a cache device provided by the present invention;

[0033] Figure 2 It is another flowchart of a client requesting hot data from a cache device provided by the present invention;

[0034] Figure 3 It is a flowchart of a cache load balancing method provided by the present invention;

[0035] Figure 4 It is a schematic structural diagram of a cache load balancing device provided by the present invention;

[0036] Figure 5 It is a schematic structural diagram of a device provided by the present invention. Detailed implementation manners

[0037] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0038] In an actual system, some data is frequently accessed data, and this frequently accessed data can be called hot data. For hot data, currently, it is usually to store the hot data in a cache device in a cache system. The cache device has a high data reading and writing speed and can quickly provide data for other devices or systems to improve the speed at which other devices or systems obtain data.

[0039] Specifically, a storage system with a cache system may include: a client device, a data server, a cache server, a cache device, an index server, and a non-cache device. Among them, the cache device can be a storage device with a high data reading and writing speed such as a memory or a solid-state storage disk. The non-cache device can be a storage device that is not a cache device such as a mechanical storage disk. The data reading and writing speed of the non-cache device is generally lower than that of the cache device. For example, the reading and writing speed of a mechanical storage disk is generally lower than that of a memory and a solid-state storage disk. Among them, the cache system may include: a cache server and a cache device.

[0040] The client device can be various electronic devices such as a computer, a mobile phone, a tablet computer, a wearable device, etc.

[0041] The data server can communicate with the client device through an SDK (Software Development Kit).

[0042] The cache server can store index information of the data stored in the cache device. When the data is stored in units of data blocks, the index information stored in the cache server can be: (file_id, block_id) => (disk_id, block_id). Here, file_id in (file_id, block_id) is the file ID, and block_id in (file_id, block_id) is the data block ID. disk_id in (disk_id, block_id) is the disk ID of the cache device, and block_id in (disk_id, block_id) is the disk block ID.

[0043] Among them, the storage space of a disk block is the same as the data volume of a complete data block.

[0044] Through the above index information, it can be determined whether each data block of each file is stored in the cache device and in which disk block of which disk in the cache device.

[0045] Similar to the cache server, the index server can store index information of the data stored in the non-cache device. When the data is stored in units of data blocks, the index information stored in the index server can be: (file_id, block_id) => (disk_id, block_id). Here, file_id in (file_id, block_id) is the file ID, and block_id in (file_id, block_id) is the data block ID. disk_id in (disk_id, block_id) is the disk ID of the non-cache device, and block_id in (disk_id, block_id) is the disk block ID.

[0046] Through the above index information, it can be determined whether each data block of each file is stored in the non-cache device and in which disk block of which disk in the non-cache device.

[0047] For easy understanding, a data reading process of a storage system with a cache system is disclosed as follows:

[0048] When the data requested by the client device for reading is saved in the cache device, the data reading process is as follows Figure 1 shown, including:

[0049] S1. The client device sends a data access request to the data server;

[0050] S2. The data server sends the data access request to the cache server;

[0051] S3. The cache server determines whether the data requested by the client device is saved in the cache device. If so, execute S4;

[0052] S4. The cache server sends the save address of the data requested by the client device in the cache device to the data server;

[0053] S5. After receiving the save address returned by the cache server, the data server obtains the data requested by the client device from this save address in the cache device;

[0054] S6. The data server returns the data requested by the client device to the client device.

[0055] When the data requested by the client device for reading is not saved in the cache device, the data reading process is as follows Figure 2 shown, including:

[0056] S1. The client device sends a data access request to the data server;

[0057] S2. The data server sends the data access request to the cache server;

[0058] S3. The cache server determines whether the data requested by the client device is saved in the cache device. If it is not saved in the cache device, execute step S7;

[0059] S7. The cache server sends a notice that the data requested by the client device is not saved in the cache device to the data server;

[0060] S8. After receiving the notice returned by the cache server that the data requested by the client device is not saved in the cache device, the data server sends the above data access request to the index server;

[0061] S9. The index server sends the save address of the data requested by the client device in the non-cache device to the data server;

[0062] S10. After receiving the save address returned by the index server, the data server obtains the data requested by the client device from this save address in the non-cache device;

[0063] S11. The data server returns the data requested by the client device to the client device.

[0064] After research by the present inventor, it is found that in the above processing flow, when the client requests data from the cache system, as long as the requested data is hot data that has been cached in the cache system, the cache system will return the data requested by the client to the client. When a large number of hot data continuously requested and accessed by the client are concentrated on one disk, due to the limited input / output performance of the disk and the network bandwidth limitation of the network card, the output of hot data will be restricted, seriously affecting the data transmission performance of the disk, and further reducing the ability of the cache system to quickly provide hot data. To solve the above problems, the present inventor provides the following solution.

[0065] As Figure 3 shown, the present invention provides a cache load balancing method, which may include:

[0066] S100. Obtain the first load parameter of the first cache disk group and the second load parameter of the second cache disk group;

[0067] It should be understood that the execution subject of the method of the present invention may be a device that is communicatively connected to both the first cache disk group and the second cache disk group, for example, it may be a cache server, and the present invention does not limit this.

[0068] It should be understood that the cache device herein may include multiple cache disk groups. The multiple cache disk groups may include the first cache disk group and the second cache disk group. The first cache disk group may be the cache disk group with the largest load parameter among the multiple cache disk groups, and the second disk cache group may be the cache disk group with the smallest load parameter among the multiple cache disk groups.

[0069] Optionally, the first cache disk group and the second cache disk group may be pre-partitioned cache disk groups. For example, multiple cache disks included in a cache device may be partitioned into multiple cache disk groups, and one cache disk group may include at least one cache disk. In this way, the cache data to be stored in the cache disk group can also be grouped. The current data processing methods generally take a complete data file as an object, store the entire data file determined as hot data in the cache device, or split a complete data file into multiple very small file blocks and store the file blocks determined as hot data in the cache device. Neither of the above two methods groups the data. The method provided by the present invention can divide the above-mentioned multiple very small file blocks into multiple file block groups, where one file block group includes multiple file blocks. The file blocks divided into the same file block group can be stored in the same cache device, for example, in the same cache disk group in the same cache device.

[0070] The first cache disk group and the second cache disk group may be connected to the same cache server or may be respectively connected to different cache servers. The present invention does not limit this.

[0071] It should be understood that the function of the cache disk group is to store cache data and provide a cache data reading service to the client when the client needs to access the cache data. Therefore, for a cache disk group, generally, the larger the storage capacity of a cache disk group, the better, and the faster the speed of providing cache data to the client. That is, in actual engineering, more attention is paid to these two indicators of storage capacity and bandwidth. The above-mentioned storage capacity may refer to the current available storage capacity of the cache disk group, and the bandwidth may refer to the speed at which the cache disk group provides cache data to the client. The present invention does not limit this.

[0072] Optionally, both the first load parameter and the second load parameter may be storage capacity parameters, or both the first load parameter and the second load parameter may be bandwidth parameters. Optionally, the storage capacity parameter may include at least one of: storage capacity usage ratio, storage capacity usage amount, storage capacity unused ratio, and storage capacity unused amount. The bandwidth parameter may include at least one of: bandwidth usage ratio, bandwidth usage amount, bandwidth unused ratio, and bandwidth unused amount.

[0073] Optionally, both the first load parameter and the second load parameter are parameters used to describe the storage capacity of the first cache disk group and the second cache disk group, or are parameters used to describe the data reading speed of the first cache disk group and the second cache disk group.

[0074] Optionally, the first load parameter and the second load parameter may have the same parameter type. For example, if the first load parameter is the storage capacity usage ratio, the second load parameter may also be the storage capacity usage ratio; if the first load parameter is the bandwidth usage ratio, the second load parameter may also be the bandwidth usage ratio. If the first load parameter is a combination of multiple parameters, the second load parameter may also be a combination of the above multiple parameters, and the present invention does not limit this.

[0075] It should be understood that the first load parameter and the second load parameter should have the same parameter type so that the first load parameter and the second load parameter are comparable. Further, when performing data migration subsequently, the result obtained by comparing the gap between the first load parameter and the second load parameter is more valuable for reference, and the present invention does not limit this.

[0076] S200. When the first load parameter is greater than the second load parameter, determine the first quantity of the file block groups to be migrated according to the gap between the first load parameter and the second load parameter;

[0077] It should be understood that if both the first load parameter and the second load parameter are storage capacity parameters, by comparing the first load parameter and the second load parameter, it can be obtained which of the first cache disk group and the second cache disk group has a larger current storage capacity and which has a smaller current storage capacity, so as to subsequently migrate part of the cached data of the cache disk group with the smaller current storage capacity to the cache disk group with the larger current storage capacity, thereby balancing the cached data in the first cache disk group and the second cache disk group, and at the same time balancing the current storage capacity of the first cache disk group and the second cache disk group.

[0078] It should be understood that if both the first load parameter and the second load parameter are bandwidth parameters, by comparing the first load parameter and the second load parameter, it can be obtained which of the first cache disk group and the second cache disk group has a larger current available bandwidth and which has a smaller current available bandwidth, so as to subsequently migrate part of the cached data of the cache disk group with the smaller current available bandwidth to the cache disk group with the larger current available bandwidth, thereby balancing the cached data in the first cache disk group and the second cache disk group, and at the same time balancing the bandwidth of the first cache disk group and the second cache disk group.

[0079] Optionally, the first load parameter and the second load parameter may also be a combination of storage capacity parameters and bandwidth parameters, that is, comprehensively referring to the storage capacity parameters and bandwidth parameters to determine which cached data is migrated to which. In this way, the obtained migration result is more reasonable. That is, from the two aspects of the current storage capacity and bandwidth, the cached data of the first cache disk group and the second cache disk group is more balanced, and the present invention does not limit this.

[0080] Optionally, step S200 may include:

[0081] When the first load parameter is greater than the second load parameter and the difference between the first load parameter and the second load parameter is greater than a preset difference, determine a first quantity of file block groups to be migrated according to the difference between the first load parameter and the second load parameter.

[0082] It should be understood that the present invention can balance the data in the first cache disk group and the second cache disk group. However, if the difference between the first load parameter and the second load parameter is not very large, the cache data in the first cache disk group and the second cache disk group may not need to be balanced. Therefore, a preset difference can be set in advance. If the difference between the first load parameter and the second load parameter is greater than the preset difference, the first quantity of file block groups to be migrated can be determined. Otherwise, this process can be ended to execute other processes, and the present invention does not limit other processes.

[0083] It should be understood that when the first load parameter is greater than the second load parameter and the difference between the first load parameter and the second load parameter is greater than the preset difference, it means that part of the cache data in one of the first cache disk group and the second cache disk group can be migrated to the other cache disk group. However, it is still necessary to further determine how much cache data needs to be migrated to make the data in the first cache disk group and the second cache disk group relatively balanced, that is, it is necessary to determine the first quantity of file block groups to be migrated.

[0084] Optionally, the first quantity may represent the number of file blocks included in the file block groups to be migrated, and the present invention does not limit this.

[0085] Optionally, the difference between the first load parameter and the second load parameter may be the absolute value of the difference between the numerical values of these two parameters, and the present invention does not limit this.

[0086] Optionally, determining the first quantity of file block groups to be migrated according to the difference between the first load parameter and the second load parameter in step S200 may include:

[0087] Determine the first quantity of file block groups to be migrated according to the difference between the first load parameter and the second load parameter, and according to the first total load capacity of the first cache disk group and the second total load capacity of the second cache disk group.

[0088] It should be understood that for any cache disk group, after the cache disk grouping is completed, the maximum amount of data it can store, that is, its total load capacity, can be fixed. Therefore, the first total load capacity and the second total load capacity can be parameters obtained after the corresponding cache disk grouping is completed, and the cache server can obtain not only the first total load capacity and the second total load capacity, but also the total load capacity of the cache disk groups of other cache devices communicatively connected to the cache server. Of course, the first total load capacity can also be obtained when obtaining the first load parameter, or the first load parameter can be obtained first and then the first total load capacity, or the first total load capacity can be obtained first and then the first load parameter. The present invention does not limit the order of obtaining the first load parameter and the first total load capacity, nor does it limit the order of obtaining the second load parameter and the second total load capacity.

[0089] It should be understood that the first total load capacity can, to a certain extent, illustrate the storage capacity of the first cache disk group. The second total load capacity can also, to a certain extent, illustrate the storage capacity of the second cache disk group. Determining the first quantity of the file block groups to be migrated based on the storage capacities of the first cache disk group and the second cache disk group is relatively scientific and can reasonably balance the cached data of the first cache disk group and the second cache disk group.

[0090] Optionally, the determining the first quantity of the file block groups to be migrated according to the difference between the first load parameter and the second load parameter, and according to the first total load capacity of the first cache disk group and the second total load capacity of the second cache disk group, may include:

[0091] According to the formula: Calculating to obtain the first quantity x of the file block groups to be migrated, where the n 1 is the first total load capacity, the u 1 is the used amount of the first total load capacity, the n 2 is the second total load capacity, and the u 2 is the used amount of the second total load capacity.

[0092] It should be understood that calculating the first quantity through the above formula is only an optional implementation manner. Any implementation manner obtained by making appropriate modifications based on the above formula or any implementation manner of calculating the first quantity according to the first total load capacity and the second total load capacity belongs to the protection scope of the present invention, and the present invention does not limit this.

[0093] Optionally, the first quantity calculated through the above formula may be the total quantity of file blocks included in the file block group to be migrated, that is, all the file blocks to be migrated. In subsequent steps, all the file blocks to be migrated may be migrated, either all at once or in multiple batches. The present invention does not limit this.

[0094] The following will provide an implementation manner of migrating the file block group to be migrated in multiple batches: Optionally, determining the first quantity of the file block group to be migrated according to the difference between the first load parameter and the second load parameter, and according to the first total load capacity of the first cache disk group and the second total load capacity of the second cache disk group, may include:

[0095] According to the formula: Calculate to obtain the first quantity x of the file block group to be migrated, where the n 1 is the first total load capacity, the u 1 is the used amount of the first total load capacity, the n 2 is the second total load capacity, the u 2 is the used amount of the second total load capacity, and the p is a preset ratio.

[0096] It should be understood that the first quantity in this implementation manner may be the quantity of file blocks that can be migrated at a single time during the process of migrating the file block group to be migrated in multiple batches. That is, the quantity of file blocks migrated each time is calculated through a preset ratio, and this preset ratio can be set according to actual needs. The present invention does not limit this.

[0097] Optionally, the value range of the preset ratio may be in the range from 0% to 100%. The present invention does not limit this.

[0098] It should be understood that by migrating the file block group to be migrated in multiple batches, it is possible to avoid the communication resources being occupied for a long time during the process of migrating the file block group, resulting in the communication of other processes being blocked for a long time and seriously affecting the progress of other processes, which is relatively reasonable. For example, the device executing the method of the present invention may, after executing the process of migrating a part of the file blocks of the file block group once, first execute at least one other process (a process other than migrating the file block group), and then return to execute the process of migrating a part of the file blocks of the file block group next time, and so on. The present invention does not limit other processes.

[0099] It should be understood that there are two solutions for migrating the file block group to be migrated in multiple batches:

[0100] First, the above n 1 、u 1 、n 2 and u2 Parameters such as these will not be updated until all the file blocks of the file block group to be migrated have been migrated, and will remain unchanged. That is, in multiple batches, the number of file blocks migrated each time can be the same.

[0101] Second, in multiple batches, after each migration is completed, the above-mentioned n 1 , u 1 , n 2 and u 2 and other parameters will all be updated. That is, when calculating the first quantity each time, the above-mentioned n 1 , u 1 , n 2 and u 2 and other parameters are all different from the previous values. Therefore, the first quantity calculated before each migration may also be different. This method can avoid over-migration. That is, as each migration is completed, the gap between the quantity of cached data stored in the first cache disk group and the quantity of cached data stored in the second cache disk group is gradually reduced. At this time, the number of file blocks migrated each time can be gradually reduced, that is, the first quantity can show a gradually decreasing trend.

[0102] S300. According to the first quantity, migrate at least part of the file block group saved in the first cache disk group to the second cache disk group.

[0103] It should be understood that the first quantity is generally less than the number of file blocks saved in the first cache disk group. Therefore, whether all the file block groups with the quantity of the first quantity are migrated to the second cache disk group or part of the file blocks of the file block group with the quantity of the first quantity are migrated to the second cache disk group, it can be understood that at least part of the file block group saved in the first cache disk group is migrated to the second cache disk group.

[0104] Optionally, what the present invention realizes is to balance the cached data of the first cache disk group and the second cache disk group. Therefore, generally, the first quantity will not be greater than or equal to the number of file blocks saved in the first cache disk group. Otherwise, after migrating the file block group with the first quantity saved in the first cache disk group to the second cache disk group, it may cause the cached data cached in the first cache disk group to be empty, and the effect of balancing the cached data of the first cache disk group and the second cache disk group is not achieved.

[0105] Optionally, the step S300 may include:

[0106] Judge whether the first quantity is greater than a preset quantity. If so, migrate the preset quantity of file block groups saved in the first cache disk group to the second cache disk group. Otherwise, migrate the file block group with the first quantity saved in the first cache disk group to the second cache disk group.

[0107] It should be understood that since the migration of the file block group involves subsequent steps of finding the first quantity of file block groups saved in the first cache disk group from the non-cache disk group; saving the found first quantity of file block groups to the second cache disk group; and deleting the first quantity of file block groups saved in the first cache disk group. Among them, the speed of finding and obtaining the first quantity of file block groups from the non-cache disk group is limited by the data reading performance and communication bandwidth of the non-cache disk group; the speed of saving the first quantity of file block groups to the second cache disk group is limited by the data writing performance and communication bandwidth of the second cache disk group. Therefore, an upper limit, that is, the preset quantity mentioned in this article, can be set for each migration.

[0108] It should be understood that the preset quantity can be a value obtained through experimental determination, and the present invention does not limit this.

[0109] Optionally, when the first quantity calculated this time is greater than or equal to the preset quantity, it is determined that the number of file blocks included in the file block group migrated this time is the preset quantity, otherwise it is determined that the number of file blocks included in the file block group migrated this time is the first quantity, and the present invention does not limit this.

[0110] Optionally, the migration of the preset quantity of file block groups saved in the first cache disk group to the second cache disk group may include: finding the preset quantity of file block groups saved in the first cache disk group in the non-cache disk group; saving the found preset quantity of file block groups to the second cache disk group; and deleting the preset quantity of file block groups saved in the first cache disk group;

[0111] and / or,

[0112] The migration of the first quantity of file block groups saved in the first cache disk group to the second cache disk group includes: finding the first quantity of file block groups saved in the first cache disk group in the non-cache disk group; saving the found first quantity of file block groups to the second cache disk group; and deleting the first quantity of file block groups saved in the first cache disk group.

[0113] As Figure 4 shown, the present invention provides a cache load balancing device, including: a load parameter obtaining unit 100, a first quantity determining unit 200, and a migration unit 300;

[0114] The load parameter obtaining unit 100 is configured to obtain the first load parameter of the first cache disk group and the second load parameter of the second cache disk group;

[0115] The first quantity determination unit 200 is configured to, when the first load parameter is greater than the second load parameter, determine a first quantity of file block groups to be migrated according to the difference between the first load parameter and the second load parameter;

[0116] The migration unit 300 is configured to migrate at least part of the file block groups saved in the first cache disk group to the second cache disk group according to the first quantity.

[0117] Optionally, in some alternative embodiments, the first quantity determination unit 200 includes: a first sub-unit;

[0118] The first sub-unit is configured to, when the first load parameter is greater than the second load parameter and the difference between the first load parameter and the second load parameter is greater than a preset difference, determine a first quantity of file block groups to be migrated according to the difference between the first load parameter and the second load parameter.

[0119] Optionally, in some alternative embodiments, the first quantity determination unit 200 includes: a second sub-unit;

[0120] The second sub-unit is configured to determine a first quantity of file block groups to be migrated according to the difference between the first load parameter and the second load parameter and according to a first total load capacity of the first cache disk group and a second total load capacity of the second cache disk group.

[0121] Optionally, in some alternative embodiments, the second sub-unit includes: a third sub-unit;

[0122] The third sub-unit is configured to execute according to the formula:

[0123] Calculate to obtain a first quantity x of file block groups to be migrated, where the n 1 is the first total load capacity, the u 1 is the used amount of the first total load capacity, the n 2 is the second total load capacity, the u 2 is the used amount of the second total load capacity, and the p is a preset ratio.

[0124] Optionally, in some alternative embodiments, the migration unit 300 includes: a fourth sub-unit, a fifth sub-unit, and a sixth sub-unit;

[0125] The fourth subunit is configured to determine whether the first quantity is greater than a preset quantity. If so, the fifth subunit is triggered; otherwise, the sixth subunit is triggered.

[0126] The fifth subunit is configured to migrate the preset quantity of file block groups saved in the first cache disk group to the second cache disk group.

[0127] The sixth subunit is configured to migrate the first quantity of file block groups saved in the first cache disk group to the second cache disk group.

[0128] Optionally, in some alternative embodiments, the fifth subunit includes: a first search subunit, a first save subunit, and a first delete subunit.

[0129] The first search subunit is configured to search for the preset quantity of file block groups saved in the first cache disk group in a non-cache disk group.

[0130] The first save subunit is configured to save the found preset quantity of file block groups to the second cache disk group.

[0131] The first delete subunit is configured to delete the preset quantity of file block groups saved in the first cache disk group.

[0132] And / or

[0133] The sixth subunit includes: a second search subunit, a second save subunit, and a second delete subunit.

[0134] The second search subunit is configured to search for the first quantity of file block groups saved in the first cache disk group in a non-cache disk group.

[0135] The second save subunit is configured to save the found first quantity of file block groups to the second cache disk group.

[0136] The second delete subunit is configured to delete the first quantity of file block groups saved in the first cache disk group.

[0137] The cache load balancing device includes a processor and a memory. The above load parameter obtaining unit 100, first quantity determining unit 200, migration unit 300, etc. are all stored in the memory as program units, and the processor executes the above program units stored in the memory to implement corresponding functions.

[0138] The processor contains cores, and the cores retrieve corresponding program units from the memory. One or more cores can be set. By adjusting the core parameters, data migration can be performed between different cache devices, so that the loads borne by different cache devices during actual operation are relatively balanced, and further the overall performance of the cache system can be fully exerted.

[0139] An embodiment of the present invention provides a storage medium, on which a program is stored, and when the program is executed by a processor, the cache load balancing method is implemented.

[0140] An embodiment of the present invention provides a processor, which is used to run a program. When the program runs, the cache load balancing method is executed.

[0141] As Figure 5 shown, an embodiment of the present invention provides a device 70, which includes at least one processor 701, at least one memory 702 connected to the processor 701, and a bus 703; wherein, the processor 701 and the memory 702 complete communication with each other through the bus 703; the processor 701 is used to call program instructions in the memory 702 to execute the above-mentioned cache load balancing method. The device herein can be a server, a PC, a PAD, a mobile phone, etc.

[0142] The present application also provides a computer program product, which is suitable for executing a program initialized with the steps included in the above-mentioned cache load balancing method when executed on a data processing device.

[0143] This application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0144] In a typical configuration, a device includes one or more processors (CPUs), a memory, and a bus. The device may also include an input / output interface, a network interface, etc.

[0145] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip. The memory is an example of a computer-readable medium.

[0146] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0147] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0148] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. It should also be noted that the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0149] Each embodiment in this specification is described in a related manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for the related parts, reference can be made to the partial description of the method embodiment.

[0150] The above are only the embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A cache load balancing method, characterized in that, comprising: obtaining a first load parameter of a first cache disk group and a second load parameter of a second cache disk group; when the first load parameter is greater than the second load parameter, determining a first quantity of file block groups to be migrated according to the difference between the first load parameter and the second load parameter; according to the first quantity, migrating at least part of the file block groups stored in the first cache disk group to the second cache disk group; the step of migrating at least part of the file block groups stored in the first cache disk group to the second cache disk group according to the first quantity includes: judging whether the first quantity is greater than a preset quantity, if so, migrating the preset quantity of file block groups stored in the first cache disk group to the second cache disk group, otherwise, migrating the first quantity of file block groups stored in the first cache disk group to the second cache disk group; the step of determining a first quantity of file block groups to be migrated according to the difference between the first load parameter and the second load parameter includes: determining a first quantity of file block groups to be migrated according to the difference between the first load parameter and the second load parameter and according to a first total load capacity of the first cache disk group and a second total load capacity of the second cache disk group; the step of determining a first quantity of file block groups to be migrated according to the difference between the first load parameter and the second load parameter and according to a first total load capacity of the first cache disk group and a second total load capacity of the second cache disk group includes: According to the formula: Calculate to obtain the first quantity x of the file block groups to be migrated, where the n 1 is the first total load capacity, and the u 1 is the used amount of the first total load capacity, and the n 2 is the second total load capacity, and the u 2 is the used amount of the second total load capacity.

2. The method according to claim 1, characterized in that, both the first load parameter and the second load parameter are storage capacity parameters, or both the first load parameter and the second load parameter are bandwidth parameters.

3. The method according to claim 1, characterized in that, a cache device includes a plurality of cache disk groups, the plurality of cache disk groups include the first cache disk group and the second cache disk group, the first cache disk group is the cache disk group with the largest load parameter among the plurality of cache disk groups, and the second cache disk group is the cache disk group with the smallest load parameter among the plurality of cache disk groups.

4. The method according to claim 1 or 3, characterized in that, the step of when the first load parameter is greater than the second load parameter, determining a first quantity of file block groups to be migrated according to the difference between the first load parameter and the second load parameter includes: when the first load parameter is greater than the second load parameter and the difference between the first load parameter and the second load parameter is greater than a preset difference, determining a first quantity of file block groups to be migrated according to the difference between the first load parameter and the second load parameter.

5. The method according to claim 1, characterized in that, Determining a first quantity of file block groups to be migrated according to the difference between the first load parameter and the second load parameter, and according to the first total load capacity of the first cache disk group and the second total load capacity of the second cache disk group, includes: According to the formula: Calculate the first quantity x of the file block groups to be migrated, where the n 1 is the first total load capacity, the u 1 is the used amount of the first total load capacity, the n 2 is the second total load capacity, the u 2 is the used amount of the second total load capacity, and p is a preset ratio.

6. The method according to claim 1, wherein, Migrating the preset quantity of file block groups saved in the first cache disk group to the second cache disk group includes: searching for the preset quantity of file block groups saved in the first cache disk group in a non-cache disk group; saving the found preset quantity of file block groups to the second cache disk group; deleting the preset quantity of file block groups saved in the first cache disk group; and / or, Migrating the first quantity of file block groups saved in the first cache disk group to the second cache disk group includes: searching for the first quantity of file block groups saved in the first cache disk group in a non-cache disk group; saving the found first quantity of file block groups to the second cache disk group; deleting the first quantity of file block groups saved in the first cache disk group.

7. A cache load balancing device, wherein, includes: A load parameter obtaining unit, a first quantity determining unit, and a migration unit; The load parameter obtaining unit is configured to obtain a first load parameter of a first cache disk group and a second load parameter of a second cache disk group; The first quantity determining unit is configured to, when the first load parameter is greater than the second load parameter, determine a first quantity of file block groups to be migrated according to the difference between the first load parameter and the second load parameter; The migration unit is configured to migrate at least part of the file block groups saved in the first cache disk group to the second cache disk group according to the first quantity; The migration unit includes: a second sub-unit, a third sub-unit, a fourth sub-unit, a fifth sub-unit, and a sixth sub-unit; The second sub-unit is configured to determine a first quantity of file block groups to be migrated according to the difference between the first load parameter and the second load parameter, and according to the first total load capacity of the first cache disk group and the second total load capacity of the second cache disk group; The third sub-unit is configured to execute according to the formula: Calculate to obtain a first quantity of file block groups to be migrated, where the n 1 is the first total load capacity, and the u 1 is the used amount of the first total load capacity, and the n 2 is the second total load capacity, and the u 2 is the used amount of the second total load capacity; The fourth sub-unit is configured to determine whether the first quantity is greater than a preset quantity, and if so, trigger the fifth sub-unit, otherwise, trigger the sixth sub-unit; The fifth sub-unit is configured to migrate the preset quantity of file block groups saved in the first cache disk group to the second cache disk group; The sixth sub-unit is configured to migrate the first quantity of file block groups saved in the first cache disk group to the second cache disk group.

Images