Hybrid storage system resource scheduling method and device, electronic equipment and storage medium
By calculating file scores based on random access frequency, access frequency, file size, and read operation ratio in a hybrid storage system, the advantages of both mechanical hard drives and SSDs are combined, solving the problems of slow mechanical hard drives and high cost of SSDs, improving data access speed and reducing storage device costs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING KINGSOFT CLOUD NETWORK TECH CO LTD
- Filing Date
- 2024-12-31
- Publication Date
- 2026-06-30
AI Technical Summary
Mechanical hard drives have slow data access speeds, while SSDs, although fast, are expensive and have small capacities, which limits their widespread adoption.
A resource scheduling method for a hybrid storage system is designed. By obtaining the random access frequency, access popularity, file size, and read operation ratio of the file to be scheduled, a file score is calculated, and the file is stored in a suitable location on a mechanical hard drive or a solid-state drive according to the score, so as to take advantage of each.
This approach combines the advantages of both hard disk drives (HDDs) and SSDs, while simultaneously reducing storage device costs and improving data access speeds.
Smart Images
Figure CN122308714A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus, electronic device and storage medium for resource scheduling of a hybrid storage system. Background Technology
[0002] A mechanical hard drive (HDD), a traditional type of hard drive, mainly consists of several parts: platters, read / write heads, a platter spindle and control motor, a head controller, a data converter, and an interface. The advantages of an HDD are its low cost and large capacity; its disadvantage is its relatively slow data access speed.
[0003] To address the slow data access speed of mechanical hard drives (HDDs), solid-state drives (SSDs) have become increasingly popular in computers due to their significantly faster data access speeds. However, compared to HDDs, SSDs are more expensive and have smaller capacities, which limits their widespread adoption.
[0004] Therefore, designing a storage system that combines the advantages of both hard disk drives (HDDs) and SSDs, thereby reducing storage device costs while improving data access speed, has become an urgent problem to be solved. Summary of the Invention
[0005] This application provides a hybrid storage system resource scheduling method, apparatus, electronic device, and storage medium to realize a storage system that combines the advantages of both hard disk drives (HDDs) and SSDs.
[0006] Firstly, this application provides a resource scheduling method for a hybrid storage system, including:
[0007] Obtain a set of files to be scheduled in a hybrid storage system, wherein the hybrid storage system includes mechanical hard disks and solid-state drives;
[0008] For each file in the set of files to be scheduled, determine the random access degree, access popularity, file size and read operation ratio of the file to be scheduled in the current period, wherein the read operation ratio is used to characterize the proportion of read operations to the total access volume;
[0009] The file score corresponding to the file to be scheduled is determined based on the random access degree, the access popularity, the file size, and the read operation ratio.
[0010] All files to be scheduled are stored in their corresponding storage locations based on their file scores.
[0011] In one possible implementation, storing all files to be scheduled to their corresponding storage locations based on the file score of each file to be scheduled includes:
[0012] Sort all files to be scheduled in descending order of their corresponding file scores;
[0013] The first preset number of files to be scheduled, which are ranked first, are stored in the solid-state drive;
[0014] The second preset number of files to be scheduled, which are ranked lower, are stored in the mechanical hard disk.
[0015] In one possible implementation, determining the random access degree corresponding to the file to be scheduled within the current period includes:
[0016] Within the current cycle, the access status of the files to be scheduled is monitored in real time;
[0017] Each time the file to be scheduled is accessed, it is determined whether the current access is consecutive to the previous access;
[0018] If the current access is not consecutive with the previous access, a step value is added to the access degree assignment corresponding to the file to be scheduled to obtain the current random access degree. The initial value of the access degree assignment is zero at the beginning of the current period.
[0019] In one possible implementation, determining the access popularity of the file to be scheduled within the current period includes:
[0020] Determine at least one unit of duration contained in the current period;
[0021] For each unit of time, determine the number of accesses to the file to be scheduled within that unit of time;
[0022] The access popularity is obtained by summing the access counts corresponding to multiple unit durations.
[0023] In one possible implementation, determining the file score corresponding to the file to be scheduled based on the random access degree, the access popularity, the file size, and the read operation ratio includes:
[0024] The access dispersion is calculated based on the access popularity and the number of accesses to the scheduled file within each unit of time.
[0025] The file score corresponding to the file to be scheduled is calculated based on the access dispersion, the random access degree, the access popularity, the file size, and the read operation ratio.
[0026] In one possible implementation, calculating the file score corresponding to the file to be scheduled based on the access dispersion, the random access degree, the access popularity, the file size, and the read operation ratio includes:
[0027] The file score corresponding to the file to be scheduled is obtained by substituting the access dispersion, the random access degree, the access popularity, the file size, and the read operation ratio into Formula 1.
[0028]
[0029] Among them, V i F represents the file score of file i. i This represents the access popularity of file i. To access the degree of discreteness, r i s represents the random access degree of file i. i R represents the size of file i. i T represents the total size of read operations on file i. i This represents the total number of accesses to file i.
[0030] In one possible implementation, calculating the access dispersion based on the access popularity and the number of accesses to the file to be scheduled within each unit of time includes:
[0031] Substitute the access popularity and the number of accesses to the scheduled file within each unit of time into Formula 2 to calculate the access dispersion.
[0032]
[0033] Where n is the number of units of time, f j Let f be the number of accesses to the file to be scheduled within the j-th unit of time, and let f be the total number of accesses to the file to be scheduled within the n-th unit of time.
[0034] Secondly, this application provides a resource scheduling device for a hybrid storage system, comprising:
[0035] The file acquisition module is used to acquire a set of files to be scheduled in a hybrid storage system, wherein the hybrid storage system includes mechanical hard drives and solid-state drives;
[0036] The first determining module is used to determine, for each file to be scheduled in the set of files to be scheduled, the random access degree, access popularity, file size and read operation ratio of the file to be scheduled in the current period, wherein the read operation ratio is used to characterize the proportion of read operations to the total access volume;
[0037] The second determining module is used to determine the file score corresponding to the file to be scheduled based on the random access degree, the access popularity, the file size and the read operation ratio;
[0038] The file storage module is used to store all files to be scheduled to the corresponding storage location according to the file score of each file to be scheduled.
[0039] In one possible implementation, the file storage module is specifically used for:
[0040] Sort all files to be scheduled in descending order of their corresponding file scores;
[0041] The first preset number of files to be scheduled, which are ranked first, are stored in the solid-state drive;
[0042] The second preset number of files to be scheduled, which are ranked lower, are stored in the mechanical hard disk.
[0043] In one possible implementation, the first determining module is specifically used for:
[0044] Within the current cycle, the access status of the files to be scheduled is monitored in real time;
[0045] Each time the file to be scheduled is accessed, it is determined whether the current access is consecutive to the previous access;
[0046] If the current access is not consecutive with the previous access, a step value is added to the access degree assignment corresponding to the file to be scheduled to obtain the current random access degree. The initial value of the access degree assignment is zero at the beginning of the current period.
[0047] In one possible implementation, the first determining module is further configured to:
[0048] Determine at least one unit of duration contained in the current period;
[0049] For each unit of time, determine the number of accesses to the file to be scheduled within that unit of time;
[0050] The access popularity is obtained by summing the access counts corresponding to multiple unit durations.
[0051] In one possible implementation, the second determining module is specifically used for:
[0052] The access dispersion is calculated based on the access popularity and the number of accesses to the scheduled file within each unit of time.
[0053] The file score corresponding to the file to be scheduled is calculated based on the access dispersion, the random access degree, the access popularity, the file size, and the read operation ratio.
[0054] In one possible implementation, the second determining module is further configured to:
[0055] The file score corresponding to the file to be scheduled is obtained by substituting the access dispersion, the random access degree, the access popularity, the file size, and the read operation ratio into Formula 1.
[0056]
[0057] Among them, V i F represents the file score of file i. i This represents the access popularity of file i. To access the degree of discreteness, r i s represents the random access degree of file i. i R represents the size of file i. i T represents the total size of read operations on file i. i This represents the total number of accesses to file i.
[0058] In one possible implementation, the second determining module is further configured to:
[0059] Substitute the access popularity and the number of accesses to the scheduled file within each unit of time into Formula 2 to calculate the access dispersion.
[0060]
[0061] Where n is the number of units of time, f j Let f be the number of accesses to the file to be scheduled within the j-th unit of time, and let f be the total number of accesses to the file to be scheduled within the n-th unit of time.
[0062] Thirdly, this application provides an apparatus comprising: a processor and a memory, the processor being configured to execute a resource scheduling program for a hybrid storage system stored in the memory, to implement the resource scheduling method for a hybrid storage system as described in any one of the first aspects.
[0063] Fourthly, this application provides a storage medium storing one or more programs that can be executed by one or more processors to implement the resource scheduling method of the hybrid storage system described in the first aspect.
[0064] Compared with the prior art, the technical solution provided in this application has the following advantages: The method provided in this application first obtains a set of files to be scheduled in a hybrid storage system, wherein the hybrid storage system includes mechanical hard disks (HDDs) and solid-state drives (SSDs). Then, for each file to be scheduled in the set of files to be scheduled, the random access degree, access popularity, file size, and read operation ratio corresponding to the file to be scheduled in the current period are determined. The read operation ratio is used to characterize the proportion of read operations to the total access volume. Based on the random access degree, access popularity, file size, and read operation ratio, the file score corresponding to the file to be scheduled is determined. Finally, all files to be scheduled are stored in their corresponding storage locations according to their file scores. Through this application, the scheduling of files in a hybrid storage system can be achieved by leveraging the respective advantages of HDDs and SSDs, thereby reducing storage device costs while improving data access speed. Attached Figure Description
[0065] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0066] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0067] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.
[0068] Figure 1 A flowchart illustrating an embodiment of a resource scheduling method for a hybrid storage system provided in this application;
[0069] Figure 2 A flowchart illustrating an embodiment of this application for determining the file score corresponding to the file to be scheduled based on the random access degree, the access popularity, the file size, and the read operation ratio;
[0070] Figure 3 A block diagram illustrating an embodiment of a resource scheduling device for a hybrid storage system provided in this application.
[0071] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0072] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0073] The following disclosure provides numerous different embodiments or examples for implementing various structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.
[0074] To address the shortcomings of existing technologies, such as the slow data access speed of mechanical hard drives and the high cost and small capacity of SSDs (Solid State Drives), which limit the widespread adoption of SSDs, this application provides a resource scheduling method for hybrid storage systems. This method leverages the advantages of both mechanical hard drives and solid state drives to schedule files within the hybrid storage system, thereby reducing storage device costs while improving data access speed.
[0075] Figure 1 This is a flowchart illustrating an embodiment of a resource scheduling method for a hybrid storage system provided in this application. Figure 1 As shown, the method includes the following steps:
[0076] Step 101: Obtain the set of files to be scheduled in the hybrid storage system, wherein the hybrid storage system includes mechanical hard disks and solid-state drives.
[0077] This application provides a resource scheduling method for a hybrid storage system, which can be applied to a hybrid storage system that includes a mechanical hard disk and an SSD (Solid State Disk).
[0078] In one embodiment, the set of all files in the current hybrid storage system can be used as the set of files to be scheduled. In this way, each file in the hybrid storage system can be treated as a file to be scheduled for subsequent file score calculations and then re-stored to an appropriate location.
[0079] In another embodiment, a set of files stored in the hybrid storage system for more than a preset duration (e.g., one day, one week, one month, etc.) can be used as the set of files to be scheduled. In this way, some files with longer storage times can be included as files to be scheduled for subsequent file score calculations and then restoring them to appropriate locations.
[0080] In another embodiment, the set of files specified by the user in the hybrid storage system can be used as the set of files to be scheduled. This allows the user to flexibly set the files that need to be re-stored according to actual needs.
[0081] Step 102: For each file to be scheduled in the set of files to be scheduled, determine the random access degree, access popularity, file size and read operation ratio of the file to be scheduled in the current period, wherein the read operation ratio is used to characterize the proportion of read operations to the total access volume.
[0082] The current period refers to the time elapsed since the last time files in the hybrid storage system were scheduled.
[0083] Random access degree: used to characterize the continuity of file access over a period of time. The lower the continuity, the higher the random access degree.
[0084] Access popularity: This measures the frequency of file access over a period of time. The higher the access frequency, the higher the access popularity.
[0085] File size: Used to represent the amount of storage space occupied by a file.
[0086] Read operation ratio: This represents the proportion of read operations to the total number of accesses.
[0087] In one embodiment, the specific implementation of determining the random access degree corresponding to the file to be scheduled in the current period may include the following steps: in the current period, the access status of the file to be scheduled is monitored in real time; each time the file to be scheduled is accessed, it is determined whether the current access is continuous with the previous access; if the current access is not continuous with the previous access, a step value is added to the access degree assignment corresponding to the file to be scheduled to obtain the current random access degree, wherein the initial value of the access degree assignment is zero at the beginning of the current period.
[0088] In this scheme, each time a file is accessed, it is determined whether the current access is consecutive to the previous access. If it is not consecutive, it means that the file is biased towards random access, and a step value is added to the current access degree assignment. For example, if the current access degree is assigned as r and the step value is 1, then when the file access is not consecutive in this instance, the access degree assignment is updated to r+1.
[0089] In another embodiment, the specific implementation of determining the access popularity of the file to be scheduled within the current period may include the following steps: determining at least one unit duration contained in the current period; for each unit duration, determining the number of accesses of the file to be scheduled within the unit duration; and summing the number of accesses corresponding to multiple unit durations to obtain the access popularity.
[0090] In a specific example, if the unit duration is one hour and the current period contains n hours of data, then the file access popularity... Where f is the access popularity, f j This represents the number of visits within the j-th hour.
[0091] Step 103: Determine the file score corresponding to the file to be scheduled based on the random access degree, the access popularity, the file size, and the read operation ratio.
[0092] Step 104: Store all files to be scheduled to their corresponding storage locations based on the file score of each file to be scheduled.
[0093] For ease of understanding, steps 103 and 104 will be explained uniformly below:
[0094] In practical applications, the advantages of mechanical hard drives are: large capacity and low price. The disadvantages are: poor random access performance (sequential access performance is relatively better than random access). The advantages of solid-state drives are: better performance than mechanical hard drives in both random and sequential access. The disadvantages are: smaller capacity and higher price than mechanical hard drives. Write operations are limited by the number of erase cycles of flash memory chips, so frequent write operations should be avoided as much as possible.
[0095] Based on the advantages of both hard disk drives (HDDs) and solid-state drives (SSDs), we will employ the following strategies for resource scheduling:
[0096] 1. If a file is accessed randomly in most scenarios, it is preferable to place it on an SSD to obtain better random access performance.
[0097] 2. If a file is frequently accessed, it is preferred to place it on an SSD for better performance. If the file is accessed infrequently, it is preferred to place it on a mechanical hard drive, which is considered cold data.
[0098] 3. If a file is particularly large and takes up a lot of space, since the storage cost of an SSD is much higher than that of a mechanical hard drive, it is more preferable to place it on a mechanical hard drive.
[0099] 4. If a file is written to much more frequently than read, it is preferable to place it on a mechanical hard drive to ensure the lifespan of the flash memory chips.
[0100] 5. For two files that are accessed f times within n hours, the f accesses to file 1 are evenly distributed over n hours, while the f accesses to file 2 occur within a shorter period of time and are not accessed for the rest of the time. Therefore, file 1 is more advantageous to the solid-state drive (SSD) as it has higher space utilization. File 2, on the other hand, is not accessed for a long period of time, but it occupies expensive storage space on the SSD.
[0101] In this embodiment of the application, firstly, combining the strategies in points 1-5 above, the file score corresponding to the file to be scheduled is calculated based on random access degree, access popularity, file size and read operation ratio. Then, all files to be scheduled are stored in the corresponding storage location (i.e., mechanical hard disk or SSD) according to the file score of each file to be scheduled.
[0102] Specifically, the implementation of storing all files to be scheduled to their corresponding storage locations based on the file score of each file to be scheduled may include the following steps:
[0103] Step A1: Sort all files to be scheduled in descending order of their corresponding file scores;
[0104] Step A2: Store the first preset number of files to be scheduled, which are ranked first, into the solid-state drive;
[0105] Step A3: Store the second preset number of files to be scheduled that are ranked lower in the order into the mechanical hard disk.
[0106] In this implementation, small files with high-frequency random access, a large proportion of which are read operations, and a relatively even distribution of all accesses will have a higher file score. The scheduling system will prioritize placing these files to be scheduled on the SSD, while files with lower file scores will be prioritized on the mechanical hard drive. This achieves the scheduling of files to be scheduled in the hybrid storage system.
[0107] In practical applications, the execution of steps 101-104 can be triggered periodically to prevent files from being stored in inappropriate locations for extended periods, thus avoiding performance degradation. The execution of steps 101-104 can also be triggered when the system load is low, or it can be manually initiated by the user. This ensures that the scheduling process itself does not affect user access.
[0108] The technical solution provided in this application first obtains a set of files to be scheduled in a hybrid storage system, wherein the hybrid storage system includes both mechanical hard drives and solid-state drives. Then, for each file to be scheduled in the set, the random access frequency, access popularity, file size, and read operation ratio corresponding to the file in the current period are determined. The read operation ratio characterizes the proportion of read operations to total access volume. Based on the random access frequency, access popularity, file size, and read operation ratio, a file score corresponding to the file to be scheduled is determined. Finally, all files to be scheduled are stored in their corresponding storage locations according to their file scores. This application leverages the respective advantages of mechanical hard drives and solid-state drives to achieve file scheduling in a hybrid storage system, thereby reducing storage device costs while improving data access speed.
[0109] Figure 2 This document provides a flowchart of an embodiment for determining the file score corresponding to the file to be scheduled based on the random access degree, the access popularity, the file size, and the read operation ratio. Figure 2 The process shown is in Figure 1 Based on the illustrated process, the following steps are included:
[0110] Step 201: Calculate the access dispersion based on the access popularity and the number of accesses to the file to be scheduled within each unit of time.
[0111] In this embodiment of the application, the specific implementation of calculating the access dispersion based on the access popularity and the number of accesses to the file to be scheduled within each unit of time may include the following steps:
[0112] Substitute the access popularity and the number of accesses to the scheduled file within each unit of time into Formula 2 to calculate the access dispersion.
[0113]
[0114] Where n is the number of units of time, f j Let f be the number of accesses to the file to be scheduled within the j-th unit of time, and let f be the total number of accesses to the file to be scheduled within the n-th unit of time.
[0115] Formula 2 is derived from the variance formula, which is as follows: Since all files to be scheduled have the same value of n, the variance formula can be simplified to Formula 2.
[0116] Since sample variance is a statistic used to measure the dispersion of a set of data, the larger the variance value, the more dispersed the statistical data. Therefore, the dispersion can be obtained through Formula 2.
[0117] Step 202: Calculate the file score corresponding to the file to be scheduled based on the access dispersion, random access degree, access popularity, file size, and read operation ratio.
[0118] In this embodiment of the application, the specific implementation of calculating the file score corresponding to the file to be scheduled based on the access dispersion, the random access degree, the access popularity, the file size, and the read operation ratio may include the following steps:
[0119] The file score corresponding to the file to be scheduled is obtained by substituting the access dispersion, the random access degree, the access popularity, the file size, and the read operation ratio into Formula 1.
[0120]
[0121] Among them, V i F represents the file score of file i. i This represents the access popularity of file i. To access the degree of discreteness, r i s represents the random access degree of file i. i R represents the size of file i. i T represents the total size of read operations on file i. i This represents the total number of accesses to file i.
[0122] As can be seen from the above formula, the higher the access frequency of the file to be scheduled, the lower the access dispersion, the higher the random access degree, the smaller the file, and the larger the proportion of read operations, the higher the file score. In this case, the file to be scheduled should be placed on the SSD; otherwise, the file to be scheduled should be placed on the mechanical hard drive first.
[0123] pass Figure 2 The process shown can calculate the file score of the file to be scheduled based on the access dispersion, random access, access popularity, file size, and read operation ratio. This allows the file to be scheduled to be placed on the SSD when the access popularity, access dispersion, random access, file size, and read operation ratio are high. Otherwise, the file to be scheduled will be placed on the mechanical hard drive first.
[0124] Figure 3 This is a block diagram illustrating an embodiment of a resource scheduling device for a hybrid storage system provided in this application. Figure 3 As shown, the device includes:
[0125] The file acquisition module 31 is used to acquire a set of files to be scheduled in the hybrid storage system, wherein the hybrid storage system includes mechanical hard disks and solid-state drives;
[0126] The first determining module 32 is used to determine, for each file to be scheduled in the set of files to be scheduled, the random access degree, access popularity, file size and read operation ratio of the file to be scheduled in the current period, wherein the read operation ratio is used to characterize the proportion of read operations to the total access volume.
[0127] The second determining module 33 is used to determine the file score corresponding to the file to be scheduled based on the random access degree, the access popularity, the file size and the read operation ratio;
[0128] The file storage module 34 is used to store all the files to be scheduled to the corresponding storage location according to the file score of each file to be scheduled.
[0129] In one possible implementation, the file storage module is specifically used for:
[0130] Sort all files to be scheduled in descending order of their corresponding file scores;
[0131] The first preset number of files to be scheduled, which are ranked first, are stored in the solid-state drive;
[0132] The second preset number of files to be scheduled, which are ranked lower, are stored in the mechanical hard disk.
[0133] In one possible implementation, the first determining module is specifically used for:
[0134] Within the current cycle, the access status of the files to be scheduled is monitored in real time;
[0135] Each time the file to be scheduled is accessed, it is determined whether the current access is consecutive to the previous access;
[0136] If the current access is not consecutive with the previous access, a step value is added to the access degree assignment corresponding to the file to be scheduled to obtain the current random access degree. The initial value of the access degree assignment is zero at the beginning of the current period.
[0137] In one possible implementation, the first determining module is further configured to:
[0138] Determine at least one unit of duration contained in the current period;
[0139] For each unit of time, determine the number of accesses to the file to be scheduled within that unit of time;
[0140] The access popularity is obtained by summing the access counts corresponding to multiple unit durations.
[0141] In one possible implementation, the second determining module is specifically used for:
[0142] The access dispersion is calculated based on the access popularity and the number of accesses to the scheduled file within each unit of time.
[0143] The file score corresponding to the file to be scheduled is calculated based on the access dispersion, the random access degree, the access popularity, the file size, and the read operation ratio.
[0144] In one possible implementation, the second determining module is further configured to:
[0145] The file score corresponding to the file to be scheduled is obtained by substituting the access dispersion, the random access degree, the access popularity, the file size, and the read operation ratio into Formula 1.
[0146]
[0147] Among them, V i F represents the file score of file i. i This represents the access popularity of file i. To access the degree of discreteness, r i s represents the random access degree of file i. i R represents the size of file i. i T represents the total size of read operations on file i. i This represents the total number of accesses to file i.
[0148] In one possible implementation, the second determining module is further configured to:
[0149] Substitute the access popularity and the number of accesses to the scheduled file within each unit of time into Formula 2 to calculate the access dispersion.
[0150]
[0151] Where n is the number of units of time, f j Let f be the number of accesses to the file to be scheduled within the j-th unit of time, and let f be the total number of accesses to the file to be scheduled within the n-th unit of time.
[0152] like Figure 4As shown in the figure, this application provides a device including a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.
[0153] Memory 113 is used to store computer programs;
[0154] In one embodiment of this application, when the processor 111 executes a program stored in the memory 113, it implements the resource scheduling method of the hybrid storage system provided in any of the foregoing method embodiments, including:
[0155] Obtain a set of files to be scheduled in a hybrid storage system, wherein the hybrid storage system includes mechanical hard disks and solid-state drives;
[0156] For each file in the set of files to be scheduled, determine the random access degree, access popularity, file size and read operation ratio of the file to be scheduled in the current period, wherein the read operation ratio is used to characterize the proportion of read operations to the total access volume;
[0157] The file score corresponding to the file to be scheduled is determined based on the random access degree, the access popularity, the file size, and the read operation ratio.
[0158] All files to be scheduled are stored in their corresponding storage locations based on their file scores.
[0159] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the resource scheduling method for a hybrid storage system as provided in any of the foregoing method embodiments.
[0160] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0161] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0162] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.
[0163] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.
Claims
1. A resource scheduling method for a hybrid storage system, characterized in that, The method includes: Obtain a set of files to be scheduled in a hybrid storage system, wherein the hybrid storage system includes mechanical hard disks and solid-state drives; For each file to be scheduled in the set of files to be scheduled, determine the random access degree, access popularity, file size and read operation ratio of the file to be scheduled in the current period, wherein the read operation ratio is used to characterize the proportion of read operations to the total access volume; The file score corresponding to the file to be scheduled is determined based on the random access degree, the access popularity, the file size, and the read operation ratio. All files to be scheduled are stored in their corresponding storage locations based on their file scores.
2. The method of claim 1, wherein, The step of storing all files to be scheduled to their corresponding storage locations based on the file score of each file to be scheduled includes: Sort all files to be scheduled in descending order of their corresponding file scores; The first preset number of files to be scheduled, which are ranked first, are stored in the solid-state drive; The second preset number of files to be scheduled, which are ranked lower, are stored in the mechanical hard disk.
3. The method of claim 1, wherein, Determining the random access degree of the file to be scheduled within the current period includes: Within the current cycle, the access status of the files to be scheduled is monitored in real time; Each time the file to be scheduled is accessed, it is determined whether the current access is consecutive to the previous access; If the current access is not consecutive with the previous access, a step value is added to the access degree assignment corresponding to the file to be scheduled to obtain the current random access degree. The initial value of the access degree assignment is zero at the beginning of the current period.
4. The method of claim 1, wherein, Determining the access popularity of the file to be scheduled within the current period includes: Determine at least one unit of duration contained in the current period; For each unit of time, determine the number of accesses to the file to be scheduled within that unit of time; The access popularity is obtained by summing the access counts corresponding to multiple unit durations.
5. The method of claim 4, wherein, The step of determining the file score corresponding to the file to be scheduled based on the random access degree, the access popularity, the file size, and the read operation ratio includes: The access dispersion is calculated based on the access popularity and the number of accesses to the scheduled file within each unit of time. The file score corresponding to the file to be scheduled is calculated based on the access dispersion, the random access degree, the access popularity, the file size, and the read operation ratio.
6. The method of claim 5, wherein, The step of calculating the file score corresponding to the file to be scheduled based on the access dispersion, the random access degree, the access popularity, the file size, and the read operation ratio includes: The file score corresponding to the file to be scheduled is obtained by substituting the access dispersion, the random access degree, the access popularity, the file size, and the read operation ratio into Formula 1. Equation One: wherein V i represents the file score of file i, F i represents the access hotness of file i, is the access dispersion, r i represents the random access degree of file i, s i represents the size of file i, R i is the total size of read operations on file i, T i is the total access amount on file i.
7. The method of claim 5, wherein, The step of calculating the access dispersion based on the access popularity and the number of accesses to the file to be scheduled within each unit of time includes: Substitute the access popularity and the number of accesses to the scheduled file within each unit of time into Formula 2 to calculate the access dispersion. Equation Two: wherein n is the number of unit time lengths, f j is the number of access times corresponding to the file to be scheduled in the jth unit time length, and f is the total number of access times corresponding to the file to be scheduled in the n unit time lengths.
8. A resource scheduling apparatus of a hybrid storage system, characterized by, The device includes: The file acquisition module is used to acquire a set of files to be scheduled in a hybrid storage system, wherein the hybrid storage system includes mechanical hard drives and solid-state drives; The first determining module is used to determine, for each file to be scheduled in the set of files to be scheduled, the random access degree, access popularity, file size and read operation ratio of the file to be scheduled in the current period, wherein the read operation ratio is used to characterize the proportion of read operations to the total access volume; The second determining module is used to determine the file score corresponding to the file to be scheduled based on the random access degree, the access popularity, the file size and the read operation ratio; The file storage module is used to store all files to be scheduled to the corresponding storage location according to the file score of each file to be scheduled.
9. An apparatus, comprising: include: A processor and a memory, the processor being configured to execute a resource scheduling program for a hybrid storage system stored in the memory, to implement the resource scheduling method for a hybrid storage system according to any one of claims 1-7.
10. A storage medium, characterized by The storage medium stores one or more programs, which can be executed by one or more processors to implement the resource scheduling method of the hybrid storage system according to any one of claims 1-7.