Storage area creating method, device and equipment
By cyclically selecting and inserting candidate hard disks in the storage system to optimize the distribution of hard disk objects, the problem of data storage imbalance between hard disks is solved, and more efficient resource utilization and business IO stability are achieved.
Patent Information
- Application Number
- CN202411962232.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-04-18
- Estimated Expiration
- 2044-12-30
AI Technical Summary
In high-performance and high-reliability storage systems, the prior art has unbalanced data storage between hard disks due to the complexity and decentralization characteristics of the CRUSH algorithm, which in turn affects resource utilization and service IO.
When creating a target storage area, the candidate hard disk is removed from the large top heap for each storage pool cycle, and the target hard disk object is determined based on the number of hard disk objects and the matching degree of the hard disk box, and the candidate hard disk is inserted back to the large top heap, ensuring that the number of available hard disks for the candidate hard disk in the large top heap is the largest, thereby achieving balanced selection of hard disk objects.
This method optimizes the selection and distribution of hard disk objects, realizes the balance of data storage between hard disks, and improves the system's resource utilization rate and business IO stability.
Smart Images

Figure CN119847440A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a storage area creation method, device and equipment. Background Art
[0002] In a high-performance and high-reliability storage system, memory is selected as the storage medium, multiple copies are used, and persistent memory technology is used in the event of failure, so that high-performance and high-reliability data storage can be achieved at a relatively low cost. Specifically, when the system is operating normally, the system directly uses memory to achieve high performance. When the system fails, the system persists the data in the memory to the hard disk. When the system is started next time, the data is read from the hard disk to the memory, thereby achieving high reliability.
[0003] In the related technology, when persisting data in the system memory to the hard disk, the storage area where the data is written is usually determined by Ceph (distributed file system). Specifically, Ceph records the location information of each hard disk in the cluster map (cluster heat map), and based on the cluster map and other information, uses the CRUSH (Controlled Replication Under Scalable Hashing) algorithm to determine the storage areas of different fault domain levels for writing data in the memory. However, since the CRUSH algorithm is relatively complex and is a decentralized algorithm, the creation of the storage area is based on the cluster map, so there are problems such as uneven data storage between hard disks. Summary of the invention
[0004] The present invention provides a storage area creation method, device and equipment for solving the problem of uneven data storage among hard disks.
[0005] In a first aspect, an embodiment of the present application provides a method for creating a storage area, the method comprising:
[0006] Acquire at least one storage pool corresponding to the target storage area to be created, wherein the storage pool includes a plurality of hard disks, and each hard disk includes a plurality of hard disk objects;
[0007] For each storage pool, the following process is executed cyclically until the determined target hard disk object meets the first preset condition, and a target storage area is constructed based on each obtained target hard disk object:
[0008] From the big-top heap obtained in the previous cycle, the candidate hard disk at the top of the heap is taken out; wherein the hard disks included in the storage pool are arranged in sequence in the big-top heap according to the number of available hard disk objects included in the hard disk;
[0009] Based on the disk enclosure corresponding to the candidate hard disk and the reference disk enclosure corresponding to the hard disk to which each determined target hard disk object belongs, when it is determined that the candidate hard disk meets the second preset condition, select a hard disk object from the candidate hard disk as the target hard disk object;
[0010] Based on the number of available hard disk objects included in the current candidate hard disk, a target position of the candidate hard disk in the big-max heap is determined, and the candidate hard disk is inserted into the target position.
[0011] The above method, when creating the target storage area, for each storage pool, cyclically takes out the candidate hard disk at the top of the big top heap, and when it is determined that the corresponding hard disk frame meets the preset conditions, obtains the target hard disk object therefrom, inserts the candidate hard disk back into the big top heap, and updates the big top heap to facilitate subsequent operations. In this way, it can be ensured that the candidate hard disk at the top of the big top heap has the largest number of available hard disks, and then it can be preferentially selected from the hard disks with the largest number of available hard disk objects, that is, it is possible to quickly select the hard disk that meets the redundancy requirements from each hard disk, and ensure that each hard disk is in a balanced state.
[0012] In a possible implementation manner, the first preset condition includes at least one of the following:
[0013] The number of determined target objects reaches a first preset threshold;
[0014] The number of cycles reaches a second preset threshold;
[0015] The number of available hard disk objects corresponding to the candidate hard disk at the top of the max-heap is zero.
[0016] In a possible implementation, determining that the candidate hard disk meets the second preset condition based on the hard disk enclosure corresponding to the candidate hard disk and the hard disk enclosure corresponding to the hard disk to which each determined target hard disk object belongs includes:
[0017] When the number of each reference hard disk frame is less than the preset number, and each reference hard disk frame does not include a hard disk frame corresponding to the candidate hard disk, determining that the candidate hard disk meets the second preset condition; or,
[0018] When the number of each reference hard disk frame is not less than the preset number, it is determined that the candidate hard disk meets the second preset condition.
[0019] The above method ensures that the target hard disk objects are evenly distributed in multiple hard disk frames, thereby improving the balance of subsequent data storage, by setting the candidate hard disk to meet the second preset condition when the number of reference hard disk frames reaches the preset number, or does not reach the preset number and the candidate hard disk does not correspond to the reference hard disk frame.
[0020] In a possible implementation, when the target storage area corresponds to the first type, each target hard disk object in the target storage area belongs to multiple hard disks in multiple storage pools, and the method further includes:
[0021] When it is determined that a hard disk to which any target hard disk object belongs in the target storage area is faulty, deleting any target hard disk object from the target storage area;
[0022] When it is determined that a disk enclosure corresponding to a hard disk to which any target hard disk object belongs in the target storage area is faulty, deleting any target hard disk object from the target storage area;
[0023] When it is determined that a storage pool corresponding to a hard disk to which any target hard disk object belongs in the target storage area has a fault, deleting any target hard disk object from the target storage area;
[0024] When it is determined that the target storage area includes only one target hard disk object and a storage pool corresponding to a hard disk to which the target hard disk object belongs fails, the storage area corresponding to the hard disk of the target hard disk object is configured as a virtual storage pool.
[0025] The above method, by setting the corresponding first type of target storage area, when the hard disk, hard disk frame, or storage pool corresponding to the target hard disk object fails, the target storage area still exists. The setting of the target storage area meets the storage requirements of data that the normal operation of the system depends on and has high reliability requirements, and provides high reliability of data storage.
[0026] In a possible implementation, when the target storage area corresponds to the second type, each target hard disk object in the target storage area belongs to multiple hard disks in the same storage pool, and the method further includes:
[0027] When it is determined that a hard disk to which any target hard disk object belongs in the target storage area is faulty, deleting any target hard disk object from the target storage area;
[0028] When it is determined that a disk enclosure corresponding to a hard disk to which any target hard disk object belongs in the target storage area is faulty, deleting any target hard disk object from the target storage area;
[0029] When it is determined that a storage pool corresponding to a hard disk to which any target hard disk object belongs in the target storage area is faulty, the target storage area is deleted.
[0030] The above method, by setting the corresponding second type of target storage area, when the hard disk or hard disk frame corresponding to the target hard disk object fails, the target storage area still exists. Only when the storage pool fails, the corresponding target storage area is deleted. The setting of the target storage area meets the storage requirements of data that are strongly bound to the life cycle of the storage pool.
[0031] In a possible implementation manner, before obtaining at least one storage pool corresponding to the target storage area to be created, the method further includes:
[0032] Create each storage pool, and for each storage pool, add the hard disk corresponding to the storage pool and each hard disk object in the hard disk to the storage pool;
[0033] Wherein, for any hard disk object, when it is determined based on the red-black tree algorithm that any hard disk object does not exist in the hard disk objects included in the storage pool, any hard disk object is added to the storage pool.
[0034] The above method ensures that each hard disk object belongs to a single hard disk, which effectively avoids adding a hard disk object to the storage pool multiple times, realizes fast verification of hard disk objects, and improves the accuracy of subsequent generation of target storage areas based on hard disk objects in the storage pool.
[0035] In a possible implementation, after constructing the target storage area, the method further includes:
[0036] Based on the target structure, encapsulate the target storage area to obtain a segment structure corresponding to the target storage area;
[0037] Among them, the target structure is: a structure used when encapsulating the storage area corresponding to the memory, and the target structure includes: area type, number of copies and copy information; the area type corresponding to the target storage area is hard disk, the number of copies is the number of hard disk objects included, and the copy information is the object information of each hard disk object.
[0038] The above method, by setting a unified segment structure for the storage areas of the memory and hard disk, provides a unified read and write interface for the hard disk and memory. When the persistent memory is unstable, the business request entry can be quickly switched to the hard disk, ensuring that the business can quickly restore IO after a short (seconds) period of unavailability.
[0039] In a second aspect, an embodiment of the present application provides a storage area creation device, the device comprising:
[0040] An acquisition module, used for acquiring at least one storage pool corresponding to the target storage area to be created, wherein the storage pool includes a plurality of hard disks, and each hard disk includes a plurality of hard disk objects;
[0041] The construction module is used to cyclically execute the following process for each storage pool until the determined target hard disk object meets the first preset condition, and construct the target storage area based on each obtained target hard disk object:
[0042] From the big-top heap obtained in the previous cycle, the candidate hard disk at the top of the heap is taken out; wherein the hard disks included in the storage pool are arranged in sequence in the big-top heap according to the number of available hard disk objects included in the hard disk;
[0043] Based on the hard disk enclosure corresponding to the candidate hard disk and the reference hard disk enclosure corresponding to the hard disk to which each determined target hard disk object belongs, when it is determined that the candidate hard disk meets the second preset condition, select a hard disk object from the candidate hard disk as the target hard disk object;
[0044] Based on the number of available hard disk objects included in the current candidate hard disk, a target position of the candidate hard disk in the big-max heap is determined, and the candidate hard disk is inserted into the target position.
[0045] In a possible implementation manner, the first preset condition includes at least one of the following:
[0046] The number of determined target objects reaches a first preset threshold;
[0047] The number of cycles reaches a second preset threshold;
[0048] The number of available hard disk objects corresponding to the candidate hard disk at the top of the max-heap is zero.
[0049] In a possible implementation manner, the above-mentioned building blocks are specifically used to:
[0050] When the number of each reference hard disk frame is less than the preset number, and each reference hard disk frame does not include a hard disk frame corresponding to the candidate hard disk, determining that the candidate hard disk meets the second preset condition; or,
[0051] When the number of each reference hard disk frame is not less than the preset number, it is determined that the candidate hard disk meets the second preset condition.
[0052] In a possible implementation, when the target storage area corresponds to the first type, each target hard disk object in the target storage area belongs to multiple hard disks in multiple storage pools, and the above-mentioned device further includes a first management module, which is specifically used to:
[0053] When it is determined that a hard disk to which any target hard disk object belongs in the target storage area is faulty, deleting any target hard disk object from the target storage area;
[0054] When it is determined that a disk enclosure corresponding to a hard disk to which any target hard disk object belongs in the target storage area is faulty, deleting any target hard disk object from the target storage area;
[0055] When it is determined that a storage pool corresponding to a hard disk to which any target hard disk object belongs in the target storage area has a fault, deleting any target hard disk object from the target storage area;
[0056] When it is determined that the target storage area includes only one target hard disk object and a storage pool corresponding to a hard disk to which the target hard disk object belongs fails, the storage area corresponding to the hard disk of the target hard disk object is configured as a virtual storage pool.
[0057] In a possible implementation, when the target storage area corresponds to the second type, each target hard disk object in the target storage area belongs to multiple hard disks in the same storage pool, and the above-mentioned device further includes a second management module, which is specifically used to:
[0058] When it is determined that a hard disk to which any target hard disk object belongs in the target storage area is faulty, deleting any target hard disk object from the target storage area;
[0059] When it is determined that a disk enclosure corresponding to a hard disk to which any target hard disk object belongs in the target storage area is faulty, deleting any target hard disk object from the target storage area;
[0060] When it is determined that a storage pool corresponding to a hard disk to which any target hard disk object belongs in the target storage area is faulty, the target storage area is deleted.
[0061] In a possible implementation manner, before acquiring at least one storage pool corresponding to the target storage area to be created, the acquisition module is further configured to:
[0062] Create each storage pool, and for each storage pool, add the hard disk corresponding to the storage pool and each hard disk object in the hard disk to the storage pool;
[0063] Wherein, for any hard disk object, when it is determined based on the red-black tree algorithm that any hard disk object does not exist in the storage objects included in the storage pool, any hard disk object is added to the storage pool.
[0064] In a possible implementation manner, after constructing the target storage area, the construction module is further used to:
[0065] Based on the target structure, encapsulate the target storage area to obtain a segment structure corresponding to the target storage area;
[0066] Among them, the target structure is: a structure used when encapsulating the storage area corresponding to the memory, and the target structure includes: area type, number of copies and copy information; the area type corresponding to the target storage area is hard disk, the number of copies is the number of hard disk objects included, and the copy information is the object information of each hard disk object.
[0067] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the storage area creation method when executing the computer program.
[0068] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the storage area creation method described above in the present application.
[0069] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which is stored in a computer-readable storage medium; when the processor of a memory access device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the memory access device executes the steps in the above-mentioned storage area creation method of the present application.
[0070] For each of the above-mentioned second to fifth aspects and the technical effects that may be achieved by each of the above-mentioned aspects, please refer to the above-mentioned description of the technical effects that can be achieved by the first aspect or various possible schemes in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS
[0071] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0072] Figure 1 A schematic diagram of a storage area creation method provided in an embodiment of the present application;
[0073] Figure 2 A schematic diagram of a storage pool and a hardware frame provided in an embodiment of the present application;
[0074] Figure 3 A schematic diagram of an example of creating a single pool storage area provided in an embodiment of the present application;
[0075] Figure 4 A schematic diagram of an example of creating a cross-pool storage area provided in an embodiment of the present application;
[0076] Figure 5 A schematic diagram of a storage area creation device provided in an embodiment of the present application;
[0077] Figure 6 A schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0078] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be arbitrarily combined with each other. In addition, although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.
[0079] The terms "first" and "second" in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive protection. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. "Multiple" in the present application can mean at least two, for example, two, three or more, and the embodiments of the present application are not limited.
[0080] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description. It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, which should be considered as exemplary, and their purpose is only to illustrate the feasibility of the implementation of the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.
[0081] In the technical solution of this application, the acquisition, transmission, storage, and use of data are in compliance with the requirements of relevant national laws and regulations.
[0082] In order to facilitate those skilled in the art to better understand the technical solutions provided by the embodiments of the present application, the following is a brief description of the professional terms involved:
[0083] (1) CRUSH (Controlled Replication Under Scalable Hashing): A hash-based data distribution algorithm that uses the data unique identifier, the current storage cluster topology, and the data backup strategy as CRUSH input. It can obtain the location of the underlying storage device where the data is located through calculation at any time and communicate with it directly, thereby avoiding table lookup operations and achieving decentralization and high concurrency.
[0084] (2) Hard disk enclosure: The physical location of the hard disk.
[0085] (3) Storage pool: The logical location of a hard disk, which serves as a logical storage space to provide services to the business.
[0086] (4) Hard disk object: The smallest storage unit on the hard disk, with a fixed storage space size, identified by the physical hard disk ID and offset.
[0087] (5) Memory segment (i.e., the storage area corresponding to the memory): provides the smallest memory-based storage unit to the outside world.
[0088] (6) Disk segment (i.e., target storage area): It provides the smallest storage unit based on the hard disk to the outside world. It consists of multiple copies of hard disk objects on multiple hard disks and carries hard disk object information.
[0089] In a high-performance and high-reliability storage system, memory is selected as the storage medium, multiple copies are used, and persistent memory technology (such as PMem media, batteries, etc.) is used in combination with failures to achieve high-performance and high-reliability data storage at a relatively low cost. Specifically, when the system is operating normally, the system directly uses memory to achieve high performance. When the system fails (such as downtime), the system persists the data in the memory to the hard disk. When the system is started next time, the data is read from the hard disk to the memory, thereby achieving high reliability.
[0090] The above solution can balance cost and high performance, and can ensure that data is not lost when the system is running normally or abnormally. However, the solution currently has the following two problems:
[0091] 1. High-performance storage systems based on memory are more expensive than ordinary hard disks. In different application scenarios, businesses expect to be able to flexibly choose between memory and hard disk media, and be able to switch between the two media relatively smoothly. To meet such requirements, a hard disk management architecture that supports high data reliability and fast allocation is required so that businesses can apply and use them on demand without pursuing high performance.
[0092] 2. When the persistent memory of a memory-based high-performance storage system is unstable, it is necessary to quickly switch the business request entry to the hard disk to ensure that the business can quickly restore IO (Input Output) after a short period of unavailability (seconds). In this scenario, it is necessary to ensure that the hard disk management is as simple as possible, and during the use of the business, the management and read and write provided by the two storage media (memory and hard disk) to the business are consistent.
[0093] In the related technology, when persisting data in the system memory to the hard disk, the storage area (i.e., PG) where the data is written is usually determined by Ceph (distributed file system). Specifically, Ceph records the location information of each hard disk in the cluster map (cluster heat map), and based on the cluster map, cluster rule and other information, uses the CRUSH (Controlled Replication Under Scalable Hashing) algorithm to determine the osd (Object-based Storage, corresponding to the hard disk) of different fault domain levels to store the PG for writing the data in the memory.
[0094] Although the above method solves the problems of single point failure, performance bottleneck and difficulty in scalability in the centralized architecture, the CRUSH algorithm is relatively complex and is a decentralized algorithm. The creation of the storage area is based on the cluster map. Therefore, there are problems such as uneven data storage between hard disks, which leads to low resource utilization of the entire system. At the same time, since the expansion of osd and the update of cluster map will trigger data synchronization of the entire system, it has a greater impact on business IO.
[0095] To overcome the above problems, an embodiment of the present application provides a storage area creation method, which uses hard disk segments (i.e., target storage areas) as storage units externally and adapts to memory storage units, and uses hard disk objects as the smallest storage units internally, thereby achieving high data reliability and quickly and evenly selecting hard disk objects to create target storage areas, so that when the business specifies hard disk storage data or the persistent memory is unstable, the business can quickly switch to this software management system.
[0096] The present application is further described in detail below in conjunction with the accompanying drawings. Figure 1 FIG. 1 is a flow chart of a storage area creation method provided in an embodiment of the present application. The specific implementation process of the method is as follows:
[0097] Step S101, obtaining at least one storage pool corresponding to the target storage area to be created,
[0098] Each storage pool includes multiple hard disks, and each hard disk includes multiple hard disk objects; Figure 2 As shown, the storage device includes multiple storage pools and multiple hard disk frames. Each hard disk frame includes multiple hard disks, such as Figure 2 As shown, disk enclosure 1 includes hard disks 1, 5, 9, and 13 (the number of hard disks here is only an example). Each storage pool includes multiple hard disks, such as Figure 2 The storage pool 0 shown includes hard disks 1 to 4 (the number here is only an example), and the hard disks in each storage pool may belong to different hard disk enclosures. Figure 2 In storage pool 1, hard disk 1 belongs to disk enclosure 1, hard disk 2 belongs to disk enclosure 2, hard disk 3 belongs to disk enclosure 3, and hard disk 4 belongs to disk enclosure 4.
[0099] In a possible implementation, before obtaining the at least one storage pool, a storage pool creation process needs to be performed, specifically:
[0100] Create each storage pool, and for each storage pool, add the hard disk corresponding to the storage pool and each hard disk object in the hard disk to the storage pool;
[0101] Wherein, for any hard disk object, when it is determined based on the red-black tree algorithm that any hard disk object does not exist in the hard disk objects included in the storage pool, any hard disk object is added to the storage pool.
[0102] In some embodiments, before creating a storage area corresponding to a hard disk, each storage pool needs to be created first in the embodiments of the present application. Specifically, when creating a storage pool, it is necessary to ensure that each hard disk object belongs to a single hard disk, that is, to avoid adding a hard disk object to the storage pool multiple times. To achieve this effect, in the embodiments of the present application, for each storage pool, when injecting the hard disk and hard disk object corresponding to the storage pool (the corresponding relationship is pre-set by the user based on demand) into the storage pool, each hard disk uses a red-black tree to manage the hard disk object, that is, when the hard disk object is injected, first check whether the hard disk object already exists on the hard disk. If the hard disk object does not exist, perform the subsequent injection operation. If the hard disk object exists, the injection of the hard disk object is no longer performed. At the same time, a corresponding error log can be generated to record the information of repeated injection of the current hard disk object for the convenience of user viewing. Since the red-black tree algorithm realizes the association of hard disk objects based on binary fractions, the time complexity corresponding to this operation is O(logn).
[0103] The above operation ensures that each hard disk object belongs to a single hard disk, that is, it effectively avoids adding a hard disk object to the storage pool multiple times, realizes the rapid verification of the hard disk object, and improves the accuracy of the subsequent generation of the target storage area based on the hard disk object in the storage pool; at the same time, since the application adopts the red-black tree algorithm to manage the hard disk object, the time complexity of the operation is O(logn), so the overall performance can meet the application and release of the target storage area in the scenario where BBU (Building Base band Unite, indoor baseband processing unit) cannot be used.
[0104] In a possible implementation, in order to provide different levels of reliability for the business, the embodiment of the present application provides two types (i.e., the first type and the second type) of target storage areas. When the target storage area corresponds to the first type, each target hard disk object in the target storage area belongs to multiple hard disks in multiple storage pools, i.e., the target storage area includes multiple hard disk objects across storage pools; when the target storage area corresponds to the second type, each target hard disk object in the target storage area belongs to multiple hard disks in the same storage pool, i.e., the target storage area includes multiple hard disk objects of a single storage pool. Among them, the first type of target storage area is suitable for data that the normal operation of the storage system depends on and has high reliability requirements, and the second type of target storage area is suitable for storing data that is strongly bound to the life cycle of the storage pool.
[0105] In a specific implementation, when the target storage area in the above step S101 is of the first type, the number of storage pools obtained is the number of storage pools included in the storage system, that is, each storage pool included in the storage system is used as the storage pool corresponding to the target storage area.
[0106] It should be noted that, for the first type of target storage area, when the target storage area is created, if a newly created storage pool appears, the number of storage pools corresponding to the target storage area is updated, that is, an operation of re-creating the target storage area is performed to ensure that the target storage area includes hard disk objects (i.e., copies) in each storage pool.
[0107] When the target storage area in step S101 is of the second type, the number of storage pools obtained is 1, that is, the target storage area corresponds to only one storage pool. It should be noted that, at this time, the storage pool corresponding to the target storage area needs to be pre-set by the user based on demand.
[0108] Step S102: for each storage pool, the steps S1021-1023 are executed cyclically until the determined target hard disk object meets the first preset condition, and a target storage area is constructed based on each obtained target hard disk object:
[0109] In a possible implementation manner, the first preset condition includes at least one of the following:
[0110] The number of the determined target hard disk objects reaches a first preset threshold;
[0111] The number of cycles reaches a second preset threshold;
[0112] The number of available hard disk objects corresponding to the candidate hard disk at the top of the max-heap is zero.
[0113] In some embodiments, different first preset thresholds and second preset thresholds can be set for different types of target storage areas and different storage pools. In specific implementations, the values of the first preset threshold and the second preset threshold can be set based on demand and are not limited in the embodiments of the present application.
[0114] In implementation, the above-mentioned first preset threshold refers to: the first preset threshold of the target hard disk object corresponding to the current storage pool, the sum of the first preset thresholds of the target hard disk objects corresponding to each storage pool, which is the preset total number of target hard disk objects included in the target storage area, wherein the preset total number can be set to: the maximum value between the number of storage pools and the preset number threshold (this value can be set based on demand, and preferably, it can be set to 4).
[0115] It should be noted that if the above total number is the number of storage pools, the first preset threshold corresponding to each storage pool is 1. If the above total number is the preset number threshold, that is, the number of storage pools is less than the preset number threshold, at this time, among the first preset thresholds corresponding to each storage pool, there is at least one that is not 1. If one of the storage pools cannot meet the requirements of the corresponding first preset threshold, it is necessary to modify the first preset thresholds corresponding to other storage pools to meet the requirements of the total number corresponding to the target storage.
[0116] For example, assuming that the number of storage pools is 3, including storage pool 1, storage pool 2 and storage pool 3, and the preset number threshold is 4, then when setting the first preset threshold corresponding to each storage pool, the first preset threshold corresponding to storage pool 1 can be set to 2, the first preset threshold corresponding to storage pool 1 can be set to 1, and the first preset threshold corresponding to storage pool 1 can be set to 1.
[0117] Step S1021, taking out the candidate hard disk at the top of the heap from the largest heap obtained in the previous cycle;
[0118] The hard disks included in the storage pool are arranged in the big-max heap in order according to the number of available hard disk objects included in the hard disks;
[0119] In some embodiments, the embodiments of the present application provide a corresponding max heap for each storage pool, and the max heap includes multiple hard disks sorted in sequence (i.e., the multiple hard disks included in the storage pool), and the multiple hard disks are sorted in sequence according to the number of available hard disk objects, wherein the available hard disk objects refer to unallocated hard disk objects, and the available hard disk objects are arranged in descending order during sorting, that is, the candidate hard disk at the top of the heap is: the hard disk with the largest number of available hard disk objects among the hard disks in the max heap.
[0120] It should be noted that after the candidate hard disk at the top of the heap is taken out from the big top heap, the big top heap no longer includes the candidate hard disk, and the hard disks in the big top heap will be re-sorted.
[0121] In some embodiments, in order to ensure the real-time accuracy of the max heap, the present application may periodically update the max heap corresponding to the storage pool based on the status of the hard disk objects included in each hard disk in the current storage pool. Specifically, the above-mentioned update period can be set based on demand. For example, the period can be set to a fixed value or a dynamic value, and support real-time modification based on demand.
[0122] In the embodiment of the present application, the above-mentioned method can ensure that the candidate hard disks at the top of the max heap have the largest number of available hard disks, thereby ensuring that when selecting the target hard disk object later, priority is given to selecting from the hard disks with the largest number of available hard disk objects, thereby ensuring data balance.
[0123] In some embodiments, before taking out the candidate hard disk at the top of the max heap obtained in the previous loop, it is necessary to determine whether the candidate hard disk includes an available hard disk object. If it is determined that it does not include an available hard disk object, it can be determined that all hard disks in the max heap (i.e., storage pool) are unavailable. At this time, the loop is terminated directly, that is, as described in the first preset condition above, the number of available hard disk objects corresponding to the candidate hard disk at the top of the max heap is zero. At this time, since only one judgment operation is performed to terminate the loop, the time complexity of this operation is O(1). If it is determined that it includes an available hard disk object, the candidate hard disk at the top of the max heap is taken out. At this time, since the hard disks are managed in the form of a tree in the max heap, the time complexity of this operation is O(logn).
[0124] Step S1022, based on the disk enclosure corresponding to the candidate hard disk and the reference disk enclosure corresponding to the hard disk to which each target hard disk object belongs, when it is determined that the candidate hard disk meets the second preset condition, select a hard disk object from the candidate hard disk as the target hard disk object;
[0125] In a possible implementation, the process of determining that a candidate hard disk meets the second preset condition based on the hard disk enclosure corresponding to the candidate hard disk and the hard disk enclosure corresponding to the hard disk to which each determined target hard disk object belongs may be:
[0126] When the number of each reference hard disk frame is less than the preset number, and each reference hard disk frame does not include a hard disk frame corresponding to the candidate hard disk, determining that the candidate hard disk meets the second preset condition; or,
[0127] When the number of each reference hard disk frame is not less than the preset number, it is determined that the candidate hard disk meets the second preset condition.
[0128] In some embodiments, the present application pre-sets a preset number corresponding to the minimum hard disk frame, that is, each target hard disk object in the target storage area needs to be distributed in a preset number of hard disk frames at a minimum. In a specific implementation, the preset number can be set based on demand, for example, it can be set to: the number of target hard disk objects in the target storage area minus 2 (this value can be set based on demand), or a preset value. For example, when the target storage area is of the first type, the preset number is set to 3, and when the target storage area is of the second type, the preset number is set to 2.
[0129] In some embodiments, during each loop, after the candidate hard disk at the top of the heap is taken out, it is necessary to obtain whether the number of hard disk frames (i.e., reference hard disk frames) corresponding to the hard disks to which each target hard disk object has been determined has reached a preset number. If it has reached the preset number, there is no need to consider the hard disk frames corresponding to the candidate hard disk. If it has not reached the preset number, it is necessary to ensure that the hard disk frame corresponding to the candidate hard disk does not belong to the reference hard disk frame.
[0130] In the implementation, if the candidate hard disk does not meet the second preset condition, the subsequent operation will not be performed, the current loop ends, and the next loop is performed. It should be noted that at this time, the candidate hard disk taken out will no longer be inserted into the big top heap, that is, in the next loop, the selected candidate hard disk will be updated, thereby ensuring that the hard disk frame corresponding to the determined target hard disk object meets the above preset number requirements.
[0131] In some embodiments, when it is determined that the candidate hard disk meets the second preset condition, a hard disk object (i.e., an available hard disk) is randomly selected from the candidate hard disks as the target hard disk object. In implementation, since each hard disk is managed based on a red-black tree algorithm, the time complexity of this operation is O(logn).
[0132] Step S1023: based on the number of available hard disk objects included in the current candidate hard disk, determine the target position of the candidate hard disk in the big-max heap, and insert the candidate hard disk into the target position.
[0133] In an embodiment of the present application, when creating a target storage area, when selecting a hard disk object on a hard disk as a copy of the storage area (i.e., a target hard disk object), it is necessary to ensure that only one hard disk object is selected on each hard disk. To achieve this effect, in an embodiment of the present application, before inserting a candidate hard disk into the target position, the candidate hard disk needs to be marked to indicate that the target hard disk object has been selected from the candidate hard disk. At this time, in the above step S1021, before taking out the hard disk at the top of the big top heap obtained in the previous cycle, it is necessary to determine whether the hard disk is marked. If it is not marked, the subsequent steps are executed. If it is marked, the hard disk at the next position at the top of the big top heap is taken out as the candidate hard disk used when executing steps S1022-S1023.
[0134] In a possible implementation manner, after constructing the target storage area, the embodiment of the present application further performs the following operations:
[0135] Based on the target structure, encapsulate the target storage area to obtain a segment structure corresponding to the target storage area;
[0136] Among them, the above-mentioned target structure is: a structure used when encapsulating the storage area corresponding to the memory, and the target structure includes: area type, number of copies and copy information; the area type corresponding to the target storage area is hard disk, the number of copies is the number of hard disk objects included, and the copy information is the object information of each hard disk object.
[0137] In some embodiments, in the present application, the storage area corresponding to the hard disk and the storage area corresponding to the memory are encapsulated into a unified segment structure, and the structure of the segment structure includes: the number of copies, the area type (memory or hard disk) and the copy information.
[0138] Specifically, for the storage area corresponding to the hard disk, its copy information is the hard disk object information (in some embodiments, it can be composed of the hard disk ID (identification) and offset (offset)), the number of copies is the number of hard disk objects included, and the area type is hard disk. In response to the user's request for business reading and writing, the request carries a segment structure (Seg structure) pointer, the starting offset of the read and write area, and the length of the read and write area, wherein the pointer is used to identify the hard disk ID and other information, call the interface encapsulated by the underlying hard disk module, input the hard disk ID and offset, and directly access the hard disk.
[0139] In some embodiments, the target storage area adopts strong consistency writing. When all hard disks are written successfully, the business is returned successfully. If any copy is read successfully, the business is returned successfully.
[0140] For the storage area corresponding to the memory, its copy information is the memory location information, the number of copies is the number of memory units included, and the area type is memory. In response to the user's request for business reading and writing, the corresponding memory block data is directly read and written through the memory location information carried in the request.
[0141] The embodiment of the present application sets a unified segment structure for the storage areas of the memory and the hard disk, and provides a unified read and write interface for the hard disk and the memory. When the persistent memory is unstable, the service request entry can be quickly switched to the hard disk, ensuring that the service can quickly restore IO after a short period of unavailability (seconds).
[0142] In some embodiments, when constructing the target storage area based on the max heap, the present application also supports the rapid deletion of faulty hard disks based on the max heap. Specifically, the max heap relies on the arrays (i.e., array subscripts) corresponding to each hard disk to implement, and each array identifies the arrangement position of the corresponding hard disk in the max heap; when a hard disk fails, the hard disk can be quickly swapped to the end of the max heap based on the array of the hard disk, and then an update operation of the max heap is performed. The time complexity of this operation is O(logn).
[0143] In a possible implementation manner, when the target storage area corresponds to the first type, the embodiment of the present application further performs the following operations:
[0144] When it is determined that a hard disk to which any target hard disk object belongs in the target storage area is faulty, deleting any target hard disk object from the target storage area;
[0145] When it is determined that a disk enclosure corresponding to a hard disk to which any target hard disk object belongs in the target storage area is faulty, deleting any target hard disk object from the target storage area;
[0146] When it is determined that a storage pool corresponding to a hard disk to which any target hard disk object belongs in the target storage area has a fault, deleting any target hard disk object from the target storage area;
[0147] When it is determined that the target storage area includes only one target hard disk object and a storage pool corresponding to a hard disk to which the target hard disk object belongs fails, the storage area corresponding to the hard disk of the target hard disk object is configured as a virtual storage pool.
[0148] In some embodiments, when the target storage area is of the first type, that is, a type that spans storage pools, the failure of any storage pool, hard disk frame, or hard disk corresponding to each target hard disk object in the target storage pool will not affect the life cycle of the target storage area. When a storage pool corresponding to a hard disk to which any target hard disk object belongs in the target storage area fails, any target hard disk object will be deleted from the target storage area. When the target storage area includes only one target hard disk object, and a storage pool corresponding to a hard disk to which the target hard disk object belongs fails, the target storage pool will be switched to the virtual storage pool and continue to exist.
[0149] In a possible implementation manner, when the target storage area corresponds to the second type, the embodiment of the present application further performs the following operations:
[0150] When it is determined that a hard disk to which any target hard disk object belongs in the target storage area is faulty, deleting any target hard disk object from the target storage area;
[0151] When it is determined that a disk enclosure corresponding to a hard disk to which any target hard disk object belongs in the target storage area is faulty, deleting any target hard disk object from the target storage area;
[0152] When it is determined that a storage pool corresponding to a hard disk to which any target hard disk object belongs in the target storage area is faulty, the target storage area is deleted.
[0153] In some embodiments, when the target storage area is of the second type, that is, a single storage pool type, the failure of any hard disk frame or hard disk corresponding to each target hard disk object in the target storage pool will not affect the life cycle of the target storage area, but the life cycle of the storage pool corresponding to the target hard disk will be consistent with the life cycle of the target UCUN out-area. When the storage pool corresponding to the hard disk to which the target hard disk object belongs in the target storage area fails or is deleted, the target storage pool will be deleted or inaccessible.
[0154] The following combination Figure 3 The process of creating a target storage area of the second type (i.e., a single pool) is described in detail with a specific example. The process specifically includes the following steps:
[0155] Step S301, obtaining a storage pool corresponding to the target storage area to be created;
[0156] Step S302, determine whether the storage pool is in an available state, if yes, proceed to step S303, if not, proceed to step S301;
[0157] If the storage pool is currently allocable and includes allocable hard disks, it is determined that the storage pool is in an available state; otherwise, it is determined that the storage pool is in an unavailable state.
[0158] Step S303, taking out the candidate hard disk at the top of the heap from the big top heap corresponding to the storage pool;
[0159] Step S304, determining whether the candidate hard disk is in a usable state, if yes, proceed to step S305, if no, proceed to step S303;
[0160] When the candidate hard disk is currently allocable and there is an available hard disk object, it is determined that the candidate hard disk is in an available state; otherwise, it is determined that the candidate hard disk is in an unavailable state.
[0161] Step S305, based on the disk enclosure corresponding to the candidate hard disk and the reference disk enclosure corresponding to the hard disks to which each target hard disk object belongs, determine whether the candidate hard disk meets the second preset condition; if so, execute step S306, if not, execute step S303;
[0162] Among them, the second preset condition is: the number of each reference hard disk frame is less than the preset number, and each reference hard disk frame does not include the hard disk frame corresponding to the candidate hard disk; or, the number of each reference hard disk frame is not less than the preset number.
[0163] Step S306, selecting a hard disk object from the candidate hard disks as the target hard disk object;
[0164] Step S307: insert the selected hard disk into a corresponding position in the max-heap based on the number of available hard disk objects included in the current candidate hard disk.
[0165] Step S308, determining whether the loop satisfies the first preset condition, if so, executing step S309, otherwise returning to step S303.
[0166] Step S309: construct a target storage area based on each target hard disk object.
[0167] The following combination Figure 4 The process of creating a target storage area of the first type (i.e., across pools) is described in detail with a specific example. The process specifically includes the following steps:
[0168] Step S401, obtaining each storage pool corresponding to the target storage area to be created;
[0169] Step S402, for a storage pool, determine whether the storage pool is in an available state, if yes, execute step S403, if not, execute step S401;
[0170] If the storage pool is currently allocable and includes allocable hard disks, it is determined that the storage pool is in an available state; otherwise, it is determined that the storage pool is in an unavailable state.
[0171] Step S403, taking out the candidate hard disk at the top of the heap from the big top heap corresponding to the storage pool;
[0172] Step S404, determining whether the candidate hard disk is in a usable state, if yes, proceed to step S405, if no, proceed to step S403;
[0173] When the candidate hard disk is currently allocable and there is an available hard disk object, it is determined that the candidate hard disk is in an available state; otherwise, it is determined that the candidate hard disk is in an unavailable state.
[0174] Step S405, based on the disk enclosure corresponding to the candidate hard disk and the reference disk enclosure corresponding to the hard disks to which each target hard disk object belongs, determine whether the candidate hard disk meets the second preset condition; if so, execute step S406, if not, execute step S403;
[0175] Among them, the second preset condition is: the number of each reference hard disk frame is less than the preset number, and each reference hard disk frame does not include the hard disk frame corresponding to the candidate hard disk; or, the number of each reference hard disk frame is not less than the preset number.
[0176] Step S406, selecting a hard disk object from the candidate hard disks as the target hard disk object;
[0177] Step S407: insert the selected hard disk into a corresponding position in the max-heap based on the number of available hard disk objects included in the current candidate hard disk.
[0178] Step S408, determining whether the loop satisfies the first preset condition, if so, proceed to step S409, otherwise, return to step S403;
[0179] Step S409, determining whether each storage pool is traversed and the number of acquired target hard disk objects reaches a preset total number, if yes, executing step S410, if no, executing step S402;
[0180] It should be noted that, when the process returns to step S402, the one storage pool refers to the next storage pool of the current storage pool.
[0181] Step S410: construct a target storage area based on each target hard disk object.
[0182] Based on the same inventive concept, the present application also provides a storage area creation device, see Figure 5 , the device comprises:
[0183] The acquisition module 501 is used to acquire at least one storage pool corresponding to the target storage area to be created, wherein the storage pool includes multiple hard disks, and each hard disk includes multiple hard disk objects;
[0184] The construction module 502 includes a first submodule 5021, a second submodule 5022 and a third submodule 5023, which is used to loop through the following process for each storage pool until the determined target hard disk object meets the first preset condition, and construct the above-mentioned target storage area based on each obtained target hard disk object:
[0185] The first submodule 5021 takes out the candidate hard disk at the top of the big-top heap obtained in the previous cycle; wherein the hard disks included in the storage pool are arranged in sequence in the big-top heap according to the number of available hard disk objects included in the hard disks;
[0186] By the second submodule 5022, based on the hard disk enclosure corresponding to the candidate hard disk and the reference hard disk enclosure corresponding to the hard disk to which each determined target hard disk object belongs, when it is determined that the candidate hard disk meets the second preset condition, a hard disk object is selected from the candidate hard disk as the target hard disk object;
[0187] Through the third submodule 5023, based on the number of available hard disk objects included in the current candidate hard disk, the target position of the candidate hard disk in the big-max heap is determined, and the candidate hard disk is inserted into the target position.
[0188] In a possible implementation manner, the first preset condition includes at least one of the following:
[0189] The number of determined target objects reaches a first preset threshold;
[0190] The number of cycles reaches a second preset threshold;
[0191] The number of available hard disk objects corresponding to the candidate hard disk at the top of the max-heap is zero.
[0192] In a possible implementation manner, the second submodule 5022 is specifically used to:
[0193] When the number of each reference hard disk frame is less than the preset number, and each reference hard disk frame does not include a hard disk frame corresponding to the candidate hard disk, determining that the candidate hard disk meets the second preset condition; or,
[0194] When the number of each reference hard disk frame is not less than the preset number, it is determined that the candidate hard disk meets the second preset condition.
[0195] In a possible implementation, when the target storage area corresponds to the first type, each target hard disk object in the target storage area belongs to multiple hard disks in multiple storage pools, and the above-mentioned device further includes a first management module, which is specifically used to:
[0196] When it is determined that a hard disk to which any target hard disk object belongs in the target storage area is faulty, deleting any target hard disk object from the target storage area;
[0197] When it is determined that a disk enclosure corresponding to a hard disk to which any target hard disk object belongs in the target storage area is faulty, deleting any target hard disk object from the target storage area;
[0198] When it is determined that a storage pool corresponding to a hard disk to which any target hard disk object belongs in the target storage area has a fault, deleting any target hard disk object from the target storage area;
[0199] When it is determined that the target storage area includes only one target hard disk object and a storage pool corresponding to a hard disk to which the target hard disk object belongs fails, the storage area corresponding to the hard disk of the target hard disk object is configured as a virtual storage pool.
[0200] In a possible implementation, when the target storage area corresponds to the second type, each target hard disk object in the target storage area belongs to multiple hard disks in the same storage pool, and the above-mentioned device further includes a second management module, which is specifically used to:
[0201] When it is determined that a hard disk to which any target hard disk object belongs in the target storage area is faulty, deleting any target hard disk object from the target storage area;
[0202] When it is determined that a disk enclosure corresponding to a hard disk to which any target hard disk object belongs in the target storage area is faulty, deleting any target hard disk object from the target storage area;
[0203] When it is determined that a storage pool corresponding to a hard disk to which any target hard disk object belongs in the target storage area is faulty, the target storage area is deleted.
[0204] In a possible implementation manner, before acquiring at least one storage pool corresponding to the target storage area to be created, the acquisition module 501 is further configured to:
[0205] Create each storage pool, and for each storage pool, add the hard disk corresponding to the storage pool and each hard disk object in the hard disk to the storage pool;
[0206] Wherein, for any hard disk object, when it is determined based on the red-black tree algorithm that any hard disk object does not exist in the storage objects included in the storage pool, any hard disk object is added to the storage pool.
[0207] In a possible implementation manner, after constructing the target storage area, the construction module 502 is further configured to:
[0208] Based on the target structure, encapsulate the target storage area to obtain a segment structure corresponding to the target storage area;
[0209] Among them, the target structure is: a structure used when encapsulating the storage area corresponding to the memory, and the target structure includes: area type, number of copies and copy information; the area type corresponding to the target storage area is hard disk, the number of copies is the number of hard disk objects included, and the copy information is the object information of each hard disk object.
[0210] Based on the same inventive concept, an electronic device is also provided in an embodiment of the present application, and the electronic device can realize the function of the aforementioned storage area creation device, referring to Figure 6 , the electronic device comprises:
[0211] At least one processor 601, and a memory 602 connected to the at least one processor 601. The specific connection medium between the processor 601 and the memory 602 is not limited in the embodiment of the present application. Figure 6 In the example, the processor 601 and the memory 602 are connected via a bus 600. Figure 6 The connection between other components is shown by bold lines, and is not intended to be limiting. The bus 600 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. Alternatively, the processor 601 can also be called a controller, and there is no limitation on the name.
[0212] In the embodiment of the present application, the memory 602 stores instructions that can be executed by at least one processor 601. The at least one processor 601 can execute the storage area creation method discussed above by executing the instructions stored in the memory 602. The processor 601 can implement Figure 5 The functions of each module in the device shown.
[0213] Among them, the processor 601 is the control center of the device, and can use various interfaces and lines to connect the various parts of the entire control device. By running or executing instructions stored in the memory 602 and calling the data stored in the memory 602, the various functions of the device and processing data, the device can be monitored as a whole.
[0214] In one possible design, the processor 601 may include one or more processing units, and the processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 601. In some embodiments, the processor 601 and the memory 602 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.
[0215] Processor 601 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the storage area creation method disclosed in the embodiments of the present application can be directly embodied as a hardware processor to be executed, or a combination of hardware and software modules in the processor can be executed.
[0216] The memory 602 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 602 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 602 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 602 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.
[0217] By designing and programming the processor 601, the code corresponding to the application anomaly detection method described in the above embodiment can be fixed into the chip, so that the chip can execute the code when running. Figure 1 The steps of the storage area creation method of the embodiment shown are as follows: How to design and program the processor 601 is a technique known to those skilled in the art and will not be described in detail here.
[0218] An embodiment of the present application also provides a computer-readable storage medium that stores computer-executable instructions required to execute the above-mentioned processor, which includes a program required to execute the above-mentioned processor.
[0219] In some possible implementations, various aspects of the storage area creation method provided in the present application may also be implemented in the form of a program product, which includes a program code. When the above-mentioned program product is run on an electronic device, the above-mentioned program code is used to enable the above-mentioned electronic device to execute the steps of the storage area creation method according to various exemplary embodiments of the present application described above in this specification.
[0220] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0221] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (apparatus), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0222] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0223] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0224] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.
[0225] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A method for creating a storage area, characterized in that: The method comprises: Acquire at least one storage pool corresponding to the target storage area to be created, wherein the storage pool includes a plurality of hard disks, and each hard disk includes a plurality of hard disk objects; For each storage pool, the following process is executed cyclically until the determined target hard disk object meets the first preset condition, and the target storage area is constructed based on each obtained target hard disk object: From the max-heap obtained in the last cycle, the candidate hard disk at the top of the heap is taken out; wherein the hard disks included in the storage pool are arranged in sequence in the max-heap according to the number of available hard disk objects included in the hard disk; Based on the hard disk enclosure corresponding to the candidate hard disk and the reference hard disk enclosure corresponding to the hard disk to which each determined target hard disk object belongs, when it is determined that the candidate hard disk meets the second preset condition, select a hard disk object from the candidate hard disk as the target hard disk object; Based on the number of available hard disk objects included in the current candidate hard disk, a target position of the candidate hard disk in the big-max heap is determined, and the candidate hard disk is inserted into the target position.
2. The method according to claim 1, characterized in that The first preset condition includes at least one of the following: The number of the determined target hard disk objects reaches a first preset threshold; The number of cycles reaches a second preset threshold; The number of available hard disk objects corresponding to the candidate hard disk at the top of the max-heap is zero.
3. The method according to claim 1, characterized in that The determining that the candidate hard disk meets the second preset condition based on the hard disk enclosure corresponding to the candidate hard disk and the reference hard disk enclosure corresponding to the hard disks to which each determined target hard disk object belongs includes: When the number of each reference hard disk frame is less than the preset number, and each reference hard disk frame does not include a hard disk frame corresponding to the candidate hard disk, determining that the candidate hard disk meets the second preset condition; or, When the number of the reference hard disk frames is not less than the preset number, it is determined that the candidate hard disk meets the second preset condition.
4. The method according to claim 1, characterized in that When the target storage area corresponds to the first type, each target hard disk object in the target storage area belongs to multiple hard disks in multiple storage pools, and the method further includes: When it is determined that a hard disk to which any target hard disk object belongs in the target storage area is faulty, deleting any target hard disk object from the target storage area; When it is determined that a hard disk enclosure corresponding to a hard disk to which any target hard disk object belongs in the target storage area is faulty, deleting the target hard disk object from the target storage area; When it is determined that a storage pool corresponding to a hard disk to which any target hard disk object belongs in the target storage area is faulty, deleting any target hard disk object from the target storage area; When it is determined that the target storage area includes only one target hard disk object and a storage pool corresponding to a hard disk to which the target hard disk object belongs fails, the storage area corresponding to the hard disk of the target hard disk object is configured as a virtual storage pool.
5. The method according to claim 1, characterized in that When the target storage area corresponds to the second type, each target hard disk object in the target storage area belongs to multiple hard disks in the same storage pool, and the method further includes: When it is determined that a hard disk to which any target hard disk object belongs in the target storage area is faulty, deleting any target hard disk object from the target storage area; When it is determined that a hard disk enclosure corresponding to a hard disk to which any target hard disk object belongs in the target storage area is faulty, deleting the target hard disk object from the target storage area; When it is determined that a storage pool corresponding to a hard disk to which any target hard disk object in the target storage area belongs has a fault, the target storage area is deleted.
6. The method according to any one of claims 1 to 5, characterized in that: Before acquiring at least one storage pool corresponding to the target storage area to be created, the method further includes: Creating each of the storage pools, and for each storage pool, adding a hard disk corresponding to the storage pool and each hard disk object in the hard disk to the storage pool; Wherein, for any hard disk object, when it is determined based on the red-black tree algorithm that the hard disk object does not exist in the hard disk objects included in the storage pool, the hard disk object is added to the storage pool.
7. The method according to any one of claims 1 to 5, characterized in that: After constructing the target storage area, the method further includes: Based on the target structure, encapsulate the target storage area to obtain a segment structure corresponding to the target storage area; Among them, the target structure is: a structure used when encapsulating the storage area corresponding to the memory, and the target structure includes: area type, number of copies and copy information; the area type corresponding to the target storage area is hard disk, the number of copies is the number of hard disk objects included, and the copy information is the object information of each hard disk object.
8. A storage area creation device, characterized in that: The device comprises: An acquisition module, used for acquiring at least one storage pool corresponding to the target storage area to be created, wherein the storage pool includes a plurality of hard disks, and each hard disk includes a plurality of hard disk objects; The construction module is used to cyclically execute the following process for each storage pool until the determined target hard disk object meets the first preset condition, and construct the target storage area based on each obtained target hard disk object: From the max-heap obtained in the last cycle, the candidate hard disk at the top of the heap is taken out; wherein the hard disks included in the storage pool are arranged in sequence in the max-heap according to the number of available hard disk objects included in the hard disk; Based on the hard disk enclosure corresponding to the candidate hard disk and the reference hard disk enclosure corresponding to the hard disk to which each determined target hard disk object belongs, when it is determined that the candidate hard disk meets the second preset condition, select a hard disk object from the candidate hard disk as the target hard disk object; Based on the number of available hard disk objects included in the current candidate hard disk, a target position of the candidate hard disk in the big-max heap is determined, and the candidate hard disk is inserted into the target position.
9. An electronic device, characterized in that: include: A memory for storing program instructions; A processor is used to call the program instructions stored in the memory, and execute the steps included in any one of claims 1-7 according to the obtained program instructions.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Method and device for allocating distributed type storage magnetic discs
CN104484134A
Equilibrium method and device for storage resource
CN104699540A
Storage user-defined data storage method, system and device
CN108874913A
Defensive heap memory management
US20040221120A1
Systems and / or methods for leveraging in-memory storage in connection with the shuffle phase of mapreduce
US20160034205A1