Method and device for creating read cache layer and storage medium
By creating a ZRAM virtual device in the cloud pool server as a read cache layer for HDD storage disks, the problem of excessive power consumption during random reads of HDD storage disks is solved, thus achieving the effect of reducing the power consumption of HDD storage disks.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA UNITED NETWORK COMM GRP CO LTD
- Filing Date
- 2023-12-12
- Publication Date
- 2026-06-16
AI Technical Summary
HDD storage disks consume excessive power when randomly reading data, leading to increased energy consumption.
Create ZRAM virtual devices in the servers within the cloud pool as read cache layers for HDD storage hard drives to reduce extensive random read access to HDD hard drives.
By creating a read cache layer, the power consumption of HDD storage drives is reduced, thus improving energy efficiency.
Smart Images
Figure CN117707989B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a method, apparatus and storage medium for creating a read cache layer. Background Technology
[0002] As solid-state drive (SSD) hardware and software technologies continue to mature, SSD capacity is constantly increasing, and its advantages are becoming more and more apparent. However, due to historical reasons, cloud providers still have a large number of traditional hard disk drives (HDDs) in their cloud pools for production environments.
[0003] HDD storage disks typically read data using a read / write head arm. However, as capacity increases, the head arm amplitude increases during random reads, leading to excessive power consumption. Summary of the Invention
[0004] This application provides a method, apparatus, and storage medium for creating a read cache layer, which solves the problem of excessive power consumption of current HDD storage disks. It can create a read cache layer for HDD storage disks, reduce large-scale random read accesses to HDD storage disks, and thus effectively reduce the power consumption of HDD storage disks.
[0005] To achieve the above objectives, this application adopts the following technical solution:
[0006] In a first aspect, this application provides a method for creating a read cache layer, the method comprising: obtaining the idle rate of the memory space of each server in a cloud pool, wherein the operating system of the server has the ability to create memory-optimized ZRAM virtual devices; the cloud pool includes multiple servers and traditional hard disks (HDDs); for a target server whose idle rate is greater than or equal to a preset threshold, creating at least one ZRAM virtual device in a target area of the memory space of the target server; and setting the at least one ZRAM virtual device as a read cache layer for the HDD.
[0007] In conjunction with the first aspect mentioned above, in one possible implementation, the method further includes: dividing the target region of the memory space according to a preset memory size to obtain at least one sub-target region; and creating a ZRAM virtual device in each sub-target region.
[0008] In conjunction with the first aspect above, in one possible implementation, the method further includes: adding at least one ZRAM virtual device to the HDD storage disk; creating ZRAM storage memory based on the at least one ZRAM virtual device; and setting the ZRAM storage memory as a read cache layer of the HDD storage disk.
[0009] In conjunction with the first aspect mentioned above, in one possible implementation, the method further includes: monitoring the idle rate of the target server's memory space based on a preset period; if the idle rate of the target server's memory space is less than a preset threshold after the ZRAM virtual device is created, then deleting the ZRAM virtual device corresponding to the target region of the target server.
[0010] In conjunction with the first aspect above, in one possible implementation, the method further includes: adding the target server to the target list; servers in the target list waiting to reclaim the target region; and sequentially removing at least one ZRAM virtual device of the target server in the target list from the HDD storage hard disk.
[0011] Secondly, this application provides a read cache layer creation apparatus, which includes: a communication unit and a processing unit; the communication unit is used to obtain the idle rate of the memory space of each server in a cloud pool, wherein the operating system of the server has the ability to create memory-optimized ZRAM virtual devices; the cloud pool includes multiple servers and traditional hard disk drives (HDDs); the processing unit is used to create at least one ZRAM virtual device in a target area of the memory space of a target server whose idle rate is greater than or equal to a preset threshold; the processing unit is also used to set the at least one ZRAM virtual device as a read cache layer of the HDD.
[0012] In conjunction with the second aspect above, in one possible implementation, the processing unit is specifically used to: divide the target region of the memory space according to a preset memory size to obtain at least one sub-target region; and for each sub-target region, create a ZRAM virtual device in each sub-target region.
[0013] In conjunction with the second aspect above, in one possible implementation, the processing unit is specifically used for: adding at least one ZRAM virtual device to the HDD storage disk; creating ZRAM storage memory based on at least one ZRAM virtual device; and setting the ZRAM storage memory as a read cache layer of the HDD storage disk.
[0014] In conjunction with the second aspect above, in one possible implementation, the processing unit is further configured to: monitor the idle rate of the target server's memory space based on a preset period; if the idle rate of the target server's memory space is less than a preset threshold after the ZRAM virtual device is created, then delete the ZRAM virtual device corresponding to the target region of the target server.
[0015] In conjunction with the second aspect above, in one possible implementation, the processing unit is specifically used for: adding the target server to the target list; the servers in the target list waiting to reclaim the target area; and sequentially removing at least one ZRAM virtual device of the target server in the target list from the HDD storage hard disk.
[0016] Thirdly, this application provides an apparatus for creating a read cache layer, the apparatus comprising: a processor and a communication interface; the communication interface and the processor are coupled, the processor being used to run computer programs or instructions to implement the read cache layer creation method as described in the first aspect and any possible implementation thereof.
[0017] Fourthly, this application provides a computer-readable storage medium storing instructions that, when executed on a terminal, cause the terminal to perform a method for creating a read cache layer as described in the first aspect and any possible implementation thereof.
[0018] Fifthly, this application provides a computer program product containing instructions that, when run on a read cache layer creation apparatus, causes the read cache layer creation apparatus to perform the read cache layer creation method as described in the first aspect and any possible implementation thereof.
[0019] In a sixth aspect, this application provides a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run computer programs or instructions to implement the method for creating a read cache layer as described in the first aspect and any possible implementation thereof.
[0020] Specifically, the chip provided in this application also includes a memory for storing computer programs or instructions.
[0021] It should be noted that the aforementioned computer instructions may be stored, in whole or in part, on a computer-readable storage medium. This computer-readable storage medium may be packaged together with the processor of the device, or it may be packaged separately from the processor of the device; this application does not impose any limitation on this.
[0022] In a seventh aspect, this application provides a system for creating a read cache layer, comprising: a server and an HDD storage disk, wherein the server is used to execute the method for creating a read cache layer as described in the first aspect and any possible implementation thereof.
[0023] The descriptions of aspects two through seven in this application can be referenced to the detailed description of aspect one; and the beneficial effects of the descriptions of aspects two through seven can be referenced to the analysis of the beneficial effects of aspect one, which will not be repeated here.
[0024] In this application, the name of the aforementioned read cache layer creation device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear under other names. As long as the function of each device or functional module is similar to that of this application, it falls within the scope of the claims of this application and its equivalents.
[0025] These or other aspects of this application will become more readily apparent in the following description.
[0026] The above solution offers at least the following advantages: Based on the above technical solution, the method for creating a read cache layer provided in this application first obtains the free space rate of the memory space of servers within the cloud pool capable of creating memory-optimized ZRAM virtual devices. Then, at least one ZRAM virtual device is created in the target area of the target server's memory space. If the free space rate of the target server is greater than or equal to a preset threshold, it indicates that the target server has a significant amount of free memory, capable of providing read caching for HDD storage disks. Furthermore, the at least one ZRAM virtual device is set as the read cache layer for the HDD storage disk. Compared to the current problem of excessively high power consumption of HDD storage disks, the above technical solution can create a read cache layer for HDD storage disks, reducing large-scale random read accesses to the HDD storage disks, thereby effectively reducing the power consumption of the HDD storage disks. Attached Figure Description
[0027] Figure 1 A schematic diagram of a cloud pool architecture provided for an embodiment of this application;
[0028] Figure 2 A schematic diagram of the architecture of a read cache layer creation system provided in this application embodiment;
[0029] Figure 3 A schematic diagram of the hardware structure of a read cache layer creation device provided in an embodiment of this application;
[0030] Figure 4 A flowchart illustrating a method for creating a read cache layer, as provided in an embodiment of this application;
[0031] Figure 5 A flowchart illustrating another method for creating a read cache layer provided in this application embodiment;
[0032] Figure 6 This is a schematic diagram of a read cache layer creation device provided in an embodiment of this application. Detailed Implementation
[0033] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0034] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.
[0035] The terms "first" and "second," etc., used in the specification and drawings of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.
[0036] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.
[0037] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.
[0038] In the description of this application, unless otherwise stated, "a plurality of" means two or more.
[0039] The following explanations of the terms used in the embodiments of this application are provided to facilitate the reader's understanding.
[0040] (1) The main resource pools within the cloud pool
[0041] like Figure 1 As shown, the cloud pool includes core backbone switches, cloud pool egress firewalls, and many hardware resources. The largest and most core resource pool is mainly divided into two categories: computing resource pool and storage resource pool.
[0042] Specifically, computing resource pools generally include computing resource pools, bare metal resource pools, and graphics processing unit (GPU) resource pools. Storage resource pools generally include block storage pools, file storage pools, and object storage pools. In addition to computing and storage resource pools, cloud pools also include security resource pools and cloud management pools.
[0043] The computing resource pool consists of N computing servers and access switches. The bare metal resource pool consists of N bare metal servers and access switches. The GPU resource pool consists of N GPU servers and access switches. Both the block storage pool and file storage pool consist of N storage servers and access switches. The object storage pool consists of N raw storage servers and access switches. The security resource pool consists of N security servers and access switches. The cloud management pool consists of N management servers and access switches.
[0044] For a storage pool, taking a block storage pool as an example, it includes the following different types of storage servers, with several of each type of storage server:
[0045] 1) Several NVMe SSD all-flash storage servers (providing high-speed cloud disks);
[0046] 2) Several SATA SSD all-flash storage servers (providing all-flash cloud disks);
[0047] 3) Several SSD+HDD hybrid flash storage servers (providing high-efficiency enhanced cloud disks);
[0048] 4) Several HDD storage servers (providing high-efficiency cloud disks).
[0049] The performance provided by the cloud disks in 1)-4) above decreases in that order. File storage pools and object storage pools can also include several of the storage servers in 1)-4) above, providing file storage and object storage with corresponding different performance indicators, which will not be elaborated here.
[0050] (2) Memory optimization technology (ZRAM)
[0051] ZRAM is a memory optimization technique for Linux systems. It supports allocating a memory region, creating ZRAM virtual devices, and features automatic compression. This means that calling the ZRAM virtual device's write interface automatically compresses the data before writing it to memory. Furthermore, calling the ZRAM virtual device's read interface automatically decompresses the data and returns the decompressed data.
[0052] (3) Distributed storage cluster (Ceph cache tier)
[0053] Within the same Ceph distributed storage cluster, a resource pool consisting of high-speed disks (such as SSD-pool) and a resource pool consisting of low-speed disks (such as HDD-pool) can be configured as a cachetier layer for HDD-pool, accelerating read and write operations for HDD-pool. The cachetier can be configured as a read cache, write cache, read-write cache, and various other caching strategies, such as cache flushing.
[0054] As solid-state drive (SSD) hardware and software technologies continue to mature, SSD capacity is constantly increasing, and its advantages are becoming increasingly apparent. HDD storage disks are mechanical disks, which consume significantly more power than SSDs. However, due to historical reasons, cloud providers still have a large number of HDD storage disks in their production cloud pools.
[0055] Distributed storage systems built on HDD storage disks offer relatively low-performance cloud disks. High-performance cloud disks are generally used in scenarios requiring large capacity and relatively low performance requirements. Currently, HDD storage disks can support capacities of up to 20 terabytes (T). Most HDD storage disks have capacities between 10T and 20T.
[0056] For random writes, high-capacity HDD storage disks are typically configured with a media cache and a flash-based non-volatile cache (NVC). During random small data writes, the data can be cached through the media cache or NVC, and then flushed to the disk media from the cache.
[0057] Specifically, data is typically sorted in a cache before being flushed to the disk media, reducing head seek time and improving random write performance to some extent. Therefore, for large-capacity HDD storage disks, the power consumption variation in random write performance is relatively stable across different capacity areas.
[0058] For random reads, HDDs lack optimization mechanisms and typically read data directly from the disk using the read / write head arm. However, as capacity increases, the head arm swings more during random reads, leading to excessive power consumption. For example, random reads in the 0-5TB range of a 10TB HDD consume significantly more power than those in the 0-1TB range.
[0059] Therefore, the read cache layer creation method provided in this application first obtains the free space rate of the memory space of servers within the cloud pool that have the capability to create memory-optimized ZRAM virtual devices. Then, at least one ZRAM virtual device is created in a target area of the target server's memory space. If the free space rate of the target server is greater than or equal to a preset threshold, it indicates that there is a significant amount of free memory in the target server, enabling it to provide read caching for HDD storage disks. Furthermore, the at least one ZRAM virtual device is set as the read cache layer for the HDD storage disk. Compared to the current problem of excessively high power consumption of HDD storage disks, the above technical solution can create a read cache layer for HDD storage disks, reducing large-scale random read accesses to the HDD storage disks, thereby effectively reducing the power consumption of the HDD storage disks.
[0060] The embodiments of this application will now be described in detail with reference to the accompanying drawings.
[0061] Figure 2 This is a schematic diagram of the architecture of a read cache layer creation system provided in an embodiment of this application. Figure 2 As shown, the system for creating the read cache layer includes: a first server 201, a second server cluster 202, and an HDD storage hard disk 203.
[0062] The first server 201 is connected to the second server cluster 202 via a communication link. The second server cluster 202 is connected to the HDD storage hard drive 203 via a communication link. The first server 201 is also connected to the HDD storage hard drive 203 via a communication link. This communication link can be a wired communication link or a wireless communication link; this application does not limit the type of link.
[0063] It should be noted that the operating systems (such as Linux kernels) of the servers in the second server cluster 202 have the ability to create ZRAM virtual devices. The first server 201, the second server cluster 202, and the HDD storage disk 203 are resources within the same cloud pool.
[0064] In one possible implementation, the first server 201 is used to monitor the memory space of the servers in the second server cluster 202, and set the memory that meets the preset conditions as the read cache layer of the HDD storage hard disk 203 to reduce the power consumption of the HDD storage hard disk 203 during random reads.
[0065] In one possible implementation, the second server cluster 202 includes computing servers and storage servers.
[0066] Specifically, compute servers can be servers within the compute resource pool of a cloud pool. Storage servers can be servers within block storage pools, file storage pools, and object storage pools.
[0067] In one possible implementation, the HDD storage hard disk 203 is used to store hot data in the read cache layer. When an instruction to read target data from the HDD storage hard disk 203 is received, if the target data has been pre-stored in the read cache layer, the target data in the read cache layer can be read directly, avoiding reading data from the HDD storage hard disk 203 through the read head arm, thereby reducing the power consumption of the HDD storage hard disk 203 during random reads.
[0068] If the target data is not pre-stored in the read cache layer, it needs to be read from the HDD storage hard disk 203 via the read / write head arm.
[0069] When implemented in hardware, the various modules in the first server 201 can be integrated into, for example... Figure 3 The read cache layer creation device shown is implemented on the hardware structure. Specifically, as... Figure 3 As shown, the basic hardware structure of the read cache layer creation device is introduced.
[0070] Figure 3 This is a schematic diagram of the hardware structure of a read cache layer creation device provided in an embodiment of this application. Figure 3 As shown, the apparatus for creating the read cache layer includes at least one processor 301, a communication line 302, and at least one communication interface 304, and may also include a memory 303. The processor 301, memory 303, and communication interface 304 are connected via the communication line 302.
[0071] The processor 301 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).
[0072] Communication line 302 may include a path for transmitting information between the aforementioned components.
[0073] Communication interface 304 is used to communicate with other devices or communication networks. It can use any transceiver-like device, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc.
[0074] The memory 303 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of including or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.
[0075] In one possible design, the memory 303 can exist independently of the processor 301, meaning the memory 303 can be an external memory of the processor 301. In this case, the memory 303 can be connected to the processor 301 via a communication line 302 to store execution instructions or application code, and its execution is controlled by the processor 301, implementing the read cache layer creation method provided in the following embodiments of this application. In another possible design, the memory 303 can also be integrated with the processor 301, meaning the memory 303 can be an internal memory of the processor 301. For example, the memory 303 can be a cache, used to temporarily store some data and instruction information.
[0076] As one possible implementation, processor 301 may include one or more CPUs, for example Figure 3 CPU0 and CPU1 in the example. As another possible implementation, the means for creating the read cache layer may include multiple processors, such as... Figure 3 The processors 301 and 307 are included. As another possible implementation, the means for creating the read cache layer may also include an output device 305 and an input device 306.
[0077] It should be noted that the various embodiments of this application can be referenced or learned from each other. For example, the same or similar steps, method embodiments, system embodiments and device embodiments can be referenced from each other without limitation.
[0078] Figure 4 A flowchart illustrating a method for creating a read cache layer provided in this application embodiment, which can be applied to, for example... Figure 3In the device for creating the read cache layer shown. For example... Figure 4 As shown, the method includes the following S401-S403.
[0079] S401. Obtain the free memory rate of each server in the cloud pool.
[0080] One possible implementation involves retrieving the operating system information of all servers within the cloud pool. Servers whose operating systems do not support ZRAM are removed, and servers whose operating systems support ZRAM are added to a first list (e.g., exclusive_list). Then, the memory space free rate of each server in the first list is iterated over.
[0081] Specifically, for each server, the ratio of the amount of idle memory in the server to the total memory space of the server is used as the idle rate of the server's memory space.
[0082] In one example, let's say the server has 100GB of memory. If 30GB of that memory is already occupied, then the server has 70GB of idle memory, meaning the memory idle rate is 70%.
[0083] S402. For a target server with an idle rate greater than or equal to a preset threshold, create at least one ZRAM virtual device in the target area of the target server's memory space.
[0084] The preset threshold can be 70%, and the preset threshold can be set according to the actual situation. This application does not limit it.
[0085] It should be noted that when the free space rate of the target server's memory is greater than or equal to the preset threshold, it indicates that there is a large amount of free space in the target server's memory, which is capable of providing read cache for HDD storage hard drives.
[0086] In one example, target servers with an idle rate greater than or equal to a preset threshold in the first list are added to a second list (e.g., a shared_list), while target servers with an idle rate less than the preset threshold in the first list remain unchanged and remain in the first list. Each target server in the second list is traversed, and at least one ZRAM virtual device is created in the target region of the target server's memory space.
[0087] In one possible implementation, the process of creating at least one ZRAM virtual device may include steps 21-22.
[0088] Step 21: Divide the target area of the memory space according to the preset memory size to obtain at least one sub-target area.
[0089] The target area can be 30% of the memory space. For example, if the memory space is 100GB, the target area can be 30GB. The default memory size is 10GB.
[0090] In one example, a target region with a memory size of 30G is divided into 3 sub-target regions with a granularity of 10G.
[0091] Step 22: For each sub-target region, create a ZRAM virtual device in each sub-target region.
[0092] In one example, a ZRAM virtual device is created in each of the three sub-target regions.
[0093] S403. Set at least one ZRAM virtual device as a read cache layer for HDD storage hard disk.
[0094] In one possible implementation, the process of setting up the read cache layer may include the following steps 31-33.
[0095] Step 31: Add at least one ZRAM virtual device to the HDD storage hard drive.
[0096] In one example, a ZRAM virtual device is added to an HDD storage disk as an object storage device (OSD) so that the HDD storage disk has the right to use the ZRAM virtual device.
[0097] Step 32: Create ZRAM storage memory based on at least one ZRAM virtual device.
[0098] In one example, multiple ZRAM virtual devices added to an HDD storage hard drive are used as ZRAM storage memory (zram_pool).
[0099] Optionally, the disaster recovery domain of zram_pool is at the OSD level.
[0100] Step 33: Set the ZRAM storage memory as the read cache layer of the HDD storage hard disk.
[0101] In one example, the zram_pool storage pool is set as a read cache tier (read-only) for HDD storage disks.
[0102] It's important to note that the zram_pool storage pool is a general-purpose storage pool. Besides providing read caching acceleration for HDD storage, it can also independently provide block, file, and object storage services. Therefore, other services in the cloud pool with caching needs can request cache space from the zram_pool storage pool. However, before using the zram_pool storage pool, other services in the cloud pool with caching needs must first obtain the memory space composition of the zram_pool storage pool. If the zram_pool storage pool meets their business scenario requirements, they can then request and use it. The zram_pool storage pool provides a global and flexible option for the memory-level caching needs of all services in the cloud pool.
[0103] Based on the above technical solution, the method for creating a read cache layer provided in this application first obtains the free space rate of the memory space of servers within the cloud pool that have the capability to create memory-optimized ZRAM virtual devices. Then, at least one ZRAM virtual device is created in a target area of the target server's memory space. If the free space rate of the target server is greater than or equal to a preset threshold, it indicates that the target server has a significant amount of free memory, capable of providing read caching for HDD storage disks. Further, the at least one ZRAM virtual device is set as the read cache layer for the HDD storage disk. Compared to the current problem of excessively high power consumption of HDD storage disks, the above technical solution can create a read cache layer for HDD storage disks, reducing large-scale random read accesses to the HDD storage disks, thereby effectively reducing the power consumption of the HDD storage disks.
[0104] As one possible embodiment of this application, combined with Figure 4 ,like Figure 5 As shown, in the method for creating the read cache layer, the process of deleting the created ZRAM virtual device through the read cache layer creation device can be implemented through the following S501-S502.
[0105] S501: Monitor the idle rate of the target server's memory space based on a preset period.
[0106] The preset period can be 10 minutes, and the preset period can be set according to actual needs. This application does not limit this.
[0107] One possible implementation involves adding target servers with idle rates greater than or equal to a preset threshold to a second list. The memory idle rate of each server in the second list is monitored at preset intervals.
[0108] Understandably, after creating a ZRAM virtual device, the target server's memory free rate will gradually decrease as its own application processes consume memory. Conversely, as the target server's own application processes terminate, the target server's memory will be released, meaning the target server's memory free rate will increase.
[0109] S502. If the free rate of the target server's memory space is less than a preset threshold after the ZRAM virtual device is created, then delete the ZRAM virtual device corresponding to the target region of the target server.
[0110] It should be noted that if the idle rate of the target server's memory space is less than the preset threshold after the ZRAM virtual device is created, it means that the target server's memory space is insufficient for its own needs. The shared target area needs to be reclaimed to maintain its own application processes and enhance its own reliability.
[0111] In one possible implementation, the process of deleting the ZRAM virtual device corresponding to the target region of the target server can be achieved through the following steps 41-42.
[0112] Step 41: Add the target server to the target list.
[0113] Among them, the servers in the target list are waiting to reclaim the target areas.
[0114] In one example, after creating a ZRAM virtual device, if the free space rate of the target server's memory is less than 70%, the target server is moved from the second list to the target list (such as shrinking_list) to wait for the created ZRAM virtual device to be deleted and the target region to be reclaimed.
[0115] Step 42: Remove at least one ZRAM virtual device from the HDD storage hard drive of the target server in the target list.
[0116] In one example, for each target server in the target list chain, the ZRAM virtual device corresponding to each target server is traversed, and the ZRAM virtual devices are kicked out of the cluster one by one.
[0117] Specifically, kicking out a ZRAM virtual device triggers a rebuild. After the rebuild is completed, a second ZRAM virtual device is kicked out, and so on, until all ZRAM virtual devices shared by the target server are kicked out, at which point the target area is released.
[0118] Furthermore, the target server that has been released is added to the first list, and the above S401-S403 are executed repeatedly to achieve the purpose of providing a read cache layer for the HDD storage hard disk again.
[0119] It should be noted that, since all ZRAM devices are memory media, their rebuild and eject speeds are very fast.
[0120] Based on the above technical solution, the idle rate of the target server's memory space is monitored at a preset period. If the idle rate of the target server's memory space is less than a preset threshold after the creation of the ZRAM virtual device, the ZRAM virtual device corresponding to the target region on the target server is deleted. This technical solution reclaims the shared target region when the target server's memory space is insufficient to meet its needs, thereby maintaining its own application processes and enhancing its reliability.
[0121] This application embodiment can divide the read cache layer creation device into functional modules or functional units according to the above method example. For example, each function can be divided into a separate functional module or functional unit, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or in software functional modules or functional units. The module or unit division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.
[0122] like Figure 6 The diagram shown is a structural schematic of a read cache layer creation device 60 provided in an embodiment of this application. The device includes a communication unit 601 and a processing unit 602.
[0123] The communication unit 601 is used to obtain the idle rate of memory space of each server in the cloud pool. The operating system of the server has the ability to create memory-optimized ZRAM virtual devices. The cloud pool includes multiple servers and traditional hard disk (HDD) storage hard disks.
[0124] The processing unit 602 is used to create at least one ZRAM virtual device in the target area of the memory space of a target server with an idle rate greater than or equal to a preset threshold.
[0125] The processing unit 602 is also configured to set at least one ZRAM virtual device as a read cache layer for HDD storage hard disks.
[0126] The processing unit 602 is specifically used to: divide the target area of the memory space according to a preset memory size to obtain at least one sub-target area; and create a ZRAM virtual device in each sub-target area.
[0127] The processing unit 602 is specifically used for: adding at least one ZRAM virtual device to the HDD storage disk; creating ZRAM storage memory based on at least one ZRAM virtual device; and setting the ZRAM storage memory as the read cache layer of the HDD storage disk.
[0128] The processing unit 602 is also configured to: monitor the idle rate of the target server's memory space based on a preset period; if the idle rate of the target server's memory space is less than a preset threshold after the ZRAM virtual device is created, then delete the ZRAM virtual device corresponding to the target region of the target server.
[0129] The processing unit 602 is specifically used for: adding the target server to the target list; the servers in the target list waiting to reclaim the target area; and sequentially removing at least one ZRAM virtual device of the target server in the target list from the HDD storage hard disk.
[0130] In one possible implementation, the read cache layer creation device 60 may further include a storage unit 603. Figure 6 (shown in dashed box) The storage unit 603 stores a program or instruction. When the processing unit 602 executes the program or instruction, the read cache layer creation device 60 can execute the read cache layer creation method described in the above method embodiment.
[0131] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0132] This application provides a computer program product containing instructions that, when run on a computer, cause the computer to execute the method for creating a read cache layer in the above method embodiments.
[0133] This application also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method for creating a read cache layer in the method flow shown in the above method embodiments.
[0134] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires; portable computer disks; hard disks; random access memory (RAM); read-only memory (ROM); erasable programmable read-only memory (EPROM); registers; hard disks; optical fibers; portable compact disc read-only memory (CD-ROM); optical storage devices; magnetic storage devices; or any suitable combination thereof; or any other form of computer-readable storage medium known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may reside in an application-specific integrated circuit (ASIC). In the embodiments of this application, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0135] Since the read cache layer creation device, computer-readable storage medium, and computer program product in the embodiments of this application can be applied to the above method, the technical effects that can be obtained can also be referred to the above method embodiments. The embodiments of this application will not be repeated here.
[0136] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0137] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0138] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0139] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method for creating a read cache layer, characterized in that, The method includes: The system obtains the free memory rate of each server in the cloud pool, wherein the operating system of the server has the ability to create memory-optimized ZRAM virtual devices; the cloud pool includes multiple servers and traditional hard disk (HDD) storage disks. For a target server with an idle rate greater than or equal to a preset threshold, at least one ZRAM virtual device is created in the target region of the target server's memory space; Add the at least one ZRAM virtual device to the HDD storage hard disk; Based on the at least one ZRAM virtual device, a ZRAM storage memory is created; wherein, the ZRAM storage memory is configured as a general storage pool within the cloud pool to provide memory-level caching space for other services within the cloud pool; The ZRAM storage memory is configured as a read cache layer for the HDD storage hard disk; the read cache layer is used to cache hot data read from the HDD storage hard disk; The method further includes: Based on a preset period, monitor the idle rate of the target server's memory space; If the free memory rate of the target server is less than the preset threshold after the ZRAM virtual device is created, the target server is added to the target list; the servers in the target list wait to reclaim the target area. At least one ZRAM virtual device of the target server in the target list is removed from the HDD storage hard drive in sequence.
2. The method according to claim 1, characterized in that, Creating at least one ZRAM virtual device in the target region of the target server's memory space includes: The target region of the memory space is divided according to a preset memory size to obtain at least one sub-target region; For each sub-target region, create a ZRAM virtual device in each sub-target region.
3. An apparatus for creating a read cache layer, characterized in that, The device includes a communication unit and a processing unit; The communication unit is used to obtain the idle rate of memory space of each server in the cloud pool. The operating system of the server has the ability to create memory-optimized ZRAM virtual devices. The cloud pool includes multiple servers and traditional hard disk (HDD) storage disks. The processing unit is used to create at least one ZRAM virtual device in a target area of the memory space of a target server with an idle rate greater than or equal to a preset threshold. The processing unit is further configured to set the at least one ZRAM virtual device as a read cache layer of the HDD storage hard disk; The processing unit is specifically used for: Add the at least one ZRAM virtual device to the HDD storage hard disk; Based on the at least one ZRAM virtual device, a ZRAM storage memory is created; wherein, the ZRAM storage memory is configured as a general storage pool within the cloud pool to provide memory-level caching space for other services within the cloud pool; The ZRAM storage memory is configured as a read cache layer for the HDD storage hard disk; the read cache layer is used to cache hot data read from the HDD storage hard disk; The processing unit is further configured to: Based on a preset period, monitor the idle rate of the target server's memory space; If the free rate of the memory space of the target server is less than the preset threshold after the ZRAM virtual device is created, then the ZRAM virtual device corresponding to the target region of the target server is deleted. The processing unit is specifically used for: The target server is added to the target list; the servers in the target list await the reclaiming of the target area. At least one ZRAM virtual device of the target server in the target list is removed from the HDD storage hard drive in sequence.
4. The apparatus according to claim 3, characterized in that, The processing unit is specifically used for: The target region of the memory space is divided according to a preset memory size to obtain at least one sub-target region; For each sub-target region, create a ZRAM virtual device in each sub-target region.
5. An apparatus for creating a read cache layer, characterized in that, include: A processor and a communication interface; the communication interface is coupled to the processor, which is used to run computer programs or instructions to implement the method for creating a read cache layer as described in any one of claims 1-2.
6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed by a computer, perform the method for creating a read cache layer as described in any one of claims 1-2.