A distributed storage asynchronous data compression method
By adopting asynchronous data compression method and Run To Completion thread model in distributed storage systems, the impact of data compression on I/O paths is stripped off, and the problem of balancing storage space and computing overhead is solved, improving storage performance and efficiency.
Patent Information
- Application Number
- CN202311605230.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-28
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2043-11-28
AI Technical Summary
When existing distributed storage systems balance storage space and compression computing overhead, data compression still has a great impact on the delay of I/O paths, and it is difficult to effectively improve read and write performance.
Using asynchronous data compression method, data compression and decompression are stripped from the I/O path, and cached and efficient secondary indexes are provided in the I/O read path, and the writing process is accelerated through the Run To Completion thread model and WAL method, independent Compression threads handle compression tasks to reduce the impact on I/O threads.
It significantly reduces the impact of data compression and decompression on read and write delay, improves storage performance, reduces lock and context switching overhead, improves I/O thread processing efficiency, and reduces storage costs.
Smart Images

Figure CN117632016B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer storage technology, in particular, to the technical field of distributed storage technology; specifically, to a distributed storage asynchronous data compression method. Background Art
[0002] The distributed storage system adopts an extensible system structure, distributing and storing data on multiple independent data servers, using multiple data servers to share the storage load, and then abstracting the storage resources into logical space for users to use. Distributed storage technology can improve the reliability and scalability of the system to meet different usage scenarios.
[0003] In distributed storage, to improve reliability, user data is generally stored in the cluster in a redundant manner, such as the commonly used multiple copies and erasure code storage. In order to save storage space, data compression is usually used to compress data before writing it to the disk, saving disk space and reducing storage costs.
[0004] Data compression is divided into lossy compression and lossless compression. Since the storage system has high requirements for data accuracy, lossless compression is used. Common lossless compression algorithms include Huffman coding, LZW, predictive coding, etc.
[0005] Since data compression requires calculations, it will consume CPU computing resources, so the selection of compression algorithms needs to be optimized: for scenarios with high real-time requirements, use algorithms with lower compression ratios or no compression to minimize delays on the I / O path; for some application scenarios with less real-time requirements, generally choose algorithms with higher compression ratios to further save storage space, however, this also increases the computing overhead and overall delay.
[0006] The current solutions still cannot strike a good balance between storage space and the computational overhead of compression; and data compression still has varying degrees of impact on the latency on the I / O path. Summary of the invention
[0007] In view of this, the first purpose of the present invention is to propose an asynchronous data compression storage solution in distributed storage, further reduce the impact of data compression on read and write delays, separate data compression and decompression from the I / O path, compress and decompress data in an asynchronous manner, and provide cache and efficient secondary indexes on the I / O read path to significantly reduce the impact of data compression and decompression on read and write delays.
[0008] The second objective of the present invention is to propose a distributed storage persistence layer data storage solution to give full play to the writing capability of the disk and further improve the reading and writing performance.
[0009] The present invention provides a distributed storage asynchronous data compression method, including: a method for deploying a server client, a method for writing data, and a method for reading data;
[0010] The method for deploying the server client comprises the following steps:
[0011] S01. Use the Run To Completion thread model to deploy multiple persistence layer server nodes. Each node Host runs a persistence layer service, and the fault domain is the Host level.
[0012] S02. Each of the persistence layer server nodes manages two disks, each of which is divided into three partitions: a data partition, a metadata partition, and a log partition, and the three partitions share the same physical disk;
[0013] S03, deploy a storage cluster metadata management node m0, the persistence layer service regularly reports the disk status to m0, and m0 records which disks in the cluster each data block is stored on;
[0014] S04. Deploy a persistence layer client process c0 to send read and write requests to the storage cluster;
[0015] S05, applying for a data block chunk0 from the storage cluster metadata management node m0 through the persistence layer client process c0, and c0 generates a request to write chunk0, for example, writing 0-260KB;
[0016] S06. m0 records which disks in the cluster the data of data block chunk0 is stored on, and the persistence layer client process c0 sends a write request to the I / O threads of the persistence layer service corresponding to the disks.
[0017] The data writing method comprises the following steps:
[0018] S11, the I / O thread of the persistence layer service is used to poll the I / O requests in the queue. After polling the request, if it is parsed as a write request from the client, the write process is entered;
[0019] S12, persist the original data of each I / O thread in the log disk in the form of a sequence seq; after writing the data of a seq to the log disk, update the super information of the log disk; write the cache at the same time, and write the original data into the JnodeBlock of the cache Jnode;
[0020] Each seq occupies a sector-aligned space in the log disk, which stores the metadata and original data of the seq. The seq metadata is the header information, which is used to describe the location of the metadata and data in this space and the location of the original data in the logical data block, and is used for log playback.
[0021] S13. After the persistence layer completes the WAL process of the data, the persistence layer server replies to the client: writing is successful;
[0022] After the data completes the WAL process, it is guaranteed that the data will not be lost;
[0023] If the process crashes suddenly, the data can be restored by replaying the log disk;
[0024] S14, checking whether the JnodeBlock of the cache Jnode is full or the number is more than 1 through the I / O thread, if the JnodeBlock is full or the number is more than 1, then starting the refresh process;
[0025] S15, submitting the data compression task to the compress thread through the I / O thread;
[0026] The first step of the flushing process is to compress the data. Since the compression calculation is time-consuming, in order not to affect the I / O thread processing other requests, the present invention assigns the data compression task to the compress thread. After the data compression is completed, a compression completion message is submitted to the I / O thread. The I / O thread parses the compression completion message and continues the flushing process of the I / O thread.
[0027] S16, updating the index information of the data block after data compression, and saving the updated index information in the metadata Cnode of the data block and the Record of the data disk;
[0028] The present invention divides the space of the data disk into blocks of the same size. The metadata Cnode describes which blocks the data block occupies on the data disk and the position of each Compressed JnodeBlock in the Block. Cnode occupies at least one sector of the metadata partition. If Cnode records more metadata, it can be expanded to occupy multiple sectors. What is stored in each Block is the compressed Compressed JnodeBlock. Each JnodeBlock is compressed in units of Cell, and the compressed Cell is a Compressed Cell. The Compressed JnodeBlock consists of multiple Compressed Cells. The header of the Compressed JnodeBlock will generate a Record to record the position of each Compressed Cell in the current Block;
[0029] In order to improve the performance of reading and writing, the present invention adopts the WAL method to accelerate the writing process: the log disk adopts high-speed storage hardware, such as NVME SSD, etc. The operation log is first written into the log disk space, and then asynchronously persisted to the data disk after responding to the user request, and the space of the log disk is released after persisting to the data disk. If the program crashes, the context before the crash is restored by replaying the log disk data. Since sequential writing can better exert the read and write capabilities of fast storage media compared to random writing, the present invention adopts the append writing method to store data to the persistence layer, and builds an index on the persistence layer to record the location of user data stored in the persistence layer. Since the compression and decompression of data are synchronous, and the submission of read and write requests by the I / O thread is asynchronous, in order not to affect the efficiency of the I / O thread processing, the present invention puts the compression and decompression of data into a separate thread. From the perspective of the I / O thread, compression and decompression are also asynchronous, which can avoid the I / O thread waiting during the compression and decompression process, avoid affecting the I / O thread processing other I / O requests, and the compression time can be completely separated from the write delay on the write path, thereby improving the write performance.
[0030] S17, submitting an I / O thread write request to the data disk and metadata disk in an asynchronous manner, and writing the compressed data block (Compressed JnodeBlock) and the metadata Cnode of the data block to the disk;
[0031] S18, persisting the compressed data and Cnode metadata of the data block to the data disk and metadata partition;
[0032] S19. After persistence is completed, the log disk space and JnodeBlock cache occupied by the I / O thread are released;
[0033] The data reading method comprises the following steps:
[0034] S21, the I / O thread is used to poll the I / O requests in the queue. After polling the request, if it is parsed as a read request from the client, the read process is entered;
[0035] S22. According to the order of logical addresses of data blocks from low to high, the complete data of each data block is composed of multiple blocks recorded by Cnode and the cache in the JnodeBlock dynamic array recorded by Jnode. The compressed data in the disk corresponds to the low address data of the data block, and the data in the JnodeBlock cache corresponds to the high address data of the data block. According to the interval read by the client, it is confirmed whether the data to be read is in the JnodeBlock cache, in the data disk, or in both the cache and the disk.
[0036] S23, if the read request interval hits the cache, then the cache read is performed; if the read request interval hits the disk space, then the data disk read is performed;
[0037] The data disk reading method comprises: locating which Compressed JnodeBlocks in which Blocks to be read according to the index information recorded in the metadata Cnode, firstly reading the Record of the Compressed JnodeBlock, locating the position of the target Compressed Cell in the Block according to the information recorded in the Record, and sending a data decompression task to the compressthread thread after the reading is completed;
[0038] S24, after the decompression is completed by the compress thread, a decompression completion message is sent to the message queue of the I / O thread; after the I / O thread polls the decompression completion message, it parses it and takes out the decompressed data;
[0039] The index layer sends I / O requests to the persistence layer server through the persistence layer client. The persistence layer compresses the received data and stores it on disk to complete persistence.
[0040] S25. Return all data read from the cache or disk to the client.
[0041] The data compression and decompression in the present invention are performed in the persistence layer.
[0042] Furthermore, the super information in step S12 of the data writing method records the writing position information of the current log disk, and the log disk adopts an append writing method, and the space is recycled.
[0043] Furthermore, in step S23 of the data reading method, if the data disk is not read, the client is directly replied after the cache read in step S23 is completed.
[0044] Furthermore, the cache reading method of step S23 of the data reading method includes:
[0045] The data in the JnodeBlock cache is not compressed, and the target data can be directly retrieved from the cache without decompression.
[0046] Furthermore, the method of decompressing by compress thread in step S24 of the data reading method includes:
[0047] 256KB-260KB data is directly read in JnodeBlock without decompression;
[0048] The 200-256KB data is on the disk. According to the Block and Compressed Block secondary index information recorded in the metadata Cnode, the target data is located on the data disk. After the data is read successfully, the data is decompressed and then concatenated with the data in the cache and returned to c0.
[0049] Furthermore, the method for setting the data partition in step S02 of the method for deploying the server client includes:
[0050] Set the maximum capacity of each data block chunk of the data partition to 128MB and the size of each block to 4M.
[0051] Furthermore, the method for setting the metadata partition in step S02 of the method for deploying the server client includes:
[0052] Set the minimum metadata space occupied by each Cnode of the metadata partition to 512B.
[0053] Furthermore, the method for setting the log partition in step S02 of the method for deploying the server client includes:
[0054] Set the size of each JnodeBlock cache interval of the log partition to 256KB; when the cached data fills a 256KB JnodeBlock, trigger the flushing and compression process; set the data compression unit to 16KB, and split a 256KB JnodeBlock into 16 16KB cells for compression.
[0055] The present invention deploys the nodes of the persistence layer server in a replica manner, runs a persistence layer service on each node (Host), and the persistence layer server adopts a Run To Completion thread model. Each I / O thread corresponds to one or more disks. The I / O threads are completely independent and there is no data sharing or interaction. The read and write operations in the I / O threads are asynchronous read and write operations. Each data disk may store data of multiple data blocks, which will be processed concurrently in the I / O threads. Figure 3 The figure shows the process of each I / O thread processing one I / O.
[0056] The present invention puts data compression and decompression into a compression thread for processing. After the processing is completed, the compression thread sends a completion message to the message queue of the I / O thread. The I / O thread obtains the completion message through polling and continues to process the next process of the I / O thread, thereby reducing the impact of data compression and decompression on reading and writing.
[0057] In addition, different compression algorithms can be selected in the compression thread, or dedicated hardware can be used to further optimize the compression rate and compression time.
[0058] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the distributed storage asynchronous data compression method as described above are implemented.
[0059] The present invention also provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the distributed storage asynchronous data compression method as described above are implemented.
[0060] Compared with the prior art, the present invention has the following beneficial effects:
[0061] The present invention puts compression and decompression into independent threads for execution, temporarily persists the client's write request using a log disk (WAL) based on a fast storage medium, and puts the timing of compressed data into an asynchronous flushing process, which basically eliminates the impact of compression on write latency. The append write writing method is disk-friendly and gives full play to the write performance. At the same time, small I / Os can be merged based on the JnodeBlock cache, further improving the write performance. For read requests, a processing method of cache + Cnode, Record secondary index + independent decompression thread is adopted, which minimizes the read latency and improves the read concurrency capability. In addition, the Run To Completion thread model is adopted to reduce the overhead of locks, interrupts and context switches, greatly improves the efficiency of I / O thread processing, and greatly improves the storage performance while reducing storage costs. BRIEF DESCRIPTION OF THE DRAWINGS
[0062] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the following detailed description of the preferred embodiment.The drawings are only for the purpose of illustrating the preferred embodiments and are not to be construed as limiting the invention.
[0063] In the attached picture:
[0064] Figure 1 A schematic diagram of the location of the persistence layer in a storage cluster according to an embodiment of the present invention;
[0065] Figure 2 This is a general architecture diagram of a distributed storage system according to an embodiment of the present invention;
[0066] Figure 3 A schematic diagram of a process for processing an I / O write request by a persistence layer server according to an embodiment of the present invention;
[0067] Figure 4 A schematic diagram of updating an index after data compression according to an embodiment of the present invention;
[0068] Figure 5 A schematic diagram of the structure of a computer device according to an embodiment of the present invention;
[0069] Figure 6 A flow chart of a method for deploying a server client according to an embodiment of the present invention;
[0070] Figure 7 A flow chart of a method for writing data according to an embodiment of the present invention;
[0071] Figure 8 The present invention is a flowchart of a method for reading data according to an embodiment of the present invention. DETAILED DESCRIPTION
[0072] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure. Instead, they are merely examples of systems and products consistent with some aspects of the present disclosure as detailed in the appended claims.
[0073] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.
[0074] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".
[0075] The embodiments of the present invention are described in further detail below.
[0076] The application scenario of the embodiment of the present invention belongs to the field of distributed storage technology, and belongs to the data persistence layer in the storage cluster. The location of the persistence layer in the storage cluster is as follows: Figure 1 shown.
[0077] The embodiment of the present invention provides a distributed storage asynchronous data compression method, including: a method for deploying a server client, a method for writing data, and a method for reading data;
[0078] The method for deploying the server client is described in Figure 6 As shown, the following steps are included:
[0079] S01. Use the Run To Completion thread model to deploy multiple persistence layer server nodes. Each node Host runs a persistence layer service, and the fault domain is the Host level.
[0080] S02. Each of the persistence layer server nodes manages two disks, each of which is divided into three partitions: a data partition, a metadata partition, and a log partition, and the three partitions share the same physical disk;
[0081] In this embodiment, the maximum capacity of each data block chunk of the data partition is set to 128MB, and the size of each Block is 4M.
[0082] The method for setting the metadata partition in step S02 of the method for deploying the server client includes:
[0083] Set the minimum metadata space occupied by each Cnode of the metadata partition to 512B.
[0084] The method for setting the log partition in step S02 of the method for deploying the server client includes:
[0085] Set the size of each JnodeBlock cache interval of the log partition to 256KB; when the cached data fills a 256KB JnodeBlock, trigger the flushing and compression process; set the data compression unit to 16KB, and split a 256KB JnodeBlock into 16 16KB cells for compression.
[0086] S03, deploy a storage cluster metadata management node m0, the persistence layer service regularly reports the disk status to m0, and m0 records which disks in the cluster each data block is stored on;
[0087] S04. Deploy a persistence layer client process c0 to send read and write requests to the storage cluster;
[0088] S05, applying for a data block chunk0 from the storage cluster metadata management node m0 through the persistence layer client process c0, c0 generates a request to write chunk0, preferably writing 0-260KB;
[0089] S06. m0 records which disks in the cluster the data of data block chunk0 is stored on, and the persistence layer client process c0 sends a write request to the I / O threads of the persistence layer service corresponding to the disks.
[0090] For the method of writing data, see Figure 7 As shown, the following steps are included:
[0091] S11, the I / O thread of the persistence layer service is used to poll the I / O requests in the queue. After polling the request, if it is parsed as a write request from the client, the write process is entered;
[0092] S12, persist the original data of each I / O thread in the log disk in the form of a sequence seq; after writing the data of a seq to the log disk, update the super information of the log disk; write the cache at the same time, and write the original data into the JnodeBlock of the cache Jnode;
[0093] The super information records the write location information of the current log disk. The log disk adopts the append-write method and the space is recycled.
[0094] Each seq occupies a sector-aligned space in the log disk, which stores the metadata and original data of the seq. The seq metadata is the header information, which is used to describe the location of the metadata and data in this space and the location of the original data in the logical data block, and is used for log playback.
[0095] S13. After the persistence layer completes the WAL process of the data, the persistence layer server replies to the client: writing is successful;
[0096] After the data completes the WAL process, it is guaranteed that the data will not be lost;
[0097] If the process crashes suddenly, the data can be restored by replaying the log disk;
[0098] S14, checking whether the JnodeBlock of the cache Jnode is full or the number is more than 1 through the I / O thread, if the JnodeBlock is full or the number is more than 1, then starting the refresh process;
[0099] S15, submitting the data compression task to the compress thread through the I / O thread;
[0100] The first step of the flushing process is to compress the data. Since the compression calculation is time-consuming, in order not to affect the I / O thread processing other requests, the present invention assigns the data compression task to the compress thread. After the data compression is completed, a compression completion message is submitted to the I / O thread. The I / O thread parses the compression completion message and continues the flushing process of the I / O thread.
[0101] S16, updating the index information of the data block after data compression, and saving the updated index information in the metadata Cnode of the data block and the Record of the data disk;
[0102] See also Figure 4As shown, in this embodiment, the space of the data disk is divided into blocks of the same size. The metadata Cnode describes which blocks the data block occupies on the data disk and the position of each Compressed JnodeBlock in the Block. Cnode occupies at least one sector of the metadata partition. If Cnode records more metadata, it can be expanded to occupy multiple sectors. What is stored in each Block is the compressed Compressed JnodeBlock. Each JnodeBlock is compressed in units of Cell. The compressed Cell is a Compressed Cell. The CompressedJnodeBlock consists of multiple Compressed Cells. The header of the Compressed JnodeBlock will generate a Record to record the position of each Compressed Cell in the current Block;
[0103] Distributed storage systems generally support all-flash and HDD. In order to improve the reading and writing performance, this embodiment adopts the WAL method to accelerate the writing process: the log disk adopts high-speed storage hardware, such as NVME SSD, etc. The operation log is first written to the log disk space, and then asynchronously persisted to the data disk after responding to the user request. After persisting to the data disk, the space of the log disk is released. If the program crashes, the context before the crash is restored by replaying the log disk data. Since sequential writing can better exert the reading and writing capabilities of fast storage media than random writing, the present invention adopts an append write method to store data to the persistence layer, and builds an index on the persistence layer to record the location of user data stored in the persistence layer. Since data compression and decompression are synchronous, and the I / O thread submits read and write requests asynchronously, in order not to affect the efficiency of I / O thread processing, the present invention puts data compression and decompression into separate threads. From the perspective of the I / O thread, compression and decompression are also asynchronous, which can avoid the I / O thread waiting during the compression and decompression process and avoid affecting the I / O thread in processing other I / O requests. In addition, on the write path, the compression time can be completely separated from the write delay, thereby improving the write performance.
[0104] The overall architecture of this embodiment is as follows Figure 2 As shown, Figure 2 The meanings of several concepts involved are explained as follows:
[0105] (1) Cnode:
[0106] Represents the metadata of the data block in the data disk, recording the space occupied by a data block on the data disk. The space on the data disk is divided into blocks. The space occupied by data blocks is allocated when writing, and the granularity of applying for space is in blocks. Cnode also records the storage location of each compressed JnodeBlock in the Block. Compression is in cells, and each JnodeBlock is split into multiple cells. The Record in the header of Compressed JnodeBlock records the location of each compressed Cell in the Block.
[0107] Cnode+Record constitutes a secondary index for accessing compressed data on disk: find the location of CompressedJnodeBlock through Cnode, and find the location of Compressed Cell through Record;
[0108] (2) Jnode:
[0109] Metadata indicating the data block in the log disk. The log disk space is recycled, and the data from the client is in sequence (sequence Figure 2 seq1, seq2, etc.) in the log disk space. Each seq contains the data to be written by a write request and the metadata of the seq itself. The metadata records the location of the metadata itself and the data in the seq area after sector alignment, as well as the location of the data in the logical data block for log playback. Jnode records which seqs in the log partition belong to this data block. Jnode also caches client data, which can merge small I / Os and reduce the number of writes to disk. The cache is based on the granularity of JnodeBlock. Each JnodeBlock caches a portion of seq data. The space occupied by JnodeBlock is a fixed value. When the cache is full, a new JnodeBlock will be applied for. Jnode maintains a dynamic array of JnodeBlocks. At the same time, Jnode also saves a pointer to Cnode, such as Figure 2 As shown in the figure, Cnode indicates the metadata that has been flushed, and Writing Cnode indicates the metadata that is being flushed. The complete data of a data block is determined by Cnode and Jnode. Cnode will be persisted to the metadata partition, and Jnode will be saved in memory. If the server process crashes, Jnode can be rebuilt by replaying the data on the log disk.
[0110] S17, submitting an I / O thread write request to the data disk and metadata disk in an asynchronous manner, and writing the compressed data block (Compressed JnodeBlock) and the metadata Cnode of the data block to the disk;
[0111] S18, persisting the compressed data and Cnode metadata of the data block to the data disk and metadata partition;
[0112] S19. After persistence is completed, the log disk space and JnodeBlock cache occupied by the I / O thread are released;
[0113] For the method of reading the data, see Figure 8 As shown, the following steps are included:
[0114] S21, the I / O thread is used to poll the I / O requests in the queue. After polling the request, if it is parsed as a read request from the client, the read process is entered;
[0115] S22. According to the order of logical addresses of data blocks from low to high, the complete data of each data block is composed of multiple blocks recorded by Cnode and the cache in the JnodeBlock dynamic array recorded by Jnode. The compressed data in the disk corresponds to the low address data of the data block, and the data in the JnodeBlock cache corresponds to the high address data of the data block. According to the interval read by the client, it is confirmed whether the data to be read is in the JnodeBlock cache, in the data disk, or in both the cache and the disk.
[0116] S23, if the read request interval hits the cache, then the cache read is performed; if the read request interval hits the disk space, then the data disk read is performed;
[0117] The data disk reading method comprises: locating which Compressed JnodeBlocks in which Blocks to be read according to the index information recorded in the metadata Cnode, firstly reading the Record of the Compressed JnodeBlock, locating the position of the target Compressed Cell in the Block according to the information recorded in the Record, and sending a data decompression task to the compressthread thread after the reading is completed;
[0118] If the data disk is not read, the client is directly replied after the cache read in step S23 is completed.
[0119] The data in the JnodeBlock cache is not compressed, and the target data can be directly retrieved from the cache without decompression.
[0120] S24, after the decompression is completed by the compress thread, a decompression completion message is sent to the message queue of the I / O thread; after the I / O thread polls the decompression completion message, it parses it and takes out the decompressed data;
[0121] 256KB-260KB data is directly read in JnodeBlock without decompression;
[0122] The 200-256KB data is on the disk. According to the Block and Compressed Block secondary index information recorded in the metadata Cnode, the target data is located on the data disk. After the data is read successfully, the data is decompressed and then concatenated with the data in the cache and returned to c0.
[0123] The index layer sends I / O requests to the persistence layer server through the persistence layer client. The persistence layer compresses the received data and stores it on disk to complete persistence.
[0124] S25. Return all data read from the cache or disk to the client.
[0125] The data compression and decompression in the present invention are performed in the persistence layer.
[0126] See also Figure 3 As shown in the figure, the nodes of the persistence layer server are deployed in a replica manner, and a persistence layer service is run on each node (Host). The persistence layer server adopts the Run To Completion thread model. Each I / O thread corresponds to one or more disks. The I / O threads are completely independent and there is no data sharing or interaction. The read and write operations in the I / O threads are asynchronous read and write operations. Each data disk may store data of multiple data blocks, which will be processed concurrently in the I / O threads. Figure 3 The figure shows the process of each I / O thread processing one I / O.
[0127] In this embodiment, data compression and decompression are processed by a compression thread. After the processing is completed, the compression thread sends a completion message to the message queue of the I / O thread. The I / O thread obtains the completion message through polling and continues to process the next process of the I / O thread, thereby reducing the impact of data compression and decompression on reading and writing.
[0128] Different compression algorithms can be selected in the compression thread, or dedicated hardware can be used to further optimize the compression rate and compression time.
[0129] Application Examples
[0130] (1) Deploy three persistence layer server nodes in a three-replica manner. Each node (host) runs a persistence layer service. The three nodes are h0, h1, and h2. The three persistence layer services are s0, s1, and s2. The failure domain is the host level.
[0131] (2) Each persistence layer server node manages 2 disks, a total of 6 disks (d0-d5), of which s0 manages d0 and d1, s1 manages d2 and d3, and s2 manages d4 and d5. Each disk is divided into 3 partitions: data partition, metadata partition and log partition, and the 3 partitions share the same physical disk. Set the maximum capacity of each data block (chunk) to 128MB; the size of each Block is 4M; the minimum metadata space occupied by each Cnode is 512B; the size of each JnodeBlock cache interval is 256KB, that is: when the cached data fills a 256KB JnodeBlock, the flush and compression process will be triggered. Set the unit of data compression to 16KB, that is: split a 256KB JnodeBlock into 16 16KB Cells for compression;
[0132] (3) Deploy a storage cluster metadata management node m0. The persistence layer services s0-s2 regularly report the disk status to m0. m0 records which disks in the cluster each data block is stored on.
[0133] (4) Deploy a persistence layer client process c0 to send read and write requests to the storage cluster;
[0134] (5) The c0 process applies for a data block chunk0 from the m0 node. c0 generates a request to write chunk0, for example, 0-260KB.
[0135] (6) Node m0 records that the data of chunk0 falls on d0 of s0, d2 of s1, and d4 of s2. c0 sends write requests to the I / O threads of d0, d2, and d4 respectively;
[0136] (7) After receiving the write request from c0, the I / O thread first writes the data to the log partition (seq1), and then writes it to the two JnodeBlock caches. JnodeBlock1 is filled with 256KB, and JnodeBlock2 is written with the first 4KB. Then, the I / O thread replies to c0 that the write is complete.
[0137] (8) After the reply is completed, the I / O thread meets the flushing and compression conditions, triggering the flushing and compression process. After compressing the data of JnodeBlock1, the Cnode metadata of chunk0 is updated and persisted to the log partition and data partition together with the compressed data. After the persistence is completed, the seq1 and JnodeBlock1 caches are released;
[0138] (9) c0 generates a request to read chunk0, for example, 200-260KB. Based on the cluster metadata information recorded by m0, c0 sends the read request to the I / O threads of d0, d2, and d4.
[0139] (10) After receiving the read request from c0, the I / O thread determines that it needs to read the cache and disk. The 256KB-260KB data is in JnodeBlock2 and can be read directly without decompression. The 200-256KB data is on the disk and needs to be located in the data disk according to the Block and Compressed Block secondary index information recorded by Cnode. After the read is successful, the data is decompressed and then concatenated with the data in the cache and returned to c0.
[0140] An embodiment of the present invention further provides a computer device, Figure 5 is a schematic diagram of the structure of a computer device provided by an embodiment of the present invention; see the accompanying drawings Figure 5 As shown, the computer device includes: an input system 23, an output system 24, a memory 22 and a processor 21; the memory 22 is used to store one or more programs; when the one or more programs are executed by the one or more processors 21, the one or more processors 21 implement the distributed storage asynchronous data compression method provided in the above embodiment; wherein the input system 23, the output system 24, the memory 22 and the processor 21 can be connected by a bus or other means, Figure 5 The example of connecting through bus is taken in the following.
[0141] The memory 22 is a readable and writable storage medium of a computing device, which can be used to store software programs and computer executable programs, such as program instructions corresponding to the distributed storage asynchronous data compression method described in the embodiment of the present invention; the memory 22 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required for a function; the data storage area can store data created according to the use of the device, etc.; in addition, the memory 22 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device; in some instances, the memory 22 can further include a memory remotely arranged relative to the processor 21, and these remote memories can be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0142] The input system 23 may be used to receive input digital or character information, and to generate key signal input related to user settings and function control of the device; the output system 24 may include display devices such as display screens.
[0143] The processor 21 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 22, that is, implements the above-mentioned distributed storage asynchronous data compression method.
[0144] The computer device provided above can be used to execute the distributed storage asynchronous data compression method provided in the above embodiment, and has corresponding functions and beneficial effects.
[0145] The embodiment of the present invention also provides a storage medium containing computer executable instructions, which are used to execute the distributed storage asynchronous data compression method provided by the above embodiment when executed by a computer processor. The storage medium is any of various types of memory devices or storage devices, and the storage medium includes: installation media, such as CD-ROM, floppy disk or tape system; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc.; the storage medium may also include other types of memory or a combination thereof; in addition, the storage medium may be located in a first computer system in which the program is executed, or may be located in a different second computer system, which is connected to the first computer system via a network (such as the Internet); the second computer system may provide program instructions to the first computer for execution. The storage medium includes two or more storage media that can reside in different locations (for example, in different computer systems connected via a network). The storage medium can store program instructions (for example, specifically implemented as a computer program) that can be executed by one or more processors.
[0146] Of course, the storage medium containing computer executable instructions provided by an embodiment of the present invention is not limited to the distributed storage asynchronous data compression method described in the above embodiment, and can also execute related operations in the distributed storage asynchronous data compression method provided by any embodiment of the present invention.
[0147] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.
[0148] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A distributed storage asynchronous data compression method, characterized in that: include: Methods for deploying server-side clients, methods for writing data, and methods for reading data; The data writing method comprises the following steps: S11, the I / O thread of the persistence layer service is used to poll the I / O requests in the queue. After polling the request, if it is parsed as a write request from the client, the write process is entered; S12, persist the original data of each I / O thread in the log disk in the form of a sequence seq; after writing the data of a seq to the log disk, update the super information of the log disk, the super information records the write position information of the current log disk, the log disk adopts the append write method, and the space is recycled; write the cache at the same time, write the original data into the JnodeBlock of the cache Jnode; Jnode represents the metadata of the data block in the log disk; S13. After the persistence layer completes the WAL process of the data, the persistence layer server replies to the client: writing is successful; If the process crashes suddenly, the data can be restored by replaying the log disk; S14, checking whether the JnodeBlock of the cache Jnode is full or the number is more than 1 through the I / O thread, if the JnodeBlock is full or the number is more than 1, then starting the refresh process; S15, submitting a data compression task to the compress thread through the I / O thread; after completing the data compression, submitting a compression completion message to the I / O thread, the I / O thread parses the compression completion message, and continues the down-flush process of the I / O thread; S16. Update the index information of the data block after data compression, and save the updated index information in the metadata Cnode of the data block and the Record of the data disk; Cnode represents the metadata of the data block in the data disk, which records what space a data block occupies on the data disk; Record records the position of each compressed Cell in the Block; S17, writing data and metadata: submitting an I / O thread write request to the data disk and metadata disk in an asynchronous manner, and writing the compressed data block and the metadata Cnode of the data block to the disk; S18, persisting the metadata Cnode of the compressed data and data block to the data disk and metadata partition; S19. After persistence is completed, the log disk space and JnodeBlock cache occupied by the I / O thread are released.
2. The distributed storage asynchronous data compression method according to claim 1, characterized in that: The data reading method comprises the following steps: S21, polling the I / O requests in the queue through the I / O thread, and after polling the request, if it is parsed as a read request from the client, the read process is entered; S22. According to the order of logical addresses of data blocks from low to high, the complete data of each data block is composed of multiple blocks recorded by Cnode and the cache in the JnodeBlock dynamic array recorded by Jnode. The compressed data in the disk corresponds to the low address data of the data block, and the data in the JnodeBlock cache corresponds to the high address data of the data block. According to the interval read by the client, it is confirmed whether the data to be read is in the JnodeBlock cache, in the data disk, or in both the cache and the disk. S23, if the read request interval hits the cache, then the cache read is performed; if the read request interval hits the disk space, then the data disk read is performed; The data disk reading method comprises: locating which Compressed JnodeBlocks in which Blocks to be read according to the index information recorded in the metadata Cnode, firstly reading the Record of the Compressed JnodeBlock, locating the position of the target Compressed Cell in the Block according to the information recorded in the Record, and sending a data decompression task to the compress thread after the reading is completed; S24, after the decompression is completed by the compress thread, a decompression completion message is sent to the I / O message queue; after the I / O thread polls the decompression completion message, it parses it and takes out the decompressed data; S25. Return all data read from the cache or disk to the client.
3. The distributed storage asynchronous data compression method according to claim 1, characterized in that: The method for deploying a server client comprises the following steps: S01. Use the Run To Completion thread model to deploy multiple persistence layer server nodes. Each node Host runs a persistence layer service, and the fault domain is the Host level. S02. Each of the persistence layer server nodes manages two disks, each of which is divided into three partitions: a data partition, a metadata partition, and a log partition, and the three partitions share the same physical disk; S03, deploy a storage cluster metadata management node m0, the persistence layer service regularly reports the disk status to m0, and m0 records which disks in the cluster each data block is stored on; S04. Deploy a persistence layer client process c0 to send read and write requests to the storage cluster; S05, applying for a data block chunk0 from the storage cluster metadata management node m0 through the persistence layer client process c0, and c0 generates a request to write chunk0; S06. m0 records which disks in the cluster the data of data block chunk0 is stored on, and the persistence layer client process c0 sends a write request to the I / O threads of the persistence layer service corresponding to the disks.
4. The distributed storage asynchronous data compression method according to claim 2, characterized in that: In step S23 of the data reading method, if the data disk is not read, the client is directly replied after the cache read in step S23 is completed.
5. The distributed storage asynchronous data compression method according to claim 2, characterized in that: The cache reading method of step S23 of the data reading method includes: The data in the JnodeBlock cache is not compressed, and the target data can be directly retrieved from the cache without decompression.
6. The distributed storage asynchronous data compression method according to claim 3, characterized in that: The method of decompressing by the compress thread in step S24 of the data reading method includes: 256KB-260KB data is directly read in JnodeBlock without decompression; The 200-256KB data is on the disk. According to the Block and Compressed Block secondary index information recorded in the metadata Cnode, the target data is located on the data disk. After the data is read successfully, the data is decompressed and then concatenated with the data in the cache and returned to c0.
7. The distributed storage asynchronous data compression method according to claim 3, characterized in that: The method for setting data partitions in step S02 of the method for deploying the server client includes: Set the maximum capacity of each data block chunk of the data partition to 128MB, and the size of each block to 4M; Set the minimum metadata space occupied by each Cnode of the metadata partition to 512B; Set the size of each JnodeBlock cache interval of the log partition to 256KB; when the cached data fills a 256KB JnodeBlock, trigger the flushing and compression process; set the data compression unit to 16KB, and split a 256KB JnodeBlock into 16 16KB cells for compression.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the distributed storage asynchronous data compression method described in any one of claims 1 to 7 are implemented.
9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the distributed storage asynchronous data compression method as described in any one of claims 1 to 7 are implemented.
