Compression method and device for realizing fixed-length data block in openGauss database

Abstract

The invention relates to a compression method and device for realizing a fixed-length data block in an openGauss database. The method comprises the steps that 127 continuous data pages in a data file are divided into PCD, the last independent data page is set as PCA, and the PCD and the PCA jointly form a compressed Extent; the method comprises the following steps: splitting compressed data of a logic page through fixed chunk (1KB / 2KB / 4KB), compactly storing the compressed data in PCD, and calling FALLOCFLPUNCHHOLE to release the residual space into sparse storage; the PCA maintains physical distribution mapping and arrangement sequence index from the logic page to the compressed chunk, and address positioning data is analyzed through the PCA when the page is accessed. According to the method, variable-length compressed data is dynamically adapted while the I / O advantage of a fixed-length storage sequence is reserved, multi-algorithm transparent compression (such as LZ4 / ZSTD) is supported, high-concurrency read-write and efficient utilization of storage resources are achieved through a lock mechanism and sparse storage optimization, and the database compression rate and transaction performance are remarkably improved.

Description

Technical Field

[0001] The present application belongs to the field of database operation technology, and in particular relates to a method, device and electronic device for compressing fixed-length data blocks in an openGauss database based on a general compression coding algorithm and FALLOC_FL_PUNCH_HOLE technology. Background Art

[0002] Against the backdrop of the rapid development of big data, cloud computing, and the Internet of Things (IoT), data volumes are growing exponentially, leading to a continuous increase in storage resource consumption. The industry currently faces a dual challenge: on the one hand, users are increasingly demanding database system performance and resource utilization, hoping to fully and efficiently utilize limited hardware resources through algorithm and software optimization; on the other hand, existing technologies struggle to achieve a balanced optimization of storage efficiency and access performance within limited hardware resources. Specifically, data compression technology, as a key means of improving storage resource utilization, must address the storage management complexity caused by variable-length encoding while ensuring compression efficiency.

[0003] The storage engines of mainstream relational databases generally adopt a three-tiered storage architecture: "Segment-Extent-Page." This architecture, through the design of fixed-size data units, achieves three major technical advantages: 1. Simplified address mapping logic for data read and write operations; 2. Support for sequential I / O optimization of disks and file systems; and 3. Reduced system overhead through highly compact metadata management. However, existing technologies have significant limitations: traditional compression algorithms struggle to ensure the stability of compressed data length. Adopting a variable-length storage solution introduces two major technical drawbacks: first, it undermines the determinism of the storage structure, leading to complex dynamic space management; second, it degrades the originally optimized sequential disk I / O into a random access mode, resulting in significant performance loss.

[0004] Existing Linux file systems provide the FALLOC_FL_PUNCH_HOLE system call flag, which creates sparse storage space (a "hole punching" operation) within a specified area of ​​a file, returning the freed physical storage space to the operating system while maintaining the file's logical size. While this mechanism enables dynamic storage space reclamation, it has yet to be effectively combined with fixed-length storage structures in database applications, particularly in ensuring data access consistency and compression efficiency. Summary of the Invention

[0005] In order to address the above problems, the present invention proposes a new method for compressing fixed-length data blocks in the openGauss database.

[0006] The present invention aims to solve the technical problem of the conflict between the adaptability of fixed-length storage structures and variable-length compressed data in existing database compression technologies. By innovatively constructing a dynamic adaptation mechanism for compressed data, a data compression solution is provided in the openGauss database that can effectively handle scenarios where the length of compressed data is unstable without changing the underlying fixed-length storage structure.

[0007] The main purpose of the present invention is as follows: (1) Maintain the inherent characteristics of the fixed-length storage structure, ensure the fixed-length properties of the page-level storage unit, and retain the original sequential I / O optimization capabilities and metadata management efficiency.

[0008] (2) Dynamically adapt to variable-length compressed data scenarios, and achieve elastic storage of compressed data by combining sparse file processing technology (such as FALLOC_FL_PUNCH_HOLE). Under the premise of ensuring logical storage continuity, the efficient encapsulation and decompression reconstruction of compressed data are completed.

[0009] (3) Resolve the technical contradiction between compression efficiency and storage stability, break through the limitation of traditional fixed-length storage architecture that is incompatible with fluctuations in the length of compressed data, and establish a dual optimization model that takes into account both storage density and access performance.

[0010] In summary, the data compression method based on the fixed-length storage structure of the present invention is specifically implemented as follows: 1. Compress Extent structure The 127 consecutive data pages in the openGauss data file are divided into a compressed data area (PCD, Page Compression Data), and the last independent data page is used as the compressed address management area (PCA, Page Compression Address) for addressing, together forming a compressed extent.

[0011] 2. Compressed data organization The compressed data of each logical page is split and reorganized according to a fixed chunk resource block size (1KB / 2KB / 4KB) and stored in the PCD in a compact arrangement; The remaining unused space of PCD is released as a sparse storage block through the Linux system call FALLOC_FL_PUNCH_HOLE and returned to the operating system.

[0012] 3. Address mapping management PCA is responsible for maintaining two-level index relationships: Record which chunk resource blocks are used by the compression block corresponding to each logical page, and the physical distribution location of these chunks in the PCD; The order of storage chunk resource blocks in PCD. When accessing a page, address resolution must be completed through PCA to locate the specific storage location of the compressed data in PCD.

[0013] Specifically, this application provides the following technical solutions: A first aspect of the present application provides a method for compressing fixed-length data blocks in an openGauss database, the method comprising: S1. Divide 127 consecutive data pages in the data file into PCDs (compressed data areas) and set the last independent data page as PCA (compressed address area). The PCD and PCA together constitute the compressed extent. S2. Split and reorganize the compressed data of each logical page into fixed-size chunks and compactly store them in the PCD. S3. Maintain a two-level index in the PCA, recording the chunk resource blocks used by the compression blocks corresponding to each logical page and the physical distribution location of these chunk resource blocks in the PCD. The order of the chunk resource blocks in the PCD is also stored for address resolution during page access. S4. When a page is accessed, the PCA resolves the address and locates the storage location of the compressed data in the PCD.

[0014] Furthermore, step S2 of the method of the present application also includes: releasing the remaining space of the PCD into a sparse storage block by calling FALLOC_FL_PUNCH_HOLE and returning it to the operating system.

[0015] Furthermore, step S2 of the method of the present application also includes: after the compressed data is split, concurrent writing to PCD is controlled through a lock mechanism, and an exclusive lock is used to update the allocchunksum field in PCA to manage chunk resource allocation.

[0016] Furthermore, in the method of the present application, the size of the chunk resource block is 1KB, 2KB or 4KB.

[0017] Furthermore, in the method of the present application, the PCA records the mapping relationship between logical pages and compressed blocks through the chunk_id array.

[0018] Furthermore, the present application method also includes: (1) When reading a page, the page information to be read is determined by the page logical id, its corresponding addr position is found in the PCA, the offset is calculated by the blocknum of the page and the structure length of addr, and the chunk information stored on addr is obtained; (2) Read chunknum in addr, read chunknum chunk_ids from the chunk_id array in sequence, and lock the chunk position from PCD according to the reading order of chunk_id, read it into the temporary buffer, and arrange it according to the reading order of chunk_id; (3) Decompress the spliced ​​buffer blocks, restore the page information before compression and store it in the buffer.

[0019] Furthermore, the present application method also includes: (1) When writing a page, compress the page and store the compressed result in a buffer; (2) Calculate the required chunks for storage based on the compressed page size and the chunk size, and round up the result after division; (3) Read the corresponding PCA and offset it to the corresponding addr position, check the allocchunknum of the allocated chunk on addr, and if it is not less than the required number of chunks, skip the subsequent steps directly; (4) If the number of chunks in the current addr is not enough to store the compressed data, apply for more chunks from PCD to supplement it until the compressed data is sufficient to store; (5) Divide the compressed file into chunks of different sizes and store them in the chunk corresponding to chunk_id. After storage is completed, return success. (6) Release the original page in the buffer for use by other threads.

[0020] Furthermore, the method of the present application also includes: the user enables the compression algorithm selection function by customizing the table options, and calculates the compression ratio by observing the physical size of the actual table data and the size of the written data.

[0021] A second aspect of the present application provides a device for compressing fixed-length data blocks in an openGauss database, the device comprising: The compressed extent partitioning module is used to partition 127 consecutive data pages in the data file into PCDs and set the last independent data page as PCA. The PCD and PCA together constitute the compressed extent. The compressed data organization module is used to split and reorganize the compressed data of each logical page into fixed-size chunk resource blocks and store them compactly in the PCD; The address mapping management module is used to maintain a two-level index in the PCA, record the chunk resource blocks used by the compression blocks corresponding to each logical page and the physical distribution locations of these chunk resource blocks in the PCD, and store the arrangement order information of the chunk resource blocks in the PCD for address resolution during page access; The positioning query module is used to locate the storage location of the compressed data in the PCD by querying the PCA resolution address when the page is accessed.

[0022] The device implements the steps of the method for compressing fixed-length data blocks in the aforementioned openGauss database during operation.

[0023] A third aspect of the present application provides an electronic device, comprising: a memory and a processor; Memory: used to store computer programs; Processor: used to execute the computer program to implement the steps of the method for compressing fixed-length data blocks in the aforementioned openGauss database.

[0024] A fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program implements the steps of the method for compressing fixed-length data blocks in the aforementioned openGauss database.

[0025] In summary, this invention, by integrating fixed-length structures with dynamic compression, resolves the contradiction between storage structure stability and compression efficiency in traditional database compression technologies. This significantly improves storage resource utilization while ensuring transaction performance, making it suitable for high-concurrency, large-data-volume OLAP and OLTP scenarios. Specifically, this invention has the following technical advantages: (1) Balance between storage density and performance: The fixed-length structure retains the advantages of disk sequential reading and writing, compresses data to save space, and sparse storage releases fragmentation, thereby comprehensively improving storage utilization.

[0026] (2) Efficient read and write optimization: Compact arrangement reduces I / O randomness, PCA index accelerates address resolution, and decompressed data is written back according to the original page logic to maintain database transaction performance.

[0027] (3) Maximizing resource utilization: Dynamically releasing unused chunk space to avoid the waste of traditional fixed-length storage reservations. This is especially suitable for scenarios with high compression rates (such as text and log data).

[0028] (4) Compatibility and scalability: Compatible with the existing storage engine of openGauss, supports flexible selection of multiple algorithms, and users can customize compression strategies through table-level parameters to adapt to different business needs.

[0029] (5) High concurrency stability: The locking mechanism and chunk pre-allocation strategy (allocchunksum unidirectional growth) ensure the atomicity of compressed data writing, avoiding concurrency conflicts and data corruption.

[0030] Other features and advantages of the present invention will be described in detail in the following description, or may be understood through implementation of the relevant technical solutions of this application. The objectives and other advantages of this application may be achieved through the technical features and technical means clearly indicated in the description, claims, and drawings, and may be obtained through the implementation of these technical contents. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] To more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings involved in the description of the embodiments. It should be noted that the drawings only illustrate some embodiments of the present application. Those skilled in the art can deduce other relevant drawings based on these drawings without engaging in creative work.

[0032] Figure 1 This is an overall implementation flow chart of the method for compressing fixed-length data blocks in the openGauss database of the present invention.

[0033] Figure 2 The figure is a schematic diagram comparing the arrangement of compressed and uncompressed table data files in the openGauss database in the method of the present invention.

[0034] Figure 3 This is the expanded diagram of the PCD and PCA structures in the method of the present invention.

[0035] Figure 4 Schematic diagram of the compressed page reading and writing process in the method of the present invention.

[0036] Figure 5 Schematic diagram of the operation mode of user-defined table options in the method of the present invention.

[0037] Figure 6 It is a structural diagram of the compression device of the present invention.

[0038] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0040] In this document, the term "including" and any variations thereof (such as "including," "comprising," etc.) are open-ended expressions and should be understood as meaning "including but not limited to," meaning that the listed contents are not exhaustive and may include other contents not explicitly mentioned. The term "based on" should be understood as meaning "based at least in part on," meaning that the basis or condition referred to may not be the only factor and may also involve other relevant factors. The term "one embodiment" should be understood as meaning "at least one embodiment," meaning that the described embodiment is not the only possible implementation method and that other similar embodiments may exist.

[0041] In this application, the terms "a" and "a plurality" are used to modify related elements or features in an illustrative, non-restrictive manner. Unless the context clearly indicates otherwise, "a" should be understood as meaning "at least one," and "a plurality" should be understood as meaning "at least two." Those skilled in the art should interpret these terms appropriately based on the semantics and logical relationships of the context to ensure that they encompass the possibility of "one or more."

[0042] Figure 1 The figure shows the overall implementation process of the method for compressing fixed-length data blocks in the openGauss database provided by this application, including the following steps: S1. Divide 127 consecutive data pages in the data file into PCDs (compressed data areas) and set the last independent data page as PCA (compressed address area). The PCD and PCA together constitute the compressed extent. S2. Split and reorganize the compressed data of each logical page into fixed-size chunks and compactly store them in the PCD. S3. Maintain a two-level index in the PCA, recording the chunk resource blocks used by the compression blocks corresponding to each logical page and the physical distribution location of these chunk resource blocks in the PCD. The order of the chunk resource blocks in the PCD is also stored for address resolution during page access. S4. When a page is accessed, the PCA resolves the address and locates the storage location of the compressed data in the PCD.

[0043] In order to more clearly illustrate the technical solution of the present application, the following will further illustrate it through embodiments of specific scenarios.

[0044] The specific implementation steps of the method of the present invention are as follows: 1. PCA content format From a storage resource perspective, chunks simply divide pages (8KB) ​​into smaller resource granularities for storage. This is because compressed metadata is stored as data pages, which are 8KB in size. After compression, they must be stored as smaller than 8KB; otherwise, the remaining free space cannot be freed up for use on other pages. However, file management systems continue to process data at the page granularity, regardless of the content within the page.

[0045] like Figure 3 As shown, the compressed data page requires additional PCA pages for management. The PCD is the entity that stores the compressed page, divided into several chunks numbered sequentially starting from 1. The PCA manages pages and compressed chunks, storing the chunk_id of each page's compressed storage chunk. The PCA page, managed by chunk address, and the PCD page, composed of the aforementioned chunk resource blocks, convert the pre-compression page data into the corresponding compressed chunk.

[0046] 2. Compressed page reading and writing process, such as Figure 4 shown like Figure 2 As shown, when the page is not compressed, the page is located by the tablespace OID, database OID, table OID, and blocknum within the page. During compression, the page logical number in the above operation needs to be converted to find the extent position of the page and the offset within the extent. The conversion relationship of the logical page is as follows: If the logical page number logic_id = x, the extno of the page is x / 127, and the offset inside the extent is x % 127, where each ext carries 127 pages.

[0047] Compressed page reading process: (1) Determine the page information to be read through the page logical id, find its corresponding addr position in PCA, calculate the offset through the page blocknum and the structure length of addr, and obtain the chunk information stored on addr.

[0048] (2) Read chunknum in addr, and read chunknum chunk_ids from the chunk_id array in sequence. It is particularly emphasized here that the reading order of chunk_id cannot be disrupted, otherwise the compressed blocks cannot be decompressed into the original page.

[0049] (3) Use chunk_id to lock the chunk position in PCD in turn and read it into the temporary buffer, arranging it in the order of chunk_id reading. When reading with multiple threads, be sure to record the subscript of chunk_id when reading. After each thread reads it, it determines the order of chunks in the buffer based on the subscript.

[0050] (4) Decompress the spliced ​​buffer blocks, restore the page information before compression and store it in the buffer.

[0051] Compared with reading a page, when a page is modified and needs to be written to disk, the following aspects need to be noted. The specific writing steps are as follows: (1) Compress the page and store the compressed result in a buffer. The compressed result is stored in a buffer instead of being stored locally. This redundant operation is mainly to ensure that the page in the buffer still exists when an abnormality occurs during storage of the compressed block, resulting in damage to the compressed block.

[0052] (2) Calculate the required chunks based on the compressed size and the chunk size, and round up the result after division.

[0053] (3) Read the corresponding PCA page and offset it to the corresponding addr position, check the allocchunknum of the allocated chunk on addr, and if it is not less than the required number of chunks, skip the following steps directly.

[0054] (4) If the number of chunks in the current addr is insufficient to store the compressed data, additional chunks must be requested from the PCD until the compressed data is sufficient. PCD chunk management is relatively simple, managed by a single allocchunksum (in most cases, allocchunksum only increases in one direction, used to request new chunks for page use). After an exclusive lock is placed on allocchunksum, the increment operation is performed, the newly added chunk_id is recorded in addr, and allochunknum is modified.

[0055] (5) Divide the compressed file into chunks of different sizes and store them in the chunk corresponding to chunk_id. After storage is completed, a success message is returned.

[0056] (6) The original page in the buffer is released for use by other threads, and the entire page is written to the disk.

[0057] In transparent page compression, page punching occurs when the data file is extended. At this time, an extent of data is requested, the PCA page is initialized, and finally the entire PCD area is punched.

[0058] 3. General compression scheme for PCD pages openGauss supports mainstream compression algorithms in the industry, including lz4 (pglz), zstd, and zlib. Whether it is ASTORE or USTORE, the page header is not compressed, only the data portion is compressed. After compression, the checksum needs to be recalculated to avoid rapid verification of the page's legitimacy during backup and recovery.

[0059] It should be pointed out that compression and decompression are black box behaviors for users, and users do not actually perceive them. However, information such as the compression rate can be calculated through some actual observations.

[0060] For users who are sensitive to storage costs, this function can be enabled by customizing table options. The operation method is as follows: Figure 5 As shown in the figure, the compresstype in the table options is set to 2, indicating that the ZLIB algorithm is used to compress the data in this table. Then, a data entry is inserted into the table and a checkpoint is executed. This data page is written to the PCD through the above process and mapped in the PCA. The compression ratio can be calculated by observing the physical size of the actual table data and the amount of written data.

[0061] Figure 6 FIG. 1 is a device for compressing fixed-length data blocks in an openGauss database proposed in this application, the device comprising: The compressed extent partitioning module is used to partition 127 consecutive data pages in the data file into PCDs and set the last independent data page as PCA. The PCD and PCA together constitute the compressed extent. The compressed data organization module is used to split and reorganize the compressed data of each logical page into fixed-size chunk resource blocks and store them compactly in the PCD; The address mapping management module is used to maintain a two-level index in the PCA, record the chunk resource blocks used by the compression blocks corresponding to each logical page and the physical distribution locations of these chunk resource blocks in the PCD, and store the arrangement order information of the chunk resource blocks in the PCD for address resolution during page access; The positioning query module is used to locate the storage location of the compressed data in the PCD by querying the PCA resolution address when the page is accessed.

[0062] When the above device is running, the steps of the method for compressing fixed-length data blocks in the openGauss database disclosed in this application are implemented.

[0063] The flowcharts and block diagrams in the accompanying drawings illustrate possible implementations of the apparatus, methods, and computer program products according to various embodiments of the present application, including architecture, functions, and operations. In these figures, each box may represent a module, a program segment, or a portion of a code, which contains one or more executable instructions for implementing a specified logical function. It should be noted that each box in the block diagram and / or flowchart, and the combination of these boxes, can be implemented using a dedicated hardware-based system to implement the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0064] like Figure 7 As shown, an embodiment of the present application further discloses an electronic device, comprising: a processor 310, a communication interface 320, a memory 330 for storing a computer program executable by the processor, and a communication bus 340. The processor 310, the communication interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 executes the executable computer program to implement the steps of the above-mentioned method for compressing fixed-length data blocks in the openGauss database.

[0065] It is understood that, in addition to the memory and processor, the electronic device may also include an input device (e.g., a keyboard), an output device (e.g., a display), and other communication modules. These input devices, output devices, and other communication modules all communicate with the processor via an I / O interface (i.e., an input / output interface).

[0066] The operation of the present application can be implemented by writing computer program code using one or more programming languages ​​or a combination thereof. The programming languages ​​include but are not limited to the following types: Object-oriented programming languages, such as Java, Smalltalk, C++, etc.; A conventional procedural programming language, such as "C" or a similar programming language.

[0067] The execution methods of the program code include but are not limited to: Executes entirely on the user's computer; Partially executed on the user's computer and partially on a remote computer; Executed as a standalone software package; Executes entirely on the remote computer or server.

[0068] In scenarios involving a remote computer, the remote computer can be connected to the user's computer via any type of network, including but not limited to a local area network (LAN) or a wide area network (WAN). Additionally, the remote computer can be connected to an external computer via an Internet service provider, such as the Internet.

[0069] Furthermore, the present application also discloses a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute the various steps of the method for compressing fixed-length data blocks in the openGauss database disclosed in the present application.

[0070] In the context of this application, computer-readable storage media refers to tangible media that can store computer program code and related data. Specific examples include, but are not limited to, the following: (1) Portable computer disk: A removable magnetic storage medium such as a floppy disk.

[0071] (2) Hard disk: includes fixed storage devices such as mechanical hard disks and solid-state hard disks.

[0072] (3) Random Access Memory (RAM): Volatile storage medium used for temporary storage of data and program code.

[0073] (4) Read-only memory (ROM): A non-volatile storage medium used to store fixed programs and data.

[0074] (5) Erasable Programmable Read-Only Memory (EPROM) or Flash Memory: A non-volatile storage medium that supports multiple erasing and programming.

[0075] (6) Fiber optic storage device: storage medium based on fiber optic technology.

[0076] (7) Compact Disc Read-Only Memory (CD-ROM): A read-only medium that stores data in the form of an optical disc.

[0077] (8) Optical storage devices: storage media based on optical principles, such as DVDs and Blu-ray discs.

[0078] (9) Magnetic storage devices: storage media based on magnetic principles, such as magnetic tapes and disks.

[0079] (10) Any suitable combination of the above: for example, combining multiple storage media to meet different storage requirements.

[0080] These computer-readable storage media can be used to store the program code and related data described in this application to support the operation of the program and the persistent storage of data.

[0081] In particular, according to embodiments of the present application, the processes described in the flowcharts can be implemented as computer software programs. For example, embodiments of the present application relate to a computer program product comprising a computer program carried on a non-transitory computer-readable medium. The computer program includes program code for executing the method for compressing fixed-length data blocks in the openGauss database disclosed in this application. When the computer program is executed by a processing device, the above-described functions defined in the embodiments of the present application can be implemented.

[0082] Although the above discussion contains several specific implementation details, these details should not be interpreted as limiting the scope of this application. The above description is only a preferred embodiment of the present application and an illustration of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in this application is not limited to the technical solutions formed by the specific combination of the above technical features. At the same time, this application should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concepts.

[0083] Those skilled in the art should also understand that they may modify the technical solutions described in the aforementioned embodiments, or replace some of the technical features therein with equivalents, without departing from the spirit and scope of the technical solutions of the embodiments of the present application. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the core spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for compressing fixed-length data blocks in an openGauss database, characterized in that: The method comprises: S1. Divide 127 consecutive data pages in the data file into PCDs and set the last independent data page as PCA. The PCD and PCA together constitute the compressed extent. S2. Split and reorganize the compressed data of each logical page into fixed-size chunks and compactly store them in the PCD. S3. Maintain a two-level index in the PCA, recording the chunk resource blocks used by the compression blocks corresponding to each logical page and the physical distribution location of these chunk resource blocks in the PCD. The order of the chunk resource blocks in the PCD is also stored for address resolution during page access. S4. When a page is accessed, the PCA resolves the address and locates the storage location of the compressed data in the PCD.

2. The method according to claim 1, characterized in that Step S2 also includes: releasing the remaining space of the PCD into a sparse storage block by calling FALLOC_FL_PUNCH_HOLE, and returning it to the operating system.

3. The method according to claim 1, characterized in that Step S2 also includes: after the compressed data is split, controlling concurrent writing to the PCD through a lock mechanism, and using an exclusive lock to update the allocchunksum field in the PCA to manage chunk resource allocation.

4. The method according to claim 1, wherein The size of the chunk resource block is 1KB, 2KB or 4KB.

5. The method according to claim 1, wherein The PCA records the mapping relationship between logical pages and compressed blocks through the chunk_id array.

6. The method according to claim 1, characterized in that The method further comprises: (1) When reading a page, the page information to be read is determined by the page logical id, its corresponding addr position is found in the PCA, the offset is calculated by the blocknum of the page and the structure length of addr, and the chunk information stored on addr is obtained; (2) Read chunknum in addr, read chunknum chunk_ids from the chunk_id array in sequence, and lock the chunk position from PCD according to the reading order of chunk_id, read it into the temporary buffer, and arrange it according to the reading order of chunk_id; (3) Decompress the spliced ​​buffer blocks, restore the page information before compression and store it in the buffer.

7. The method according to claim 1, characterized in that The method further comprises: (1) When writing a page, compress the page and store the compressed result in a buffer; (2) Calculate the required chunks for storage based on the compressed page size and the chunk size, and round up the result after division; (3) Read the corresponding PCA and offset it to the corresponding addr position, check the allocchunknum of the allocated chunk on addr, and if it is not less than the required number of chunks, skip the subsequent steps directly; (4) If the number of chunks in the current addr is not enough to store the compressed data, apply for more chunks from PCD to supplement it until the compressed data is sufficient to store; (5) Divide the compressed file into chunks of different sizes and store them in the chunk corresponding to chunk_id. After storage is completed, return success. (6) Release the original page in the buffer for use by other threads.

8. The method according to claim 1, characterized in that The method further includes: the user enabling a compression algorithm selection function by customizing table options, and calculating a compression ratio by observing the physical size of actual table data and the size of written data.

9. A device for compressing fixed-length data blocks in an openGauss database, characterized in that: The device comprises: The compressed extent partitioning module is used to partition 127 consecutive data pages in the data file into PCDs and set the last independent data page as PCA. The PCD and PCA together constitute the compressed extent. The compressed data organization module is used to split and reorganize the compressed data of each logical page into fixed-size chunk resource blocks and store them compactly in the PCD; The address mapping management module is used to maintain a two-level index in the PCA, record the chunk resource blocks used by the compression blocks corresponding to each logical page and the physical distribution locations of these chunk resource blocks in the PCD, and store the arrangement order information of the chunk resource blocks in the PCD for address resolution during page access; The positioning query module is used to locate the storage location of the compressed data in the PCD by querying the PCA resolution address when the page is accessed.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for compressing fixed-length data blocks in the openGauss database according to any one of claims 1 to 8 are implemented.

Images