A method for compressed storage of a database and related products

Abstract

The application provides a compressed storage method of a database and related products. The method comprises the following steps: receiving a write request for a target logical page in a data table, determining an extended area to which the target logical page belongs and an offset position of the target logical page in the extended area according to a logical page number of the target logical page; determining data domain usage information of the extended area according to a control domain, and performing a compression operation on original data carried by the target logical page based on the data domain usage information; writing target data obtained based on the compression operation to a physical disk space of the data domain; and storing a mapping entry of one or more compression blocks corresponding to the target logical page in the control domain. The scheme of the application enables the target data obtained by the compression operation to be written to the data domain in units of compression blocks, makes it possible to store and manage variable-length blocks, can adapt to compression ratio differences of different data, and avoids internal fragmentation or space reservation waste caused by size fluctuations after compression.

Description

Technical Field

[0001] This invention relates to database technology, and in particular to a method for compressed storage of databases and related products. Background Technology

[0002] In database systems, data is stored and managed in fixed-size logical pages. This fixed-size page mechanism simplifies buffer pool management and concurrency control logic. Each logical page is associated with a fixed-size physical disk page, so even if the actual amount of data stored within a logical page is small, the database system must still allocate a complete physical page space, resulting in unused free space within the page. As data volume increases, the free space within a large number of logical pages accumulates page by page, causing a significant increase in overall disk usage and consequently increasing storage costs. Summary of the Invention

[0003] One objective of this invention is to improve the utilization rate of disk space in database systems and reduce storage waste caused by fixed-length page mechanisms.

[0004] A further objective of this invention is to reduce the write amplification effect in compressed storage scenarios and improve the update performance of compressed tables.

[0005] Specifically, the present invention provides a method for compressed storage of a database, comprising: The system receives a write request for a target logical page in a data table, and determines the extended region to which the target logical page belongs and the offset position of the target logical page in the extended region based on the logical page number of the target logical page. The extended region is a contiguous logical page range on a logical address, used to manage space allocation and reclamation. Each extended region contains a data field and a control field. The data domain usage information of the extended area is determined according to the control domain, and the original data carried by the target logical page is compressed based on the data domain usage information. The target data obtained based on the compression operation is written into the physical disk space of the data domain; The control domain stores mapping entries for one or more compressed blocks corresponding to the target logical page. Each mapping entry includes the logical page number, the offset of the compressed block within the extension area, and the length of the compressed data.

[0006] Optionally, before receiving a write request for a target logical page in the data table, the method further includes: The logical page address space of the data table is sequentially divided into multiple logically contiguous extension regions, each of which contains the same number of logical pages; The extended area is logically divided into a data domain and a control domain, wherein the data domain is used to store the target data; and the control domain is used to store control pages that record physical location mappings.

[0007] Optionally, determining the extended region to which the target logical page belongs and the offset position of the target logical page in the extended region based on the logical page number of the target logical page includes: Perform an integer division operation between the logical page number of the target logical page and the number of logical pages contained in a single extension region, and determine the extension region to which the target logical page belongs based on the quotient obtained from the integer division operation; A modulo operation is performed between the logical page number of the target logical page and the number of logical pages contained in a single extension region. The offset position of the target logical page within its extension region is determined based on the remainder obtained from the modulo operation.

[0008] Optionally, determining the data domain usage information of the extended area based on the control domain includes: The control page stored in the control domain is read, the space usage information of each logical page recorded in the control page is parsed, and the data domain usage information of the extended area is determined based on the space usage information; wherein, the space usage information is used to indicate the occupancy status of the compressed data in the data domain.

[0009] Optionally, performing compression on the original data carried by the target logical page based on the data field usage information includes: The upper limit of the length of the compressed data is determined based on the data field usage information. Based on the aforementioned upper limit value, a target compression algorithm is selected from a variety of candidate compression algorithms; The original data is compressed using the target compression algorithm.

[0010] Optionally, writing the target data obtained based on the compression operation into the physical disk space of the data domain includes: The compressed intermediate data is obtained according to the compression operation; Determine whether the byte length of the intermediate data is less than a preset threshold, the preset threshold being determined based on the compression ratio of the selected target compression algorithm; If so, the intermediate data is used as the target data and written to the physical disk space of the data field in compressed blocks, wherein the byte length of the logical page is a positive integer multiple of the byte length of the compressed block; If not, the original data is used as the target data and written to the physical disk space of the data domain in uncompressed form.

[0011] Optionally, after obtaining the compressed intermediate data according to the compression operation, the method further includes: If the length of the intermediate data is greater than or equal to the length of the original data, compression is skipped, and the original data is written to the physical disk space of the data field in uncompressed form.

[0012] Optionally, after storing the mapping entries of one or more compressed blocks corresponding to the target logical page in the control domain, the method further includes: If the target data written to the physical disk space of the data domain is deleted, the corresponding mapping entry in the control domain is marked as invalid, and the physical disk space is released through a hole-punching operation.

[0013] Optionally, after storing the mapping entries of one or more compressed blocks corresponding to the target logical page in the control domain, the method further includes: If the target data written to the physical disk space of the data domain is modified, update the offset and data length of the corresponding mapping entry in the control domain, and release the physical disk space occupied by the target data through a hole-punching operation.

[0014] According to another aspect of the present invention, a computer-readable storage medium is also provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of the compressed storage method for the database described above.

[0015] According to another aspect of the present invention, a computer program product is also provided, comprising a computer program that, when executed by a processor, implements the steps of any of the above-described database compression storage methods.

[0016] According to another aspect of the present invention, a computer device is also provided, which includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any database compression storage method.

[0017] The database compression storage method of this invention decouples logical pages from physical disk space, eliminating the need to allocate a complete physical page for each logical page. When the actual data volume of the target logical page is small, the compression operation compresses it into smaller target data, occupying only the physical space actually required in the data domain, thus avoiding waste of free space within the page due to incomplete logical pages. Mapping entries are stored in the control domain within the extended area. Each mapping entry records the logical page number, the offset of the compressed block within the extended area, and the length of the compressed data. When a specific logical page needs to be read, it is only necessary to locate the extended area to which it belongs based on its logical page number and search for the corresponding mapping entry in the control domain to obtain the physical location and length information of the compressed block, without scanning the entire extended area. The target data obtained from the compression operation is written to the data domain in units of compressed blocks. The compressed data of one logical page can correspond to one or more compressed blocks. The mapping entry records the length of the compressed data, making the storage and management of variable-length blocks possible. This adapts to differences in compression ratios of different data and avoids internal fragmentation or wasted space due to fluctuations in the size after compression.

[0018] Furthermore, the database compression storage method of the present invention logically divides the extended area into independent data domains and control domains. The data domain is dedicated to storing compressed data, and the control domain is dedicated to storing mapping entries. Based on physical isolation, when data is updated, the database system only needs to reallocate space within the data domain and synchronously update the mapping entries in the control domain, without having to move other data.

[0019] Furthermore, the database compression storage method of the present invention directly calculates the extended region and its offset position within the region to which the target logical page belongs through integer division and modulo operations. The positioning process involves only integer division and modulo operations, does not rely on any in-memory mapping table, hash structure, or tree index, and does not require access to metadata on the disk, reducing the positioning time complexity from O(log n) or O(1) but requiring memory access to pure O(1) arithmetic operations, significantly reducing CPU instruction overhead. The positioning process does not rely on any persistent or in-memory mapping state, and multiple threads concurrently positioning different logical pages do not need to acquire any locks, eliminating lock contention in the positioning process.

[0020] Furthermore, the database compression storage method of the present invention, after completing the compression operation, compares the byte length of the intermediate compressed data with a preset threshold determined based on the compression ratio of the compression algorithm. When the length of the compressed data is not less than the preset threshold, the compression result is discarded and the original data is stored in uncompressed form, avoiding the negative benefit of occupying more space and ensuring that the storage efficiency is no worse than the uncompressed storage scheme under any data characteristics. The byte length of the logical page is specified to be a positive integer multiple of the byte length of the compressed block, making the block boundaries predictable. The starting position of the compressed block in the data domain can be calculated without reading the control domain, supporting block-level parallel reading and partial decompression optimization. Fixed-size compressed blocks are used as the smallest allocation granularity, avoiding the fragmentation management complexity caused by variable-length blocks.

[0021] The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments of the invention in conjunction with the accompanying drawings. Attached Figure Description

[0022] The following sections will describe some specific embodiments of the invention in detail by way of example and not limitation, with reference to the accompanying drawings. The same reference numerals in the drawings denote the same or similar parts or portions. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings: Figure 1 This is a schematic flowchart of a database compression storage method according to an embodiment of the present invention; Figure 2 This is a schematic flowchart illustrating the determination of an extended area in a database compression storage method according to an embodiment of the present invention; Figure 3 This is a schematic flowchart of the compression operation in a database compression storage method according to an embodiment of the present invention; Figure 4 This is a schematic flowchart illustrating the writing of target data to physical disk space in a database compression storage method according to an embodiment of the present invention; Figure 5 This is a schematic diagram of a computer program product according to an embodiment of the present invention; Figure 6 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention; Figure 7 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0023] In database systems, data is stored and managed in fixed-size logical pages. Database engines perform data read / write, buffer pool management, and concurrency control operations at the logical page level. This fixed-size page design ensures uniform slot sizes in the buffer pool, a regular page table structure, and simplified lock management, thereby reducing the implementation complexity of the storage engine.

[0024] In some scenarios, a fixed mapping relationship is established between each logical page and a physical disk page of a fixed size. When creating a table or inserting data, the database system pre-allocates or allocates a physical disk page of equal size for each logical page, and this mapping relationship remains stable throughout the data's lifecycle. This design simplifies address translation logic; the database system only needs to calculate the address offset to complete the translation from logical page number to physical disk location.

[0025] However, while this fixed-size page design simplifies management, it also introduces space waste. Because logical pages and physical pages are of fixed sizes and bound one-to-one, even if the actual amount of data stored within a logical page is much smaller than the logical page capacity, the database system must still allocate the entire physical page space for it, and cannot use the remaining free space to store other data. This results in a general abundance of unusable free space within each logical page.

[0026] As data volumes continue to grow, the free space within a large number of logical pages accumulates page by page, leading to a continuously amplified waste of overall disk space. For example, under OLTP (Online Transaction Processing) workloads, the disk space utilization rate of fixed-length page mechanisms is typically only 50% to 70%. This space waste directly manifests as a significant increase in the overall disk usage of the database, which in turn increases the enterprise's storage hardware procurement costs, power consumption, and operational expenses.

[0027] This embodiment provides a database compression storage solution. It achieves on-demand compressed data storage without changing the upper-layer transaction engine's practice of accessing data in logical page units.

[0028] Figure 1 This is a schematic flowchart of a database compression storage method according to an embodiment of the present invention. Figure 1 As shown, the compressed storage method for this database can generally include: Step S101: Receive a write request for a target logical page in the data table. Based on the logical page number of the target logical page, determine the extended region to which the target logical page belongs and the offset position of the target logical page in the extended region. The extended region is a contiguous logical page range on the logical address, used to manage space allocation and reclamation. Each extended region contains a data field and a control field. Step S102: Determine the data domain usage information of the extended area based on the control domain, and perform compression operation on the original data carried by the target logical page based on the data domain usage information; Step S103: Write the target data obtained based on the compression operation into the physical disk space of the data domain; Step S104: Store mapping entries for one or more compressed blocks corresponding to the target logical page in the control domain. Each mapping entry includes the logical page number, the offset of the compressed block in the extension area, and the length of the compressed data.

[0029] In this embodiment, the database system's storage engine receives a write request for a target logical page of a data table and extracts the logical page number carried in the write request as a retrieval identifier. Based on preset space management rules, it determines the extended region to which the target logical page belongs and the page offset within that extended region, wherein: An extension region is a set of logically contiguous logical pages, serving as the basic unit for space allocation and reclamation. Each extension region contains a control field and a data field: the control field stores the mapping entries for each logical page within the extension region (including logical page number, offset, and compressed data length) and a space usage bitmap for the data field; the data field stores compressed data blocks.

[0030] The preset space management rules are used to define the mapping method from logical page numbers to the extension area, as well as the space allocation and reclamation mechanism for data fields within the extension area.

[0031] In this embodiment, the space usage bitmap in the target extended area control domain is read, and the distribution of free areas and the remaining free space in the data domain are parsed. Based on this information, it is determined whether the data domain has the conditions to accommodate the data to be written. If the conditions are met, a compression operation is performed on the original data carried by the target logical page, the target data is generated based on the result of the compression operation, and the target data is written to the physical disk space of the data domain.

[0032] In this embodiment, the target data is divided into compressed blocks to obtain one or more compressed blocks. Based on the space usage bitmap in the control domain, the distribution and physical order of free areas in the data domain are determined. Following the physical order of the free areas, each compressed block is written to the physical disk space of the data domain. A single compressed block is allowed to cross physical sector boundaries within the data domain, but the entire write operation must not cross the boundary of the current extended area.

[0033] In this embodiment, a corresponding mapping entry is stored in the control domain of the target extended area for each written compressed block. Each mapping entry includes at least the following fields: the logical page number of the target logical page, the offset of the compressed block within the extended area, and the length of the compressed data of the compressed block. Mapping entries for multiple compressed blocks corresponding to the same target logical page are stored consecutively according to the writing order of the blocks in the data domain.

[0034] Based on the above steps, by decoupling logical pages from physical disk space, a complete physical page is no longer allocated for each logical page. When the actual data volume of the target logical page is small, the compression operation compresses it into smaller target data, occupying only the physical space actually required in the data domain, avoiding waste of free space within the page due to incomplete logical pages. Mapping entries are stored in the control domain within the extended area. Each mapping entry records the logical page number, the offset of the compressed block within the extended area, and the length of the compressed data. When a specific logical page needs to be read, it is only necessary to locate the extended area to which it belongs based on its logical page number, and then search for the corresponding mapping entry in the control domain to obtain the physical location and length information of the compressed block, without scanning the entire extended area. The target data obtained from the compression operation is written to the data domain in units of compressed blocks. The compressed data of one logical page can correspond to one or more compressed blocks. The mapping entry records the length of the compressed data, making the storage and management of variable-length blocks possible. This adapts to differences in compression ratios of different data and avoids internal fragmentation or wasted space reservation due to fluctuations in the size after compression.

[0035] In the embodiment of step S101, before receiving a write request for a target logical page in the data table, the method further includes: The logical page address space of the data table is sequentially divided into multiple logically contiguous extension regions, each of which contains the same number of logical pages; The extended area is logically divided into a data domain and a control domain. The data domain is used to store target data, while the control domain is used to store control pages that record physical location mappings.

[0036] In this embodiment, when a data table is created or accessed for the first time, the database system determines the logical page address range based on the data table's capacity parameters (such as tablespace size). According to a preset single extension area logical page capacity, the logical page address space is sequentially divided into multiple logically contiguous extension areas, each containing a fixed number of logical pages. Except for the last extension area, which may contain fewer logical pages than the preset value, all other extension areas contain the same number of logical pages. The extension area serves as the basic management unit for subsequent space allocation and reclamation.

[0037] In this embodiment, two logical regions are defined for each extended region: a data region and a control region. The data region is located at the beginning of the extended region, occupying a contiguous and fixed-size address space within the extended region, and is used to store the compressed target data. The control region follows the data region, located at the end of the extended region, occupying the remaining contiguous address space, and is used to store control pages. The control page contains at least a space usage bitmap and mapping entries to record the physical location mapping relationship from the logical page to the compressed block.

[0038] Based on the above implementation, the extended area is logically divided into independent data and control domains. The data domain is dedicated to storing compressed data, while the control domain is dedicated to storing mapping entries. This physical isolation ensures that when data is updated, the database system only needs to reallocate space within the data domain and synchronously update the mapping entries in the control domain, without needing to move other data.

[0039] Figure 2 This is a schematic flowchart illustrating the determination of the extended area in a database compression storage method according to an embodiment of the present invention. Figure 2 As shown, determining the extended region to which the target logical page belongs and its offset position within the extended region, based on the target logical page's logical page number, includes: Step S201: Perform an integer division operation on the logical page number of the target logical page and the number of logical pages contained in a single extension region, and determine the extension region to which the target logical page belongs based on the quotient obtained from the integer division operation; Step S202: Perform a modulo operation between the logical page number of the target logical page and the number of logical pages contained in a single extension region, and determine the offset position of the target logical page within its extension region based on the remainder obtained from the modulo operation.

[0040] Based on the above steps, the extended region and its offset within the region to which the target logical page belongs are directly calculated through integer division and modulo operations. The location process involves only integer division and modulo operations, without relying on any in-memory mapping tables, hash structures, or tree indexes, and without needing to access metadata on disk. This reduces the location time complexity from O(log n) or O(1) with memory access requirements to pure O(1) arithmetic operations, significantly reducing CPU instruction overhead. The location process does not rely on any persistent or in-memory mapping states, and multiple threads concurrently locating different logical pages do not need to acquire any locks, eliminating lock contention in the location process.

[0041] In the implementation of step S102, determining the data domain usage information of the extended area based on the control domain includes: Read the control pages stored in the control domain, parse the space usage information of each logical page recorded in the control page, and determine the data domain usage information of the extension area based on the space usage information; wherein, the space usage information is used to indicate the occupancy status of compressed data in the data domain.

[0042] In this embodiment, control pages stored in the target extended area control domain are read. Each control page contains mapping entries for all logical pages within the extended area. Each mapping entry records the offset and compressed length of the corresponding logical page's compressed block within the data domain. Each mapping entry is traversed, and based on the offset and length information in each entry, the space occupied by compressed data in the data domain is calculated. Based on the occupied space and the total capacity of the data domain, the distribution of free space in the data domain at the compressed block level and the remaining free space are calculated and output as data domain usage information.

[0043] Based on the above implementation, by reading the control pages stored in the extended area control domain and parsing the space usage information of each logical page recorded therein, the distribution and occupancy status of the free area of ​​the data domain can be obtained. This process only involves sequential reading and structured parsing of the control domain, without traversing the entire data domain or scanning all physical disk space. Compared to scanning data pages to obtain space usage information, this significantly reduces the IO overhead and CPU consumption of space information queries. Since the space usage information is stored in the control pages, when new compressed blocks are written to the data domain or existing space is released, the database system only needs to update the corresponding mapping entries and space usage bitmap in the control pages to complete the incremental maintenance of space usage information. During system crash recovery, only the control pages of each extended area need to be reread to reconstruct the complete data domain usage information, without scanning the data domain, greatly shortening the recovery time.

[0044] Figure 3 This is a schematic flowchart of the compression operation in a database compression storage method according to an embodiment of the present invention. Figure 3 As shown, performing compression operations on the raw data carried by the target logical page based on data domain usage information includes: Step S301: Determine the upper limit of the length of the compressed data based on the data field usage information; Step S302: Based on the upper limit of length, select the target compression algorithm from multiple candidate compression algorithms; Step S303: Perform compression operation on the original data using the target compression algorithm.

[0045] In this embodiment, the free space size and compression block granularity constraints of the current data domain are determined based on the data domain usage information. The upper limit of the compressed data length is calculated, and this upper limit dynamically changes with the free space of the data domain and is rounded down to the nearest integer based on the compression block size. Based on this upper limit, the database system selects from a variety of pre-configured candidate compression algorithms (including high compression ratio algorithms, balanced algorithms, and high-speed lightweight algorithms): when the upper limit is small and space is limited, a high compression ratio algorithm is prioritized to ensure that the data can be stored in the limited space; when the upper limit is sufficient and space is abundant, a high-speed lightweight algorithm is prioritized to reduce CPU overhead and improve write throughput. After selecting the target compression algorithm, the database system performs compression operations on the original data and verifies the compression result to confirm that the length of the compressed data does not exceed the upper limit. If it does, the subsequent write process begins; otherwise, a degradation strategy is triggered (e.g., using a higher compression ratio algorithm, using uncompressed storage, or allocating a new extension area), ultimately generating target data that meets the space constraints.

[0046] Based on the above steps, the upper limit of the compressed data length is determined according to the real-time usage information of the data domain. This allows the compression operation to be aware of the current free space status of the expansion area, ensuring that the compressed target data can be completely written into the data domain. This avoids problems such as insufficient space, write overflow, or cross-boundary anomalies caused by excessively large compressed data, thus improving the success rate of write operations. Based on the upper limit of length, a target compression algorithm is selected from multiple candidate compression algorithms to achieve dynamic matching between compression strategy and space constraints. When free space is scarce, a high compression ratio algorithm is selected to meet space constraints; when free space is sufficient, a high-speed, lightweight algorithm is selected to reduce CPU overhead and increase write throughput, achieving a balance between space efficiency and time efficiency.

[0047] Figure 4 This is a schematic flowchart illustrating the writing of target data to physical disk space in a database compression storage method according to an embodiment of the present invention. Figure 4 As shown, writing the target data obtained based on the compression operation into the physical disk space of the data domain includes: Step S401: Obtain the compressed intermediate data according to the compression operation; Step S402: Determine whether the byte length of the intermediate data is less than a preset threshold. The preset threshold is determined based on the compression ratio of the selected target compression algorithm. Step S403: If yes, use the intermediate data as the target data and write it to the physical disk space of the data field in units of compressed blocks, where the byte length of the logical page is a positive integer multiple of the byte length of the compressed block. Step S404: If not, write the original data as the target data to the physical disk space of the data domain in uncompressed form.

[0048] In this embodiment, the database system calls the target compression algorithm (e.g., LZ4, ZSTD, or Snappy) determined in step S302 to perform lossless compression on the original data carried by the target logical page, generate compressed intermediate data, and record its byte length.

[0049] In this embodiment, a preset threshold corresponding to the selected target compression algorithm is obtained. This preset threshold is pre-calculated based on the standard compression ratio of the target compression algorithm (for example, if the typical compression ratio of the algorithm is 0.4 and the logical page size is 8KB, then the threshold can be set to 3.2KB). The database system compares the actual byte length of the intermediate data with the preset threshold: if it is less than the preset threshold, the compression is deemed effective; if it is greater than or equal to the preset threshold, the compression is deemed ineffective or has a negative impact.

[0050] In this embodiment, if compression is deemed effective, the database system uses the intermediate data as the final target data. The target data is then divided according to a fixed compressed block size. Since the logical page length is a positive integer multiple of the compressed block length, the data in a single logical page can be evenly divided into an integer number of blocks. The database system uses the compressed block as the smallest I / O unit and writes it sequentially to the physical disk space of the data domain. During writing, it adheres to the extended region boundary constraints to ensure that each block is stored contiguously and does not cross regions.

[0051] In this embodiment, if compression is deemed invalid, the database system discards the compression result and uses the original data as the final target data. The original data is written to the physical disk space of the data domain in its uncompressed, raw form. The length of the original data is equal to the logical page size, and it is also divided into compressed blocks (i.e., divided into a fixed number of complete blocks), while maintaining the consistency of the write path.

[0052] Based on the above steps, after the compression operation is completed, the byte length of the intermediate compressed data is compared with a preset threshold determined based on the compression ratio of the compression algorithm. When the length of the compressed data is not less than the preset threshold, the compression result is discarded and the original data is stored in uncompressed form. This avoids the negative benefit of occupying more space and ensures that the storage efficiency is no worse than the uncompressed storage scheme under any data characteristics. The byte length of the logical page is specified to be a positive integer multiple of the compressed block byte length, making the block boundaries predictable. The starting position of the compressed block in the data field can be calculated without reading the control field, supporting block-level parallel reading and partial decompression optimization. Fixed-size compressed blocks are used as the smallest allocation granularity to avoid the fragmentation management complexity caused by variable-length blocks.

[0053] In the embodiment of step S401, after obtaining the compressed intermediate data according to the compression operation, the method further includes: if the byte length of the intermediate data is greater than or equal to the byte length of the original data, skip the compression and write the original data into the physical disk space of the data field in an uncompressed form.

[0054] In this embodiment, after the compression operation is completed, the database system compares the actual byte length of the intermediate compressed data with the byte length of the original data. If the intermediate data length is greater than or equal to the original data length, it indicates that the compression has caused bloat or is completely invalid. In this case, the database system immediately abandons the compression result, releases temporary memory, confirms the original data as the target data, and switches to the uncompressed writing process. If the intermediate data length is less than the original data length, it indicates that the compression has generated positive benefits. The database system continues to compare the intermediate data length with a preset threshold based on the algorithm's compression ratio to evaluate the benefits of compression. Through this pre-judgment mechanism, the database system can identify and filter scenarios of invalid compression with low decision-making cost, avoid space waste caused by compression bloat, and reduce CPU overhead caused by invalid compression.

[0055] In the implementation of step S104, after storing the mapping entries of one or more compressed blocks corresponding to the target logical page in the control domain, the method further includes: if the target data written to the physical disk space of the data domain is deleted, marking the corresponding mapping entry in the control domain as invalid, and releasing the physical disk space through a hole-punching operation.

[0056] In this implementation, upon receiving a deletion request for a target logical page, the database system locates the mapping entry in the extended region and control domain to which the logical page belongs based on its logical page number. The mapping entry is marked as invalid, ensuring that subsequent requests to read the target logical page can identify that the page no longer contains valid data. Based on the compressed block offset and length information recorded in the mapping entry, the database system calls the hole-punching interface provided by the operating system to release the corresponding physical disk space and updates the space usage bitmap in the control domain, marking the released allocation units as free. If all mapping entries in the entire extended region are marked as invalid, a hole-punching operation is performed on the entire extended region, adding it to the free extended region pool. This two-stage release mechanism, combining marking and hole punching, achieves efficient metadata updates for deletion operations and on-demand reclamation of physical space, avoiding space waste.

[0057] In the implementation of step S104, after storing the mapping entries of one or more compressed blocks corresponding to the target logical page in the control domain, the method further includes: if the target data written to the physical disk space of the data domain is modified, updating the offset and data length of the corresponding mapping entry in the control domain, and releasing the physical disk space occupied by the target data through a hole punching operation.

[0058] In this implementation, upon receiving an update request for a target logical page, the database system locates the original mapping entry in its extended area and control domain based on the logical page number, and reads the old compressed block offset and length information recorded therein. The database system performs a compression operation on the new data, writing the compressed target data into a free area of ​​the data domain in compressed blocks, recording the offset and length of the new blocks. After writing, the database system performs an atomic update operation on the mapping entry in the control domain, updating it to the new location information; after the update, subsequent read requests will locate the new data based on the new mapping entry. Based on the offset and length recorded in the old mapping entry, the database system calls the hole-punching interface provided by the operating system to release the physical disk space occupied by the old data, and updates the space using a bitmap to mark the corresponding allocation unit as free. If the length of the newly compressed data does not exceed the length of the old data and there is sufficient space in the original location, the database system can choose an in-place update strategy, directly overwriting the original location and adjusting the length field, avoiding the overhead of reallocation and hole-punching release, thereby achieving efficient atomicity of data updates and on-demand reclamation of physical space.

[0059] The flowchart provided in this embodiment is not intended to indicate that the operations of the method will be performed in any particular order, or that all operations of the method are included in every case. Furthermore, the method may include additional operations. Within the scope of the technical concept provided by the method in this embodiment, additional variations can be made to the above method.

[0060] It should be understood that in some embodiments, the components may be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods may be implemented using software or firmware stored in memory and executed by a suitable instruction execution system.

[0061] This embodiment also provides a computer program product 10, a computer-readable storage medium 20, and a computer device 30. Figure 5 This is a schematic diagram of a computer program product 10 according to an embodiment of the present invention. Figure 6 This is a schematic diagram of a computer-readable storage medium 20 according to an embodiment of the present invention. Figure 7 This is a schematic diagram of a computer device 30 according to an embodiment of the present invention. Figure 5 As shown, the computer program product 10 includes a computer program 11, which, when executed by the processor 32, implements the steps of any of the above-described database compression storage methods. For example... Figure 6 As shown, a computer-readable storage medium 20 stores the aforementioned computer program 11, which, when executed by the processor 32, implements the steps of the database compression storage method of any of the above embodiments. Figure 7As shown, the computer device 30 may include a memory 31, a processor 32, and a computer program 11 stored on the memory 31 and running on the processor 32.

[0062] The computer program 11 used to perform the operations of this invention may be assembly instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages ​​and procedural programming languages. The computer program 11 may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer. In some embodiments, to perform aspects of this invention, electronic circuits, including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute computer-readable program instructions to personalize the electronic circuits by utilizing status information of the computer-readable program instructions.

[0063] For the purposes of this embodiment, computer program product 10 is a related product that includes computer program 11.

[0064] For the purposes of this embodiment, computer-readable storage medium 20 is a tangible device capable of holding and storing a computer program 11. It can be any device capable of containing, storing, communicating, propagating, or transmitting the program 11 for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable storage medium 20 include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanical encoding device, and any suitable combination thereof.

[0065] Computer device 30 can be, for example, a server, desktop computer, laptop computer, tablet computer, or smartphone. In some examples, computer device 30 can be a cloud computing node. Computer device 30 can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., that perform specific tasks or implement specific abstract data types. Computer device 30 can be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can reside on local or remote computing system storage media, including storage devices.

[0066] Computer device 30 may include a processor 32 adapted to execute stored instructions and a memory 31 that provides temporary storage space for the operation of said instructions during operation. The processor 32 may be a single-core processor, a multi-core processor, a computing cluster, or any other configuration. The memory 31 may include random access memory (RAM), read-only memory, flash memory, or any other suitable storage system.

[0067] Computer device 30 may also include a network adapter / interface and an input / output (I / O) interface. The I / O interface allows external devices that can be connected to the computer device to input and output data. The network adapter / interface provides communication between the computer device and a network, typically represented as a communication network.

[0068] Therefore, those skilled in the art should recognize that although numerous exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications conforming to the principles of the present invention can be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Thus, the scope of the present invention should be understood and construed as covering all such other variations or modifications.

Claims

1. A method for compressed storage of a database, characterized in that, include: The system receives a write request for a target logical page in a data table, and determines the extended region to which the target logical page belongs and the offset position of the target logical page in the extended region based on the logical page number of the target logical page. The extended region is a contiguous logical page range on a logical address, used to manage space allocation and reclamation. Each extended region contains a data field and a control field. The data domain usage information of the extended area is determined according to the control domain, and the original data carried by the target logical page is compressed based on the data domain usage information. The target data obtained based on the compression operation is written into the physical disk space of the data domain; The control domain stores mapping entries for one or more compressed blocks corresponding to the target logical page. Each mapping entry includes the logical page number, the offset of the compressed block within the extension area, and the length of the compressed data.

2. The database compression storage method according to claim 1, characterized in that, Before receiving a write request for a target logical page in the data table, the following steps are also included: The logical page address space of the data table is sequentially divided into multiple logically contiguous extension regions, each of which contains the same number of logical pages; The extended area is logically divided into a data domain and a control domain, wherein the data domain is used to store the target data; and the control domain is used to store control pages that record physical location mappings.

3. The database compression storage method according to claim 1, characterized in that, Determining the extended region to which the target logical page belongs, and the offset position of the target logical page within the extended region, based on the logical page number of the target logical page, includes: Perform an integer division operation between the logical page number of the target logical page and the number of logical pages contained in a single extension region, and determine the extension region to which the target logical page belongs based on the quotient obtained from the integer division operation; A modulo operation is performed between the logical page number of the target logical page and the number of logical pages contained in a single extension region. The offset position of the target logical page within its extension region is determined based on the remainder obtained from the modulo operation.

4. The database compression storage method according to claim 1, characterized in that, Determining the data domain usage information of the extended area based on the control domain includes: The control page stored in the control domain is read, the space usage information of each logical page recorded in the control page is parsed, and the data domain usage information of the extended area is determined based on the space usage information; wherein, the space usage information is used to indicate the occupancy status of the compressed data in the data domain.

5. The database compression storage method according to claim 1, characterized in that, Performing compression operations on the original data carried by the target logical page based on the data field usage information includes: The upper limit of the length of the compressed data is determined based on the data field usage information. Based on the aforementioned upper limit value, a target compression algorithm is selected from a variety of candidate compression algorithms; The original data is compressed using the target compression algorithm.

6. The database compression storage method according to claim 1, characterized in that, Writing the target data obtained based on the compression operation into the physical disk space of the data domain includes: The compressed intermediate data is obtained according to the compression operation; Determine whether the byte length of the intermediate data is less than a preset threshold, the preset threshold being determined based on the compression ratio of the selected target compression algorithm; If so, the intermediate data is used as the target data and written to the physical disk space of the data field in compressed blocks, wherein the byte length of the logical page is a positive integer multiple of the byte length of the compressed block; If not, the original data is used as the target data and written to the physical disk space of the data domain in uncompressed form.

7. The database compression storage method according to claim 6, characterized in that, After obtaining the compressed intermediate data according to the compression operation, the method further includes: If the length of the intermediate data is greater than or equal to the length of the original data, compression is skipped, and the original data is written to the physical disk space of the data field in uncompressed form.

8. The database compression storage method according to claim 1, characterized in that, After storing the mapping entries of one or more compressed blocks corresponding to the target logical page in the control domain, the method further includes: If the target data written to the physical disk space of the data domain is deleted, the corresponding mapping entry in the control domain is marked as invalid, and the physical disk space is released through a hole-punching operation.

9. The database compression storage method according to claim 1, characterized in that, After storing the mapping entries of one or more compressed blocks corresponding to the target logical page in the control domain, the method further includes: If the target data written to the physical disk space of the data domain is modified, update the offset and data length of the corresponding mapping entry in the control domain, and release the physical disk space occupied by the target data through a hole-punching operation.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the compressed storage method for the database according to any one of claims 1 to 9.

Images