OpenGauss database space management and data layout method and device

Abstract

The invention relates to an openGauss database space management and data layout method and device. The method comprises the steps that various data objects in a database are jointly stored in a size class file, and the page use condition of the data objects is recorded through metadata; a progressive spatial expansion strategy is adopted, small page expansion is initially carried out, and the page expansion number is doubled along with the increase of the data size until the maximum expansion spatial threshold value is reached; and returning the space applied by the deleted object to the corresponding size class, and updating the related mark for subsequent reuse. According to the method, the number of file objects can be reduced to the maximum extent, management is simplified, the space distribution and recovery efficiency is improved, the database performance is remarkably improved, and the method has remarkable advantages especially for large-scale data storage and frequent operation scenes.

Description

Technical Field

[0001] The present application relates to the technical field of database operation, and in particular to an openGauss database space management and data layout method, device and electronic device. Background Art

[0002] OpenGauss uses a page-based storage method to manage data space. Figure 1 As shown, each data object (table, index, sequence, temporary table, temporary space used for sorting, etc.) occupies a file. Each file is divided into pages (hereinafter referred to as pages). Each page corresponds to a page number. For example, page 0 corresponds to data offsets from 0 to 8191, page 1 corresponds to data offsets from 8192 to 16383, and so on.

[0003] The page-based space management solution has advantages such as simple implementation and convenient addressing. However, this space management method also has the following obvious shortcomings: (1) When using traditional page space management, as the data in the table grows (especially when a large partition table is created), a large number of file objects will appear inside the database, and the number of file handles required by the database will increase exponentially, which can easily lead to problems such as too many open files.

[0004] (2) Using traditional page space management, each page expansion can only add one page (i.e. 8Kb). For scenarios where a large amount of data is imported, the page expansion speed is slow, which becomes a system performance bottleneck.

[0005] (3) When using traditional page space management, temporary tables, temporary sort spaces, etc. will frequently create, expand, and delete file objects. Such scenarios can easily lead to database performance degradation.

[0006] Furthermore, while new data file management methods are constantly being developed, for example, Chinese patent application "A Smart Card File System Management Method" (CN101593193A) employs a DAT (Data Allocation Table) approach to manage the internal space of smart cards. However, this approach still has numerous drawbacks, including: low file management flexibility; the file system is divided into multiple fixed areas, with different data or files occupying discontinuous clusters, making it difficult to store multiple data objects in a single file; and increased management complexity as the number of files increases; fixed-size clusters are used as units for space expansion, making space expansion inconvenient; low space reclamation and utilization efficiency; the need to use the DAT table to find the cluster occupied by the file and then calculate the address, resulting in high addressing complexity; files are stored in discontinuous clusters in a chained manner, requiring data reading and writing to jump between clusters, resulting in poor data read and write continuity; narrow application scenarios, and limited scalability.

[0007] To solve the above problems, we need a new space management and data layout solution. Summary of the Invention

[0008] In order to solve the above problems faced by traditional page space management, the present invention proposes a new openGauss database space management and data layout method.

[0009] Figure 2 The overall structure of the present invention is shown below. The present invention mainly adopts the following technical strategies: (1) Data object common storage strategy: Abandon the traditional page space management method of using independent files to store each data object, and instead store different data objects together in size class files, such as Figure 2 As shown in the figure, object1 shares the same size class file with other data objects. By using metadata to accurately record the pages used by each data object in the size class file, we can efficiently store and manage multiple data objects in a unified file, effectively reducing the number of file objects generated and simplifying the file management structure.

[0010] (2) Progressive space expansion strategy: To address the limitation of traditional page space management that can only expand one page space (8KB) ​​at a time, this solution adopts a progressive expansion method. In the initial stage, smaller page units are expanded as needed. As the amount of data grows, the number of pages expanded each time is dynamically increased. Specifically, every 8 expansions, the size class number is increased by 1, and the number of pages contained in a class increases by a doubling rule. In this way, space can be reasonably expanded according to the data growth trend, the overhead of frequent expansion operations can be reduced, and system performance can be improved. The advantages are particularly significant in scenarios such as data import.

[0011] (3) Efficient space recovery and reuse strategy: Unlike traditional page space management, which directly deletes the corresponding files when deleting a table and makes the page space unusable, this solution returns the allocated space to the system when deleting a table. Other data objects can reuse this recovered space, optimizing the space allocation process, improving the space utilization efficiency of the database system, and further enhancing the overall database performance.

[0012] Specifically, this application provides the following technical solutions: A first aspect of the present application provides an openGauss database space management and data layout method, the method comprising: S1. Data object creation: Store various database data objects (including tables, temporary tables, sequences, indexes, sort spaces, etc.) in size class files. Metadata is used to record the page usage of each data object in the size class file. S2. Space Request and Page Expansion: Space expansion is performed according to a progressive space expansion strategy based on the size class usage of the data object. The number of pages expanded each time doubles as the size class number increases, until the preset expansion threshold is reached. S3. Data object deletion: The space allocated for the deleted object is returned to the corresponding size class and the relevant mark is updated for subsequent reuse.

[0013] Furthermore, in the progressive space expansion strategy described in step S2 of the present method, the size class number is increased by 1 every 8 expansions, and the number of pages expanded once for size class (M) is 2 M , the expansion space size is 8Kb×2 M , and it is stipulated that a maximum of 64Mb of space can be expanded at a time, that is, when M = 13, the expansion space no longer grows.

[0014] Furthermore, the method of the present application also includes: setting an FSM page at the head of each size class to record the space usage of the current size class. When the page is expanded, free space is first searched from the FSM for allocation. If there is no free space in the FSM, it is expanded from the tail of the size class, and the usage mark in the FSM and the metadata page record corresponding to the object are updated.

[0015] Furthermore, in the present application method, each data object corresponds to a metadata page, which is used to store all class spaces applied for by the data object, and can store up to 227 SizeClassBMS, that is, a data object can use up to 227 size classes and up to 100G space.

[0016] Furthermore, the formula for calculating the physical location of the class space in the present application method is: pageNo = 10 + P + (x–total_size_class(M - 1))% 2 M Formula Description: pageNo is the physical location of page(x); 10 is a fixed value because the size class starts with 10 pages for FSM management. x is the logical number of the page; total_size_class (M - 1) indicates how many pages are allocated to all size classes before M.

[0017] Furthermore, the specific process of creating the data object in step S1 of the present method is as follows: Search for unused pages in the metadata header. If there are no free pages, extend a new page from the end of the metadata. Initialize the metadata page of the newly created data object to record the data space used by the object. The size class of the newly created data object is initially empty.

[0018] Furthermore, the specific process of space application and page expansion in step S2 of this application method is as follows: On the metadata page, query the size class usage of the object and determine the starting size class for expansion. According to the number of expansions N of the current object in size class (M), if N is less than 8, the next expansion is to space class (N+1) in size class (M); if N is equal to 8, the next expansion is to space class (1) in size class (M+1); The number of pages expanded at a time by size class (M) is 2 M The page is expanded according to the rules until the preset expansion space threshold is reached.

[0019] Furthermore, the specific process of deleting the data object in step S3 of the present method is as follows: Read the metadata page, return all class space requested by the current object to the corresponding size class, and mark the class space as free in the FSM corresponding to the size class; Return the metadata page to metadata, and mark the metadata page corresponding to the deleted object as free in the header page.

[0020] Furthermore, in the present application method, (1) The specific steps of the page application process are as follows: Read the metadata page of the object; Calculate the size class that the object needs to apply for based on metadata; Find the FSM for the corresponding size class, apply for an available class space, and set the corresponding bit in the FSM to 1 to indicate that it is in use. At this time, the bit multiplied by 8 is the class space address; Record the location of the class in metadata; (2) The specific steps of the page addressing process are as follows: Read the metadata page of the object; Get the class space address; According to pageNo = 10 + P + (x–total_size_class(M - 1)) % 2 M The formula calculates the physical location of the page and reads the location to obtain the page content; (3) The specific steps of deleting the corresponding page return process are as follows: Read the metadata page of the object; Traverse all class space addresses applied for by all objects, divide the class space address by 8 to get the FSM bit, and set the bit to 0 to indicate that the class is idle; Returns the metadata page to the metadata file.

[0021] A second aspect of the present application provides an openGauss database space management and data layout device, the device comprising: Data object creation module: used to store multiple data objects in the database in size class files, and use metadata to record the page usage of each data object in the size class file; Space application and page expansion module: This module is used to expand space based on the size class usage of data objects using a progressive space expansion strategy. The number of pages expanded each time doubles as the size class number increases until the preset expansion space threshold is reached. Data object deletion module: used to return the space requested by the deleted object to the corresponding size class and update the relevant tags for subsequent reuse.

[0022] The device implements the steps of the aforementioned openGauss database space management and data layout method when running.

[0023] A third aspect of the present application provides an electronic device, comprising: a memory and a processor; Memory: used to store computer programs; Processor: used to execute the computer program to implement the steps of the aforementioned openGauss database space management and data layout method.

[0024] A fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program implements the steps of the aforementioned openGauss database space management and data layout method.

[0025] In summary, the present invention has the following technical advantages: (1) Integrated storage: By designing a unique space management and data layout solution, a file can store multiple data objects, including tables, indexes, sequences, temporary files, etc. This effectively avoids the problem of generating a large number of file objects due to the separate storage of each data object in traditional solutions, simplifies the file management structure, and improves storage efficiency.

[0026] (2) Gradual expansion: Abandoning the traditional model of expanding only one page at a time, a gradual expansion strategy is adopted. In the initial stage, smaller page units are expanded as needed. As the amount of data grows, the number of pages expanded each time is gradually increased. This achieves dynamic expansion based on data growth trends, reduces frequent expansion operations, and significantly improves performance in scenarios such as data import.

[0027] (3) Efficient space recovery and reuse: After table deletion, temporary table cleanup, and sorting, free space can be recovered promptly and efficiently. Furthermore, when creating a table and starting sorting, space can be allocated and reused from free space first, optimizing the space allocation process and significantly improving the space utilization efficiency and overall performance of the database system.

[0028] Other features and advantages of the present invention will be described in detail in the following description, or may be understood through implementation of the relevant technical solutions of this application. The objectives and other advantages of this application may be achieved through the technical features and technical means clearly indicated in the description, claims, and drawings, and may be obtained through the implementation of these technical contents. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] To more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings involved in the description of the embodiments. It should be noted that the drawings only illustrate some embodiments of the present application. Those skilled in the art can deduce other relevant drawings based on these drawings without engaging in creative work.

[0030] Figure 1 Implementation architecture diagram for traditional page space management.

[0031] Figure 2This is the overall design architecture diagram of the present invention.

[0032] Figure 3 This is a flowchart of the overall implementation of the openGauss database space management and data layout method of the present invention.

[0033] Figure 4 Create an example graph for the data objects in the method of the present invention.

[0034] Figure 5 This is an example diagram of space application and page expansion in the method of the present invention.

[0035] Figure 6 It is a structural diagram of the composition of the device of the present invention.

[0036] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0038] In this document, the term "including" and any variations thereof (such as "including," "comprising," etc.) are open-ended expressions and should be understood as meaning "including but not limited to," meaning that the listed contents are not exhaustive and may include other contents not explicitly mentioned. The term "based on" should be understood as meaning "based at least in part on," meaning that the basis or condition referred to may not be the only factor and may also involve other relevant factors. The term "one embodiment" should be understood as meaning "at least one embodiment," meaning that the described embodiment is not the only possible implementation method and that other similar embodiments may exist.

[0039] In this application, the terms "a" and "a plurality" are used to modify related elements or features in an illustrative, non-restrictive manner. Unless the context clearly indicates otherwise, "a" should be understood as meaning "at least one," and "a plurality" should be understood as meaning "at least two." Those skilled in the art should interpret these terms appropriately based on the semantics and logical relationships of the context to ensure that they encompass the possibility of "one or more."

[0040] Figure 3 The figure shows the overall implementation process of the openGauss database space management and data layout method provided by this application, including the following steps: S1. Data object creation: Store various database data objects (including tables, temporary tables, sequences, indexes, sort spaces, etc.) in size class files. Metadata is used to record the page usage of each data object in the size class file. S2. Space Request and Page Expansion: Space expansion is performed according to a progressive space expansion strategy based on the size class usage of the data object. The number of pages expanded each time doubles as the size class number increases, until the preset expansion threshold is reached. S3. Data object deletion: The space allocated for the deleted object is returned to the corresponding size class and the relevant mark is updated for subsequent reuse.

[0041] In order to more clearly illustrate the technical solution of the present application, the following will further illustrate it through embodiments of specific scenarios.

[0042] The specific implementation process of the method of the present invention is as follows: (1) Data object creation Data objects refer to various objects within the database that require space to store data, including tables, temporary tables, sequences, indexes, sorting spaces, etc. Figure 4 As shown, create data object object1, the process is as follows: 1. Find an unused page in the metadata header. If there is no free page, extend a new page from the end of the metadata to store it.

[0043] 2. Initialize the metadata page of object1, and then use object1 to record the data space used by the object. The size class of the newly created data object is empty. Figure 4 .

[0044] (2) Space application and page expansion like Figure 5 As shown, the data object object1 applies for new storage space (also known as page expansion). The process is as follows: 1. In the metadata page, query the size class usage of the object. The newly created table sizeclass(0) starts to expand. Every 8 expansions, the size class number increases by 1, that is, size class(0) expands 8 times, then size class(1) expands 8 times, then size class(2) expands 8 times, and so on. See Figure 5, the current object has been expanded N times in size class (M). If N is less than 8, the next expansion will be in size class (M) and space class (N+1). If N is equal to 8, the next expansion will be in size class (M+1) and space class (1).

[0045] This solution stipulates that the number of pages that size class (M) can expand at a time is 2 M , the expansion space size is 8Kb×2 M As the amount of data increases, the number of pages to be expanded increases exponentially, thus avoiding frequent page expansion. At the same time, to avoid allocating too much space at once, this solution stipulates that a maximum of 64MB of space can be expanded at a time. That is, when M = 13, the expansion space no longer increases. The following table shows the relationship between M, the number of expansion pages, and the size of the expansion space:

[0046] (3) Each size class header contains a free space map (FSM) page, which records the space usage of the current size class. When a page is expanded, free space is first searched in the FSM. If no free space is found, the page is expanded from the end of the size class. The newly allocated class is then marked as used in the FSM, and the newly allocated class space is recorded in the object's corresponding metadata page (object1 page).

[0047] (IV) Deletion of Data Objects The deletion process is as follows: 1. Read the metadata page and return all the class space requested by the current object to the corresponding size class. In the FSM corresponding to the size class, mark the class space as free. This class space can be allocated to other data objects for reuse later.

[0048] 2. Return the metadata page to metadata, and mark the metadata page corresponding to the deleted object as free on the header page. This page can be reallocated to other objects later.

[0049] The following uses a specific example to illustrate the specific implementation process of page application, addressing, and page return when deleting objects under this solution architecture.

[0050] (1) FSM free space management Each size class header contains a fixed-size FSM page, which marks whether a class space is allocated to an object file. When a class is allocated, the FSM searches for free space. If free space is available, it is allocated directly from the FSM. If no free space is found in the FSM, a new class space is allocated from the end of the size class and the newly allocated class space is marked as in use in the FSM.

[0051] The FSM occupies 10 pages, with a total size of 80 KB. The FSM internally uses a bitmap structure, where a single bit indicates whether a class (N) is free. The maximum number of classes that can be marked by the FSM is 655,360 (8 * 80 KB). Since this solution stipulates that a size class can be expanded a maximum of eight times, the maximum number of objects that can be allocated per size class is: 655,360 / 8 = 81,920. This means that this solution limits the creation of a maximum of 81,920 data objects per database, which is sufficient for most production environments.

[0052] In addition, the metadata free space management method is the same as FSM.

[0053] (2) Object metadata page and class space storage Each data object has a corresponding metadata page to store all the class spaces applied for by the object. The structure of the object metadata page is shown in the following code:

[0054] 1. Use a 32-bit unsigned integer to identify the address of the class, see the bms member of SizeClassBMS.

[0055] 2. According to the page expansion rules, a size class can allocate up to 8 class spaces.

[0056] 3. To fit the ObjectHeadPage structure into a single page (limiting its size to 8 KB), the structure stores a maximum of 227 SizeClassBMSs. This means that a data object can use up to 227 size classes. Based on page expansion rules, the maximum space available for a single data object is 100 GB, sufficient for most production environments.

[0057] (3) Get the location of the class space In traditional paging space management, each data object occupies a dedicated data file. Page addressing is straightforward: to obtain the data for page(x), the page's offset in the physical file is 8 * 8KB. Because multiple files share size class files, this solution requires using the size class information recorded in the metadata page to determine the size class and class space location of the page.

[0058] Since the size and number of class space allocations for each size class are fixed, as shown in the following table, we can calculate how many pages are allocated to an object when it is extended to the size class (M):

[0059] Taking page (50) as an example, according to the above table, we can conclude that the size class of page (50) is 2, that is, M=2.

[0060] Since size class (2) requires 2 pages per application 2 =4, then the class number of page(50) is: N = 50% ×4 = 2.

[0061] Therefore, the class space address = ObjectHeadPage.classes[M].bms[N].

[0062] (IV) Mapping class address to file physical location In the previous solution, we calculated the class address (denoted as P). In this section, we use the class address to obtain the physical page number where page(x) is located. The calculation formula is: pageNo = 10 + P + (x–total_size_class(M - 1))% 2 M Formula Description: 10 is a fixed value because the size class starts with 10 pages for FSM management.

[0063] x is the logical number of the page, and the physical number of the page is calculated using the formula.

[0064] total_size_class (M - 1) indicates how many pages are allocated in total for all size classes before M. In particular, when M is 0, total_size_class (M - 1) is 0, indicating that no page is allocated before size class (0).

[0065] The current size class allocates 2 at a time M pages, divided by 2 M Take the remainder and get the relative position of the physical page corresponding to x in the current class.

[0066] Finally, the physical location of page (x) is: pageNo * 8Kb. Reading the offset position can obtain the content of page x.

[0067] (V) Implementation steps of each process 1. Page application process: Read the metadata page of the object; Calculate the size class that the object needs to apply for based on metadata; Find the FSM for the size class, apply for an available class space, set the corresponding bit in the FSM to 1 to indicate that it is in use, and the bit * 8 is the class space address; Record the location of the class in metadata.

[0068] 2. Page addressing process: Read the metadata page of the object; Use the above method to get the class space address; Use the above method to get the physical location of the page, and read the location to get the page content.

[0069] 3. Delete the corresponding page return process: Read the metadata page of the object; Traverse all class space addresses applied for by all objects. The class space address / 8 is the FSM bit. If the bit is set to 0, it means that the class is idle. The metadata page is returned to the metadata file in the same way.

[0070] Figure 6 The figure shows an openGauss database space management and data layout device proposed in this application, the device comprising: Data object creation module: used to store multiple data objects in the database in size class files, and use metadata to record the page usage of each data object in the size class file; Space application and page expansion module: This module is used to expand space based on the size class usage of data objects using a progressive space expansion strategy. The number of pages expanded each time doubles as the size class number increases until the preset expansion space threshold is reached. Data object deletion module: used to return the space requested by the deleted object to the corresponding size class and update the relevant tags for subsequent reuse.

[0071] When the above device is running, the steps of the openGauss database space management and data layout method disclosed in this application are implemented.

[0072] The flowcharts and block diagrams in the accompanying drawings illustrate possible implementations of the apparatus, methods, and computer program products according to various embodiments of the present application, including architecture, functions, and operations. In these figures, each box may represent a module, a program segment, or a portion of a code, which contains one or more executable instructions for implementing a specified logical function. It should be noted that each box in the block diagram and / or flowchart, and the combination of these boxes, can be implemented using a dedicated hardware-based system to implement the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0073] like Figure 7 As shown, an embodiment of the present application further discloses an electronic device, comprising: a processor 310, a communication interface 320, a memory 330 for storing a computer program executable by the processor, and a communication bus 340. The processor 310, the communication interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 executes the executable computer program to implement the steps of the above-mentioned openGauss database space management and data layout method.

[0074] It is understood that, in addition to the memory and processor, the electronic device may also include an input device (e.g., a keyboard), an output device (e.g., a display), and other communication modules. These input devices, output devices, and other communication modules all communicate with the processor via an I / O interface (i.e., an input / output interface).

[0075] The operation of the present application can be implemented by writing computer program code using one or more programming languages ​​or a combination thereof. The programming languages ​​include but are not limited to the following types: Object-oriented programming languages, such as Java, Smalltalk, C++, etc.; A conventional procedural programming language, such as "C" or a similar programming language.

[0076] The execution methods of the program code include but are not limited to: Executes entirely on the user's computer; Partially executed on the user's computer and partially on a remote computer; Executed as a standalone software package; Executes entirely on the remote computer or server.

[0077] In scenarios involving a remote computer, the remote computer can be connected to the user's computer via any type of network, including but not limited to a local area network (LAN) or a wide area network (WAN). Additionally, the remote computer can be connected to an external computer via an Internet service provider, such as the Internet.

[0078] Furthermore, the present application also discloses a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can perform the various steps of the openGauss database space management and data layout method disclosed in the present application.

[0079] In the context of this application, computer-readable storage media refers to tangible media that can store computer program code and related data. Specific examples include, but are not limited to, the following: (1) Portable computer disk: A removable magnetic storage medium such as a floppy disk.

[0080] (2) Hard disk: includes fixed storage devices such as mechanical hard disks and solid-state hard disks.

[0081] (3) Random Access Memory (RAM): Volatile storage medium used for temporary storage of data and program code.

[0082] (4) Read-only memory (ROM): A non-volatile storage medium used to store fixed programs and data.

[0083] (5) Erasable Programmable Read-Only Memory (EPROM) or Flash Memory: A non-volatile storage medium that supports multiple erasing and programming.

[0084] (6) Fiber optic storage device: storage medium based on fiber optic technology.

[0085] (7) Compact Disc Read-Only Memory (CD-ROM): A read-only medium that stores data in the form of an optical disc.

[0086] (8) Optical storage devices: storage media based on optical principles, such as DVDs and Blu-ray discs.

[0087] (9) Magnetic storage devices: storage media based on magnetic principles, such as magnetic tapes and disks.

[0088] (10) Any suitable combination of the above: for example, combining multiple storage media to meet different storage requirements.

[0089] These computer-readable storage media can be used to store the program code and related data described in this application to support the operation of the program and the persistent storage of data.

[0090] In particular, according to embodiments of the present application, the processes described in the flowcharts can be implemented as computer software programs. For example, embodiments of the present application relate to a computer program product comprising a computer program embodied on a non-transitory computer-readable medium. The computer program includes program code for executing the openGauss database space management and data layout methods disclosed herein. When the computer program is executed by a processing device, the aforementioned functions defined in the embodiments of the present application can be implemented.

[0091] Although the above discussion contains several specific implementation details, these details should not be interpreted as limiting the scope of this application. The above description is only a preferred embodiment of the present application and an illustration of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in this application is not limited to the technical solutions formed by the specific combination of the above technical features. At the same time, this application should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concepts.

[0092] Those skilled in the art should also understand that they may modify the technical solutions described in the aforementioned embodiments, or replace some of the technical features therein with equivalents, without departing from the spirit and scope of the technical solutions of the embodiments of the present application. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the core spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for space management and data layout of openGauss database, characterized in that: The method comprises: S1. Data object creation: Store multiple data objects within the database in size class files and use metadata to record the page usage of each data object in the size class file. S2. Space Request and Page Expansion: Space expansion is performed according to a progressive space expansion strategy based on the size class usage of the data object. The number of pages expanded each time doubles as the size class number increases, until the preset expansion threshold is reached. S3. Data object deletion: The space allocated for the deleted object is returned to the corresponding size class and the relevant mark is updated for subsequent reuse.

2. The method according to claim 1, characterized in that In the progressive space expansion strategy described in step S2, the size class number is increased by 1 every 8 expansions, and the number of pages expanded at one time for size class (M) is 2 M , the expansion space size is 8Kb×2 M , and it is stipulated that a maximum of 64Mb of space can be expanded at a time, that is, when M = 13, the expansion space no longer grows.

3. The method according to claim 1, characterized in that The method further includes: setting an FSM page at the head of each size class to record the space usage of the current size class; when expanding a page, first searching for free space in the FSM for allocation; if there is no free space in the FSM, expanding from the tail of the size class, and updating the usage mark in the FSM and the metadata page record corresponding to the object.

4. The method according to claim 1, wherein In the method, each data object corresponds to a metadata page, which is used to store all class spaces applied for by the data object. A maximum of 227 SizeClassBMS can be stored, that is, a data object can use a maximum of 227 size classes and a maximum of 100G space.

5. The method according to claim 1, wherein The formula for calculating the physical position of the class space in the method is: pageNo = 10 + P +(x–total_size_class(M - 1))% 2 M Formula Description: pageNo is the physical location of page(x); 10 is a fixed value because the size class starts with 10 pages for FSM management. x is the logical number of the page; total_size_class (M - 1) indicates how many pages are allocated to all size classes before M.

6. The method according to claim 1, characterized in that The specific process of creating the data object in step S1 is as follows: Search for unused pages in the metadata header. If there are no free pages, extend a new page from the end of the metadata. Initialize the metadata page of the newly created data object to record the data space used by the object. The sizeclass of the newly created data object is initially empty.

7. The method according to claim 1, characterized in that The specific process of space application and page expansion in step S2 is as follows: On the metadata page, query the size class usage of the object and determine the size class to start with. According to the number of expansions N of the current object in size class (M), if N is less than 8, the next expansion is to space class (N+1) in size class (M); if N is equal to 8, the next expansion is to space class (1) in size class (M+1); The number of pages expanded at a time by size class (M) is 2 M The page is expanded according to the rules until the preset expansion space threshold is reached.

8. The method according to claim 1, characterized in that The specific process of deleting the data object in step S3 is as follows: Read the metadata page, return all class space requested by the current object to the corresponding size class, and mark the class space as free in the FSM corresponding to the size class; Return the metadata page to metadata, and mark the metadata page corresponding to the deleted object as free in the header page.

9. The method according to claim 1, characterized in that In the method, (1) The specific steps of the page application process are as follows: Read the metadata page of the object; Calculate the size class that the object needs to apply for based on metadata; Find the FSM for the corresponding size class, apply for an available class space, and set the corresponding bit in the FSM to 1 to indicate that it is in use. At this time, the bit multiplied by 8 is the class space address; Record the location of the class in metadata; (2) The specific steps of the page addressing process are as follows: Read the metadata page of the object; Get the class space address; According to pageNo = 10 + P + (x–total_size_class(M - 1)) % 2 M The formula calculates the physical location of the page and reads the location to obtain the page content; (3) The specific steps of deleting the corresponding page return process are as follows: Read the metadata page of the object; Traverse all class space addresses applied for by all objects, divide the class space address by 8 to get the FSM bit, and set the bit to 0 to indicate that the class is idle; Returns the metadata page to the metadata file.

10. An openGauss database space management and data layout device, characterized in that: The device comprises: Data object creation module: used to store multiple data objects in the database in size class files, and use metadata to record the page usage of each data object in the size class file; Space application and page expansion module: This module is used to expand space based on the size class usage of data objects using a progressive space expansion strategy. The number of pages expanded each time doubles as the size class number increases until the preset expansion space threshold is reached. Data object deletion module: used to return the space requested by the deleted object to the corresponding size class and update the relevant tags for subsequent reuse.

Images