Index bloat suppression method and apparatus, and computing device cluster

Abstract

An index bloat suppression method, comprising: determining that an index is bloated; identifying reclaimable pages contained in the index, and sequentially adding the reclaimable pages into a page reclamation queue, wherein the reclaimable pages include sparse pages or empty pages; and when the index requires a new page, sequentially reusing the pages contained in the page reclamation queue. In this way, when an index is bloated, reclaimable pages in the index are sequentially added to a page reclamation queue, and the pages in the page reclamation queue are sequentially reused, thereby suppressing index bloat.

Description

A method, apparatus, and computing device cluster for suppressing index inflation

[0001] This application claims priority to Chinese Patent Application No. 202311690375.0, filed on December 8, 2023, entitled "A Method, Apparatus and Computing Device Cluster for Suppressing Index Inflation", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of information technology (IT) technology, and in particular to an index expansion suppression method, apparatus and computing device cluster. Background Technology

[0003] Indexes are a crucial component for accelerating database query operations. However, over time, indexes can bloat, leading to performance degradation and wasted storage resources. For example, index skew, out-of-order writes and deletions, or mark-and-delete operations where the deleted space isn't promptly reclaimed and reused can all cause indexes to occupy significantly more storage space than the amount of data they need, resulting in substantial database storage waste and index bloat. To maintain database stability, it's often necessary to suppress index bloat. However, currently, index bloat is frequently not detected in a timely manner, leading to its continuous expansion.

[0004] Summary of the Invention

[0005] This application provides a method, apparatus, computing device cluster, computer storage medium, and computer product for suppressing index bloat, which can effectively suppress index bloat.

[0006] Firstly, this application provides a method for suppressing index bloat, comprising: determining that an index is bloated; identifying recyclable pages contained in the index; and sequentially adding each recyclable page to a page reclamation queue, wherein recyclable pages include sparse pages or empty pages; and reusing the pages contained in the page reclamation queue sequentially when the index requires new pages. In this way, when an index bloats, adding the recyclable pages in the index to the page reclamation queue sequentially and reusing the pages in the page reclamation queue sequentially achieves the suppression of index bloat.

[0007] In some embodiments, determining that an index has bloated includes: obtaining a first size of the key inserted in the index during an insert operation, a second size of the key deleted in the index during a delete operation, and a third size of the extended pages generated by splitting the index; and determining that the index has bloated based on the first size, the second size, the third size, and the bloated size contained in the statistics in the database. This approach no longer relies solely on statistics as the sole source of bloating information; instead, it adds bloating detection capabilities during index insertion and deletion. Index bloating detection originates from the integration of insertion, deletion, and statistics, ensuring real-time detection while addressing the problem of untimely detection of bloating in statistics.

[0008] In some embodiments, determining that an index has bloated based on a first size, a second size, a third size, and the bloat size contained in the database statistics includes: determining the bloat space size of the index based on the first size, the second size, the third size, and the bloat size; determining the bloat rate of the index based on the bloat space size, the total number of pages in the index, and the size of each page in the index; and determining that the index has bloated if the bloat space of the index is greater than a preset space size and the bloat rate of the index is greater than a preset percentage threshold. Thus, by using the bloat space size and the bloat rate of the index, it is possible to determine whether the index has bloated.

[0009] In some embodiments, the method further includes: the size of the expansion space for the persistent index. This prevents the loss of expansion information in the event of abnormal restarts, which could lead to inaccurate expansion detection.

[0010] In some embodiments, adding any recyclable page to the page reclamation queue includes: updating the maximum commit / modification timestamp to the latest transaction's commit / modification timestamp if the commit / modification timestamp of the latest transaction on any recyclable page is greater than the maximum commit / modification timestamp in the index's metadata; adding a reclamation timestamp to any recyclable page; and updating the reuse timestamp in the metadata to the reclamation timestamp; and adding any recyclable page to the page reclamation queue. In this way, the reclamation of pages in the index is achieved using two timestamps in the metadata.

[0011] In some embodiments, reusing any page in the page recycling queue includes: reusing any page, and updating the reuse timestamp in the metadata to the recycling timestamp of any page. This achieves page reuse.

[0012] In some embodiments, the method further includes allowing access to / modification of the index if the time for accessing / modifying the index is greater than or equal to the maximum commit timestamp, and the time for accessing / modifying a page in the index is greater than or equal to the recycling time. This ensures that index bloat is suppressed and that concurrent business access and index modification do not interfere with or block each other.

[0013] Secondly, this application provides an index bloat suppression device, comprising: a determination module for determining that an index has bloated; a recycling module for identifying recyclable pages contained in the index, and sequentially adding each recyclable page to a page recycling queue, wherein the recyclable pages include sparse pages or empty pages; and a reuse module for sequentially reusing the pages contained in the page recycling queue when the index requires new pages.

[0014] In some embodiments, when the determining module determines that an index has bloated, it is specifically configured to: obtain a first size of the key inserted in the index during an insert operation, a second size of the key deleted in the index during a delete operation, and a third size of the extended pages generated by splitting in the index; and determine that the index has bloated based on the first size, the second size, the third size, and the bloated size contained in the statistics in the database.

[0015] In some embodiments, when the determining module determines that an index has swelled based on the first size, the second size, the third size, and the expansion size contained in the database statistics, it is specifically used to: determine the expansion space size of the index based on the first size, the second size, the third size, and the expansion size; determine the expansion rate of the index based on the expansion space size of the index, the total number of pages in the index, and the size of each page of the index; and determine that the index has swelled when the expansion space of the index is greater than a preset space size and the expansion rate of the index is greater than a preset ratio threshold.

[0016] In some embodiments, the determining module is also used to: determine the expansion space size of the persistent index.

[0017] In some embodiments, when adding any recyclable page to the page recycling queue, the recycling module specifically performs the following: if the commit / modification timestamp of the latest transaction on any recyclable page is greater than the maximum commit / modification timestamp in the index's metadata, update the maximum commit / modification timestamp to the commit / modification timestamp of the latest transaction; add a recycling timestamp to any recyclable page, and update the reuse timestamp in the metadata to the recycling timestamp; and add any recyclable page to the page recycling queue.

[0018] In some embodiments, when the reuse module contains any page in the reuse page recycling queue, it is specifically used to: reuse any page, and update the reuse timestamp in the metadata to the recycling timestamp of any page.

[0019] In some embodiments, the reuse module is further configured to: allow access to / modify the index if the time of accessing / modifying the index is greater than or equal to the maximum commit timestamp, and the time of accessing / modifying a page in the index is greater than or equal to the recycling time.

[0020] Thirdly, this application provides a computing device cluster, including at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the method described in the first aspect or any possible implementation of the first aspect.

[0021] Fourthly, this application provides a computer-readable storage medium including computer program instructions, which, when executed by a cluster of computing devices, perform the method described in the first aspect or any possible implementation thereof. The computing device cluster may include one or more computing devices.

[0022] Fifthly, this application provides a computer program product containing instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method described in the first aspect or any possible implementation thereof. The cluster of computing devices may include one or more computing devices.

[0023] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0024] Figure 1 is a schematic diagram of the logical architecture of a database management system provided in an embodiment of this application;

[0025] Figure 2 is a flowchart illustrating an index inflation suppression method provided in an embodiment of this application;

[0026] Figure 3 is a schematic diagram of the steps for adding a recyclable page to the page recycling queue according to an embodiment of this application;

[0027] Figure 4 is a schematic diagram of the page recycling and reuse process provided in an embodiment of this application;

[0028] Figure 5 is a schematic diagram of an index expansion suppression device provided in an embodiment of this application;

[0029] Figure 6 is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0030] Figure 7 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of this application;

[0031] Figure 8 is a schematic diagram of another computing device cluster provided in an embodiment of this application. Detailed Implementation

[0032] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0033] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0034] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0035] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0036] First, the relevant technical terms involved in the embodiments of this application will be introduced.

[0037] (1) Index bloat

[0038] Index bloat refers to the gradual increase in the size of an index structure over time, causing the storage space occupied by the index to become larger than the actual data stored. Index bloat is usually caused by frequent data deletion and update operations, but index maintenance operations (such as rebuilding the index and cleaning up fragmentation) are not performed in a timely manner, resulting in the index structure becoming less compact and containing a large amount of invalid or redundant data that has not been cleaned up.

[0039] (2) Inflation inhibition

[0040] Inflation suppression refers to taking measures or performing operations (such as reclaiming free space in the index) to limit or slow down the expansion process of the index structure, so that the storage space occupied by the index is close to the space required by the actual data stored in the index.

[0041] Next, the technical solutions provided in the embodiments of this application will be introduced.

[0042] For example, Figure 1 is a schematic diagram of the logical architecture of a database management system provided in an embodiment of this application. As shown in Figure 1, the database management system may include: a client 100 and a database 200. The database 200 may include: an execution engine 210 and a storage engine 220. The client 100 refers to various forms of connecting to the database, such as ActiveX Data Objects (ADO) connection used in .NET, Java Database Connect (JDBC) connection used in Java, etc.

[0043] The execution engine 210 is primarily responsible for generating an efficient execution plan from the Structured Query Language (SQL) statement input by the client 110 under the current load scenario, and then running that execution plan. Additionally, during the process of accessing and modifying indexes in the storage engine 220, the execution engine 210 can also statistically analyze index bloat and / or persist the statistically analyzed index bloat data to the storage engine 220. The execution engine 210 may include: a connector 211, a query cache 212, a parser 213, an optimizer 214, and an executor 215. The connector 211 is primarily responsible for communication with the client 110, and for handling business logic such as connection authentication, connection count determination, and connection pooling. The query cache 212 primarily aims to improve query efficiency; the cache is stored in a key-value hash table, where the key is the specific SQL statement and the value is the set of results. When an SQL statement arrives, if the query caching function is enabled, the execution engine 210 can first check if there is a matching data in the query cache 212. If a match is found, the matching data is directly returned to the client 110 without parsing the corresponding SQL statement. However, if the SQL statement contains user-defined functions, stored functions, user variables, or temporary tables, it will not go through the query cache 212. If no match is found in the query cache 212, the parser 213 will be used to parse the corresponding SQL statement. The parser 213 is mainly responsible for parsing the SQL statement according to syntax rules and generating an internally recognizable parse tree. The optimizer 214 is responsible for optimizing the parse tree generated by the parser 213 to find an optimal execution plan. After the optimizer 214 finds the optimal execution plan, the executor 215 is mainly responsible for calling the interface of the storage engine 220 to execute queries or other operations according to the optimal execution plan, and finally returning the query result set to the client 110.

[0044] Storage engine 220 is primarily responsible for data storage, retrieval, and management. It defines important characteristics such as how the database management system organizes data, executes queries and transactions, and ensures data security and reliability. Additionally, storage engine 220 can also be responsible for storing indexes or managing the size of index expansion spaces.

[0045] For example, Figure 2 shows a schematic flowchart of an index bloat suppression method provided in an embodiment of this application. It is understood that this method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. As shown in Figure 2, the index bloat suppression method may include the following steps:

[0046] S201. Determine that the index has bloated, and add the index to the bloating suppression queue.

[0047] In this embodiment, the presence of index bloat can be determined by analyzing statistical information from the database associated with the index. This statistical information includes the calculated bloat size of the index. When the bloat size exceeds a preset size threshold, index bloat is confirmed. Furthermore, to improve accuracy, the index bloat rate can be incorporated into the determination. When both the bloat size and bloat rate exceed a preset bloat rate threshold, index bloat is confirmed. The bloat rate can be determined by the bloat size, the total number of pages in the index, and the size of each page. Specifically, the bloat rate is calculated as: expand_ration = expand_size ÷ (real_total_pages × pages_size), where expand_size is the bloat size, real_total_pages is the total number of pages, and page_size is the size of each page. After determining that an index is bloated, it can be added to a bloat suppression queue. For example, the bloat suppression queue may contain identifiers of each bloated index. For example, the index may be, but is not limited to, a B-tree or B+tree.

[0048] As one possible implementation, to achieve real-time awareness of index bloat, the following methods can be used: When performing an insert operation, obtain the first size of the inserted key; when performing a delete operation, obtain the second size of the deleted key; and obtain the third size of the expanded pages generated by the index split. Then, based on the first, second, and third sizes and the bloat size in the statistics, determine whether index bloat has occurred. Specifically, an bloat size can be determined first from the first, second, and third sizes. For example, this bloat size could be expanded_size + deleted_size - insert_size, where expanded_size is the third size, deleted_size is the second size, and insert_size is the first size. Then, compare this bloat size with the bloat size in the statistics and select the largest one as the index bloat space size. When this bloat space size exceeds a preset size threshold, index bloat is determined. Alternatively, the index bloat rate can also be introduced during the determination. At this point, the index expansion rate is determined by the index expansion space size, the total number of pages in the index, and the size of each page in the index.

[0049] Furthermore, after determining the size of the index's expansion space, this expansion space size can be persisted to prevent the loss of expansion information in the event of abnormal restarts, which would lead to inaccurate expansion perception.

[0050] S202, Identify at least one recyclable page contained in the index in the page reclamation queue, and sequentially add each recyclable page to the page reclamation queue, wherein the recyclable page includes: sparse page or empty page.

[0051] In this embodiment, recyclable pages contained in the index of the anti-expansion suppression queue can be identified sequentially. During identification, all pages in the index can be traversed. After identifying a recyclable page, the identified recyclable pages in the index can be added to the page recycling queue sequentially. Recyclable pages include sparse pages or empty pages. A sparse page refers to a page where the space occupied by data on the page is less than a preset space threshold. In some embodiments, adding any recyclable page to the page recycling queue may include the following steps: S301, if the commit / modification timestamp of the latest transaction on any recyclable page is greater than the maximum commit / modification timestamp in the index's metadata, update the maximum commit / modification timestamp in the metadata to the commit / modification timestamp of the latest transaction. S302, add a recycling timestamp to the recyclable page, and update the reuse timestamp in the metadata to the recycling timestamp. S303, add the recyclable page to the page recycling queue. For ease of understanding, the following example will illustrate this. Please refer to Figure 4. As shown in Figure 4(A), index a has bloated, with reclaimable pages 1, 3, and 4. The commit time of the latest transaction in page 1 is T1, in page 3 it is T2, and in page 4 it is T3, with T2 < T1 < T3. Additionally, in Figure 4(A), the maximum commit time (Max_commit_time) and reuse time (Reused_time) of the metadata area of ​​index a are initial values. When traversing the reclaimable pages in index a, page 1 is first identified and can be reclaimed. During reclamation, as shown in Figure 4(B), the Max_commit_time in the metadata area of ​​index a is updated to T1, and the Reused_time is updated to the reclamation timet t1 of page 1. Additionally, t1 is added to page 1, and page 1 is added to the page reclamation queue, thus completing the reclamation of page 1. When traversing to page 2, it is identified as not a reclaimable page, and then page 3 is identified. Page 3 is identified as a recyclable page and is reclaimed. During reclamation, as shown in Figure 4(C), since T2 < T1, the Max_commit_time in the metadata area can remain unchanged. Simultaneously, the Reused_time is updated to the reclamation timestamp t2 of page 3, and t2 is added to page 3. Page 3 is then added to the page reclamation queue, thus completing the reclamation of page 3. When traversing to page 4, it is identified as a recyclable page and is reclaimed. During reclamation, as shown in Figure 4(D), since T1 < T3, the Max_commit_time in the metadata area is updated from T1 to T3. Simultaneously, the Reused_time is updated to the reclamation timestamp t3 of page 4, and t3 is added to page 4. Page 4 is then added to the page reclamation queue, thus completing the reclamation of page 4.In some embodiments, access to / modification of the index is permitted if the time for accessing / modifying the index is greater than or equal to the maximum commit timestamp and the time for accessing / modifying a page in the index is greater than or equal to the recycling time. This ensures that index bloat is suppressed and that concurrent business access and index modification do not interfere with or block each other.

[0052] S203. If a new page is needed for indexing, reuse the pages contained in the page recycling queue in sequence.

[0053] In this embodiment, when a new page is needed in the index, pages contained in the page recycling queue can be reused sequentially. This eliminates the need to wait for long transactions or long queries to finish, avoiding the expansion caused by requesting new space. For example, when reusing any page in the page recycling queue, the reuse timestamp in the metadata can be updated to the recycling timestamp of the reused page after reuse. For instance, referring to Figure 4, as shown in Figure 4(E), when reusing page 4, since the Reused_time in the metadata area is the recycling timestamp t3 of page 4, it is not necessary to update the Reused_time. As shown in Figure 4(F), when reusing page 3, t3 in the Reused_time in the metadata area can be updated to the recycling timestamp t2 of page 3. As shown in Figure 4(G), when reusing page 1, t2 in the Reused_time in the metadata area can be updated to the recycling timestamp t1 of page 1.

[0054] In this way, when the index swells, the recyclable pages in the index are added to the page reclamation queue in sequence, and the pages in the page reclamation queue are reused in sequence, thus suppressing the index swell.

[0055] It is understandable that, as shown in Figure 2, each time an index is identified as bloated, the reclaimable pages within that index can be identified. Alternatively, bloated indexes can be added to a queue for index bloat suppression, and then the reclaimable pages within those queues can be identified sequentially. The specific approach depends on the actual situation and is not limited here. Furthermore, the different steps in Figure 2 can be executed, but are not limited to, using different threads, thereby improving the efficiency of index bloat detection and suppression.

[0056] It is understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. In addition, the various embodiments described above can be combined according to actual conditions, and the combined solutions are still within the protection scope of this application.

[0057] Based on the methods in the above embodiments, this application provides an index inflation suppression device.

[0058] For example, Figure 5 shows a schematic diagram of an index bloat suppression device provided in an embodiment of this application. As shown in Figure 5, the index bloat suppression device 500 includes: a determination module 501, a recycling module 502, and a reuse module 503. The determination module 501 is used to determine that index bloat has occurred. The recycling module 502 is used to identify recyclable pages contained in the index and sequentially add each recyclable page to a page recycling queue. Recyclable pages include sparse pages or empty pages. The reuse module 503 is used to sequentially reuse the pages contained in the page recycling queue when the index requires new pages.

[0059] In some embodiments, when determining that an index has bloated, the determining module 501 is specifically configured to: obtain a first size of the key inserted in the index during an insert operation, a second size of the key deleted in the index during a delete operation, and a third size of the extended pages generated by splitting in the index; and determine that the index has bloated based on the first size, the second size, the third size, and the bloated size contained in the statistics in the database.

[0060] In some embodiments, when the determining module 501 determines that an index has swelled based on the first size, the second size, the third size, and the expansion size contained in the database statistics, it is specifically used to: determine the expansion space size of the index based on the first size, the second size, the third size, and the expansion size; determine the expansion rate of the index based on the expansion space size of the index, the total number of pages in the index, and the size of each page of the index; and determine that the index has swelled when the expansion space of the index is greater than a preset space size and the expansion rate of the index is greater than a preset ratio threshold.

[0061] In some embodiments, the determining module 501 is further configured to: determine the expansion space size of the persistent index.

[0062] In some embodiments, when adding any recyclable page to the page recycling queue, the recycling module 502 is specifically configured to: update the maximum commit / modification timestamp to the latest transaction's commit / modification timestamp if the commit / modification timestamp of the latest transaction on any recyclable page is greater than the maximum commit / modification timestamp in the index's metadata; add a recycling timestamp to any recyclable page, and update the reuse timestamp in the metadata to the recycling timestamp; and add any recyclable page to the page recycling queue.

[0063] In some embodiments, when the reuse module 503 contains any page in the reuse page recycling queue, it is specifically used to: reuse any page, and update the reuse timestamp in the metadata to the recycling timestamp of any page.

[0064] In some embodiments, the reuse module 503 is further configured to: allow access to / modify the index when the time of accessing / modifying the index is greater than or equal to the maximum commit timestamp, and the time of accessing / modifying a page in the index is greater than or equal to the recycling time.

[0065] In some embodiments, the determining module 501, recycling module 502, and reuse module 503 shown in FIG. 5 can all be implemented in software or in hardware. For example, the implementation of the determining module 501 will be described below. Similarly, the implementation of the recycling module 502 and the reuse module 503 can refer to the implementation of the determining module 501.

[0066] As an example of a software functional unit, module 501 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the aforementioned computing instance may be one or more. For example, module 501 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed within the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed within the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0067] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0068] As an example of a hardware functional unit, module 501 may include at least one computing device, such as a server. Alternatively, module 501 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0069] The multiple computing devices included in module 501 can be distributed in the same region or in different regions. Similarly, the multiple computing devices included in module 501 can be distributed in the same Availability Zone (AZ) or in different AZs. Likewise, the multiple computing devices included in module 501 can be distributed in the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0070] It should be noted that, in other embodiments, the determining module 501 can be used to execute any step in the index inflation suppression method described in the above embodiments, the recycling module 502 can also be used to execute any step in the index inflation suppression method described in the above embodiments, and the reuse module 503 can also be used to execute any step in the index inflation suppression method described in the above embodiments. The steps implemented by the determining module 501, the recycling module 502, and the reuse module 503 can be specified as needed. By implementing different steps in the index inflation suppression method described in the above embodiments through the determining module 501, the recycling module 502, and the reuse module 503, all the functions of the index inflation suppression device 500 shown in FIG. 5 are realized.

[0071] This application also provides a computing device 600. As shown in FIG6, the computing device 600 includes: a bus 602, a processor 604, a memory 606, and a communication interface 608. The processor 604, the memory 606, and the communication interface 608 communicate with each other via the bus 602. The computing device 600 may be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 600.

[0072] Bus 602 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in Figure 6, but this does not imply that there is only one bus or one type of bus. Bus 604 can include pathways for transmitting information between various components of computing device 600 (e.g., memory 606, processor 604, communication interface 608).

[0073] Processor 604 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0074] The memory 606 may include volatile memory, such as random access memory (RAM). The processor 1004 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0075] The memory 606 stores executable program code, and the processor 604 executes the executable program code to implement the functions of the determination module 501, the recycling module 502, and the reuse module 503 shown in FIG. 5, thereby implementing the index bloat suppression method described in the above embodiments. That is, the memory 606 stores instructions for executing the index bloat suppression method described in the above embodiments.

[0076] Alternatively, the memory 606 stores executable code, which the processor 604 executes to implement the functions of the index inflation suppression device 500 shown in FIG. 5, thereby implementing the index inflation suppression method described in the above embodiments. That is, the memory 606 stores instructions for executing the index inflation suppression method described in the above embodiments.

[0077] The communication interface 603 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 600 and other devices or communication networks.

[0078] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0079] As shown in Figure 7, the computing device cluster includes at least one computing device 600. The memory 606 of one or more computing devices 600 in the computing device cluster may store the same instructions for executing the index bloat suppression method described in the above embodiments.

[0080] In some possible implementations, the memory 606 of one or more computing devices 600 in the computing device cluster may also store partial instructions for executing the index bloat suppression method described in the above embodiments. In other words, a combination of one or more computing devices 600 can jointly execute instructions for executing the index bloat suppression method described in the above embodiments.

[0081] It should be noted that the memory 606 in different computing devices 600 within the computing device cluster can store different instructions, which are used to execute some of the functions of the index inflation suppression device 500 shown in Figure 5. That is, the instructions stored in the memory 606 of different computing devices 600 can implement the functions of one or more modules among the determination module 501, the recycling module 502, and the reuse module 503.

[0082] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 8 illustrates one possible implementation. As shown in Figure 8, two computing devices 600A and 600B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this type of possible implementation, the memory 606 in computing device 600A stores instructions for executing the functions of the determination module 501. Simultaneously, the memory 606 in computing device 600B stores instructions for executing the functions of the recycling module 502 and the reuse module 503.

[0083] It should be understood that the functions of computing device 600A shown in Figure 8 can also be performed by multiple computing devices 600. Similarly, the functions of computing device 600B can also be performed by multiple computing devices 600.

[0084] This application also provides another computing device cluster. The connection relationship between the computing devices in this computing device cluster can be similarly referred to the connection method of the computing device cluster described in Figures 7 and 8. The difference is that the memory 606 of one or more computing devices 600 in this computing device cluster can store the same instructions for executing the methods in the above embodiments.

[0085] In some possible implementations, the memory 606 of one or more computing devices 600 in the computing device cluster may also store partial instructions for executing the aforementioned index bloat suppression method. In other words, a combination of one or more computing devices 600 can jointly execute the instructions for executing the aforementioned index bloat suppression method.

[0086] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program. When the computer program runs on a cluster of computing devices including at least one computing device, it causes the cluster of computing devices to perform the methods described in the above embodiments. Exemplarily, the computer-readable storage medium can be any available medium that the computing device can store, or a data storage device such as a data center containing one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives).

[0087] Based on the methods in the above embodiments, this application provides a computer program product containing instructions that, when run on a computing device cluster including at least one computing device, cause the computing device cluster to execute the methods in the above embodiments.

[0088] It is understood that the processor in the embodiments of this application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor.

[0089] The method steps in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0090] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0091] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A method for suppressing index inflation, characterized in that, include: The index has become bloated; Identify the recyclable pages contained in the index, and sequentially add each of the recyclable pages to the page recycling queue, wherein the recyclable pages include: sparse pages or empty pages; If the index requires a new page, the pages contained in the page recycling queue are reused sequentially.

2. The method according to claim 1, characterized in that, The determination that the index is bloated includes: Get the first size of the key inserted by performing an insert operation in the index, the second size of the key deleted by performing a delete operation in the index, and the third size of the extended pages generated by splitting in the index; Based on the first size, the second size, the third size, and the bloat size contained in the statistics in the database, it is determined that the index has bloated.

3. The method according to claim 2, characterized in that, The step of determining that the index has bloated based on the first size, the second size, the third size, and the bloat size contained in the statistics in the database includes: The expansion space size of the index is determined based on the first size, the second size, the third size, and the expansion size; The expansion rate of the index is determined based on the expansion space size of the index, the total number of pages in the index, and the size of each page in the index; If the expansion space of the index is greater than a preset space size and the expansion rate of the index is greater than a preset ratio threshold, it is determined that the index has expanded.

4. The method according to claim 3, characterized in that, Also includes: Persist the expansion space size of the index.

5. The method according to any one of claims 1-4, characterized in that, Add any recyclable page to the page reclamation queue, including: If the commit / modification timestamp of the latest transaction on any of the reclaimable pages is greater than the maximum commit / modification timestamp in the metadata of the index, the maximum commit / modification timestamp shall be updated to the commit / modification timestamp of the latest transaction. Add a recycling timestamp to any of the recyclable pages, and update the reuse timestamp in the metadata to the recycling timestamp; Add any one of the recyclable pages to the page recycling queue.

6. The method according to claim 5, characterized in that, Reusing any page contained in the page reclamation queue includes: Reuse any one of the pages, and update the reuse timestamp in the metadata to the recycling timestamp of the any one page.

7. The method according to claim 5 or 6, characterized in that, Also includes: Access to / modification of the index is permitted if the time for accessing / modifying the index is greater than or equal to the maximum commit timestamp, and the time for accessing / modifying a page in the index is greater than or equal to the recycling time.

8. An index inflation suppression device, characterized in that, include: The determination module is used to identify index bloat. The recycling module is used to identify recyclable pages contained in the index, and to add each of the recyclable pages to the page recycling queue in sequence. The recyclable pages include sparse pages or empty pages. A reuse module is used to sequentially reuse the pages contained in the page recycling queue when the index requires a new page.

9. The apparatus according to claim 8, characterized in that, When the determining module determines that an index bloat has occurred, it is specifically used for: Get the first size of the key inserted by performing an insert operation in the index, the second size of the key deleted by performing a delete operation in the index, and the third size of the extended pages generated by splitting in the index; Based on the first size, the second size, the third size, and the bloat size contained in the statistics in the database, it is determined that the index has bloated.

10. The apparatus according to claim 9, characterized in that, The determining module determines when the index bloates based on the first size, the second size, the third size, and the bloat size contained in the database statistics. Specifically, it is used to: The expansion space size of the index is determined based on the first size, the second size, the third size, and the expansion size; The expansion rate of the index is determined based on the expansion space size of the index, the total number of pages in the index, and the size of each page in the index; If the expansion space of the index is greater than a preset space size and the expansion rate of the index is greater than a preset ratio threshold, it is determined that the index has expanded.

11. The apparatus according to claim 10, characterized in that, The determining module is further configured to: Persist the expansion space size of the index.

12. The apparatus according to any one of claims 8-11, characterized in that, When adding any recyclable page to the page recycling queue, the recycling module is specifically used for: If the commit / modification timestamp of the latest transaction on any of the reclaimable pages is greater than the maximum commit / modification timestamp in the metadata of the index, the maximum commit / modification timestamp shall be updated to the commit / modification timestamp of the latest transaction. Add a recycling timestamp to any of the recyclable pages, and update the reuse timestamp in the metadata to the recycling timestamp; Add any one of the recyclable pages to the page recycling queue.

13. The apparatus according to claim 12, characterized in that, When reusing any page contained in the page recycling queue, the reuse module is specifically used for: Reuse any one of the pages, and update the reuse timestamp in the metadata to the recycling timestamp of the any one page.

14. The apparatus according to claim 12 or 13, characterized in that, The multiplexing module is also used for: Access to / modification of the index is permitted if the time for accessing / modifying the index is greater than or equal to the maximum commit timestamp, and the time for accessing / modifying a page in the index is greater than or equal to the recycling time.

15. A computing device cluster, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1-7.

16. A computer-readable storage medium storing a computer program that, when run on a cluster of computing devices including at least one computing device, causes the cluster of computing devices to perform the method as described in any one of claims 1-7.

17. A computer program product, characterized in that, When the computer program product is run on a cluster of computing devices including at least one computing device, the cluster of computing devices causes the cluster of computing devices to perform the method as described in any one of claims 1-7.

Images