A method for implementing parallel bitmap heap scanning on openGauss database

Abstract

The present invention relates to a method for implementing parallel bitmap heap scanning on an OpenGauss database. The method comprises the steps of generating a parallel bitmap heap scanning plan, initializing the parallel bitmap heap scanning plan, and executing the parallel bitmap heap scanning plan. This method implements parallel bitmap heap scanning based on an existing bitmap heap scanning operator on the OpenGauss database. By adjusting the number of threads used during the bitmap heap scanning operation and enabling multiple threads to execute the bitmap heap scan in parallel, the execution time of the bitmap heap scanning operator is reduced and the efficiency of the scan execution is improved. Furthermore, during the parallel execution of the bitmap heap scan, the method ensures the correctness of the parallel execution through the use of mutual exclusion locks.

Description

Technical Field

[0001] The present invention belongs to the technical field of database parallel query, and in particular relates to a method and system for implementing parallel bitmap heap scanning on an openGauss database. Background Art

[0002] The existing bitmap heap scan method is as follows: a thread is launched to scan index records based on the query conditions, sorting the involved data blocks in ascending order by number, and then accessing the data in the heap table one by one. However, when there are many data blocks involved, since a thread can only execute serially, the scan time will be too long, affecting service throughput.

[0003] Therefore, how to improve the efficiency of bitmap heap scanning has become a technical problem that needs to be solved urgently. Summary of the Invention

[0004] In order to overcome the shortcomings of the existing technology and improve the efficiency of bitmap heap scanning, the present invention proposes a new method for implementing parallel bitmap heap scanning on the openGauss database. This method can significantly improve the execution efficiency of bitmap heap scanning.

[0005] Explanation of terms

[0006] Transaction: A database transaction is a mechanism, a sequence of operations, encompassing a set of database operation commands. A transaction submits or cancels all commands as a whole to the system. This means that either all or none of the commands in the set are executed, making it an indivisible logical unit of work.

[0007] Snapshot: Snapshot is a technology used to implement transaction isolation and concurrency control in databases. A snapshot records the state of the database at a certain point in time during runtime. The snapshot can be used to determine the visibility of data in a transaction.

[0008] TID: tuple identifier, the location information of a tuple in a heap table data block. In openGauss, it consists of a block number and an offset within the block.

[0009] Index: Index is auxiliary data used to improve the efficiency of data retrieval in a database. There are two basic types of indexes: sequential index and hash index. This invention only involves sequential index.

[0010] Bitmap Heap Scan: A bitmap heap scan is a physical operator in the openGauss database. A traditional index scan (IndexScan) operator reads data from the heap table based on the TID for each index entry. When the index selectivity is high and the data in the heap table is distributed discretely, index scans can result in significant random I / O. A bitmap heap scan first retrieves and sorts all the heap table data blocks to be scanned from the index, then sequentially reads the tuples in the heap table, avoiding this random I / O issue.

[0011] In a first aspect, the present invention provides a method for implementing a parallel bitmap heap scan (Parallel BitmapHeapScan) on an openGauss database, the method comprising:

[0012] S1. Generate a parallel bitmap heap scan plan;

[0013] S2. Initialize the parallel bitmap heap scan plan;

[0014] S3. Execute the parallel bitmap heap scan plan.

[0015] Furthermore, the generating of the parallel bitmap heap scan plan in step S1 of the method of the present invention includes:

[0016] S101. Determine whether the parallel query function is enabled in the database;

[0017] S102. Determine whether the query table and query conditions on the table support parallel query;

[0018] S103. Generate a parallel bitmap heap scan path for the corresponding table;

[0019] S104 . Generate a parallel bitmap heap scan plan according to the parallel bitmap heap scan path.

[0020] Furthermore, the initialization of the parallel bitmap heap scan plan in step S2 of the method of the present invention includes:

[0021] S201. Initialize the parameter information required for parallel bitmap heap scanning;

[0022] S202. Register the background thread to be started, set this thread as the leader thread, and set the remaining threads as worker threads;

[0023] S203. Start the worker thread and initialize it.

[0024] Furthermore, the parameter information described in step S201 of the above method includes memory, communication queue, plan statement, execution parameters and snapshot information.

[0025] Preferably, the number of worker threads in step S202 of the above method does not exceed a threshold set by the system.

[0026] Furthermore, the execution of the parallel bitmap heap scan plan in step S3 of the method of the present invention includes:

[0027] S301. The leader thread reads the tuples returned by the worker threads from the communication queue. If no tuples are returned by any worker threads and the worker threads have not completed processing, the leader thread goes into sleep mode.

[0028] S302. The first worker thread that performs the bitmap heap scan is responsible for reading the location information TIDs of all tuples to be scanned from the index, while other worker threads wait;

[0029] S303. Each worker thread reads one block in TIDs at a time in ascending order of block numbers;

[0030] S304. Each worker thread performs a bitmap heap scan based on the read block, and writes the scan results into the communication queue. After all TIDs are scanned, the entire parallel bitmap heap scan process is completed.

[0031] Preferably, each worker thread in step S303 of the above method reads one block from TIDs at a time in ascending order of block numbers, and a mutex is added when the worker thread accesses the TIDs.

[0032] In a second aspect, the present invention provides a system for implementing parallel bitmap heap scanning on an openGauss database, the system comprising:

[0033] Scan plan generation module: used to generate parallel bitmap heap scan plans;

[0034] Scan plan initialization module: used to initialize the parallel bitmap heap scan plan;

[0035] Scan plan execution module: used to execute parallel bitmap heap scan plans.

[0036] Each module is implemented and operated according to the above method for implementing parallel bitmap heap scanning on the openGauss database.

[0037] Finally, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned method for implementing parallel bitmap heap scanning on an openGauss database.

[0038] In summary, the method for implementing parallel bitmap heap scanning on the openGauss database of the present invention has the following advantages:

[0039] (1) The method of the present invention improves the scanning execution efficiency by starting multiple threads to execute bitmap heap scanning in parallel.

[0040] (2) The method of the present invention ensures the correctness of parallel execution through mutual exclusion locks during the parallel execution of bitmap heap scanning.

[0041] (3) The embodiment of the present invention shows a method for implementing parallel bitmap heap scanning based on the existing bitmap heap scanning operator on the openGauss database. By adjusting the number of threads during bitmap heap scanning, the execution time of the bitmap heap scanning operator is reduced and the execution efficiency of the operator is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the following drawings are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0043] Figure 1 The figure is a flow chart of the overall implementation of the method of the present invention.

[0044] Figure 2 4 is an implementation flow chart of step S1 in the method of the present invention.

[0045] Figure 3 4 is an implementation flow chart of step S2 in the method of the present invention.

[0046] Figure 4 4 is an implementation flow chart of step S3 in the method of the present invention.

[0047] Figure 5 It is a structural diagram of the system of the present invention. DETAILED DESCRIPTION

[0048] To make the objectives, technical solutions, and advantages of the present invention more clearly apparent, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The present invention may also be implemented or applied through different specific implementation methods, and the details in this specification may be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention.

[0049] At the same time, it should be understood that the scope of protection of the present invention is not limited to the specific embodiments described below; it should also be understood that the terms used in the embodiments of the present invention are for describing specific embodiments rather than for limiting the scope of protection of the present invention.

[0050] Example 1: A method for implementing parallel bitmap heap scanning on an openGauss database (eg Figure 1 The method comprises the following steps:

[0051] (1) Generate a parallel bitmap heap scan plan (such as Figure 2 shown)

[0052] S101. Determine whether the parallel query function is enabled in the database;

[0053] S102. Determine whether the query table and query conditions on the table support parallel query;

[0054] S103. Generate a parallel bitmap heap scan path for the corresponding table;

[0055] S104 . Generate a parallel bitmap heap scan plan according to the parallel bitmap heap scan path.

[0056] (2) Initialize the parallel bitmap heap scan plan (such as Figure 3 shown)

[0057] S201. Initialize the memory, communication queue, plan statement, execution parameters and snapshot information required for parallel bitmap heap scanning;

[0058] S202. Register the background thread to be started, set this thread as the leader thread, and set the remaining threads as worker threads. The number of worker threads does not exceed the threshold set by the system;

[0059] S203. Start the worker thread and initialize it.

[0060] (3) Execute the parallel bitmap heap scan plan (such as Figure 4 shown)

[0061] S301. The leader thread reads the tuples returned by the worker threads from the communication queue. If no tuples are returned by any worker threads and the worker threads have not completed processing, the leader thread goes into sleep mode.

[0062] S302. The first worker thread that performs the bitmap heap scan is responsible for reading the location information TIDs of all tuples to be scanned from the index, while other worker threads wait;

[0063] S303. Each worker thread reads a block in TIDs in ascending order of block number. To ensure the correctness of parallel execution, a mutex is added when the worker thread accesses TIDs.

[0064] S304. Each worker thread performs a bitmap heap scan based on the read block, and writes the scan results into the communication queue until all TIDs are scanned. The entire parallel bitmap heap scan process is completed. Example 2: A system for implementing parallel bitmap heap scanning on an openGauss database (e.g. Figure 5 As shown), this system includes:

[0065] Scan plan generation module: used to generate parallel bitmap heap scan plans;

[0066] Scan plan initialization module: used to initialize the parallel bitmap heap scan plan;

[0067] Scan plan execution module: used to execute parallel bitmap heap scan plan;

[0068] Each module is implemented and operated according to the method for implementing parallel bitmap heap scanning on the openGauss database described in Example 1.

[0069] The foregoing is merely an embodiment of the present invention and is not intended to limit the present invention. It will be apparent to those skilled in the art that various modifications and variations of the present invention are possible. Any modifications, substitutions, etc. made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. A method for implementing parallel bitmap heap scanning on an openGauss database, characterized in that: The method comprises: S1. Generate a parallel bitmap heap scan plan. S2. Initialize the parallel bitmap heap scan plan; S3. Execute a parallel bitmap heap scan plan, including: S301. The leader thread reads the tuple returned by the worker thread from the communication queue. If no tuple is returned by any worker thread and the worker thread has not completed processing, it sleeps and waits. S302. The first worker thread that performs a bitmap heap scan is responsible for reading all tuple location information TIDs that need to be scanned from the index, while other worker threads wait. S303. Each worker thread reads a block in TIDs in ascending order of block number each time, and adds a mutex lock when the worker thread accesses TIDs; S304. Each worker thread performs a bitmap heap scan based on the read block, and writes the scan results to the communication queue. After all TIDs are scanned, the entire parallel bitmap heap scan process is completed.

2. The method for implementing parallel bitmap heap scanning on an openGauss database according to claim 1, characterized in that: The generation of a parallel bitmap heap scan plan in step S1 includes: S101 determines whether the database parallel query function is enabled; S102 determines whether the query table and the query conditions on the table involved support parallel query; S103 generates a parallel bitmap heap scan path corresponding to the table; S104. Generate a parallel bitmap heap scan plan according to the parallel bitmap heap scan path.

3. The method for implementing parallel bitmap heap scanning on an openGauss database according to claim 2, characterized in that: The initialization parallel bitmap heap scan plan described in step S2 includes: S201 initializes the parameter information required for parallel bitmap heap scanning; S202 registers the background thread to be started, sets this thread as the leader thread, and sets the remaining threads as worker threads; S203. Start and initialize the worker thread.

4. The method for implementing parallel bitmap heap scanning on an openGauss database according to claim 3, characterized in that: The parameter information in step S201 includes memory, communication queue, plan statement, execution parameters and snapshot information.

5. The method for implementing parallel bitmap heap scanning on an openGauss database according to claim 3, characterized in that: The number of worker threads in step S202 does not exceed a threshold set by the system.

6. A system for implementing parallel bitmap heap scanning on an openGauss database, characterized in that: The system comprises: Scan plan generation module: used to generate parallel bitmap heap scan plans; Scan plan initialization module: used to initialize the parallel bitmap heap scan plan; Scan plan execution module: used to execute parallel bitmap heap scan plans, including: (1) The leader thread reads the tuples returned by the worker threads from the communication queue. If no tuples are returned by any worker threads and the worker threads have not completed the processing, the leader thread goes into sleep mode. (2) The first worker thread that performs the bitmap heap scan is responsible for reading the location information TIDs of all tuples that need to be scanned from the index, while other worker threads wait; (3) Each worker thread reads one block from TIDs at a time in ascending order of block numbers, and adds a mutex lock when the worker thread accesses TIDs; (4) Each worker thread performs a bitmap heap scan based on the read block, and the scan results are written to the communication queue until all TIDs are scanned. The entire parallel bitmap heap scan process is completed.

7. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the program implements the steps of the method for implementing parallel bitmap heap scanning on an openGauss database according to any one of claims 1 to 5.

Images