Lock-free marking method for invalid index, storage medium and computer device

Abstract

The present invention provides a lock-free marking method, storage medium, and computer device for invalid indexes. The method includes prohibiting access to and physical modification of an index page; searching for invalid index entries on the index page and recording the first modification sequence number of the index page; upon completing the search for invalid index entries on the index page, allowing access to and physical modification of the index page; recording the invalid index entries found on the index page; prohibiting access to and physical modification of the index page; recording the second modification sequence number of the index page; determining whether the recorded invalid index entry of the index page is the final invalid index entry based on the first modification sequence number and the second modification sequence number; if so, marking the final invalid index entry as invalid; and allowing access to and physical modification of the index page. During the period of prohibiting access to and physical modification of the index page and allowing access to and physical modification of the index page, the accuracy of marking invalid index entries can be guaranteed.

Description

Technical Field

[0001] The present invention relates to database technology, and in particular to a lock-free marking method for invalid indexes, a storage medium and a computer device. Background Art

[0002] Currently, data in relational databases can be organized as follows: table data is stored in a heap table format; index data is stored separately, and the physical location of the table data row is stored in the index item of the index leaf node to associate the index with the corresponding data. To improve query efficiency, when the table data obtained through the index item is invalid, the invalid index item can be recorded and marked. In this way, subsequent scans will skip the marked index item and avoid retrieving table data through the invalid index item. However, if other operations in the above process affect the physical organization of the index page, the position of the marked invalid index item will change, resulting in inaccurate marking of the invalid index item. If the entire above process prohibits access to and physical modification of the index page, it will block operations that require changing the physical organization of the index page, reducing concurrency and affecting the throughput of the database system. Summary of the Invention

[0003] One object of the present invention is to prohibit access to and physical modification of index pages in some steps of the process of marking invalid index entries, thereby avoiding excessive reduction in concurrency while ensuring the accuracy of marking invalid index entries.

[0004] In particular, the present invention provides a lock-free marking method for invalid indexes, comprising:

[0005] Access to and physical modification of index pages where there are multiple index entries is prohibited;

[0006] Searching for invalid index entries on the index page and recording the first modification sequence number of the index page;

[0007] When the search for invalid index entries on the index page is completed, access and physical modification of the index page are allowed;

[0008] Recording invalid index entries found on the index page;

[0009] Recording the second modification serial number of the index page;

[0010] determining whether the recorded invalid index entry of the index page is a final invalid index entry according to the first modification sequence number and the second modification sequence number;

[0011] If so, mark the final invalid index entry as invalid.

[0012] Optionally, determining whether the recorded invalid index item of the index page is a final invalid index item according to the first modification sequence number and the second modification sequence number includes:

[0013] Determine whether the first modification number is the same as the second modification number.

[0014] Optionally, after marking the final invalid index item as invalid, the method further includes:

[0015] Cleans up final invalid index entries that are marked as invalid.

[0016] Optionally, after marking the final invalid index item as invalid, the method further includes:

[0017] The second modification sequence number of the index page is modified.

[0018] Optionally, searching for invalid index items on the index page includes:

[0019] Invalid index entries on the index page are searched by scanning through the index page.

[0020] Optionally, searching for invalid index items on the index page by scanning and traversing the index page includes:

[0021] The index items of the index page are scanned and traversed, and table data of the index items are obtained, and index items corresponding to invalid table data in the table data are recorded as invalid index items.

[0022] Optionally, marking the final invalid index item as invalid includes:

[0023] The final invalid index items are marked as invalid in batches.

[0024] Optionally, if the first modification sequence number is different from the second modification sequence number, the step of marking the final invalid index item as invalid is abandoned.

[0025] According to another aspect of the present invention, a machine-readable storage medium is provided, on which a machine executable program is stored. When the machine executable program is executed by a processor, it can implement any of the above-mentioned lock-free marking methods for invalid indexes.

[0026] According to another aspect of the present invention, a computer device is provided, comprising a memory, a processor, and a machine executable program stored in the memory and running on the processor, and the processor implements any of the above-mentioned lock-free marking methods for invalid indexes when executing the machine executable program.

[0027] The lock-free marking method for invalid indexes of the present invention prohibits access to and physical modification of index pages, searches for invalid index entries on the index pages, and records the first modification sequence number of the index pages. Once the search for invalid index entries on the index pages is complete, access to and physical modification of the index pages are permitted. During this period, access to and physical modification of the index pages are avoided, preventing or reducing changes in the positions of invalid index entries, thereby ensuring the accuracy of marking invalid index entries. Once the search for invalid index entries on the index pages is complete, considering that some subsequent steps may be time-consuming, such as recording the invalid index entries found on the index pages, access to and physical modification of the index pages are permitted. This avoids reducing database concurrency and impacting the throughput of the database system. Compared to solutions that prohibit access to and physical modification of index pages throughout the entire process, this embodiment relatively improves database concurrency and ensures database system throughput. Furthermore, after recording the invalid index entries found on the index pages, access to and physical modification of the index pages are again prohibited, further ensuring the accuracy of invalid index determination. This embodiment strikes a good balance between ensuring the accuracy of marking invalid index items and ensuring the concurrency of the database, and has outstanding substantive features and significant progress.

[0028] Based on the following detailed description of specific embodiments of the present invention in conjunction with the accompanying drawings, those skilled in the art will become more aware of the above and other objects, advantages and features of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Hereinafter, some specific embodiments of the present invention will be described in detail in an illustrative and non-limiting manner with reference to the accompanying drawings. The same reference numerals in the accompanying drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. In the accompanying drawings:

[0030] Figure 1 is a schematic flowchart of a lock-free marking method for invalid indexes according to an embodiment of the present invention;

[0031] Figure 2 is a schematic diagram of marking invalid index items according to another embodiment of the present invention;

[0032] Figure 3 is a schematic flowchart of a lock-free marking method for invalid indexes according to another embodiment of the present invention;

[0033] Figure 4 is a schematic diagram of a machine-readable storage medium according to one embodiment of the present invention;

[0034] Figure 5 is a schematic diagram of a computer device according to one embodiment of the present invention. DETAILED DESCRIPTION

[0035] Figure 1 FIG. 1 is a schematic flow chart of a lock-free marking method for invalid indexes according to an embodiment of the present invention. The lock-free marking method for invalid indexes may generally include:

[0036] Step S102: prohibiting access to and physical modification of the index page, wherein the index page has multiple index entries. This step can also be understood as controlling physical modification of the index page.

[0037] Step S104: Search for invalid index entries on the index page and record the first modification sequence number of the index page. Modifications that involve changes to the physical organization of the index page generate a unique identifier for the entire database system. This identifier is typically recorded by modifying the location in the log or by modifying the database system's change sequence number. This identifier is recorded on the index page at the same time the modification occurs. The identifier, here, is also the sequence number. The first modification sequence number is the sequence number that indicates that the physical organization of the index page has not changed after access and physical modification to the index page is prohibited.

[0038] Step S106: When the search for invalid index entries on the index page is completed, access to and physical modification of the index page is allowed.

[0039] Step S108: Record the invalid index entries found on the index page. This step may involve random I / O and thus take a long time.

[0040] Step S110: prohibit access to and physical modification of the index page.

[0041] Step S112: Record the second modification serial number of the index page.

[0042] Step S114: determining whether the invalid index entry of the recorded index page is a final invalid index entry according to the first modification sequence number and the second modification sequence number.

[0043] If so, execute step S116: mark the final invalid index item as invalid.

[0044] Step S118: Access to and physical modification of the index page is allowed.

[0045] In this embodiment, access to and physical modification of index pages is prohibited, invalid index entries on the index pages are searched, and the first modification sequence number of the index pages is recorded. Once the search for invalid index entries on the index pages is complete, access to and physical modification of the index pages are permitted. During this period, access to and physical modification of the index pages are avoided, preventing or minimizing changes in the positions of invalid index entries, thereby ensuring the accuracy of invalid index entry marking. Once the search for invalid index entries on the index pages is complete, considering that some subsequent steps may be time-consuming, such as recording the invalid index entries found on the index pages in step S108, access to and physical modification of the index pages are permitted. This avoids reducing database concurrency and impacting the throughput of the database system. Compared to solutions that prohibit access to and physical modification of index pages throughout the entire process, this embodiment significantly improves database concurrency and ensures database system throughput. Furthermore, after recording the invalid index entries found on the index pages in step S108, access to and physical modification of the index pages is again prohibited, further ensuring the accuracy of invalid index determination. This embodiment strikes a good balance between ensuring the accuracy of marking invalid index items and ensuring the concurrency of the database, and has outstanding substantive features and significant progress.

[0046] In one embodiment of the present invention, determining whether the invalid index entry of the recorded index page is a final invalid index entry according to the first modification sequence number and the second modification sequence number may include:

[0047] Determine whether the first modification number is the same as the second modification number.

[0048] In this embodiment, if other operations affect the physical organization of the index page between step S104 and step S112, the position of the marked invalid index item will be changed, that is, the first modification number will be changed, and a different second modification number will be obtained, resulting in inaccurate marking of the invalid index item. Therefore, by determining whether the first modification number is the same as the second modification number, it is possible to accurately determine whether the invalid index item of the recorded index page is the final invalid index item. Specifically, for example, Figure 2This is a schematic diagram of marking invalid index items according to another embodiment of the present invention; assuming that the index page includes index item 1, the first modification number is 1 (that is, value 1); index item 2, the first modification number is 1 (that is, value 1). Index item 2, the first modification number is 1 (that is, value 1) and the corresponding table data is row 2, the first modification number is 1 (that is, A value 1), the second modification number is 3 (that is, B value 3), the first modification number and the second modification number are different, so index item 2 is valid. Index item 1, the first modification number is 1 (that is, value 1) and the corresponding table data is row 1, the first modification number is 1 (that is, A value 1), the second modification number is 1 (that is, B value 1), the first modification number and the second modification number are the same, so index item 1 is invalid.

[0049] In one embodiment of the present invention, after marking the final invalid index item as invalid, the following steps may also be performed:

[0050] Cleans up final invalid index entries that are marked as invalid.

[0051] In this embodiment, after the final invalid index items are marked as invalid, the final invalid index items marked as invalid are cleaned up, which can reduce the space occupied by the invalid final invalid index items and is beneficial to improving the performance of the database.

[0052] In one embodiment of the present invention, after marking the final invalid index item as invalid, the following steps may also be performed:

[0053] Modify the second modification sequence number of the index page.

[0054] In this embodiment, after the final invalid index entry is marked as invalid, the second modification sequence number of the index page is modified to indicate that the index page has completed the marking of the final invalid index entry, thereby avoiding repeated marking of the index page and avoiding repetitive work.

[0055] In one embodiment of the present invention, searching for invalid index entries on an index page may include:

[0056] Search for invalid index entries on index pages by scanning through the index pages.

[0057] In this embodiment, invalid index entries on an index page are searched for by scanning through the index page, which can efficiently search for invalid index entries on the index page and save time.

[0058] In one embodiment of the present invention, searching for invalid index entries on an index page by scanning through the index page may include:

[0059] Scan and traverse the index items of the index page, obtain the table data of the index items, and record the index items corresponding to the invalid table data in the table data as invalid index items.

[0060] In this embodiment, when table data uses multi-version concurrency control and multiple versions of data are mixed and stored, some table data may become invalid. For example, multiple versions of data in the same row also correspond to multiple index items. When the old version of the table data becomes invalid, the corresponding index items are also logically invalid. Therefore, the index items corresponding to the invalid table data in the table data are recorded as invalid index items to ensure the accuracy of the obtained invalid index items.

[0061] In one embodiment of the present invention, marking the final invalid index entry as invalid may include:

[0062] Mark the final invalid index items in batches as invalid.

[0063] In this embodiment, the final invalid index items are marked as invalid in batches, which realizes batch marking, has high marking efficiency, and is very simple and convenient.

[0064] In one embodiment of the present invention, if the first modification sequence number is different from the second modification sequence number, the step of marking the final invalid index entry as invalid is abandoned.

[0065] In this embodiment, if the first modification number is different from the second modification number, the invalid index item has changed. If it is still marked, the marking result may be inaccurate. Therefore, the step of marking the final invalid index item as invalid is abandoned.

[0066] Figure 3 FIG. 1 is a schematic flowchart of a lock-free marking method for invalid indexes according to another embodiment of the present invention, which may include:

[0067] Step S202: prohibiting access to and physical modification of an index page, where the index page has multiple index entries.

[0068] Step S204: Scan and traverse the index items of the index page, and obtain the table data of the index items, record the index items corresponding to the invalid table data in the table data as invalid index items, and record the first modification sequence number of the index page.

[0069] Step S206: When the search for invalid index entries on the index page is completed, access and physical modification of the index page are allowed.

[0070] Step S208: Record invalid index items found on the index page.

[0071] Step S210: prohibit access to and physical modification of the index page.

[0072] Step S212: Record the second modification serial number of the index page.

[0073] Step S214: Determine whether the first modification number is the same as the second modification number.

[0074] If so, execute step S216 and then execute step S220.

[0075] Step S216: Mark the final invalid index entry as invalid.

[0076] Step S218: Modify the second modification sequence number of the index page.

[0077] Step S220: Allow access to and physical modification of the index page.

[0078] Step S222: Abandon marking the final invalid index entry as invalid.

[0079] The above embodiments may be combined in any way. According to any one of the above preferred embodiments or a combination of multiple preferred embodiments, the embodiments of the present invention can achieve the following beneficial effects:

[0080] The lock-free marking method for invalid indexes of the present invention prohibits access to and physical modification of index pages, searches for invalid index entries on the index pages, and records the first modification sequence number of the index pages. When the search for invalid index entries on the index pages is completed, access to and physical modification of the index pages are permitted. During this period, access to and physical modification of the index pages are avoided, preventing or reducing changes in the positions of invalid index entries, thereby ensuring the accuracy of marking invalid index entries. When the search for invalid index entries on the index pages is completed, considering that some subsequent steps may be time-consuming, such as the step of recording the invalid index entries found on the index pages, access to and physical modification of the index pages are permitted. This avoids reducing the concurrency of the database and affecting the throughput of the database system. Compared to a solution that prohibits access to and physical modification of index pages throughout the entire process, this embodiment relatively improves the concurrency of the database and ensures the throughput of the database system. This embodiment strikes a good balance between ensuring the accuracy of marking invalid index entries and ensuring the concurrency of the database, and has outstanding substantive features and significant improvements.

[0081] This embodiment also provides a machine-readable storage medium and a computer device. Figure 4 is a schematic diagram of a machine-readable storage medium 830 according to one embodiment of the present invention; Figure 5 8 is a schematic diagram of a computer device 900 according to an embodiment of the present invention. A machine-readable storage medium 830 stores a machine executable program 840 thereon. When the machine executable program 840 is executed by a processor, the lock-free marking method for invalid indexes according to any of the above embodiments is implemented.

[0082] The computer device 900 may include a memory 920, a processor 910, and a machine executable program 840 stored in the memory 920 and running on the processor 910. When the processor 910 executes the machine executable program 840, the lock-free marking method for invalid indexes according to any of the above embodiments is implemented.

[0083] It should be noted that the logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any machine-readable storage medium for use by an instruction execution system, device or equipment (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, device or equipment and execute instructions), or used in combination with these instruction execution systems, devices or equipment.

[0084] For the purposes of the description of this embodiment, the machine-readable storage medium 830 can be any device that can contain, store, communicate, propagate, or transmit a program for use with an instruction execution system, device, or apparatus or in conjunction with such instruction execution system, device, or apparatus. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection having one or more wirings (electronic device), a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium 40 can even be paper or other suitable medium on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting, or processing in other suitable ways as necessary, and then stored in a computer memory.

[0085] It should be understood that each part of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system.

[0086] The computer device 900 can be, for example, a server, a desktop computer, a laptop computer, a tablet computer, or a smartphone. In some examples, the computer device 900 can be a cloud computing node. The computer device 900 can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc. that perform specific tasks or implement specific abstract data types. The computer device 900 can be implemented in a distributed cloud computing environment where remote processing devices linked via a communication network perform tasks. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0087] The computer device 900 may include a processor 910 adapted to execute stored instructions, and a memory 920 that provides temporary storage for the instructions during operation. The processor 910 may be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. The memory 920 may include random access memory (RAM), read-only memory, flash memory, or any other suitable storage system.

[0088] The processor 910 can be connected to an I / O interface (input / output interface) suitable for connecting the computer device 900 to one or more I / O devices (input / output devices) via a system interconnect (e.g., PCI, PCI-Express, etc.). The I / O devices may include, for example, a keyboard and a pointing device, wherein the pointing device may include a touchpad or a touch screen, etc. The I / O devices may be built-in components of the computer device 900, or may be devices externally connected to the computing device.

[0089] Processor 910 can also be linked to the display interface that is suitable for connecting computer device 900 to display device by system interconnection.Display device can comprise the display screen that is built-in component of computer device 900.Display device can also comprise the computer monitor, television or projector etc. that are externally connected to computer device 900.In addition, network interface controller (network interface controller, NIC) can be suitable for connecting computer device 900 to network by system interconnection.In certain embodiments, NIC can use any suitable interface or protocol (such as Internet Small Computer System Interface etc.) to transmit data.Network can be cellular network, radio network, wide area network (WAN)), local area network (LAN) or Internet etc.Remote device can be connected to computing device by network.

[0090] The flowchart provided in this embodiment is not intended to indicate that the operations of the method will be performed in any particular order, or that all operations of the method are included in all every case. In addition, the method may include additional operations. Within the scope of the technical ideas provided by the method of this embodiment, additional changes can be made to the above method.

[0091] At this point, those skilled in the art will recognize that, although a number of exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications consistent with the principles of the present invention may be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the present invention. Therefore, the scope of the present invention should be understood and deemed to cover all such other variations or modifications.

Claims

1. A lock-free marking method for invalid indexes, comprising: Access to and physical modification of index pages where there are multiple index entries is prohibited; Searching for invalid index entries on the index page and recording the first modification sequence number of the index page; When the search for invalid index entries on the index page is completed, access and physical modification of the index page are allowed; Recording invalid index entries found on the index page; Access and physical modification of index pages are prohibited; Recording the second modification serial number of the index page; determining whether the recorded invalid index entry of the index page is a final invalid index entry according to the first modification sequence number and the second modification sequence number; If so, mark the final invalid index item as invalid; Access and physical modification of the index pages are allowed.

2. The lock-free marking method for invalid indexes according to claim 1, wherein: The determining, according to the first modification sequence number and the second modification sequence number, whether the invalid index entry of the recorded index page is a final invalid index entry comprises: Determine whether the first modification number is the same as the second modification number.

3. The lock-free marking method for invalid indexes according to claim 1, wherein: After marking the final invalid index item as invalid, the method further includes: Cleans up final invalid index entries that are marked as invalid.

4. The lock-free marking method for invalid indexes according to claim 1, wherein: After marking the final invalid index item as invalid, the method further includes: The second modification sequence number of the index page is modified.

5. The lock-free marking method for invalid indexes according to claim 1, wherein: The searching for invalid index items on the index page includes: Invalid index entries on the index page are searched by scanning through the index page.

6. The lock-free marking method for invalid indexes according to claim 5, wherein: The searching for invalid index entries on the index page by scanning through the index page includes: The index items of the index page are scanned and traversed, and table data of the index items are obtained, and index items corresponding to invalid table data in the table data are recorded as invalid index items.

7. The lock-free marking method for invalid indexes according to claim 1, wherein: Marking the final invalid index item as invalid includes: The final invalid index items are marked as invalid in batches.

8. The lock-free marking method for invalid indexes according to claim 1, wherein: If the first modification sequence number is different from the second modification sequence number, abandon the step of marking the final invalid index item as invalid.

9. A machine-readable storage medium having a machine-executable program stored thereon, wherein when the machine-executable program is executed by a processor, the lock-free marking method for invalid indexes according to any one of claims 1 to 8 is implemented.

10. A computer device comprising a memory, a processor, and a machine executable program stored in the memory and running on the processor, wherein the processor implements the lock-free marking method for invalid indexes according to any one of claims 1 to 8 when executing the machine executable program.

Images