Optimize the marking method, storage medium and device for invalid indexes caused by concurrency conflicts

Abstract

The present invention provides a marking method, storage medium, and device for optimizing concurrent conflicts in invalid indexes. The method includes allowing access to an index page and prohibiting physical modification of the index page; searching for invalid index items on the index page and recording the first modification number of the index page; when the search for invalid index items on the index page is completed, allowing physical modification of the index page; recording the invalid index items found on the index page, and prohibiting physical modification of the index page after the recording is completed; recording the second modification number of the index page; determining whether the invalid index item of the recorded index page is the final invalid index item based on the first modification number and the second modification number; if so, marking the final invalid index item as invalid, and allowing physical modification of the index page. The technical solution of the present invention can increase the throughput of the data system while ensuring the accuracy of invalid index item marking.

Description

Technical Field

[0001] The present invention relates to database technology, in particular to a marking method, storage medium and device for optimizing invalid indexes of concurrent conflicts. Background Art

[0002] Currently, data in relational databases can be organized as follows: table data is stored in a heap table format; index data is stored separately, and the physical location of the table data row is stored in the index item of the index leaf node to associate the index with the corresponding data. To improve query efficiency, when the table data obtained through the index item is invalid, the invalid index item can be recorded and marked. In this way, subsequent scans will skip the marked index item and avoid retrieving table data through the invalid index item. However, if other operations in the above process affect the physical organization of the index page, the position of the marked invalid index item will change, resulting in inaccurate marking of the invalid index item. If the entire above process prohibits access to and physical modification of the index page, it will block operations that require changing the physical organization of the index page, reducing concurrency and affecting the throughput of the database system. Summary of the Invention

[0003] An object of the present invention is to enable access to index pages and prohibit physical modification of index pages in some steps of the process of marking invalid indexes, thereby increasing the throughput of the data system while ensuring the accuracy of marking invalid index entries.

[0004] In particular, the present invention provides a method for marking invalid indexes that optimize concurrency conflicts, comprising:

[0005] allowing access to an index page and disallowing physical modification of the index page, wherein the index page has a plurality of index entries;

[0006] Searching for invalid index entries on the index page and recording the first modification sequence number of the index page;

[0007] When the search for invalid index entries on the index page is completed, physical modification of the index page is allowed;

[0008] Recording invalid index entries found on the index page, and prohibiting physical modification of the index page after recording is completed;

[0009] Recording the second modification serial number of the index page;

[0010] determining whether the recorded invalid index entry of the index page is a final invalid index entry according to the first modification sequence number and the second modification sequence number;

[0011] If so, the final invalid index entry is marked as invalid, and physical modification of the index page is allowed.

[0012] Optionally, recording the first modification number of the index page includes:

[0013] Record the first modification sequence number through an atomic operation; and / or

[0014] The recording of the second modification serial number of the index page includes:

[0015] The second modification number is recorded only through atoms.

[0016] Optionally, determining whether the recorded invalid index item of the index page is a final invalid index item according to the first modification serial number and the second modification serial number includes:

[0017] Determine whether the first modification number is the same as the second modification number.

[0018] Optionally, after marking the final invalid index item as invalid, the method further includes:

[0019] The second modification sequence number of the index page is modified using an atomic operation.

[0020] Optionally, searching for invalid index items on the index page includes:

[0021] Invalid index entries on the index page are searched by scanning through the index page.

[0022] Optionally, searching for invalid index items on the index page by scanning and traversing the index page includes:

[0023] The index items of the index page are scanned and traversed, and table data of the index items are obtained, and index items corresponding to invalid table data in the table data are recorded as invalid index items.

[0024] Optionally, marking the final invalid index item as invalid includes:

[0025] The final invalid index items are marked as invalid in batches.

[0026] Optionally, if the first modification sequence number is different from the second modification sequence number, the step of marking the final invalid index item as invalid is abandoned.

[0027] According to another aspect of the present invention, a machine-readable storage medium is provided, on which a machine executable program is stored. When the machine executable program is executed by a processor, it can implement any of the above-mentioned methods for marking invalid indexes that optimize concurrency conflicts.

[0028] According to another aspect of the present invention, a computer device is also provided, comprising a memory, a processor, and a machine executable program stored in the memory and running on the processor, and the processor can implement any of the above-mentioned methods for marking invalid indexes that optimize concurrency conflicts when executing the machine executable program.

[0029] The invalid index marking method provided by the present invention allows access to the index page and prohibits physical modification of the index page, searches for invalid index items on the index page, and records the first modification number of the index page through atomic operations; when the search for invalid index items on the search page is completed, physical modification of the index page is allowed; after the invalid index items are searched for on the index page, physical modification of the index page is prohibited again, and the second modification number of the index page is recorded through atomic operations; finally, the invalid index items of the index page are determined based on the above-mentioned first modification number and second modification number.

[0030] The invalid index marking method provided by the present invention has some steps that allow access to index pages and prohibit physical modification of index pages. Enabling access to index pages can increase the throughput of the database system, and prohibiting physical modification of index pages can reduce or even prevent changes in the positions of invalid index items when searching for invalid index items on the index page, and when recording the first modification sequence number and the second modification sequence number of the index page, thereby ensuring the accuracy of marking invalid index items on the index page. Furthermore, when the search for invalid index items on the index page is completed, considering that some subsequent steps may be time-consuming, such as the step of recording the invalid index items found on the index page, access to and physical modification of the index page are allowed. Compared to a solution that prohibits access to and physical modification of index pages throughout the entire process, the technical solution of the present invention prohibits physical modification of index pages in only some steps and does not prohibit access to index pages. Therefore, it can minimize the reduction in database concurrency and avoid affecting the throughput of the database system.

[0031] Furthermore, in the invalid index marking scheme of the present invention, the steps between recording invalid index entries found on the index page and marking the final invalid index entry as invalid are relatively short. Therefore, access to the index page and physical modification of the index page are prohibited between these steps. This ensures the accuracy of marking invalid index entries while also avoiding or reducing the impact on database concurrency and thus the throughput of the database system. The technical scheme of the present invention, because it strikes a good balance between ensuring the accuracy of marking invalid index entries and maintaining database concurrency, possesses outstanding substantive features and represents a significant advancement.

[0032] Based on the following detailed description of specific embodiments of the present invention in conjunction with the accompanying drawings, those skilled in the art will become more aware of the above and other objects, advantages and features of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Hereinafter, some specific embodiments of the present invention will be described in detail in an exemplary and non-limiting manner with reference to the accompanying drawings. The same reference numerals in the accompanying drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. In the accompanying drawings:

[0034] Figure 1 is a schematic flow chart of a method for marking invalid indexes for optimizing concurrency conflicts according to an embodiment of the present invention;

[0035] Figure 2 is a schematic diagram of marking invalid index items according to another embodiment of the present invention;

[0036] Figure 3 is a schematic flow chart of a method for marking invalid indexes for optimizing concurrency conflicts according to another embodiment of the present invention;

[0037] Figure 4 is a schematic diagram of a machine-readable storage medium according to one embodiment of the present invention;

[0038] Figure 5 is a schematic diagram of a computer device according to one embodiment of the present invention. DETAILED DESCRIPTION

[0039] Figure 1 FIG. 1 is a schematic flow chart of a method for marking invalid indexes for optimizing concurrency conflicts according to an embodiment of the present invention. The invalid index marking method may generally include:

[0040] Step S102: Allow access to the index page and prohibit physical modification of the index page, wherein the index page has multiple index items. This step can also be understood as controlling the physical modification of the index page.

[0041] Step S104: Search for invalid index entries on the index page and record the first modification sequence number of the index page. Modifications that involve changes to the physical organization of the index page generate a unique identifier for the entire database system. This identifier is typically recorded by modifying the location in the log or by modifying the database system's change sequence number. This identifier is recorded on the index page at the same time the modification occurs. The identifier, here, is also the sequence number. The first modification sequence number is the sequence number that indicates that the physical organization of the index page has not changed after access and physical modification to the index page is prohibited.

[0042] Step S106: When the search for invalid index entries on the index page is completed, access to and physical modification of the index page is allowed.

[0043] Step S108: Record the invalid index entries found on the index page, and allow access to the index page again and prohibit physical modification of the index page. Recording the invalid index entries found on the index page in this step may involve random I / O and is therefore time-consuming.

[0044] Step S110: Record the second modification serial number of the index page.

[0045] Step S112: determining whether the invalid index entry of the recorded index page is a final invalid index entry according to the first modification sequence number and the second modification sequence number.

[0046] If so, step S114 is executed: the final invalid index entry is marked as invalid, and physical modification of the index page is allowed.

[0047] In this embodiment, access to and physical modification of index pages are permitted, invalid index entries on the index pages are searched for, and the first modification sequence number of the index pages is recorded. When the search for invalid index entries on the index pages is completed, access to and physical modification of the index pages are permitted. After recording the invalid index entries found on the index pages, physical modification of the index pages is prohibited, and the second modification sequence number of the index pages is recorded. Because the position changes of invalid index entries can be reduced or even avoided when physical modification of the index pages is prohibited, the accuracy of the marking of invalid index entries can be guaranteed. When the search for invalid index entries on the index pages is completed, considering that some subsequent steps may be time-consuming, such as recording the invalid index entries found on the index pages in step S108, access to and physical modification of the index pages are permitted to avoid reducing the concurrency of the database and affecting the throughput of the database system.

[0048] Compared to solutions that completely prohibit access to and physical modification of index pages throughout the entire process, the technical solution of the present invention prohibits physical modification of index pages in only some steps and does not prohibit access to index pages. This minimizes the reduction of database concurrency and the impact on database system throughput. Because this embodiment strikes a good balance between ensuring the accuracy of marking invalid index entries and maintaining database concurrency, it possesses outstanding substantive features and represents a significant advancement.

[0049] In one embodiment of the present invention, the method for recording the first modification sequence number of the index page may include:

[0050] Record the first modification sequence number of the index page using an atomic operation; and / or

[0051] The method for recording the second modification sequence number of the index page may include:

[0052] Use atomic operations to record the second modification sequence number of the index page.

[0053] In this embodiment, atomic operations are used when recording the first and second modification numbers of index pages. Atomic operations are another method for providing concurrent access control. Atomic operations are operations that maintain atomicity during execution. Furthermore, atomic operations do not require locking during execution, achieving lock-free operation. This ensures concurrency control while reducing the impact on system concurrency performance. Therefore, the technical solution of this embodiment can reduce or even avoid concurrency conflicts that arise during the process of recording the first and second modification numbers of a page, thereby further improving the accuracy of invalid index marking.

[0054] In one embodiment of the present invention, determining whether the invalid index entry of the recorded index page is a final invalid index entry according to the first modification sequence number and the second modification sequence number may include:

[0055] Determine whether the first modification number is the same as the second modification number.

[0056] In this embodiment, if other operations affect the physical organization of the index page between step S104 and step S110, the position of the marked invalid index item will be changed, that is, the first modification number will be changed, and a different second modification number will be obtained, resulting in inaccurate marking of the invalid index item. Therefore, by determining whether the first modification number is the same as the second modification number, it is possible to accurately determine whether the invalid index item of the recorded index page is the final invalid index item. Specifically, for example, Figure 2 This is a schematic diagram of marking invalid index items according to another embodiment of the present invention; assuming that the index page includes index item 1, the first modification number is 1 (that is, value 1); index item 2, the first modification number is 1 (that is, value 1). Index item 2, the first modification number is 1 (that is, value 1) and the corresponding table data is row 2, the first modification number is 1 (that is, A value 1), the second modification number is 3 (that is, B value 3), the first modification number and the second modification number are different, so index item 2 is valid. Index item 1, the first modification number is 1 (that is, value 1) and the corresponding table data is row 1, the first modification number is 1 (that is, A value 1), the second modification number is 1 (that is, B value 1), the first modification number and the second modification number are the same, so index item 1 is invalid.

[0057] In one embodiment of the present invention, after marking the final invalid index item as invalid, the following steps may also be performed:

[0058] Use atomic operations to modify the second modification sequence number of the index page.

[0059] In this embodiment, after the final invalid index entry is marked as invalid, the second modification sequence number of the index page is modified to indicate that the index page has completed the marking of the final invalid index entry, thereby avoiding repeated marking of the index page and avoiding repetitive work.

[0060] Furthermore, by using atomic operations to modify the second modification number of the index page, concurrent conflicts when modifying the second modification number can be reduced or avoided, thereby ensuring the reliability of modifying the second modification number and improving the accuracy of marking invalid index entries in the index page.

[0061] In one embodiment of the present invention, searching for invalid index entries on an index page may include:

[0062] Search for invalid index entries on index pages by scanning through the index pages.

[0063] In this embodiment, invalid index entries on an index page are searched for by scanning through the index page, which can efficiently search for invalid index entries on the index page and save time.

[0064] In one embodiment of the present invention, searching for invalid index entries on an index page by scanning through the index page may include:

[0065] Scan and traverse the index items of the index page, obtain the table data of the index items, and record the index items corresponding to the invalid table data in the table data as invalid index items.

[0066] In this embodiment, when table data uses multi-version concurrency control and multiple versions of data are mixed and stored, some table data may become invalid. For example, multiple versions of data in the same row also correspond to multiple index items. When the old version of the table data becomes invalid, the corresponding index items are also logically invalid. Therefore, the index items corresponding to the invalid table data in the table data are recorded as invalid index items to ensure the accuracy of the obtained invalid index items.

[0067] In one embodiment of the present invention, marking the final invalid index entry as invalid may include:

[0068] Mark the final invalid index items in batches as invalid.

[0069] In this embodiment, the final invalid index items are marked as invalid in batches, which realizes batch marking, has high marking efficiency, and is very simple and convenient.

[0070] In one embodiment of the present invention, if the first modification sequence number is different from the second modification sequence number, the step of marking the final invalid index entry as invalid is abandoned.

[0071] In this embodiment, if the first modification number is different from the second modification number, the invalid index item has changed. If it is still marked, the marking result may be inaccurate. Therefore, the step of marking the final invalid index item as invalid is abandoned.

[0072] Figure 3 FIG. 1 is a schematic flow chart of a method for marking invalid indexes according to another embodiment of the present invention, which may include:

[0073] Step S202: allowing access to the index page and prohibiting physical modification of the index page, wherein the index page has multiple index items.

[0074] Step S204: Scan and traverse the index items of the index page, obtain the table data of the index items, record the index items corresponding to the invalid table data in the table data as invalid index items, and record the first modification sequence number of the index page through an atomic operation.

[0075] Step S206: When the search for invalid index entries on the index page is completed, access and physical modification of the index page are allowed.

[0076] Step S208: Record invalid index items found on the index page.

[0077] Step S210: Allow access to the index page and prohibit physical modification of the index page.

[0078] Step S212: Record the second modification sequence number of the index page through an atomic operation.

[0079] Step S214: Determine whether the first modification number is the same as the second modification number.

[0080] If so, go to step S216; if not, go to step S222.

[0081] Step S216: Mark the final invalid index entry as invalid.

[0082] Step S218: Modify the second modification sequence number of the index page through an atomic operation.

[0083] Step S220: Allow access to and physical modification of the index page.

[0084] Step S222: Abandon marking the final invalid index entry as invalid.

[0085] The above embodiments may be combined in any way. According to any one of the above preferred embodiments or a combination of multiple preferred embodiments, the embodiments of the present invention can achieve the following beneficial effects:

[0086] The lock-free marking method for invalid indexes of the present invention prohibits access to and physical modification of index pages, searches for invalid index entries on the index pages, and records the first modification sequence number of the index pages. When the search for invalid index entries on the index pages is completed, access to and physical modification of the index pages are permitted. Access and physical modification of the index pages are avoided during this period, avoiding or reducing changes in the positions of invalid index entries, thereby ensuring the accuracy of marking invalid index entries. When the search for invalid index entries on the index pages is completed, considering that some subsequent steps may be time-consuming, such as the step of recording the invalid index entries found on the index pages, access to and physical modification of the index pages are permitted. This avoids reducing the concurrency of the database and affecting the throughput of the database system. Compared to a solution that prohibits access to and physical modification of index pages throughout the entire process, this embodiment relatively improves the concurrency of the database and ensures the throughput of the database system. This embodiment strikes a good balance between ensuring the accuracy of marking invalid index entries and ensuring the concurrency of the database, and has outstanding substantive features and significant improvements.

[0087] Moreover, in this embodiment, atomic operations are used when recording the first modification number and the second modification number of the index page, which can reduce or even avoid concurrent conflicts in the process of recording the first modification number and the second modification number of the page, thereby further improving the accuracy of invalid index marking items.

[0088] This embodiment also provides a machine-readable storage medium and a computer device. Figure 4 is a schematic diagram of a machine-readable storage medium 830 according to one embodiment of the present invention; Figure 5 8 is a schematic diagram of a computer device 900 according to an embodiment of the present invention. A machine-readable storage medium 830 stores a machine executable program 840, which, when executed by a processor, implements the method for marking invalid indexes for optimizing concurrency conflicts according to any of the above embodiments.

[0089] The computer device 900 may include a memory 920, a processor 910, and a machine executable program 840 stored in the memory 920 and running on the processor 910, and when the processor 910 executes the machine executable program 840, it implements the method for marking invalid indexes that optimize concurrency conflicts of any of the above embodiments.

[0090] It should be noted that the logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any machine-readable storage medium for use by an instruction execution system, device or equipment (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, device or equipment and execute instructions), or used in combination with these instruction execution systems, devices or equipment.

[0091] For the purposes of the description of this embodiment, the machine-readable storage medium 830 can be any device that can contain, store, communicate, propagate, or transmit a program for use with an instruction execution system, device, or apparatus or in conjunction with such instruction execution system, device, or apparatus. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection having one or more wirings (electronic device), a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium 40 can even be paper or other suitable medium on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting, or processing in other suitable ways as necessary, and then stored in a computer memory.

[0092] It should be understood that each part of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system.

[0093] The computer device 900 can be, for example, a server, a desktop computer, a laptop computer, a tablet computer, or a smartphone. In some examples, the computer device 900 can be a cloud computing node. The computer device 900 can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc. that perform specific tasks or implement specific abstract data types. The computer device 900 can be implemented in a distributed cloud computing environment where remote processing devices linked via a communication network perform tasks. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0094] The computer device 900 may include a processor 910 adapted to execute stored instructions, and a memory 920 that provides temporary storage for the instructions during operation. The processor 910 may be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. The memory 920 may include random access memory (RAM), read-only memory, flash memory, or any other suitable storage system.

[0095] The processor 910 can be connected to an I / O interface (input / output interface) suitable for connecting the computer device 900 to one or more I / O devices (input / output devices) via a system interconnect (e.g., PCI, PCI-Express, etc.). The I / O devices may include, for example, a keyboard and a pointing device, wherein the pointing device may include a touchpad or a touch screen, etc. The I / O devices may be built-in components of the computer device 900, or may be devices externally connected to the computing device.

[0096] Processor 910 can also be linked to the display interface that is suitable for connecting computer device 900 to display device by system interconnection.Display device can comprise the display screen that is built-in component of computer device 900.Display device can also comprise the computer monitor, television or projector etc. that are externally connected to computer device 900.In addition, network interface controller (network interface controller, NIC) can be suitable for connecting computer device 900 to network by system interconnection.In certain embodiments, NIC can use any suitable interface or protocol (such as Internet Small Computer System Interface etc.) to transmit data.Network can be cellular network, radio network, wide area network (WAN)), local area network (LAN) or Internet etc.Remote device can be connected to computing device by network.

[0097] The flowchart provided in this embodiment is not intended to indicate that the operations of the method will be performed in any particular order, or that all operations of the method are included in all every case. In addition, the method may include additional operations. Within the scope of the technical ideas provided by the method of this embodiment, additional changes can be made to the above method.

[0098] At this point, those skilled in the art will recognize that, although a number of exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications consistent with the principles of the present invention may be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the present invention. Therefore, the scope of the present invention should be understood and deemed to cover all such other variations or modifications.

Claims

1. A method for marking invalid indexes that optimize concurrency conflicts, comprising: allowing access to an index page and disallowing physical modification of the index page, wherein the index page has a plurality of index entries; Searching for invalid index entries on the index page and recording the first modification sequence number of the index page; When the search for invalid index entries on the index page is completed, physical modification of the index page is allowed; Recording invalid index entries found on the index page, and prohibiting physical modification of the index page after recording is completed; Recording the second modification serial number of the index page; determining whether the recorded invalid index entry of the index page is a final invalid index entry according to the first modification sequence number and the second modification sequence number; If so, the final invalid index entry is marked as invalid, and physical modification of the index page is allowed.

2. The marking method for optimizing invalid indexes due to concurrency conflicts according to claim 1, wherein: The recording of the first modification serial number of the index page includes: Record the first modification sequence number through an atomic operation; and / or The recording of the second modification serial number of the index page includes: The second modification sequence number is recorded through an atomic operation.

3. The marking method for optimizing invalid indexes due to concurrency conflicts according to claim 1, wherein: The determining, according to the first modification sequence number and the second modification sequence number, whether the recorded invalid index entry of the index page is a final invalid index entry includes: Determine whether the first modification number is the same as the second modification number.

4. The marking method for optimizing invalid indexes for concurrency conflicts according to claim 1, wherein: After marking the final invalid index item as invalid, the method further includes: The second modification sequence number of the index page is modified using an atomic operation.

5. The marking method for optimizing invalid indexes for concurrency conflicts according to claim 1, wherein: The searching for invalid index items on the index page includes: Invalid index entries on the index page are searched by scanning through the index page.

6. The marking method for optimizing invalid indexes for concurrency conflicts according to claim 5, wherein: The searching for invalid index entries on the index page by scanning through the index page includes: The index items of the index page are scanned and traversed, and table data of the index items are obtained, and index items corresponding to invalid table data in the table data are recorded as invalid index items.

7. The marking method for optimizing invalid indexes for concurrency conflicts according to claim 1, wherein: Marking the final invalid index item as invalid includes: The final invalid index items are marked as invalid in batches.

8. The method for marking invalid indexes for optimizing concurrency conflicts according to claim 1, wherein: If the first modification sequence number is different from the second modification sequence number, abandon the step of marking the final invalid index item as invalid.

9. A machine-readable storage medium having a machine executable program stored thereon, wherein when the machine executable program is executed by a processor, the method for marking invalid indexes for optimizing concurrency conflicts according to any one of claims 1 to 8 is implemented.

10. A computer device comprising a memory, a processor, and a machine executable program stored in the memory and running on the processor, wherein the processor implements the method for marking invalid indexes that optimize concurrency conflicts according to any one of claims 1 to 8 when executing the machine executable program.

Images