Defragmentation method and device, electronic equipment, storage medium and program product

Abstract

The invention discloses a defragmentation method and device, electronic equipment, a storage medium and a program product. According to the implementation scheme, the method comprises the steps of determining a defragmentation transaction, and a to-be-defragmented table and a to-be-defragmented B + tree corresponding to the defragmentation transaction; blocking the to-be-sorted table, and creating a temporary table corresponding to the to-be-sorted table; executing degradation processing on blocking of the to-be-sorted table, and inserting the to-be-sorted data into the temporary table to generate an inserted temporary table; operation corresponding to the logic log is redone on the inserted temporary table, and a redone temporary table and a redone temporary B + tree are obtained; and exchanging the to-be-sorted B + tree corresponding to the to-be-sorted table into the remade temporary B + tree to obtain a fragment sorting table. When the defragmentation transaction is executed, exclusive blocking does not need to be executed on the to-be-sorted table in the whole process, the to-be-sorted table is made to be in the available state, defragmentation of the to-be-sorted table is completed, and the concurrency and availability of a database system are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database, and in particular to a method and device for defragmentation, an electronic device, a storage medium and a program product. BACKGROUND

[0002] In a database system, table data is usually stored in a B+ tree structure, and a data page is a B+ tree node. After inserting a large amount of data into a table, the B+ tree will have more index nodes and data nodes, occupying more space. When data is deleted from the table, the table data page is not immediately reorganized or the B+ tree structure is not adjusted, but the record in the data page corresponding to the B+ tree is directly marked as deleted, and the transaction number stored in the record is modified to the current transaction number, indicating that the data has been deleted by the current transaction. The transaction that deletes the data can be cleaned by a cleaning thread to clean the deleted records (marked as deleted) on the page and reorganize the page, but the B+ tree structure is usually not adjusted. Therefore, although the data is deleted from the table, the storage space occupied by the B+ tree does not decrease.

[0003] If the amount of table data is large, or a large number of operations (such as insertion, update, deletion, etc.) are performed on the table, the data in the table can be frequently deleted, and there can be a lot of wasted space in the data page, the record storage becomes sparse, and the storage space occupied expands. This situation is called fragmentation of table data pages. When the table data page is severely fragmented, operations such as sequential scanning and range query of the table need to access more pages, the read-write performance of the table decreases, and the overall performance of the database system is affected. At this time, defragmentation is usually needed to reorganize and clean the severely wasted data pages to solve the fragmentation problem. The existing table fragmentation and reorganization strategy needs to be exclusively locked for the table throughout the process, and operations cannot be performed on the table during this period, which severely reduces the concurrency and availability of the database system. SUMMARY

[0004] The present application provides a method and device for defragmentation, an electronic device, a storage medium and a program product to enable the defragmentation of a table without affecting the operation of concurrent transactions on the table, thereby improving the availability of the database system.

[0005] According to an aspect of the present application, a method for defragmentation is provided, comprising:

[0006] determining a defragmentation transaction, and a table to be reorganized and a B+ tree to be reorganized corresponding to the defragmentation transaction, the B+ tree to be reorganized including a B+ tree corresponding to the table to be reorganized;

[0007] locking the table to be reorganized, and creating a temporary table corresponding to the table to be reorganized, the temporary table including a table having the same structure as the table to be reorganized;

[0008] performing downgrade processing on the lock of the to-be-arranged table, and inserting to-be-arranged data into the temporary table to generate an inserted temporary table, the to-be-arranged data including data in the to-be-arranged table that is visible to the fragmentation arrangement transaction;

[0009] redoing operations corresponding to logical logs on the inserted temporary table to obtain a redone temporary table and a redone temporary B+ tree, the logical logs including logs generated by the to-be-arranged table after the fragmentation arrangement transaction starts to execute, and the redone temporary B+ tree including a B+ tree corresponding to the redone temporary table;

[0010] swapping the to-be-arranged B+ tree corresponding to the to-be-arranged table into the redone temporary B+ tree to obtain a fragmentation arrangement table.

[0011] According to another aspect of the present application, there is provided a fragmentation arrangement device, comprising:

[0012] a determination module configured to determine a fragmentation arrangement transaction, and a to-be-arranged table and a to-be-arranged B+ tree corresponding to the fragmentation arrangement transaction, the to-be-arranged B+ tree including a B+ tree corresponding to the to-be-arranged table;

[0013] a creation module configured to perform a lock on the to-be-arranged table, and create a temporary table corresponding to the to-be-arranged table, the temporary table including a table having the same structure as the to-be-arranged table;

[0014] an insertion module configured to perform downgrade processing on the lock of the to-be-arranged table, and insert to-be-arranged data into the temporary table to generate an inserted temporary table, the to-be-arranged data including data in the to-be-arranged table that is visible to the fragmentation arrangement transaction;

[0015] a redo module configured to redo operations corresponding to logical logs on the inserted temporary table to obtain a redone temporary table and a redone temporary B+ tree, the logical logs including logs generated by the to-be-arranged table after the fragmentation arrangement transaction starts to execute, and the redone temporary B+ tree including a B+ tree corresponding to the redone temporary table;

[0016] an update module configured to swap the to-be-arranged B+ tree corresponding to the to-be-arranged table into the redone temporary B+ tree to obtain a fragmentation arrangement table.

[0017] According to another aspect of the present application, there is provided an electronic device, comprising:

[0018] at least one processor; and

[0019] a memory connected to the at least one processor in communication; wherein

[0020] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the defragmentation method according to any one of the embodiments of the present application.

[0021] According to another aspect of the present application, there is provided a computer readable storage medium storing computer instructions for enabling a processor to implement the defragmentation method according to any one of the embodiments of the present application when executed by the processor.

[0022] According to another aspect of the present application, there is provided a computer program product comprising a computer program for implementing the defragmentation method according to any one of the embodiments of the present application when executed by a processor.

[0023] The technical scheme of the embodiments of the present application determines a defragmentation transaction and a to-be-defragmented table and a to-be-defragmented B+ tree corresponding to the defragmentation transaction; performs blocking on the to-be-defragmented table and creates a temporary table corresponding to the to-be-defragmented table; performs degradation processing on the blocking of the to-be-defragmented table and inserts to-be-defragmented data into the temporary table to generate an inserted temporary table; re-does operations corresponding to logical logs on the inserted temporary table to obtain a re-done temporary table and a re-done temporary B+ tree; and exchanges the to-be-defragmented B+ tree corresponding to the to-be-defragmented table into the re-done temporary B+ tree to obtain a defragmented table. By performing degradation processing on the blocking of the to-be-defragmented table, the to-be-defragmented table is kept in a usable state without exclusive blocking on the to-be-defragmented table throughout the execution of the defragmentation transaction, and then the to-be-defragmented data and operations corresponding to the logical logs are stored in the temporary table to obtain the re-done temporary table, and the to-be-defragmented B+ tree corresponding to the to-be-defragmented table is exchanged into the re-done temporary B+ tree to obtain the defragmented table, thereby completing the defragmentation of the to-be-defragmented table without affecting other operations on the to-be-defragmented table, and improving the concurrency and availability of the database system.

[0024] It should be understood that the content described in this section is not intended to identify key or critical features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort based on these drawings.

[0026] Figure 1is a flow chart of a fragment arrangement method according to an embodiment of the present application;

[0027] Figure 2 is a flow chart of a temporary table generation method after insertion according to an embodiment of the present application;

[0028] Figure 3 is a structural schematic diagram of a fragment arrangement device according to an embodiment of the present application;

[0029] Figure 4 is a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the personnel in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0031] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0032] Embodiment one

[0033] Figure 1 is a flow chart of a fragment arrangement method according to an embodiment of the present application, and the present embodiment can be applicable to the case of arranging table fragments. The method can be executed by a fragment arrangement device, which can be realized in the form of hardware and / or software, and the fragment arrangement device can be configured in an electronic device. As shown in the figure, the method comprises: Figure 1

[0034] S110, determining a fragment arrangement transaction, and a table to be arranged and a B+ tree to be arranged corresponding to the fragment arrangement transaction.

[0035] ​The to-be-arranged B+ tree includes a B+ tree corresponding to the to-be-arranged table.

[0036] In the embodiment, the fragment arrangement transaction can be understood as a transaction of performing fragment arrangement on the to-be-arranged table in the case that the to-be-arranged table is seriously fragmented. The to-be-arranged table can be understood as a table in the database that is seriously fragmented. When the data amount of the to-be-arranged table is large, or a large number of operations (such as insertion, update, deletion, etc.) are performed on the to-be-arranged table, the records in the to-be-arranged table can be frequently deleted, and there can be a lot of wasted space in the data page where the to-be-arranged table is located, resulting in sparse record storage and inflated storage space. This is called fragmentation. The to-be-arranged B+ tree can be understood as a B+ tree corresponding to the to-be-arranged table. The B+ tree is a balanced multi-way lookup tree data structure that can be used to efficiently store and retrieve data in the to-be-arranged table. The to-be-arranged B+ tree can be used to indicate the data structure in the to-be-arranged table.

[0037] Specifically, when the to-be-arranged table is seriously fragmented, a fragment arrangement transaction is determined to perform fragment arrangement on the to-be-arranged table. For the to-be-arranged table, a to-be-arranged B+ tree indicating the data structure in the to-be-arranged table is determined. The data in the to-be-arranged table can be in the form of a B+ tree as a data structure, and a data page is a node of a B+ tree. After inserting data into the to-be-arranged table, the B+ tree will have more index nodes and data nodes.

[0038] For example, the B+ tree can be a clustered index B+ tree or a secondary index B+ tree.

[0039] S120, performing locking on the to-be-arranged table, and creating a temporary table corresponding to the to-be-arranged table.

[0040] The temporary table includes a table having the same structure as the to-be-arranged table.

[0041] In the embodiment, the temporary table can be understood as an empty table having the same structure as the to-be-arranged table, and the structure of the temporary table is completely identical to that of the to-be-arranged table.

[0042] Specifically, after starting to execute the fragment arrangement transaction, locking is first performed on the to-be-arranged table, i.e., the to-be-arranged table is locked, for example, an exclusive lock. After the to-be-arranged table is locked, no transaction concurrent with the fragment arrangement transaction is allowed to access the to-be-arranged table, and operations such as query, manipulation (DML), definition (DDL), etc. on the to-be-arranged table are prevented. Then, an empty table having the same structure as the to-be-arranged table is created as a temporary table.

[0043] For example, when the temporary table is created, a B+ tree corresponding to the temporary table is automatically created, and the structure of the B+ tree corresponding to the temporary table is the same as the structure of the to-be-organized B+ tree. If there is a secondary index on the to-be-organized table, it can be determined whether to create the same secondary index on the temporary table in response to a selection of a user. If the selection is to re-create, all secondary indexes on the to-be-organized table are created on the temporary table.

[0044] S130, performing a downgrade process on the lock of the to-be-organized table, and inserting the to-be-organized data into the temporary table to generate an inserted temporary table.

[0045] The to-be-organized data includes data in the to-be-organized table that is visible to the fragmentation organization transaction.

[0046] In this embodiment, the to-be-organized data can be understood as data in the to-be-organized table that needs to be stored in the temporary table, and the to-be-organized data is data in the to-be-organized table that is visible to the fragmentation organization transaction. The inserted temporary table can be understood as a data table obtained after the to-be-organized data is stored in the temporary table.

[0047] Specifically, after the temporary table is created, a downgrade process is performed on the lock of the to-be-organized table, so that the to-be-organized table can be queried or manipulated (DML) by concurrent transactions, but the to-be-organized table is not allowed to be defined (DDL) by concurrent transactions. The to-be-organized data visible to the fragmentation organization transaction is determined, and the to-be-organized data is inserted into the temporary table by querying to obtain an inserted temporary table.

[0048] S140, redoing operations corresponding to the logical log on the inserted temporary table to obtain a redone temporary table and a redone temporary B+ tree.

[0049] The logical log includes logs generated by the to-be-organized table after the fragmentation organization transaction starts to execute, and the redone temporary B+ tree includes a B+ tree corresponding to the redone temporary table.

[0050] In this embodiment, the logical log can be understood as a log storing records of operations on the to-be-organized table by concurrent transactions captured after the fragmentation organization transaction starts to execute. The redone temporary table can be understood as a data table obtained by redoing all records contained in the logical log on the temporary table. The redone temporary B+ tree can be understood as a B+ tree corresponding to the redone temporary table, and the redone temporary B+ tree can be used to indicate the data structure in the redone temporary table.

[0051] Specifically, after performing the downgrade processing on the lock of the table to be defragmented, the concurrent transaction can perform a query or a DML operation on the table to be defragmented. At this time, the logical log function of the table to be defragmented needs to be enabled, the operation performed by the concurrent transaction on the table to be defragmented is captured, and recorded in the logical log. Then, the operation corresponding to the logical log is redone on the post-insertion temporary table, and the post-redone temporary table is obtained. At this time, the post-redone temporary table has all the data in the table to be defragmented, and the operation of the concurrent transaction on the table to be defragmented is redone, and finally the post-redone temporary B+ tree corresponding to the post-redone temporary table is determined.

[0052] For example, the content recorded in the logical log includes a timestamp, a transaction number, a row of modified data, and specific data of the modified data. The commit or rollback of the concurrent transaction is also recorded in the logical log, and the operation performed by the concurrent transaction on the table can be restored by analyzing the record in the logical log.

[0053] S150, exchange the table to be defragmented B+ tree corresponding to the table to be defragmented into the post-redone temporary B+ tree, and obtain a defragmented table.

[0054] In this embodiment, the defragmented table can be understood as a data table after performing the defragmentation on the table to be defragmented.

[0055] Specifically, the post-redone temporary B+ tree is newly created, and there is almost no wasted space, and there is no fragmentation problem. Therefore, the table to be defragmented B+ tree corresponding to the table to be defragmented is exchanged into the post-redone temporary B+ tree, and the defragmented table is obtained. The defragmented table can use the post-redone temporary B+ tree to solve the fragmentation problem.

[0056] For example, the B+ tree of the table to be defragmented and the post-redone temporary table (if there is a secondary index, the B+ tree corresponding to the secondary index is also included here) is exchanged, that is, the table to be defragmented uses the post-redone temporary B+ tree of the post-redone temporary table without fragmentation problem, and the post-redone temporary table uses the table to be defragmented B+ tree of the table to be defragmented with serious fragmentation problem.

[0057] The technical scheme of the embodiment of the present application determines a fragment arrangement transaction and a to-be-arranged table and a to-be-arranged B+ tree corresponding to the fragment arrangement transaction; performs blocking on the to-be-arranged table, and creates a temporary table corresponding to the to-be-arranged table; performs degradation processing on the blocking of the to-be-arranged table, inserts to-be-arranged data into the temporary table, and generates an inserted temporary table; re-does operations corresponding to logical logs on the inserted temporary table, and obtains a re-done temporary table and a re-done temporary B+ tree; exchanges the to-be-arranged B+ tree corresponding to the to-be-arranged table into the re-done temporary B+ tree, and obtains a fragment arrangement table. By performing degradation processing on the blocking of the to-be-arranged table, the to-be-arranged table is kept in a usable state without exclusive blocking on the to-be-arranged table during the execution of the fragment arrangement transaction, and then the to-be-arranged data and the operations corresponding to the logical logs are stored in the temporary table to obtain the re-done temporary table, the to-be-arranged B+ tree corresponding to the to-be-arranged table is exchanged into the re-done temporary B+ tree to obtain the fragment arrangement table, the fragment arrangement of the to-be-arranged table is completed, and other operations on the to-be-arranged table are not affected during the process, thereby improving the concurrency and availability of the database system.

[0058] On the basis of the above embodiment, a variant embodiment of the above embodiment is proposed. It should be noted that, in order to make the description brief, only the differences from the above embodiment are described in the variant embodiment.

[0059] In one embodiment, the re-doing of the operations corresponding to the logical logs on the inserted temporary table to obtain the re-done temporary table and the re-done temporary B+ tree comprises:

[0060] For the operation record in the logical log, the operation corresponding to the operation record is re-done on the inserted temporary table to obtain an initial re-done temporary table;

[0061] During the acquisition of the initial re-done temporary table, when no newly generated logical log appears, the initial re-done temporary table is taken as the re-done temporary table, and a re-done temporary B+ tree corresponding to the re-done temporary table is determined, otherwise, a newly generated logical log is determined, and the initial re-done temporary table is re-done according to the newly generated logical log until no newly generated logical log appears.

[0062] In the present embodiment, the operation record can be understood as a record of the operation of the concurrent transaction on the to-be-arranged table, for example, a query or a DML operation performed by the concurrent transaction on the to-be-arranged table. The initial re-done temporary table can be understood as a data table obtained by re-doing the operation record in the logical log on the inserted temporary table, and the initial re-done temporary table can include the operation performed by the concurrent transaction on the to-be-arranged table.

[0063] Specifically, after obtaining the post-insertion temporary table, the logical log corresponding to the table to be arranged can be obtained. Each operation record in the logical log is parsed, the operation of the concurrent transaction indicated by the operation record on the table to be arranged is restored, and the operation is redone on the post-insertion temporary table to obtain an initial post-redo temporary table. During the obtaining of the initial post-redo temporary table, there can be an operation of the concurrent transaction on the table to be arranged, which can be determined by whether a new logical log is generated. When no new logical log is generated, it is determined that there is no operation of the concurrent transaction on the table to be arranged during the redoing of the logical log operation, and thus the initial post-redo temporary table can be used as the post-redo temporary table, and a post-redo temporary B+ tree corresponding to the post-redo temporary table is determined. Otherwise, it is determined that there is an operation of the concurrent transaction on the table to be arranged during the redoing of the logical log operation, and thus a new logical log needs to be determined, and the initial post-redo temporary table is redone according to the operation record in the new logical log until no new logical log is generated, and thus the post-redo temporary table is determined.

[0064] For example, after each operation record in the logical log is redone, the table to be arranged can be locked again. After the locking, it is checked whether there is an operation of the concurrent transaction on the table to be arranged, that is, whether a new logical log is generated. If a new logical log is generated, the exclusive lock on the table to be arranged is released, and the operation in the new logical log is redone. Then the locking and the checking of the generation of the logical log are continued until no new logical log is generated.

[0065] Optionally, the method for obtaining the logical log comprises:

[0066] After the table arrangement transaction is started, the operation of the concurrent transaction on the table to be arranged is captured, and an operation record is stored in the logical log, the operation record comprising a log generated when the concurrent transaction performs the operation on the table to be arranged.

[0067] Specifically, after the temporary table is created and before the exclusive lock on the table to be arranged is downgraded, the logical log function of the table to be arranged can be started. After the starting, the operation of the concurrent transaction on the table to be arranged is captured, and the operation record is stored in the logical log. The content of the operation record includes a timestamp, a transaction number, a row number (ROWID) of modified data, and specific modified data. In addition, the commit or rollback of the concurrent transaction also records the logical log, and the operation of the concurrent transaction on the table to be arranged can be restored by parsing the record in the logical log.

[0068] In one embodiment, after the table to be arranged B+ tree corresponding to the table to be arranged is exchanged into the post-redo temporary B+ tree to obtain a table to be arranged, the method further comprises:

[0069] swap the redo-after temporary B+ tree corresponding to the redo-after temporary table to the to-be-organized B+ tree;

[0070] commit the defragmentation transaction and release the lock on the defragmentation table.

[0071] Specifically, the B+ tree of the to-be-organized table and the redo-after temporary table can be swapped by modifying the table dictionary. The redo-after temporary table uses the redo-after temporary B+ tree that has no fragmentation problem for the to-be-organized table. After the redo-after temporary table uses the to-be-organized B+ tree that is heavily fragmented, the redo-after temporary table is deleted. When the redo-after temporary table is deleted, the to-be-organized B+ tree that is heavily fragmented is also deleted. Finally, the defragmentation transaction is committed, and the lock on the defragmentation table corresponding to the to-be-organized table after the defragmentation is released.

[0072] Embodiment Two

[0073] Figure 2 is a flowchart of a method for generating an insertion-after temporary table according to Embodiment Two of the present application. This embodiment is directed to the method for generating an insertion-after temporary table in the above-mentioned embodiments. As shown in Figure 2 the method comprises:

[0074] S210, determining a defragmentation transaction, and a to-be-organized table and a to-be-organized B+ tree corresponding to the defragmentation transaction.

[0075] S220, performing a lock on the to-be-organized table, and creating a temporary table corresponding to the to-be-organized table.

[0076] S230, performing a downgrade process on the lock of the to-be-organized table, and performing S231-S233.

[0077] S231, determining initial data in the to-be-organized table.

[0078] The initial data comprises at least one.

[0079] In this embodiment, the initial data can be understood as data stored in the to-be-organized table. The initial data can be data processed by a concurrent transaction.

[0080] Specifically, the lock of the to-be-organized table is downgraded. After the downgrade process, the to-be-organized table can be queried or manipulated (DML) by a concurrent transaction, but the to-be-organized table is not allowed to be defined (DDL) by a concurrent transaction. Therefore, the initial data in the to-be-organized table can be data processed by a concurrent transaction.

[0081] S232, for each initial data, determine the visibility of the initial data to the defragmentation transaction based on the transaction isolation level, and determine the initial data as the data to be defragmented when the visibility of the initial data is visible.

[0082] The transaction isolation level includes an isolation level of the defragmentation transaction.

[0083] In this embodiment, the transaction isolation level is used to control the access and modification behavior of data when multiple transactions are concurrently executed, so as to ensure the consistency and integrity of the data. Different isolation levels will bring different concurrent performance and data consistency guarantee. The visibility can be understood as whether the initial data is visible to the defragmentation transaction.

[0084] Specifically, the transaction isolation level of the defragmentation transaction can be determined as a serializable isolation level. For each initial data in the logical log, the visibility of the initial data is determined based on the transaction isolation level. For the serializable isolation level, the initial data at the start time is visible to the defragmentation transaction, and the initial data modified by the subsequent concurrent transaction is not visible to the defragmentation transaction. In the case where the visibility of the initial data is visible, the initial data is determined as the data to be defragmented.

[0085] Optionally, the determination of the visibility of the initial data based on the transaction isolation level comprises:

[0086] When there is a concurrent transaction executing an operation on the initial data, the operation type is determined, and the visibility of the initial data is determined according to the operation type and the transaction isolation level, and the concurrent transaction includes a transaction concurrent with the defragmentation transaction;

[0087] Otherwise, the transaction isolation level indicates that the visibility of the initial data is visible.

[0088] In this embodiment, the operation type can be understood as the type corresponding to the operation of the concurrent transaction on the initial data in the table to be defragmented, and the operation type can include insertion, modification, or deletion.

[0089] Specifically, when there is a concurrent transaction executing an operation on the table to be defragmented, the initial data to be operated and the operation type are determined, and the visibility of the initial data is determined according to different operation types under the indication of the transaction isolation level. When there is no concurrent transaction executing an operation on the table to be defragmented, the transaction isolation level indicates that the visibility of the initial data in the table to be defragmented is visible.

[0090] Optionally, the determination of the visibility of the initial data based on the operation type and the transaction isolation level comprises:

[0091] In a case where the operation type is an insertion operation, the transaction isolation level indicates that the initial data is invisible;

[0092] In a case where the operation type is a modification operation, the transaction isolation level indicates that the initial data is invisible, historical version data corresponding to the initial data is determined, and the historical version data is determined as the data to be arranged, the historical version data including data visible to the fragmentation arrangement transaction before the initial data is modified.

[0093] In a case where the operation type is a deletion operation, the transaction isolation level indicates that the initial data is visible.

[0094] In the embodiment, the historical version data can be understood as data before a concurrent transaction operation corresponding to the initial data, and the data after the operation of the concurrent transaction on the historical version data in the table to be arranged is the initial data.

[0095] For example, if the operation type is an insertion operation, that is, the initial data is inserted by the concurrent transaction, the initial data is invisible to the fragmentation arrangement transaction, and the initial data does not need to be arranged. If the operation type is a modification operation, that is, the initial data is modified by the concurrent transaction, the last historical version data visible before the modification is assembled according to the rollback log, and the historical version data is arranged as the data to be arranged. If the operation type is a deletion operation, that is, the initial data is deleted by the concurrent transaction, that is, the initial data is marked with a deletion mark, and the initial data is visible to the fragmentation arrangement transaction, and the initial data is arranged as the data to be arranged.

[0096] S233, at least one piece of data to be arranged corresponding to the table to be arranged is determined, and the at least one piece of data to be arranged is inserted into the temporary table to generate an inserted temporary table.

[0097] Specifically, all the data to be arranged in the table to be arranged is determined, and all the data to be arranged is inserted into the temporary table to generate the inserted temporary table. The data in the inserted temporary table is all the data in the table to be arranged at the start time of the fragmentation arrangement transaction, and does not include data inserted, deleted or modified by the concurrent transaction, and is not affected by the concurrent transaction.

[0098] S240, redoing operations corresponding to the logical log on the inserted temporary table to obtain a redone temporary table and a redone temporary B+ tree.

[0099] S250, exchanging the B+ tree corresponding to the table to be arranged into the redone temporary B+ tree to obtain a fragmentation arrangement table.

[0100] The technical scheme of the embodiment of the present application determines the initial data in the table to be arranged; for each initial data, determines the visibility of the initial data based on a transaction isolation level, and in the case that the visibility of the initial data is visible, determines the initial data as data to be arranged; determines at least one piece of data to be arranged corresponding to the table to be arranged, and inserts the at least one piece of data to be arranged into the temporary table to generate an inserted temporary table. Through the transaction isolation level, the data to be arranged inserted into the temporary table is determined, which ensures that the data to be arranged is not affected by concurrent transactions, and the correctness of the operation corresponding to the redo logical log is ensured.

[0101] Embodiment Three

[0102] Figure 3 is a structural schematic diagram of a fragment arrangement device according to Embodiment Three of the present application. As shown in the figure, the device comprises: Figure 3

[0103] The determining module 310 is configured to determine a fragment arrangement transaction, and a table to be arranged and a B+ tree to be arranged corresponding to the fragment arrangement transaction, wherein the B+ tree to be arranged comprises a B+ tree corresponding to the table to be arranged.

[0104] The creating module 320 is configured to perform locking on the table to be arranged, and create a temporary table corresponding to the table to be arranged, wherein the temporary table comprises a table having the same structure as the table to be arranged.

[0105] The inserting module 330 is configured to perform downgrade processing on the locking of the table to be arranged, and insert data to be arranged into the temporary table to generate an inserted temporary table, wherein the data to be arranged comprises data in the table to be arranged that is visible to the fragment arrangement transaction.

[0106] The redo module 340 is configured to redo an operation corresponding to a logical log on the inserted temporary table to obtain a redone temporary table and a redone temporary B+ tree, wherein the logical log comprises a log generated by the table to be arranged after the fragment arrangement transaction starts to be executed, and the redone temporary B+ tree comprises a B+ tree corresponding to the redone temporary table.

[0107] The updating module 350 is configured to exchange the B+ tree to be arranged corresponding to the table to be arranged into the redone temporary B+ tree to obtain a fragment arrangement table

[0108] ​The fragment arrangement device provided by the embodiment of the application comprises: a determining module configured to determine a fragment arrangement transaction and a to-be-arranged table and a to-be-arranged B+ tree corresponding to the fragment arrangement transaction; a creating module configured to perform blocking on the to-be-arranged table and create a temporary table corresponding to the to-be-arranged table; an inserting module configured to perform degradation processing on the blocking of the to-be-arranged table and insert to-be-arranged data into the temporary table to generate an inserted temporary table; a redoing module configured to redo an operation corresponding to a logical log on the inserted temporary table to obtain a redone temporary table and a redone temporary B+ tree; and an updating module configured to exchange the to-be-arranged B+ tree corresponding to the to-be-arranged table into the redone temporary B+ tree to obtain a fragment arrangement table. Through mutual cooperation between the modules, degradation processing is performed on the blocking of the to-be-arranged table, so that the to-be-arranged table is in a usable state without exclusive blocking of the to-be-arranged table during execution of the fragment arrangement transaction, to-be-arranged data and an operation corresponding to the logical log are stored in the temporary table to obtain the redone temporary table, the to-be-arranged B+ tree corresponding to the to-be-arranged table is exchanged into the redone temporary B+ tree to obtain the fragment arrangement table, and the fragment arrangement of the to-be-arranged table is completed, without affecting other operations on the to-be-arranged table during the process, thereby improving the concurrency and availability of the database system.

[0109] In one embodiment, the inserting module 330 comprises:

[0110] A first determining unit is configured to determine initial data in the to-be-arranged table, wherein the initial data comprises at least one transaction.

[0111] A second determining unit is configured to determine, for each piece of initial data, the visibility of the initial data to the fragment arrangement transaction based on a transaction isolation level, and determine the initial data as to-be-arranged data when the visibility of the initial data is visible, wherein the transaction isolation level comprises an isolation level of the fragment arrangement transaction.

[0112] A third determining unit is configured to determine at least one piece of to-be-arranged data corresponding to the to-be-arranged table and insert the at least one piece of to-be-arranged data into the temporary table to generate an inserted temporary table.

[0113] In one embodiment, the second determining unit comprises:

[0114] A determining subunit is configured to determine an operation type when a concurrent transaction performs an operation on the initial data, and determine the visibility of the initial data according to the operation type and the transaction isolation level, wherein the concurrent transaction comprises a transaction concurrent with the fragment arrangement transaction.

[0115] An indicating subunit is configured to indicate that the transaction isolation level indicates that the visibility of the initial data is visible, otherwise.

[0116] In one embodiment, the determining subunit is specifically configured to:

[0117] In a case where the operation type is an insertion operation, the transaction isolation level indicates that the initial data is invisible;

[0118] In a case where the operation type is a modification operation, the transaction isolation level indicates that the initial data is invisible, historical version data corresponding to the initial data is determined, and the historical version data is determined as the data to be arranged, the historical version data including data visible to the fragment arrangement transaction before the modification operation is performed on the initial data;

[0119] In a case where the operation type is a deletion operation, the transaction isolation level indicates that the initial data is visible.

[0120] In one embodiment, the redo module 340 includes:

[0121] The redo unit is configured to, for an operation record in the logical log, redo the operation corresponding to the operation record on the inserted temporary table to obtain an initial redone temporary table;

[0122] The updating unit is configured to, during acquisition of the initial redone temporary table, determine the initial redone temporary table as a redone temporary table and a redone temporary B+ tree corresponding to the redone temporary table when no newly generated logical log appears, or determine a newly generated logical log and redo the initial redone temporary table according to the newly generated logical log until no newly generated logical log appears.

[0123] In one embodiment, the updating unit is specifically configured to:

[0124] After the fragment arrangement transaction starts, operations performed on the table to be arranged by a concurrent transaction are captured, and operation records are stored in the logical log, the operation records including logs generated when the concurrent transaction performs operations on the table to be arranged.

[0125] In one embodiment, the fragment arrangement apparatus further includes a deletion module specifically configured to:

[0126] The redone temporary B+ tree corresponding to the redone temporary table is exchanged for the B+ tree to be arranged;

[0127] The fragment arrangement transaction is committed, and the lock on the fragment arrangement table is released.

[0128] The fragment arrangement device provided by the embodiment of the present application can execute the fragment arrangement method provided by any embodiment of the present application, and through mutual cooperation and collaborative work among the modules, the arrangement of the table fragments is completed, and the function modules and beneficial effects corresponding to the execution method are possessed.

[0129] Embodiment four

[0130] According to the embodiments of the present application, the present application also provides an electronic device, a computer readable storage medium and a computer program product.

[0131] Figure 4 is a block diagram of an electronic device provided according to the fourth embodiment of the present application, and the electronic device can implement the fragment arrangement method described in the embodiments of the present application. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0132] As shown in Figure 4 The electronic device 410 includes at least one processor 411, and a memory, such as a read-only memory (ROM) 412, a random access memory (RAM) 413, etc., which is communicatively connected to the at least one processor 411, wherein the memory stores a computer program that can be executed by the at least one processor. The processor 411 can execute various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 412 or loaded into the random access memory (RAM) 413 from the storage unit 418. In the RAM 413, various programs and data required for the operation of the electronic device 410 can also be stored. The processor 411, the ROM 412, and the RAM 413 are connected to each other through a bus 414. An input / output (I / O) interface 415 is also connected to the bus 414.

[0133] A plurality of components in the electronic device are connected to the I / O interface 415, including: an input unit 416, such as a keyboard, a mouse, etc.; an output unit 417, such as various types of displays, speakers, etc.; a storage unit 418, such as a magnetic disk, an optical disk, etc.; and a communication unit 419, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 419 allows the electronic device to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunications networks.

[0134] The processor 411 can be various general-purpose and / or special-purpose processing components having processing and computing capabilities. Some examples of the processor 411 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, and the like. The processor 411 performs various methods and processes described above, such as the defragmentation method.

[0135] In some embodiments, the defragmentation method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 418. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 410 via the ROM 412 and / or the communication unit 419. When the computer program is loaded onto the RAM 413 and executed by the processor 411, one or more steps of the defragmentation method described above can be performed. Alternatively, in other embodiments, the processor 411 can be configured to perform the defragmentation method by any other appropriate means, such as by means of firmware.

[0136] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0137] Computer programs used to implement the methods of the application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed by the processor of the machine, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, and partially on a machine or a remote machine or a server.

[0138] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0139] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0140] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), blockchain network, and the Internet.

[0141] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0142] In some embodiments, the computer program product includes a computer program which, when executed by a processor, implements the defragmentation method provided by the embodiments of the present application.

[0143] The technical scheme of the embodiments of the present application is a defragmentation method, device, electronic equipment, storage medium and program product. A defragmentation transaction and a to-be-defragmented table and a to-be-defragmented B+ tree corresponding to the defragmentation transaction are determined; a lock is executed on the to-be-defragmented table, and a temporary table corresponding to the to-be-defragmented table is created; a downgrade process is executed on the lock of the to-be-defragmented table, and to-be-defragmented data is inserted into the temporary table to generate an inserted temporary table; an operation corresponding to a logical log is redone on the inserted temporary table to obtain a redone temporary table and a redone temporary B+ tree; the to-be-defragmented B+ tree corresponding to the to-be-defragmented table is exchanged into the redone temporary B+ tree to obtain a defragmented table. By executing the downgrade process on the lock of the to-be-defragmented table, the exclusive lock of the to-be-defragmented table is not required throughout the execution of the defragmentation transaction, so that the to-be-defragmented table is in a usable state. Then, the to-be-defragmented data and the operation corresponding to the logical log are stored in the temporary table to obtain the redone temporary table, and the to-be-defragmented B+ tree corresponding to the to-be-defragmented table is exchanged into the redone temporary B+ tree to obtain the defragmented table, so that the defragmentation of the to-be-defragmented table is completed. In this process, other operations on the to-be-defragmented table are not affected, and the concurrency and availability of the database system are improved.

[0144] It should be understood that the various forms of flow shown above can be reordered, steps added or removed. For example, the steps described in the present application can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical scheme of the present application can be achieved, which is not limited herein.

[0145] The above detailed description does not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A defragmentation method, characterized by, The method comprises the following steps: determining a fragment arrangement transaction, and a to-be-arranged table and a to-be-arranged B+ tree corresponding to the fragment arrangement transaction, the to-be-arranged B+ tree comprising a B+ tree corresponding to the to-be-arranged table; performing locking on the to-be-arranged table, and creating a temporary table corresponding to the to-be-arranged table, the temporary table comprising a table having the same structure as the to-be-arranged table; performing downgrading processing on the locking of the to-be-arranged table, and inserting to-be-arranged data into the temporary table to generate an inserted temporary table, the to-be-arranged data comprising data in the to-be-arranged table that is visible to the fragment arrangement transaction; redoing operations corresponding to logical logs on the inserted temporary table to obtain a redone temporary table and a redone temporary B+ tree, the logical logs comprising logs generated by the to-be-arranged table after the fragment arrangement transaction starts to be executed, and the redone temporary B+ tree comprising a B+ tree corresponding to the redone temporary table; swapping the to-be-arranged B+ tree corresponding to the to-be-arranged table into the redone temporary B+ tree to obtain a fragment arrangement table.

2. The method of claim 1, wherein, The inserting of the to-be-arranged data into the temporary table to generate an inserted temporary table comprises the following steps: determining initial data in the to-be-arranged table, the initial data comprising at least one piece of data; for each piece of initial data, determining the visibility of the initial data to the fragment arrangement transaction based on a transaction isolation level, and determining the initial data as to-be-arranged data when the visibility of the initial data is visible, the transaction isolation level comprising an isolation level of the fragment arrangement transaction; determining at least one piece of to-be-arranged data corresponding to the to-be-arranged table, and inserting the at least one piece of to-be-arranged data into the temporary table to generate an inserted temporary table.

3. The method of claim 2, wherein, The determining of the visibility of the initial data based on the transaction isolation level comprises the following steps: when there is a concurrent transaction performing an operation on the initial data, determining an operation type, and determining the visibility of the initial data according to the operation type and the transaction isolation level, the concurrent transaction comprising a transaction concurrent with the fragment arrangement transaction; otherwise, the transaction isolation level indicates that the visibility of the initial data is visible.

4. The method of claim 3, wherein, The determining of the visibility of the initial data according to the operation type and the transaction isolation level comprises the following steps: when the operation type is an insertion operation, the transaction isolation level indicates that the visibility of the initial data is invisible; when the operation type is a modification operation, the transaction isolation level indicates that the visibility of the initial data is invisible, historical version data corresponding to the initial data is determined, and the historical version data is determined as to-be-arranged data, the historical version data comprising data visible to the fragment arrangement transaction before the initial data is executed by the modification operation; when the operation type is a deletion operation, the transaction isolation level indicates that the visibility of the initial data is visible.

5. The method of claim 1, wherein, The redoing of operations corresponding to the logical logs on the inserted temporary table to obtain a redone temporary table and a redone temporary B+ tree comprises the following steps: Redo the operation corresponding to the operation record in the logical log on the inserted temporary table to obtain an initial redone temporary table; During the obtaining of the initial redone temporary table, when no newly generated logical log occurs, the initial redone temporary table is taken as a redone temporary table, and a redone temporary B+ tree corresponding to the redone temporary table is determined, otherwise, a newly generated logical log is determined, and the initial redone temporary table is redone according to the newly generated logical log until no newly generated logical log occurs.

6. The method of claim 5, wherein, The method for obtaining the logical log comprises: After the start of the defragmentation transaction, capturing an operation performed on the table to be defragmented by a concurrent transaction, and storing an operation record into the logical log, the operation record comprising a log generated when the concurrent transaction performs the operation on the table to be defragmented.

7. The method of claim 1, wherein, After the exchanging of the table to be defragmented B+ tree corresponding to the table to be defragmented into the redone temporary B+ tree to obtain a defragmented table, further comprising: Exchanging the redone temporary B+ tree corresponding to the redone temporary table into the table to be defragmented B+ tree; Committing the defragmentation transaction, and releasing the lock on the defragmented table.

8. A debris management device, characterized by, Comprise: A determination module for determining a defragmentation transaction, and a table and a table to be defragmented B+ tree corresponding to the defragmentation transaction, the table to be defragmented B+ tree comprising a B+ tree corresponding to the table to be defragmented; A creation module for performing a lock on the table to be defragmented, and creating a temporary table corresponding to the table to be defragmented, the temporary table comprising a table having the same structure as the table to be defragmented; An insertion module for performing a downgrade process on the lock of the table to be defragmented, and inserting defragmentation data into the temporary table to generate an inserted temporary table, the defragmentation data comprising data in the table to be defragmented visible to the defragmentation transaction; A redo module for redoing an operation corresponding to a logical log on the inserted temporary table to obtain a redone temporary table and a redone temporary B+ tree, the logical log comprising a log generated by the table to be defragmented after the start of the defragmentation transaction, and the redone temporary B+ tree comprising a B+ tree corresponding to the redone temporary table; An update module for exchanging the table to be defragmented B+ tree corresponding to the table to be defragmented into the redone temporary B+ tree to obtain a defragmented table.

9. An electronic device, comprising: The electronic device comprises: At least one processor; and The memory is in communication with the at least one processor; wherein The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the defragmentation method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for enabling the processor to execute the defragmentation method of any one of claims 1-7 when executed.

11. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program implements the defragmentation method according to any one of claims 1-7 when executed by the processor.

Images