Data recovery method and device during parallel insertion of insert select in database transaction

Abstract

The invention relates to a method and a device for recovering data during parallel insertion of insert select in database transactions, and the method comprises the following steps: inserting data in a source table into a target table in a concurrent manner through a plurality of sub-threads in the same transaction in a database, generating a physical log and a logical log, and writing the logical log into a cache of the database; if it is determined that all the sub-threads successfully insert the data, all the sub-threads are notified to carry out data submission on the data in the transaction; and if abnormal downtime of the database occurs when submission of all the sub-threads is not finished, data recovery is performed according to the physical log and the logic log, and data submission is performed again after data recovery. The data consistency in the database is improved.

Description

Technical Field

[0001] This application relates to the field of database technology, and particularly to a data recovery method and apparatus for parallel insertion of insert select within a database transaction. Background Art

[0002] Using the insert select method to import source table data into the target table is a common application scenario in the database field. When the source table data volume is large, the parallel insertion method can be used to import the data into the target table. This process involves data insertion and data submission. However, if the database crashes abnormally before the data is submitted, it is easy to cause data inconsistency between the source table and the target table, and data insertion errors may occur. Summary of the Invention

[0003] This application provides a data recovery method and apparatus for parallel insertion of insert select within a database transaction to solve the problem of data insertion errors.

[0004] In a first aspect, this application provides a data recovery method for parallel insertion of insert select within a database transaction, the method including:

[0005] After concurrently inserting the data in the source table into the target table through multiple child threads within the same transaction in the database, generating physical logs and logical logs, and writing the logical logs into the cache of the database;

[0006] If it is determined that all child threads have successfully inserted data, notifying all child threads to commit the data in the transaction;

[0007] If the database crashes abnormally before all child threads finish committing, performing data recovery according to the physical logs and the logical logs, and re-performing data submission after the data recovery.

[0008] Optionally, generating logical logs includes:

[0009] Obtaining the log cache generated corresponding to each child thread;

[0010] Merging the log cache of each child thread into the main thread, where the main thread also serves as a child thread;

[0011] Writing the merged log cache into the logical logs.

[0012] Optionally, before writing the logical logs into the cache of the database, the method further includes:

[0013] If the database crashes abnormally, notifying all child threads to roll back according to the physical logs;

[0014] Send a failure message to the terminal, where the failure message is used to indicate the failure of the insertselect concurrent import within the database transaction.

[0015] Optionally, after generating the physical log and the logical log, the method further includes:

[0016] If there is a situation where a child thread fails to insert data, notify all child threads to roll back according to the physical log;

[0017] Send a failure message to the terminal, where the failure message is used to indicate the failure of the insertselect concurrent import within the database transaction.

[0018] Optionally, the database is a single-point database;

[0019] After writing the logical log into the cache of the database, the method further includes: locking the target table, where the locking is used to block the access of other transactions to the target table in the transaction during the data submission process;

[0020] After determining that all child threads have completed the submission, the method further includes: unlocking the target table.

[0021] Optionally, during the process of merging the log caches of each child thread into the main thread, the method further includes:

[0022] Record the position of the log cache corresponding to each child thread, where the position is used to record the location of copying the log cache;

[0023] Merge the log caches according to the order of the positions.

[0024] In a second aspect, the present application provides a data recovery device for parallel insertion of insert select within a database transaction, and the device includes:

[0025] A generation module, configured to generate a physical log and a logical log after concurrently inserting data in a source table into a target table by multiple child threads within the same transaction in the database, and write the logical log into the cache of the database;

[0026] A submission module, configured to notify all child threads to submit the data in the transaction if it is determined that all child threads have successfully inserted data;

[0027] A recovery module, configured to perform data recovery according to the physical log and the logical log if the database crashes abnormally when all child threads have not completed the submission, and re-perform data submission after the data recovery.

[0028] Optionally, the apparatus is further configured to:

[0029] Record the position of the log cache corresponding to each sub-thread, where the position is used to record the location of copying the log cache;

[0030] Merge the log caches according to the order of the positions.

[0031] In a third aspect, the present application provides an electronic device, including: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; and at least one memory connected to the at least one bus.

[0032] In a fourth aspect, the present application further provides a computer storage medium storing computer-executable instructions for executing the data recovery method during insert select parallel insertion in the database transaction described in any one of the above of the present application.

[0033] The above technical solutions provided by the embodiments of the present application have the following advantages compared with the prior art: After the data is successfully inserted and before the data is committed, if the database crashes abnormally, then data recovery is achieved according to the physical log and the logical log, avoiding inconsistent contents between the source table and the target table, and the present application improves the data consistency in the database. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present invention and used together with the specification to explain the principles of the present invention.

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0036] One or more embodiments are illustrated by way of example in the pictures in the corresponding accompanying drawings. These exemplary illustrations do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements, unless otherwise stated, and the drawings in the drawings do not constitute a proportional limitation.

[0037] Figure 1 It is a system schematic diagram for data recovery during insert select parallel insertion in a database transaction provided by an embodiment of the present application;

[0038] Figure 2A flowchart of a data recovery method for parallel insert of insert select within a database transaction provided by an embodiment of the present application;

[0039] Figure 3 A flowchart of a method for parallel insert of insert select within a database transaction provided by an embodiment of the present application;

[0040] Figure 4 A signaling diagram of parallel insert of insert select within a database transaction provided by an embodiment of the present application;

[0041] Figure 5 A schematic structural diagram of a data recovery device for parallel insert of insert select within a database transaction provided by an embodiment of the present application;

[0042] Figure 6 A schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0043] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are some but not all of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the scope of protection of the present application.

[0044] The following disclosure provides many different embodiments or examples for implementing different structures of the present invention. To simplify the disclosure of the present invention, components and settings of specific examples are described below. Of course, they are only examples and are not intended to limit the present invention. In addition, the present invention may repeat reference numerals and / or letters in different examples. This repetition is for the purpose of simplification and clarity and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0045] To solve the problems mentioned in the background art, according to one aspect of the embodiments of the present application, an embodiment of a data recovery method for parallel insert of insert select within a database transaction is provided.

[0046] Optionally, in the embodiments of the present application, the above data recovery method for parallel insert of insert select within a database transaction may be applied to a hardware environment composed of a terminal 101 and a server 103 as shown in Figure 1 As shown in Figure 1As shown in the figure, the server 103 is connected to the terminal 101 through a network, and can be used to feedback the parallel insertion result of insert select to the terminal. The database 105 can be set on the server or independently of the server, and is used to provide data storage services for the server 103. The above network includes but is not limited to: wide area network, metropolitan area network or local area network. The terminal 101 includes but is not limited to PC, mobile phone, tablet computer, etc.

[0047] In the embodiment of the present application, a data recovery method for parallel insertion of insert select within a database transaction can be executed by the server 103.

[0048] The embodiment of the present application provides a data recovery method for parallel insertion of insert select within a database transaction, which can be applied to a server and is used to improve data consistency in the database.

[0049] Next, in combination with specific implementation manners, a data recovery method for parallel insertion of insert select within a database transaction provided by the embodiment of the present application will be described in detail. As Figure 2 shown, the specific steps are as follows:

[0050] Step 201: After concurrently inserting the data in the source table into the target table through multiple sub-threads within the same transaction in the database, generate physical logs and logical logs, and write the logical logs into the cache of the database.

[0051] Step 202: If it is determined that all sub-threads have successfully inserted data, notify all sub-threads to commit the data in the transaction.

[0052] Step 203: If the database crashes abnormally when all sub-threads have not finished committing, perform data recovery according to the physical logs and logical logs, and re-perform data submission after the data recovery.

[0053] In the present application, the database is a software system for storing and managing data. In this system, various data operations can be performed, such as insertion, update, deletion, etc.; the source table is a table containing original data, and this data will be imported into the target table. Exemplarily, the source table is table t1; the target table is a table used to receive the data imported from the source table. Exemplarily, the target table is table t2. The database is an overall management system, while the source table t1 and the target table t2 are two specific data tables among them. By means of the SQL statement insert into t2 select * from t1, the data in the source table t1 can be imported into the target table t2. This method can be realized by concurrently inserting data through multiple sub-threads within the same transaction, and can improve the database import performance.

[0054] The specific process of data insertion is as follows: The terminal sends an insert select request to the database. The insert select request is used to request the database to copy the data in the source table to the target table, so as to achieve data migration and synchronization. The user node thread of the database receives the insert select request sent by the client. This thread serves as the coordinator of the entire parallel insertion operation, that is, the main thread of the insert select parallelism.

[0055] The main thread evenly splits the source table data into corresponding data ranges according to the parallelism of data parallel insertion. The main thread creates or wakes up the corresponding number of child threads according to the number of data ranges. The main thread assigns each split data range to a child thread, and multiple child threads concurrently perform data insertion on the target table. Specifically, the main thread splits the data in the table into multiple data ranges, each data range includes multiple data, and multiple child threads simultaneously perform insertion operations on multiple data ranges, so as to achieve parallel insertion of data in multiple data ranges into the target table. When each child thread performs insertion operations on multiple data in the corresponding data range, it still performs data insertion one by one in a serial manner.

[0056] In the above process of concurrent data insertion, if the database crashes abnormally, it will cause the data submitted by the child threads to the database to be inconsistent, resulting in the content of the source table and the target table being different, and the insert select being inaccurate. Therefore, after the present application concurrently inserts the data in the source table into the target table through multiple child threads, it will generate physical logs and logical logs, and write the logical logs into the cache of the database.

[0057] The physical logs of the database record the changes in the underlying data structure of the database, such as the changes in data pages on the disk. The physical logs focus on the specific implementation of data storage and do not pay attention to the logical changes of data. The physical logs can be used for data recovery.

[0058] The logical logs of the database record all change operations (such as insert, update, delete, etc.) performed on the database and the time when these operations occur. The logical logs can be used by downstream data nodes or synchronization tools to synchronize data.

[0059] By controlling the consistency of the modification of the database physical logs and the uniqueness of the logical logs by the concurrent insertion threads, the atomicity of the insert select transaction can be guaranteed. Among them, the consistency of the physical log modification means that the modifications of the physical logs by each child thread need to be all committed or all rolled back. The uniqueness of the logical log means that the insert operations concurrently executed by each child thread after splitting finally write only one logical log for downstream data nodes or synchronization tools to synchronize data. This is also a necessary guarantee for the insert select to be regarded as a whole transaction although it is independently inserted by child threads.

[0060] When a transaction starts, the system generates a record containing the XID (unique identification information) of the transaction and writes it into the physical log and the logical log. During the execution of the transaction, all data change operations are recorded in the logical log, while the physical log records the specific implementation details of these operations.

[0061] After the main thread determines that all child threads have successfully inserted data, it also needs to notify all child threads to commit the data in the transaction. Since data insertion and data submission are two different processes, data insertion is the process of copying data from the source table to the target table, which includes copying data rows from one table to another, but does not immediately permanently save these data rows to the target table. And data submission is to officially write the inserted data rows into the target table to ensure the persistence of these data in the database. This step is usually irreversible. Once submitted, the changes to these data cannot be revoked. The data submission operation marks the completion of a transaction, ensuring that all changes made in the transaction are permanently saved to the database.

[0062] After the child threads execute data insertion and before data submission, these data are not permanently saved in the target table. If the database crashes abnormally, these data will disappear. Therefore, the physical log and the logical log can be combined for data recovery. Specifically, if the database crashes abnormally before a transaction is committed, it will first check whether the physical log of the transaction exists. If it exists, it will further check whether the corresponding logical log of the transaction exists. If both the physical log and the logical log exist, then read the latest data page change information from the physical log and restore it to the data file on the disk. Then read the operation records of the transaction from the logical log and apply these operations to the data in the database. After data recovery, re-perform data submission for multiple child threads. After the data submission is successful, feedback a success message to the terminal.

[0063] In this application, after data insertion is successful and before data submission, if the database crashes abnormally, then data recovery is implemented according to the physical log and the logical log to avoid inconsistent contents between the source table and the target table, and this application improves the data consistency in the database.

[0064] As an optional implementation manner, generating the logical log includes: obtaining the log cache generated by each child thread; merging the log caches of each child thread into the main thread, where the main thread also acts as a child thread; and writing the merged log cache into the logical log.

[0065] After each sub - thread inserts the data it is responsible for from the source table into the target table, it will generate a corresponding log cache. The log cache is a mechanism for temporarily storing transaction log information. During the transaction processing, the operations of each sub - thread will be recorded in the log cache. These log messages include the start time, end time, operation type of the transaction, and information about the relevant data rows. To ensure data consistency and integrity, it is necessary to merge and synchronize the operations of these sub - threads. The main thread will regularly merge the log caches of each sub - thread and write the merged log information into the logical log. In this way, it can ensure the final data consistency and integrity and provide rollback and recovery functions when necessary.

[0066] As an alternative implementation, before writing the logical log into the cache of the database, the method further includes: if the database crashes abnormally, notify all sub - threads to roll back according to the physical log; send a failure message to the terminal, where the failure message is used to indicate that the insert - select concurrent import within the database transaction fails.

[0067] After the main thread merges the log caches of multiple sub - threads, if the logical log has not been written into the database cache at this time, it means that these log records have not been persistently saved. In this case, if the database crashes abnormally, these logical log records will be lost. At this time, restoring the data will result in incomplete data. To ensure data consistency and integrity, it is necessary to notify all sub - threads to perform rollback operations according to the physical log, return to the initial step of splitting the data into multiple data intervals, and feedback a failure message to the terminal, notifying the terminal that the insert - select concurrent import within the database transaction fails, so that the personnel at the terminal can take timely measures for processing and optimization.

[0068] As an alternative implementation, after generating the physical log and the logical log, the method further includes: if there is a situation where a sub - thread fails to insert data, notify all sub - threads to roll back according to the physical log; send a failure message to the terminal, where the failure message is used to indicate that the insert - select concurrent import within the database transaction fails.

[0069] If there is a situation where some sub - threads insert successfully and some sub - threads insert fail, or if all sub - threads insert fail, to ensure data consistency and integrity, it is necessary to notify all sub - threads to perform rollback operations according to the physical log, return to the initial step of splitting the data into multiple data intervals, and feedback a failure message to the terminal, notifying the terminal that the insert - select concurrent import within the database transaction fails, so that the personnel at the terminal can take timely measures for processing and optimization.

[0070] As an alternative implementation, the database is a single-point database; after writing the logical log into the cache of the database, the method further includes: locking the target table, where the locking is used to block the access of other transactions to the target table during the data submission process; after determining that all child threads have finished submitting, the method further includes: unlocking the target table.

[0071] There is only one data node in the single-point database, and the data within a transaction is also only on one data node, which means that all data is stored in a single physical location. In a single-point database, all read and write operations need to be completed through the same node, which requires this node to be able to handle a large number of concurrent requests and ensure data consistency and integrity. Therefore, some measures need to be taken to prevent the access of other transactions to the target table from interfering with the processing of the current transaction.

[0072] After the main thread notifies all child threads to insert data, the logical log is written into the database cache, and then the target table in the single-point database is locked to block the access of other transactions to the target table during the submission process, so as to ensure the isolation of the transaction and avoid data conflict and inconsistency problems. After the main thread determines that all child threads have successfully inserted data, it notifies all child threads to perform the submission action. After the child threads complete the submission, they notify the main thread, and the main thread releases the lock on the target table. At this time, other transactions can access the data inserted by the insert select transaction.

[0073] Before the child threads submit data, by locking the target table in the single-point database, the isolation and consistency of the insert select transaction can be ensured, and it is guaranteed that different transactions will not interfere with each other, thereby improving the stability and reliability of the database system.

[0074] As an alternative implementation, during the process of merging the log cache of each child thread into the main thread, the method further includes: recording the position of the log cache corresponding to each child thread, where the position is used to record the location of copying the log cache; merging the log caches according to the order of the positions.

[0075] The position is used to identify the starting position when copying incremental data from the log cache of the child thread each time. The purpose of doing this is to ensure that when the main thread merges the log cache, it can accurately know which data is included in the log cache of each child thread, thereby avoiding duplicate copying and data duplication problems.

[0076] By recording the positions, the main thread can, when merging the log cache, merge the data in the log caches of each child thread in the correct order according to the position information, which can ensure that the order of data changes recorded in the logical log is consistent with the actual situation, thereby improving the stability and reliability of the database system.

[0077] Figure 3 A flowchart of a method for parallel insertion of insert select within a database transaction provided by an embodiment of the present application is as follows Figure 3 shown, and includes the following steps.

[0078] Step 301: Multiple child threads within the same transaction concurrently insert data and generate physical logs and logical logs.

[0079] Step 303: The main thread writes the logical logs into the database cache.

[0080] Step 305: The main thread determines whether the data insertions of all child threads are all successful. If all are successful, then execute Step 307; if not all are successful, then execute Step 309.

[0081] Step 307: Notify all child threads to synchronously commit the data.

[0082] Step 309: Notify all child threads to roll back.

[0083] Among them, before Step 303, if a database abnormal shutdown occurs, then execute Step 309.

[0084] Among them, before Step 307, if a database abnormal shutdown occurs, then execute Step 306: Perform data recovery according to the physical logs and logical logs.

[0085] In the present application, the transaction has atomicity, that is, the transaction is regarded as an indivisible minimum unit. The operations in the transaction are either all executed or all not executed to ensure data integrity. The insert select operation of the database can be part of the transaction, that is, the final insert result is either all threads insert successfully or all threads insert fail, and it is not allowed that some threads insert successfully and some threads insert fail. Therefore, it is necessary for the main thread to coordinate and control the insertions and submissions of each child thread to ensure that the insertions and submissions of all threads are executed as a whole.

[0086] A transaction also has isolation. When multiple users access the database concurrently, for example, when operating on the same table, the database starts a transaction for each user, and the operations of other transactions cannot interfere before the transaction is committed. Multiple concurrent transactions need to be isolated from each other. From the perspective of optimal submission efficiency, the main thread and child threads of the insert select operation need to submit simultaneously. Since there is a sequence in which each thread completes the submission action, it is possible that the data submitted by some threads before the main thread responds to the client is accessed by other transactions, resulting in the mixing of data between transactions. Therefore, before the insert select is submitted, a lock needs to be added to the target table to block other transactions from accessing the data in the target table, and the lock is released after all the threads of the insert select have completed the submission.

[0087] Data has consistency. If the database crashes abnormally before a transaction is committed, it will first check whether the physical log of the transaction exists. If it exists, it will further check whether the corresponding logical log of the transaction exists (we refer to the identification information in the logical log as XID, which is used to uniquely identify the transaction, and XID exists in both the physical log and logical log corresponding to the transaction). If it exists, the transaction will be finally committed, otherwise it will be rolled back.

[0088] Figure 4 A signaling diagram for parallel insertion of insert select within a database transaction provided by an embodiment of the present application is as Figure 4 shown and includes the following steps.

[0089] Step 1: The terminal sends an insert select request to the main thread of the database.

[0090] Step 2: After the main thread determines the data range, it creates multiple child threads.

[0091] Step 3: The child threads perform data insertion.

[0092] Step 4: The main thread merges the log caches of the child threads and records the positions.

[0093] Step 5: Write the merged log cache into the logical log and generate the physical log.

[0094] Step 6: Lock the target table.

[0095] Step 7: The child threads submit the data.

[0096] Step 8: The main thread releases the lock on the target table.

[0097] Step 9: Feedback the result of parallel insertion to the terminal.

[0098] Based on the same technical concept, such as Figure 5 shown, this application provides a data recovery device for parallel insertion of insert select within a database transaction. The device includes:

[0099] A generation module 501, configured to generate physical logs and logical logs after concurrently inserting data from a source table into a target table through multiple sub-threads within the same transaction in the database, and write the logical logs into the cache of the database;

[0100] A submission module 502, configured to notify all sub-threads to commit the data in the transaction if it is determined that all sub-threads have successfully inserted data;

[0101] A recovery module 503, configured to perform data recovery based on the physical logs and logical logs if the database crashes abnormally when all sub-threads have not completed the submission, and re-perform data submission after the data recovery.

[0102] Optionally, the generation module 501 is configured to:

[0103] Obtain the log cache generated by each sub-thread;

[0104] Merge the log caches of each sub-thread into the main thread, where the main thread also serves as a sub-thread;

[0105] Write the merged log cache into the logical log.

[0106] Optionally, the device is further configured to:

[0107] If the database crashes abnormally, notify all sub-threads to roll back according to the physical logs;

[0108] Send a failure message to the terminal, where the failure message is used to indicate that the concurrent import of insert select within the database transaction fails.

[0109] Optionally, the device is further configured to:

[0110] If there is a situation where a sub-thread fails to insert data, notify all sub-threads to roll back according to the physical logs;

[0111] Send a failure message to the terminal, where the failure message is used to indicate that the concurrent import of insert select within the database transaction fails.

[0112] Optionally, the database is a single-point database;

[0113] The device is further configured to: after writing the logical log into the cache of the database, lock the target table, where the locking is used to block the access of other transactions to the target table during the data submission process; and unlock the target table after determining that all child threads have finished submitting.

[0114] Optionally, the device is further configured to:

[0115] Record the position of the log cache corresponding to each child thread, where the position is used to record the location of copying the log cache;

[0116] Merge the log caches according to the order of the positions.

[0117] As Figure 6 shown, an embodiment of the present application provides an electronic device, including a processor 601, a communication interface 602, a memory 603, and a communication bus 604, where the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0118] The memory 603 is used to store a computer program.

[0119] In an embodiment of the present application, when the processor 601 is configured to execute the program stored on the memory 603, it implements the data recovery method for parallel insert of insert select within a database transaction provided in any of the foregoing method embodiments, including:

[0120] After concurrently inserting the data in the source table into the target table through multiple child threads within the same transaction in the database, generating physical logs and logical logs, and writing the logical logs into the cache of the database;

[0121] If it is determined that all child threads have successfully inserted data, notify all child threads to submit the data in the transaction;

[0122] If the database crashes abnormally when all child threads have not finished submitting, perform data recovery based on the physical logs and logical logs, and re-perform data submission after the data recovery.

[0123] Optionally, generating the logical log includes:

[0124] Obtain the log cache generated by each child thread;

[0125] Merge the log caches of each child thread into the main thread, where the main thread is also regarded as a child thread;

[0126] Write the merged log cache into the logical log.

[0127] Optionally, before writing the logical log into the cache of the database, the method further includes:

[0128] If the database crashes abnormally, all child threads are notified to roll back according to the physical log;

[0129] A failure message is sent to the terminal, where the failure message is used to indicate the failure of concurrent import of insert select within the database transaction.

[0130] Optionally, after generating the physical log and the logical log, the method further includes:

[0131] If there is a situation where a child thread fails to insert data, all child threads are notified to roll back according to the physical log;

[0132] A failure message is sent to the terminal, where the failure message is used to indicate the failure of concurrent import of insert select within the database transaction.

[0133] Optionally, the database is a single-point database;

[0134] After writing the logical log into the cache of the database, the method further includes: locking the target table, where the locking is used to block the access of other transactions to the target table in the data submission process;

[0135] After determining that all child threads have finished committing, the method further includes: unlocking the target table.

[0136] Optionally, in the process of merging the log caches of each child thread into the main thread, the method further includes:

[0137] Recording the position of the log cache corresponding to each child thread, where the position is used to record the position of copying the log cache;

[0138] Merging the log caches according to the order of the positions.

[0139] The embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the data recovery method for parallel insertion of insert select within a database transaction provided by any one of the foregoing method embodiments are implemented.

[0140] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0141] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the relevant technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiments.

[0142] It should be understood that the terms used herein are for the purpose of describing specific example embodiments only and are not intended to be limiting. Unless the context clearly indicates otherwise, as used herein, the singular forms "a", "an", and "the" may also include the plural forms. The terms "comprising", "including", "containing", and "having" are inclusive and thus specify the presence of the stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or their combinations. The method steps, processes, and operations described herein are not to be construed as necessarily requiring them to be performed in the particular order described or illustrated, unless the order of performance is explicitly stated. It should also be understood that additional or alternative steps may be used.

[0143] The above are only specific embodiments of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A data recovery method for insert select parallel insertion in a database transaction, characterized in that: The method comprises: After concurrently inserting the data in the source table into the target table through multiple sub-threads in the same transaction in the database, a physical log and a logical log are generated, and the logical log is written into the cache of the database; If it is determined that all child threads have successfully inserted data, then all child threads are notified to submit the data in the transaction; If the database crashes abnormally before all sub-threads are submitted, data recovery is performed according to the physical log and the logical log, and data is resubmitted after data recovery.

2. The method according to claim 1, characterized in that Generating logical logs includes: Get the log cache generated by each child thread; Merge the log cache of each child thread into the main thread, where the main thread also serves as a child thread; Write the merged log buffer to the logical log.

3. The method according to claim 1, characterized in that Before writing the logical log into the cache of the database, the method further includes: If the database crashes abnormally, all child threads are notified to roll back according to the physical log; A failure message is sent to the terminal, wherein the failure message is used to indicate that the insertselect concurrent import within the database transaction has failed.

4. The method according to claim 1, characterized in that After generating the physical log and the logical log, the method further includes: If there is a situation where a child thread fails to insert data, all child threads are notified to roll back according to the physical log; A failure message is sent to the terminal, wherein the failure message is used to indicate that the insertselect concurrent import within the database transaction has failed.

5. The method according to claim 1, characterized in that The database is a single-point database; After writing the logic log into the cache of the database, the method further includes: locking the target table, wherein the locking is used to block other transactions from accessing the target table in the transaction during the data submission process; After determining that all sub-threads have completed submission, the method further includes: unlocking the target table.

6. The method according to claim 1, characterized in that In the process of merging the log cache of each child thread into the main thread, the method further includes: Record the location of the log cache corresponding to each child thread, wherein the location is used to record the location where the log cache is copied; The log buffers are merged according to the order of the sites.

7. A data recovery device for insert select parallel insertion in a database transaction, characterized in that: The device comprises: A generation module, used for concurrently inserting data in a source table into a target table through multiple sub-threads in the same transaction in a database, generating a physical log and a logical log, and writing the logical log into a cache of the database; A submission module, used for notifying all sub-threads to submit the data in the transaction if it is determined that all sub-threads have successfully inserted data; The recovery module is used to recover data according to the physical log and the logical log if the database crashes abnormally before all sub-threads are submitted, and to resubmit the data after the data is recovered.

8. The device according to claim 7, characterized in that The device is also used for: Record the location of the log cache corresponding to each child thread, wherein the location is used to record the location where the log cache is copied; The log buffers are merged according to the order of the sites.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-6 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images