Data processing method and apparatus, and system

Abstract

Provided in the embodiments of the present disclosure are a data processing method and apparatus, and a system, the method comprising: executing a target transaction by means of a first coroutine associated with a worker thread, so as to obtain a transaction log of the target transaction; sending a transaction commit task of the target transaction to a log storage thread, and, by means of suspending the first coroutine, releasing the worker thread, the log storage thread being used for storing the transaction log; and when storage information returned by the log storage thread is received, performing a recovery operation on the first coroutine, and, by means of the first coroutine, resuming execution of the transaction commit task. The present invention meets the requirements of high-concurrency transaction request scenarios and avoids the occurrence of multi-thread lock contention, thus improving database throughput, and simultaneously improving thread execution efficiency and allowing databases to maintain good write performance.

Description

Data processing method, system and device TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of computer, and particularly relate to a data processing method, system and device. BACKGROUND

[0002] With the continuous development of Internet technology, databases have been widely used in various project fields. In order to ensure the reliability of data operation in the case of high concurrency and system failure, the database management system needs to follow the ACID principle. The persistence of transaction log is a key step to ensure that the database meets the persistence requirement in the ACID principle. The database management system writes the transaction log into the disk to complete the log landing operation, thereby realizing the persistence of the transaction log. However, the log landing operation is very time-consuming, so the number of concurrent threads is often increased to solve this problem. However, this method can cause lock competition, CPU contention and other situations, resulting in poor thread execution efficiency and reducing the write performance of the database. Therefore, in the high concurrency scenario, how to improve the database write throughput and avoid the increase of thread scheduling overhead is a problem that needs to be solved at present. SUMMARY

[0003] Therefore, the embodiments of the present disclosure provide a data processing method. One or more embodiments of the present disclosure also provide a data processing system, a data processing device, a computing device, a computer readable storage medium and a computer program product to solve the technical defects in the prior art.

[0004] According to a first aspect of the embodiments of the present disclosure, a data processing method is provided, applied to a worker thread in a database management system, comprising:

[0005] executing a target transaction by a first coroutine associated with the worker thread, obtaining a transaction log of the target transaction;

[0006] sending a transaction commit task of the target transaction to a log storage thread, and releasing the worker thread by suspending the first coroutine, wherein the log storage thread is configured to store the transaction log;

[0007] in a case where storage information returned by the log storage thread is received, performing a recovery operation on the first coroutine, and continuing to execute the transaction commit task by the first coroutine.

[0008] According to a second aspect of the embodiments of the present disclosure, a database management system is provided, comprising a worker thread and a log storage thread, wherein,

[0009] The work thread is configured to execute a target transaction through a first coroutine, obtain a transaction log of the target transaction, send a transaction commit task of the target transaction to a log storage thread, and release the work thread by suspending the first coroutine.

[0010] The log storage thread is configured to store the transaction log, obtain storage information, and return the storage information to the work thread.

[0011] The work thread is configured to perform a recovery operation on the first coroutine according to the storage information, and continue to execute the transaction commit task through the first coroutine.

[0012] According to a third aspect of the embodiments of the present disclosure, a data processing apparatus is provided, which is applied to a work thread in a database management system, and includes:

[0013] An execution module is configured to execute a target transaction through a first coroutine associated with the work thread, and obtain a transaction log of the target transaction.

[0014] A suspension module is configured to send a transaction commit task of the target transaction to a log storage thread, and release the work thread by suspending the first coroutine, wherein the log storage thread is configured to store the transaction log.

[0015] A recovery module is configured to perform a recovery operation on the first coroutine according to storage information returned by the log storage thread, and continue to execute the transaction commit task through the first coroutine.

[0016] According to a fourth aspect of the embodiments of the present disclosure, a computing device is provided, which includes:

[0017] A memory and a processor.

[0018] The memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions, so as to implement the steps of the above data processing method.

[0019] According to a fifth aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, which stores computer executable instructions, and the instructions are executed by a processor to implement the steps of the above data processing method.

[0020] According to a sixth aspect of the embodiments of the present disclosure, a computer program product is provided, which includes a computer program or instructions, and the computer program or instructions are executed by a processor to implement the steps of the above data processing method.

[0021] One embodiment of the present disclosure is based on a work thread in a database management system, and a target transaction is executed by a first coroutine associated with the work thread when scheduling a transaction request, so that the transaction request is scheduled in a coroutine manner, and the scheduling overhead of the operating system for the thread is reduced. When performing a transaction log landing operation, a transaction submission task of the target transaction is sent to a log storage thread, and the first coroutine is suspended to release the work thread, so that the released work thread can be switched to other connections or transactions for processing when the log storage thread performs the log landing operation, so as to meet the needs of a high-concurrency transaction request scenario, avoid the occurrence of multi-thread lock competition, improve the database throughput, and also improve the thread execution efficiency, so that the database has good write performance. BRIEF DESCRIPTION OF DRAWINGS

[0022] FIG. 1 shows an execution framework diagram of a database management system in a data processing method according to one embodiment of the present disclosure;

[0023] FIG. 2 shows a flowchart of a data processing method according to one embodiment of the present disclosure;

[0024] FIG. 3 shows a flowchart of a processing process of a data processing method according to one embodiment of the present disclosure;

[0025] FIG. 4 shows a structural diagram of a data processing system according to one embodiment of the present disclosure;

[0026] FIG. 5 shows a structural diagram of a data processing apparatus according to one embodiment of the present disclosure;

[0027] FIG. 6 shows a structural block diagram of a computing device according to one embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, the present disclosure can be practiced without the specific details, other than those described herein, and it is understood that the present disclosure is not limited to the embodiments described herein and can be practiced with or without the specific details. Therefore, the present disclosure is not limited to the specific details and should not be construed as being limited to such specific details.

[0029] The terminology used in one or more embodiments of the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of one or more embodiments of the present disclosure. As used in one or more embodiments of the present disclosure and the accompanying claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in one or more embodiments of the present disclosure, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0030] It should be understood that although the terms first, second, etc. can be employed in describing various information in one or more embodiments of the present disclosure, the information should not be limited to these terms. These terms are only used to distinguish one type of information from another type of information. For example, without departing from the scope of one or more embodiments of the present disclosure, first can also be referred to as second, and similarly, second can also be referred to as first. Depending on the context, the word "if" as used herein can be interpreted as "when" or "upon" or "in response to determining".

[0031] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in one or more embodiments of the present disclosure are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0032] First, the nomenclature involved in one or more embodiments of the present disclosure is explained.

[0033] PolarDB MySQL: PolarDB MySQL is a relational database management system developed independently based on MySQL. It not only optimizes and enhances MySQL, but also introduces many innovative technologies to improve performance, scalability and availability. In the embodiments of the present disclosure, PolarDB MySQL is selected as the database management system.

[0034] InnoDB: InnoDB is the default storage engine of MySQL database, as one of the core components of modern databases, it plays an important role in many application scenarios with its rich features and powerful functions. InnoDB is a transaction-safe (ACID compliant), row-level locking and foreign key constraint storage engine, especially suitable for application scenarios that require high reliability, transaction support and concurrent performance. In the embodiments of the present disclosure, based on the transaction support provided by the InnoDB transactional storage engine, transaction operations are performed to ensure the consistency, integrity and reliability of database data.

[0035] Thread: Thread is a thread in the database management system, which is used for internal tasks such as log disk or external user requests such as foreground user threads. In the embodiments of the present disclosure, thread can include worker threads, log storage threads, listener threads, etc.

[0036] Thread Pool: Thread Pool is a thread pool module in a database management system that can manage multiple threads simultaneously and execute user requests concurrently. Thread pooling is an optimization technique designed to improve server performance and resource utilization, especially in high-concurrency environments. Thread pools manage a group of pre-created threads to handle client requests, rather than creating a new thread for each new connection, reducing the overhead of thread creation and destruction, and improving system response speed and throughput.

[0037] Coroutine: Coroutine is a lightweight execution unit that allows concurrent operations within a single thread. Compared to traditional threads, coroutines have lower switching overhead and can share resources more efficiently between different parts of a program. In the embodiments of the present disclosure, coroutines are used for transaction management, such as transaction execution, commit, and rollback.

[0038] Connection: In a database management system, Connection refers to a communication connection established between a client application and a database. Through this connection, the client can send SQL query commands to the database and receive the execution results. In the embodiments of the present disclosure, connection refers to the connection between the client application and the database, through which the application can send SQL statements to the database to perform corresponding transactions and complete data operations.

[0039] Transaction: Transaction is an operation on data in a database. In a database management system, a transaction is an execution unit composed of one or more SQL statements, which are processed as a whole, either all successfully or not at all. Transactions are commonly used to ensure data consistency and integrity. A transaction can include various operations on a database, such as data reading or data writing.

[0040] ACID Principle: ACID Principle refers to the four basic principles that a database management system's transaction must meet to ensure the reliability of data operations:

[0041] Atomicity: All operations of a transaction must be completed or not executed at all, there is no partial completion state. If an error occurs during transaction execution, the entire transaction will be rolled back, as if the transaction never started.

[0042] Consistency: A transaction must change the database from one consistent state to another. That is, before and after the transaction, the integrity constraints of the database are not violated, and all rules must be applied to the transaction's modifications to maintain the consistent state of the database.

[0043] Isolation: multiple transactions executing concurrently are isolated from each other, and do not affect each other. That is, the operations and data used within a transaction are isolated from other concurrent transactions, and concurrent transactions do not interfere with each other.

[0044] Durability: once a transaction is committed, its changes to the data in the database are permanent, and should not be affected even if the system fails. Even if the system crashes or encounters hardware problems, committed data will not be lost.

[0045] At present, based on the ACID characteristics of the transaction, the transaction log must be persisted, that is, written to the disk when the transaction is committed, however, the log disk operation is very time-consuming, in order to improve the database write throughput, it is often solved by increasing the write concurrency threads, but increasing the concurrency threads can bring other side effects, such as lock sys, trx sys and other non-IO scenarios of lock competition, CPU contention under high concurrency, frequent context switching, resulting in poor thread execution efficiency, and unable to obtain good write performance of the database.

[0046] Therefore, in the present disclosure, a data processing method is provided, and the present disclosure simultaneously relates to a data processing system, a data processing device, a computing device, a computer readable storage medium and a computer program product, which are described in detail one by one in the following embodiments.

[0047] Referring to FIG. 1, FIG. 1 shows an execution framework diagram of a database management system in a data processing method according to an embodiment of the present disclosure. In the execution framework of the database management system, a plurality of core components are included, such as Group and Timer thread. Group represents a thread group, which includes a low-priority queue, a high-priority queue, and an asynchronous preparation queue. These queues are used to store connections, and a worker thread is scheduled and executed according to the priority of the queue. The priority of the three queues is in descending order, i.e., asynchronous preparation queue, high-priority queue, and low-priority queue. The worker thread represents a worker thread that executes user requests. In the embodiment of the present disclosure, the worker thread is used to process transaction operations corresponding to the connection. A thread group can include multiple worker threads, and each worker thread can concurrently execute multiple transaction operations of the connection according to the execution strategy. The coroutine cache pool is used to pre-allocate coroutine objects to execute asynchronous requests. The listener thread is used to listen to IO events, such as whether there is a new request or a log landing event. The timer thread is used to execute background timing tasks, such as detecting connection timeout, whether the current thread number meets the expectation, waking up or creating additional threads, etc. Through the execution framework of the database management system mentioned above, the data processing method provided by the present disclosure is executed, so that based on the worker thread in the database management system, the target transaction is executed by the first coroutine associated with the worker thread when scheduling the transaction request, the transaction request is scheduled in the coroutine mode, the scheduling overhead of the operating system for the thread is reduced. When performing the transaction log landing operation, the transaction submission task of the target transaction is sent to the log storage thread, and the first coroutine is suspended to release the worker thread, so that the released worker thread can be switched to other connections or transactions for processing when the log storage thread performs the log landing operation, meeting the demand of high-concurrency transaction request scenario, avoiding the occurrence of multi-thread lock competition, improving the database throughput and also improving the thread execution efficiency, so that the database has good write performance.

[0048] Referring to FIG. 2, FIG. 2 shows a flowchart of a data processing method according to an embodiment of the present disclosure, which specifically includes the following steps.

[0049] Step 202: executing the target transaction through the first coroutine associated with the worker thread to obtain the transaction log of the target transaction.

[0050] The work thread can be understood as a certain work thread worker thread in the thread group, and the work thread is used to execute a transaction request sent by an application program to a database. In order to cope with a high concurrency scenario, the work thread can execute a transaction request through a coroutine when executing the transaction request. In actual implementation, a corresponding coroutine is created for each connection, so after a target transaction of a connection is determined, the target transaction can be executed through a first coroutine associated with the work thread. The first coroutine is the coroutine corresponding to the connection, and the target transaction is the transaction corresponding to the connection. After the target transaction is executed through the first coroutine associated with the work thread, a transaction log corresponding to the target transaction can be obtained. The transaction log can be understood as a redo log of the target transaction. In the transaction mechanism provided by the InnoDB storage engine, in order to ensure the ACID principle of the transaction, the transaction log of the target transaction needs to be written into a storage system, that is, the log is landed.

[0051] In actual application, when an application program of a client performs data operation on a database, a connection request is sent to a database management system. Through the connection between the application program of the client and the database, the application program can send an SQL statement based on the connection, so as to implement corresponding operation on the data stored in the database.

[0052] After the connection is established, the existing database processing mode usually allocates a corresponding thread for the connection, which is used to execute and process all operations on the connection. In this processing mode, if a certain operation is blocked, such as waiting for an I / O operation to complete, the entire thread will be blocked until the operation is completed and other tasks can continue to be processed. This processing mode has several obvious limitations: high resource consumption, large context switching overhead, and large blocking impact. High resource consumption can be understood as allocating an independent work thread for each client connection will cause a large amount of memory and CPU resources to be occupied, especially when facing a large number of concurrent connections, the system may run out of resources due to the creation of too many threads. Large context switching overhead can be understood as when there are a large number of threads running simultaneously, the operating system needs to frequently switch contexts between threads, which will bring additional performance overhead and reduce the overall efficiency of the system. Large blocking impact can be understood as since the thread cannot process other tasks when encountering a blocking operation, even if there are available CPU resources in the system, these resources cannot be utilized to improve throughput or response speed.

[0053] Based on this, in the embodiments of the present disclosure, when a worker thread processes a transaction request corresponding to a connection, a coroutine is selected and allocated to the worker thread, and the transaction request is processed through the coroutine. In contrast, the worker thread model using the coroutine can significantly overcome the above problems. The coroutine is lightweight, multiple coroutines can be run simultaneously in a single worker thread, and the overhead of creation and destruction is much smaller than that of a thread. When a coroutine needs to wait, it can actively yield execution rights to other coroutines, so that the same worker thread can continue to process other tasks, thereby improving resource utilization and the concurrent processing capability of the system. The coroutine provides a more intuitive way to write asynchronous code. Developers can implement complex control flow by suspending and resuming coroutines without relying on complex mechanisms such as callback functions. In this way, the system can significantly improve the ability to process large-scale concurrent connections while reducing resource consumption.

[0054] Further, in order to call the coroutine to execute the transaction request corresponding to the connection, the coroutine needs to be allocated and associated with the connection. Specifically, before the first connection is associated with the target transaction, the method further comprises: obtaining a first connection and determining a target transaction corresponding to the first connection; creating an initial coroutine, associating the initial coroutine with the first connection, and obtaining a first coroutine.

[0055] The first connection can be understood as a connection sent by a client application to a database. Through the connection, communication can be performed between the client and the database, so that the application can send subsequent SQL statements corresponding to transactions to the database based on the connection. After the database management system obtains the first connection, it can determine the transaction corresponding to the connection, i.e., the target transaction. The first connection can include multiple transactions to perform different data operations. In order to process tasks in a high-concurrency scenario, the database management system can allocate a corresponding coroutine to the first connection for processing, thereby avoiding the problem of transaction operation waiting and thread blocking.

[0056] In actual application, the database management system will allocate connection requests to one of the multiple thread groups according to the scheduling strategy, and the thread group will call one of the worker threads to respond to the connection request. The worker thread can obtain a connection connection that is ready to execute an SQL statement through a get_even function, and then select a coroutine from a coroutine cache pool and associate the coroutine with the connection. The transaction operation included in the connection is executed through the associated coroutine. It should be noted that the database management system also involves a timer thread for executing background timing tasks, such as detecting connection timeout, whether the current number of threads meets the expectation, waking up or creating additional threads, etc. Thus, the timer thread cooperates with the scheduling strategy to realize the scheduling and allocation of connections, thereby improving the throughput of the database.

[0057] In an embodiment of the present disclosure, the worker thread obtains a connection connection that is ready to execute a SQL statement through the get_even function, and the connection is a first connection. An initial coroutine is created by calling the tp_handle_request_wrapper function, and the initial coroutine is associated with the first connection through the conn_task_prepare function. The associated coroutine is used as a first coroutine corresponding to the first connection. It should be noted that the entry function of the first coroutine can be determined during the binding process. The entry function is the first function executed when the first coroutine executes the subsequent logic, such as a query, insertion, update, or other SQL statement. The worker thread can start executing the coroutine logic through the conn_task_run function, thereby completing the corresponding transaction operation.

[0058] Based on this, by allocating a connection to a bound coroutine, the subsequent transaction operation contained in the connection is executed through the coroutine, which avoids the situation that the worker thread is blocked when the transaction operation is waiting, and improves the thread execution efficiency.

[0059] Further, in order to normally use the asynchronous processing function, it is also necessary to determine the global variable corresponding to the connection. Specifically, the method further includes: determining an asynchronous execution variable corresponding to the first connection; if the asynchronous execution variable is on, continuing to execute the step of creating an initial coroutine; and if the asynchronous execution variable is off, executing the target transaction according to a synchronous strategy.

[0060] The asynchronous execution variable can be understood as a global variable corresponding to the asynchronous commit function. By exposing the dynamic switch of the function, the user can dynamically open or close the asynchronous commit function of the transaction through the SQL command. The asynchronous execution variable can be a global variable set global polar_async_commit set in the database. By changing the value of the asynchronous execution variable, the asynchronous commit function of the transaction can be dynamically opened or closed. Correspondingly, after obtaining the connection, the asynchronous execution variable can be used to determine how the worker thread processes the transaction of the connection. If the asynchronous commit function is determined to be opened according to the asynchronous execution variable, the transaction needs to be executed through the coroutine. If the asynchronous commit function is determined to be closed according to the asynchronous execution variable, the worker thread is directly responsible for executing and processing the transaction operation on the connection.

[0061] In a specific implementation, for an existing connection, before executing a next request, it is detected whether the global variable is changed. If not, the original strategy is continued to be executed. If changed, the new setting is saved in the connection object. In the case of starting the asynchronous submission function, if the strategy of the existing connection is the synchronous strategy, the coroutine object is applied, and the working thread changes the strategy to the asynchronous strategy to execute the transaction request of the connection, that is, the step of creating the initial coroutine is continued to be executed. In the case of closing the asynchronous submission function, if the strategy of the existing connection is the asynchronous strategy, the coroutine object is released, and the working thread changes the strategy to the synchronous strategy. The synchronous strategy is that the working thread is directly responsible for executing and processing the transaction operation on the connection. The above dynamic starting or closing of the asynchronous submission function has no effect on the newly created connection. However, it needs to be noted that in the dynamic closing process, the suspended coroutine object is actively woken up and returned, and it is detected whether the log landing is completed. If not, the working thread continues to execute the synchronous log landing and waits.

[0062] In a specific implementation of the present disclosure, after the connection is acquired, the working thread determines the asynchronous execution variable corresponding to the current connection. The current asynchronous execution variable set global polar_async_commit=on indicates that the asynchronous submission function is started. The working thread executes the step of creating the initial coroutine, associates and binds the coroutine for the connection, and asynchronously processes the transaction request of the connection through the coroutine.

[0063] Therefore, by changing the asynchronous execution variable, the asynchronous submission function is dynamically started and closed, so that the user can select the synchronous execution or the asynchronous execution of the transaction request of the connection according to the actual situation, and the user can adjust the execution strategy of the database according to the actual situation.

[0064] Further, in order to execute the target transaction according to the first coroutine, it is necessary to determine the execution logic function of the first coroutine. Specifically, the transaction log of the target transaction is obtained by executing the target transaction through the first coroutine associated with the working thread, including: determining the execution information of the first coroutine associated with the working thread; and executing the target transaction through the first coroutine according to the execution information to obtain the transaction log of the target transaction.

[0065] The execution information of the first coroutine can be understood as the first function, that is, the entry function, of the first coroutine when executing the target transaction. By determining the execution information of the first coroutine, the target transaction is executed according to the execution information subsequently.

[0066] In actual application, after the association between the connection and the coroutine is made, the coroutine may involve a newly built operation when executed, or the coroutine has been built and can directly execute the SQL statement, so it is necessary to determine the entry function of the coroutine, and the subsequent coroutines execute the target transaction according to the entry function, so as to obtain the transaction log of the target transaction. The transaction log can be understood as the log generated by the operation corresponding to the SQL statement executed by the coroutine. Under the transaction mechanism provided by the InnoDB storage engine, once a transaction is committed, the changes made by it are permanent, and these changes will not be lost even if the system fails. In order to ensure this, the database system usually writes the transaction log redo log to the disk, and the transaction log is a combination of log entries of data operations corresponding to each SQL statement in the transaction operation.

[0067] In summary, the first coroutine executes the target transaction through the determined execution information, i.e., the entry function, and generates the corresponding transaction log, which facilitates subsequent rollback or recovery operations based on the transaction log, thereby ensuring the integrity and consistency of data in the database.

[0068] Further, the first coroutine executes the target transaction according to the execution information to obtain the transaction log of the target transaction, including: the first coroutine executes the target transaction according to the execution information to obtain the execution record corresponding to the execution information; and generating the transaction log of the target transaction according to the execution record.

[0069] The execution record corresponding to the execution information can be understood as the execution record corresponding to the operation of the coroutine in the process of executing the target transaction, which can be the execution record corresponding to the SQL statement, and the transaction log of the target transaction can be generated through the execution record.

[0070] In actual application, the transaction log is a redo log, and the main role of the transaction log is to recover the unfinished transaction after the system crashes or abnormally closes, to ensure the consistency and integrity of the data. The redo log records all physical changes (such as insert, update or delete operations) made to the database. These log entries are used to reapply (i.e., "redo") changes that have not yet been permanently saved to data files when the database is restarted, thereby ensuring the durability of transactions. When a transaction is executed, the relevant redo log record is written to the redo log buffer before the transaction is committed, and is flushed from the buffer to the redo log file on the disk at the appropriate time.

[0071] In an embodiment of the present disclosure, the first coroutine executes the target transaction according to the execution information, generates a corresponding redo log record when executing the first update statement, and writes the redo log record into a redo log buffer. When executing the second update statement, a redo log record is also generated and written into the buffer. Subsequently, when the transaction is committed, the log record corresponding to the transaction in the redo log buffer is stored into a disk, that is, the log landing operation is completed.

[0072] Based on this, by generating the transaction log of the target transaction according to the execution record, the database can recover the unfinished transaction in the case of crash or exception, and ensure the consistency and integrity of the data.

[0073] Step 204: sending a transaction commit task of the target transaction to a log storage thread, and releasing the worker thread by suspending the first coroutine, wherein the log storage thread is used for storing the transaction log.

[0074] The transaction commit task of the target transaction can be understood as a task of committing the target transaction. After the execution of the target transaction is completed, the transaction commit of the target transaction is needed, and at this time, the landing operation of the transaction log of the target transaction is needed. The worker thread sends the transaction commit task to the log storage thread. The log storage thread can be understood as a thread for storing the transaction log.

[0075] In actual application, when the transaction is committed, it is necessary to ensure that the transaction log is written into the storage system. At this time, the worker thread notifies the background log writer log storage thread that there is a transaction log to be landed. Therefore, the worker thread sends the transaction commit task to the log storage thread, and the log storage thread executes the log landing operation in the transaction commit task. Through the transaction commit task, the log storage thread can determine the target transaction and the corresponding transaction log to be stored.

[0076] In specific implementation, after notifying the log storage thread, the worker thread does not enter the sleep state, but performs a suspension operation on the first coroutine, and suspends the first coroutine through a conn_task_suspend function. After suspending the current coroutine, the worker thread can immediately return to the get_event logic and continue to acquire and process other client connections requiring to execute SQL statements. In this way, even in the process of waiting for the log to be written, the worker thread does not idle, but can better utilize resources and improve the overall throughput of the database.

[0077] Further, before sending the transaction commit task of the target transaction to the log storage thread, the method further includes: obtaining a transaction commit command by the first connection corresponding to the target transaction; and generating the transaction commit task in response to the transaction commit command.

[0078] The transaction commit command can be understood as a command sent by an application program for committing the target transaction. The transaction commit command can be a commit command. After the worker thread receives the transaction commit command, the transaction commit process is triggered. The transaction commit command is generated to notify the coroutine or other threads such as the log storage thread, the monitoring thread, and the like to cooperate in processing.

[0079] In actual application, the application program of the client can determine whether the transaction operation of each SQL statement is successfully completed according to the return result of the operation. If a certain operation fails, the application program can choose to terminate the transaction immediately and roll back all changes made (by sending a rollback command). If all SQL operations are successfully completed, the application program sends a transaction commit command to the database. The worker thread can receive the transaction commit command through the first connection, generate a transaction commit task, and thus perform the transaction commit process.

[0080] Based on this, the transaction commit process can be triggered by the transaction commit task, which facilitates storing the transaction log of the target transaction during transaction commit, so that the database system can guarantee data consistency and persistence.

[0081] Further, before releasing the worker thread by suspending the first coroutine, the method further includes: determining log sequence information of the transaction log; and adding the first connection to a connection waiting list according to the log sequence information, wherein the log storage thread stores logs of the connections in the connection waiting list according to the log sequence information; detecting the connections in the connection waiting list based on the storage information by a log notification thread, and obtaining a detection result returned by the log notification thread by a monitoring thread.

[0082] The log sequence information can be understood as a log sequence number (LSN), which is a unique number used by a database system to identify the order of redo log entries. A new, incremental LSN value is assigned each time a new redo log entry is generated. The log sequence number is crucial for ensuring the order of log entries and the correctness during the recovery process. The connection waiting list can be understood as a list for managing connections waiting for the completion of the writing of the redo log. When a transaction needs to wait for its redo log to be written to the disk before proceeding (e.g., committing), it will be added to this list. The connection waiting list (waiter list) is designed to ensure that these connections can continue to execute in the correct order. Arranging these connections in LSN order can ensure that all operations are executed in the correct order, thereby maintaining data consistency and transaction isolation.

[0083] In actual applications, when a transaction is committed, the transaction log of the target transaction needs to be written to the disk. At this time, the worker thread will determine the log sequence information of the transaction log, add the connection to the connection waiting list according to the log sequence information, and the subsequent log storage thread will store the transaction log corresponding to each connection in the connection waiting list according to the log sequence information.

[0084] In specific implementations, the worker thread also registers the file descriptor eventfd object in the connection to the epoll instance of the thread pool, and the listener thread (listener thread) listens to whether the transaction log is completed and written to the disk. Specifically, the log notification thread (log notifie) detects the connections in the connection waiting list according to the storage information, determines that the transaction log corresponding to the connection is completed and written to the disk through the storage information, generates a detection result of the completion of writing to the disk, and sends a message of the detection result to the listener thread through the eventfd object. The listener thread receives the detection result, and based on the detection result, processes the next step of the log writing connection.

[0085] In a specific embodiment of the present disclosure, during the process of waiting for the completion of the redo log writing, the worker thread suspends the first coroutine using the conn_task_suspend function to avoid blocking the worker thread. The worker thread registers the eventfd object in the connection to the epoll instance of the thread pool. The eventfd is a special file descriptor used for inter-process or inter-thread event notification. After registering with the epoll instance, the listener thread can listen to these eventfds through the epoll mechanism to receive notifications after the completion of the redo log writing. The background log writing thread is responsible for writing the redo log from the memory buffer to the disk, and after the completion of the log writing, the log notifier thread checks the waiter list to find all coroutines in the suspended state and corresponding redo logs that have been written to the disk. For each coroutine that meets the conditions, the log notifier thread sends a write completion message through its associated eventfd. The listener thread listens to all registered eventfds through epoll_wait. When an event occurs on an eventfd (i.e., a write completion message is received), the listener thread processes the event.

[0086] In summary, through the efficient I / O multiplexing mechanism (such as epoll) and event notification using eventfd, an efficient asynchronous log writing and transaction submission process is achieved. It not only improves the concurrent processing capability of the system, but also guarantees the consistency and reliability of the data.

[0087] Further, in the case where the detection result is log writing, the method further includes: selecting the first connection in the connection waiting list by the listener thread, and adding the first connection to an asynchronous preparation queue; based on the asynchronous preparation queue, obtaining the first connection, determining the first coroutine corresponding to the first connection, and continuing to perform the recovery operation on the first coroutine, and continuing to execute the transaction submission task through the first coroutine.

[0088] Wherein, after the listener thread receives the write completion message, the first connection is selected from the connection waiting list and added to the asynchronous preparation queue. The asynchronous preparation queue is used to store asynchronous tasks that have been prepared but have not been executed, so the first connection in the asynchronous preparation queue needs to be scheduled and processed by the worker thread. Specifically, the worker thread obtains the first connection based on the asynchronous preparation queue, determines the corresponding first coroutine, resumes the suspended first coroutine, and continues to execute the transaction submission task corresponding to the target transaction of the connection.

[0089] In practical applications, when the redo log of a certain client connection is detected to have completed writing, the system puts the connection into an async ready asynchronous preparation queue in the thread pool, to ensure that the transaction waiting for the completion of the redo log writing can be processed as soon as possible. The working thread obtains the connection marked as writing completion from the asynchronous preparation queue, and uses the conn_task_resume function to resume the coroutine context corresponding to the connection, to allow it to continue to execute other logic after the transaction is submitted. For example, cleaning and releasing the memory resources occupied by the transaction.

[0090] Therefore, after the listening thread receives the message of writing completion, the first connection is added to the asynchronous preparation queue, so that the first connection can be resumed and processed by the working thread as soon as possible. The working thread resumes the coroutine corresponding to the connection of writing completion, to improve the concurrent processing capability and efficiency of the system.

[0091] Further, to improve the concurrent processing capability and efficiency of the system, while the first coroutine is suspended, the working thread can process transactions under other connections. Specifically, after the working thread is released by suspending the first coroutine, the method further includes: determining that a second connection is associated with a second coroutine; and executing a transaction corresponding to the second connection based on the second coroutine.

[0092] The second connection can be understood as a connection other than the first connection, and the second coroutine can be understood as a coroutine associated with the second connection. The transaction under the second connection can be executed through the second coroutine.

[0093] In practical applications, after the working thread suspends the first coroutine associated with the first connection, to avoid wasting time while the thread is waiting for the log to be written, the working thread can execute transactions of other connections while the transaction log of the target transaction of the first coroutine is being written. In specific implementation, the working thread can obtain other client connections that need to execute SQL through get_event logic, and continue to execute, so that one working thread can process multiple transaction requests at the same time, to improve the processing efficiency of the thread.

[0094] Step 206: In the case of receiving the storage information returned by the log storage thread, the first coroutine is resumed, and the transaction submission task is continued to be executed through the first coroutine.

[0095] According to the above content, when the working thread confirms that the transaction log of the connection is written, the first coroutine is resumed, and the transaction submission task is continued to be executed by the first coroutine.

[0096] In practical applications, the log notification thread checks the connections in the connection waiting queue based on the storage information returned by the log storage thread. When it detects that a connection has completed log writing based on the storage information, it notifies the listening thread, which then removes the connection from the connection waiting queue and adds it to the asynchronous preparation queue. The worker thread then performs the next scheduling process on the connections in the asynchronous preparation queue.

[0097] This disclosure provides a data processing method applied to worker threads in a database management system. The method includes executing a target transaction through a first coroutine associated with the worker thread to obtain the transaction log of the target transaction; sending the transaction commit task of the target transaction to a log storage thread; and releasing the worker thread by suspending the first coroutine, wherein the log storage thread is used to store the transaction log. Upon receiving storage information returned by the log storage thread, the first coroutine is resumed, and the transaction commit task continues to be executed through the first coroutine. Based on worker threads in a database management system, the target transaction is executed through a first coroutine associated with the worker thread during transaction request scheduling, achieving transaction request scheduling in a coroutine manner and reducing the operating system's scheduling overhead for threads. During the transaction log write-to-disk operation, the transaction commit task of the target transaction is sent to the log storage thread, and the first coroutine is suspended to release the worker thread. This allows the released worker thread to be scheduled and switched to other connections or transactions for processing while the log storage thread performs the log write-to-disk operation, meeting the needs of high-concurrency transaction request scenarios, avoiding multi-threaded lock contention, improving database throughput while also improving thread execution efficiency, and ensuring good database write performance.

[0098] Referring to Figure 3, the data processing method provided in this disclosure will be further described using the application of the data processing method in an order database as an example. Figure 3 shows a flowchart of the processing procedure of a data processing method according to an embodiment of this disclosure, specifically including the following steps.

[0099] Step 302: Obtain the first connection and determine the target transaction corresponding to the first connection.

[0100] In one feasible approach, the client application is a shopping application through which users can place orders. The backend service receives the order requests and persists the order information to a database. The application establishes a first connection with the database provided by the backend service and determines the target transaction corresponding to the first connection. The target transaction may include data operations for inserting orders.

[0101] Step 304: Create an initial coroutine and associate it with the first connection to obtain the first coroutine.

[0102] In an implementable manner, determine the asynchronous execution variable corresponding to the first connection. When it is determined that the asynchronous transaction submission function is enabled according to the asynchronous execution variable, create an initial coroutine, associate the initial coroutine with the first connection, and obtain the first coroutine.

[0103] Step 306: Determine the execution information of the first coroutine associated with the working thread. Execute the target transaction through the first coroutine according to the execution information, obtain the execution record corresponding to the execution information, and generate the transaction log of the target transaction according to the execution record.

[0104] In an implementable manner, determine the execution information of the first coroutine, that is, the entry function. Execute the target transaction according to the entry function, obtain the operation execution record corresponding to the SQL statement, and generate the transaction log redo log according to the execution record.

[0105] Step 308: Obtain the transaction submission command through the first connection corresponding to the target transaction, and generate a transaction submission task in response to the transaction submission command.

[0106] In an implementable manner, when the application program confirms that the operation result returned by the SQL statement is normal, send a transaction submission command to the database. After the working thread obtains the transaction submission command, it generates a transaction submission task.

[0107] Step 310: Send the transaction submission task of the target transaction to the log storage thread.

[0108] In an implementable manner, the working thread sends the transaction submission task to the log storage thread, and notifies the log storage thread to perform the disk write operation of the transaction log redo log.

[0109] Step 312: Determine the log sequence information of the transaction log, and add the first connection to the connection waiting list according to the log sequence information.

[0110] In an implementable manner, the working thread determines the log sequence number LSN of the transaction log, and adds the first connection to the connection waiting list waiter list according to the log sequence information.

[0111] Step 314: Release the working thread by suspending the first coroutine, determine the second connection associated with the second coroutine, and execute the transaction corresponding to the second connection based on the second coroutine.

[0112] In an implementable manner, the working thread suspends the first coroutine, determines the second connection and its corresponding second coroutine through the event acquisition function, and executes the transaction corresponding to the second connection by using the second coroutine.

[0113] Step 316: The log notification thread checks the connections in the connection wait list based on the stored information, and the listening thread obtains the detection results returned by the log notification thread.

[0114] In one possible implementation, the log notifier thread checks the connections in the wait list based on stored information, and returns the detection result to the listener thread when it determines that the transaction log of the connection has been written.

[0115] Step 318: Select the first connection from the connection waiting list through the listening thread and add the first connection to the asynchronous preparation queue.

[0116] In one possible implementation, the listener thread identifies the first connection in the waiter list and adds it to the asynchronous ready queue, waiting for the worker thread to schedule it.

[0117] Step 320: Obtain the first connection based on the asynchronous preparation queue, determine the first coroutine corresponding to the first connection, continue to perform the recovery operation on the first coroutine, and continue to execute the transaction commit task through the first coroutine.

[0118] In one possible implementation, the worker thread obtains the first connection from the asynchronous ready queue and resumes the first thread corresponding to the first connection, thus continuing to execute other operations of the transaction commit task through the first coroutine. After the transaction is committed, the relevant operations on the order data are completed.

[0119] The data processing method disclosed herein implements transaction request scheduling using coroutines, reducing the operating system's overhead on thread scheduling. During transaction log write-to-disk operations, the transaction commit task of the target transaction is sent to the log storage thread, and the first coroutine is suspended to release the worker thread. This allows the released worker thread to be scheduled and switched to other connections or transactions for processing while the log storage thread performs the log write-to-disk operation. This meets the needs of high-concurrency transaction request scenarios, avoids multi-threaded lock contention, improves database throughput while also increasing thread execution efficiency, and ensures good database write performance.

[0120] Corresponding to the above method embodiments, this disclosure also provides a data processing system embodiment. Figure 4 shows a schematic diagram of the structure of a data processing system provided in one embodiment of this disclosure. As shown in Figure 4, the system includes a worker thread 402 and a log storage thread 404, wherein...

[0121] The worker thread 402 is used to execute the target transaction through the first coroutine, obtain the transaction log of the target transaction, send the transaction commit task of the target transaction to the log storage thread, and release the worker thread by suspending the first coroutine.

[0122] The log storage thread 404 is used to store the transaction log, obtain the storage information, and return it to the worker thread;

[0123] The worker thread 402 is used to perform a recovery operation on the first coroutine based on the stored information, and to continue executing the transaction commit task through the first coroutine.

[0124] The processing procedure of the data processing system provided in this disclosure can be found in the detailed description of the data processing method described above, and will not be repeated here. The data processing system provided in this disclosure implements transaction request scheduling using coroutines, reducing the operating system's overhead on thread scheduling. When performing transaction log write-to-disk operations, the transaction commit task of the target transaction is sent to the log storage thread, and the first coroutine is suspended to release the worker thread. This allows the released worker thread to be scheduled and switched to other connections or transactions for processing while the log storage thread performs the log write-to-disk operation, meeting the needs of high-concurrency transaction request scenarios, avoiding multi-threaded lock contention, improving database throughput while also improving thread execution efficiency, and ensuring good database write performance.

[0125] Corresponding to the above method embodiments, this disclosure also provides a data processing apparatus embodiment. Figure 5 shows a schematic diagram of the structure of a data processing apparatus provided in one embodiment of this disclosure. As shown in Figure 5, the apparatus includes:

[0126] The execution module 502 is configured to execute the target transaction through the first coroutine associated with the worker thread and obtain the transaction log of the target transaction;

[0127] The suspension module 504 is configured to send the transaction commit task of the target transaction to the log storage thread, and to release the worker thread by suspending the first coroutine, wherein the log storage thread is used to store the transaction log;

[0128] The recovery module 506 is configured to perform a recovery operation on the first coroutine upon receiving storage information returned by the log storage thread, and to continue executing the transaction commit task through the first coroutine.

[0129] Optionally, the device further includes an association module configured to acquire a first connection and determine the target transaction corresponding to the first connection; create an initial coroutine and associate the initial coroutine with the first connection to obtain the first coroutine.

[0130] Optionally, the association module is further configured to determine the asynchronous execution variable corresponding to the first connection; if the asynchronous execution variable is enabled, continue to execute the step of creating the initial coroutine; if the asynchronous execution variable is disabled, execute the target transaction according to the synchronization strategy.

[0131] Optionally, the execution module 502 is further configured to determine the execution information of the first coroutine associated with the worker thread; and to execute the target transaction through the first coroutine according to the execution information to obtain the transaction log of the target transaction.

[0132] Optionally, the execution module 502 is further configured to execute the target transaction according to the execution information through the first coroutine, obtain the execution record corresponding to the execution information, and generate a transaction log of the target transaction based on the execution record.

[0133] Optionally, the apparatus further includes a generation module configured to obtain a transaction commit command through a first connection corresponding to the target transaction; and generate a transaction commit task in response to the transaction commit command.

[0134] Optionally, the device further includes an adding module configured to determine the log sequence information of the transaction log; add the first connection to the connection wait list according to the log sequence information, wherein the log storage thread stores logs of the connections in the connection wait list according to the log sequence information; and detect the connections in the connection wait list based on the stored information through a log notification thread, and obtain the detection results returned by the log notification thread through a listening thread.

[0135] Optionally, the device further includes a selection module configured to select the first connection from the connection waiting list via a listening thread, add the first connection to an asynchronous preparation queue, obtain the first connection based on the asynchronous preparation queue, determine the first coroutine corresponding to the first connection, continue to perform a recovery operation on the first coroutine, and continue to execute the transaction commit task through the first coroutine.

[0136] Optionally, the apparatus further includes a determining module configured to determine a second connection associated with a second coroutine; and to execute a transaction corresponding to the second connection based on the second coroutine.

[0137] This disclosure provides a data processing apparatus that implements transaction request scheduling using coroutines, reducing the operating system's overhead on thread scheduling. During transaction log write-to-disk operations, the transaction commit task of the target transaction is sent to the log storage thread, and the first coroutine is suspended to release the worker thread. This allows the released worker thread to be scheduled and switched to other connections or transactions for processing while the log storage thread performs the log write-to-disk operation. This meets the needs of high-concurrency transaction request scenarios, avoids multi-threaded lock contention, improves database throughput while also increasing thread execution efficiency, and ensures good database write performance.

[0138] The above is an illustrative scheme of a data processing apparatus according to this embodiment. It should be noted that the technical solution of this data processing apparatus and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the data processing apparatus, please refer to the description of the technical solution of the data processing method described above.

[0139] Figure 6 shows a structural block diagram of a computing device 600 according to an embodiment of the present disclosure. The components of the computing device 600 include, but are not limited to, a memory 610 and a processor 620. The processor 620 is connected to the memory 610 via a bus 630, and a database 650 is used to store data.

[0140] The computing device 600 also includes an access device 640, which enables the computing device 600 to communicate via one or more networks 660. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 640 may include one or more of any type of wired or wireless network interface (e.g., a network interface controller (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0141] In one embodiment of this disclosure, the aforementioned components of the computing device 600, as well as other components not shown in FIG. 6, may also be connected to each other, for example, via a bus. It should be understood that the computing device block diagram shown in FIG. 6 is merely for illustrative purposes and is not intended to limit the scope of this disclosure. Those skilled in the art can add or replace other components as needed.

[0142] The computing device 600 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 600 can also be a mobile or stationary server.

[0143] The processor 620 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-described data processing method.

[0144] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the data processing method described above.

[0145] An embodiment of this disclosure also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described data processing method.

[0146] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the data processing method described above.

[0147] An embodiment of this disclosure also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described data processing method.

[0148] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the data processing method described above.

[0149] The foregoing has described specific embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0150] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0151] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this disclosure are not limited to the described order of actions, because according to the embodiments of this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this disclosure.

[0152] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0153] The preferred embodiments disclosed above are merely illustrative of this disclosure. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments of this disclosure. This disclosure selects and specifically describes these embodiments to better explain the principles and practical applications of the embodiments of this disclosure, thereby enabling those skilled in the art to better understand and utilize this disclosure.

Claims

1. A data processing method, applied to a worker thread in a database management system, comprising: The target transaction is executed by the first coroutine associated with the worker thread, and the transaction log of the target transaction is obtained. The transaction commit task of the target transaction is sent to the log storage thread, and the worker thread is released by suspending the first coroutine, wherein the log storage thread is used to store the transaction log; Upon receiving the storage information returned by the log storage thread, a recovery operation is performed on the first coroutine, and the transaction commit task continues to be executed through the first coroutine.

2. The method according to claim 1, before executing the target transaction through the first coroutine associated with the worker thread, the method further includes: Obtain the first connection and determine the target transaction corresponding to the first connection; Create an initial coroutine and associate the initial coroutine with the first connection to obtain a first coroutine.

3. The method according to claim 2, wherein before creating the initial coroutine, the method further comprises: Determine the asynchronous execution variables corresponding to the first connection; If the asynchronous execution variable is enabled, continue executing the steps to create the initial coroutine; When the asynchronous execution variable is turned off, the target transaction is executed according to the synchronous strategy.

4. The method according to claim 1, wherein the target transaction is executed through a first coroutine associated with the worker thread to obtain the transaction log of the target transaction, comprising: Determine the execution information of the first coroutine associated with the worker thread; The first coroutine executes the target transaction according to the execution information to obtain the transaction log of the target transaction.

5. The method according to claim 4, wherein the first coroutine executes the target transaction according to the execution information to obtain the transaction log of the target transaction, comprising: The first coroutine executes the target transaction according to the execution information to obtain the execution record corresponding to the execution information; The transaction log of the target transaction is generated based on the execution records.

6. The method according to claim 1, before sending the transaction commit task of the target transaction to the log storage thread, the method further includes: Obtain the transaction commit command through the first connection corresponding to the target transaction; A transaction commit task is generated in response to the transaction commit command.

7. The method according to claim 6, further comprising, before releasing the worker thread by suspending the first coroutine: Determine the log sequence information of the transaction log; The first connection is added to the connection waiting list according to the log sequence information, wherein the log storage thread stores logs of the connections in the connection waiting list according to the log sequence information; The log notification thread detects connections in the connection wait list based on the stored information, and the listening thread obtains the detection results returned by the log notification thread.

8. The method according to claim 7, wherein if the detection result is a log write, the method further comprises: The listening thread selects the first connection from the connection waiting list and adds the first connection to the asynchronous preparation queue. The steps of obtaining the first connection based on the asynchronous preparation queue, determining the first coroutine corresponding to the first connection, continuing to perform the recovery operation on the first coroutine, and continuing to execute the transaction commit task through the first coroutine.

9. The method according to claim 1, after releasing the worker thread by suspending the first coroutine, the method further includes: Determine the association between the second connection and the second coroutine; The transaction corresponding to the second connection is executed based on the second coroutine.

10. A database management system, the system comprising worker threads and log storage threads, wherein, The worker thread is used to execute the target transaction through the first coroutine, obtain the transaction log of the target transaction, send the transaction commit task of the target transaction to the log storage thread, and release the worker thread by suspending the first coroutine. The log storage thread is used to store the transaction log, obtain the storage information, and return it to the worker thread; The worker thread is used to perform a recovery operation on the first coroutine based on the stored information, and to continue executing the transaction commit task through the first coroutine.

11. A data processing apparatus, applied to a worker thread in a database management system, comprising: The execution module is configured to execute the target transaction through a first coroutine associated with the worker thread and obtain the transaction log of the target transaction; The suspension module is configured to send the transaction commit task of the target transaction to the log storage thread, and to release the worker thread by suspending the first coroutine, wherein the log storage thread is used to store the transaction log; The recovery module is configured to perform a recovery operation on the first coroutine upon receiving storage information returned by the log storage thread, and to continue executing the transaction commit task through the first coroutine.

12. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they perform the following steps: The target transaction is executed by the first coroutine associated with the worker thread, and the transaction log of the target transaction is obtained. The transaction commit task of the target transaction is sent to the log storage thread, and the worker thread is released by suspending the first coroutine, wherein the log storage thread is used to store the transaction log; Upon receiving the storage information returned by the log storage thread, a recovery operation is performed on the first coroutine, and the transaction commit task continues to be executed through the first coroutine.

13. The computing device of claim 12, wherein when the computer-executable instructions are executed by the processor, the following steps are further performed: Obtain the first connection and determine the target transaction corresponding to the first connection; Create an initial coroutine and associate the initial coroutine with the first connection to obtain a first coroutine.

14. The computing device of claim 13, wherein when the computer-executable instructions are executed by the processor, the following steps are further performed: Determine the asynchronous execution variables corresponding to the first connection; If the asynchronous execution variable is enabled, continue executing the steps to create the initial coroutine; When the asynchronous execution variable is turned off, the target transaction is executed according to the synchronous strategy.

15. The computing device of claim 12, wherein when the computer-executable instructions are executed by the processor, the following steps are further performed: Determine the execution information of the first coroutine associated with the worker thread; The first coroutine executes the target transaction according to the execution information to obtain the transaction log of the target transaction.

16. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, perform the following steps: The target transaction is executed by the first coroutine associated with the worker thread, and the transaction log of the target transaction is obtained. The transaction commit task for the target transaction is sent to the log storage thread, and the worker thread is released by suspending the first coroutine. The log storage thread is used to store the transaction log; Upon receiving the storage information returned by the log storage thread, a recovery operation is performed on the first coroutine, and the transaction commit task continues to be executed through the first coroutine.

17. The computer-readable storage medium of claim 16, wherein the computer-executable instructions, when executed by a processor, further perform the following steps: Obtain the first connection and determine the target transaction corresponding to the first connection; Create an initial coroutine and associate the initial coroutine with the first connection to obtain a first coroutine.

18. The computer-readable storage medium of claim 17, wherein the computer-executable instructions, when executed by a processor, further perform the following steps: Determine the asynchronous execution variables corresponding to the first connection; If the asynchronous execution variable is enabled, continue executing the steps to create the initial coroutine; When the asynchronous execution variable is turned off, the target transaction is executed according to the synchronous strategy.

19. The computer-readable storage medium of claim 16, wherein the computer-executable instructions, when executed by a processor, further perform the following steps: Determine the execution information of the first coroutine associated with the worker thread; The first coroutine executes the target transaction according to the execution information to obtain the transaction log of the target transaction.

20. The computer-readable storage medium of claim 19, wherein the computer-executable instructions, when executed by a processor, further perform the following steps: The first coroutine executes the target transaction according to the execution information to obtain the execution record corresponding to the execution information; The transaction log of the target transaction is generated based on the execution records.

Images