Data writing method, device, electronic device and storage medium

Abstract

The present application provides a data writing method, device, electronic device and storage medium, which relate to the field of cloud computing technology. The method includes: obtaining log events of multiple transactions; determining multiple temporary storage areas corresponding to the multiple transactions; writing the log events of the multiple transactions into a persistent storage area based on the multiple temporary storage areas by parallel processing, and the log events stored in the persistent storage area are committed based on the storage engine. In an embodiment of the present application, the log events of the multiple transactions are written into a persistent storage area by parallel processing based on the temporary storage areas corresponding to the multiple transactions. The use of parallel processing can increase the data writing speed, thereby improving the performance of the database management system.

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to a data writing method, device, electronic device, and storage medium. Background Art

[0002] A database is a computer application system that manages data resources. Data can take many forms, such as text, numbers, symbols, graphics, images, and sounds. Data is the object that all computer systems process. The software that manages a database is called a database management system.

[0003] MySQL (Structured Query Language) is a relational database management system that provides multiple database storage engines suitable for different application scenarios. The MySQL transaction commit strategy for writing log files is to write all transaction log events to the log file and persist them in the database when multiple transactions are committed simultaneously. However, this slow data write speed cannot meet the needs of applications with high write speed requirements. Summary of the Invention

[0004] The embodiments of the present application provide a data writing method, device, electronic device and storage medium to improve the data writing speed.

[0005] In a first aspect, an embodiment of the present application provides a data writing method, comprising:

[0006] Get log events for multiple transactions;

[0007] Determine multiple temporary storage areas corresponding to multiple transactions;

[0008] Through parallel processing, log events of multiple transactions are written to a persistent storage area based on multiple temporary storage areas. The log events stored in the persistent storage area are committed to transactions based on the storage engine.

[0009] In a second aspect, an embodiment of the present application provides a data writing device, comprising:

[0010] The acquisition module is used to obtain log events of multiple transactions;

[0011] A determination module, configured to determine a plurality of temporary storage areas corresponding to a plurality of transactions;

[0012] The writing module is used to write the log events of multiple transactions into the persistent storage area based on multiple temporary storage areas in a parallel processing manner. The log events stored in the persistent storage area are committed to transactions based on the storage engine.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor implements any of the above methods when executing the computer program.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, any of the above methods is implemented.

[0015] Compared with the prior art, this application has the following advantages:

[0016] The data writing method, device, electronic device and storage medium provided in the embodiments of the present application determine temporary storage areas corresponding to multiple transactions, and write log events of multiple transactions into a persistent storage area based on the temporary storage areas corresponding to the multiple transactions through parallel processing. The use of parallel processing can increase the data writing speed, thereby improving the performance of the database management system.

[0017] The above description is only an overview of the technical solution of this application. In order to more clearly understand the technical means of this application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of this application more obvious and easy to understand, the specific implementation methods of this application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar components or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments according to the present application and should not be regarded as limiting the scope of the present application.

[0019] Figure 1 A schematic diagram of an application scenario of the data writing method provided in this application;

[0020] Figure 2 A schematic diagram of a data writing method in related art;

[0021] Figure 3 This is a flow chart of a data writing method according to an embodiment of the present application;

[0022] Figure 4 is a schematic diagram of a data writing method according to another embodiment of the present application;

[0023] Figure 5 is a schematic diagram of a data writing method according to an embodiment of the present application;

[0024] Figure 6 This is a schematic diagram of persisting binary log events to redo log files according to an embodiment of the present application;

[0025] Figure 7 This is a schematic diagram of recovering binary log events of transactions from redo log files to binary log files according to an embodiment of the present application;

[0026] Figure 8 is a structural block diagram of a data writing device according to another embodiment of the present application; and

[0027] Figure 9 A block diagram of an electronic device used to implement an embodiment of the present application. DETAILED DESCRIPTION

[0028] Hereinafter, only certain exemplary embodiments are briefly described. As will be appreciated by those skilled in the art, the described embodiments may be modified in various ways without departing from the spirit or scope of the present application. Therefore, the drawings and description are to be regarded as illustrative in nature and not restrictive.

[0029] To facilitate understanding of the technical solutions of the embodiments of the present application, the following describes the related technologies of the embodiments of the present application. The following related technologies can be combined with the technical solutions of the embodiments of the present application as optional solutions, and all of them fall within the scope of protection of the embodiments of the present application.

[0030] The following is an introduction to the application scenarios of the technical solution of this application. Figure 1 The following is a schematic diagram illustrating an exemplary application scenario for implementing the method of an embodiment of the present application. The technical solution of the present application can be applied to scenarios where data is written to a database through a database management system. When selecting a database management system, data writing performance is a key metric. Improving writing performance can better meet user needs and reduce costs. Figure 2This diagram illustrates a data writing method in related technologies. Taking MySQL as an example, the implementation strategy for writing the binary log (Binlog) to the database during transaction commit includes the write phase (called the Flush Stage), the persistence phase (called the Sync Stage), and the commit phase (called the Commit Stage). This strategy has two characteristics: group commit and serial operation. Group commit means that when multiple transactions (as shown in the figure, Trx1, Trx2, ..., Trx12) are submitted simultaneously, they are divided into different groups. Within each group, a leader transaction (called the Leader) is identified and is responsible for completing the commit operations for all transactions within the group. During the three stages of the commit process—write, persist, and commit—only one group can operate simultaneously, meaning operations are serial and non-concurrent. Because these three stages are serial, slow execution in any one stage can cause blockages in the other stages. This is especially true during the write phase, where each transaction must perform numerous operations, including persisting redo logs and writing to the binary log file. Because these transactions are serial, subsequent transactions experience long wait times, limiting database performance.

[0031] In view of this, the embodiment of the present application proposes a data writing method, such as Figure 1 As shown, log events (for example, Binlog Event) of multiple transactions (Trx1, Trx2, ... Trx4 as shown in the figure) are obtained, and multiple temporary storage areas corresponding to multiple transactions are determined in a temporary storage area (Binlog Buffer) shared by multiple transactions; by a parallel execution mode, based on the temporary storage areas corresponding to multiple transactions, the log events of multiple transactions are written (called flush) to the log file from the temporary storage areas corresponding to the multiple transactions, and then the log file is persisted (called sync) and stored in the database, and the stored log events are submitted (called commit) to the transaction based on the storage engine. In the present embodiment, the temporary storage areas corresponding to multiple transactions are determined in the temporary storage area shared by multiple transactions, and by a parallel processing mode, based on the temporary storage areas corresponding to multiple transactions, the log events of multiple transactions are written to the persistent storage area, and the parallel processing mode can improve the data write speed, thereby improving the performance of the database management system.

[0032] Figure 3 This is a flow chart of a data writing method according to an embodiment of the present application. Figure 3 As shown, the data writing method may include:

[0033] Step S301: Obtain log events of multiple transactions.

[0034] In this embodiment, the execution entity may include a server, a user terminal, or any electronic device. A transaction (referred to as a transaction) is a sequence of database operations that accesses and potentially manipulates various data items and is an indivisible unit of work. A transaction is the entire database operation performed between the start and end of a transaction. Log events are used to record transactions and the database modifications made by transactions. Logs may include binary logs, redo logs, and so on.

[0035] Step S302: determining a plurality of temporary storage areas corresponding to the plurality of transactions.

[0036] A temporary storage area corresponding to each of the multiple transactions is determined in a temporary storage area shared by the multiple transactions. For example, a temporary storage area corresponding to each of the transactions is determined in a binary log buffer area shared by the multiple transactions.

[0037] Optionally, multiple transactions may be grouped into multiple groups, each group including multiple transactions, and a corresponding temporary storage area may be allocated to each group of transactions in the shared temporary storage area. After each group of transactions has been allocated a corresponding temporary storage area, log events corresponding to each group of transactions are stored in the corresponding temporary storage area for each group of transactions.

[0038] Optionally, a corresponding temporary storage area is allocated to each transaction in the shared temporary storage area. After each transaction is allocated a corresponding temporary storage area, the log events corresponding to each transaction are stored in the temporary storage area corresponding to each transaction.

[0039] Exemplarily, the multiple temporary storage areas corresponding to the multiple transactions may be the Binlog Buffers corresponding to each group of transactions, or the Binlog Buffers corresponding to each transaction.

[0040] Step S303: writing log events of multiple transactions into a persistent storage area based on multiple temporary storage areas in a parallel processing manner, and performing transaction commit on the log events stored in the persistent storage area based on the storage engine.

[0041] By processing multiple groups of transactions or multiple transactions in parallel, log events are retrieved from the temporary storage area corresponding to each group of transactions or each transaction and persistently stored on disk. The persistent storage area may include a database stored on disk. After the transaction log events are written to the persistent storage area, the transaction status is marked as committed, thus implementing the engine transaction commit.

[0042] The data writing method provided in the embodiment of the present application determines temporary storage areas corresponding to multiple transactions in a temporary storage area shared by multiple transactions, and writes log events of multiple transactions into a persistent storage area based on the temporary storage areas corresponding to the multiple transactions through parallel processing. The use of parallel processing can increase the data writing speed, thereby improving the performance of the database management system.

[0043] The specific implementation of step S303 is shown in the following embodiment:

[0044] In one possible implementation, step S303 writes the log events of multiple transactions into a persistent storage area based on multiple temporary storage areas through parallel processing, including: writing the log events of multiple transactions into their respective corresponding temporary storage areas through parallel processing, obtaining the log events of multiple transactions from their respective corresponding temporary storage areas, writing the log files, and writing the log files into the persistent storage area.

[0045] Among them, each transaction corresponds to its own transaction thread. The transaction thread writes its own log events into the temporary storage area allocated to the group to which the transaction belongs in the shared temporary storage area. By processing multiple groups of transactions in parallel, the log events of each group of transactions are written into the temporary storage area corresponding to each group respectively, and the log events of the group of transactions are obtained from the temporary storage area corresponding to each group, written into the log file, and the log file is written into the persistent storage area.

[0046] Optionally, data can also be written according to each transaction. The transaction thread writes its own log events into the temporary storage area allocated for the transaction in the shared temporary storage area. By processing multiple transactions in parallel, the log events of each transaction are written into the temporary storage area corresponding to the transaction. The log events of the transaction are obtained from the temporary storage area corresponding to each transaction, written into the log file, and the log file is written into the persistent storage area.

[0047] In one possible implementation, log events of multiple transactions are written to a persistent storage area based on multiple temporary storage areas through parallel processing, including:

[0048] Through parallel processing, based on multiple temporary storage areas, the foreground thread is used to write the logical log events of multiple transactions into the persistent storage area through the physical log file, and the background thread is used to write the logical log events into the persistent storage area through the logical log file.

[0049] Among them, the logical log event of the transaction, for example, the binary log event Binlog Event. The logical log file, for example, the Binlog file. The physical log file, for example, the redo log Redolog file. Through parallel processing, the foreground thread is used to write the logical log event to the temporary storage area corresponding to each transaction, and the logical log events of multiple transactions are obtained from the corresponding temporary storage area, written to the physical log file, and then the physical log file is written to the persistent storage area. Then, the logical log event is sent to the background thread, and the background thread is used to write the logical log event to the temporary storage area corresponding to each transaction, and the logical log events of multiple transactions are obtained from the corresponding temporary storage area, written to the logical log file, and then the logical log file is written to the persistent storage area.

[0050] In this embodiment, the foreground thread is used to write the logical log event into the physical log file and persist it together with the physical log file. The background thread is used to write the logical log file into the logical log file and persist it. This can reduce the number of write operations in the input and output processes during the transaction submission process and improve write performance.

[0051] For example, Figure 6 Provides a schematic diagram for persisting binary log events to redo log files. Figure 6 As shown in the figure, when a transaction is about to enter the write process, a foreground thread retrieves binary log events from the temporary storage area for multiple transactions, writes these events to the redo log file in parallel, and persists the redo log file to disk. The Binlog events are then handed off to a background thread, which writes and persists them to the Binlog file. The background thread's writing and persisting operations can be asynchronous with the foreground thread's transaction commit. After handing off the Binlog events to the background thread, it waits for the Redo log file to be persisted. Once the Redo log file is persisted to disk, the transaction is committed, and a transaction commit notification message is returned to the client.

[0052] In a possible implementation, the method further includes:

[0053] When the server starts, if the physical log file records logical log events that are not recorded in the logical log file, the unrecorded logical log events are written to the logical log file.

[0054] In actual applications, the server may crash and need to be restarted. In this case, the physical log file is checked to see if there are any logical log events recorded. If the physical log file records logical log events that are not recorded in the logical log file, the logical log events that are not recorded in the logical log file are written to the logical log file, thereby ensuring the data persistence of the logical log events and the consistency of the logical log events recorded in the physical log file and the logical log file.

[0055] For example, after a server crash, the number of logical log events recorded in the Binlog file may be less than that in the Redolog file. Therefore, when the server starts, the Redolog file recovery mechanism redoes all Redolog files after the checkpoint. If there are Binlog events recorded in the Redolog file, the Binlog file is checked and the missing Binlog events are written back to the Binlog file. Figure 7 This is a diagram of the process of recovering the binary log events of transactions from the redo log files to the binary log files. Figure 7 As shown in the figure, after the server restarts, the redo log file is checked. The redo log file records binary log events: binary log Trx2 and binary log Trx3. The binary log file records binary log Trx1, but does not have binary log Trx2 or binary log Trx3. In this case, binary log Trx2 and binary log Trx3 are added to the binary log file.

[0056] In one possible implementation, log events of multiple transactions are obtained from respective corresponding temporary storage areas, written into log files, and the log files are written into a persistent storage area, including: using a writing thread to obtain log events of multiple transactions from respective corresponding temporary storage areas, write into log files, and using a persistent thread to write the log files into the persistent storage area.

[0057] In practical applications, two different threads, a write thread and a persistence thread, can be used to perform write and persistence operations respectively. For any transaction, the write thread writes the log events of the current transaction to the log file, and the persistence thread performs the log file persistence operation. Because the two threads perform the write and persistence operations separately, the log events of the next transaction can be written as soon as the current transaction is written to the log file. There is no need to wait for the current transaction to be persisted before the next transaction is persisted, thereby improving data writing speed.

[0058] It should be noted that the writing thread and the persistence thread can also perform writing and persistence operations for each group of transactions. The specific process is similar to the process of operating for each transaction and will not be repeated here.

[0059] For example, when a transaction needs to be committed, the transaction thread is responsible for writing its own Binlog Event from the BinlogCache to the Binlog Buffer. Binlog Events in the Binlog Buffer are asynchronously written to the Binlog file and flushed to disk by independent threads for persistent storage.

[0060] In one possible implementation, using a write thread to obtain log events of multiple transactions from their respective corresponding temporary storage areas and write them into a log file includes: according to a write notification message, using a write thread to obtain log events of multiple transactions from their respective corresponding temporary storage areas and write them into a log file.

[0061] In actual applications, after the log events of each transaction or each group of transactions are written to the corresponding temporary storage area, a write notification message is sent to the writing thread. The writing thread writes the log events of each transaction or each group of transactions into the log file according to the write notification message. After the writing is completed, a persistence notification message is sent to the persistence thread.

[0062] In a possible implementation, using a persistence thread to write the log file into the persistent storage area includes: using a persistence thread to write the log file into the persistent storage area according to the persistence notification message.

[0063] In actual applications, the persistence thread performs persistence operations based on the received persistence notification message and writes the log file into the persistent storage area.

[0064] In a possible implementation, the method further includes: determining that the configuration information indicates a first execution strategy.

[0065] In practical applications, a transaction commit execution strategy is preconfigured, and the timing of transaction commit can be determined based on the execution strategy. The execution strategy can be configured with corresponding execution conditions for the write thread or the persistence thread, and the execution conditions can be preset values ​​corresponding to the write thread or the persistence thread. A first execution strategy can be to use the write thread to write log events for multiple transactions to a persistent storage area, and then commit the transaction based on the storage engine, such as the execution process in step S303 above.

[0066] In addition, the transaction can also be committed by executing the second execution strategy, as shown in the following embodiment:

[0067] In one possible implementation, the method further includes: determining that the configuration information indicates a second execution strategy; for any transaction among the multiple transactions, obtaining a log event of the transaction from a second temporary storage area corresponding to the transaction, writing the log event to a log file, committing the log file based on a storage engine, and writing the log file to a persistent storage area.

[0068] In practical applications, if the configuration information indicates the second execution strategy, for any transaction, after the transaction's log events are written to the log file, the log file can be committed, and then the log file can be written to the persistent storage area. In other words, the transaction can be committed without waiting for the log file to be persisted. Alternatively, the transaction can be committed simultaneously with the log file being written to the persistent storage area. In other words, log file persistence and transaction commit do not have a strict temporal relationship and can be configured according to specific needs.

[0069] For example, if the preset value of the persistence thread is sync_binlog=1, the engine transaction is committed after the persistence operation is completed; otherwise, the transaction is committed after the log event of the transaction is written into the log file.

[0070] In a possible implementation, the method further includes: when determining multiple temporary storage areas corresponding to the multiple transactions, performing a locking operation on a code block that executes an operation of determining the multiple temporary storage areas.

[0071] In actual applications, since the first temporary storage area is a shared storage area, in order to ensure the security of data during operation, when allocating a corresponding temporary storage area for each transaction in the first temporary storage area, the code block that executes the operation of determining multiple temporary storage areas is locked and protected by locking Lock_log.

[0072] In a possible implementation, determining multiple temporary storage areas corresponding to multiple transactions includes: for any transaction among the multiple transactions, determining a temporary storage area corresponding to the transaction according to a data volume of a log event of the transaction.

[0073] Optionally, when allocating a corresponding temporary storage area for each transaction in the shared temporary storage area, the size of the temporary storage area may be determined according to the data volume of the log events of the transaction.

[0074] Optionally, when allocating a corresponding temporary storage area for each group of transactions in the shared temporary storage area, the size of the temporary storage area may be determined according to the data volume of the log events of each group of transactions.

[0075] In order to more clearly introduce the technical ideas of the technical solution of this application, a detailed introduction is given below through specific embodiments.

[0076] Figure 4 Schematic diagram of a data writing method according to an embodiment of the present application. Figure 4 As shown, log events (for example, binary log events Binlog Event) of multiple transactions Trx1, Trx2, Trx3, and Trx4 are obtained, a global transaction identifier (GTID) is assigned to each transaction, and a logical clock is assigned, and a corresponding cache is assigned to each transaction in the binary log file shared cache, and the log events of each transaction are written to the corresponding cache in parallel, that is, the log events are written to the log file in the cache, and the log file (for example, a binary log file) is persisted and stored in a persistent storage area, and the engine transaction is committed after the persistence operation is completed. Among them, the allocation of global identifiers, the allocation of logical clocks, and the allocation of caches can be executed sequentially. In order to ensure that the GTID and the Logical Clock are ordered in the log file, the operations of allocating GTIDs and Logical Clocks and allocating caches are locked (Lock_log) together for protection. In addition, the operations of writing to the cache, waiting for persistence, and engine submission can be performed in parallel by multiple transactions. The log events of each transaction are written to the log file by the write thread. After the write operation is completed, a persistence notification message is sent to the persistence thread. The persistence thread performs a persistence operation on the log file, and the stored log events are submitted to the storage engine based on the transaction. In this embodiment, temporary storage areas corresponding to multiple transactions are determined in a temporary storage area shared by multiple transactions. Through parallel processing, the log events of multiple transactions are written to the persistent storage area based on the temporary storage areas corresponding to the multiple transactions. The use of parallel processing can increase the data write speed, thereby improving the performance of the database management system.

[0077] Figure 5 Schematic diagram of a data writing method according to an embodiment of the present application. Figure 5The following illustrates log operations using the MySQL binary log (Binlog) as an example. When a transaction enters the commit phase, the following process is executed: allocating a Gtid, allocating a logical clock, and allocating space for the Binlog buffer. These steps are performed serially, and only one transaction can perform these operations at a time. Each transaction thread writes its own Binlog events to the allocated Binlog buffer. It then notifies the Binlog Write thread that this buffer is ready for writing. The transaction waits for its Binlog events to be written to the Binlog file. When the write is complete, it generates a write completion notification message. Upon receiving the notification, the write thread copies the portion that can be written to the Binlog file from the Binlog buffer to the Binlog file, then notifies the persistence thread to persist the event and notify the waiting transaction that it can continue. Upon receiving the notification, the persistence thread performs the Binlog persistence operation and notifies the waiting transaction that it can continue. Upon receiving the notification, the transaction confirms that its Binlog events have been written or persisted and then performs a commit operation. The timing of transaction commit can be determined based on the pre-configured default values ​​corresponding to the write thread or persistence thread. For example, if sync_binlog = 1, the transaction commit will be performed after persistence is completed, otherwise the transaction commit will be performed after writing is completed.

[0078] Corresponding to the application scenario and method of the method provided in the embodiment of the present application, the embodiment of the present application also provides a data writing device. Figure 8 FIG2 is a block diagram of a data writing device according to an embodiment of the present application. The data writing device may include:

[0079] The acquisition module 801 is used to acquire log events of multiple transactions.

[0080] The determination module 802 is configured to determine a plurality of temporary storage areas corresponding to a plurality of transactions.

[0081] The writing module 803 is used to write the log events of multiple transactions into the persistent storage area based on multiple temporary storage areas in a parallel processing manner. The log events stored in the persistent storage area are committed to transactions based on the storage engine.

[0082] The data writing device provided in the embodiment of the present application determines temporary storage areas corresponding to multiple transactions in a temporary storage area shared by multiple transactions, and writes log events of multiple transactions into a persistent storage area based on the temporary storage areas corresponding to the multiple transactions through parallel processing. The use of parallel processing can increase the data writing speed, thereby improving the performance of the database management system.

[0083] In a possible implementation, the writing module 803 is specifically configured to:

[0084] By means of parallel processing, log events of multiple transactions are written into their respective corresponding temporary storage areas, log events of multiple transactions are obtained from their respective corresponding temporary storage areas, written into log files, and the log files are written into the persistent storage area.

[0085] In a possible implementation, the writing module 803 is specifically configured to:

[0086] Through parallel processing, based on multiple temporary storage areas, the foreground thread is used to write the logical log events of multiple transactions into the persistent storage area through the physical log file, and the background thread is used to write the logical log events into the persistent storage area through the logical log file.

[0087] In a possible implementation, the writing module 803 is further configured to:

[0088] When the server starts, if the physical log file records logical log events that are not recorded in the logical log file, the unrecorded logical log events are written to the logical log file.

[0089] In one possible implementation, when the writing module 803 obtains log events of multiple transactions from the corresponding temporary storage areas, writes the log files, and writes the log files to the persistent storage area, it is configured to:

[0090] The writing thread is used to obtain log events of multiple transactions from the corresponding temporary storage area and write them into the log file. The persistence thread is used to write the log file into the persistent storage area.

[0091] In one possible implementation, when the writing module 803 uses a writing thread to obtain log events of multiple transactions from the corresponding temporary storage areas and writes them into the log file, it is configured to:

[0092] According to the write notification message, the write thread is used to obtain log events of multiple transactions from the corresponding temporary storage areas and write them into the log file.

[0093] In one possible implementation, when the writing module 803 writes the log file to the persistent storage area using the persistence thread, it is configured to:

[0094] According to the persistence notification message, the persistence thread is used to write the log file into the persistent storage area.

[0095] In a possible implementation, the apparatus further includes a policy determination module configured to:

[0096] It is determined that the configuration information indicates a first execution strategy.

[0097] In a possible implementation, the policy determination module is further configured to:

[0098] Determine that the configuration information indicates a second execution strategy; for any transaction among the multiple transactions, obtain the log event of the transaction from the second temporary storage area corresponding to the transaction, write it into the log file, commit the log file based on the storage engine, and write the log file into the persistent storage area.

[0099] In a possible implementation, the device further includes a locking module, configured to:

[0100] When multiple temporary storage areas corresponding to multiple transactions are determined, a locking operation is performed on a code block that executes an operation of determining the multiple temporary storage areas.

[0101] In a possible implementation, the determining module 802 is specifically configured to:

[0102] For any transaction among the multiple transactions, a temporary storage area corresponding to the transaction is determined according to the data volume of the log events of the transaction.

[0103] The functions of each module in the device of the embodiment of the present application can be referred to the corresponding description in the above method, and have corresponding beneficial effects, which will not be repeated here.

[0104] Figure 9 FIG. 1 is a block diagram of an electronic device for implementing an embodiment of the present application. Figure 9 As shown, the electronic device includes: a memory 910 and a processor 920. The memory 910 stores a computer program that can be run on the processor 920. When the processor 920 executes the computer program, the method in the above embodiment is implemented. The number of the memory 910 and the processor 920 can be one or more.

[0105] The electronic device also includes:

[0106] The communication interface 930 is used to communicate with external devices and perform data exchange transmission.

[0107] If the memory 910, processor 920, and communication interface 930 are implemented independently, the memory 910, processor 920, and communication interface 930 can be connected to each other via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0108] Optionally, in a specific implementation, if the memory 910, the processor 920 and the communication interface 930 are integrated on a chip, the memory 910, the processor 920 and the communication interface 930 can communicate with each other through an internal interface.

[0109] An embodiment of the present application provides a computer-readable storage medium storing a computer program, which implements the method provided in the embodiment of the present application when the program is executed by a processor.

[0110] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes the method provided in the embodiment of the present application.

[0111] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory. The input interface, the output interface, the processor and the memory are connected through an internal connection path. The processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the application.

[0112] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor that supports the Advanced RISC Machines (ARM) architecture.

[0113] Furthermore, optionally, the above-mentioned memory may include a read-only memory and a random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM) and direct memory bus random access memory (DR RAM).

[0114] In the above embodiments, all or part of the embodiments may be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0115] In the description of this specification, the reference terms "one embodiment," "some embodiments," "example," "specific example," or "some examples" mean that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials, or characteristics described may be combined in any appropriate manner in any one or more embodiments or examples. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification, as well as features of different embodiments or examples, unless they are mutually inconsistent.

[0116] In addition, the terms "first", "first", "second", "third", "third" and "third" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, a feature specified with "first", "second", "third", "third" may explicitly or implicitly include at least one of the features. In the description of this application, "plurality" means two or more, unless otherwise clearly specified.

[0117] Any process or method described in the flowchart or otherwise described herein can be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process. The scope of the preferred embodiments of the present application includes other implementations in which the functions may be performed in a different order than shown or discussed, including performing the functions substantially simultaneously or in reverse order depending on the functions involved.

[0118] The logic and / or steps described in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in combination with such instruction execution systems, apparatuses or devices.

[0119] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0120] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the aforementioned integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium. The storage medium may be a read-only memory, a magnetic disk, or an optical disk, etc.

[0121] The above is merely an exemplary embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various modifications or substitutions within the technical scope described in this application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A data writing method, characterized in that: include: Get log events for multiple transactions; Determining a plurality of temporary storage areas corresponding to the plurality of transactions respectively; Writing the log events of the multiple transactions into a persistent storage area based on the multiple temporary storage areas in a parallel processing manner, and performing transaction commit on the log events stored in the persistent storage area based on a storage engine, including: using a foreground thread to write the logical log events of the multiple transactions into the persistent storage area through a physical log file, and using a background thread to write the logical log events into the persistent storage area through a logical log file; When the server is started, if the physical log file records a logical log event that is not recorded in the logical log file, then the unrecorded logical log event is written into the logical log file; Determine that the configuration information indicates a second execution strategy, that is, for any transaction among the multiple transactions, obtain the log event of the transaction from the temporary storage area corresponding to the transaction, write it into the log file, commit the transaction based on the storage engine, and write the log file into the persistent storage area.

2. The method according to claim 1, characterized in that The step of writing the log events of the multiple transactions into the persistent storage area based on the multiple temporary storage areas in a parallel processing manner includes: By means of parallel processing, the log events of the multiple transactions are written into their respective corresponding temporary storage areas, the log events of the multiple transactions are obtained from their respective corresponding temporary storage areas, written into log files, and the log files are written into a persistent storage area.

3. The method according to claim 2, characterized in that The acquiring log events of the plurality of transactions from the respective corresponding temporary storage areas, writing the log events into log files, and writing the log files into the persistent storage area includes: The log events of the multiple transactions are acquired from the corresponding temporary storage areas by using a writing thread, and written into a log file, and the log file is written into a persistent storage area by using a persistence thread.

4. The method according to claim 3, characterized in that The step of obtaining the log events of the plurality of transactions from the corresponding temporary storage areas by using a writing thread and writing the log events into a log file includes: According to the write notification message, the log events of the multiple transactions are acquired from the corresponding temporary storage areas by using a write thread, and are written into the log file.

5. The method according to claim 3, characterized in that The step of writing the log file into the persistent storage area by using a persistent thread includes: According to the persistence notification message, the log file is written into the persistent storage area using a persistence thread.

6. The method according to claim 1, characterized in that The method further comprises: It is determined that the configuration information indicates a first execution strategy, wherein the first execution strategy is to write log events of multiple transactions into a persistent storage area using a write thread and then commit the transactions based on a storage engine.

7. The method according to claim 1, characterized in that The method further comprises: When determining the multiple temporary storage areas respectively corresponding to the multiple transactions, a locking operation is performed on a code block that executes an operation of determining the multiple temporary storage areas.

8. The method according to claim 1, characterized in that The determining of the multiple temporary storage areas corresponding to the multiple transactions respectively includes: For any transaction among the multiple transactions, a temporary storage area corresponding to the transaction is determined according to the data volume of the log event of the transaction.

9. A data writing device, characterized in that: include: The acquisition module is used to obtain log events of multiple transactions; a determination module, configured to determine a plurality of temporary storage areas corresponding to the plurality of transactions; A writing module is configured to write the log events of the multiple transactions into a persistent storage area based on the multiple temporary storage areas in a parallel processing manner, wherein the log events stored in the persistent storage area are committed to a transaction based on a storage engine, including: utilizing a foreground thread to write the logical log events of the multiple transactions into the persistent storage area through a physical log file, and utilizing a background thread to write the logical log events into the persistent storage area through a logical log file; When the server is started, if the physical log file records a logical log event that is not recorded in the logical log file, then the unrecorded logical log event is written into the logical log file; Determine that the configuration information indicates a second execution strategy, that is, for any transaction among the multiple transactions, obtain the log event of the transaction from the temporary storage area corresponding to the transaction, write it into the log file, commit the transaction based on the storage engine, and write the log file into the persistent storage area.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory, wherein the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

11. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images