A method, apparatus, readable storage medium, and electronic device for writing logs.

Abstract

This specification discloses a method, apparatus, readable storage medium, and electronic device for writing logs. This method merges the log buffers of multiple other write nodes reserved by the write node in a distributed database into a shared buffer corresponding to two or more write nodes. When multiple write nodes write logs to the same write node, they only need to operate on the shared buffer in the write node, thus avoiding wasted storage space. When adding nodes to the distributed database, it is not necessary to add a new log buffer for each added node, thus not limiting the overall size of the distributed database. Furthermore, when multiple write nodes write logs to the same write node simultaneously, this method can use atomic operations to assign mutually non-conflicting write start positions to different write nodes, thereby avoiding log write conflicts while maintaining a shared log buffer.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a method, apparatus, readable storage medium, and electronic device for writing logs. Background Technology

[0002] Currently, with the development of the internet, distributed databases are widely used to better protect users' privacy data. To address data loss due to node failures in distributed databases, backing up the data on those nodes is essential.

[0003] Typically, when the second node (hereinafter referred to as the write node) backs up the data in the first node (hereinafter referred to as the write node), the write node needs to write the log of its own data operation to the write node. The write node then replays the written log to achieve the backup of the write node's data.

[0004] How to enable the writing node to write logs to the writing node is a problem that urgently needs to be solved. Summary of the Invention

[0005] This specification provides a method, apparatus, readable storage medium, and electronic device for writing logs, to at least partially solve the aforementioned problems existing in the prior art.

[0006] The following technical solution is adopted in this specification:

[0007] This specification provides a method for writing logs, which is applied to a write node in a distributed database, wherein the write node in the distributed database includes a shared buffer corresponding to two or more write nodes; the method includes:

[0008] The write node generates logs to be written;

[0009] Initiate an atomic operation on the node being written to, and receive the return value returned by the node being written to based on the atomic operation;

[0010] Based on the return value, determine the starting position for writing the log to be written in the shared buffer;

[0011] Based on the starting position, the log to be written is written to the shared buffer of the node being written.

[0012] Optionally, the write node generates a log to be written, specifically including:

[0013] The write node generates a log to be written with a preset length;

[0014] When two or more write nodes simultaneously initiate atomic operations on the written node, the return values ​​returned by the written node to the two or more write nodes are all different, and the interval between each return value is at least the preset length.

[0015] Optionally, an atomic operation is initiated on the node being written to, specifically including:

[0016] Initiate a FAA operation based on Remote Direct Data Access (RDMA) on the node to be written.

[0017] Optionally, the shared buffer is a circular buffer;

[0018] Writing the log to be written to the shared buffer of the node being written to specifically includes:

[0019] The writing node determines the header position of the valid log stored in the shared buffer of the node being written to, and determines the tail position of the valid log stored in the shared buffer after writing the log to be written to the shared buffer.

[0020] Determine whether the storage space from the head position to the tail position is greater than the storage space of the shared buffer;

[0021] If not, the log to be written is written to the shared buffer of the node being written to.

[0022] Optionally, when the storage space corresponding to the head position to the tail position is greater than the storage space of the shared buffer, the method further includes:

[0023] Read the header position of the valid log stored in the shared buffer from the node being written to;

[0024] Determine whether the storage space from the beginning position to the end position of the read data is greater than the storage space of the shared buffer;

[0025] If so, the header position of the valid log stored in the shared buffer is read again from the written node until the storage space from the read header position to the tail position is not greater than the storage space of the shared buffer.

[0026] If not, the log to be written is written to the shared buffer of the node being written to.

[0027] Optionally, after reading the header position of the valid log stored in the shared buffer from the written node, the method further includes:

[0028] The header position of the valid log stored in the shared buffer of the written node, as recorded by the writing node itself, is updated to the read header position.

[0029] Optionally, writing the log to be written to the shared buffer of the node being written to specifically includes:

[0030] The log to be written is written to the shared buffer of the node being written through a write operation based on Remote Direct Data Access (RDMA).

[0031] This specification provides an apparatus for writing logs, the apparatus being applied to a write node in a distributed database, wherein the write node in the distributed database includes a shared buffer corresponding to two or more write nodes; the apparatus includes:

[0032] The generation module is used to generate logs to be written.

[0033] The positioning module is used to initiate an atomic operation on the node being written to and to receive the return value returned by the node being written to based on the atomic operation.

[0034] The determination module is used to determine the starting position for writing the log to be written in the shared buffer based on the return value;

[0035] The writing module is used to write the log to be written into the shared buffer of the node being written, according to the starting position.

[0036] Optionally, the generation module is specifically used to generate a log to be written with a preset length; when two or more writing nodes simultaneously initiate atomic operations on the written node, the return values ​​returned by the written node to the two or more writing nodes are different, and the interval between each return value is at least the preset length.

[0037] Optionally, the positioning module is specifically used to initiate a FAA operation based on Remote Direct Data Access (RDMA) on the node being written to.

[0038] Optionally, the shared buffer is a circular buffer;

[0039] The writing module is specifically used to: determine the head position of the valid log stored in the shared buffer of the node to be written, as recorded by the device; determine the tail position of the valid log stored in the shared buffer after the log to be written is written to the shared buffer; determine whether the storage space corresponding to the head position to the tail position is greater than the storage space of the shared buffer; if not, write the log to be written to the shared buffer of the node to be written.

[0040] Optionally, the writing module is further configured to: when the storage space corresponding to the head position to the tail position is greater than the storage space of the shared buffer, read the head position of the valid log stored in the shared buffer from the node being written; determine whether the storage space corresponding to the read head position to the tail position is greater than the storage space of the shared buffer; if so, reread the head position of the valid log stored in the shared buffer from the node being written until the storage space corresponding to the read head position to the tail position is not greater than the storage space of the shared buffer; if not, write the log to be written into the shared buffer of the node being written.

[0041] Optionally, the writing module is further configured to, after reading the header position of the valid log stored in the shared buffer from the written node, update the header position of the valid log stored in the shared buffer of the written node recorded by the device to the read header position.

[0042] Optionally, the writing module is specifically used to write the log to be written into the shared buffer of the node being written through a write operation based on Remote Direct Data Access (RDMA).

[0043] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for writing logs.

[0044] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described method for writing logs.

[0045] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:

[0046] As can be seen from the above method, this approach merges the log buffers of multiple other write nodes reserved by the write node in the distributed database into a shared buffer corresponding to two or more write nodes. When multiple write nodes write logs to the same write node, they can all operate on the shared buffer in the write node, thus avoiding storage space waste. When the distributed database adds nodes, it is not necessary to add a new log buffer for each added node, and the scale of the entire distributed database is not limited. Furthermore, when multiple write nodes write logs to the same write node simultaneously, this method can use atomic operations to assign mutually exclusive write start positions to different write nodes, thereby avoiding log write conflicts while sharing the log buffer. Attached Figure Description

[0047] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:

[0048] Figure 1 This is a flowchart illustrating a method for writing logs as provided in this specification.

[0049] Figure 2 This is a schematic diagram of the node structure of the distributed database provided in this specification;

[0050] Figure 3 This is a schematic diagram of the circular buffer of a node in the distributed database provided in this specification;

[0051] Figure 4 This is a schematic diagram showing the various locations where logs are written during execution in this manual;

[0052] Figure 5 This is a schematic diagram of a log writing device provided in this specification;

[0053] Figure 6 The corresponding information provided in this specification Figure 1 A schematic diagram of an electronic device. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort are within the scope of protection of this application.

[0055] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0056] Figure 1 This document provides a flowchart illustrating a method for writing logs, which includes the following steps:

[0057] S100: The writing node generates a log to be written.

[0058] In this embodiment of the application, the write node in the distributed database can perform actions such as... Figure 1The method shown is used to write the logs of the writing node to other nodes in the distributed database. Here, the writing node refers to the node that needs to write logs to other nodes, and the corresponding writing node refers to the node that needs to receive and store the logs written by the writing node. The writing node and the writing node mentioned in this specification can be any electronic device such as a desktop computer or laptop computer capable of executing the scheme described in this specification. Based on this, in this specification, the writing node generates a log to be written. The log to be written is generated based on data operation behavior, where the data operation is an operation performed on the data stored by the writing node itself. It is called a log to be written because it is to be written but not yet written to the writing node, and can be written to the writing node later.

[0059] The nodes being written to in the aforementioned distributed database include shared buffers corresponding to two or more write nodes, such as... Figure 2 As shown. If multiple write nodes writing logs to the write node correspond to the same shared buffer, then all write nodes write logs to this shared buffer, instead of writing to the log buffer reserved for each write node by the write node, thus solving the problem of wasted storage space. For example, in a distributed database with 100 nodes, the write node has a shared buffer 1 corresponding to 50 of the 100 nodes and a shared buffer 2 corresponding to the other 50 nodes. Of course, the write node can also have only one shared buffer corresponding to all nodes in the distributed database. In this case, other nodes in the distributed database, i.e., multiple write nodes, only need to operate on this one shared buffer when writing logs to the write node.

[0060] However, when different write nodes simultaneously write logs to the same write node, the different write nodes may write to the same location in the shared buffer of the same write node, which will cause write log conflicts.

[0061] S102: Initiate an atomic operation on the node being written to, and receive the return value returned by the node being written to based on the atomic operation.

[0062] To resolve the aforementioned conflict issue, when two or more write nodes simultaneously write logs to the node being written to—that is, different write nodes simultaneously write to the same location in the shared buffer of the node being written to—a write conflict will occur if atomic operations are not used. However, by initiating atomic operations on the node being written to, the return values ​​returned by the node being written to the two or more write nodes can be made different.

[0063] For example, node 1 (the writing node) and node 2 (the writing node) simultaneously write logs to node 3 (the node being written to). Node 1 and node 2 simultaneously initiate atomic operations on node 3. Node 1 receives the return value x returned by node 3 based on the atomic operation, and node 2 receives the return value x+c returned by node 3 based on the atomic operation, where c is a positive integer.

[0064] Specifically, the aforementioned distributed database can be a distributed database based on Remote Direct Memory Access (RDMA) technology, and the atomic operation initiated on the node to be written can be an atomic fetch and atomic add (FAA) operation of RDMA.

[0065] S104: Based on the return value, determine the starting position for writing the log to be written in the shared buffer.

[0066] Based on the return value received in step S102, the starting position for writing the log to be written in the shared buffer of the node being written can be determined. For example, the return value received in step S102 can be directly used as the starting position for writing the log to be written.

[0067] Of course, besides directly using the received return value as the starting position for writing the log, other methods can be used to determine the starting position based on the received return value, as long as the starting position determined by different return values ​​is different.

[0068] S106: Based on the starting position, write the log to be written to the shared buffer of the node being written.

[0069] Since different write nodes write logs to the same write node at the same time, the write position is the same, which will cause write conflict. However, according to step S104, the starting position of the log to be written in the shared buffer can be different for different write nodes, that is, the log writing position is different, so the write conflict problem can be avoided.

[0070] Specifically, when the aforementioned distributed database is a distributed database based on RDMA technology, the log to be written can be written to the shared buffer of the node being written through the RDMA write operation.

[0071] based on Figure 1The log writing method shown merges the log buffers reserved for other write nodes in the distributed database into a shared buffer corresponding to two or more write nodes. When multiple write nodes write logs to the same write node, they all operate on the shared buffer in the write node, thus avoiding storage space waste. When adding nodes to the distributed database, there is no need to add a new log buffer for each added node, which does not limit the size of the entire distributed database. Although the shared buffer solves the space waste problem, it may cause write conflicts during application. This method, however, uses atomic operations so that different write nodes receive different return values, thereby determining different write log start positions. That is, it assigns non-conflicting write start positions to different write nodes, thus avoiding write log conflicts while avoiding storage space waste by using a shared log buffer.

[0072] The log to be written in the above method can be of any length. To facilitate the operation in step S102, a fixed-length log to be written can also be generated in step S100. The fixed length can be preset, that is, each time the write node generates a log to be written in step S100, it generates a log of a preset length. If the log to be written is of a preset length, then in step S102, the interval between different return values ​​received by different write nodes is at least the preset length. For example, if two write nodes simultaneously initiate atomic operations on the same written node, and one write node receives a return value of x, while the other write node receives a return value of x+c, where c is at least the preset length, then in step S104, the interval between the starting positions of the write logs determined by different write nodes based on the different return values ​​received is also at least the preset length, so that the logs written by different write nodes do not conflict.

[0073] In the embodiments described in this specification, the shared buffer of the node being written to can specifically be a circular buffer, such as... Figure 3 As shown, the shared buffer is a circular buffer, and the clockwise writing of logs is used as an example for illustration.

[0074] In step S106 above, when a node writes logs, it may overwrite logs that have already been written to but not committed or rolled back in the shared buffer of the node being written to (hereinafter, logs that have been written to but not committed or rolled back in the shared buffer will be referred to as valid logs). When the storage space of the circular buffer of the node being written to is completely filled with logs, if writing continues, it can only overwrite the already written logs and continue writing the logs to be written. The overwritten logs may be valid logs, for example, in Figure 3In the current system, logs 1-8 have already been written to the circular buffer. If log 9 needs to be written, it can only be done by overwriting log 1 within the storage space occupied by log 1. If log 1 is a valid log, then overwriting it with log 9 will cause errors in subsequent operations.

[0075] To avoid overwriting valid logs during writing, it is necessary to determine whether the writing of the log to be written will overwrite valid logs before proceeding to step S106.

[0076] In the embodiments of this specification, the writing node can record the header position of the valid log stored in the shared buffer of the writing node. Therefore, when determining whether the log to be written will overwrite the valid log after being written, it can specifically determine the tail position of the valid log stored in the shared buffer after the log to be written is written to the shared buffer of the writing node. Then, it can determine whether the storage space corresponding to the determined header position of the valid log stored in the shared buffer to the determined tail position of the valid log after the log to be written is written to the shared buffer of the writing node is greater than the storage space of the shared buffer of the writing node. If not, it is determined that the log to be written will not overwrite the valid log after being written, and the log to be written is written to the shared buffer of the writing node. If yes, it is determined that the log to be written will overwrite the valid log after being written, and the log to be written cannot be written to the shared buffer of the writing node for the time being.

[0077] The reason for using the above method to determine whether the log to be written will overwrite the valid log is because the shared buffer is a circular buffer. Figure 4 Taking this as an example, the log is written in a clockwise direction. Figure 4 The shared buffer shown indicates that if the header position of the valid log recorded by the currently writing node is position 'a', then... Figure 1 The starting position of the log to be written, determined in step S104, is position b, which is the end of log 8. After writing the log to be written, the end position of the valid log is position c.

[0078] It should be noted that, in Figure 4 Although positions a and b appear to be at the same location, position b is actually obtained by adding 8 logs (log 1 to log 8) clockwise from position a. Therefore, the length from position a to position b is actually the length of the entire shared buffer's storage space, which is 8 logs. If more logs to be written are added, the length of 1 log is added from the starting position b. The end position of the valid log is position c. At this time, the length from position a to position c is 9 logs, which is greater than the length of the entire shared buffer. Therefore, the valid logs will be overwritten, that is, log 1 will be overwritten.

[0079] As can be seen, the above-described judgment method relies on the header position of the valid log stored in the shared buffer of the node being written to, which is recorded by the writing node itself. In practical applications, the node being written to also records the header position of the valid log stored in its own shared buffer. The header position of the valid log stored in the shared buffer of the node being written to, as recorded by the writing node itself, can be updated synchronously with the header position of the valid log stored in the shared buffer of the node being written to. For example, whenever the header position of the valid log stored in the shared buffer of the node being written to changes, the writing node is notified of the changed header position, causing the writing node to update the header position of the valid log stored in the shared buffer of the node being written to, as recorded by itself. Alternatively, the writing node periodically reads the header position of the valid log stored in the shared buffer of the node being written to, as recorded by itself, and updates the header position of the valid log stored in the shared buffer of the writing node itself based on the read header position.

[0080] However, the synchronous update method described above incurs significant network overhead between the writing node and the written node. Therefore, in this embodiment, the header position of the valid log stored in the shared buffer of the written node, recorded by the writing node itself, is updated asynchronously. Specifically, when the writing node determines that the log to be written will not overwrite the valid log, it does not need to update the header position of the valid log stored in the shared buffer. However, when the writing node determines that the log to be written will overwrite the valid log, it needs to read the header position of the valid log stored in the shared buffer from the written node and update the header position of the valid log stored in the shared buffer of the written node recorded by the writing node itself to the read header position.

[0081] After updating the header position of the valid log stored in the shared buffer of its own records, it can again determine whether the storage space corresponding to the read header position to the determined tail position is greater than the storage space of the shared buffer. If so, the header position of the valid log stored in the shared buffer is read again from the node being written to, until the storage space corresponding to the read header position to the determined tail position is not greater than the storage space of the shared buffer. If not, the log to be written is written to the shared buffer of the node being written to.

[0082] Continue with Figure 4Taking an example, position a is the beginning of the valid log stored in the shared buffer of the node being written, as recorded by the writing node itself. Position b is the starting position for writing the log to be written. The length from position a to position b is actually the length of the entire shared buffer's storage space, i.e., the length of 8 logs. Position c is the end of the log to be written after it has been written into the shared buffer of the node being written. Position c is determined based on position b and the length of the log to be written. Position d is the beginning of the valid log stored in the shared buffer of the node being written itself. Figure 4 As shown, the storage space between positions a and c is the storage space of the circular buffer plus the storage space occupied by log 1, which is the length of 9 logs. This is greater than the storage space of the shared buffer of the node being written to. Therefore, the log to be written will overwrite the valid log, specifically log 1. The writing node reads the header position of the valid log stored in the shared buffer from the node being written to, i.e., position d, and updates position a to the read position d. In other words, the position pointed to by position a is updated to the position pointed to by position d.

[0083] After updating the header position of the valid log stored in its own shared buffer, the node again checks whether the storage space between the read header position (i.e., position d) and position c is greater than the storage space of the shared buffer. In this example, the length of the storage space between position d and position c is the length of log 5 to log 1 in a clockwise direction, i.e., the length of 5 logs, which is less than the length of the entire shared buffer storage space, i.e., the length of 8 logs. Therefore, the writing node determines that the log to be written will not overwrite the valid log after writing, and then writes the log to be written to the shared buffer of the node being written to.

[0084] Additionally, the way the node being written to records the header position of the valid log stored in the shared buffer can be achieved by maintaining a pointer to the shared buffer. Specifically, the node being written to can maintain a pointer to the header position of the valid log in the shared buffer, and record the header position of the valid log in the shared buffer by using the position pointed to by this pointer. (Continuing with...) Figure 4 Taking an example, if the pointer of the shared buffer of the currently being written node points to position d, then the header position of the valid log stored in the shared buffer of the node being written to is position d. Assuming log 5 has been committed or rolled back, then log 5 is invalid for the node being written to. The pointer of the shared buffer moves clockwise by one log length, and the position it points to is updated to the header of log 6. Therefore, the header position of the valid log recorded by the node being written to is updated to the header position of log 6. Each time a log is committed or rolled back, the pointer of the shared buffer moves forward by the length of the committed or rolled-back log, and the position it points to is updated to the header position of the next log. Therefore, the header position of the valid log recorded by the node being written to is updated to the header position of the next log.

[0085] Based on the above method of writing logs, writing nodes can write logs without wasting storage space and without conflicting with each other, avoiding possible log overwriting.

[0086] Figure 5 This is a schematic diagram of a log writing device provided in this specification. The device is applied to a write node in a distributed database, wherein the write node in the distributed database includes a shared buffer corresponding to two or more write nodes. The device specifically includes:

[0087] Module 500 is used to generate logs to be written.

[0088] The positioning module 502 is used to initiate an atomic operation on the node being written to and to receive the return value returned by the node being written to based on the atomic operation.

[0089] The determining module 504 is used to determine the starting position of the log to be written in the shared buffer based on the return value;

[0090] The writing module 506 is used to write the log to be written into the shared buffer of the node being written, according to the starting position.

[0091] Optionally, the generation module 500 is specifically used to generate a log to be written with a preset length; when two or more writing nodes simultaneously initiate atomic operations on the written node, the return values ​​returned by the written node to the two or more writing nodes are different, and the interval between each return value is at least the preset length.

[0092] Optionally, the positioning module 502 is specifically used to initiate a FAA operation based on Remote Direct Data Access (RDMA) on the node being written to.

[0093] Optionally, the shared buffer is a circular buffer;

[0094] The writing module 506 is specifically used to: determine the head position of the valid log stored in the shared buffer of the node to be written, as recorded by the device; determine the tail position of the valid log stored in the shared buffer after the log to be written is written to the shared buffer; determine whether the storage space corresponding to the head position to the tail position is greater than the storage space of the shared buffer; if not, write the log to be written to the shared buffer of the node to be written.

[0095] Optionally, the writing module 506 is further configured to: when the storage space corresponding to the head position to the tail position is greater than the storage space of the shared buffer, read the head position of the valid log stored in the shared buffer from the node being written; determine whether the storage space corresponding to the read head position to the tail position is greater than the storage space of the shared buffer; if so, reread the head position of the valid log stored in the shared buffer from the node being written until the storage space corresponding to the read head position to the tail position is not greater than the storage space of the shared buffer; if not, write the log to be written into the shared buffer of the node being written.

[0096] Optionally, the writing module 506 is further configured to, after reading the header position of the valid log stored in the shared buffer from the written node, update the header position of the valid log stored in the shared buffer of the written node recorded by the device to the read header position.

[0097] Optionally, the writing module 506 is specifically used to write the log to be written into the shared buffer of the node being written through a write operation based on Remote Direct Data Access (RDMA).

[0098] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 Provides methods for writing logs.

[0099] This instruction manual also provides Figure 6 The diagram shows the structure of the electronic device. Figure 6 At the hardware level, the autonomous driving device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for various operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 1 The provided method for writing to the log. Of course, in addition to the software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0100] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0101] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, ASICs, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0102] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0103] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0104] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0105] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0106] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0107] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0108] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0109] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0110] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0111] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0112] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0113] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0114] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0115] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this application.

Claims

1. A method for writing logs, the method being applied to a write node in a distributed database, wherein the write node in the distributed database has a shared buffer, the shared buffer being used by two or more write nodes to jointly write logs; the method includes: The write node generates logs to be written; Initiate an atomic operation on the node being written to, and receive the return value returned by the node being written to based on the atomic operation; Based on the return value, determine the starting position for writing the log to be written in the shared buffer; Based on the starting position, the log to be written is written to the shared buffer of the node being written.

2. The method as described in claim 1, wherein the writing node generates the log to be written, specifically including: The write node generates a log to be written with a preset length; When two or more write nodes simultaneously initiate atomic operations on the written node, the return values ​​returned by the written node to the two or more write nodes are all different, and the interval between each return value is at least the preset length.

3. The method as described in claim 1, wherein an atomic operation is initiated on the node being written to, specifically including: Initiate a FAA operation based on Remote Direct Data Access (RDMA) on the node to be written.

4. The method as described in claim 1, wherein the shared buffer is a circular buffer; Writing the log to be written to the shared buffer of the node being written to specifically includes: The writing node determines the header position of the valid log stored in the shared buffer of the node being written to, and determines the tail position of the valid log stored in the shared buffer after writing the log to be written to the shared buffer. Determine whether the storage space from the head position to the tail position is greater than the storage space of the shared buffer; If not, the log to be written is written to the shared buffer of the node being written to.

5. The method of claim 4, wherein when the storage space corresponding to the head position to the tail position is greater than the storage space of the shared buffer, the method further comprises: Read the header position of the valid log stored in the shared buffer from the node being written to; Determine whether the storage space from the beginning position to the end position of the read data is greater than the storage space of the shared buffer; If so, the header position of the valid log stored in the shared buffer is read again from the written node until the storage space from the read header position to the tail position is not greater than the storage space of the shared buffer. If not, the log to be written is written to the shared buffer of the node being written to.

6. The method of claim 5, after reading the header position of the valid log stored in the shared buffer from the written node, the method further includes: The header position of the valid log stored in the shared buffer of the written node, as recorded by the writing node itself, is updated to the read header position.

7. The method as described in claim 1, wherein writing the log to be written to the shared buffer of the node being written to specifically includes: The log to be written is written to the shared buffer of the node being written through a write operation based on Remote Direct Data Access (RDMA).

8. An apparatus for writing logs, the apparatus being applied to a write node in a distributed database, wherein the write node in the distributed database has a shared buffer for two or more write nodes to jointly write logs; the apparatus comprises: The generation module is used to generate logs to be written. The positioning module is used to initiate an atomic operation on the node being written to and to receive the return value returned by the node being written to based on the atomic operation. The determination module is used to determine the starting position for writing the log to be written in the shared buffer based on the return value; The writing module is used to write the log to be written into the shared buffer of the node being written, according to the starting position.

9. The apparatus of claim 8, wherein the generation module is specifically used to generate a log to be written with a preset length; when two or more writing nodes simultaneously initiate atomic operations on the written node, the return values ​​returned by the written node to the two or more writing nodes are all different, and the interval between each return value is at least the preset length.

10. The apparatus of claim 8, wherein the positioning module is specifically configured to initiate a FAA operation based on Remote Direct Data Access (RDMA) on the written node.

11. The apparatus of claim 8, wherein the shared buffer is a circular buffer; The writing module is specifically used to: determine the head position of the valid log stored in the shared buffer of the node to be written, as recorded by the device; determine the tail position of the valid log stored in the shared buffer after the log to be written is written to the shared buffer; determine whether the storage space corresponding to the head position to the tail position is greater than the storage space of the shared buffer; if not, write the log to be written to the shared buffer of the node to be written.

12. The apparatus of claim 11, wherein the writing module is further configured to: when the storage space corresponding to the head position to the tail position is greater than the storage space of the shared buffer, read the head position of the valid log stored in the shared buffer from the node being written; determine whether the storage space corresponding to the read head position to the tail position is greater than the storage space of the shared buffer; if so, reread the head position of the valid log stored in the shared buffer from the node being written until the storage space corresponding to the read head position to the tail position is not greater than the storage space of the shared buffer; if not, write the log to be written into the shared buffer of the node being written.

13. The apparatus of claim 12, wherein the writing module is further configured to, after reading the header position of the valid log stored in the shared buffer from the written node, update the header position of the valid log stored in the shared buffer of the written node recorded by the apparatus to the read header position.

14. The apparatus of claim 8, wherein the writing module is specifically configured to write the log to be written into the shared buffer of the node being written by means of a write operation based on Remote Direct Data Access (RDMA).

15. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any one of claims 1 to 7.

16. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any one of claims 1 to 7.

Images