A log sharing system, method, apparatus, storage medium and electronic device

Abstract

The embodiment of the specification discloses a log sharing system, which is applied to a distributed database system, comprises a shared storage node and a plurality of computing nodes, each computing node generates a database log corresponding to a data operation according to a data operation request, and then stores the database log in a local storage space; when an uploading condition is met, the database log in the local storage space is uploaded to the shared storage node, the shared storage node stores each database log uploaded by each computing node, and when receiving a log acquisition request sent by any computing node, the corresponding database log is returned to the computing node. Through the above system, the database logs generated by each computing node can be uniformly stored by the shared storage node in the distributed database system, so that the entire event process can be restored according to all the database logs when analyzing system behavior and troubleshooting system faults, thereby facilitating analysis and troubleshooting.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a log sharing system, method, apparatus, storage medium, and electronic device. Background Technology

[0002] With the development of the times, the demand for data storage has increased dramatically. Currently, distributed database systems are often used to meet the storage needs of ultra-large-scale data.

[0003] Generally, distributed database systems typically have multiple compute nodes, which can collectively handle the same business logic through multiple process instances or microservices. This can lead to logs generated when data related to that business logic changes being scattered across different physical nodes or containers on these compute nodes.

[0004] When developers, testers, and operations personnel analyze system behavior, the logs from the aforementioned distributed database systems cannot fully reconstruct the course of events, nor are they conducive to centralized analysis and rapid troubleshooting. Therefore, how to aggregate logs scattered across various nodes is an urgent problem to be solved. Summary of the Invention

[0005] This specification provides a log sharing system, method, apparatus, storage medium, and electronic device to partially solve the problems existing in the prior art.

[0006] The embodiments in this specification adopt the following technical solutions: This specification provides a log sharing system, which includes a shared storage node and several computing nodes; wherein: The computing node is used to receive data operation requests, determine each data operation in the database transaction corresponding to the data operation request, generate a database log corresponding to each data operation, store the database log in the local storage space of the computing node, and upload each database log stored in the local storage space to the shared storage node when the upload conditions are met. The shared storage node is used to store the database logs uploaded by each computing node from their respective local storage space, and when it receives a log retrieval request from any computing node, it returns the database logs stored in the shared storage node corresponding to the log retrieval request.

[0007] This specification provides a log sharing method, the method comprising: The compute node receives data operation requests; Determine the data operations within the database transaction corresponding to the data operation request; Generate database logs for each data operation; The database logs are stored in the local storage space of the computing node; When the upload conditions are met, the database logs stored in the local storage space are uploaded to the shared storage node. The shared storage node is used to store the database logs stored in the local storage space of each computing node and, upon receiving a log retrieval request from any computing node, returns the database logs stored in the shared storage node corresponding to the log retrieval request.

[0008] This specification provides a log sharing method, the method comprising: The shared storage node receives database logs stored in the local storage space of the computing node when the upload conditions are met. The database logs are generated by the computing node after receiving a data operation request, based on the data operations in the database transaction corresponding to the data operation request. Store the received logs from each database; Upon receiving a log retrieval request from the computing node, the system returns the stored database log corresponding to the log retrieval request.

[0009] This specification provides a log sharing device, the device comprising: The receiving module is used to receive data operation requests; The determination module is used to determine each data operation in the database transaction corresponding to the data operation request; The generation module is used to generate database logs for each data operation. A storage module is used to store the database logs in the local storage space of the computing node; The upload module is used to upload the database logs stored in the local storage space to the shared storage node when the upload conditions are met. The shared storage node is used to store the database logs uploaded by each computing node from their respective local storage spaces, and when it receives a log retrieval request from any computing node, it returns the database logs stored in the shared storage node corresponding to the log retrieval request.

[0010] This specification provides a log sharing device, the device comprising: The receiving module is used to receive the database logs stored in the local storage space of the computing node when the upload conditions are met. The database logs are generated by the computing node after receiving the data operation request, based on the data operations in the database transaction corresponding to the data operation request. The storage module is used to store the received database logs; The log sharing module is used to return the stored database log corresponding to the log retrieval request when it receives a log retrieval request sent by the computing node.

[0011] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described log sharing method.

[0012] This specification provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described log sharing method.

[0013] This specification provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described log sharing method.

[0014] The above-described at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects: This specification discloses a log sharing system applied to a distributed database system. The system includes a shared storage node and several computing nodes. Each computing node generates a database log corresponding to a data operation based on a data operation request, and first stores the database log in its local storage space. When upload conditions are met, the database log in the local storage space is uploaded to the shared storage node, which stores all the database logs uploaded by the computing nodes. When a log retrieval request is received from any computing node, the system returns the corresponding database log to that computing node. Through this system, the shared storage node in the distributed database system can uniformly store the database logs generated by each computing node. This facilitates the reconstruction of the entire event process based on all the database logs when analyzing system behavior and troubleshooting system faults, thereby aiding in analysis and troubleshooting. Attached Figure Description

[0015] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings: Figure 1 A schematic diagram of a log sharing system provided in the embodiments of this specification; Figure 2 This is a schematic flowchart of the log sharing method provided in the embodiments of this specification; Figure 3 This is a schematic diagram of a first type of log sharing device provided in the embodiments of this specification; Figure 4This is a schematic diagram of a second log sharing device provided in the embodiments of this specification; Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this specification. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0017] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0018] Figure 1 This diagram illustrates a log sharing system provided in an embodiment of this specification. The system is applied to a distributed database system, which includes at least shared storage nodes and several computing nodes. Wherein: The compute node, located at the front end, provides database services to the business, including basic database services such as adding, deleting, modifying, and querying stored data based on received data operation requests. These data operation requests include both Data Manipulation Language (DML) operation requests (such as DML operations executed via SQL statements for adding, deleting, modifying, and querying data) and Data Definition Language (DDL) operation requests (used to define or modify the metadata corresponding to the stored data).

[0019] In a distributed database system, a single compute node can store copies of data corresponding to different partitions within a database instance. These copies on different compute nodes contain identical data. This is because a database instance often contains multiple tables, each with a massive amount of data. To ensure reliability, these tables are typically partitioned. For example, data starting with the key a~m might be partitioned into one partition, and data starting with the key n~z into another. Each compute node in the distributed database system stores a copy of each partition. This way, if one compute node fails, other compute nodes can immediately take over and continue providing database services, achieving disaster recovery.

[0020] It's important to note that while data replicas stored on different compute nodes may correspond to the same data, data replicas stored on different data stores may differ. This is because data replicas are categorized into three types: full replicas, original data replicas, and log replicas. A full replica includes both original and log replicas, while the original data replica only contains the data itself. For example, the data for users Zhang San and Li Si exists only in the original data replica. A log replica, on the other hand, only includes the database logs generated by data operations. For instance, if Zhang San's account balance is updated from 100 yuan to 90 yuan, and Li Si's account balance is updated from 50 yuan to 60 yuan, the corresponding database logs for these two data operations only exist in the log replica. In other words, although data replicas stored on different compute nodes may correspond to the same data, the data replicas stored on different compute nodes may not be identical. Figure 1 Compute nodes 1 and 2 may store full replicas, but compute node 3 only stores log replicas to save storage overhead.

[0021] These computing nodes include a master node and several other slave nodes. The master node receives the aforementioned data operation request, determines the database transaction corresponding to the data operation request, determines each data operation in the database transaction, and then generates a database log corresponding to each data operation.

[0022] It should be noted that the database logs described in this specification are not ordinary event logs, but database logs based on the Write-Ahead Logging (WAL) mechanism. The WAL mechanism means that when the database system receives a data operation request, it does not directly modify the data according to the data operation corresponding to the request. Instead, it first generates the corresponding database log for the data operation, and then modifies the data based on that database log.

[0023] WAL is a mechanism developed to ensure data atomicity. Generally speaking, WAL is often used in conjunction with two-phase commit.

[0024] For example, suppose Zhang San's account balance is 100 yuan and Li Si's account balance is 50 yuan. Now, Zhang San wants to transfer 10 yuan to Li Si. The database transaction corresponding to the data operation request sent by the business system to the aforementioned distributed database system includes two data operations: first, changing Zhang San's account balance from 100 yuan to 90 yuan; and second, changing Li Si's account balance from 50 yuan to 60 yuan. If Zhang San's data and Li Si's data reside in different partitions, this will result in cross-partition data operations. Although each compute node has a copy of the data corresponding to each partition, the master nodes corresponding to different partitions may not be the same compute node. This will lead to the master node corresponding to the partition where Zhang San's data resides (hereinafter referred to as node A) being responsible for changing Zhang San's data, and the master node corresponding to the partition where Li Si's data resides (hereinafter referred to as node B) being responsible for changing Li Si's data. If node A successfully changes Zhang San's data from 100 yuan to 90 yuan, while node B fails to change Li Si's data from 50 yuan to 60 yuan, then, from an overall perspective, Zhang San's 10 yuan has "disappeared." This is the result of violating the atomicity of data.

[0025] To ensure data atomicity—meaning that all data operations within the same database transaction either succeed or fail—the current WAL (Write-Ahead Log) mechanism combined with two-phase commit works as follows: First, each data operation within a database transaction generates a corresponding database log. Then, one of nodes, A or B, acts as the coordinator, and the other acts as a participant. The coordinator initiates the preparation phase. During the preparation phase, the coordinator and all participants need to confirm that they have obtained the database logs corresponding to the data operations within the database transaction (e.g., in the example above, node A has obtained the database log for changing Zhang San's data from 100 yuan to 90 yuan, and node B has obtained the database log for changing Li Si's data from 50 yuan to 60 yuan). If at least one participant or coordinator has not obtained the corresponding database log, the two-phase commit ends, and all nodes temporarily refrain from changing data. If all participants and the coordinator have obtained the corresponding database logs, the coordinator then initiates the commit phase. During the commit phase, all participants and coordinators perform corresponding data operations based on the database log corresponding to their data operations within the database transaction. That is, they modify the data according to the database log. If at least one participant or coordinator fails to modify the data according to the database log, all participants or coordinators abandon their current data changes, maintaining the data in its pre-modification state. Only if all participants and coordinators successfully modify the data is the database transaction considered successfully executed. This is the current method of ensuring data atomicity through WAL (Write-Ahead Logging) in conjunction with two-phase commit. The database log described in this specification is precisely the database log generated based on the WAL mechanism.

[0026] As seen through the WAL combined with two-phase commit method described above, the reason for the breach in data atomicity mainly stems from cross-partition data operations. This is because the logs generated when data in different partitions are operated on logically belong to different log streams. Simply put, each partition corresponds to a different log stream, and cross-partition data operations are cross-log stream data operations. If it is a single log stream data operation (such as Zhang San and Li Si's data being in the same partition in the example above), the problem of data atomicity being breached will not occur. In reality, when the database instance is large, a database instance often has millions of partitions, which means millions of log streams. Using the WAL combined with two-phase commit method described above will generate huge computational costs. If these log streams can be unified into a single log stream, then the data operation will become a single log stream data operation, which can naturally avoid the problem of breaching data atomicity.

[0027] Therefore, in order to facilitate the provision of complete database logs when centrally analyzing system failures, and to transform the multiple log streams corresponding to different partitions into a single log stream, this specification stipulates that after the master node determines the corresponding database transaction based on the received data operation request and generates the database log corresponding to each data operation in the database transaction, it can first store the database log in the local storage space of the master node. When the preset upload conditions are met, the database logs stored in the local storage space are uploaded to the shared storage node.

[0028] Figure 1 The shared storage node in the system shown is located in the background and is used to store the database logs uploaded by each compute node to its local storage space. The shared storage node uses shared storage to store the database logs, while each compute node uses object storage to store data copies of its partitions.

[0029] Specifically, for a compute node, it can either directly store data copies or store data copies through its corresponding storage node. Regardless of the method, data is stored using object storage. This is because object storage requires data access via an API. When data needs to be modified, the entire data to be modified is retrieved as an access object and placed into the compute node's memory. The modified access object is then stored in memory at its original location. This approach is suitable for data modification, especially for transactional (TP) databases.

[0030] For shared storage nodes, their function is to store database logs. They provide the necessary database logs only when needed, without requiring modification. Instead, they need to handle frequent access to the stored database logs from other nodes. Therefore, shared storage nodes use a shared storage method to store database logs. Shared storage refers to block storage or file storage. Each compute node can logically access this shared storage node as its own storage space (e.g., through RDMA). Each compute node can directly create a file system on the shared storage node or perform byte-level read and write operations directly without needing to access it through an API.

[0031] Shared storage nodes store database logs uploaded by each compute node. When a shared storage node receives a log retrieval request from any compute node, it can determine the corresponding database log from its own stored database logs and return it to the compute node. For example, if a master node generates database logs corresponding to each data operation in a database transaction based on the WAL mechanism, subsequent execution of these operations not only by the master node but also by other slave nodes is necessary to maintain data consistency. After the master node uploads the generated database logs to the shared storage node, each slave node can send a log retrieval request to read the database log from the shared storage node. Once the database log is read, the slave node can perform the corresponding data operations based on it, thus maintaining data consistency with the master node.

[0032] Based on the above Figure 1 The system shown in this specification provides embodiments as follows: Figure 2 The log sharing method shown.

[0033] Figure 2 This is a schematic flowchart of the log sharing method provided in the embodiments of this specification, which specifically includes the following steps: S200: The compute node receives a data operation request.

[0034] S201: Determine each data operation in the database transaction corresponding to the data operation request.

[0035] S202: Generate the database log corresponding to each data operation.

[0036] S203: Store the database logs in the local storage space of the computing node.

[0037] In the embodiments described in this specification, the local storage space of the computing node can be implemented through a local cloud disk. A local cloud disk refers to a storage medium that is physically located on the computing node, but whose management is not handled by the computing node itself, but by other devices in the cloud.

[0038] Since the database logs temporarily stored in the local cloud disk will be uploaded to the shared storage node later, in the embodiments of this specification, the shared storage node can act as the cloud manager of the local cloud disk of the settlement node, that is, the shared storage node performs cloud management of the local cloud disk of the computing node.

[0039] Specifically, the shared storage node can pre-send cloud management rules to the settlement node for cloud management of the compute node's local cloud disk. These cloud management rules include at least the following upload conditions: The upload conditions may include: the time elapsed since the last time the compute node uploaded database logs to the shared storage node reaches a preset time length; and / or, the number of database logs currently stored in the compute node's local cloud disk reaches a preset number; and / or, the space occupied by the database logs currently stored in the compute node's local cloud disk reaches a preset space size. These upload conditions can be set as needed, and are not limited in the embodiments described in this specification.

[0040] S204: When the upload conditions are met, upload the database logs stored in the local storage space to the shared storage node.

[0041] The compute node can periodically or in real-time determine whether the database logs stored in the local cloud disk meet the upload conditions based on the cloud management rules for the local cloud disk pre-issued by the shared storage node. If they do, the compute node will upload the database logs stored in the local cloud disk to the shared storage node.

[0042] Correspondingly, after uploading the database logs stored in the local cloud disk to the shared storage node, the compute node can release the space occupied by the uploaded database logs in the local cloud disk, that is, clear the uploaded database logs stored in the local cloud disk. Then, when the compute node needs to execute corresponding database transactions based on the database logs, i.e., when replaying the database logs, it can first query the database logs to be replayed in its own local cloud disk. If the database log is found, it can be replayed directly. If it is not found, it can send a log retrieval request to the shared storage node to read the database log from the shared storage node and replay it.

[0043] S205: Shared storage nodes store the received database logs.

[0044] After receiving the database logs uploaded by each compute node, the shared storage node can store these database logs using a shared storage method.

[0045] The shared storage nodes described in the embodiments of this specification are not limited to one. These shared storage nodes can serve as backups for each other, each storing a copy of the logs composed of these database logs. These shared storage nodes can reach a consensus on the database logs received by any shared storage node based on a preset consensus protocol, and only store the database log after the consensus is passed. Specifically, the multiple computing nodes and multiple shared storage nodes can have a one-to-one correspondence, that is, a computing node only uploads database logs to its corresponding shared storage node. Alternatively, a computing node can have a correspondence with at least two of the multiple shared storage nodes, that is, a computing node uploads database logs to each of its corresponding shared storage nodes.

[0046] Regardless of the mapping between compute nodes and shared storage nodes, after receiving a database log uploaded by a compute node, a shared storage node can initiate consensus on that database log with other shared storage nodes. Once the consensus is reached with other shared storage nodes, all shared storage nodes store the database log. The consensus protocol mentioned above includes the Paxos protocol. The Paxos protocol allows multiple database logs stored and agreed upon by different shared storage nodes to be stored in different orders. In this case, the distributed database system combines the Paxos consensus protocol and the WAL mechanism, and can be called PALF (Paxos-backed Append-only LogFile System). If a strict consistency in the order of multiple database logs stored and agreed upon by all shared storage nodes is required, the consensus protocol mentioned above can also be the Raft protocol; this specification does not impose any restrictions on this.

[0047] The clearing of database logs differs between shared storage nodes and compute nodes. Compute nodes determine whether database logs have been uploaded from their local cloud disk to the shared storage node, while shared storage nodes determine this based on the length of time the database logs have been stored there. Specifically, shared storage nodes can clear database logs whose timestamps are longer than a preset minimum replay time, based on this minimum. For example, if the minimum replay time is set to one year, database logs older than one year will no longer be replayed and can be cleared. Therefore, shared storage nodes can clear database logs whose timestamps are longer than one year.

[0048] S206: Upon receiving a log retrieval request sent by the computing node, return the stored database log corresponding to the log retrieval request.

[0049] In addition, in the embodiments of this specification, besides storing the database logs generated by each of the aforementioned computing nodes, the shared storage node can also use shared storage to store the metadata corresponding to the data in different partitions of the database instance, as well as the database logs corresponding to this metadata. This is because in real-world application scenarios, in many cases, even if the data itself has not changed, its corresponding metadata will frequently change. For example, Zhang San's account balance data has not changed, but due to business needs, the business system will periodically send heartbeat messages for this data to the distributed database system to confirm that Zhang San's account balance data is still normally stored in the distributed database system. In addition, in a distributed database system using an LSM-Tree storage architecture, operations such as MemTable dumping, compression or format changes without data modification will cause a large number of metadata changes, but the data itself has not changed.

[0050] If the data itself and its corresponding metadata are stored in the compute node (or its corresponding storage node) using object storage, then the data itself and its corresponding metadata will be treated as an access object. This means that even if the data itself has not changed, as long as its corresponding metadata has changed, the compute node will need to read the entire access object into memory, modify it, and then write it back to the previously stored location. This will lead to a significant read / write amplification problem. Therefore, in this embodiment, the data and its corresponding metadata are no longer centrally stored on the compute nodes. Instead, the data and metadata are separated. The metadata and its corresponding database logs (which are also generated by the compute nodes when the metadata changes; the only difference between the database logs generated by changing data and those generated by changing metadata is that the latter are based on DML operations, while the former are based on DDL operations; however, once the database logs are generated, the subsequent processes are the same) are stored on shared storage nodes. Since the shared storage nodes use a shared storage method to store the metadata and its corresponding database logs, when the metadata changes, the shared storage nodes only need to read and modify the metadata, without needing to read other data. The processing of the database logs corresponding to the metadata is the same as that of the database logs corresponding to the data, and will not be repeated here. In other words, the shared storage nodes store the database logs corresponding to the metadata and also replay the database logs corresponding to the metadata to update the metadata, but the data copies of the data corresponding to these metadata are still stored on the compute nodes or the storage nodes corresponding to the compute nodes.

[0051] The above describes a log sharing system and log sharing method provided in the embodiments of this specification. Based on the same idea, this specification also provides corresponding devices, storage media, and electronic devices.

[0052] Figure 3 This is a schematic diagram of a first log sharing device provided in an embodiment of this specification. The device is applied to a computing node in a distributed database, and the device includes: The receiving module 300 is used to receive data operation requests; The determining module 301 is used to determine each data operation in the database transaction corresponding to the data operation request; The generation module 302 is used to generate database logs corresponding to each data operation. Storage module 303 is used to store the database logs in the local storage space of the computing node; The upload module 304 is used to upload the database logs stored in the local storage space to the shared storage node when the upload conditions are met. The shared storage node is used to store the database logs stored in the local storage space of each computing node and to return the database logs corresponding to the log acquisition request stored in the shared storage node when it receives a log acquisition request sent by any computing node.

[0053] Optionally, the device is applied to the master node; The master node stores data copies corresponding to data in different partitions of the database instance; The data stored on other computing nodes corresponds to the same data as the data stored on the master node.

[0054] Optionally, the storage module 303 is specifically used to store the database logs in the local cloud disk of the computing node, wherein the local cloud disk is managed by the shared storage node. The upload module 304 is specifically used to upload each database log stored in the local cloud disk to the shared storage node based on the cloud management rules of the shared storage node for the local cloud disk, when the database logs stored in the local cloud disk meet the upload conditions.

[0055] Optionally, the upload module 304 is further configured to, after uploading the database logs stored in the local storage space to the shared storage node, clear the uploaded database logs in the local storage space.

[0056] Optionally, the device uses object storage to store data copies; The shared storage node uses a shared storage method to store the logs of each database.

[0057] Figure 4 This is a schematic diagram of a second log sharing device provided in the embodiments of this specification. The device is applied to a shared storage node in a distributed database, and the device includes: The receiving module 400 is used to receive the database logs stored in the local storage space of the computing node when the upload conditions are met. The database logs are generated by the computing node after receiving the data operation request, based on the data operations in the database transaction corresponding to the data operation request. Storage module 401 is used to store the received database logs; The log sharing module 402 is used to return the stored database log corresponding to the log acquisition request when it receives a log acquisition request sent by the computing node.

[0058] Optionally, the database logs are stored in the local cloud disk of the computing node; The log sharing module 402 is further configured to send cloud management rules for cloud management of the local cloud disk of the computing node in advance, so that when the database logs stored in the local cloud disk of the computing node meet the upload conditions, the computing node uploads each database log stored in the local cloud disk.

[0059] Optionally, the storage module 401 is further configured to clear database logs whose generation timestamp to the current time is longer than the minimum log replay time, based on a preset minimum log replay time.

[0060] Optionally, the computing nodes use object storage to store data copies. One computing node stores data copies corresponding to different partitions in the database instance, and the data copies stored on different computing nodes correspond to the same data. The storage module 401 is specifically used to store the logs of each database using a shared storage method.

[0061] Optionally, the storage module 401 is specifically used to reach a consensus with other shared storage nodes on the received database logs according to a preset consensus protocol; and to store the received database logs after the consensus is reached.

[0062] Optionally, the storage module 401 is further configured to store the metadata corresponding to the data in the different partitions and the database logs corresponding to the metadata in a shared storage manner.

[0063] This specification also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can be used to perform the log sharing method provided above.

[0064] This specification also provides a computer program product comprising a computer program that, when executed by a processor, implements the above-described log sharing method.

[0065] based on Figure 2 The log sharing method shown in this specification also provides embodiments that provide... Figure 5 The diagram shows the structure of the electronic device. Figure 5 At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile storage, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile storage into memory and then runs it to implement the log sharing method described above.

[0066] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A log sharing system, the system comprising a shared storage node and several computing nodes; wherein: The computing node is used to receive data operation requests, determine each data operation in the database transaction corresponding to the data operation request, generate a database log corresponding to each data operation, store the database log in the local storage space of the computing node, and upload each database log stored in the local storage space to the shared storage node when the upload conditions are met. The shared storage node is used to store the database logs uploaded by each computing node from their respective local storage space, and when it receives a log retrieval request from any computing node, it returns the database logs stored in the shared storage node corresponding to the log retrieval request.

2. In the system as described in claim 1, a computing node stores data copies corresponding to data in different partitions of the database instance; The data replicas stored on different computing nodes correspond to the same data. The computing node includes a master node and several slave nodes; The master node is used to receive the data operation request and generate database logs, and upload the database logs to the shared storage node.

3. The system as described in claim 1, wherein the computing node is specifically used to store the database logs in the local cloud disk of the computing node; The shared storage node is specifically used for cloud management of the local cloud disk of the computing node, so that when the database logs stored in the local cloud disk of the computing node meet the upload conditions, the database logs stored in the local cloud disk are uploaded to the shared storage node.

4. The system as described in claim 1, wherein the computing node is further configured to, after uploading the database logs stored in the local storage space to the shared storage node, clear the database logs that have been uploaded to the local storage space; The shared storage node is also used to clear database logs whose generation timestamp to the current time is longer than the minimum log replay time, based on a preset minimum log replay time.

5. The system as described in claim 2, wherein the computing node stores data copies using an object storage method; The shared storage node uses a shared storage method to store the logs of each database.

6. The system as described in claim 2, wherein there are multiple shared storage nodes; Specifically, any shared storage node is used to receive database logs uploaded by any computing node, and according to a preset consensus protocol, to reach a consensus with other shared storage nodes on the received database logs. After the consensus is passed, the received database logs are stored.

7. The system as described in claim 5, wherein the shared storage node is further configured to store metadata corresponding to the data in the different partitions and database logs corresponding to the metadata using a shared storage method.

8. A log sharing method, the method comprising: The compute node receives data operation requests; Determine the data operations within the database transaction corresponding to the data operation request; Generate database logs for each data operation; The database logs are stored in the local storage space of the computing node; When the upload conditions are met, the database logs stored in the local storage space are uploaded to the shared storage node. The shared storage node is used to store the database logs stored in the local storage space of each computing node and, upon receiving a log retrieval request from any computing node, returns the database logs stored in the shared storage node corresponding to the log retrieval request.

9. The method as described in claim 8, wherein the computing node is the master node; The master node stores data copies corresponding to data in different partitions of the database instance; The data stored on other computing nodes corresponds to the same data as the data stored on the master node.

10. The method of claim 8, wherein storing the database log in the local storage space of the computing node specifically includes: The database logs are stored in the local cloud disk of the computing node, and the local cloud disk is managed by the shared storage node. When the upload conditions are met, the database logs stored in the local storage space are uploaded to the shared storage node, specifically including: Based on the cloud management rules of the shared storage node for the local cloud disk, when the database logs stored in the local cloud disk meet the upload conditions, each database log stored in the local cloud disk is uploaded to the shared storage node.

11. The method of claim 8, further comprising: After uploading the database logs stored in the local storage space to the shared storage node, clear all the uploaded database logs from the local storage space.

12. The method of claim 9, wherein the computing node stores data copies using an object storage method; The shared storage node uses a shared storage method to store the logs of each database.

13. A method for sharing a log, the method comprising: The shared storage node receives database logs stored in the local storage space of the computing node when the upload conditions are met. The database logs are generated by the computing node after receiving a data operation request, based on the data operations in the database transaction corresponding to the data operation request. Store the received logs from each database; Upon receiving a log retrieval request from the computing node, the system returns the stored database log corresponding to the log retrieval request.

14. The method of claim 13, wherein the database log is stored in the local cloud disk of the computing node; The method further includes: Cloud management rules for cloud management of the local cloud disk of the computing node are sent in advance, so that when the database logs stored in the local cloud disk of the computing node meet the upload conditions, the computing node uploads the database logs stored in the local cloud disk.

15. The method of claim 13, further comprising: The shared storage node clears database logs whose timestamps from generation to the current moment are longer than the preset minimum log replay time, based on a preset minimum log replay time.

16. The method as described in claim 13, wherein the computing node stores data copies using an object storage method, one computing node stores data copies corresponding to data in different partitions of the database instance, and the data copies stored by different computing nodes correspond to the same data; The shared storage node uses a shared storage method to store the logs of each database.

17. The method of claim 13, wherein storing the received database logs specifically includes: According to the preset consensus protocol, it reaches a consensus with other shared storage nodes on the received database logs; After consensus is reached, the received database logs are stored.

18. The method of claim 16, further comprising: The shared storage node uses a shared storage method to store the metadata corresponding to the data in different partitions and the database logs corresponding to the metadata.

19. A log sharing device, the device comprising: The receiving module is used to receive data operation requests; The determination module is used to determine each data operation in the database transaction corresponding to the data operation request; The generation module is used to generate database logs for each data operation. A storage module is used to store the database logs in the local storage space of the computing node; The upload module is used to upload the database logs stored in the local storage space to the shared storage node when the upload conditions are met. The shared storage node is used to store the database logs uploaded by each computing node from their respective local storage spaces, and when it receives a log retrieval request from any computing node, it returns the database logs stored in the shared storage node corresponding to the log retrieval request.

20. A log sharing device, the device comprising: The receiving module is used to receive the database logs stored in the local storage space of the computing node when the upload conditions are met. The database logs are generated by the computing node after receiving the data operation request, based on the data operations in the database transaction corresponding to the data operation request. The storage module is used to store the received database logs; The log sharing module is used to return the stored database log corresponding to the log retrieval request when it receives a log retrieval request sent by the computing node.

21. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any one of claims 8-18.

22. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any one of claims 8-18.

23. A computer program product comprising a computer program that, when executed by a processor, implements the method described in any one of claims 8-18.

Images