A distributed system processing method based on optimized Raft algorithm

Abstract

The present invention discloses a distributed system processing method and program product based on an optimized Raft algorithm, including: establishing a unique transaction ID for a user's operation transaction, the transaction ID being used as an identifier for subsequent operations; when executing a transaction, the corresponding API function is called together with the transaction ID; an independent cache area is allocated to the corresponding transaction according to the transaction ID to store all operations during the transaction; after the transaction is completed, the data in the cache area is persisted to the database at one time, and the logs related to the transaction are cleaned up. By establishing a unique transaction ID for the user's operation transaction as an identifier for subsequent operations, the atomicity and consistency of operations within the transaction are guaranteed; all operations are completed in the cache area, frequent hard disk operations are avoided, and the processing speed is greatly improved; by persisting the data in the cache to the database at one time, the overhead of persisting one by one is reduced, the logs related to the transaction are cleaned up in time, memory waste is prevented, and the efficient operation of the system is maintained.

Description

Technical Field

[0001] The present invention belongs to the technical field of distributed system transaction processing methods, and in particular relates to a distributed system processing method based on an optimized Raft algorithm. Background Art

[0002] With the rapid development of the digital economy, the widespread penetration of cloud computing, big data and AI technologies has greatly accelerated the pace of digital transformation of enterprises. Faced with the growing traffic peaks, complex task processing needs, and strict requirements for high availability, excellent scalability and high-performance data processing capabilities, distributed systems have become an indispensable core technology to support modern enterprise architecture. In this technical field, there is currently a consensus algorithm, the Raft algorithm, which aims to replace the Paxos algorithm. It is easier to understand than Paxos through logical separation, but it has also been formally proven to be safe and provides some additional features. The Raft algorithm provides a general method for distributing state machines in a computing system cluster, ensuring that each node in the cluster agrees on a series of identical state transitions. It has many open source reference implementations with complete specification implementations in Go, C++, Java and Scala. The Raft algorithm is widely used in distributed databases, file systems and coordination services because of its simplicity, efficiency, ease of understanding and implementation, and excellent performance, ensuring data consistency and high availability of the system. Compared with the Paxos algorithm, Raft shows stronger operability and lower complexity, so it has become the preferred solution in engineering practice.

[0003] In the vast field of distributed systems, the focus of technical solutions is mainly on two core areas: distributed coordination services (DCS) and distributed consensus protocols (DCP). These technologies are designed to solve the most difficult challenges in distributed environments, including but not limited to key issues such as data consistency maintenance, service coordination, naming service management, configuration synchronization and optimization. The following is a detailed analysis of the current mainstream distributed systems and the key technologies they use:

[0004] (1) ZooKeeper: As a pioneer in distributed coordination services carefully developed by Yahoo Research, ZooKeeper relies on its unique ZAB (ZooKeeper Atomic Broadcast) protocol, which is actually an efficient variant of the Paxos algorithm. It effectively ensures the high consistency of cluster status and high availability of the system;

[0005] (2) Chubby: A distributed lock service developed by Google. Its core lies in the deep integration of the Paxos algorithm, which enables efficient management of distributed locks and reliable coordination of services.

[0006] (3) Consul: An open source masterpiece launched by HashiCorp. Consul's core advantage lies in its distributed consistency guarantee based on the Raft algorithm. This algorithm, with its easy-to-understand and easy-to-implement features, has laid a solid foundation for the widespread application of Consul in distributed systems.

[0007] (4) etcd: A key-value storage system carefully built by the CoreOS team. etcd also uses the Raft algorithm as its distributed consistency engine, relying on its high performance, high availability and strong consistency.

[0008] (5) Doozer: As another distributed coordination service, Doozer stands out with its unique distributed consensus algorithm;

[0009] These systems have their own advantages and disadvantages, and together they have promoted the continuous development of distributed system technology. However, the above existing technologies lack global transaction support.

[0010] Current mainstream distributed systems, whether based on Paxos or Raft algorithms, generally have the problem of insufficient support for global transactions:

[0011] (1) ZooKeeper: Although its multi operation provides certain transaction capabilities, it is limited to a single client session and cannot span multiple clients or services, limiting its applicability in complex distributed applications.

[0012] (2) Chubby: The design focuses on distributed locks, but has limited support for global transaction processing and is difficult to meet the needs of complex scenarios.

[0013] (3) Consul: Although it is based on Raft to ensure data consistency, its transaction processing capabilities are weak and it mainly focuses on service discovery and health checks.

[0014] (4) etcd: High-performance key-value storage, but transaction processing focuses on simple KV operations and lacks support for complex transactions.

[0015] (5) Doozer: The transaction management mechanism is unclear, which poses risks when dealing with high transaction requirements. The lack of global transactions is particularly evident in high-concurrency scenarios, reducing command execution efficiency and system performance.

[0016] Although distributed systems such as Zookeeper have demonstrated excellent performance in their respective fields, they still face a series of secondary challenges and shortcomings, which are particularly prominent in complex and large-scale distributed environments;

[0017] (1) Inefficiency of log management: Challenge: As the system runs longer, the number of log files increases dramatically, which puts tremendous pressure on storage and management. The disadvantage is that the existing log cleaning strategies often lack intelligence and accuracy, and may mistakenly delete key logs or retain too many useless logs, thus affecting system performance and resource utilization.

[0018] (2) Leader election bottleneck and scalability limitations: In a cluster with a large number of nodes, the leader election process may be prolonged due to factors such as network delays and node failures, reducing the system response speed. The disadvantage is that as the cluster scale increases, the scalability of the system is challenged, and the maintenance difficulty increases, affecting the overall availability.

[0019] (3) Data storage limitations and lack of flexibility: When faced with the need to process large amounts of data or complex data structures, existing systems may show performance bottlenecks or lack of flexibility. The disadvantage is that this limits the applicability and scalability of the system in specific application scenarios and cannot meet the growing business needs.

[0020] (4) Complexity of deployment and maintenance: The deployment, configuration and subsequent maintenance of the cluster are cumbersome and complex, and require high technical skills from the operation and maintenance personnel. The disadvantage is that this not only increases the operation and maintenance costs, but may also cause delays or errors in a rapidly changing or expanding distributed environment, affecting business continuity.

[0021] However, with the continuous upgrading of business needs, the Raft algorithm has encountered new challenges in large-scale, high-concurrency scenarios, and it urgently needs further optimization and improvement in handling large-scale concurrency and pursuing higher performance. Summary of the invention

[0022] The purpose of the present invention is to provide a distributed system processing method based on an optimized Raft algorithm, the core innovation of which is to comprehensively introduce and deepen the transaction mechanism, thereby significantly improving the performance, data consistency and operational reliability of the system. Through the design and implementation of the transaction mechanism, the present invention can better adapt to complex distributed environments and meet the requirements of efficient, stable and flexible systems.

[0023] In order to achieve the above object, the present invention adopts the following technical solutions:

[0024] The present invention provides a distributed system processing method based on an optimized Raft algorithm, a distributed system processing method based on an optimized Raft algorithm, characterized in that it includes:

[0025] Based on the transaction operation response of the user when executing the system API function, a unique transaction ID is established for the user's operation transaction, and the transaction ID is used as an identifier for subsequent operations; when executing the transaction, the corresponding API function is called together with the transaction ID;

[0026] Allocate an independent buffer area for the corresponding transaction according to the transaction ID, and the buffer area is used to store all operations during the transaction;

[0027] After the transaction is completed, the data in the cache area is persisted to the database at one time, and the logs related to the transaction are cleared.

[0028] According to the above technology, a unique transaction ID is established for the user's operation transaction. The transaction ID is used as an identifier for subsequent operations to ensure the atomicity and consistency of operations within the transaction. The system allocates an independent cache area for each transaction ID, which is used to store all operations during the transaction. All operations, including node creation, deletion, and modification, are completed in the cache area to avoid frequent hard disk operations and greatly improve the processing speed. After the transaction is completed, the data in the cache is persisted to the database at one time to reduce the overhead of persisting one by one. The system cleans up the logs related to the transaction in a timely manner to prevent memory waste and maintain efficient operation of the system.

[0029] Furthermore, after the transaction is completed, the method of persisting the data in the cache to the database at one time includes:

[0030] The transaction ID is shared with the application layer to establish linkage with the application layer. When all operations in the transaction are completed, the application layer data and system data are persisted using a common persistence handle. Efficient persistence operations are achieved through a unified transaction handle.

[0031] Furthermore, the method of using a common persistence handle with the application layer to perform persistence processing on application layer data and system data at the same time includes:

[0032] In the initial stage, the application layer obtains a persistent transaction handle, which will be used as the identifier of the shared transaction; the application layer first synchronizes the cached data to the database, and then seamlessly calls the system's synchronization function to synchronize the system data to the same database; after the data is successfully synchronized, the persistence operations at the application layer and the system level are completed at the same time by submitting the shared persistent transaction handle.

[0033] Furthermore, the cache area is an independent in-memory database corresponding to the persistent database structure.

[0034] Furthermore, the transaction management logic of the transaction is encapsulated in the API function so that all database operations are executed under transaction control. The transaction management logic includes transaction opening, committing, rolling back and concurrency control; when multiple transactions operate on the same node, concurrency conflicts between transactions are managed and processed.

[0035] Furthermore, exception handling logic is added during the transaction execution process, so that any exception occurring at any stage during the transaction execution process will immediately trigger the system's exception handling logic; when performing the exception handling logic, the system will roll back the state and release resources, while recording log error data to track problems and / or restore the system.

[0036] Furthermore, in the system's Raft consensus mechanism, the previous leader is retained, and batch synchronization is used to quickly synchronize the previous leader's log to the system's follower nodes; at the end of the transaction, the cached data and the previous leader's log are persisted at once. In the leader election, the system is set to tend to retain the previous leader to reduce performance loss and work interruption, and batch synchronization technology is used to quickly synchronize the leader's log to the follower node to ensure data consistency.

[0037] Beneficial effects:

[0038] 1. The present invention provides a distributed system processing method based on an optimized Raft algorithm. By establishing a unique transaction ID for the user's operation transaction, the transaction ID is used as an identifier for subsequent operations to ensure the atomicity and consistency of operations within the transaction. The system allocates an independent cache area for each transaction ID, and the cache area is used to store all operations during the transaction. All operations including node creation, deletion, and modification are completed in the cache area to avoid frequent hard disk operations and greatly improve the processing speed. After the transaction is completed, the data in the cache is persisted to the database at one time to reduce the overhead of persisting one by one. The system cleans up the logs related to the transaction in a timely manner to prevent memory waste and maintain efficient operation of the system.

[0039] 2. The present invention provides a distributed system processing method based on the optimized Raft algorithm. It has carried out in-depth optimization and functional expansion on the basis of the existing Raft algorithm, innovatively integrated the transaction mechanism into the algorithm, and significantly improved the performance and stability of the system. At the same time, through the refined improvement of the leadership election mechanism, the in-depth optimization of the log replication process, and the comprehensive improvement of the log cleaning mechanism, the present invention further strengthens the data consistency and high availability of the system in high-concurrency scenarios, provides enterprises with efficient, stable, and scalable computing support, and strongly promotes the healthy development of the digital economy. DETAILED DESCRIPTION

[0040] The present invention is further described below in conjunction with specific embodiments. It should be noted that although the description of these embodiments is used to help understand the present invention, it does not constitute a limitation of the present invention.

[0041] The core innovation of the distributed system processing method based on the optimized Raft algorithm provided in the first aspect of this embodiment is the comprehensive introduction and deepening of the transaction mechanism, thereby significantly improving the performance, data consistency and operational reliability of the system. Through the careful design and implementation of the transaction mechanism, the present invention can better adapt to complex distributed environments and meet the requirements of efficient, stable and flexible systems.

[0042] This embodiment achieves comprehensive support and optimization of transactions through the following designs:

[0043] Application and management of transaction ID:

[0044] Before executing a system API operation, the user applies for a unique transaction ID. The transaction ID is used as an identifier for subsequent operations to ensure atomicity and consistency of operations within the transaction. When executing a transaction, the corresponding API function is called together with the transaction ID.

[0045] Transaction Cache Processing:

[0046] The system allocates an independent cache area for each transaction ID. All operations, such as node creation, deletion, and modification, are completed in the cache area, avoiding frequent hard disk operations and greatly improving processing speed.

[0047] Transaction persistence and log cleaning:

[0048] After the transaction is completed, the data in the cache is persisted to the database at one time, reducing the overhead of persisting each data one by one. The system cleans up the logs related to the transaction in a timely manner to prevent memory waste and maintain efficient operation of the system.

[0049] Furthermore, based on the basic transaction support, this embodiment further explores the in-depth application and integration strategy of the transaction mechanism: using the innovation of persistent transaction handles:

[0050] A close linkage mechanism is established between the application system and this system through a shared transaction ID, and a shared persistent transaction handle is used to ensure that the two work together within the same transaction framework. This design simplifies the operation process and improves the coherence and consistency of data processing.

[0051] The persistence process without sharing the persistent transaction handle: First, the application layer needs to independently obtain its own persistent transaction handle; then, synchronize the data cached by the application layer to the database; after completing the data synchronization, the application layer needs to commit the transaction handle to complete the persistence operation at its level; then, the application layer needs to obtain another independent persistent transaction handle for this system again; call the synchronization function of this system to synchronize the data inside this system to the database; finally, commit the transaction handle of this system to complete the persistence operation at this system level

[0052] Furthermore, the persistence process of the shared persistent transaction handle: in the initial stage, the application layer only needs to obtain the persistent transaction handle once, which will be used as the identifier of the shared transaction; the application layer first synchronizes its cached data to the database; then, it seamlessly calls the synchronization function of this system to synchronize the data of this system to the same database. Seamless call means that when operating between different systems or components, data transfer and function execution can be achieved smoothly, continuously and without human intervention.

[0053] The specific method of seamlessly calling the synchronization function of this system is: when the application layer needs to synchronize data to this system and then update it to the same database, a seamless call mechanism is adopted. The core of this mechanism is that the application layer passes the persistent transaction handle it has applied for to this system. In this way, the application layer and this system can work together under the framework of the same persistent transaction and operate the database together. Since both parties share the same transaction, when the transaction is committed, the data changes of the application layer and the system layer will be completed synchronously, ensuring data consistency and seamless synchronization. However, compared with the ordinary call of the synchronization function of this system, the ordinary call method does not involve the sharing of the persistent transaction handle. In this mode, when the application layer calls the synchronization function of this system, it will not pass the persistent transaction handle it holds. Therefore, this system needs to independently apply for a persistent transaction handle to perform database operations. Since the application layer and the system layer use different transaction handles, their database operations will be performed in their respective transactions. This means that when the transaction is committed, the data changes of the application layer and the system layer will be performed independently. Although this increases the flexibility of the system, it may also affect the overall performance in some cases, especially in scenarios where highly coordinated data changes are required.

[0054] After the data is successfully synchronized, the persistence operations at both the application layer and the system level can be completed simultaneously by submitting this shared persistence transaction handle.

[0055] By comparison, it is not difficult to find that the shared persistent transaction handle significantly reduces the number of times the transaction handle is opened and closed, thereby effectively improving operational efficiency and ensuring high consistency of data during cross-system synchronization. This optimization not only simplifies the operational process, but also brings users a more convenient and efficient experience.

[0056] Implementation of unified persistence strategy: When all operations in a transaction are completed, the application system and the system will use a shared persistence handle to persist both the application layer data and the system layer data. This strategy effectively avoids the inefficiency and complexity of separate persistence, ensuring data consistency and optimizing overall system performance.

[0057] Furthermore, in order to realize comprehensive transaction functions, the API function design of the present invention focuses on solving the following core problems and optimizes the implementation strategy to ensure efficient and reliable transaction processing: the cache area is an independent memory database corresponding to the persistent database structure; in specific implementation, an efficient memory database is constructed to support transaction caching. In specific implementation, an independent and efficient memory database that closely corresponds to the persistent database structure is constructed to cache data changes during the transaction process; lightweight, high-performance memory database technology is used to ensure seamless integration with the main system and high controllability; in terms of data consistency, the data structure in the memory database is ensured to be completely consistent with the persistent database to facilitate fast synchronization; in terms of cache strategy, intelligent cache management is implemented to dynamically adjust the cache strategy according to the activity of the transaction and the importance of the data.

[0058] During the specific implementation, mature database systems are integrated to achieve transaction persistence, and mature database systems such as MySQL are seamlessly integrated into this system to take advantage of their stability and transaction processing capabilities; MySQL is directly integrated into the application through embedded database technology to reduce network latency and data transmission overhead; in terms of configuration optimization, MySQL configuration parameters such as transaction logs and lock strategies are optimized to improve performance based on transaction processing needs.

[0059] When implementing the synchronization mechanism between the in-memory database and the persistent database, it ensures that when a transaction is committed, the data in the in-memory database can be accurately synchronized to the persistent database.

[0060] Furthermore, in terms of comprehensive transaction management functions, this embodiment implements complete transaction management logic in the API function, including transaction opening, committing, rolling back, and concurrency control; encapsulates the transaction management logic in the API function to ensure that all database operations are executed under transaction control; and effectively manages concurrency conflicts between transactions when multiple transactions operate on the same node;

[0061] Furthermore, in terms of exception handling: exception handling logic is added during transaction execution, so that exceptions occurring at any stage during transaction execution will immediately trigger the system's exception handling logic; when performing exception handling logic, the system will roll back the state and release resources, while recording log error data to track problems and / or restore the system.

[0062] In the specific implementation, the operation of transaction cache, all operations on nodes are performed in the transaction cache. In the specific application, at the application system layer, users can use the API functions provided by this system to perform transaction operations such as adding, modifying and deleting nodes. When executing these API functions, they are closely integrated with the transaction mechanism to ensure the atomicity and consistency of the operation. Since the system uses an in-memory database to implement the cache, transaction management is completed by directly operating the in-memory database. This method not only makes the management of transaction cache more efficient and convenient, but also benefits from the fast query and management capabilities of the in-memory database. In addition, the record format of the in-memory database and the persistent database is completely consistent. When synchronizing data from memory to persistent storage, there is no need to convert the format, which further improves the efficiency of data synchronization; this design provides higher performance and efficiency compared to the traditional simple cache mechanism. By processing the transaction cache completely in memory, the system can achieve extremely fast response speed when executing commands, greatly optimizing the overall execution efficiency of the program.

[0063] Timely and accurate deletion of transaction logs:

[0064] At the end of the transaction, the application layer calls the preset log deletion instruction to clear the logs related to the transaction. In contrast, existing distributed systems usually require manual or scheduled log deletion. This precise and timely log cleaning method can not only effectively save a lot of memory and hard disk space, but also significantly improve the operating efficiency of the system. When the transaction ends, the application layer calls the preset persistence API function to persist the node data and the required logs at one time. This design avoids the persistence of a single function separately, ensures data consistency, and greatly improves persistence efficiency.

[0065] Application system data is persisted together with this system data:

[0066] When calling the persistence function, the persistence transaction handle is passed into the system through the persistence API function, so that the data of the two systems can be persisted simultaneously under the same persistence handle. This strategy not only improves the persistence efficiency, but also ensures the data consistency between the application system and the system. Through the introduction of the full transaction mechanism, the present invention separates the operation command from the persistence operation, realizes efficient transaction processing and data consistency, and greatly improves the overall performance of the system.

[0067] Furthermore, in the system's Raft consensus mechanism, the previous leader is retained, and batch synchronization is used to quickly synchronize the previous leader log to the system follower node; at the end of the transaction, the data in the cache and the previous leader log are persisted at one time.

[0068] In specific implementation, this embodiment provides the following auxiliary functions and optimizations:

[0069] Batch operation support: The system supports batch processing of common commands, which improves the efficiency of command execution.

[0070] Local caching of node data: By caching node data locally, the need for remote access is reduced, thereby speeding up queries.

[0071] Accurate log cleaning: Logs are automatically cleared when transactions are closed, which is more efficient than traditional manual or scheduled cleanup methods.

[0072] Leader priority election: In leader election, the system tends to retain the previous leader to reduce performance loss and work interruption.

[0073] Batch log synchronization: When the logs of the Leader and Follower are inconsistent, the system can synchronize all the difference logs at one time, improving efficiency.

[0074] Improved user experience: The system supports unlimited node data size and multiple data formats (such as numbers, strings, JSON, etc.), which enhances the system's flexibility and user experience.

[0075] In specific implementation, cluster server deployment methods include:

[0076] (1) Single-device deployment: Suitable for low-concurrency scenarios, effectively reducing deployment costs.

[0077] (2) Dual-station deployment: Applicable to medium-concurrency scenarios and further optimizes cost-effectiveness.

[0078] (3) Multiple cluster deployment: meet high concurrency and high performance requirements, and support cluster deployment of multiple servers

[0079] Flexible deployment methods:

[0080] (1) Independent cluster deployment: The system can be deployed as an independent cluster, and users can use the system independently to improve independence and flexibility.

[0081] (2) Co-deployment with application clusters: In high-concurrency scenarios, the system can be deployed in the same cluster as the application cluster to achieve deep integration and performance optimization.

[0082] (3) Deployed in the same process as the application: In low-concurrency scenarios, the system can be deployed in the same process as the application, reducing remote call overhead and improving command execution efficiency.

[0083] Simplified installation and maintenance:

[0084] (1) Simplified installation process: The installation process of this system is very simple and does not require complex configuration. It is suitable for beginners and organizations that lack a dedicated operation and maintenance team, greatly reducing the threshold for use.

[0085] (2) Low maintenance cost: The system requires almost no maintenance and the maintenance cost is extremely low, making it an efficient and economical solution.

[0086] As an implementation mode, the system architecture of the present invention includes:

[0087] The transaction management module includes a transaction ID generation and management unit, which is used to assign a unique transaction ID before each transaction starts. The ID is used to identify and manage all operations of the transaction.

[0088] The transaction cache unit is used to correspond to an independent cache area for each transaction ID, which is used to store all operations during the transaction, avoiding frequent hard disk access and improving performance;

[0089] The transaction log management unit is used to manage transaction-related log records and perform intelligent cleanup after the transaction is completed to ensure efficient operation of the system.

[0090] Raft consensus module, including the leader election unit: through the optimized leader election mechanism, the system gives priority to retaining the previous leader, reducing the performance overhead and system interruption time caused by the election;

[0091] The log replication unit uses batch synchronization technology to quickly synchronize the Leader's log to the Follower node to ensure data consistency.

[0092] The data persistence unit, at the end of a transaction, uses a unified persistence strategy to write the data and logs in the cache to the hard disk at one time to ensure the atomicity and consistency of the data.

[0093] API layer: API units with full transaction support: including node creation, deletion, modification and other operations. These APIs all require the transaction ID to ensure the transactionality and consistency of the operation. Persistence operation API unit: provides operations such as persisting node data, deleting logs and persisting them. Through a unified transaction handle, efficient persistence operations are achieved.

[0094] As well as the operation and maintenance and real-time monitoring modules, they are responsible for the operation and real-time monitoring of the entire system.

[0095] Application scenarios of this embodiment:

[0096] In high-concurrency transaction systems: Applicable to scenarios such as finance and e-commerce, it provides efficient transaction management and data consistency. The optimized leader election and log management improve the stability and response speed of the system under peak load.

[0097] In distributed database management: Through the optimized Raft algorithm and transaction mechanism, data consistency among multiple nodes is ensured, which is particularly suitable for complex queries and data processing, improving system performance and high availability.

[0098] In large-scale distributed file system systems: provide efficient log management and persistence mechanisms to ensure the security and consistency of file operations, and flexibly deploy and optimize resource utilization.

[0099] In cloud computing and microservice architecture: support efficient distributed coordination services, ensure service reliability and stability in high-concurrency scenarios, and provide solid technical support for enterprise-level applications.

[0100] IoT Data Management: Ensure data consistency among distributed IoT nodes through efficient transaction processing and data synchronization, and improve large-scale device management and data processing performance.

[0101] Specific examples:

[0102] (1) System deployment and configuration:

[0103] In order to cope with the growing concurrent requests and transaction processing needs, a large e-commerce platform decided to introduce the optimized Raft algorithm distributed system in this embodiment. According to its business scale and needs, it chose to adopt a multi-cluster deployment mode to ensure high performance and high availability.

[0104] Standardization of transaction processing procedures:

[0105] After the system is deployed, the application layer first initiates a transaction start request through the API, and the system assigns a unique transaction ID. This ID will be used to associate all subsequent related operations to ensure that the atomicity and consistency of the transaction are maintained.

[0106] (3) Node operation and transaction cache mechanism:

[0107] When the application layer needs to perform a specific operation (such as creating, modifying, or deleting a node), the corresponding API will be called with the transaction ID. The system creates an independent cache area specifically for the transaction based on the transaction ID. All node operations are performed in this cache, effectively avoiding frequent access to the hard disk and significantly improving operational efficiency.

[0108] (4) Persistence and log management:

[0109] After the transaction is completed, the system will provide a one-click persistence interface to the application layer. When the application layer confirms that all operations within the transaction have been completed, it requests data persistence through this interface. The system then writes all cached data and logs to the database at once, and cleans up the used logs, which not only ensures data consistency but also optimizes system resource usage.

[0110] (5) End of transaction and resource cleanup:

[0111] After data persistence is completed, the application layer notifies the system of the end of the transaction through the API. At this point, the system completes the final resource cleanup, including completely deleting the cache and logs related to the transaction to ensure that the system remains in an efficient operating state.

[0112] (7) Exception handling and recovery mechanism:

[0113] During the transaction processing, any exception that occurs at any stage will immediately trigger the system's exception handling mechanism. The system will attempt to roll back the state and release related resources, while recording the details of the error through logs to support problem tracking and system recovery.

[0114] (8) System monitoring and optimization:

[0115] In addition, the system also includes a comprehensive monitoring module that monitors the system status and performance indicators in real time. With this monitoring data, system administrators can make timely optimization adjustments to ensure that the system always runs in the best state.

[0116] The embodiments described above are merely illustrative. If a unit is described as a separate component, it may or may not be physically separated; if a component is displayed as a unit, it may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment. Those of ordinary skill in the art can understand and implement it without creative work.

[0117] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A distributed system processing method based on optimized Raft algorithm, characterized in that: include: Based on the transaction operation response of the user when executing the system API function, a unique transaction ID is established for the user's operation transaction, and the transaction ID is used as an identifier for subsequent operations; When executing a transaction, the corresponding API function is called together with the transaction ID; Allocate an independent buffer area for the corresponding transaction according to the transaction ID, and the buffer area is used to store all operations during the transaction; After the transaction is completed, the data in the cache area is persisted to the database at one time, and the logs related to the transaction are cleared; After the transaction is completed, the methods for persisting the data in the cache to the database at one time include: The transaction ID is shared with the application layer to establish linkage with the application layer. When all operations in the transaction are completed, a common persistence handle is used with the application layer to perform persistence processing on the application layer data and the system data at the same time. The method of using a common persistence handle with the application layer to persist both application layer data and system data includes: In the initial stage, the application layer obtains a persistent transaction handle, which will be used as the identifier of the shared transaction. The application layer first synchronizes the cached data to the database, and then seamlessly calls the system synchronization function to synchronize the system data to the same database. After the data is successfully synchronized, the shared persistent transaction handle is submitted to complete the persistence operations at both the application layer and the system level. The transaction management logic of the transaction is encapsulated in the API function so that all database operations are executed under transaction control. The transaction management logic includes transaction opening, committing, rolling back, and concurrency control. When multiple transactions operate on the same node, concurrency conflicts between transactions are managed and handled.

2. The distributed system processing method based on the optimized Raft algorithm according to claim 1 is characterized in that: The cache area is an independent memory database corresponding to the persistent database structure.

3. The distributed system processing method based on optimized Raft algorithm according to claim 1 is characterized in that: Exception handling logic is added during the transaction execution process so that exceptions occurring at any stage during the transaction execution process will immediately trigger the system's exception handling logic; when performing the exception handling logic, the system will roll back the state and release resources, while recording log error data to track problems and / or restore the system.

4. The distributed system processing method based on the optimized Raft algorithm according to claim 1 is characterized in that: In the system's Raft consensus mechanism, the previous leader is retained, and batch synchronization is used to quickly synchronize the previous leader log to the system's follower node; at the end of the transaction, the data in the cache and the previous leader log are persisted at one time.