A request processing method and electronic device for a distributed system
By constructing request events based on sequence numbers and persisting them as key-value pairs in a distributed system, the problem of lack of operation tracing and fault diagnosis is solved, and efficient fault handling and state recovery are achieved.
Patent Information
- Application Number
- CN202511367662.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-24
- Publication Date
- 2026-01-30
- Estimated Expiration
- 2045-09-24
AI Technical Summary
Distributed systems lack the ability to trace the source of operations, making it difficult to diagnose faults and track responsibility. Furthermore, when a node crashes or the network partitions, the state of the operations being processed is completely lost, resulting in low recovery efficiency and high resource consumption.
The request event is constructed based on the sequence number corresponding to the request event, and the event is persisted by constructing key-value pairs and written directly to the database to record the request event and processing status, thereby realizing fault diagnosis and responsibility tracking.
It improves the system's operational traceability capabilities, reduces system performance loss, avoids loss of operational status, and improves the efficiency of fault handling.
Smart Images
Figure CN120872640B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of system control technology, and in particular to a request processing method and electronic device for a distributed system. Background Technology
[0002] In a distributed system architecture, the control plane is responsible for receiving, scheduling, and managing external requests to ensure stable system operation. The control plane typically involves the collaborative work of multiple components, such as a scheduler, an API (Application Programming Interface) gateway, and a state manager. However, current distributed systems often only record the final state of resources, losing the original context of operation instructions. This results in a lack of operation tracing capabilities, making fault diagnosis and accountability difficult. Furthermore, control nodes rely on volatile memory to maintain operation processing states; in the event of a node crash or network partition, the state of ongoing operations is completely lost, and the system can only recover through a full state comparison, which is inefficient and resource-intensive. Summary of the Invention
[0003] This application provides a request processing method and electronic device for a distributed system. The method constructs a request event based on the sequence number corresponding to the request event, then constructs key-value pairs for event persistence and records the request event and its corresponding processing status, which are directly written to the database. This helps to diagnose and track faults related to request events and improves the efficiency of fault handling.
[0004] This application provides a request processing method for a distributed system, applied to the master node in a control plane process cluster of a distributed system, including:
[0005] The system retrieves the request data packet sent by the external requesting process based on the remote procedure call framework, and constructs the corresponding request event based on the request data packet and the preset sequence number. The request data packet includes the request type and request content, and the preset sequence number corresponds one-to-one with the request event.
[0006] A first key-value pair is generated based on the request event and the corresponding preset sequence number, and a first status marker is generated for each request event. A second key-value pair is generated based on the first status marker and the corresponding preset sequence number. The first status marker is a marker indicating that the request event is an event to be processed.
[0007] The first key-value pair and the second key-value pair are stored in a preset database, and the request event in the first key-value pair is read from the preset database, and the corresponding request event is processed according to the first status flag in the second key-value pair.
[0008] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the request processing method of any of the above-described distributed systems.
[0009] This application is applied to the master node in the control plane process cluster of a distributed system. First, it can obtain request data packets, including request type and request content, sent by external requesting processes based on the remote procedure call framework. Then, it constructs corresponding request events based on the request data packets and preset sequence numbers that correspond one-to-one with the request events. Next, it generates a first key-value pair based on the request events and preset sequence numbers, and generates a first status flag representing each request event as an event to be processed. It generates a second key-value pair based on the first status flag and the corresponding preset sequence number. Then, it stores the first key-value pair and the second key-value pair in a preset database, and reads the request events in the first key-value pair from the preset database to process the corresponding request events according to the first status flag in the second key-value pair.
[0010] Through the above technical solution, after obtaining the request data packet sent by the external requesting process based on the remote procedure call framework, this application constructs a request event based on a preset sequence number that corresponds one-to-one with the request event. Then, it persists the request event by constructing corresponding key-value pairs and directly writes the request event to the database. This eliminates the need for middleware such as message queues, reducing dependencies and latency, and lowering system performance losses. Furthermore, it records the request event and its corresponding processing status in the database, improving the system's operational traceability capabilities. This facilitates fault diagnosis and responsibility tracking of request events, avoiding the problem of the system lacking operational traceability and being unable to perform fault diagnosis due to only recording the final state of resources. Moreover, the system can recover request events based on the event status, preventing the complete loss of the operational status of events being processed and improving the efficiency of fault handling. Attached Figure Description
[0011] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0012] Figure 1 A flowchart of a request processing method for a distributed system provided in this application embodiment;
[0013] Figure 2 This application provides a schematic diagram of an event state change.
[0014] Figure 3 A flowchart of a master node determination method provided in an embodiment of this application;
[0015] Figure 4 This is a schematic diagram of the structure of a request processing device for a distributed system provided in an embodiment of this application. Detailed Implementation
[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.
[0017] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.
[0018] In distributed system architecture, the control plane is responsible for receiving, scheduling, and managing external requests to ensure stable system operation. However, current distributed systems typically only record the final state of resources, losing the original context of operation instructions. This results in a lack of operation tracing capabilities, making fault diagnosis and accountability difficult. Furthermore, control nodes rely on volatile memory to maintain operation processing states. When a node crashes or a network partition occurs, the state of the operation being processed is completely lost, leading to low recovery efficiency and high resource consumption. This application constructs request events based on the sequence number corresponding to the request event, then builds key-value pairs for event persistence and records the request event and its corresponding processing state, directly writing it to the database. This facilitates fault diagnosis and accountability for request events, improving the efficiency of fault handling.
[0019] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0020] like Figure 1 As shown, embodiments of this application provide a request processing method for a distributed system, applied to the master node in a control plane process cluster of a distributed system. The execution flow of the request processing method for a distributed system will be described in detail below, including:
[0021] Step S11: Obtain the request data packet sent by the external requesting process based on the remote procedure call framework, and construct the corresponding request event based on the request data packet and the preset sequence number; the request data packet includes the request type and request content, and the preset sequence number corresponds one-to-one with the request event.
[0022] First of all, it should be pointed out that, such as Figure 2 As shown, the distributed system in this embodiment adopts a layered architecture, containing three core modules: an external request process, such as a command line, agent, or web (World Wide Web) frontend, which sends requests to the control plane via RPC (Remote Procedure Call); a control plane process, which has a built-in event packaging engine that transforms external requests into structured events and adds metadata such as timestamps, user IDs (Identity Documents), and operation types; and a database process, which uses an event sourcing compatible database such as ETCD (Etcd Is A Distributed and Reliable Key-Value Store), persists events, and supports idempotency verification and transaction logs.
[0023] Accordingly, in this embodiment, the request data packet sent by the external requesting process based on the Remote Procedure Call (RPC) framework can be obtained first, and a corresponding request event can be constructed based on the request data packet and the preset sequence number. The request data packet includes a request type and a request content, and the preset sequence number corresponds one-to-one with the request event. That is, when submitting an external request, the external requesting process sends a structured request data packet to the control plane through the high-performance RPC framework. The request body contains two core fields: request type and request content.
[0024] Specifically, when constructing corresponding request events based on request data packets and preset sequence numbers, the corresponding event type can be determined according to the request type in the request data packet based on a preset mapping table, and the corresponding request event can be constructed based on the event type. The preset mapping table stores the mapping relationship between request types and request events. Each request event includes a preset sequence number, event type, and request content. In other words, the control plane can perform event transformation on request data packets. The control plane converts requests into standardized events through the request type-event mapping table. The event data structure contains three layers: a metadata layer, which assigns a unique, monotonically increasing event sequence number to each request event; a semantic layer, representing the event type; and a content layer, which fully encapsulates the original request data.
[0025] Step S12: Generate a first key-value pair based on the request event and the corresponding preset sequence number, and generate a first status marker for each request event. Generate a second key-value pair based on the first status marker and the corresponding preset sequence number. The first status marker is a marker that indicates that the request event is an event to be processed.
[0026] In this embodiment, request events can be persisted. A first key-value pair can be generated based on the request event and its corresponding preset sequence number, and a first status marker for each request event can be generated. A second key-value pair can then be generated based on the first status marker and its corresponding preset sequence number. The first status marker is a marker indicating that the request event is an event to be processed. Specifically, a first key can be constructed based on the preset sequence number, and the request content in the request event can be used as the first value to generate a first key-value pair. Correspondingly, a second key can be constructed based on the preset sequence number and the first status marker, and the event status corresponding to the first status marker can be used as the second value to generate a second key-value pair.
[0027] In other words, when performing event persistence in this embodiment, the event processing module first utilizes distributed transactions to ensure the atomicity of the following operations: First, the event body is stored, using / event / n (where n is a monotonically increasing sequence number) as the key and the event content as the value, and written to the database. This ensures, on the one hand, the persistent storage of unprocessed events in the event of a control plane failure, preventing data loss; and on the other hand, it forms a complete operation audit chain, allowing all historical events to be traced sequentially by sequence number. Then, the event status is marked, using / waiting / event / n as the key to record the status as WAITING, and persisted to the database.
[0028] Step S13: Store the first key-value pair and the second key-value pair in a preset database, read the request event from the first key-value pair from the preset database, and process the corresponding request event according to the first status flag in the second key-value pair.
[0029] In this embodiment, the first key-value pair and the second key-value pair can be stored in a preset database. Request events from the first key-value pair can be read from the preset database, and the corresponding request events can be processed based on the first status flag in the second key-value pair. Specifically, a preset monitoring component can be used to monitor the preset database. When the preset monitoring component detects the existence of the first key / event / n in the preset database, the request event corresponding to the first key in the preset database is read based on the first key-value pair, and the request event is saved to a preset global message queue. That is, in this embodiment, the control plane process can also perform event listening in the global message queue. The listening module monitors the key space of the / event / prefix in real time through the database's change data capture function. When the key of / event / n is written to the database, the database pushes the changed content to the listener. When the listener detects the change, it can save the event to the global message queue.
[0030] Accordingly, when processing the corresponding request event based on the first status marker in the second key-value pair, a preset worker thread can be used to monitor the preset global message queue. When a request event exists in the preset global message queue, the request event in the preset global message queue is read, and the request event is processed according to the first status marker in the second key-value pair. Specifically, processing the corresponding request event based on the first status marker in the second key-value pair can generate a second status marker corresponding to the request event, indicating that the request event is an event in progress, and delete the first status marker in the second key-value pair. The second key-value pair is then updated based on the second status marker to obtain the initially updated second key-value pair. Then, the corresponding event handling component is determined according to the request type corresponding to the request event, the request event is routed to the event handling component, and the event handling component processes the request event. After the request event processing is completed, the initially updated second key-value pair in the preset database is updated again. In other words, in this embodiment, the worker thread continuously checks the queue status. When it detects that the global message queue is not empty, it retrieves the earliest enqueued event from the queue for processing. For example: using the second state marker / processing / event / n as the key to record the state of event n as processing, write / processing / event / nKey through an atomic transaction and delete the waiting record of the first state marker / waiting / event / n to ensure the consistency of event state. Then, route the event to the registered processing module for processing according to the event type. After processing is completed, delete the / processing / event / n event from the database.
[0031] Furthermore, during the subsequent update of the second key-value pair after the initial update in the preset database, if the request event is successfully processed, a third status flag indicating successful processing is generated, the second status flag in the initially updated second key-value pair is deleted, the second key-value pair is updated based on the third status flag, and the event processing result is returned to the external request process. If the request event fails to process, a fourth status flag indicating failure is generated, the second status flag in the initially updated second key-value pair is deleted, the second key-value pair is updated based on the fourth status flag, and the event processing result is returned to the external request process. In other words, if event processing fails, the status of event n is recorded as failed using the fourth status flag / failed / event / n as the key. An atomic transaction is used to write the / failed / event / n key and delete the processing record of the / processing / event / n key. If processing is successful, the status of event n is recorded as "success" using the third status marker / success / event / n as the key. An atomic transaction is then used to write the record to the key / success / event / n and delete the "processing" record from the key / processing / event / n. It's also understandable that regardless of whether event processing succeeds or fails, a result needs to be returned to the external requesting process via RPC. The external requesting process will then perform subsequent operations such as retries or alerts based on the returned result.
[0032] Based on the above technical solution, it should be noted that this embodiment involves multiple atomic transactions. All steps must be executed successfully. If any step fails, the entire transaction will roll back to the initial state, ensuring logical consistency and data integrity across multiple operations. For example, during the event persistence phase, atomic transactions ensure the following operations are indivisible: using / event / n as the key, the event content is written to the database as the value to achieve event body persistence; using / waiting / event / n as the key, the event state is marked as WAITING and written to the database to achieve initial state persistence. If any operation fails, the entire transaction rolls back, avoiding situations where the event is stored but the state is not marked, or the state is marked but the event is not stored. During the event state change phase, when the event state changes from WAITING to PROCESSING, SUCCESS, or FAILED, atomic transactions are used to complete both state writing and old state deletion. For example, WAITING→PROCESSING, the atomic transaction simultaneously executes writing / processing / event / n and deleting / waiting / event / n to ensure that no new or old state coexists or state is lost during the state transition. In this way, by using fine-grained status control with keys prefixed with specific prefixes such as / waiting / and / processing / , incomplete events can be accurately identified, enabling rapid recovery without full data synchronization and significantly shortening system failure recovery time. Furthermore, standardized status markers such as / waiting / , / processing / , and / failed / can intuitively display the event lifecycle, facilitating real-time monitoring of system operation status by operations and maintenance personnel. Therefore, end-to-end metrics are observable; key indicators such as event backlog and processing latency can be easily extracted based on status markers and integrated into the monitoring system to achieve visualized operations and maintenance and automated alerts.
[0033] In this embodiment, after obtaining the request data packet sent by the external requesting process based on the remote procedure call framework, a request event is constructed based on a preset sequence number that corresponds one-to-one with the request event. Then, the request event is persisted by constructing corresponding key-value pairs and directly written to the database. This eliminates the need for middleware such as message queues, reducing dependencies and latency, and lowering system performance overhead. Furthermore, the database records the request event and its corresponding processing status, improving the system's operational traceability and facilitating fault diagnosis and accountability for request events. This avoids the problem of lacking operational traceability and difficulty in fault diagnosis due to only recording the final state of resources. Simultaneously, the system can recover request events based on their status, preventing the complete loss of the operational status of events currently being processed and improving fault handling efficiency. For example, when an external process submits an event request to the control plane process, the control plane persists the initial event state as "waiting." Subsequently, the control plane updates the event state to "processing" to indicate the start of processing. Upon successful processing, the state changes from "processing" to "success," and upon failure, it changes to "failed." If a rollback is required, the state returns to "waiting" to await further processing. The entire process clearly demonstrates all possible state transition paths an event may undergo under different processing outcomes, forming a complete state machine closed loop of waiting→processing→success / failed / waiting. Through the complete lifecycle of an event from submission to final processing, an event sourcing and storage mechanism is implemented, ensuring that all operations are persistently stored as immutable events. This not only provides comprehensive data auditing capabilities but also supports system state reconstruction at any point in time, facilitating fault recovery or historical data backtracking and analysis. Simultaneously, atomic transaction management strictly controls the event state transition (WAITING→PROCESSING→COMPLETED / FAILED), effectively preventing data loss or duplicate processing, ensuring eventual consistency of data operations, and achieving high system reliability and data integrity.
[0034] As can be seen from the previous embodiment, this application can construct a request event based on the sequence number corresponding to the request event, then construct key-value pairs for event persistence and record the request event and its corresponding processing status, directly write it to the database, and process the request event. It is understood that the above process is implemented based on the master node in the control plane process cluster of a distributed system. Next, this embodiment will elaborate on the master node determination and subsequent time processing flow. See [link to previous embodiment]. Figure 3 As shown in the figure, this application discloses a method for determining a master node, including:
[0035] Step S21: Determine the identity of each control plane process in the control plane process cluster, and perform the operation of writing the identity to the preset master node key-value pair.
[0036] It is understood that this embodiment uses a master-slave architecture to deploy the control plane process cluster. By competing for the globally unique key in the database through atomic operations, such as / controller / leader, the identity of each control plane process in the control plane process cluster can be determined first, and the operation of writing the identity to the preset master node key-value pair can be performed.
[0037] Step S22: Determine the target identity identifier to be written into the preset master node key-value pair, and use the control plane process corresponding to the target identity identifier as the master node.
[0038] In this embodiment, the target identity identifier can be determined and written into the preset master node key-value pair, and the control plane process corresponding to the target identity identifier is designated as the master node. Furthermore, if the master node fails, the operation of writing the identity identifier into the preset master node key-value pair is re-executed, and the control plane process corresponding to the newly written target identity identifier in the preset master node key-value pair is designated as the new master node. In other words, in this embodiment, when a node starts, it immediately attempts to write its own ID to the LeaderKey. The node that successfully writes its ID is promoted to the Active node, i.e., the master node, and periodically renews its contract using the TTL (Time to Live) mechanism. Other nodes enter a Standby state, continuously monitoring for key changes. When the Active node crashes or fails to renew its contract due to network issues, the database deletes the / controller / leader key. All surviving nodes immediately initiate a new round of preemption to select a new Active control plane to take over control of the business.
[0039] Furthermore, after designating the control plane process corresponding to the target identity identifier written in the preset master node key-value pair as the new master node, it can also scan the second state flag in the preset database and update the second state flag using the first state flag to obtain the corresponding updated second key-value pair. Then, it determines the target event corresponding to the preset sequence number in the updated second key-value pair and saves the target event to the preset global message queue according to the preset sequence number in the updated second key-value pair. In other words, this embodiment adopts an event-driven architecture, and the control plane process is completely stateless. When the new control plane process is selected, it does not need to synchronize any data from the old control plane process. The unfinished events of the old control plane can be recovered through the following steps to achieve seamless master node switching: First, scan all keys with the prefix / processing / event / in the database, use transactional rollback to ensure atomicity, delete the / processing / event / n key from the database after rollback, and write the / waiting / event / n key. Then, read all keys with the prefix / waiting / event from the database and write them to the global message queue in order according to the size of the event sequence number. After the new control plane completes the event recovery, it opens the service port to receive new requests.
[0040] Furthermore, it is understandable that, similar to the atomic transactions in the previous embodiment, when the new control plane node takes over the business during the fault recovery rollback phase in this embodiment, it is necessary to roll back the events being processed before the fault. This rollback operation is completed through atomic transactions: first, the key / processing / event / n in the database is deleted to clear the PROCESSING state, and then the key / waiting / event / n is written to reset the event to the WAITING state, ensuring that the event state is unique and correct after the rollback. Through the stateless control plane design combined with the Leader election mechanism, seamless switching between primary and backup nodes is ensured, improving the overall availability of the system and realizing the system's rapid recovery capability after a fault. This avoids subsequent duplicate processing or omissions, facilitates system fault recovery, and prevents unprocessed events from being difficult to track and continue processing in the new control plane during control plane switching, or from failing to accurately record the currently executing state, which would lead to the inability to guarantee idempotency during recovery. In this way, combined with the above embodiment, through the three-layer mechanism of event persistence, event listening, and database recovery, zero-loss recovery of unprocessed events after a control plane fault is ensured, while maintaining rapid fault recovery.
[0041] like Figure 4 As shown, embodiments of this application also provide a request processing apparatus for a distributed system, applied to the master node in a control plane process cluster of a distributed system, including:
[0042] Event construction module 11 is used to obtain the request data packet sent by the external request process based on the remote procedure call framework, and to construct the corresponding request event based on the request data packet and the preset sequence number; the request data packet includes the request type and request content, and the preset sequence number corresponds one-to-one with the request event;
[0043] The key-value pair generation module 12 is used to generate a first key-value pair based on the request event and the corresponding preset sequence number, generate a first status marker for each request event, and generate a second key-value pair based on the first status marker and the corresponding preset sequence number; the first status marker is a marker indicating that the request event is an event to be processed.
[0044] The event handling module 13 is used to store the first key-value pair and the second key-value pair into a preset database, read the request event from the first key-value pair from the preset database, and process the corresponding request event according to the first status flag in the second key-value pair.
[0045] In some specific embodiments, the event construction module 11 specifically includes:
[0046] The type determination unit is used to determine the corresponding event type based on the request type in the request data packet according to a preset mapping table; the preset mapping table is a table that stores the mapping relationship between request types and request events.
[0047] The event building unit is used to build the corresponding request event for the request data packet based on the event type; the request event includes a preset sequence number, event type and request content.
[0048] In some specific embodiments, the key-value pair generation module 12 specifically includes:
[0049] The first key-value pair generation unit is used to construct a first key based on a preset sequence number and use the request content in the request event as the first value to generate a first key-value pair;
[0050] The second key-value pair generation unit is used to construct a second key based on a preset sequence number and a first state marker, and to use the event state corresponding to the first state marker as the second value to generate a second key-value pair.
[0051] In some specific embodiments, the event handling module 13 specifically includes:
[0052] The data monitoring unit is used to monitor a preset database using preset monitoring components;
[0053] The event storage unit is used to read the request event corresponding to the first key in the preset database based on the first key-value pair when the preset monitoring component detects the existence of the first key in the preset database, and save the request event to the preset global message queue.
[0054] In some specific embodiments, the event handling module 13 specifically includes:
[0055] The queue monitoring unit is used to monitor a preset global message queue using a preset worker thread.
[0056] The event handling unit is used to read the request event in the preset global message queue when there is a request event in the preset global message queue, and process the request event according to the first status flag in the second key-value pair.
[0057] In some specific embodiments, the event handling module 13 specifically includes:
[0058] The tag generation submodule is used to generate the second status tag corresponding to the request event; the second status tag is a tag that indicates that the request event is an event in progress.
[0059] The first tag update submodule is used to delete the first state tag in the second key-value pair and update the second key-value pair based on the second state tag to obtain the second key-value pair after the first update.
[0060] The event routing submodule is used to determine the appropriate event handling component based on the request type corresponding to the request event, and to route the request event to the event handling component.
[0061] The second tag update submodule is used to process request events using the event handling component, and after the request event processing is completed, to update the second key-value pair in the preset database again after the initial update.
[0062] In some specific embodiments, the second marker update submodule specifically includes:
[0063] The first tag generation unit is used to generate a third status tag corresponding to the request event if the request event is successfully processed; the third status tag is a tag that indicates that the request event was successfully processed.
[0064] The first marker deletion unit is used to delete the second status marker in the second key-value pair after the initial update, update the second key-value pair based on the third status marker, and return the event processing result corresponding to the request event to the external request process.
[0065] The second marker generation unit is used to generate a fourth status marker corresponding to the request event if the request event processing fails; the fourth status marker is a marker that indicates the failure of the request event processing.
[0066] The second marker deletion unit is used to delete the second status marker in the second key-value pair after the initial update, update the second key-value pair based on the fourth status marker, and return the event processing result corresponding to the request event to the external request process.
[0067] In some specific embodiments, the request processing apparatus of the distributed system further includes:
[0068] The identifier writing module is used to determine the identity identifier of each control plane process in the control plane process cluster and to perform the operation of writing the identity identifier into a preset master node key-value pair;
[0069] The node determination module is used to determine the target identity identifier to be written into the preset master node key-value pair, and to take the control plane process corresponding to the target identity identifier as the master node; if the master node fails, the operation of writing the identity identifier into the preset master node key-value pair is re-executed, and the control plane process corresponding to the new target identity identifier written into the preset master node key-value pair is taken as the new master node.
[0070] In some specific embodiments, the request processing apparatus of the distributed system further includes:
[0071] The tag scanning module is used to scan the second state tag in the preset database and update the second state tag using the first state tag to obtain the corresponding updated second key-value pair;
[0072] The event saving module is used to determine the target event corresponding to the preset sequence number in the updated second key-value pair, and save the target event to the preset global message queue according to the preset sequence number in the updated second key-value pair.
[0073] For a description of the features in the embodiments corresponding to the request processing device of the above-mentioned distributed system, please refer to the relevant descriptions in the embodiments corresponding to the request processing method of the distributed system, which will not be repeated here.
[0074] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.
[0075] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above-described distributed system request processing method embodiments.
[0076] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described distributed system request processing method embodiments when running.
[0077] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.
[0078] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described distributed system request processing method embodiments.
[0079] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described distributed system request processing method embodiments.
[0080] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0081] The above provides a detailed description of a request processing method and electronic device for a distributed system provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only intended to help understand the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.
Claims
1. A request processing method of a distributed system, characterized by, A master node applied to a cluster of control plane processes of the distributed system, comprising: obtaining a request data packet sent by an external request process based on a remote procedure call framework, and constructing a corresponding request event based on the request data packet and a preset sequence number; the request data packet comprises a request type and a request content, and the preset sequence number corresponds to the request event one by one; generating a first key-value pair based on the request event and the corresponding preset sequence number, and generating a first state marker of each request event, and generating a second key-value pair based on the first state marker and the corresponding preset sequence number; the first state marker is a marker representing that the request event is a to-be-processed event; storing the first key-value pair and the second key-value pair to a preset database, reading the request event in the first key-value pair from the preset database, and processing the corresponding request event according to the first state marker in the second key-value pair; wherein, the first key-value pair is generated based on the request event and the corresponding preset sequence number, comprising: constructing a first key based on the preset sequence number, and taking the request content in the request event as a first value to generate the first key-value pair; correspondingly, the second key-value pair is generated based on the first state marker and the corresponding preset sequence number, comprising: constructing a second key based on the preset sequence number and the first state marker, and taking the event state corresponding to the first state marker as a second value to generate the second key-value pair; and if the master node is changed, scanning the second state marker representing that the request event is a processing event in the preset database, and updating the second state marker with the first state marker to obtain the corresponding updated second key-value pair; correspondingly, the request event is processed according to the first state marker in the second key-value pair, comprising: monitoring a preset global message queue by using a preset worker thread; when the request event exists in the preset global message queue, reading the request event in the preset global message queue, and generating a second state marker corresponding to the request event; the second state marker is a marker representing that the request event is a processing event; deleting the first state marker in the second key-value pair based on an atomic transaction, and updating the second key-value pair based on the second state marker to obtain a first updated second key-value pair; determining a corresponding event processing component according to the request type corresponding to the request event, and routing the request event to the event processing component; processing the request event by using the event processing component, and updating the first updated second key-value pair in the preset database again after the request event is processed.
2. The request processing method of a distributed system according to claim 1, characterized by, the request event is constructed based on the request data packet and a preset sequence number, comprising: determining a corresponding event type according to the request type in the request data packet based on a preset mapping table; the preset mapping table is a table storing the mapping relationship between the request type and the request event; constructing the request event corresponding to the request data packet based on the event type; the request event comprising the preset sequence number, the event type and the request content.
3. The request processing method of a distributed system according to claim 2, wherein The reading of the request event in the first key-value pair from the preset database comprises: monitoring the preset database by using a preset monitoring component; when the preset monitoring component monitors that the first key exists in the preset database, reading the request event corresponding to the first key in the preset database based on the first key-value pair, and saving the request event to a preset global message queue.
4. The request processing method of a distributed system according to claim 3, characterized by, The re-updating of the second key-value pair in the preset database after the request event processing is completed comprises: if the request event processing is successful, generating a third state mark corresponding to the request event; the third state mark being a mark representing that the request event processing is successful; deleting the second state mark in the second key-value pair after the initial update, updating the second key-value pair based on the third state mark, and returning an event processing result corresponding to the request event to the external request process; if the request event processing fails, generating a fourth state mark corresponding to the request event; the fourth state mark being a mark representing that the request event processing fails; deleting the second state mark in the second key-value pair after the initial update, updating the second key-value pair based on the fourth state mark, and returning an event processing result corresponding to the request event to the external request process.
5. The request processing method of a distributed system according to claim 4, characterized by, Before the obtaining of the request data packet sent by the external request process based on the remote procedure call framework, the method further comprises: determining the identity of each control plane process in the control plane process cluster, and performing an operation of writing the identity into a preset master node key-value pair; determining a target identity written into the preset master node key-value pair, and taking the control plane process corresponding to the target identity as the master node; correspondingly, after taking the control plane process corresponding to the target identity as the master node, the method further comprises: if the master node fails, re-performing the operation of writing the identity into the preset master node key-value pair, and taking the control plane process corresponding to a new target identity written into the preset master node key-value pair as a new master node.
6. The request processing method of a distributed system according to claim 5, wherein, After taking the control plane process corresponding to the new target identity written into the preset master node key-value pair as the new master node, the method further comprises: scanning a second state mark in the preset database, and updating the second state mark by using the first state mark to obtain an updated second key-value pair; determining a target event corresponding to the preset sequence number in the updated second key-value pair, and saving the target event to the preset global message queue according to the preset sequence number in the updated second key-value pair.
7. An electronic device, comprising: comprise: a memory for storing a computer program; a processor for executing the computer program to implement the steps of the request processing method of the distributed system according to any one of claims 1 to 6.
Citation Information
Patent Citations
One-way data transmission method and system and extranet client
CN115913742A