Request deduplication processing method and electronic device

Abstract

This application discloses a method and electronic device for request deduplication, relating to the field of storage system technology. The method includes: for an event processing request sent from a control terminal to a data terminal, the control terminal collects multiple request information corresponding to the event processing request, including an event identifier, request type, object identifier, and request content. A first identifier is obtained from this request information. This first identifier can uniquely identify and distinguish the event processing request, enabling the event processing request received by the control terminal from the data terminal to have business semantic-level deduplication capabilities. Consequently, when the data terminal receives a duplicate request, it can directly return historical processing results, avoiding duplicate execution caused by retries after a control terminal failure switch. Simultaneously, since the first identifier contains the request content, requests of the same type but with different parameters will not be misjudged as duplicate requests, thereby ensuring the consistency of the storage state and improving the accuracy of request deduplication.

Description

Technical Field

[0001] This application relates to the field of storage system technology, and in particular to a method and electronic device for deduplicating requests. Background Technology

[0002] As storage system architecture continues to evolve from monolithic to a separation of control and data planes, communication between the control and data planes via Remote Procedure Calls (PRCs) has become a crucial means of improving system scalability, fault isolation capabilities, and the convenience of rolling upgrades. In a control-data plane separated environment, the control plane sending requests to the data plane (C2A_RPC) and the data plane reporting requests to the control plane (A2C_RPC) constitute a bidirectional RPC interaction link. However, duplicate requests that may occur during RPC calls can affect storage state consistency. Related technologies typically design deduplication for requests based on unidirectional RPC calls: the client attaches a unique identifier when initiating an RPC request, usually a combination of the client ID and the request sequence number, which the server uses to identify whether the RPC request is duplicated; cache cleanup relies on timeouts, heartbeats, or a client-carried list of requests to be reclaimed. These methods struggle to accurately distinguish between requests of the same type but with different parameters and cannot adapt to the differentiated scenarios of bidirectional RPCs, resulting in poor accuracy in request deduplication. Therefore, improving the accuracy of request deduplication is a pressing technical problem that needs to be solved. Summary of the Invention

[0003] This application provides a method and electronic device for deduplication of requests, in order to at least solve the problem of how to improve the accuracy of request deduplication in related technologies.

[0004] This application provides a method for deduplicating requests, including:

[0005] In response to an event handling request, obtain the corresponding first request information; wherein, the first request information includes the event identifier, request type, object identifier, and request content;

[0006] Generate corresponding first identifier information based on the first request information;

[0007] The first request information and the first identification information are sent to the data terminal so that the data terminal can determine whether the event handling request is a processed request based on the first identification information.

[0008] The receiving end sends a first return result; wherein, if the data end determines that the event handling request is a processed request, the first return result is the corresponding first historical processing result; if the data end determines that the event handling request is not a processed request, the first return result is the first processing result generated by the data end after executing the event handling request.

[0009] This application also provides a method for deduplicating requests, including:

[0010] Receive an event processing request and first identification information sent by the control terminal; wherein, the first identification information is generated by the control terminal in response to the event processing request, based on the first request information corresponding to the event processing request, and the first request information includes event identifier, request type, object identifier and request content;

[0011] Based on the first identifier information, determine whether the event handling request is a processed request;

[0012] Send a first return result to the control terminal; wherein, if it is determined that the event handling request is a processed request, the first return result is the corresponding first historical processing result; if it is determined that the event handling request is not a processed request, the first return result is the first processing result generated by the data terminal after executing the event handling request.

[0013] This application also provides a method for deduplicating requests, including:

[0014] The control terminal responds to the event handling request and obtains the corresponding first request information; wherein, the first request information includes the event identifier, request type, object identifier and request content;

[0015] The control terminal generates corresponding first identification information based on the first request information;

[0016] The control terminal sends the first request information and the first identification information to the data terminal;

[0017] The data terminal determines whether the event handling request is a processed request based on the first identifier information;

[0018] The data terminal sends a first return result to the control terminal; wherein, if the data terminal determines that the event handling request is a processed request, the first return result is the corresponding first historical processing result; if the data terminal determines that the event handling request is not a processed request, the first return result is the first processing result generated by the data terminal after executing the event handling request.

[0019] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing the deduplication method of any of the above requests when executing the computer program.

[0020] This application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of the deduplication method for any of the above requests.

[0021] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the deduplication method for any of the above requests.

[0022] This application provides an accurate request deduplication scheme. For event processing requests sent from the control end to the data end, the control end collects multiple request information corresponding to the event processing request, including event identifier, request type, object identifier, and request content. Through these request information, a first identifier is obtained. The first identifier can uniquely identify and distinguish the event processing request, enabling the event processing requests received by the control end from the data end to have business semantic-level deduplication capabilities. Consequently, when the data end receives a duplicate request, it can directly return the historical processing result, avoiding duplicate execution caused by retries after control end failure switching. At the same time, since the first identifier includes the request content, requests of the same type but different parameters will not be misjudged as duplicate requests, thereby ensuring the consistency of the storage state and improving the accuracy of request deduplication. Attached Figure Description

[0023] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A schematic diagram of a request deduplication system architecture provided for an embodiment of this application;

[0025] Figure 2 A flowchart illustrating the deduplication method for requests provided in this application embodiment. Figure 1 ;

[0026] Figure 3 A flowchart illustrating the deduplication method for requests provided in this application embodiment. Figure 2 ;

[0027] Figure 4 A flowchart illustrating the deduplication method for requests provided in this application embodiment. Figure 3 ;

[0028] Figure 5 A schematic diagram of the component structure of a request deduplication system provided in an embodiment of this application;

[0029] Figure 6 A schematic diagram of a C2A_RPC distribution process is provided for embodiments of this application;

[0030] Figure 7A schematic diagram of A2C_RPC reporting interaction provided for an embodiment of this application;

[0031] Figure 8 A schematic diagram of the structure of the request deduplication processing device provided in the embodiments of this application;

[0032] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0033] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0034] It should be noted that, in the description of the embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in the embodiments of this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0035] First, the key terms in the embodiments of this application will be explained:

[0036] Control plane: The side responsible for receiving, distributing, and processing RPC requests. In this article, it refers to the controller-side service process of the storage system.

[0037] Data plane: The side responsible for initiating RPC requests for specific business scenarios. In this article, it refers to the device-side or node-side processes of the storage system.

[0038] C2A_RPC: A control plane to data plane RPC. In the downlink direction, the control plane actively initiates the request, and the data plane acts as the server to receive and process it.

[0039] A2C_RPC: The RP from the data plane to the control plane. The reporting direction RPC is initiated by the data plane, and the control plane acts as the server to receive and process the requests.

[0040] Event ID (event_id): A global identifier used to uniquely identify a business event, generated and maintained during the event processing in the control plane.

[0041] Object ID (object_id): An identifier used to uniquely identify a node.

[0042] Request type (request_type): The type of RPC request used to distinguish different categories of RPC operations.

[0043] Request content (request_msg): The complete message body of the RPC request, which contains the specific parameters of this business operation.

[0044] Global ID / Globally Unique ID (rpc_global_id): A globally unique ID generated locally by the data plane when reporting an RPC request.

[0045] Idempotent key / idempotent determination key: A fixed-length string generated by hashing a combination of several identifier fields, used to uniquely identify an RPC request and its semantics.

[0046] Key-value (KV) cache: A key-value storage structure with idempotent keys as keys and processing status and processing result as values, used in C2A_RPC scenarios.

[0047] Database (DB) cache: A database storage structure with idempotent keys as keys and processing status and processing results as values, used in A2C_RPC scenarios.

[0048] In the development of storage systems, the architecture has evolved from monolithic to decoupled. Early storage systems ran the control and data processing modules within the same process, communicating directly through function calls. However, as business scale and reliability requirements increased, the scalability and fault isolation capabilities of monolithic architectures became insufficient. Splitting the control plane into independent processes, communicating with the data plane via network RPC, became the direction of storage architecture evolution. This decoupling brought advantages such as clear module boundaries, sufficient fault isolation, and convenient rolling upgrades, but also introduced new technical challenges: when the control plane process fails over, the newly started control plane process cannot accurately know whether previously issued RPC requests have been completed by the data plane. The core function of the control plane is to schedule storage volumes and handle various business events. When the control plane cannot continue operating due to network interruptions, process crashes, or node failures, the system quickly activates a backup control plane to take over. The first thing the backup control plane does after taking over is to rerun any unprocessed events to ensure that every instruction is ultimately executed. The problem is that the control plane has no idea how far its RPC requests have progressed.

[0049] For example, control plane A sends an RPC request to the data plane to increment the replica count of a storage volume by 1. The data plane receives the request and executes the operation, updating the replica count, but the response is lost on the way back. Control plane A crashes before receiving the result. At this point, a new control plane B takes over, unaware of whether the RPC was actually processed. If B chooses to resend the request, the data plane will execute the replica count increment operation again, ultimately resulting in a replica count that is one more than expected. This phenomenon is particularly prominent in scenarios involving replica count adjustments and storage quota control. Once an RPC is executed repeatedly, the state of the storage system will deviate from expectations. On the other hand, the data plane does not simply passively receive commands from the control plane. In actual operation, the data plane needs to actively report node link changes, alarm information, and statistical metrics to the control plane. For example, when a node link change is detected, the data plane needs to notify the control plane of this change so that the control plane can update the global view. If a report request sent by the data plane is lost on the return path, the data plane may repeatedly send the same report content, causing the control plane to process it repeatedly.

[0050] Therefore, in an architecture where the control plane and data plane are separated, both bidirectional RPCs need to address the idempotency issue. Although the downlink (C2A_RPC) and uplink (A2C_RPC) have different business semantics, they both face the same core problem: how to identify duplicate requests and return stable results.

[0051] In distributed systems, various technical solutions exist to address the issue of duplicate requests that may occur during RPC calls. The basic idea behind these solutions is to uniquely identify each request on the server side and use this identifier to determine whether the request is a duplicate, thereby preventing repeated execution.

[0052] A common practice is for the client to attach a unique identifier to the RPC request. This unique identifier is typically a combination of the client ID and the request sequence number, in the format "client ID + incrementing sequence number". Upon receiving the request, the server first checks its local cache to see if the processing result corresponding to that identifier already exists. If the cache is hit, it means the request has already been processed, and the server directly returns the cached result without repeating the process. If the cache is not hit, the server proceeds with the normal business logic and writes the processing result to the cache. In this approach, the server typically maintains a processing status field for each request identifier to indicate whether the request is currently "processing" or "completed". When the server receives a duplicate request and finds that the request status is "processing", it returns a specific error code or status code to inform the client that the request is being processed, and the client executes a backoff and retry strategy accordingly. When the status is "completed", the cached processing result is returned directly. The cache cleanup mechanism typically relies on a time dimension. One approach is for the server to maintain a heartbeat lease for each client. If a client does not send a new request or heartbeat within a certain period, the server considers the client offline and treats all its corresponding cached records as reclaimable. Another approach is for the client to include a "list of requests to be reclaimed" in subsequent requests, marking requests that have been processed but not yet actively cleaned up by the server. The server then uses this list for targeted cleanup. These solutions can provide some deduplication in single-point RPC call scenarios, but their limitations become apparent when fault recovery is involved.

[0053] Many related technologies are designed with unidirectional RPC calls in mind, failing to provide a unified model for the differentiated needs of bidirectional RPC. In architectures where the control plane and data plane are separated, the usage patterns, data characteristics, and reliability requirements of the control plane sending requests to the data plane differ significantly from those of the data plane reporting requests to the control plane. Related technologies typically only address idempotency issues in one direction, lacking a systematic processing mechanism for the other. Identification designs do not include the request content itself. The combination of client number and request sequence number can only distinguish different request sequences from the same client, failing to differentiate scenarios where business content changes within the same call sequence. When multiple requests with the same operation type but different parameters are sent in the same call chain, related solutions may misjudge them as duplicate requests, leading to the incorrect deduplication of some legitimate requests. Cache eviction mechanisms lack linkage with business progress. Cache cleanup is driven by time or heartbeats, unrelated to whether business requests have been correctly processed. This passive eviction strategy may result in: cached records existing for a long time without timely cleanup occupying memory, or cache being cleaned before the business process is fully completed, causing subsequent retries to fail. The different storage requirements of RPCs in different directions are not considered. Sending RPCs typically requires high real-time performance, necessitating fast write and query caching; reporting RPCs, on the other hand, may require persistence to meet auditing requirements. Related technologies usually employ the same storage strategy to handle all types of RPCs, failing to optimize for different scenarios. Based on the above analysis, the accuracy of request deduplication using these technologies is poor.

[0054] To address the aforementioned technical issues, embodiments of this application provide a method and electronic device for deduplicating requests. The control terminal collects multiple request information corresponding to an event processing request, including an event identifier, a request type, an object identifier, and a request content. Through this request information, a first identifier is obtained. The first identifier can uniquely identify and distinguish the event processing request, enabling the event processing request obtained by the control terminal to have business semantic-level deduplication capabilities.

[0055] Optionally, embodiments of this application aim to address how to ensure the idempotency of bidirectional RPC (C2A_RPC and A2C_RPC) operations in a storage system where the control plane and data plane are separated, and to avoid unexpected changes in the storage state due to RPC retries.

[0056] Optionally, embodiments of this application also aim to address at least one of the following problems: How to uniquely identify C2A_RPC requests sent from the control plane to the data plane, enabling the data plane to accurately identify duplicate RPC requests and distinguish between requests of the same type but with different parameters. How to effectively manage C2A_RPC cache in the data plane and implement event-based collaborative recycling, so that the cache lifecycle is precisely bound to the business execution cycle. How to uniquely identify A2C_RPC requests reported from the data plane to the control plane, enabling the control plane to accurately identify duplicate reporting requests and adopt storage strategies suitable for the reporting scenario. How to achieve efficient expiration recycling of A2C_RPC cache in the control plane to avoid unlimited growth of database storage. How to adopt differentiated storage strategies for different characteristics of bidirectional RPC, so that the system achieves a reasonable balance between real-time performance and persistence.

[0057] The specific application environment architecture or hardware architecture on which the deduplication method for requests depends is described here. (References) Figure 1 , Figure 1 This is a schematic diagram of a request deduplication system architecture provided in an embodiment of this application. The request deduplication system includes a data terminal 10 and a control terminal 11.

[0058] Optionally, the data terminal 10 can be the data plane in this embodiment of the application, or it can be a device-side process or a node-side process in the storage system that initiates RPC requests for specific business scenarios. As an example, the data terminal 10 can run on the device side or node side of the storage system, and is responsible for receiving C2A_RPC requests issued by the control terminal 11 and executing the corresponding processing logic, while also being responsible for actively initiating A2C_RPC reporting requests to the control terminal 11.

[0059] Optionally, the control terminal 11 can be the control plane in this embodiment of the application, or it can be a controller-side service process in the storage system responsible for receiving, distributing and processing RPC requests. As an example, the control terminal 11 can run on the controller side of the storage system and is responsible for scheduling storage volumes, processing business events, sending C2A_RPC requests to the data terminal 10, and receiving and processing A2C_RPC requests reported by the data terminal 10.

[0060] Optionally, communication between the data terminal 10 and the control terminal 11 is achieved through Remote Procedure Call (RPC). Specifically, the RPC interaction between the data terminal 10 and the control terminal 11 includes two directions: C2A_RPC, in which the control terminal 11 sends a request to the data terminal 10, and A2C_RPC, in which the data terminal 10 reports a request to the control terminal 11. C2A_RPC is initiated by the control terminal 11, and the data terminal 10, acting as the server, receives and processes it; A2C_RPC is initiated by the data terminal 10, and the control terminal 11, acting as the server, receives and processes it.

[0061] Optionally, the data terminal 10 is also configured with a key-value cache, or the data terminal 10 is connected to the key-value cache, for storing the processing status and processing result corresponding to the C2A_RPC request. Specifically, the key-value cache stores the first identification information as the key and the third status flag and the first processing result as the value. The first identification information is generated by the control terminal 11 based on the first request information corresponding to the event processing request. The first request information includes the event identifier, request type, object identifier, and request content; the third status flag includes a completed flag and a processing flag. After receiving the C2A_RPC request, the data terminal 10 queries the key-value cache using the first identification information as the key. If a match is found and the third status flag is set to completed, the first historical processing result is returned directly. If a match is found and the third status flag is set to processing, the processing result is returned after processing is completed. If no match is found, the event processing request is executed, and a corresponding event processing record is created in the key-value cache. The third status flag is set to processing, updated to completed after processing, and written to the first processing result.

[0062] As an example, key-value caching can be implemented using memory caching to meet the requirements of high real-time performance, batch operations, and fast response in C2A_RPC. Optionally, an expiration time can be set in the key-value cache, or the control terminal 11 can actively trigger a cache cleanup request after a business event is completed. The data terminal 10 then deletes key-value cache records corresponding to the first identifier information in batches according to the cache cleanup request, so that the lifecycle of the key-value cache is precisely bound to the business event processing cycle.

[0063] Optionally, the control terminal 11 is also configured with a database, or the control terminal 11 is connected to a database, for storing the processing status and processing result corresponding to the A2C_RPC request. Specifically, the database stores the second identification information as the key and the second status flag and the second processing result as the value. The second identification information is generated by the data terminal 10 based on the second request information corresponding to the reporting request. The second request information includes the request type, object identifier, request content, and globally unique number; the second status flag includes a completed flag and a processing flag. After receiving the A2C_RPC request, the control terminal 11 queries the database using the second identification information as the key. If a match is found and the second status flag is set to completed, the second historical processing result is returned directly. If a match is found and the second status flag is set to processing, the processing result is returned after processing is completed. If no match is found, the reporting request is executed, and a corresponding reporting request processing record is created in the database. The second status flag is set to processing, and after processing is completed, it is updated to completed and written to the second processing result.

[0064] As an example, the database can be implemented using persistent storage to meet the auditing requirements of A2C_RPC and the need for a longer fault recovery window. Optionally, the control terminal 11 can also be configured with a scheduled cleanup task to scan the database and delete reporting request processing records whose creation time exceeds a preset duration, thus preventing the database records from growing indefinitely. The preset duration can be set according to the longest recovery time for fault switching on the control terminal 11, for example, set to 30 minutes, with the scheduled task scanning the database every 30 minutes.

[0065] As an example, the control terminal 11 may include an event manager, an RPC idempotency manager, an A2C_RPC cache table, and a scheduled cleanup task. The event manager is used to schedule and distribute business events and maintain the event processing state machine; the RPC idempotency manager is used to generate the first identifier information for C2A_RPC and maintain the mapping relationship between event identifiers and the first identifier information, triggering a cache cleanup request after the business event processing is completed; the A2C_RPC cache table is used to store request records reported by the data terminal 10; and the scheduled cleanup task is used to periodically clean up expired A2C_RPC cache records. The data terminal 10 may include a C2A_RPC processor, a key-value cache, a global number manager, and an A2C_RPC reporter. The C2A_RPC processor is used to receive C2A_RPC requests from the control terminal 11, interact with the key-value cache to complete idempotency judgment, and actually execute the processing logic; the key-value cache is used to store the processing status and processing result with the first identification information as the key; the global number manager is used to assign a globally unique number to the A2C_RPC reporting request initiated by this node; the A2C_RPC reporter is used to actively report the status changes of the data terminal 10 to the control terminal 11.

[0066] As an example, data endpoint 10 and control endpoint 11 can be deployed on different physical nodes, or in different processes or containers on the same physical node. There can be one or more data endpoints 10, and one or more control endpoints 11. When a control endpoint 11 fails over, the backup control endpoint takes over and reschedules unfinished business events, resending C2A_RPC requests carrying the same first identifier information to data endpoint 10. Data endpoint 10 can identify whether the request has been processed based on the first identifier information and third status flag in the key-value cache, thus returning historical processing results or executing processing to avoid duplicate execution leading to storage state deviations. When data endpoint 10 repeatedly sends A2C_RPC reporting requests, control endpoint 11 can identify whether the reporting request has been processed based on the second identifier information and second status flag in the database, thus returning historical processing results or executing processing to avoid duplicate processing.

[0067] As an example, the request deduplication system provided in this application can be applied to storage systems, as well as to scenarios with a control-end and data-end separation architecture and bidirectional RPC interaction requirements, such as message queue systems and cloud service resource scheduling systems. In a message queue system, when a producer sends a message to a consumer, it corresponds to the C2A_RPC direction, and when a consumer reports consumption progress to a producer, it corresponds to the A2C_RPC direction. In a cloud service resource scheduling system, when the scheduler issues resource allocation instructions to computing nodes, it corresponds to the C2A_RPC direction, and when a computing node reports resource status to the scheduler, it corresponds to the A2C_RPC direction.

[0068] Optionally, both the data terminal 10 and the control terminal 11 may be configured with processing devices to execute the deduplication method of the request in the embodiments of this application.

[0069] It should be understood that the aforementioned processing device can be implemented by a processor reading instructions from memory and executing those instructions, or it can be implemented by a chip circuit.

[0070] Furthermore, the network architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0071] Figure 2 A flowchart illustrating the deduplication method for requests provided in this application embodiment. Figure 1 ,like Figure 2As shown, this application embodiment provides a method for deduplicating requests, and the execution subject of this method is the aforementioned control terminal 11. The method is described in detail below:

[0072] S201: In response to the event handling request, obtain the corresponding first request information.

[0073] The first request information includes the event identifier, request type, object identifier, and request content.

[0074] Optionally, the event processing request here can be a request generated by the event manager on the control end during business event scheduling, such as storage volume creation, replica count adjustment, and storage quota control. The event identifier is used to uniquely identify a business event; the request type is used to distinguish different RPC operation categories; the object identifier is used to uniquely identify a node or storage object; and the request content is the specific parameters of the business operation. The control end can obtain the aforementioned first request information from the event state machine, event context, or business request parameters.

[0075] S202: Generate corresponding first identification information based on the first request information.

[0076] Optionally, based on the first request information, corresponding first identification information is generated, including: concatenating the event identifier, request type, object identifier, and request content to obtain a first string; and performing a hash operation on the first string to obtain the first identification information.

[0077] In one possible implementation, the first identifier is an idempotent key.

[0078] Alternatively, the C2A_RPC idempotency key (c2a_idempotency_key) is calculated as follows:

[0079] c2a_idempotency_key=H(event_id||request_type||object_id||request_msg)

[0080] The formula above indicates that a hash function is used to perform a hash operation on the concatenated string of the four fields: event identifier (event_id), request type (request_type), object identifier (object_id), and request content (request_msg), resulting in a fixed-length idempotent key. H() represents the hash operation. As long as these four fields are completely identical, the calculated idempotent key will always be the same.

[0081] Optionally, the above concatenation process can be performed in a preset order, and the hash operation can use any type of hash function; this application embodiment does not impose specific limitations. The first identification information can be used as the key of the key-value cache, so its length can be fixed. The first identification information is not only used for deduplication but also serves as an association index between events and cache records, facilitating collaborative cache cleanup after subsequent events are completed.

[0082] Here, in this embodiment of the application, the event identifier, request type, object identifier, and request content are concatenated and hashed to transform the four business fields into a fixed-length first identifier. The first identifier is identical when all four fields are identical, and different when any field changes. Therefore, it contains complete business semantics and is convenient to use as a key for key-value caching, improving query efficiency and preventing requests of the same type but with different parameters from being misjudged as duplicate requests.

[0083] S203: Send the first request information and the first identification information to the data terminal so that the data terminal can determine whether the event processing request is a processed request based on the first identification information.

[0084] Optionally, the control end can send the first request information and the first identification information to the data end via an RPC message. The first identification information can be carried in the message header or message body of the RPC message. The RPC request can be a synchronous call or an asynchronous call. The control end can also set a timeout period to trigger a retry if the first return result is not received within the timeout period. During the retry, the control end should reuse the same first identification information so that the data end can perform idempotency deduplication.

[0085] S204: The first return result sent by the receiving data terminal.

[0086] If the data terminal determines that the event handling request is a processed request, the first return result is the corresponding first historical processing result; if the data terminal determines that the event handling request is not a processed request, the first return result is the first processing result generated by the data terminal after executing the event handling request.

[0087] Optionally, when determining whether an event handling request is a processed request, the data end can also distinguish between two states: "completed" and "processing". If the first identifier information hits the key-value cache and the state is "completed", the first historical processing result is returned; if it hits and the state is "processing", the first processing result is returned after the request is processed; if it misses, the event handling request is executed and written to the cache. This avoids concurrent duplicate requests being executed simultaneously. The first return result received by the control end can include the processing result itself, or it can include the corresponding status identifier or error code, so that the control end can determine whether to retry or wait.

[0088] Optionally, after sending the first request information and the first identification information to the data terminal, the method further includes: creating a first status flag corresponding to the event processing request; wherein the first status flag is used to identify the sending status of the event processing request; the first status flag includes a pending confirmation status and a completed status; in response to the first return result, the first status flag is updated from the pending confirmation status to the completed status.

[0089] Optionally, the first status flag can be stored in the event state machine or local storage of the control end, and optionally persisted, so that after a failover of the control end, the new control end can determine which event processing requests have been sent but have not yet received the first return result based on the first status flag, thereby triggering a retrieval. The first status flag is different from the third status flag in the key-value cache of the data end: the first status flag represents the sending confirmation status on the control end side, and the third status flag represents the service processing status on the data end side. Retry conditions may include: no first return result received after a preset time, rescheduling the event processing request after a failover of the control end, or retrying according to the backoff strategy after receiving a response indicating that processing is underway.

[0090] In one possible implementation, the C2A_RPC cache's state management works as follows: When an RPC request is sent, the data plane first creates a record with a status of "processing". After processing is complete, it updates the status to "completed" and writes the actual processing result into the cache. If the same request is received subsequently, and the data plane finds that the cached record already has a status of "completed", it directly returns the cached result to the control plane without re-executing the processing logic. If the status is "processing", it waits for processing to complete before updating the status to "completed", writing the actual processing result into the cache, and then returning the result.

[0091] This application embodiment creates a first status flag on the control side and distinguishes between the pending confirmation status and the completed status, which can track the sending and confirmation status of event processing requests. When a response is lost or the control side fails, the control side can identify pending confirmation requests that have not yet received the first return result and trigger a retry; after receiving the first return result, it updates to the completed status, so that the status on the control side is consistent with the processing result on the data side, thereby improving the reliability of retry.

[0092] Optionally, embodiments of this application also configure a coordinated cache reclamation mechanism. Specifically, after receiving the first return result sent by the data terminal, the method further includes: generating a cache cleanup request; wherein the cache cleanup request carries an event identifier and / or first identifier information; and sending the cache cleanup request to the data terminal so that the data terminal deletes the first identifier information and / or the cache record data corresponding to the first identifier information according to the cache cleanup request.

[0093] Optionally, the cache cleanup request can be triggered after the business event processing is complete, rather than after the first return result. This is because a business event may contain multiple C2A_RPC requests; if cleanup is triggered upon receiving only one return result, cache records for other incomplete requests under the same event might be mistakenly deleted. The cache cleanup request can carry an event identifier and at least one first identifier generated during the event processing. Upon receiving this, the data end deletes the key-value cache records corresponding to these first identifiers in batches. If the cleanup request is lost or partially fails, a fallback expiration time can be set for final cleanup.

[0094] One possible implementation of the C2A_RPC cache's collaborative cleanup mechanism is as follows: During event processing, the control plane records the idempotent keys of these RPCs. After the event ends, it proactively sends a cache cleanup request to the data plane, containing the event_id and the corresponding list of idempotent keys. Upon receiving this request, the data plane deletes these cache records in batches. This mechanism, where the cleanup is proactively triggered by the control plane, differs from the traditional approach of passively collecting data based on timeouts. It allows for more precise cleanup of old data based on the event execution progress, thus clearing the cache more quickly.

[0095] This application embodiment generates a cache cleanup request carrying an event identifier and / or a first identifier information at the control end and sends it to the data end, enabling the data end to delete the corresponding cache record by event or by identifier, thus realizing event-driven collaborative cache reclamation. Compared with passive reclamation methods that rely on timeouts and heartbeats, this method links the cache lifecycle with the business processing process, reduces the long-term memory occupation of completed caches, and also reduces the risk of cache reclamation becoming mismatched with business progress.

[0096] In one possible implementation, the steps for ensuring idempotency in C2A_RPC (control plane sending RPC requests to data plane) are as follows: When processing an event, the control plane often needs to send multiple RPC requests to the data plane consecutively. Before each request is sent, the control plane calculates the idempotency key for this batch of requests and sends it to the data plane along with the request content. After receiving the request, the data plane first checks its local key-value cache using the idempotency key. If the cache already contains the record, it means it has been processed before, and the data plane directly returns the result stored in the cache. If the record is not in the cache, the request is processed normally, and the processing result is written to the cache.

[0097] This application embodiment addresses event processing requests sent from the control terminal to the data terminal. The control terminal collects multiple request information corresponding to the event processing request, including event identifier, request type, object identifier, and request content. Through this request information, a first identifier is obtained. The first identifier can uniquely identify and distinguish the event processing request, enabling the event processing request received by the control terminal from the data terminal to have business semantic-level deduplication capability. Consequently, when the data terminal receives a duplicate request, it can directly return the historical processing result, avoiding repeated execution due to retries after control terminal failure switching. At the same time, since the first identifier contains the request content, requests of the same type but different parameters will not be misjudged as duplicate requests, thereby ensuring the consistency of the stored state and improving the accuracy of request deduplication processing.

[0098] Optionally, this application embodiment configures a deduplication method for bidirectional requests. Accordingly, the above method further includes: receiving a reporting request and second identification information sent by the data terminal; wherein, the second identification information is generated by the data terminal in response to the reporting request based on the second request information corresponding to the reporting request, and the second request information includes a request type, object identifier, request content, and globally unique number; determining whether the reporting request is a processed request based on the second identification information; and sending a second return result to the data terminal.

[0099] If it is determined that the reported request is a processed request, the second return result is the corresponding second historical processing result; if it is determined that the reported request is not a processed request, the second return result is the second processing result generated after executing the reported request.

[0100] Optionally, A2C_RPC can be initiated proactively by the data terminal when it detects status changes such as node link changes, alarm information, or statistical indicator changes. The globally unique identifier can be generated and managed locally by the data terminal 10, for example, based on the data terminal's local node identifier and local incrementing sequence number, ensuring that the globally unique identifier for different reporting requests within the same data terminal node is not duplicated. After receiving the second identifier information, the control terminal can query the database cache. If a match is found, the historical result is returned; otherwise, the reporting request is executed and written to the database cache.

[0101] Optionally, the client / server roles of the control end and the data end in bidirectional RPC can be interchanged: in C2A_RPC, the control end is the client and the data end is the server; in A2C_RPC, the data end is the client and the control end is the server.

[0102] This application embodiment receives reporting requests and second identification information sent by the data terminal from the control terminal, and determines whether the reporting request has been processed. If processed, it returns a second historical processing result; if not processed, it executes the request and returns the second processing result, thus achieving idempotent processing in the A2C_RPC direction. When the data terminal reports repeatedly, the control terminal can identify the repetition and return the historical result, avoiding duplicate processing. This is particularly suitable for scenarios where the data terminal resends reporting requests after a control terminal failure switchover.

[0103] Optionally, determining whether a reporting request is a processed request based on the second identification information includes: querying a preset database based on the second identification information to determine whether a reporting request is a processed request; wherein the preset database stores multiple historical reporting request processing records corresponding to reporting requests, and the reporting request processing records are associated with the second identification information.

[0104] Optionally, the default database can be located at the control end or be a persistent database accessible to the control end. The reporting request processing record may include fields such as second identification information, second status flag, second processing result, and creation time. The second identification information can serve as the primary key or unique index of the database table to ensure that only one record exists for the same reporting request in the database. In concurrent repeated reporting scenarios, database unique constraints or transaction mechanisms can be used to avoid duplicate writes and processing.

[0105] Here, this embodiment of the application persistently stores multiple historical reporting request processing records corresponding to reporting requests in a preset database, and associates the records with second identification information, so that the control terminal can determine whether the reporting request has been processed based on database queries. Since the database has persistent characteristics, the control terminal can still query historical processing records after a fault switch, continue to ensure the idempotency of the reporting direction, and meet the requirements of auditing and a longer fault recovery window.

[0106] Optionally, the reporting request processing record includes a second status marker corresponding to historical reporting requests; the second status marker includes a completed marker and a processing marker; based on the second identification information, a query is performed in a preset database to determine whether the reporting request is a processed request, including: querying the preset database for reporting request processing records associated with the second identification information; if a reporting request processing record is found and the second status marker is a completed marker, the reporting request is determined to be a processed request, and the second historical processing result in the reporting request processing record is used as the second return result; if a reporting request processing record is found and the second status marker is a processing marker, the process waits for the reporting request processing to complete before using the second processing result obtained as the second return result; if no reporting request processing record associated with the second identification information is found, the reporting request is determined not to be a processed request, the reporting request is executed, a second processing result is generated, and a reporting request processing record associated with the second identification information is created in the preset database, the second status marker is set to a processing marker, and after the reporting request processing is completed, the second status marker is updated to a completed marker and written into the second processing result.

[0107] The "waiting for the reported request to complete processing" can be understood as waiting for the reported request corresponding to the second identifier information, which is already being processed, to complete its processing and then obtaining its result, rather than re-executing the current duplicate request. If processing fails, a failure flag can be set or the processing record can be deleted, and retries can be allowed according to business needs. A timeout can be set for the processing wait, after which a specific status code will be returned. The second status flag is different from the first status flag: the first status flag indicates the C2A_RPC send confirmation status on the control side; the second status flag indicates the A2C_RPC processing status in the control side database; and the third status flag indicates the C2A_RPC processing status in the data side key-value cache.

[0108] In one possible implementation, after receiving a report request, the control plane first queries the database using an idempotent key. If a match is found, it means the request has been processed previously, and the processing result is returned directly. If no match is found, the request is processed normally, and the processing result is written to the database.

[0109] This application embodiment improves the accuracy and stability of A2C_RPC deduplication by setting a second status flag in the database record, including a completed flag and a processing flag, and processing the data according to three conditions: "completed," "processing," and "not found." This avoids duplicate reporting that leads to duplicate execution and also prevents concurrent duplicate reporting from triggering multiple processing steps simultaneously. Completed requests directly return historical results, processing requests wait for existing processing to complete and return results, and requests without a found record execute the reporting request and write it to the database.

[0110] Optionally, based on the above method, the control terminal further includes: scanning a preset database through a preset timed task to obtain reporting request processing records whose creation time exceeds a preset duration; and deleting reporting request processing records whose creation time exceeds a preset duration.

[0111] Optionally, the scheduled task can delete only completed reporting request processing records whose creation time exceeds a preset duration, or process records that are still being processed but have timed out separately to avoid accidentally deleting records that are still being processed. The preset duration can be set to be greater than or equal to the maximum recovery time for control terminal failover, such as 30 minutes. The scheduled task can scan the database every 30 minutes and delete records whose creation time exceeds 30 minutes. The creation time field can be written when the reporting request processing record is created.

[0112] It is understood that the preset timed tasks and preset durations can be determined according to the actual situation, and this application embodiment does not impose specific restrictions.

[0113] In one possible implementation, the timed expiration and recycling mechanism for the A2C_RPC cache in this embodiment is configured as follows: To prevent the reported records in the database from accumulating indefinitely, the control plane runs a scheduled task. This task scans the database every 30 minutes and deletes records whose creation time (created_at) exceeds 30 minutes. This time window ensures that the database does not grow indefinitely, while allowing sufficient time to cover the maximum duration of most control plane failovers.

[0114] This application embodiment prevents the database from growing indefinitely by using a pre-set scheduled task to scan a pre-set database and delete reporting request processing records whose creation time exceeds a pre-set duration. The pre-set duration can be set according to the longest recovery time for control terminal fault switching, thereby controlling the database storage size while retaining sufficient fault recovery window and audit records, and avoiding database bloat caused by long-term system operation.

[0115] Figure 3 A flowchart illustrating the deduplication method for requests provided in this application embodiment. Figure 2 ,like Figure 3 As shown, this application embodiment provides a method for deduplicating requests, and the execution subject of this method is the aforementioned data terminal 10. The method is described in detail below:

[0116] S301: Receive the event processing request and first identification information sent by the control terminal.

[0117] The first identification information is generated by the control terminal in response to the event processing request, based on the first request information corresponding to the event processing request. The first request information includes the event identifier, request type, object identifier, and request content.

[0118] Optionally, the data terminal can receive event processing requests and first identification information sent by the control terminal through the RPC service interface. The first identification information can be carried in the RPC message header or message body. The data terminal 10 can use the first identification information as the key of the key-value cache for subsequent idempotency deduplication.

[0119] S302: Determine whether the event handling request is a processed request based on the first identification information.

[0120] Optionally, determining whether an event handling request is a processed request based on the first identification information includes:

[0121] Based on the first identifier information, a query is performed in the preset key-value cache to determine whether the event handling request is a processed request.

[0122] The preset key-value cache stores event processing records with the first identifier information as the key and the third status flag and the first processing result as the value.

[0123] Optionally, the preset key-value cache can be a memory-based KV cache, which stores event processing records that may include: first identification information (key), third status flag, first processing result, creation time, etc. The third status flag may include a processing flag and a completed flag. The data terminal 10 can utilize the atomic operations, local locks, or single-threaded processing mechanisms of the key-value cache to ensure that records corresponding to the same first identification information are not created or executed concurrently.

[0124] This application embodiment provides a foundation for fast deduplication queries on the data end by setting a preset key-value cache and storing a third status flag and a first processing result using the first identifier information as the key. The key-value cache is suitable for high real-time delivery scenarios, and can quickly determine whether a request has been processed and obtain historical processing results, thereby improving the efficiency of C2A_RPC deduplication.

[0125] Optionally, based on the first identification information, a query is performed in a preset key-value cache to determine whether the event processing request is a processed request. This includes: querying the preset key-value cache for event processing records associated with the first identification information; if an event processing record is found and the third status marker is set to "completed", the event processing request is determined to be a processed request, and the first processing result in the event processing record is used as the first historical processing result; if an event processing record is found and the third status marker is set to "processing", the process waits for the event processing request corresponding to the event processing record to be processed until it is completed, and then uses the first processing result obtained from the execution as the first return result; if no event processing record associated with the first identification information is found, the event processing request is determined not to be a processed request, the event processing request is executed, the first processing result is generated, and an event processing record associated with the first identification information is created in the preset key-value cache, the third status marker is set to "processing", and after the event processing request is completed, the third status marker is updated to "completed" and written to the first processing result.

[0126] Optionally, a timeout period can be set while waiting for a request to complete. If the timeout occurs, a processing status code or error code can be returned, allowing the control unit to decide whether to retry. If processing fails, the third status flag can be set to a failure flag or the record can be deleted for subsequent retrying. The first returned result may include the first processing result, status flag, or error code. The data terminal can also receive a cache cleanup request sent by the control unit after the business event is completed, and delete the key-value cache record corresponding to the first identifier information according to the cache cleanup request.

[0127] This application embodiment achieves three-state deduplication on the data side by setting a third state flag, including a completed flag and a processing flag, in the key-value cache, and processing it according to three cases: "completed," "processing," and "not found." Completed requests return historical results, processing requests wait for existing processing to complete and return results, and if no record is found, an event processing request is executed and written to the cache. This simultaneously avoids duplicate requests and duplicate execution caused by concurrent duplicate requests, ensuring the idempotency of C2A_RPC.

[0128] S303: Send the first return result to the control terminal.

[0129] If the event handling request is determined to be a processed request, the first return result is the corresponding first historical processing result; if the event handling request is determined not to be a processed request, the first return result is the first processing result generated by the data end after executing the event handling request.

[0130] In some possible implementations, the data plane does not simply passively receive instructions; it also actively reports state changes to the control plane. For example, when a node state change is detected, the data plane needs to notify the control plane of this change so that the control plane can update the global view.

[0131] Specifically, the A2C_RPC idempotency key (a2c_idempotency_key) is calculated as follows: The data plane assigns a globally unique ID (rpc_global_id) to each locally reported request. This ID is managed by the data plane itself, ensuring that there are no duplicates within the same data plane node. Then, the idempotency key is calculated using the following formula:

[0132] a2c_idempotency_key=H(request_type||object_id||request_msg||rpc_global_id)

[0133] Here, in this embodiment, the data terminal receives the event processing request and first identification information sent by the control terminal, and determines whether the event processing request is a processed request based on the first identification information. If it is processed, the first historical processing result is returned; if it is not processed, the first processing result is returned after execution. This ensures idempotent execution of C2A_RPC from the data terminal side. During control terminal failover and retry, the data terminal can accurately identify duplicate requests, avoid repeated execution of business operations, and ensure the consistency of the stored state.

[0134] Optionally, the above method further includes: in response to a reporting request, obtaining corresponding second request information; wherein the second request information includes a request type, an object identifier, a request content, and a globally unique number; generating second identification information based on the second request information; sending the reporting request and the second identification information to the control terminal, so that the control terminal determines whether the reporting request is a processed request based on the second identification information; receiving a second return result sent by the control terminal; wherein if the control terminal determines that the reporting request is a processed request, the second return result is the corresponding second historical processing result; if the control terminal determines that the reporting request is not a processed request, the second return result is the second processing result generated after executing the reporting request.

[0135] Optionally, before generating the second identification information, the data end can first obtain the local node identifier and local incrementing sequence number, generate a globally unique number, and then concatenate the request type, object identifier, request content, and globally unique number and hash them to obtain the second identification information. The second identification information can be sent to the control end in an A2C_RPC message. The control end queries the database cache and returns the second result. If the database hits and the status is "completed," the second historical processing result is returned; if the hits and the status is "processing," the second processing result is returned after processing is complete; if the hits do not occur, a reporting request is executed and written to the database.

[0136] Here, by responding to the reporting request from the data end, the system obtains second request information, including the request type, object identifier, request content, and a globally unique number. It then generates second identification information and sends it to the control end. Finally, it receives the second return result from the control end, enabling idempotent processing of the data end's proactive reporting direction. The control end can use this information to identify duplicate reports and return historical or new processing results, preventing duplicate processing of reported content such as link changes, alarms, and statistical indicators.

[0137] Optionally, before responding to the reporting request and obtaining the corresponding second request information, the method further includes: obtaining the local node identifier and the local incrementing sequence number; generating a globally unique number based on the local node identifier and the local incrementing sequence number; wherein the globally unique number is used to uniquely identify the reporting request, and the globally unique number of different reporting requests is not repeated.

[0138] Different data endpoint nodes can be distinguished by their local node identifiers. Locally incrementing sequence numbers can be maintained within the data endpoint node and restored from persistent storage or otherwise guaranteed to be unique after a node restart. Globally unique identifiers are managed locally at the data endpoint and do not require coordination across data endpoint nodes.

[0139] Here, a globally unique number is generated by obtaining the local node identifier and the local incrementing sequence number. The data end can assign a unique number to the reporting request without cross-node coordination. After this globally unique number is incorporated into the second request information, the second identifier information can distinguish reporting requests of the same type but different instances, and the globally unique numbers of different reporting requests within the same data end node are not repeated, thereby improving the accuracy of A2C_RPC deduplication.

[0140] Figure 4 A flowchart illustrating the deduplication method for requests provided in this application embodiment. Figure 3 ,like Figure 4 As shown, this application provides a method for deduplicating requests, and the execution subject of this method is the aforementioned request deduplication system. The method is described in detail below:

[0141] S401: The control terminal responds to the event handling request and obtains the corresponding first request information.

[0142] The first request information includes the event identifier, request type, object identifier, and request content.

[0143] S402: The control terminal generates corresponding first identification information based on the first request information.

[0144] S403: The control terminal sends the first request information and the first identification information to the data terminal.

[0145] S404: The data terminal determines whether the event handling request is a processed request based on the first identification information.

[0146] S405: The data terminal sends the first return result to the control terminal.

[0147] If the data terminal determines that the event handling request is a processed request, the first return result is the corresponding first historical processing result; if the data terminal determines that the event handling request is not a processed request, the first return result is the first processing result generated by the data terminal after executing the event handling request.

[0148] It should be noted that, Figure 4 This describes the end-to-end interaction process between the control end and the data end. Optionally, the process may further include: the control end sending a cache cleanup request to the data end after processing a business event; the data end receiving the request and deleting the key-value cache record corresponding to the first identification information. The process may also include: the data end responding to the reporting request, generating second identification information and sending it to the control end; the control end querying the database cache based on the second identification information and returning a second result. The roles of the control end and the data end in bidirectional RPC can be interchanged: in C2A_RPC, the control end is the client and the data end is the server; in A2C_RPC, the data end is the client and the control end is the server.

[0149] This application embodiment achieves idempotent processing of the entire C2A_RPC process through end-to-end interaction, where the control terminal generates and sends first identification information, and the data terminal determines duplicates based on the first identification information and returns a first return result to the control terminal. In the event of control terminal failure switching, request retry, or response loss, the data terminal can return historical processing results or execute the event processing request only once, avoiding repeated execution and ensuring the consistency of the stored state.

[0150] This application also provides a complete system architecture for bidirectional RPC interaction between C2A_RPC and A2C_RPC, as well as a differentiated storage strategy.

[0151] Optionally, Figure 5 This is a schematic diagram of the component structure of a request deduplication system provided in an embodiment of this application. Figure 5 As shown, the deduplication system for this request includes a control plane side, a data plane side, and a database. The control plane side includes an event manager, a remote procedure call idempotency manager, a remote procedure call cache table from the data plane to the control plane, and scheduled cleanup tasks. The event manager is connected to the remote procedure call idempotency manager, and triggers the remote procedure call idempotency manager when an event completes. The remote procedure call idempotency manager sends a control plane-to-data plane remote procedure call request to the control plane-to-data plane remote procedure call processor on the data plane side. The data plane side includes a control plane-to-data plane remote procedure call processor, a control plane-to-data plane key-value cache, a global number manager, and a data plane-to-control plane remote procedure call reporter. The remote procedure call (RPC) processor from the control plane to the data plane is connected to the key-value cache from the control plane to the data plane, used to query or write processing status and results using the first identifier information as the key. The global number manager is connected to the remote procedure call (RPC) reporter from the data plane to the control plane, used to assign globally unique numbers to reporting requests. The remote procedure call reporter from the data plane to the control plane reports remote procedure call requests from the data plane to the control plane to the remote procedure call cache table on the control plane side. The remote procedure call cache table from the data plane to the control plane is connected to the database, used to query and write reporting request processing records. A scheduled cleanup task is connected to the database, used to periodically delete records in the database whose creation time exceeds a preset duration.

[0152] Optionally, the control plane includes an event manager, an RPC idempotency manager, an A2C_RPC cache table, and scheduled cleanup tasks. The event manager is responsible for scheduling and distributing events and maintaining the event processing state machine. The RPC idempotency manager is responsible for generating idempotency keys for C2A_RPC and maintaining the mapping between events and idempotency keys, triggering cache cleanup when an event ends. The A2C_RPC cache table stores request records reported from the data plane. The scheduled cleanup tasks are responsible for periodically cleaning up expired A2C_RPC cache records.

[0153] Optionally, the data plane side includes a C2A_RPC processor, a C2A_KV cache, a global number manager, and an A2C_RPC reporter. The C2A_RPC processor receives requests from the control plane, executes the processing logic, and interacts with the KV cache to perform idempotency checks. The C2A_KV cache stores the mapping between requests and processing results, using the idempotency key as the key. The global number manager assigns a unique number to the reporting requests of this node. The A2C_RPC reporter is responsible for proactively reporting state changes in the data plane to the control plane; state changes include, for example, changes in link connections between nodes.

[0154] Optionally, this embodiment employs differentiated storage strategies based on the different business characteristics of bidirectional RPC. For C2A_RPC, the direction is from the control plane to the data plane, the triggering method is event-driven, the storage medium is KV cache (memory), the real-time requirement is high, the cleanup mechanism is collaborative recycling, i.e., triggered upon event completion, and the idempotent key composition is event_id + request_type + object_id + request_msg. For A2C_RPC, the direction is from the data plane actively reporting to the control plane, the triggering method is data plane actively reporting, the storage medium is database (persistent), the real-time requirement is medium, the cleanup mechanism is timed expiration, such as 30 minutes, and the idempotent key composition is request_type + object_id + request_msg + rpc_global_id.

[0155] Optionally, C2A_RPC uses a key-value cache because RPC deployments are typically batch operations, associated with specific events, and require rapid responses. After the event is completed, the control plane proactively triggers cleanup, and the cache lifecycle is precisely bound to the business execution cycle, eliminating the need for persistence. A2C_RPC uses a database because RPC reporting may involve auditing requirements, necessitating persistent storage. During control plane failover, the data plane may repeatedly send the same reporting content, and database storage can withstand a longer fault recovery window. Scheduled cleanup tasks control the database size.

[0156] Through the above-mentioned division of responsibilities among modules and differentiated storage strategies, this embodiment can simultaneously support idempotent processing in both C2A_RPC and A2C_RPC directions, taking into account both real-time performance and persistence, and ensuring the consistency of storage status in scenarios such as control plane fault switching and repeated data plane reporting.

[0157] Optionally, Figure 6 This is a schematic diagram illustrating a C2A_RPC distribution process provided in an embodiment of this application. For example... Figure 6As shown, this interaction flow involves the control plane, data plane, and key-value cache. The control plane begins processing event E001, generating an RPC request R1 and an idempotent key K1, where the idempotent key K1 = hash(event identifier E001 || update operation || object identifier OBJ-001 || request content msg1). The control plane sends the RPC request R1 and idempotent key K1 to the data plane. The data plane queries the key-value cache with the idempotent key K1. If a match is found, a "processing in progress" record is created, the processing logic is executed, the processing result is written to the key-value cache, the status is updated to "completed," and the processing result is returned to the control plane. The control plane continues sending RPC requests R2 and idempotent key K2. The data plane queries the key-value cache with the idempotent key K2. If a match is found, the processing logic is executed, and the status is set to "processing in progress." At this point, a failover occurs in the control plane, and the new control plane resends the RPC request R1 and idempotent key K1. The data plane queries the key-value cache with the idempotent key K1. If a match is found and the status is "completed," the cached processing result is returned directly without duplicate processing. The new control plane resends the RPC request R2 and the idempotent key K2. The data plane queries the key-value cache with the idempotent key K2. If a match is found and the status is "processing," it waits for processing to complete. Once completed, the processing result is written, the status is updated to "completed," and the processing result is returned. After event E001 is processed, the control plane sends a cache cleanup request to the data plane. This request includes the event identifier E001 and the list of idempotent keys [K1, K2]. The data plane deletes the records corresponding to K1 and K2 in the key-value cache in batches according to the cache cleanup request.

[0158] Optionally, Figure 7 This is a schematic diagram of an A2C_RPC reporting interaction provided for an embodiment of this application. For example... Figure 7 As shown, this interaction process involves the data plane, control plane, and database. The data plane generates a locally unique global identifier G-001 and calculates the idempotent key K1 = hash(node ​​state || object identifier OBJ-001 || request content msg1 || globally unique identifier G-001). The data plane reports request R1 and idempotent key K1 to the control plane. The control plane queries the database using idempotent key K1. If no match is found, it executes the processing logic, writes the processing result to the database, and returns the processing result to the data plane. At this point, a failover occurs in the control plane, and the data plane resends the reported request R1 and idempotent key K1. The control plane queries the database using idempotent key K1. If a historical processing record is found, the cached processing result is returned directly without reprocessing. After 30 minutes, the control plane runs a scheduled task to delete records in the database created more than 30 minutes ago.

[0159] Understandable Figure 6 and Figure 7 R1, K1, E001, etc. are all applicable to the current graph, and can refer to different data in different graphs.

[0160] Figure 8 This is a schematic diagram of the deduplication processing device for requests provided in an embodiment of this application, applied to the control terminal. For example... Figure 8 As shown in the embodiments of this application, a request deduplication processing device is also provided, which includes: an acquisition module 801, a generation module 802, a first sending module 803, and a first receiving module 804.

[0161] The acquisition module is used to acquire the corresponding first request information in response to the event handling request; the first request information includes the event identifier, request type, object identifier and request content.

[0162] The generation module is used to generate corresponding first identifier information based on the first request information;

[0163] The first sending module is used to send the first request information and the first identification information to the data terminal, so that the data terminal can determine whether the event processing request is a processed request based on the first identification information.

[0164] The first receiving module is used to receive the first return result sent by the data terminal; wherein, if the data terminal determines that the event processing request is a processed request, the first return result is the corresponding first historical processing result; if the data terminal determines that the event processing request is not a processed request, the first return result is the first processing result generated by the data terminal after executing the event processing request.

[0165] Optionally, the generation module is specifically used for:

[0166] The event identifier, request type, object identifier, and request content are concatenated to obtain the first string;

[0167] Perform a hash operation on the first string to obtain the first identifier information.

[0168] Optionally, after the first sending module sends the first request information and the first identification information to the data terminal, the above apparatus further includes an update module, used for:

[0169] Create a first status flag corresponding to the event handling request; wherein, the first status flag is used to identify the sending status of the event handling request; the first status flag includes a pending confirmation status and a completed status;

[0170] In response to the first returned result, the first status flag is updated from pending confirmation to completed.

[0171] Optionally, after the first receiving module receives the first return result sent by the data terminal, the above apparatus further includes a cleaning module for:

[0172] Generate a cache cleanup request; the cache cleanup request carries an event identifier and / or first identifier information;

[0173] A cache cleanup request is sent to the data end so that the data end can delete the first identification information and / or the cache record data corresponding to the first identification information according to the cache cleanup request.

[0174] Optionally, the above-mentioned device further includes a first processing module, configured to: receive a reporting request and second identification information sent by a data terminal; wherein the second identification information is generated by the data terminal in response to the reporting request based on the second request information corresponding to the reporting request, and the second request information includes a request type, an object identifier, a request content, and a globally unique number; determine whether the reporting request is a processed request based on the second identification information; and send a second return result to the data terminal; wherein if it is determined that the reporting request is a processed request, the second return result is the corresponding second historical processing result, and if it is determined that the reporting request is not a processed request, the second return result is the second processing result generated after executing the reporting request.

[0175] Optionally, the first processing module is specifically used to: query a preset database based on the second identification information to determine whether the reporting request is a processed request; wherein, the preset database stores multiple historical reporting request processing records corresponding to reporting requests, and the reporting request processing records are associated with the second identification information.

[0176] Optionally, the reporting request processing record includes a second status marker corresponding to historical reporting requests; the second status marker includes a completed marker and a processing marker; the first processing module is specifically used to: query the reporting request processing record associated with the second identification information in the preset database; if a reporting request processing record is found and the second status marker is a completed marker, then the reporting request is determined to be a processed request, and the second historical processing result in the reporting request processing record is used as the second return result; if a reporting request processing record is found and the second status marker is a processing marker, then after the reporting request processing is completed, the second processing result obtained is used as the second return result; if no reporting request processing record associated with the second identification information is found, then the reporting request is determined not to be a processed request, the reporting request is executed, a second processing result is generated, and a reporting request processing record associated with the second identification information is created in the preset database, the second status marker is set to a processing marker, and after the reporting request processing is completed, the second status marker is updated to a completed marker and written to the second processing result.

[0177] Optionally, the above device further includes a deletion module, used to: scan a preset database through a preset timed task to obtain reporting request processing records whose creation time exceeds a preset duration; and delete reporting request processing records whose creation time exceeds the preset duration.

[0178] This application also provides a request deduplication processing device applied at a data terminal. The device includes: a second receiving module, a second processing module, and a second sending module.

[0179] The second receiving module is used to receive an event processing request and first identification information sent by the control terminal; wherein, the first identification information is generated by the control terminal in response to the event processing request based on the first request information corresponding to the event processing request, and the first request information includes an event identifier, a request type, an object identifier, and a request content;

[0180] The second processing module is used to determine whether the event processing request is a processed request based on the first identification information;

[0181] The second sending module is used to send a first return result to the control terminal; wherein, if it is determined that the event processing request is a processed request, the first return result is the corresponding first historical processing result; if it is determined that the event processing request is not a processed request, the first return result is the first processing result generated by the data terminal after executing the event processing request.

[0182] Optionally, the second processing module is specifically used to: query a preset key-value cache based on the first identification information to determine whether the event processing request is a processed request; wherein the preset key-value cache stores event processing records with the first identification information as the key and the third status flag and the first processing result as the value.

[0183] Optionally, the second processing module is specifically used to: query the event processing record associated with the first identification information in the preset key-value cache; if an event processing record is found and the third status mark is marked as completed, then the event processing request is determined to be a processed request, and the first processing result in the event processing record is used as the first historical processing result; if an event processing record is found and the third status mark is marked as processing, then after the event processing request corresponding to the event processing record is processed, the first processing result obtained is used as the first return result; if no event processing record associated with the first identification information is found, then the event processing request is determined not to be a processed request, the event processing request is executed, the first processing result is generated, and an event processing record associated with the first identification information is created in the preset key-value cache, the third status mark is set to processing, and after the event processing request is processed, the third status mark is updated to completed and written to the first processing result.

[0184] Optionally, the above-mentioned device further includes a third processing module, configured to: in response to a reporting request, obtain corresponding second request information; wherein the second request information includes a request type, an object identifier, a request content, and a globally unique number; generate second identification information based on the second request information; send the reporting request and the second identification information to a control terminal, so that the control terminal determines whether the reporting request is a processed request based on the second identification information; and receive a second return result sent by the control terminal; wherein if the control terminal determines that the reporting request is a processed request, the second return result is the corresponding second historical processing result; if the control terminal determines that the reporting request is not a processed request, the second return result is the second processing result generated after executing the reporting request.

[0185] Optionally, before the third processing module is used to respond to the reporting request and obtain the corresponding second request information, it is also used to: obtain the local node identifier and the local incrementing sequence number; generate a globally unique number based on the local node identifier and the local incrementing sequence number; wherein, the globally unique number is used to uniquely identify the reporting request, and the globally unique number of different reporting requests is not repeated.

[0186] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 9 As shown, the electronic device 90 provided in this embodiment includes at least one processor 901 and a memory 902. In one possible implementation, the electronic device 90 further includes a communication component 903. The processor 901, memory 902, and communication component 903 are connected via a bus.

[0187] In a specific implementation, at least one processor 901 executes computer execution instructions stored in memory 902, causing at least one processor 901 to execute the above-described deduplication processing embodiment of the request.

[0188] The specific implementation process of processor 901 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0189] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0190] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0191] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings of this application's embodiments are not limited to only one bus or one type of bus.

[0192] An embodiment of this application also provides a computer-readable storage medium storing a computer program configured to execute the steps in the deduplication method embodiment of any of the above requests when running.

[0193] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0194] An embodiment of this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps in the deduplication method embodiment for any of the above requests.

[0195] The embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in the deduplication processing method embodiments of any of the above requests.

[0196] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of the embodiments of this application.

[0197] The foregoing has provided a detailed description of a method for deduplicating requests and an electronic device according to embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of the embodiments of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the methods and core ideas of the embodiments of this application. It should be noted that those skilled in the art can make several improvements and modifications to the embodiments of this application without departing from the principles of the embodiments of this application, and these improvements and modifications also fall within the protection scope of the embodiments of this application.

Claims

1. A method for deduplicating requests, characterized in that, include: In response to an event handling request, obtain the corresponding first request information; wherein, the first request information includes an event identifier, a request type, an object identifier, and a request content; Generate corresponding first identification information based on the first request information; The first request information and the first identification information are sent to the data terminal so that the data terminal can determine whether the event processing request is a processed request based on the first identification information. The system receives a first return result sent by the data terminal; wherein, if the data terminal determines that the event processing request is a processed request, the first return result is the corresponding first historical processing result; if the data terminal determines that the event processing request is not a processed request, the first return result is the first processing result generated by the data terminal after executing the event processing request.

2. The method according to claim 1, characterized in that, The step of generating corresponding first identifier information based on the first request information includes: The event identifier, the request type, the object identifier, and the request content are concatenated to obtain a first string; Perform a hash operation on the first string to obtain the first identifier information.

3. The method according to claim 1, characterized in that, After sending the first request information and the first identification information to the data terminal, the method further includes: Create a first status flag corresponding to the event processing request; wherein, the first status flag is used to identify the sending status of the event processing request; the first status flag includes a pending confirmation status and a completed status; In response to the first returned result, the first status flag is updated from the pending confirmation status to the completed status.

4. The method according to any one of claims 1 to 3, characterized in that, After the first return result sent by the receiving end, the method further includes: Generate a cache cleanup request; wherein the cache cleanup request carries the event identifier and / or the first identifier information; The cache cleanup request is sent to the data terminal so that the data terminal deletes the first identification information and / or the cache record data corresponding to the first identification information according to the cache cleanup request.

5. The method according to any one of claims 1 to 3, characterized in that, Also includes: The system receives a reporting request and second identification information sent by the data terminal; wherein the second identification information is generated by the data terminal in response to the reporting request based on the second request information corresponding to the reporting request, and the second request information includes a request type, an object identifier, a request content, and a globally unique number. Based on the second identification information, determine whether the reporting request is a processed request; Send a second return result to the data terminal; wherein, if it is determined that the reporting request is a processed request, the second return result is the corresponding second historical processing result; if it is determined that the reporting request is not a processed request, the second return result is the second processing result generated after executing the reporting request.

6. The method according to claim 5, characterized in that, The step of determining whether the reported request is a processed request based on the second identification information includes: Based on the second identification information, a query is performed in a preset database to determine whether the reporting request is a processed request; wherein, the preset database stores multiple historical reporting request processing records corresponding to reporting requests, and the reporting request processing records are associated with the second identification information.

7. The method according to claim 6, characterized in that, The reporting request processing record includes a second status flag corresponding to the historical reporting request; the second status flag includes a completed flag and a processing flag. The step of querying a preset database based on the second identification information to determine whether the reported request is a processed request includes: Query the preset database for the reporting request processing record associated with the second identification information; If the reporting request processing record is found and the second status mark is the completed mark, then the reporting request is determined to be a processed request, and the second historical processing result in the reporting request processing record is used as the second return result; If the reporting request processing record is found and the second status mark is the "processing" mark, then wait for the reporting request processing to be completed, and use the second processing result obtained from the execution as the second return result. If no reporting request processing record associated with the second identification information is found, it is determined that the reporting request is not a processed request. The reporting request is executed, a second processing result is generated, and a reporting request processing record associated with the second identification information is created in the preset database. The second status flag is set to the processing flag. After the reporting request processing is completed, the second status flag is updated to the completed flag and written into the second processing result.

8. The method according to claim 6 or 7, characterized in that, Also includes: By using a pre-set scheduled task, the pre-set database is scanned to obtain reporting request processing records whose creation time exceeds a pre-set duration; Delete the reporting request processing record whose creation time exceeds the preset time.

9. A method for deduplicating requests, characterized in that, include: The system receives an event processing request and first identification information sent by the control terminal; wherein the first identification information is generated by the control terminal in response to the event processing request based on the first request information corresponding to the event processing request, and the first request information includes an event identifier, a request type, an object identifier, and a request content; Based on the first identification information, determine whether the event handling request is a processed request; Send a first return result to the control terminal; wherein, if it is determined that the event processing request is a processed request, the first return result is the corresponding first historical processing result; if it is determined that the event processing request is not a processed request, the first return result is the first processing result generated by the data terminal after executing the event processing request.

10. The method according to claim 9, characterized in that, The step of determining whether the event handling request is a processed request based on the first identification information includes: Based on the first identification information, a query is performed in a preset key-value cache to determine whether the event processing request is a processed request; wherein, the preset key-value cache stores event processing records with the first identification information as the key, the third status flag and the first processing result as the value.

11. The method according to claim 10, characterized in that, The step of querying a preset key-value cache based on the first identifier information to determine whether the event handling request is a processed request includes: Query the event processing record associated with the first identification information in the preset key-value cache; If the event processing record is found and the third status is marked as completed, then the event processing request is determined to be a processed request, and the first processing result in the event processing record is taken as the first historical processing result. If the event processing record is found and the third status is marked as being in process, then after the event processing request corresponding to the event processing record is completed, the first processing result obtained will be used as the first return result. If no event processing record associated with the first identification information is found, the event processing request is determined to be a processed request. The event processing request is executed, a first processing result is generated, and an event processing record associated with the first identification information is created in the preset key-value cache. The third status flag is set to a processing flag. After the event processing request is completed, the third status flag is updated to a completed flag and written to the first processing result.

12. The method according to any one of claims 9 to 11, characterized in that, Also includes: In response to the reporting request, obtain the corresponding second request information; wherein, the second request information includes the request type, object identifier, request content and globally unique number; Generate second identification information based on the second request information; The reporting request and the second identification information are sent to the control terminal so that the control terminal can determine whether the reporting request is a processed request based on the second identification information. The control terminal receives a second return result; wherein, if the control terminal determines that the reporting request is a processed request, the second return result is the corresponding second historical processing result; if the control terminal determines that the reporting request is not a processed request, the second return result is the second processing result generated after executing the reporting request.

13. The method according to claim 12, characterized in that, Before responding to the reporting request and obtaining the corresponding second request information, the method further includes: Obtain the local node identifier and the local incrementing sequence number; The globally unique number is generated based on the local node identifier and the local incrementing sequence number; wherein, the globally unique number is used to uniquely identify the reporting request, and the globally unique number of different reporting requests is not repeated.

14. A method for deduplicating requests, characterized in that, include: The control terminal responds to the event handling request and obtains the corresponding first request information; wherein, the first request information includes event identifier, request type, object identifier and request content; The control terminal generates corresponding first identification information based on the first request information; The control terminal sends the first request information and the first identification information to the data terminal; The data terminal determines whether the event processing request is a processed request based on the first identification information; The data terminal sends a first return result to the control terminal; wherein, if the data terminal determines that the event processing request is a processed request, the first return result is the corresponding first historical processing result; if the data terminal determines that the event processing request is not a processed request, the first return result is the first processing result generated by the data terminal after executing the event processing request.

15. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the deduplication method as claimed in any one of claims 1 to 8, or any one of claims 9 to 13, or as claimed in claim 14.

Images