Idempotent processing method and device, electronic equipment and medium
By acquiring and detecting the execution time and server status in the idempotent record table in a distributed system, and extending or deleting idempotent records, the problem of idempotent records being processed in extreme cases is solved, and efficient and accurate idempotent processing is achieved and data consistency.
Patent Information
- Application Number
- CN202410588059.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-13
- Publication Date
- 2024-11-08
AI Technical Summary
In distributed systems, in extreme cases, such as server downtime or network oscillation, the state of idempotent records may always be processed, resulting in the inability to successfully execute business requests, and the existing technology requires low manual processing efficiency.
By obtaining records in the idempotent record table of the distributed system whose execution time exceeds the threshold, detecting the idempotent execution table of the server, judging the thread's survival status, and extending or deleting records according to the renewal rules, ensuring the normal execution of idempotent processing.
It realizes efficient and accurate processing of idempotent requests in a distributed system, avoids business process failures caused by repeated requests, and improves the idempotence and data consistency of the system.
Smart Images

Figure CN118916363A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to computer technology, and in particular to an idempotent processing method, device, electronic device and medium. Background Art
[0002] Data consistency refers to maintaining the correctness and consistency of data among multiple nodes in a distributed system. In a distributed system, since multiple nodes can access and modify data at the same time, ensuring data consistency, that is, achieving idempotence of the system, is an important challenge.
[0003] In the prior art, idempotency records can be generated for received business requests, and by checking whether there are identical idempotency records, it can be determined whether the current business request is repeated. However, in some extreme cases such as server downtime and network shock, the status of individual idempotency records is always in processing, and the response received by the requester after sending repeated requests is "concurrent request, please try again later", so that the business request can never be successfully executed. At this time, data can only be processed manually, but there is a problem of low efficiency. Summary of the invention
[0004] The present application provides an idempotent processing method, device, electronic device and medium for performing idempotent processing efficiently and accurately.
[0005] On the one hand, the present application provides an idempotent processing method, which is applied to a server of a distributed system, wherein the distributed system includes multiple servers; the method includes:
[0006] Obtaining a first idempotent record in the idempotent record table of the distributed system whose execution time exceeds a corresponding overtime threshold; wherein the idempotent record table of the distributed system is used to record the service requests received by the distributed system, and the idempotent record includes a server identifier and a thread identifier;
[0007] Detecting whether the first idempotent record exists in the idempotent execution table of the server; wherein the idempotent execution table of the server is used to record the idempotent record being executed by the server;
[0008] If the first idempotent record exists, judging whether the thread corresponding to the first idempotent record is alive according to the thread identifier in the first idempotent record; if alive, extending the overtime threshold corresponding to the first idempotent record according to the preset renewal rule; if not alive, deleting the first idempotent record in the idempotent record table;
[0009] According to the idempotent record table, the received business requests are processed idempotently.
[0010] Optionally, the method further comprises:
[0011] If the first idempotent record does not exist in the idempotent execution tables of the multiple servers, the first idempotent record in the idempotent record table is deleted.
[0012] Optionally, the method further includes:
[0013] Archiving processing is performed regularly, and the archiving processing includes: obtaining a distributed lock. If the acquisition fails, the archiving processing is terminated; if the acquisition is successful, a second idempotent record is obtained from the idempotent record table, the status of the second idempotent record is successful execution, and the time of successful execution is more than a preset threshold from the current time; the second idempotent record is inserted into the archiving table, and the second idempotent record in the idempotent record table is deleted.
[0014] Optionally, performing idempotent processing on the received service request according to the idempotent record table includes:
[0015] Receive a business request, construct a third idempotent record according to the business request, and set the state of the third idempotent record to be in execution; wherein the idempotent record includes: request parameters and state of the business request, and a server identifier, thread identifier, and time of receiving the business request; wherein the request parameter is a unique key of the idempotent record;
[0016] Determine whether there is a fourth idempotent record in the idempotent record table, and the unique key of the fourth idempotent record is the same as the unique key of the third idempotent record; if not, write the third idempotent record into the idempotent record table, execute the business process according to the business request, and return the execution result; and update the status of the third idempotent record in the idempotent record table to successful execution, and add the execution result to the third idempotent record.
[0017] Optionally, the method further includes:
[0018] If an exception occurs when executing the business process, the third idempotent record in the idempotent record table is deleted.
[0019] Optionally, the method further includes:
[0020] If the fourth idempotent record exists in the idempotent record table, query the status of the fourth idempotent record;
[0021] If the status of the fourth idempotent record is successful execution, the execution result corresponding to the fourth idempotent record is obtained from the idempotent record table and returned;
[0022] If the status of the fourth idempotent record is being executed, the first information is returned, where the first information is used to prompt that the same business request is being processed.
[0023] On the other hand, the present application provides an idempotent processing device, which is applied to a server of a distributed system, wherein the distributed system includes a plurality of servers; the device includes:
[0024] An acquisition module is used to acquire a first idempotent record whose execution time exceeds a corresponding overtime threshold in the idempotent record table of the distributed system; wherein the idempotent record table of the distributed system is used to record the service requests received by the distributed system, and the idempotent record includes a server identifier and a thread identifier;
[0025] A detection module, used to detect whether the first idempotent record exists in the idempotent execution table of the server; wherein the idempotent execution table of the server is used to record the idempotent record being executed by the server;
[0026] a renewal module, configured to, if the first idempotent record exists, determine whether the thread corresponding to the first idempotent record is alive according to the thread identifier in the first idempotent record; if alive, extend the overtime threshold corresponding to the first idempotent record according to a preset renewal rule; if not alive, delete the first idempotent record in the idempotent record table;
[0027] The processing module is used to perform idempotent processing on the received business request according to the idempotent record table.
[0028] Optionally, the device further comprises:
[0029] The deleting module is used to delete the first idempotent record in the idempotent record table if the first idempotent record does not exist in the idempotent execution tables of the multiple servers.
[0030] Optionally, the device further comprises:
[0031] The archiving module is used to perform archiving processing regularly, and the archiving processing includes: obtaining a distributed lock. If the acquisition fails, the archiving processing is terminated; if the acquisition is successful, a second idempotent record is obtained from the idempotent record table, and the status of the second idempotent record is successful execution, and the time of successful execution is more than a preset threshold from the current time; the second idempotent record is inserted into the archiving table, and the second idempotent record in the idempotent record table is deleted.
[0032] Optionally, the processing module is specifically used to:
[0033] Receive a business request, construct a third idempotent record according to the business request, and set the state of the third idempotent record to be in execution; wherein the idempotent record includes: request parameters and state of the business request, and a server identifier, thread identifier, and time of receiving the business request; wherein the request parameter is a unique key of the idempotent record;
[0034] Determine whether there is a fourth idempotent record in the idempotent record table, and the unique key of the fourth idempotent record is the same as the unique key of the third idempotent record; if not, write the third idempotent record into the idempotent record table, execute the business process according to the business request, and return the execution result; and update the status of the third idempotent record in the idempotent record table to successful execution, and add the execution result to the third idempotent record.
[0035] Optionally, the processing module is further used to:
[0036] If an exception occurs when executing the business process, the third idempotent record in the idempotent record table is deleted.
[0037] Optionally, the processing module is further used to:
[0038] If the fourth idempotent record exists in the idempotent record table, query the status of the fourth idempotent record;
[0039] If the status of the fourth idempotent record is successful execution, the execution result corresponding to the fourth idempotent record is obtained from the idempotent record table and returned;
[0040] If the status of the fourth idempotent record is being executed, the first information is returned, where the first information is used to prompt that the same business request is being processed.
[0041] On the other hand, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method described above.
[0042] On the other hand, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method as described above.
[0043] In the idempotent processing method, device, electronic device and medium provided by the present application, a server of a distributed system obtains a first idempotent record whose execution time exceeds a corresponding threshold in an idempotent record table of the distributed system, and detects whether the first idempotent record exists in the idempotent execution table of the server, so as to accurately determine whether the server is a server that receives a business request corresponding to the first idempotent record, and when the server is a server that receives a business request corresponding to the first idempotent record, whether the server is executing the business request corresponding to the first idempotent record; if the first idempotent record exists, then according to the thread identifier in the first idempotent record, it is determined whether the thread corresponding to the first idempotent record is alive, and if it is alive, the corresponding threshold of the first idempotent record is extended, so that when the business request corresponding to the first idempotent record is still executed normally, the first idempotent record is no longer in an overdue state; if it does not exist, then the first idempotent record in the sealed record table is deleted, so that when the distributed system receives a repeated request corresponding to the first idempotent record, it is avoided that the corresponding business process cannot be executed normally due to the existence of the first idempotent record in the idempotent record table, so that the idempotent record table can be effectively maintained, so as to efficiently and accurately perform idempotent processing on the received business requests. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0045] Figure 1 The schematic diagram of the architecture of the distributed system provided in the first embodiment of the present application is exemplarily shown in FIG.
[0046] Figure 2 The flowchart of the idempotent processing method provided in the first embodiment of the present application is exemplarily shown in FIG.
[0047] Figure 3 The flowchart of idempotent processing of a service request provided in the first embodiment of the present application is exemplarily shown in FIG.
[0048] Figure 4 The flowchart of the idempotent record table archiving process provided by the first embodiment of the present application is shown in the figure;
[0049] Figure 5 The structural diagram of the idempotent processing device provided in the second embodiment of the present application is exemplarily shown in FIG.
[0050] Figure 6 The structural diagram of the idempotent processing electronic device provided in Example 3 of the present application is exemplified in FIG.
[0051] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION
[0052] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.
[0053] The module in this application refers to a functional module or a logical module. It can be in software form, and its functions are realized by executing program code by a processor; it can also be in hardware form. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects before and after are in an "or" relationship.
[0054] Data consistency refers to maintaining the correctness and consistency of data among multiple nodes in a distributed system. In a distributed system, since multiple nodes can access and modify data at the same time, ensuring data consistency, that is, achieving idempotence of the system, is an important challenge.
[0055] In the prior art, idempotency records can be generated for received business requests, and by checking whether there are identical idempotency records, it can be determined whether the current business request is repeated. However, in some extreme cases such as server downtime and network shock, the status of individual idempotency records is always in processing, and the response received by the requester after sending repeated requests is "concurrent request, please try again later", so that the business request can never be successfully executed. At this time, data can only be processed manually, but there is a problem of low efficiency.
[0056] The idempotent processing method, device, electronic device and medium provided in this application are intended to solve the above-mentioned technical problems.
[0057] In the idempotent processing method, device, electronic device and medium provided by the present application, a server of a distributed system obtains a first idempotent record whose execution time exceeds a corresponding threshold in an idempotent record table of the distributed system, and detects whether the first idempotent record exists in the idempotent execution table of the server, so as to accurately determine whether the server is a server that receives a business request corresponding to the first idempotent record, and when the server is a server that receives a business request corresponding to the first idempotent record, whether the server is executing the business request corresponding to the first idempotent record; if the first idempotent record exists, then according to the thread identifier in the first idempotent record, it is determined whether the thread corresponding to the first idempotent record is alive, and if it is alive, the corresponding threshold of the first idempotent record is extended, so that when the business request corresponding to the first idempotent record is still executed normally, the first idempotent record is no longer in an overdue state; if it does not exist, then the first idempotent record in the sealed record table is deleted, so that when the distributed system receives a repeated request corresponding to the first idempotent record, it is avoided that the corresponding business process cannot be executed normally due to the existence of the first idempotent record in the idempotent record table, so that the idempotent record table can be effectively maintained, so as to efficiently and accurately perform idempotent processing on the received business requests.
[0058] The technical solution of the present application is illustrated below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
[0059] Embodiment 1
[0060] For ease of understanding, the following Figure 1 , the application scenarios involved in the embodiments of the present application are described.
[0061] Figure 1 A schematic diagram of the structure of a distributed system provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the distributed system includes multiple machines, and an idempotent execution table is set in the memory of each machine.
[0062] The embodiments of the present application do not limit the specific type of the machine. For example, the machine may be a server, a desktop computer, a tablet computer, or an artificial intelligence device. The server may be, for example, a graphics processing unit (GPU) server or an artificial intelligence (AI) server.
[0063] It should be noted that the number of computing devices may be two or more. Figure 1 The example of two computing devices is used only for illustration and does not constitute a limitation on the technical solution provided in the embodiment of the present application.
[0064] Exemplarily, the idempotent execution table may be in the form of a hash table.
[0065] Figure 2 The idempotent processing method provided in this embodiment is applied to a server of a distributed system, and the distributed system includes multiple servers, such as Figure 2 As shown, the method may include:
[0066] S201, obtaining the first idempotent record whose execution time exceeds the corresponding overtime threshold in the idempotent record table of the distributed system; wherein the idempotent record table of the distributed system is used to record the business requests received by the distributed system, and the idempotent record includes a server identifier and a thread identifier.
[0067] Specifically, the distributed system database is provided with an idempotent record table of the distributed system, which is used to record the service requests received by the distributed system. Based on the idempotent record table, the distributed system can determine whether the received service request is a repeated service request.
[0068] Exemplarily, the server may run a monitor thread to periodically obtain the first idempotent record whose execution time exceeds a corresponding timeout threshold in the idempotent record table of the distributed system.
[0069] Exemplarily, the server identifier is used to uniquely identify a server that receives a service request corresponding to the idempotent record, and the thread identifier is used to uniquely identify a thread on the server that receives a service request corresponding to the idempotent record.
[0070] S202, detecting whether there is a first idempotent record in the idempotent execution table of the server; wherein the idempotent execution table of the server is used to record the idempotent records being executed by the server.
[0071] Specifically, detecting whether there is a pending idempotent record belonging to the first idempotent record in the idempotent execution table of the server may include: judging whether the server identifier corresponds to the current server based on the server identifier in the first idempotent record; if the server identifier corresponds to the current server, then detecting whether the first idempotent record exists in the idempotent execution table of the current server based on the unique key of the first idempotent record; if the server identifier does not correspond to the current server, then determining that the first idempotent record does not exist in the idempotent execution table of the current server.
[0072] Exemplarily, the idempotent record may also include key business request parameters. For example, for a shopping scenario, the key business parameter may be a payment order number. The unique key of the idempotent record may be the key business request parameter.
[0073] S203, if there is a first idempotent record, determine whether the thread corresponding to the pending idempotent record is alive according to the thread identifier in the first idempotent record; if alive, extend the overtime threshold corresponding to the pending idempotent record according to the preset renewal rule; if not alive, delete the first idempotent record in the idempotent record table.
[0074] Specifically, if there is a first idempotent record in the idempotent execution table of the current server, then according to the thread identifier in the first idempotent record, it is determined whether the corresponding thread in the current server is alive. If alive, the server extends the overtime threshold corresponding to the first idempotent record according to the preset renewal rule, and can also postpone the execution start time corresponding to the first idempotent record; if not alive, the first idempotent record in the idempotent record table is deleted, thereby clearing the idempotent records with abnormal execution in the idempotent record table, and ensuring that the distributed system can re-execute the same business request. At the same time, the server that receives the business request corresponding to the first idempotent record processes the first idempotent record, which can effectively avoid parallel errors in the distributed system.
[0075] Exemplarily, the idempotent record renewal configuration may be read from the execution configuration file, and the preset renewal rule may be obtained from the idempotent record renewal configuration.
[0076] S204: Perform idempotent processing on the received service request according to the idempotent record table.
[0077] Specifically, after receiving a service request, the server determines whether the distributed system has received the same service request based on the idempotent record table and performs idempotent processing, so that the distributed system can return the same result for repeated service requests and achieve data consistency.
[0078] In this example, the server of the distributed system obtains the first idempotent record whose execution time exceeds the corresponding threshold in the idempotent record table of the distributed system, and detects whether the first idempotent record exists in the idempotent execution table of the server, so as to accurately determine whether the server is a server that receives the business request corresponding to the first idempotent record, and whether the server is executing the business request corresponding to the first idempotent record when the server is a server that receives the business request corresponding to the first idempotent record; if the first idempotent record exists, then according to the thread identifier in the first idempotent record, it is determined whether the thread corresponding to the first idempotent record is alive, and if it is alive, the corresponding threshold of the first idempotent record is extended, so that when the business request corresponding to the first idempotent record is still executed normally, the first idempotent record is no longer in an overdue state; if it does not exist, then the first idempotent record in the sealed record table is deleted, so that when the distributed system receives a repeated request corresponding to the first idempotent record, it is avoided that the corresponding business process cannot be executed normally due to the existence of the first idempotent record in the idempotent record table, so that the idempotent record table can be effectively maintained, so as to efficiently and accurately perform idempotent processing on the received business requests.
[0079] The above example is a case where there is a first idempotent record in the idempotent execution table of the server. In one example, the method further includes:
[0080] If the first idempotent record does not exist in the idempotent execution tables of multiple servers, the first idempotent record in the idempotent record table is deleted.
[0081] Specifically, the first idempotent record does not exist in the idempotent execution tables of multiple servers, and at least the following two situations exist:
[0082] Case 1: A server corresponding to the server identifier in the first idempotent record among multiple servers, and the first idempotent record does not exist in the idempotent execution table of the server.
[0083] In this case, due to server crash, restart or network oscillation, the first idempotent record does not exist in the idempotent execution table of the server that receives the business request corresponding to the first idempotent record. The server is not currently executing the business request corresponding to the first idempotent record. The first idempotent record in the idempotent record table is deleted so that the distributed system can execute the business processing flow normally when it receives the repeated business request corresponding to the first idempotent record again.
[0084] Case 2: A server corresponding to a server identifier that does not exist in the first idempotent record among multiple servers.
[0085] In this case, due to reasons such as the scaling down of the distributed system or the server going offline, the identifier of the server that receives the service request corresponding to the first idempotent record will not appear in the distributed system.
[0086] After determining that a server corresponding to a server identifier in the first idempotent record does not exist in the multiple servers in the distributed system, the distributed system issues a cleanup task, and the server assigned to the cleanup task in the multiple servers cleans up the first idempotent record in the idempotent record table. The cleanup process may include: sending an alarm email to a management personnel so that the management personnel manually processes the corresponding data; or deleting the first idempotent record in the idempotent record table.
[0087] In this example, when the first idempotent record does not exist in the idempotent execution tables of multiple servers, the first idempotent record in the idempotent record table is deleted, so that the idempotent records corresponding to the business requests that have not been successfully executed can be accurately cleaned up, so that when the requester retries the request, the distributed system can execute the corresponding business process normally, avoiding the requester receiving the response of "concurrent request execution, please try again later" every time it retries.
[0088] In actual applications, after receiving a service request, the server will determine whether the distributed system has received repeated requests based on the idempotent records in the idempotent record table, and then perform idempotent processing. In one example, the method also includes:
[0089] Receive a business request, construct a third idempotent record according to the business request, and set the status of the third idempotent record to be in execution; wherein the idempotent record includes: request parameters and status of the business request, and a server identifier, thread identifier, and time of receiving the business request; wherein the request parameter is a unique key of the idempotent record;
[0090] Determine whether there is a fourth idempotent record in the idempotent record table, and the unique key of the fourth idempotent record is the same as the unique key of the third idempotent record; if not, write the third idempotent record into the idempotent record table, execute the business process according to the business request, and return the execution result; and update the status of the third idempotent record in the idempotent record table to successful execution, and add the execution result to the third idempotent record.
[0091] Specifically, Figure 3 A flowchart of idempotent processing of a business request provided in an embodiment of the present application is shown in FIG. Figure 3As shown, the idempotent record includes the server ID, thread ID, time, and request parameters and status of the business request that receives the business request, wherein the status includes being executed and being executed successfully, and the request parameter is the unique key of the idempotent record. After the server receives the business request sent by the requester, it constructs a third idempotent record according to the business request, and sets the status of the third idempotent record to being executed. According to the unique key of the third idempotent record, it is determined whether there is a fourth idempotent record with the same unique key in the idempotent record table. If not, it indicates that there is no duplicate request. The third idempotent record is written into the idempotent record table, and the corresponding business process is executed according to the business request. After the execution is successful, the execution result is returned to the requester, and the status of the third idempotent record in the idempotent record table is updated to be executed successfully, and the execution result is written. Therefore, when the distributed system receives the duplicate request corresponding to the third idempotent record next time, the execution result can be conveniently returned according to the idempotent record table, and the idempotence of the system is realized.
[0092] In this example, a third idempotent record is constructed according to the received business request, the status of the third idempotent record is set to being executed, and it is determined whether there is a fourth idempotent record with the same request parameters in the idempotent record table. If the fourth idempotent record does not exist, the third idempotent record is written into the idempotent record table, and the business process is executed according to the business request. The execution result is returned, and the status of the third idempotent record in the idempotent record table is updated to successful execution. The execution result is added to the third idempotent record, which can accurately implement idempotent processing.
[0093] In one example, the method further includes:
[0094] If an exception occurs when executing the business process, the third idempotent record in the idempotent record table is deleted.
[0095] Specifically, if an exception occurs during the execution of the business process, the third idempotent record in the idempotent record table is deleted, and the exception is thrown to the upper layer, so that the distributed system can execute the business process normally after the requester retries the request next time.
[0096] In this example, if an exception occurs when executing the business process, deleting the third idempotent record in the idempotent record table enables the distributed system to execute the business process normally after receiving a retry request.
[0097] If the fourth idempotent record exists in the idempotent record table, it indicates that the distributed system has received the corresponding repeated request. In order to achieve idempotency, in one example, the method further includes:
[0098] If there is a fourth idempotent record in the idempotent record table, query the status of the fourth idempotent record;
[0099] If the status of the fourth idempotent record is successful execution, the execution result corresponding to the fourth idempotent record is obtained from the idempotent record table and returned;
[0100] If the status of the fourth idempotent record is being executed, the first information is returned, and the first information is used to prompt that the same business request is being processed.
[0101] Specifically, after detecting that there is a fourth idempotent record in the idempotent record table, the status of the fourth idempotent record can be queried. If the status of the fourth idempotent record is successful execution, it indicates that the distributed system has successfully executed the repeated request corresponding to the fourth idempotent record, and the execution result corresponding to the fourth idempotent record can be directly obtained from the idempotent record table and returned, so as to achieve a fast and accurate response to the requester; if the status of the fourth idempotent record is in execution, it indicates that the distributed system is executing the repeated request corresponding to the fourth idempotent record, and returns the first information to the requester, and the first information is used to prompt that the same business request is being processed, please try again later.
[0102] In this example, if there is a fourth idempotent record in the idempotent record table, it indicates that the current business request is repeated. By querying the status of the fourth idempotent record, when the status is successful execution, the execution result corresponding to the fourth idempotent record is obtained and returned; when the status of the fourth idempotent record is executing, it is prompted to the requester that the same business request is being processed, so that the business request can be responded to quickly and accurately.
[0103] In order to improve the efficiency of the server querying the idempotent record table, in one example, the method further includes:
[0104] Archiving processing is performed regularly, and the archiving processing includes: obtaining a distributed lock. If the acquisition fails, the archiving processing is terminated; if the acquisition is successful, the second idempotent record is obtained from the idempotent record table, and the status of the second idempotent record is successful execution, and the time of successful execution is more than the preset time from the current time; the second idempotent record is inserted into the archiving table, and the second idempotent record in the idempotent record table is deleted.
[0105] Specifically, Figure 4 A schematic diagram of a process flow for archiving an idempotent record table provided in an embodiment of the present application is shown in FIG. Figure 4 As shown in the figure, after the requester sends a business request, it often retries in a short time. Therefore, some idempotent records in the idempotent record table have low utilization rate after a long time after successful execution. The server can perform archiving processing regularly. The archiving processing includes: obtaining a distributed lock. If the acquisition fails, it indicates that other servers in the distributed system are performing archiving processing, and the archiving processing ends; if the acquisition is successful, the idempotent record with a status of successful execution and a time difference from the current time exceeding the preset time is obtained from the idempotent record table as the second idempotent record, and the second idempotent record is inserted into the archiving table, and the second idempotent record in the idempotent record table is deleted, which can effectively improve the hit rate of the idempotent record table.
[0106] In this example, the server periodically deletes idempotent records in the idempotent record table whose status is successful and whose successful execution time is more than a preset time away from the current time. This can effectively improve the hit rate of the idempotent record table, thereby improving the idempotent processing efficiency of the distributed system.
[0107] In the idempotent processing method provided in this embodiment, the server of the distributed system obtains the first idempotent record whose execution time exceeds the corresponding threshold in the idempotent record table of the distributed system, and detects whether the first idempotent record exists in the idempotent execution table of the server, so as to accurately judge whether the server is a server that receives the business request corresponding to the first idempotent record, and when the server is a server that receives the business request corresponding to the first idempotent record, whether the server is executing the business request corresponding to the first idempotent record; if the first idempotent record exists, then according to the thread identifier in the first idempotent record, it is judged whether the thread corresponding to the first idempotent record is alive, and if it is alive, the corresponding threshold of the first idempotent record is extended, so that when the business request corresponding to the first idempotent record is still executed normally, the first idempotent record is no longer in an overdue state; if it does not exist, then the first idempotent record in the sealed record table is deleted, so that when the distributed system receives a repeated request corresponding to the first idempotent record, it is avoided that the corresponding business process cannot be executed normally due to the existence of the first idempotent record in the idempotent record table, so that the idempotent record table can be effectively maintained, so as to efficiently and accurately perform idempotent processing on the received business request.
[0108] Embodiment 2
[0109] Figure 5 A flowchart of an idempotent processing device provided in one embodiment of the present application. The idempotent processing device provided in this embodiment is applied to a server of a distributed system, which includes multiple servers. In practical applications, the idempotent processing device can be implemented by a computer program, such as application software, etc.; or, it can also be implemented as a medium storing relevant computer programs, such as a USB flash drive, a cloud disk, etc.; or, it can also be implemented by a physical device integrated or installed with relevant computer programs, such as a chip, a server, etc. Figure 5 As shown, the device may include:
[0110] The acquisition module 51 is used to obtain the first idempotent record whose execution time exceeds the corresponding overtime threshold in the idempotent record table of the distributed system; wherein the idempotent record table of the distributed system is used to record the business requests received by the distributed system, and the idempotent record includes a server identifier and a thread identifier.
[0111] Specifically, the distributed system database is provided with an idempotent record table of the distributed system, which is used to record the service requests received by the distributed system. Based on the idempotent record table, the distributed system can determine whether the received service request is a duplicate service request.
[0112] Exemplarily, the server identifier is used to uniquely identify a server that receives a service request corresponding to the idempotent record, and the thread identifier is used to uniquely identify a thread on the server that receives a service request corresponding to the idempotent record.
[0113] The detection module 52 is used to detect whether there is a first idempotent record in the idempotent execution table of the server; wherein the idempotent execution table of the server is used to record the idempotent records being executed by the server.
[0114] Specifically, detecting whether there is a pending idempotent record belonging to the first idempotent record in the idempotent execution table of the server may include: judging whether the server identifier corresponds to the current server based on the server identifier in the first idempotent record; if the server identifier corresponds to the current server, then detecting whether the first idempotent record exists in the idempotent execution table of the current server based on the unique key of the first idempotent record; if the server identifier does not correspond to the current server, then determining that the first idempotent record does not exist in the idempotent execution table of the current server.
[0115] Exemplarily, the idempotent record may also include key business request parameters. For example, for a shopping scenario, the key business parameter may be a payment order number. The unique key of the idempotent record may be the key business request parameter.
[0116] The renewal module 53 is used to determine whether the thread corresponding to the pending idempotent record is alive according to the thread identifier in the first idempotent record if there is a first idempotent record; if alive, extend the overtime threshold corresponding to the pending idempotent record according to the preset renewal rule; if not alive, delete the first idempotent record in the idempotent record table.
[0117] Specifically, if there is a first idempotent record in the idempotent execution table of the current server, then according to the thread identifier in the first idempotent record, it is determined whether the corresponding thread in the current server is alive. If alive, the server extends the overtime threshold corresponding to the first idempotent record according to the preset renewal rule, and can also postpone the execution start time corresponding to the first idempotent record; if not alive, the first idempotent record in the idempotent record table is deleted, thereby clearing the idempotent records with abnormal execution in the idempotent record table, and ensuring that the distributed system can re-execute the same business request. At the same time, the server that receives the business request corresponding to the first idempotent record processes the first idempotent record, which can effectively avoid parallel errors in the distributed system.
[0118] The processing module 54 is used to perform idempotent processing on the received service request according to the idempotent record table.
[0119] Specifically, after receiving a service request, the server determines whether the distributed system has received the same service request based on the idempotent record table and performs idempotent processing, so that the distributed system can return the same result for repeated service requests and achieve data consistency.
[0120] In this example, the server of the distributed system obtains the first idempotent record whose execution time exceeds the corresponding threshold in the idempotent record table of the distributed system, and detects whether the first idempotent record exists in the idempotent execution table of the server, so as to accurately determine whether the server is a server that receives the business request corresponding to the first idempotent record, and whether the server is executing the business request corresponding to the first idempotent record when the server is a server that receives the business request corresponding to the first idempotent record; if the first idempotent record exists, then according to the thread identifier in the first idempotent record, it is determined whether the thread corresponding to the first idempotent record is alive, and if it is alive, the corresponding threshold of the first idempotent record is extended, so that when the business request corresponding to the first idempotent record is still executed normally, the first idempotent record is no longer in an overdue state; if it does not exist, then the first idempotent record in the sealed record table is deleted, so that when the distributed system receives a repeated request corresponding to the first idempotent record, it is avoided that the corresponding business process cannot be executed normally due to the existence of the first idempotent record in the idempotent record table, so that the idempotent record table can be effectively maintained, so as to efficiently and accurately perform idempotent processing on the received business requests.
[0121] The above example is a case where there is a first idempotent record in the idempotent execution table of the server. In one example, the apparatus further includes:
[0122] The deletion module is used to delete the first idempotent record in the idempotent record table if the first idempotent record does not exist in the idempotent execution tables of multiple servers.
[0123] Specifically, the first idempotent record does not exist in the idempotent execution tables of multiple servers, and at least the following two situations exist:
[0124] Case 1: A server corresponding to the server identifier in the first idempotent record among multiple servers, and the first idempotent record does not exist in the idempotent execution table of the server.
[0125] In this case, due to server crash, restart or network oscillation, the first idempotent record does not exist in the idempotent execution table of the server that receives the business request corresponding to the first idempotent record. The server is not currently executing the business request corresponding to the first idempotent record. The first idempotent record in the idempotent record table is deleted so that the distributed system can execute the business processing flow normally when it receives the repeated business request corresponding to the first idempotent record again.
[0126] Case 2: A server corresponding to a server identifier that does not exist in the first idempotent record among multiple servers.
[0127] In this case, due to reasons such as the scaling down of the distributed system or the server going offline, the identifier of the server that receives the service request corresponding to the first idempotent record will not appear in the distributed system.
[0128] After determining that a server corresponding to a server identifier in the first idempotent record does not exist in the multiple servers in the distributed system, the distributed system issues a cleanup task, and the server assigned to the cleanup task in the multiple servers cleans up the first idempotent record in the idempotent record table. The cleanup process may include: sending an alarm email to a management personnel so that the management personnel manually processes the corresponding data; or deleting the first idempotent record in the idempotent record table.
[0129] In this example, when the first idempotent record does not exist in the idempotent execution tables of multiple servers, the first idempotent record in the idempotent record table is deleted, so that the idempotent records corresponding to the business requests that have not been successfully executed can be accurately cleaned up, so that when the requester retries the request, the distributed system can execute the corresponding business process normally, avoiding the requester receiving the response of "concurrent request execution, please try again later" every time it retries.
[0130] In actual applications, after receiving a service request, the server will determine whether the distributed system has received repeated requests based on the idempotency records in the idempotency record table, and then perform idempotency processing. In an example, the processing module can be specifically used to:
[0131] Receive a business request, construct a third idempotent record according to the business request, and set the status of the third idempotent record to be in execution; wherein the idempotent record includes: request parameters and status of the business request, and a server identifier, thread identifier, and time of receiving the business request; wherein the request parameter is a unique key of the idempotent record;
[0132] Determine whether there is a fourth idempotent record in the idempotent record table, and the unique key of the fourth idempotent record is the same as the unique key of the third idempotent record; if not, write the third idempotent record into the idempotent record table, execute the business process according to the business request, and return the execution result; and update the status of the third idempotent record in the idempotent record table to successful execution, and add the execution result to the third idempotent record.
[0133] Specifically, the idempotent record includes the server ID, thread ID, time of receiving the business request, and request parameters and status of the business request, where the status includes "in execution" and "successful execution", and the request parameter is the unique key of the idempotent record. After the server receives the business request sent by the requester, it constructs a third idempotent record according to the business request and sets the status of the third idempotent record to "in execution". According to the unique key of the third idempotent record, it is determined whether there is a fourth idempotent record with the same unique key in the idempotent record table. If not, it indicates that there is no duplicate request. The third idempotent record is written into the idempotent record table, and the corresponding business process is executed according to the business request. After successful execution, the execution result is returned to the requester, and the status of the third idempotent record in the idempotent record table is updated to "successful execution", and the execution result is written. Therefore, when the distributed system receives a duplicate request corresponding to the third idempotent record next time, it can conveniently return the execution result according to the idempotent record table and realize the idempotence of the system.
[0134] In this example, a third idempotent record is constructed according to the received business request, the status of the third idempotent record is set to being executed, and it is determined whether there is a fourth idempotent record with the same request parameters in the idempotent record table. If the fourth idempotent record does not exist, the third idempotent record is written into the idempotent record table, and the business process is executed according to the business request. The execution result is returned, and the status of the third idempotent record in the idempotent record table is updated to successful execution. The execution result is added to the third idempotent record, which can accurately implement idempotent processing.
[0135] In one example, the processing module may also be used to:
[0136] If an exception occurs when executing the business process, the third idempotent record in the idempotent record table is deleted.
[0137] Specifically, if an exception occurs during the execution of the business process, the third idempotent record in the idempotent record table is deleted, and the exception is thrown to the upper layer, so that the distributed system can execute the business process normally after the requester retries the request next time.
[0138] In this example, if an exception occurs when executing the business process, deleting the third idempotent record in the idempotent record table enables the distributed system to execute the business process normally after receiving a retry request.
[0139] If there is a fourth idempotent record in the idempotent record table, it indicates that the distributed system has received corresponding repeated requests. In order to achieve idempotency, in an example, the processing module can also be used to:
[0140] If there is a fourth idempotent record in the idempotent record table, query the status of the fourth idempotent record;
[0141] If the status of the fourth idempotent record is successful execution, the execution result corresponding to the fourth idempotent record is obtained from the idempotent record table and returned;
[0142] If the status of the fourth idempotent record is being executed, the first information is returned, and the first information is used to prompt that the same business request is being processed.
[0143] Specifically, after detecting that there is a fourth idempotent record in the idempotent record table, the status of the fourth idempotent record can be queried. If the status of the fourth idempotent record is successful execution, it indicates that the distributed system has successfully executed the repeated request corresponding to the fourth idempotent record, and the execution result corresponding to the fourth idempotent record can be directly obtained from the idempotent record table and returned, so as to achieve a fast and accurate response to the requester; if the status of the fourth idempotent record is in execution, it indicates that the distributed system is executing the repeated request corresponding to the fourth idempotent record, and returns the first information to the requester, and the first information is used to prompt that the same business request is being processed, please try again later.
[0144] In this example, if there is a fourth idempotent record in the idempotent record table, it indicates that the current business request is repeated. By querying the status of the fourth idempotent record, when the status is successful execution, the execution result corresponding to the fourth idempotent record is obtained and returned; when the status of the fourth idempotent record is executing, it is prompted to the requester that the same business request is being processed, so that the business request can be responded to quickly and accurately.
[0145] In order to improve the efficiency of the server querying the idempotent record table, in one example, the device further includes:
[0146] The archiving module is used to perform archiving processing regularly. The archiving processing includes: obtaining a distributed lock. If the acquisition fails, the archiving processing is terminated; if the acquisition is successful, the second idempotent record is obtained from the idempotent record table. The status of the second idempotent record is successful execution, and the time of successful execution is more than a preset time from the current time; the second idempotent record is inserted into the archiving table, and the second idempotent record in the idempotent record table is deleted.
[0147] Specifically, after the requester sends a service request, it often retries in a short time. Therefore, some idempotent records in the idempotent record table have low utilization rate after a long time after successful execution. The server can perform archiving processing regularly, and the archiving processing includes: obtaining a distributed lock. If the acquisition fails, it indicates that other servers in the distributed system are performing archiving processing, and the current archiving processing ends; if the acquisition is successful, the idempotent record with a status of successful execution and a time difference from the current time exceeding the preset time is obtained from the idempotent record table as the second idempotent record, and the second idempotent record is inserted into the archiving table, and the second idempotent record in the idempotent record table is deleted, which can effectively improve the hit rate of the idempotent record table.
[0148] In this example, the server periodically deletes idempotent records in the idempotent record table whose status is successful and whose successful execution time is more than a preset time away from the current time. This can effectively improve the hit rate of the idempotent record table, thereby improving the idempotent processing efficiency of the distributed system.
[0149] In the idempotent processing device provided by the present embodiment, the server of the distributed system obtains the first idempotent record whose execution time exceeds the corresponding threshold in the idempotent record table of the distributed system, and detects whether the first idempotent record exists in the idempotent execution table of the server, so as to accurately judge whether the server is a server that receives the business request corresponding to the first idempotent record, and when the server is a server that receives the business request corresponding to the first idempotent record, whether the server is executing the business request corresponding to the first idempotent record; if the first idempotent record exists, then according to the thread identifier in the first idempotent record, it is judged whether the thread corresponding to the first idempotent record is alive, and if it is alive, the corresponding threshold of the first idempotent record is extended, so that when the business request corresponding to the first idempotent record is still executed normally, the first idempotent record is no longer in an overdue state; if it does not exist, then the first idempotent record in the sealed record table is deleted, so that when the distributed system receives a repeated request corresponding to the first idempotent record, it is avoided that the corresponding business process cannot be executed normally due to the existence of the first idempotent record in the idempotent record table, so that the idempotent record table can be effectively maintained, so as to efficiently and accurately perform idempotent processing on the received business requests.
[0150] Embodiment 3
[0151] Figure 6 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure, such as Figure 6 As shown, the electronic device includes:
[0152] The electronic device includes a processor 291 and a memory 292; it may also include a communication interface 293 and a bus 294. The processor 291, the memory 292, and the communication interface 293 may communicate with each other through the bus 294. The communication interface 293 may be used for information transmission. The processor 291 may call the logic instructions in the memory 292 to execute the method of the above embodiment.
[0153] In addition, the logic instructions in the above-mentioned memory 292 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.
[0154] The memory 292 is a computer-readable storage medium that can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the methods in the embodiments of the present disclosure. The processor 291 executes the software programs, instructions, and modules stored in the memory 292 to perform functional applications and data processing, that is, to implement the methods in the above method embodiments.
[0155] The memory 292 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 292 may include a high-speed random access memory and may also include a non-volatile memory.
[0156] An embodiment of the present disclosure provides a non-transitory computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the method described in the above embodiment.
[0157] Embodiment 4
[0158] An embodiment of the present disclosure provides a computer program product, including a computer program. When the computer program is executed by a processor, the method provided in any embodiment of the present disclosure is implemented.
[0159] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.
[0160] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
Claims
1. An idempotent processing method, characterized in that: A server applied to a distributed system, wherein the distributed system includes a plurality of servers; the method includes: Obtaining a first idempotent record in the idempotent record table of the distributed system whose execution time exceeds a corresponding overtime threshold; wherein the idempotent record table of the distributed system is used to record the service requests received by the distributed system, and the idempotent record includes a server identifier and a thread identifier; Detecting whether the first idempotent record exists in the idempotent execution table of the server; wherein the idempotent execution table of the server is used to record the idempotent record being executed by the server; If the first idempotent record exists, judging whether the thread corresponding to the first idempotent record is alive according to the thread identifier in the first idempotent record; if alive, extending the overtime threshold corresponding to the first idempotent record according to the preset renewal rule; if not alive, deleting the first idempotent record in the idempotent record table; According to the idempotent record table, the received business requests are processed idempotently.
2. The method according to claim 1, characterized in that The method further comprises: If the first idempotent record does not exist in the idempotent execution tables of the multiple servers, the first idempotent record in the idempotent record table is deleted.
3. The method according to claim 1, characterized in that The method further comprises: Archiving processing is performed regularly, and the archiving processing includes: obtaining a distributed lock. If the acquisition fails, the archiving processing is terminated; if the acquisition is successful, a second idempotent record is obtained from the idempotent record table, the status of the second idempotent record is successful execution, and the time of successful execution is more than a preset threshold from the current time; the second idempotent record is inserted into the archiving table, and the second idempotent record in the idempotent record table is deleted.
4. The method according to any one of claims 1 to 3, characterized in that: The step of performing idempotent processing on the received service request according to the idempotent record table includes: Receive a business request, construct a third idempotent record according to the business request, and set the state of the third idempotent record to be in execution; wherein the idempotent record includes: request parameters and state of the business request, and a server identifier, thread identifier, and time of receiving the business request; wherein the request parameter is a unique key of the idempotent record; Determine whether there is a fourth idempotent record in the idempotent record table, and the unique key of the fourth idempotent record is the same as the unique key of the third idempotent record; if not, write the third idempotent record into the idempotent record table, execute the business process according to the business request, and return the execution result; and update the status of the third idempotent record in the idempotent record table to successful execution, and add the execution result to the third idempotent record.
5. The method according to claim 4, characterized in that The method further comprises: If an exception occurs when executing the business process, the third idempotent record in the idempotent record table is deleted.
6. The method according to claim 4, characterized in that The method further comprises: If the fourth idempotent record exists in the idempotent record table, query the status of the fourth idempotent record; If the status of the fourth idempotent record is successful execution, the execution result corresponding to the fourth idempotent record is obtained from the idempotent record table and returned; If the status of the fourth idempotent record is being executed, the first information is returned, where the first information is used to prompt that the same business request is being processed.
7. An idempotent processing device, characterized in that: A server applied to a distributed system, wherein the distributed system includes a plurality of servers; the device includes: An acquisition module is used to acquire a first idempotent record whose execution time exceeds a corresponding overtime threshold in the idempotent record table of the distributed system; wherein the idempotent record table of the distributed system is used to record the service requests received by the distributed system, and the idempotent record includes a server identifier and a thread identifier; A detection module, used to detect whether the first idempotent record exists in the idempotent execution table of the server; wherein the idempotent execution table of the server is used to record the idempotent record being executed by the server; a renewal module, configured to, if the first idempotent record exists, determine whether the thread corresponding to the first idempotent record is alive according to the thread identifier in the first idempotent record; if alive, extend the overtime threshold corresponding to the first idempotent record according to a preset renewal rule; if not alive, delete the first idempotent record in the idempotent record table; The processing module is used to perform idempotent processing on the received business request according to the idempotent record table.
8. The device according to claim 7, characterized in that The device also includes: The deleting module is used to delete the first idempotent record in the idempotent record table if the first idempotent record does not exist in the idempotent execution tables of the multiple servers.
9. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 6.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 6 when executed by a processor.
Citation Information
Patent Citations
Task processing method and device in distributed system
CN105446801A
Service processing method and apparatus
CN109063027A
Transaction processing method and device, electronic equipment and storage medium
CN116797366A
