Atomic commit and delivery methods for database transactions and asynchronous messages
Through the atomic submission method of database transactions and asynchronous messages, the problem of inconsistent database submission and message delivery in microservice scenarios is solved, low latency, high throughput and decentralized fault tolerance are achieved, business intrusion is reduced, and system reliability and scalability are improved.
Patent Information
- Application Number
- CN202511080610.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-04
- Publication Date
- 2025-09-26
- Estimated Expiration
- 2045-08-04
AI Technical Summary
Existing technologies cannot simultaneously meet the technical requirements of low latency and high throughput, decentralized fault tolerance, and business non-intrusion in microservice scenarios, resulting in problems such as database submission being successful but messages not being delivered, or message delivery being successful but database transactions not being committed.
Adopt the atomic submission method of database transactions and asynchronous messages, create database transactions through preset transaction message packages and target functions, configure demand strategies, ensure the consistency of messages and data, and use atomic submission to trigger the submission of the first type of information only after the successful submission of the second type of information, shielding the differences in message middleware APIs and realizing the decoupling of business code and middleware.
It achieves the synchronization and consistency of database transactions and asynchronous messages, reduces message delivery delays, improves compensation efficiency, reduces business intrusion and operation and maintenance costs, and enhances system reliability and scalability.
Smart Images

Figure CN120567924B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to an atomic submission method and delivery method for database transactions and asynchronous messages. Background Art
[0002] In microservice scenarios, business operations often require the simultaneous completion of local database transaction submission and cross-service message notifications, such as sending an inventory deduction message after an order is created. However, due to network partitions, service downtime, and other failures, cross-system status inconsistencies can easily occur, where the database submission succeeds but the message is not delivered, or the message is delivered successfully but the database transaction is not committed.
[0003] Existing technologies primarily employ the following three solutions. The first is a basic local message table solution. When executing business operations, business services poll the message table through an independent process and deliver it to the message middleware. This solution suffers from excessive I / O load, insufficient real-time performance, and high business intrusion. Message table polling uses a scheduled task, resulting in significant message delivery delays. The second is a transactional outbox solution, which captures data change events through database transaction logs and forwards them to the message middleware using CDC tools. This solution is architecturally complex, requiring the deployment of independent CDC components and parsing of database log formats. It relies heavily on database versions and configurations, increasing operational and maintenance costs, uncontrollable message processing delays, and the inability to ensure that message order aligns with transaction submission order. Furthermore, there is a lack of compensation mechanisms, requiring the development of additional retry strategies when message delivery fails, making cross-component state rollback difficult.
[0004] The third option is to use 2PC (two-phase commit). The traditional 2PC protocol uses a coordinator to implement atomic commits across resource managers, but there is a synchronization blocking problem. Participants lock resources in the Prepare phase until Commit / Rollback, which can easily cause thread blocking and deadlock under high concurrency. The coordinator also has a single point of failure. A downtime will cause the loss of global transaction status. After recovery, manual intervention is required to handle suspended transactions, reducing system availability. Existing solutions cannot simultaneously meet the technical requirements of low latency, high throughput, decentralized fault tolerance, and non-intrusive business operations. Summary of the Invention
[0005] In view of this, the embodiments of the present disclosure provide an atomic submission method and delivery method for database transactions and asynchronous messages, which can solve the problem that existing solutions cannot simultaneously meet technical requirements such as low latency and high throughput, decentralized fault tolerance, and non-intrusive business.
[0006] In a first aspect, an embodiment of the present disclosure provides a method for atomically committing database transactions and asynchronous messages, including:
[0007] In response to a first type of information in a first field initiated by a user in a target business scenario, parsing first message content and first business data in the first type of information;
[0008] Calling a preset transaction message package; the preset transaction message package is configured with a target function, several message middlewares, a business data writing interface, and a business message publishing interface;
[0009] Creating a database transaction through the target function; configuring a plurality of demand strategies in the second domain in the database transaction;
[0010] Generating, in the database transaction, a first write request for writing the first message content and the first service data into a target service database;
[0011] In response to the instruction to generate the first write request, in the database transaction, calling a corresponding demand policy in the second domain according to the first business data, which is recorded as a target demand policy;
[0012] Verifying the identity information of the user according to the target demand policy, and when the identity information passes the verification, executing the target demand policy and generating second type of information, the second type of information including second message content and second business data;
[0013] Submitting the second type of information to the target business database through the business message writing interface;
[0014] In response to a successful submission instruction of the second type of information, the execution of the first write request is triggered to submit the first type of information to the target business database.
[0015] In a second aspect, an embodiment of the present disclosure further provides a method for atomic delivery of database transactions and asynchronous messages, including:
[0016] In response to a first type of information in a first field initiated by a user in a target business scenario, parsing first message content and first business data in the first type of information;
[0017] Calling a target function in a preset transaction message package to create a database transaction; the database transaction is configured with a plurality of demand strategies in the second domain;
[0018] Generating, in the database transaction, a first write request for writing the first message content and the first service data into a target service database;
[0019] In response to the instruction to generate the first write request, in the database transaction, calling a corresponding demand policy in the second domain according to the first business data, which is recorded as a target demand policy;
[0020] Verifying the identity information of the user according to the target demand policy, and when the identity information passes the verification, executing the target demand policy and generating second type of information, the second type of information including second message content and second business data;
[0021] Submitting the second type of information to the target business database through a business message writing interface;
[0022] In response to a successful submission instruction for the second type of information, triggering execution of the first write request to submit the first type of information to the target business database, initiating a request to call a business message publishing interface after the submission is successful, and based on the call request of the business message publishing interface, calling the message middleware corresponding to the first domain to convert the first message content into a first message parameter supported by the corresponding middleware, and calling the message middleware corresponding to the second domain to convert the second message content into a second message parameter supported by the corresponding middleware;
[0023] The first message parameter is delivered through the three-party message sending interface of the message middleware corresponding to the first domain; and the second message parameter is delivered through the three-party message sending interface of the message middleware corresponding to the second domain.
[0024] In a third aspect, the embodiments of the present disclosure further provide a computer device that adopts the following technical solution:
[0025] The computer device comprises:
[0026] at least one processor; and,
[0027] a memory communicatively connected to the at least one processor; wherein,
[0028] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any of the above-mentioned atomic submission methods of database transactions and asynchronous messages or atomic delivery methods of database transactions and asynchronous messages.
[0029] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute any of the above-mentioned atomic submission methods of database transactions and asynchronous messages or atomic delivery methods of database transactions and asynchronous messages.
[0030] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, comprising a computer program / instruction, which implements the steps of any of the above methods when executed by a processor.
[0031] The present application discloses an atomic submission method for database transactions and asynchronous messages, comprising: responding to a first type of information in a first field initiated by a user in a target business scenario, parsing a first message content and first business data in the first type of information, calling a preset transaction message package, and creating a database transaction through a target function; the database transaction is configured with a number of demand strategies in a second field, generating a first write request in the database transaction to write the first message content and the first business data into the target business database, responding to a generation instruction for the first write request, calling a corresponding demand strategy in the second field in the database transaction according to the first business data, recorded as a target demand strategy, verifying the user's identity information according to the target demand strategy, and when the identity information passes the verification, executing the target demand strategy and generating a second type of information, the second type of information including a second message content and a second business data, through The business message write interface submits the second type of information to the target business database. In response to the successful submission instruction of the second type of information, it triggers the execution of the first write request to submit the first type of information to the target business database. This application can shield the API differences of message middleware such as MNS and RocketMQ, provide a unified message publishing interface for the business layer, and achieve complete decoupling of business code and middleware technology. It adopts an atomic submission method to associate database transactions with the submission of asynchronous messages. In database transactions, various operations are performed in a specific order. Only when the second type of information is successfully submitted to the target business database will the first write request be triggered to submit the first type of information to the target business database. This atomic submission method avoids the delay problem caused by message table polling, ensures the synchronization of messages and data, and improves compensation efficiency.
[0032] The above description is only an overview of the technical solution of the present disclosure. In order to more clearly understand the technical means of the present disclosure, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present disclosure more obvious and easy to understand, the following specifically cites preferred embodiments and describes them in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0034] Figure 1 A flowchart of a method for atomically committing database transactions and asynchronous messages provided in an embodiment of the present disclosure.
[0035] Figure 2A flowchart of a method for atomically delivering database transactions and asynchronous messages provided in an embodiment of the present disclosure.
[0036] Figure 3 A flowchart of a method for calling an asynchronous daemon process strategy for redelivery provided in an embodiment of the present disclosure.
[0037] Figure 4 A flowchart of a method for synchronously triggering redelivery of a target message provided by an embodiment of the present disclosure.
[0038] Figure 5 A schematic diagram of a process for dynamically adjusting the scanning frequency according to the total number of redelivery times and triggering message redelivery provided in an embodiment of the present disclosure.
[0039] Figure 6 A schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0040] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.
[0041] It should be clear that the following embodiments of the present disclosure are described through specific concrete examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.
[0042] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on this disclosure, it should be understood by those skilled in the art that an aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein can be used to implement this device and / or practice this method.
[0043] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The illustrations only show components related to the present disclosure and are not drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component can be changed at will, and the component layout type may also be more complicated.
[0044] Additionally, in the following description, specific details are provided to provide a thorough understanding of the examples. However, one skilled in the art will appreciate that the aspects described can be practiced without these specific details.
[0045] Reference Figure 1 This application discloses a method for atomically submitting database transactions and asynchronous messages, specifically a method for atomically submitting database transactions and asynchronous messages under a microservice architecture, which specifically includes:
[0046] S100 , in response to first-category information in a first domain initiated by a user in a target business scenario, parsing first message content and first business data in the first-category information.
[0047] Among them, the target business scenarios include insurance business scenarios, e-commerce business scenarios, banking and finance scenarios, logistics and warehousing scenarios, and other scenarios involving data storage and message sending.
[0048] Parsing user-initiated information can clearly separate message content and business data, providing a clear data basis for subsequent processing, allowing subsequent steps to independently process different types of data, improving the system's processing efficiency and accuracy.
[0049] S200, calling a preset transaction message package; the preset transaction message package is configured with a target function, several message middlewares, a business data writing interface and a business message publishing interface.
[0050] The business message publishing interface shields API differences between different message middleware. The pre-configured transaction message package is a pre-configured transaction message package—a pre-developed component. By configuring the pre-configured transaction message package, API differences between message middleware like MNS and RocketMQ can be shielded, providing a unified message publishing interface for the business layer and achieving complete decoupling of business code from middleware technology.
[0051] The preset transaction message package encapsulates related functions together, improving the reusability and maintainability of the code; through unified interfaces and functions, it simplifies system development and management and reduces the duplication of work for developers.
[0052] S300, creating a database transaction through the target function; the database transaction is configured with several demand strategies in the second domain.
[0053] Specifically, the target function is the qorm.Transaction() function in the transaction message package. The database transaction created by the qorm.Transaction() function can ensure the consistency of order data and message data at the database level.
[0054] When the target business scenario is an insurance business scenario, the multiple demand strategies refer to multiple insurance strategies. When the target business scenario is other scenarios, the multiple demand strategies correspond to information related to different demands.
[0055] Database transactions ensure data consistency and integrity. By configuring demand policies, you can strictly control and verify data during transaction execution to avoid data inconsistencies. For example, when deducting inventory, if there is insufficient inventory, the transaction can be rolled back to ensure data correctness.
[0056] S400: Generate a first write request in a database transaction to write first message content and first business data into a target business database.
[0057] Generating a write request within a database transaction ensures the atomicity of data write operations. If the transaction fails, the write request will not take effect, thus avoiding data inconsistency. At the same time, binding the write request to the transaction facilitates subsequent unified management and rollback operations.
[0058] S500 , in response to a generation instruction of a first write request, calling a corresponding demand policy in a second domain according to first business data in a database transaction, which is recorded as a target demand policy.
[0059] Dynamically calling demand strategies based on business data enables the system to respond flexibly to different business situations and adjust strategies based on specific business needs, thereby improving the adaptability and flexibility of the system.
[0060] S600, verifying the user's identity information according to the target demand policy. When the identity information passes the verification, executing the target demand policy and generating the second type of information. The second type of information includes the second message content and the second business data.
[0061] Verifying user identity information can ensure the security of business operations and prevent illegal users from performing operations. After identity authentication is passed, the required strategy is executed and new information is generated, ensuring the correctness and consistency of business processes.
[0062] S700: Submit the second type of information to the target business database through the business message writing interface.
[0063] Submitting the second type of information to the target business database in a timely manner ensures the real-time and accuracy of the data in the database, allowing other business modules to obtain the latest business data in a timely manner and improve the overall performance of the system.
[0064] S800 , in response to a successful submission instruction of the second type of information, triggering execution of a first write request to submit the first type of information to a target business database.
[0065] This method ensures the atomic submission of database transactions and asynchronous messages. The first type of information will only be submitted after the second type of information is successfully submitted, ensuring the consistency of order information and inventory information, and avoiding the situation where the order has been generated but the inventory has not been deducted, or the inventory has been deducted but the order has not been generated.
[0066] The atomic submission method of database transactions and asynchronous messages disclosed in this application ensures the consistency of business data between different database tables through database transactions and atomic submission mechanisms. In the example of an e-commerce platform, it ensures the synchronous update of order information and inventory information, avoiding the problem of data inconsistency; the entire solution integrates user-initiated business operations with related business policies and message processing, ensuring the integrity of business processes, such as from user ordering to inventory deduction, and then to order generation, each link can be processed according to predetermined policies and processes; the use of message middleware and transaction mechanism improves the reliability of the system. When an abnormal situation occurs, the database transaction can be rolled back, and the message middleware can ensure the reliable delivery of messages, ensuring the security and integrity of business data; the design of preset transaction message packages and dynamic call demand policies makes the system have good scalability, and can easily add new message middleware, demand policies and business interfaces according to the development and changes of the business to meet different business needs.
[0067] In this embodiment, when the business layer calls the unified message publishing interface, it can pass parameters according to certain specifications. Through this input parameter specification, the adaptation layer can more conveniently process the parameters passed by the business layer, and can also check the validity of the parameters to avoid exceptions caused by parameter errors. After completing the message publishing operation, the adaptation layer can return the result to the business layer according to the unified specification. For example, the return value is specified as an object containing a status code and a message. The status code can be SUCCESS, indicating that the message is successfully published, and FAILED, indicating that the message fails to be published; the message can be a detailed description of the status, such as Message published successfully or Failed to publish message due to network error. The business layer can perform corresponding processing based on the returned status code and message, such as logging, retrying, etc. Through the standardization of the message protocol, the interaction between the business layer and the adaptation layer is clearer and more reliable, which improves the maintainability and scalability of the system.
[0068] The present application discloses an atomic submission method for database transactions and asynchronous messages, comprising: responding to a first type of information in a first field initiated by a user in a target business scenario, parsing a first message content and first business data in the first type of information, calling a preset transaction message package, and creating a database transaction through a target function; the database transaction is configured with a number of demand strategies in a second field, generating a first write request in the database transaction to write the first message content and the first business data into the target business database, responding to a generation instruction for the first write request, calling a corresponding demand strategy in the second field in the database transaction according to the first business data, recorded as a target demand strategy, verifying the user's identity information according to the target demand strategy, and when the identity information passes the verification, executing the target demand strategy and generating a second type of information, the second type of information including a second message content and a second business data, through The business message write interface submits the second type of information to the target business database. In response to the successful submission instruction of the second type of information, it triggers the execution of the first write request to submit the first type of information to the target business database. This application can shield the API differences of message middleware such as MNS and RocketMQ, provide a unified message publishing interface for the business layer, and achieve complete decoupling of business code and middleware technology. It adopts an atomic submission method to associate database transactions with the submission of asynchronous messages. In database transactions, various operations are performed in a specific order. Only when the second type of information is successfully submitted to the target business database will the first write request be triggered to submit the first type of information to the target business database. This atomic submission method avoids the delay problem caused by message table polling, ensures the synchronization of messages and data, and improves compensation efficiency.
[0069] When the target business scenario is insurance, the first area involved is payment, and the second area is insurance. When the target business scenario is e-commerce, the first area involved is business information entry, and the second area is inventory changes. When the target business scenario is banking and finance, the first area involved is payment, and the second area is capital flow. When the target business scenario is logistics and warehousing, the first area involved is inventory quantity changes, and the second area is logistics scheduling.
[0070] The following describes the target business scenario, using the insurance business scenario as an example, in detail. In this scenario, the atomic submission method for database transactions and asynchronous messages specifically involves the following: In the insurance business scenario, a user who wishes to purchase insurance must first successfully pay. In response to the first type of information (i.e., a payment success request) in the first field (i.e., the payment field) initiated by the user in the target business scenario (i.e., the insurance business scenario), for example, after a user selects an insurance product on the insurance platform and clicks the payment button, the system receives payment-related information such as the payment amount, payment method (bank card payment, online payment, etc.), and the user's payment account number. The system parses this information to separate the first message content and the first business data. The first message content might be "User Zhang San paid a premium of [insurance product name] of [X] yuan," while the first business data includes specific data such as the payment amount, payment method code, and the user's payment account number.
[0071] The system calls a preset transaction message package, which is configured with a target function, several message middleware (such as Kafka, RabbitMQ, etc.), a business data writing interface, and a business message publishing interface. In the insurance business, these components will be used to coordinate the processing of payment data and the delivery of insurance-related messages. The target function will be responsible for managing the entire transaction process. The message middleware is used to transmit messages between different services. The business data writing interface is used to write payment and insurance data to the corresponding database. The business message publishing interface is used to publish messages related to the insurance business.
[0072] A database transaction is created using the target function in the preset transaction message package. This transaction configures several insurance policy requirements, such as checking whether the user meets insurance requirements (age, health status, etc.), verifying the validity of the insurance product (whether it is within the sales period, whether there is any remaining balance, etc.), and determining the effective date of the insurance contract. These policy requirements will be checked and executed during subsequent transaction execution.
[0073] In the database transaction, a first write request is generated to write the first message content and the first business data into the target business database (such as the payment database). This means that the system will prepare to write the user's payment information (such as payment amount, payment method, payment time, etc.) into the corresponding table of the payment database. The request may be an SQL insert statement used to insert the payment information into the payment record table.
[0074] In response to the instruction to generate the first write request, a corresponding demand policy in the insurance domain is invoked based on the first business data in a database transaction, recorded as the target demand policy. For example, based on the user's payment amount and the selected insurance product, the corresponding insurance eligibility verification policy is invoked. If the payment amount is consistent with the insurance product's premium, and the user's age, health status, and other conditions meet the insurance product's eligibility requirements, the demand policy is determined to be a valid target demand policy.
[0075] The user's identity information is verified according to the target demand strategy. For example, the user's ID number and name are verified to be consistent with the payment account and insurance information. If the identity information is verified, the target demand strategy is executed, such as generating the insurance contract, determining the insurance period, calculating the insurance benefits, etc., and generating the second type of information. The second type of information includes the second message content and the second business data. The second message content may be "User Zhang San has successfully insured [insurance product name], the insurance contract number is [XXXX]." The second business data includes the insurance contract number, insurance period, insurance amount, beneficiary information, etc.
[0076] The second type of information is submitted to the target business database (such as the insurance database) through the business message writing interface. The system will write the generated insurance contract information, user insurance information, etc. into the corresponding table of the insurance database and update the user's insurance record. This step ensures the accuracy and completeness of the insurance information in the database.
[0077] In response to the successful submission instruction of the second type of information, the first write request is triggered to submit the first type of information (payment information) to the target business database (payment database). Only when the insurance information is successfully submitted to the insurance database will the payment information be officially written to the payment database, thereby ensuring the consistency of the payment information and the insurance information. This can avoid the situation where the user's payment is successful but the insurance is unsuccessful or the insurance is successful but the payment information is not recorded, and realize the atomic submission of database transactions and asynchronous messages.
[0078] Through the above steps, in the insurance business scenario, this solution can ensure data consistency and transaction integrity in the payment and insurance areas, and improve the reliability and accuracy of insurance business processing.
[0079] The atomic submission method for database transactions and asynchronous messages disclosed in this application can effectively solve the problems of high business intrusion, low compensation efficiency and poor cross-middleware compatibility in the existing technology. The specific analysis is as follows: In the existing local message table solution, the business logic and message processing logic are coupled, and developers need to explicitly write message table operation code and adapt to different message middleware APIs. However, this application encapsulates the message processing-related functions in a preset transaction message package by calling the preset transaction message package. The preset transaction message package is configured with a target function, several message middlewares, a business data writing interface and a business message publishing interface. Developers only need to call this preset package, and there is no need to explicitly write message table operation code and adapt to different message middleware APIs in the business code. This reduces the coupling between the business code and the message processing logic, realizes the decoupling of business and message processing, and thus reduces business intrusion.
[0080] The business data writing interface and business message publishing interface provide developers with a unified operation method. During the entire business processing process, developers only need to focus on the logic of the business itself and complete data writing and message publishing through these unified interfaces without having to consider the specific implementation details of different message middleware. This further reduces the dependence of business code on message processing logic and reduces business intrusion.
[0081] The message table polling mechanism in the prior art relies on fixed-interval scanning, resulting in uncontrollable delays and a lack of sharding processing capabilities. This application adopts an atomic submission method to associate database transactions with the submission of asynchronous messages. In database transactions, various operations are performed in a specific order, such as generating a write request (S400), calling a demand policy (S500), and verifying user identity (S600). Only after the second type of information is successfully submitted to the target business database (S700) will the first write request be triggered to submit the first type of information to the target business database (S800). This atomic submission method avoids the delay problem caused by message table polling, ensures the synchronization of messages and data, and improves compensation efficiency.
[0082] This solution processes business logic and message-related operations in real time within database transactions, rather than relying on scheduled tasks like message table polling. During transaction execution, the corresponding operations are executed immediately once the conditions are met, reducing message delivery delays and improving real-time performance, thereby solving the problem of low compensation efficiency.
[0083] Reference Figure 2 In the second aspect, the present application discloses a method for atomic delivery of database transactions and asynchronous messages, which is based on the atomic submission method for database transactions and asynchronous messages. The delivery method specifically includes:
[0084] S10 , in response to a successful submission instruction for submitting the first type of information to the target business database, initiating a request for calling a business message publishing interface.
[0085] This step ensures that message publishing occurs only after a successful database transaction. For example, a message is only published after payment information has been accurately written to the database. This avoids unnecessary message publishing in the event of a database operation failure, ensures data and message consistency, and improves system reliability.
[0086] In this embodiment, the atomic delivery method of database transactions and asynchronous messages disclosed in the present application is triggered after the execution of the atomic submission method of database transactions and asynchronous messages disclosed in the present application is completed. Furthermore, it is triggered after the first type of information is successfully submitted to the target business database.
[0087] S20, in response to the call request of the business message publishing interface, call the message middleware corresponding to the first domain to convert the first message content into the first message parameter supported by the corresponding middleware, and call the message middleware corresponding to the second domain to convert the second message content into the second message parameter supported by the corresponding middleware.
[0088] In this step, the business service calls the message publishing interface by introducing the transaction message package, without having to be aware of the message table and message middleware details.
[0089] Taking e-commerce as an example, the first domain is payment, and the second domain is order. The message middleware for the payment domain might be Kafka, while the message middleware for the order domain might be RabbitMQ. Upon receiving a call to the business message publishing interface, the system converts the first message content (e.g., "User Zhang San paid [product name] for [X] yuan") into the first message parameters supported by Kafka. This may include setting the message subject (e.g., "payment_success"), the message key (e.g., the order number), and the message body (i.e., the specific payment information). Similarly, for the second message content (e.g., "Order [order number] has been paid and is ready for shipment"), the system converts it into the second message parameters supported by RabbitMQ, including setting the queue name and message properties. Different message middlewares have different message formats and parameter requirements. By converting message content into parameters supported by the corresponding middleware, the system can adapt to multiple message middlewares, improving system compatibility and flexibility. Regardless of the message middleware used, it ensures that messages are correctly processed and delivered.
[0090] Furthermore, the message middleware corresponding to the first domain and the message middleware corresponding to the second domain may be the same or different, and both are within the protection scope of this application.
[0091] S30, delivering the first message parameter through the three-party message sending interface of the message middleware corresponding to the first domain; delivering the second message parameter through the three-party message sending interface of the message middleware corresponding to the second domain.
[0092] Specifically, the three-party message sending interface of the message middleware corresponding to the first domain is called, and the first message parameter is delivered through the interface; the three-party message sending interface of the message middleware corresponding to the second domain is called, and the second message parameter is delivered through the interface.
[0093] Message delivery utilizes the message-based middleware's three-party messaging interface, enabling asynchronous message processing. Services are decoupled through messaging, so a failure in one service won't affect the normal operation of others. For example, even if the inventory management service temporarily fails, payment messages can still be delivered normally. These messages can then be processed once the inventory management service is restored, improving the system's scalability and fault tolerance.
[0094] Specifically, in this embodiment, it is preferred to use the after-commit hook of the database transaction to asynchronously trigger message delivery.
[0095] The atomic delivery method of database transactions and asynchronous messages disclosed in this application is based on the atomic submission method of database transactions and asynchronous messages, which ensures the consistency of database operations and message publishing. Messages are published only when the database transaction is successfully submitted, avoiding the problem of inconsistency between data and messages and ensuring the accuracy and integrity of system data. By converting the message content into parameters supported by different message middleware, the system can adapt to a variety of message middleware, and users can choose appropriate message middleware according to actual needs, thereby improving the compatibility and flexibility of the system. Using asynchronous message delivery, each service communicates through messages to achieve decoupling. Changes in one service will not directly affect other services. The system can easily add, modify or delete services, thereby improving the scalability of the system. At the same time, when a service fails, it will not affect the normal operation of the entire system, thereby improving the fault tolerance of the system.
[0096] In this embodiment, developers do not need to write message table operation code, message publishing is decoupled from business logic, and the amount of code is reduced; delivery is triggered immediately after transaction submission (non-polling), and the first delivery delay is reduced from seconds to milliseconds.
[0097] Furthermore, after the first message parameter and the second message parameter are delivered, the method further includes: determining whether the message is delivered successfully according to a result of whether the call of the three-party message sending interface is successful;
[0098] When message delivery fails, the asynchronous daemon strategy is called for redelivery.
[0099] In the previous steps, we've created order data, written message data to the database, and triggered message delivery after the database transaction is committed. However, message delivery can fail for various reasons, such as network fluctuations or temporary failures in the message queue service. Therefore, we need to introduce an asynchronous daemon process to perform retries to ensure that messages are ultimately successfully sent to a message queue like RocketMQ, thereby ensuring the smooth operation of the business process.
[0100] Specific reference Figure 3 , calling the asynchronous daemon strategy for redelivery, specifically including:
[0101] A100, in response to the message delivery failure instruction, calls the distributed lock scan to obtain the content of all messages that failed to be delivered in the preset historical period, and records them as target messages.
[0102] When a Pod starts processing a batch of messages by acquiring a distributed lock, other Pods will not process the same batch of messages because they cannot acquire the lock, thus ensuring the uniqueness of message processing.
[0103] For example, in an e-commerce system, after a user places an order, the system needs to send a message to the inventory service to deduct inventory. When a message middleware (such as Kafka) attempts to send this message to the inventory service, network fluctuations may cause the message to fail to be delivered. The message middleware will issue a message delivery failure instruction. At this point, the system will invoke a distributed lock, assuming that a distributed lock is implemented using Redis. The system uses this distributed lock to lock access to the message record database, preventing multiple processes from scanning the same data simultaneously. The system then scans the content of all failed delivery messages within a preset historical period (for example, the past hour). These messages may be stored in a dedicated message record table. Each record contains detailed information about the message, such as the message content, target service, delivery time, and delivery status. The system will filter out messages with a failed delivery status and mark them as target messages.
[0104] Using distributed locks can prevent multiple processes from scanning and processing the same failed message at the same time, prevent duplicate delivery, and ensure data consistency and accuracy. By scanning messages within a preset historical period, recent failed messages can be processed centrally to avoid missing important business messages. At the same time, outdated messages will not be processed, thereby improving processing efficiency.
[0105] A200 , in response to the acquired instruction of the target message, synchronously triggering the redelivery of the target message.
[0106] Once the target message is received, it is immediately triggered to redeliver, enabling timely recovery of failed business processes. For example, in e-commerce scenarios, timely redelivery of inventory reduction messages ensures timely inventory updates and prevents overselling. Furthermore, synchronous triggering reduces message processing delays, improving system responsiveness and business processing efficiency.
[0107] The A100-A200 solution significantly increases the likelihood of successful message delivery by re-delivering failed messages. In complex distributed systems, message delivery can fail due to various factors, such as network failures and temporary service unavailability. This re-delivery mechanism can overcome these issues to a certain extent, ensuring the smooth operation of business processes. Many business scenarios rely on the accurate delivery of messages to ensure data consistency. For example, in e-commerce systems, the accurate delivery of order messages, inventory messages, and payment messages is crucial for order processing, inventory management, and financial settlement. When message delivery fails, timely re-delivery can avoid data inconsistencies caused by message loss or errors, improving system reliability and stability. This solution automatically re-delivers failed messages, reducing the need for manual intervention. Administrators no longer need to manually locate and process each failed message; the system automatically completes these tasks, improving operational efficiency and reducing labor costs. Furthermore, the system can uniformly process failed messages based on pre-set rules, avoiding errors that can be caused by manual operation.
[0108] Furthermore, when the daemon retries and successfully sends a message to a message queue like RocketMQ, the status field of the corresponding message record in the tx_message table is marked as SENT, indicating that the message was successfully sent. If the message still fails to send, the status field is marked as FAILED, and the number of retries is recorded. The retry count is incremented by 1 for each failed retry, allowing for subsequent handling strategies based on the number of retries.
[0109] Further references Figure 4 , the method for synchronously triggering the redelivery of the target message in A200 specifically includes:
[0110] A201, obtain the message ID of each target message, and perform a hash operation on the message ID to obtain a hash value.
[0111] The hash operation can convert different message IDs into a fixed-length hash value. This hash value can be evenly distributed within a larger numerical range. Through the hash value, messages can be grouped and processed more conveniently later, avoiding the complexity and unevenness that may be caused by directly using the message ID.
[0112] A202, divides the hash value by the preset number of virtual shards to obtain a remainder.
[0113] When the preset number of virtual shards is 1024, the remainder ranges from 0 to 1023. In other words, all messages are logically divided into 1024 intervals in this way. Each interval is a virtual shard. The advantage of this is that messages can be evenly distributed to different virtual shards to avoid data skew.
[0114] A203, based on the remainder, determine the fragment number corresponding to each target message.
[0115] Specifically, the number of all processors is determined, and the remainder modulo the number of processors is calculated. The result is the physical shard number corresponding to the virtual shard. For example, assuming there are three processors numbered 0, 1, and 2, the remainder is divided by 3 and the modulo result (0, 1, or 2) is the physical shard number corresponding to the virtual shard. In this way, 1024 virtual shards are mapped to three physical shards. As this example shows, different messages are assigned to different virtual shards through hashing and modulo operations, and then mapped to different physical shards. This evenly distributes data across multiple processors, improving system processing power and concurrency. The hashing and modulo operations evenly distribute messages across different physical shards, preventing any particular processor from being overloaded. When adding or removing processors, simply adjust the mapping rules without requiring large-scale data migration.
[0116] A204, sends the target message to the processor corresponding to the shard number and performs redelivery.
[0117] Sending messages to the corresponding processors for redelivery enables parallel message processing. Different processors can simultaneously process messages from different shards, greatly improving the efficiency of message redelivery. This sharded processing approach also facilitates system monitoring and maintenance. If a processor fails, only the corresponding shard is affected, without disrupting the normal operation of the entire system.
[0118] Furthermore, the atomic delivery method for database transactions and asynchronous messages disclosed in the present application also includes: obtaining the total number of redelivery times of the target message, dynamically adjusting the scanning frequency according to the total number of redelivery times, and triggering the redelivery of the message.
[0119] Reference Figure 5 , a method for dynamically adjusting the scanning frequency and triggering message redelivery based on the total number of redelivery times, specifically including:
[0120] B110: When the total number of redelivery times meets the first condition, the corresponding first update strategy is called, which is recorded as the target strategy.
[0121] When the total number of redelivery times meets the first condition, the corresponding first update strategy is called, including: when 1≤total number of redelivery times≤3, the first update strategy called includes: preset historical period = 2 n-1 *Initial preset interval.
[0122] When the number of redelivery attempts is small, the delivery failure may be caused by occasional network fluctuations or temporary service failures. By shortening the preset historical period, messages with recent delivery failures can be scanned more frequently and redelivery attempts can be made, increasing the probability of successful message delivery and accelerating the recovery of business processes.
[0123] B120: When the total number of redelivery times meets the second condition, the corresponding second update strategy is called, which is recorded as the target strategy.
[0124] When the total number of redelivery times meets the second condition, calling the corresponding second update strategy includes: when 4≤total number of redelivery times≤5, calling the second update strategy includes: preset historical period=5*initial preset interval.
[0125] B130, when the total number of redelivery times meets the third condition, call the corresponding third update strategy, which is recorded as the target strategy.
[0126] When the total number of redelivery times satisfies the third condition, the corresponding third update strategy is called, including: when 6≤total number of redelivery times≤15, the preset historical period=10*initial preset interval.
[0127] When the number of redelivery is high, there may be some underlying problems, such as a serious failure of the target service. Extending the preset historical period can reduce the number of unnecessary scans, lower system resource consumption, and reduce the number of messages obtained in each scan. This allows the system to have more time to process each message and avoid excessive and ineffective redelivery when the target service has not recovered.
[0128] B140, updates the preset historical period based on the target strategy.
[0129] By updating the preset historical period, the system can dynamically adjust the scanning range according to the message redelivery situation to adapt to different failure scenarios, while ensuring message processing efficiency and rationally utilizing system resources.
[0130] B150, calling the distributed lock to scan all the messages that failed to be delivered within the preset historical period after the update, recording them as target messages, and synchronously triggering the redelivery of the message in response to the acquired instruction of the target message.
[0131] Using distributed locks can prevent multiple processes from scanning the same data simultaneously, avoiding duplicate processing; scanning messages according to the updated preset historical period can ensure that the system focuses on failed messages within the appropriate time period for redelivery, improving the targetedness and efficiency of message processing.
[0132] Furthermore: the scanning frequency is dynamically adjusted according to the number of retries. In order to avoid excessive pressure on the system caused by high-frequency retries, the daemon process will dynamically adjust the scanning frequency according to the number of message retries.
[0133] Furthermore, the atomic delivery method for database transactions and asynchronous messages disclosed in this application also includes: when the total number of redelivery times is greater than a preset threshold, the corresponding target message is moved to a dead letter queue and an alarm is triggered; the dead letter queue is used to store messages that cannot be processed normally.
[0134] Assume that the system's preset redelivery threshold is 10 times. After the user places an order, the system will send a message to the inventory service to deduct the inventory. However, due to continuous failures in the inventory service, the message failed to be delivered multiple times. The system will record the total number of redeliveries of the message. When the total number of redeliveries reaches 11 times, the condition of "the total number of redeliveries is greater than the preset threshold" is met. The system will remove the inventory deduction message from the original message processing process and move it to the dead letter queue. The dead letter queue is like a "special warehouse" that stores messages that cannot be processed normally. At the same time, the system's alarm mechanism will be triggered, such as notifying the relevant operation and maintenance personnel or developers through email, SMS, or the system's internal alarm platform, informing them that a message has entered the dead letter queue and requires timely attention and processing.
[0135] Separating messages that fail to process properly from the normal message processing flow prevents these abnormal messages from continuing to occupy normal message processing resources and impacting the delivery and processing efficiency of other normal messages. For example, in an e-commerce system, if messages regarding failed inventory deductions are not moved to a dead letter queue, they may be repeatedly retried, consuming system resources such as network bandwidth and CPU, slowing down the processing of other normal order messages. The dead letter queue centrally stores all messages that fail to process properly, facilitating subsequent manual investigation and resolution. Operations and maintenance personnel or developers can directly view detailed information about these messages, such as message content, delivery history, and failure reasons, in the dead letter queue, allowing them to quickly identify the issue and implement appropriate resolution measures. Promptly detecting and resolving abnormal messages can reduce business losses caused by message delivery failures. For example, in a financial system, if a transfer message fails to be delivered, a timely alert allows staff to address it promptly, avoiding delays or errors in funds and ensuring normal business operations.
[0136] Furthermore, the atomic delivery method of database transactions and asynchronous messages disclosed in the present application also includes: when the updated preset historical period is greater than the preset interval, setting the updated preset historical period as the preset interval.
[0137] If the updated preset historical period is not limited, the scanning period may be too long. An excessively long scanning period means that the system may not process failed messages until a long time has passed. This will cause serious delays in the redelivery of failed messages and affect the timeliness of business. A shorter scanning period allows the system to detect and process failed messages more promptly, enhancing the system's responsiveness to abnormal situations. When a message delivery fails, the system can attempt to redeliver it again in a relatively short period of time, increasing the probability of successful message delivery and enabling the system to recover from failures more quickly, ensuring normal system operation. In some complex failure scenarios, policy adjustments may cause the preset historical period to become extremely long. This extreme situation may negatively impact system stability. For example, an excessively long scanning period may prevent the system from detecting and processing urgent failed messages in a timely manner, leading to the accumulation and exacerbation of problems. By limiting the updated preset historical period to the preset interval, this extreme situation can be avoided, enhancing the stability and reliability of the system and ensuring its continuous and stable operation.
[0138] In a third aspect, the present application discloses a system for atomically submitting database transactions and asynchronous messages. Based on the atomic submission method for database transactions and asynchronous messages disclosed in the first aspect of the present application, the system includes:
[0139] A parsing module, configured to respond to a first category of information in a first field initiated by a user in a target business scenario, and parse first message content and first business data in the first category of information;
[0140] The calling module is used to call the preset transaction message package; the preset transaction message package is configured with a target function, several message middlewares, a business data writing interface, and a business message publishing interface;
[0141] A creation module is used to create a database transaction through a target function; the database transaction is configured with a plurality of demand strategies in the second domain;
[0142] A request generation module, configured to generate, in a database transaction, a first write request for writing the first message content and the first business data into a target business database;
[0143] a demand policy calling module, configured to call a corresponding demand policy in the second domain according to the first business data in a database transaction in response to a generation instruction of the first write request, which is recorded as a target demand policy;
[0144] a verification module, configured to verify the identity information of the user according to the target demand policy, and when the identity information passes the verification, execute the target demand policy and generate the second type of information, the second type of information including the second message content and the second business data;
[0145] A first submission module is used to submit the second type of information to the target business database through the business message writing interface;
[0146] The second submission module is configured to trigger the execution of the first write request in response to a successful submission instruction of the second type of information to submit the first type of information to the target business database.
[0147] In a fourth aspect, the present application discloses an atomic delivery system for database transactions and asynchronous messages. Based on the atomic delivery system for database transactions and asynchronous messages disclosed in the third aspect, the system further includes:
[0148] An interface call request initiating module, configured to initiate a request for calling a business message publishing interface in response to a successful submission instruction for submitting the first type of information to the target business database;
[0149] a conversion module, configured to, in response to a call request of the business message publishing interface, call the message middleware corresponding to the first domain to convert the first message content into a first message parameter supported by the corresponding middleware, and call the message middleware corresponding to the second domain to convert the second message content into a second message parameter supported by the corresponding middleware;
[0150] A first delivery module is configured to deliver the first message parameter through a three-party message sending interface of the message middleware corresponding to the first domain;
[0151] The second delivery module is used to deliver the second message parameters through the three-party message sending interface of the message middleware corresponding to the second domain.
[0152] Specifically, in this embodiment, the atomic delivery system for database transactions and asynchronous messages disclosed in this application includes the atomic delivery system for database transactions and asynchronous messages disclosed in the third aspect and the above modules.
[0153] A computer device according to an embodiment of the present disclosure includes a memory and a processor. The memory is used to store non-transitory computer-readable instructions. Specifically, the memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc.
[0154] The processor can be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and can control other components in the computer device to perform desired functions. In one embodiment of the present disclosure, the processor is used to execute the computer-readable instructions stored in the memory, causing the computer device to execute all or part of the steps of the aforementioned methods for atomic committing database transactions and asynchronous messages or the methods for atomic delivery of database transactions and asynchronous messages in various embodiments of the present disclosure.
[0155] Those skilled in the art should understand that in order to solve the technical problem of how to obtain a good user experience, this embodiment may also include well-known structures such as a communication bus and an interface, and these well-known structures should also be included in the scope of protection of this disclosure.
[0156] like Figure 6 The present invention provides a schematic diagram of the structure of a computer device according to an embodiment of the present invention. Figure 6 The computer device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.
[0157] like Figure 6 As shown, a computer device may include a processor (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) or programs loaded from a storage device into a random access memory (RAM). The RAM also stores various programs and data required for the operation of the computer device. The processor, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.
[0158] Typically, the following devices can be connected to the I / O interface: input devices such as sensors or visual information acquisition devices; output devices such as display screens; storage devices such as tapes and hard disks; and communication devices. The communication device can allow the computer device to communicate with other devices (such as edge computing devices) wirelessly or by wire to exchange data. Figure 6 A computer device having various devices is shown, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.
[0159] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processor, all or part of the steps of the atomic submission method of database transactions and asynchronous messages or the atomic delivery method of database transactions and asynchronous messages of the embodiment of the present disclosure are executed.
[0160] For detailed description of this embodiment, please refer to the corresponding description in the aforementioned embodiments, which will not be repeated here.
[0161] According to an embodiment of the present disclosure, a computer-readable storage medium stores non-transitory computer-readable instructions. When the non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the aforementioned methods for atomically committing database transactions and asynchronous messages or the methods for atomically delivering database transactions and asynchronous messages according to various embodiments of the present disclosure are performed.
[0162] The above-mentioned computer-readable storage media include, but are not limited to, optical storage media (e.g., CD-ROMs and DVDs), magneto-optical storage media (e.g., MOs), magnetic storage media (e.g., magnetic tapes or mobile hard disks), media with built-in rewritable non-volatile memory (e.g., memory cards), and media with built-in ROM (e.g., ROM cartridges).
[0163] For detailed description of this embodiment, please refer to the corresponding description in the aforementioned embodiments, which will not be repeated here.
[0164] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this disclosure are merely illustrative and not restrictive, and should not be construed as necessarily possessed by each embodiment of the present disclosure. Furthermore, the specific details disclosed above are provided for illustrative purposes and to facilitate understanding, rather than as limitations. These details do not limit the present disclosure to necessarily being implemented using these specific details.
[0165] In the present disclosure, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. The block diagrams of the devices, devices, equipment, and systems involved in the present disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "including," "comprising," "having," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.
[0166] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.
[0167] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.
[0168] Various changes, substitutions, and modifications may be made to the technology described herein without departing from the teachings defined by the appended claims. Moreover, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of things, means, methods, and actions described above. Currently existing or later developed processes, machines, manufactures, compositions of things, means, methods, or actions that perform substantially the same function or achieve substantially the same results as the corresponding aspects described herein may be utilized. Accordingly, the appended claims include within their scope such processes, machines, manufactures, compositions of things, means, methods, or actions.
[0169] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.
[0170] The above description has been provided for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.
Claims
1. A method for atomically committing database transactions and asynchronous messages, characterized in that: include: In response to a first type of information in a first field initiated by a user in a target business scenario, parsing first message content and first business data in the first type of information; Call the preset transaction message package; The preset transaction message package is configured with a target function, several message middlewares, a business data writing interface and a business message publishing interface; Creating a database transaction through the target function; configuring a plurality of demand strategies in the second domain in the database transaction; Generating, in the database transaction, a first write request for writing the first message content and the first service data into a target service database; In response to the instruction to generate the first write request, in the database transaction, calling a corresponding demand policy in the second domain according to the first business data, which is recorded as a target demand policy; Verifying the identity information of the user according to the target demand policy, and when the identity information passes the verification, executing the target demand policy and generating second type of information, the second type of information including second message content and second business data; Submitting the second type of information to the target business database through the business message writing interface; In response to a successful submission instruction of the second type of information, the execution of the first write request is triggered to submit the first type of information to the target business database.
2. The atomic commit method for database transactions and asynchronous messages according to claim 1, characterized in that: The target business scenario is a scenario including data storage and message sending; The target business scenarios include insurance business scenarios, e-commerce business scenarios, banking and finance scenarios, or logistics and warehousing scenarios; The service message publishing interface has the function of shielding API differences between different message middleware.
3. A method for atomic delivery of database transactions and asynchronous messages, characterized in that: include: In response to a first type of information in a first field initiated by a user in a target business scenario, parsing first message content and first business data in the first type of information; Call the target function in the preset transaction message package to create a database transaction; The database transaction is configured with several demand strategies in the second domain; Generating, in the database transaction, a first write request for writing the first message content and the first service data into a target service database; In response to the instruction to generate the first write request, in the database transaction, calling a corresponding demand policy in the second domain according to the first business data, which is recorded as a target demand policy; Verifying the identity information of the user according to the target demand policy, and when the identity information passes the verification, executing the target demand policy and generating second type of information, the second type of information including second message content and second business data; Submitting the second type of information to the target business database through a business message writing interface; In response to a successful submission instruction for the second type of information, triggering execution of the first write request to submit the first type of information to the target business database, initiating a request to call a business message publishing interface after the submission is successful, and based on the call request of the business message publishing interface, calling the message middleware corresponding to the first domain to convert the first message content into a first message parameter supported by the corresponding middleware, and calling the message middleware corresponding to the second domain to convert the second message content into a second message parameter supported by the corresponding middleware; Delivering the first message parameters through the three-party message sending interface of the message middleware corresponding to the first domain; The second message parameter is delivered through the three-party message sending interface of the message middleware corresponding to the second domain.
4. The atomic delivery method for database transactions and asynchronous messages according to claim 3, characterized in that: After the first message parameter and the second message parameter are delivered, the method further includes: determining whether the message is delivered successfully according to whether the call of the three-party message sending interface is successful; When message delivery fails, the asynchronous daemon strategy is called for redelivery.
5. The atomic delivery method for database transactions and asynchronous messages according to claim 4, characterized in that: When the message delivery fails, the asynchronous daemon process strategy is called to redeliver the message, including: In response to the message delivery failure instruction, a distributed lock scan is called to obtain the content of all messages that failed to be delivered in a preset historical period, which are recorded as target messages; In response to the acquired instruction of the target message, the redelivery of the target message is synchronously triggered.
6. The atomic delivery method for database transactions and asynchronous messages according to claim 5, characterized in that: The synchronously triggering the redelivery of the target message includes: Obtain the message ID of each target message, and perform a hash operation on the message ID to obtain a hash value; Divide the hash value by the preset number of virtual shards to obtain a remainder; Determine, according to the remainder, a fragment number corresponding to each of the target messages; The target message is sent to the processor corresponding to the fragment number and redelivered.
7. The atomic delivery method for database transactions and asynchronous messages according to claim 5, characterized in that: Also includes: Obtaining the total number of redelivery times of the target message, dynamically adjusting the scanning frequency according to the total number of redelivery times and triggering the redelivery of the message; The dynamically adjusting the scanning frequency according to the total number of redelivery times and triggering the redelivery of the message includes: When the total number of redelivery times meets the first condition, calling the corresponding first update strategy, which is recorded as the target strategy; When the total number of redelivery times meets the second condition, calling the corresponding second update strategy, which is recorded as the target strategy; When the total number of redelivery times satisfies the third condition, calling the corresponding third update strategy, which is recorded as the target strategy; Updating the preset historical period based on the target strategy; The distributed lock is called to scan all the messages that failed to be delivered within the preset historical period after the update, and recorded as target messages. In response to the acquired instruction of the target message, the message redelivery is synchronously triggered.
8. The atomic delivery method for database transactions and asynchronous messages according to claim 7, characterized in that: When the total number of redelivery times meets the first condition, calling the corresponding first update strategy includes: when 1≤the total number of redelivery times≤3, calling the first update strategy includes: preset historical period=2 n-1 * Initial preset interval; When the total number of redeliveries satisfies the second condition, calling the corresponding second update strategy includes: when 4≤the total number of redeliveries≤5, calling the second update strategy includes: preset historical period=5*initial preset interval; When the total number of redelivery times satisfies a third condition, calling a corresponding third update strategy includes: when 6≤the total number of redelivery times≤15, the preset historical period=10*initial preset interval.
9. The atomic delivery method for database transactions and asynchronous messages according to claim 8, characterized in that: When the total number of redelivery attempts exceeds a preset threshold, the corresponding target message is moved to a dead letter queue and an alarm is triggered. The dead letter queue is used to store messages that cannot be processed normally.
10. The atomic delivery method for database transactions and asynchronous messages according to claim 7, characterized in that: When the updated preset historical period is greater than the preset interval, the updated preset historical period is defined as the preset interval.
11. A database transaction and asynchronous message atomic submission system, characterized in that: include: a parsing module, configured to respond to a first category of information in a first field initiated by a user in a target business scenario, and parse first message content and first business data in the first category of information; A calling module, used to call a preset transaction message package; The preset transaction message package is configured with a target function, several message middlewares, a business data writing interface and a business message publishing interface; A creation module, configured to create a database transaction using the target function; the database transaction is configured with a plurality of demand strategies in the second domain; a request generating module, configured to generate, in the database transaction, a first write request for writing the first message content and the first business data into a target business database; a demand policy calling module, configured to call a corresponding demand policy in the second domain according to the first business data in the database transaction in response to a generation instruction of the first write request, which is recorded as a target demand policy; a verification module, configured to verify the identity information of the user according to the target demand policy, and when the identity information passes the verification, execute the target demand policy and generate second-category information, wherein the second-category information includes second message content and second service data; A first submission module, configured to submit the second type of information to the target business database through the business message writing interface; The second submission module is configured to trigger the execution of the first write request in response to a successful submission instruction of the second type of information to submit the first type of information to the target business database.
12. An atomic delivery system for database transactions and asynchronous messages, characterized in that: include: a content data parsing module, configured to respond to first-category information of a first field initiated by a user in a target business scenario and parse first message content and first business data in the first-category information; The execution module is used to call the target function in the preset transaction message package to create a database transaction; The database transaction is configured with several demand strategies in the second domain; A first write request generating module, configured to generate, in the database transaction, a first write request for writing the first message content and the first business data into a target business database; a target demand policy acquisition module, configured to, in response to a generation instruction of the first write request, invoke a corresponding demand policy in the second domain according to the first business data in the database transaction, which is recorded as a target demand policy; A second type of information generating module is configured to verify the identity information of the user according to the target demand policy, and when the identity information passes the verification, execute the target demand policy and generate the second type of information, wherein the second type of information includes the second message content and the second service data; A second type of information submission module, configured to submit the second type of information to the target business database through a business message writing interface; an interface call request initiating module, configured to trigger, in response to a successful submission instruction of the second type of information, execution of the first write request to submit the first type of information to the target business database, and initiate a request to call a business message publishing interface after successful submission; a conversion module, configured to, in response to a call request of the business message publishing interface, call the message middleware corresponding to the first domain to convert the first message content into a first message parameter supported by the corresponding middleware, and call the message middleware corresponding to the second domain to convert the second message content into a second message parameter supported by the corresponding middleware; A first delivery module, configured to deliver the first message parameters through a three-party message sending interface of a message middleware corresponding to the first domain; The second delivery module is used to deliver the second message parameters through the three-party message sending interface of the message middleware corresponding to the second domain.
13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, which are used to enable a computer to execute the atomic submission method of database transactions and asynchronous messages described in any one of claims 1-2 or the atomic delivery method of database transactions and asynchronous messages described in any one of claims 3-10.
Citation Information
Patent Citations
A distributed transaction processing method based on distributed message middleware
CN109933412A
Distributed transaction processing method based on message queue, computer equipment and medium
CN119847786A