API adaptation method and system supporting non-stop switching of heterogeneous message middleware

Abstract

The invention provides an API (Application Program Interface) adaptation method and system supporting non-stop switching of heterogeneous message middleware, which are used for solving the problems of complex switching, high risk, service interruption and the like when RabbitMQ (RabbitMQ) is switched to Rocket MQ. An adaptation layer is arranged between an application system and message middleware, and operation of unified API packaging on two kinds of message middleware is provided. And the adaptation layer dynamically selects the message middleware to send and consume the message, supports a double-monitoring mode, and ensures reliable transmission of the message in the switching process. The switching process is completed step by step in stages, and the adaptation layer comprises a producer module, a consumer module and a message format conversion module and can analyze, convert and serialize message formats to adapt to different middleware. The system further comprises a configuration management system and a monitoring module, and dynamic configuration management and real-time monitoring are achieved. According to the method, the switching workload is remarkably reduced, the risk is reduced, the service continuity is guaranteed, and remarkable economic and social benefits are achieved.

Description

Technical Field

[0001] This invention belongs to the field of message middleware technology, and particularly relates to an API adaptation method and system that supports switching heterogeneous message middleware without downtime. Background Technology

[0002] As business volume continues to grow, message queues are playing an increasingly important role in distributed systems. RabbitMQ, a widely used open-source message queue system, has won the favor of many enterprises due to its high timeliness and stability. However, with the expansion of business scale, RabbitMQ has gradually revealed its limitations in terms of throughput, cluster support, and functional features. In contrast, RocketMQ, with its high throughput, high availability, and rich functional features, has become a better choice.

[0003] The transition from RabbitMQ to RocketMQ for enterprise users typically involves the following steps: business system code modification, message middleware switching, and application verification. Among these steps, the most labor-intensive is often not the business system code modification, but the message middleware switching process. During the switch, it's crucial to ensure that both upstream and downstream systems have been modified and that messages can be reliably transmitted, which consumes significant time and manpower. This process introduces considerable resistance to the business cutover and deployment, and may even prevent the task from proceeding smoothly. Summary of the Invention

[0004] (a) Purpose of the invention To overcome the above shortcomings, the purpose of this invention is to provide an API adaptation method and system that supports non-stop switching of heterogeneous message middleware, so as to solve the above technical problems.

[0005] (II) Technical Solution To achieve the above objectives, the technical solution provided in this application is as follows: An API adaptation method that supports seamless switching of heterogeneous message middleware includes the following steps: S1 sets up an adaptation layer between the application system and the message middleware. The adaptation layer provides a unified API interface to encapsulate operations on the first message middleware and the second message middleware. In the adaptation layer, S2 dynamically selects whether to use the first message middleware or the second message middleware for message sending and consumption based on the configuration. During the switching process, the S3 adaptation layer supports dual listening mode, simultaneously listening to messages from the first message middleware and the second message middleware; S4 gradually switches the target middleware for message sending and consumption according to preset conditions, and finally completes the non-stop switch from the first message middleware to the second message middleware. The switch process includes the following stages: Phase 1: The message is sent to the first message middleware, and the adaptation layer listens for messages from both the first and second message middleware. Second stage: The message is sent to the second message middleware. The adaptation layer continues to listen to messages in both the first and second message middleware simultaneously until all messages in the first message middleware have been consumed. Phase 3: The adaptation layer stops listening to the first message middleware and only listens to messages from the second message middleware. Phase 4: Remove code and configuration related to the first message middleware.

[0006] Preferably, the adapter layer includes: The producer module is used to send messages to the first message middleware or the second message middleware according to the configuration. The consumer module is used to listen to messages from both the first and second message middleware simultaneously, and dynamically switch the message middleware to consume based on the configuration. The message format conversion module is used to convert message formats between different message middleware.

[0007] Preferably, the adaptation layer dynamically obtains the configuration information of the message middleware through the configuration management system, and automatically switches the message middleware according to the configuration information. The first message middleware is RabbitMQ, and the second message middleware is RocketMQ.

[0008] Preferably, the adaptation layer further includes a monitoring module for monitoring the running status of the message middleware and the sending and consumption of messages.

[0009] Preferably, the switching method further includes the following preliminary preparation steps: Review the existing application systems and clarify the message middleware and business logic used by each application system; Design a message middleware adapter layer architecture to encapsulate operations on the first and second message middleware; The adaptation layer implements client integration for the first and second message middleware and provides a unified API interface. Add message format conversion logic to the adaptation layer to ensure message format compatibility between different message middleware; Design a configuration management system that supports dynamic modification of message middleware configuration information.

[0010] Preferably, the message format conversion module performs message format conversion using the following steps: (1) Parse the message format of the first message middleware: Receive a message from the first message middleware, the message including a message header, message body, timestamp, and message type; Use regular expressions or predefined parsing templates to parse the message and extract key fields; The extracted key fields are stored as intermediate data structures for subsequent transformation processing; (2) Perform field conversion according to the preset mapping rules; Load a preset mapping rule table, which defines the correspondence between the key fields of the first message middleware and the fields supported by the second message middleware. The mapping rule table can be a configuration file, a database table, or a data structure in memory. Traverse each key field in the intermediate data structure, use a hash table or search tree algorithm to quickly match the field name, and convert it into a field format supported by the second message middleware according to the mapping rules; For fields that cannot be directly mapped, the following algorithm is used for processing: Check if the field is an optional field. If it is an optional field and there is no corresponding mapping, fill it with the default value. If a field is required and there is no corresponding mapping, the error handling mechanism is triggered, the log is recorded and the error information is returned; For fields that require data type conversion, use a type conversion algorithm to ensure that the converted fields meet the format requirements of the second message middleware. (3) The message after serialization: The transformed fields are organized into a message structure supported by the second message middleware, which can be JSON, XML, Protobuf or other custom binary formats; The message is serialized using a serialization algorithm, which includes, but is not limited to: For JSON format, a recursive traversal algorithm is used to convert the intermediate data structure into a JSON string; For the Protobuf format, the serialization code generated by the Protobuf compiler is used to serialize the intermediate data structure into binary format; During the serialization process, the message is verified to ensure its integrity and correctness. The verification algorithm includes, but is not limited to, checksum algorithm, hash algorithm or custom verification logic. The serialized message is sent to the second message middleware, completing the entire message format conversion process.

[0011] An API adaptation system that supports seamless switching of heterogeneous message middleware includes: The adaptation layer is set up between the application system and the message middleware, encapsulating the operations on the first and second message middleware, and providing a unified API interface to the upper layer. A configuration management system is used to dynamically manage the configuration information of the message middleware; The monitoring module is used to monitor the running status of the message middleware and the sending and consumption of messages.

[0012] Preferably, the adapter layer includes: The producer module is used to send messages to the first message middleware or the second message middleware according to the configuration. The consumer module is used to listen to messages from both the first and second message middleware simultaneously, and dynamically switch the message middleware to consume based on the configuration. The message format conversion module is used to convert message formats between different message middleware.

[0013] Preferably, the configuration management system supports dynamically modifying the configuration information of the message middleware and triggering the adaptation layer to automatically switch the message middleware.

[0014] Preferably, the monitoring module includes: The message queue monitoring unit is used to monitor the message queue status of the message middleware. The performance monitoring unit is used to monitor performance metrics such as latency and throughput for message sending and consumption. The alarm unit is used to issue alarm notifications when monitoring indicators are abnormal.

[0015] Beneficial effects: 1. Reduced switching workload: By introducing an adaptation layer, application systems do not need to interact directly with the message middleware. Instead, they send and consume messages through a unified API interface. This significantly reduces the workload of modifying business system code and lowers development and testing costs.

[0016] 2. Reduced Switching Risk: The adaptation layer supports a dual-listening mode, simultaneously monitoring messages from two message brokers during the switching process to ensure messages are not lost or consumed repeatedly. This mechanism effectively reduces the risks during switching and improves system reliability.

[0017] 3. Ensure business continuity: This invention enables seamless switching without downtime. Through dynamic configuration management and a gradual switching process, it ensures that services are not affected during the switching process, thus avoiding business interruptions caused by downtime.

[0018] 4. Improved system flexibility and scalability: The adaptation layer can dynamically select different message middleware based on configuration, supporting dynamic switching at runtime without downtime. This design is not only applicable to switching from RabbitMQ to RocketMQ, but can also be extended to switching other message middleware, improving system flexibility and scalability. Attached Figure Description

[0019] Figure 1 This is a flowchart of the first stage of the present invention; Figure 2 This is a flowchart of the second stage according to an embodiment of the present invention; Figure 3 This is a flowchart of the third stage according to one embodiment of the present invention; Figure 4 This is a flowchart of the fourth stage according to one embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the following detailed embodiments are described in conjunction with the appendix. Figure 1-4 The present invention will be described in further detail below. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0021] This invention provides an API adaptation method that supports seamless switching of heterogeneous message middleware, comprising the following steps: S1 sets up an adaptation layer between the application system and the message middleware. The adaptation layer provides a unified API interface to encapsulate the operation of the first message middleware and the second message middleware. The application system does not need to interact directly with the message middleware, but sends and consumes messages through the unified API interface, which reduces the large amount of code modification work caused by switching message middleware. The existence of the adaptation layer makes it easier to replace other message middleware in the future. Only adjustments need to be made in the adaptation layer, without modifying the application system code. In the adaptation layer, S2 dynamically selects either the first or second message middleware for message sending and consumption based on the configuration. It can dynamically switch message middleware at runtime without downtime, reducing the impact of switching on business. During the switching process, the message sending and consumption targets can be gradually adjusted according to actual needs to ensure that business logic is not affected. During the handover process, the S3 adaptation layer supports a dual-listening mode, simultaneously listening to messages from the first and second message middleware to ensure that messages are not lost or consumed repeatedly. Through the dual-listening mechanism, potential problems can be detected and handled in a timely manner during the handover process, reducing the handover risk. S4 gradually switches the target middleware for message sending and consumption according to preset conditions, ultimately completing a non-stop switch from the first message middleware to the second message middleware. This phased, gradual switch ensures that the switching conditions and operation steps for each stage are clear, reducing switch risks, improving switch reliability, and minimizing the impact on business operations. The switch process includes the following stages: Phase 1: The message is sent to the first message middleware, and the adaptation layer listens for messages from both the first and second message middleware. Second stage: The message is sent to the second message middleware. The adaptation layer continues to listen to messages in both the first and second message middleware simultaneously until all messages in the first message middleware have been consumed. Phase 3: The adaptation layer stops listening to the first message middleware and only listens to messages from the second message middleware. Phase 4: Remove code and configuration related to the first message middleware.

[0022] Preferably, the adapter layer includes: The producer module is used to send messages to the first message middleware or the second message middleware according to the configuration. The consumer module is used to listen to messages from both the first and second message middleware simultaneously, and dynamically switch the message middleware to consume based on the configuration. The message format conversion module is used to convert message formats between different message middleware.

[0023] Preferably, the adaptation layer dynamically obtains the configuration information of the message middleware through the configuration management system, and automatically switches the message middleware according to the configuration information. The first message middleware is RabbitMQ, and the second message middleware is RocketMQ.

[0024] Preferably, the adaptation layer further includes a monitoring module for monitoring the running status of the message middleware and the sending and consumption of messages.

[0025] Preferably, the switching method further includes the following preliminary preparation steps: Review the existing application systems and clarify the message middleware and business logic used by each application system; Design a message middleware adapter layer architecture to encapsulate operations on the first and second message middleware; The adaptation layer implements client integration for the first and second message middleware and provides a unified API interface. Add message format conversion logic to the adaptation layer to ensure message format compatibility between different message middleware; Design a configuration management system that supports dynamic modification of message middleware configuration information.

[0026] By conducting thorough preliminary planning and design, we can ensure a smooth transition process, reduce problems caused by insufficient preparation, clarify the business logic of each application system and the usage of message middleware, and help formulate reasonable transition strategies and improve the reliability of the transition.

[0027] Preferably, the message format conversion module performs message format conversion using the following steps: (1) Parse the message format of the first message middleware: Receive a message from the first message middleware, the message including a message header, message body, timestamp, and message type; Use regular expressions or predefined parsing templates to parse the message and extract key fields; The extracted key fields are stored as intermediate data structures for subsequent transformation processing; (2) Perform field conversion according to the preset mapping rules; Load a preset mapping rule table, which defines the correspondence between the key fields of the first message middleware and the fields supported by the second message middleware. The mapping rule table can be a configuration file, a database table, or a data structure in memory. Traverse each key field in the intermediate data structure, use a hash table or search tree algorithm to quickly match the field name, and convert it into a field format supported by the second message middleware according to the mapping rules; For fields that cannot be directly mapped, the following algorithm is used for processing: Check if the field is an optional field. If it is an optional field and there is no corresponding mapping, fill it with the default value. If a field is required and there is no corresponding mapping, the error handling mechanism is triggered, the log is recorded and the error information is returned; For fields that require data type conversion, use a type conversion algorithm to ensure that the converted fields meet the format requirements of the second message middleware. (3) The message after serialization: The transformed fields are organized into a message structure supported by the second message middleware, which can be JSON, XML, Protobuf or other custom binary formats; The message is serialized using a serialization algorithm, which includes, but is not limited to: For JSON format, a recursive traversal algorithm is used to convert the intermediate data structure into a JSON string; For the Protobuf format, the serialization code generated by the Protobuf compiler is used to serialize the intermediate data structure into binary format; During the serialization process, the message is verified to ensure its integrity and correctness. The verification algorithm includes, but is not limited to, checksum algorithm, hash algorithm or custom verification logic. The serialized message is sent to the second message middleware, completing the entire message format conversion process.

[0028] By employing parsing and mapping steps, message format compatibility between different message middleware is ensured, avoiding message loss or errors caused by format incompatibility. Hash tables or search tree algorithms are used for field matching, improving the efficiency of field conversion. For fields that cannot be directly mapped, error handling mechanisms and default value filling are used to enhance the system's robustness.

[0029] An API adaptation system that supports seamless switching of heterogeneous message middleware includes: The adaptation layer is set up between the application system and the message middleware, encapsulating the operations on the first and second message middleware, and providing a unified API interface to the upper layer. A configuration management system is used to dynamically manage the configuration information of the message middleware; The monitoring module is used to monitor the running status of the message middleware and the sending and consumption of messages.

[0030] Preferably, the adapter layer includes: The producer module is used to send messages to the first message middleware or the second message middleware according to the configuration. The consumer module is used to listen to messages from both the first and second message middleware simultaneously, and dynamically switch the message middleware to consume based on the configuration. The message format conversion module is used to convert message formats between different message middleware.

[0031] Preferably, the configuration management system supports dynamically modifying the configuration information of the message middleware and triggering the adaptation layer to automatically switch the message middleware.

[0032] Preferably, the monitoring module includes: The message queue monitoring unit is used to monitor the message queue status of the message middleware. The performance monitoring unit is used to monitor performance metrics such as latency and throughput for message sending and consumption. The alarm unit is used to issue alarm notifications when monitoring indicators are abnormal.

[0033] The adaptation layer, configuration management system, and monitoring module work together to provide a complete non-stop switching solution. Through dynamic switching of the adaptation layer, flexible configuration of the configuration management system, and real-time monitoring by the monitoring module, the overall performance of the system is improved.

[0034] This invention significantly reduces the amount of code modification work required when switching message middleware by setting up an adaptation layer between the application system and the message middleware, providing a unified API interface to encapsulate operations on different message middleware. The adaptation layer supports a dual-listening mode, simultaneously monitoring messages from both message middleware during the switching process to ensure that messages are not lost or consumed repeatedly, thereby effectively reducing switching risks and improving system reliability. Through a dynamic configuration management system, this invention achieves non-stop switching, ensuring that business operations are unaffected during the switching process, avoiding business interruptions caused by downtime, and enhancing the system's flexibility and scalability to adapt to different business needs and message middleware switching. In addition, the modular design of the adaptation layer improves the system's maintainability and stability, while the real-time monitoring function of the monitoring module further ensures the reliable operation of the system. The message format conversion module ensures message format compatibility between different message middleware through parsing, mapping, and serialization steps, enhancing the system's robustness. Overall, this invention provides a complete non-stop switching solution, optimizes resource utilization, improves user experience, supports smooth transition, and ensures data integrity and consistency, providing enterprises with an efficient, reliable, and flexible message middleware switching solution with significant economic and social benefits.

[0035] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0036] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An API adaptation method supporting non-stop switching of heterogeneous message middleware, characterized in that, Includes the following steps: S1 sets up an adaptation layer between the application system and the message middleware. The adaptation layer provides a unified API interface to encapsulate operations on the first message middleware and the second message middleware. In the adaptation layer, S2 dynamically selects whether to use the first message middleware or the second message middleware for message sending and consumption based on the configuration. During the switching process, the S3 adaptation layer supports dual listening mode, simultaneously listening to messages from the first message middleware and the second message middleware; S4 gradually switches the target middleware for message sending and consumption according to preset conditions, and finally completes the non-stop switch from the first message middleware to the second message middleware. The switch process includes the following stages: Phase 1: The message is sent to the first message middleware, and the adaptation layer listens for messages from both the first and second message middleware. Second stage: The message is sent to the second message middleware. The adaptation layer continues to listen to messages in both the first and second message middleware simultaneously until all messages in the first message middleware have been consumed. Phase 3: The adaptation layer stops listening to the first message middleware and only listens to messages from the second message middleware. Phase 4: Remove code and configuration related to the first message middleware.

2. The API adaptation method for supporting non-stop switching of heterogeneous message middleware according to claim 1, characterized in that, The adapter layer includes: The producer module is used to send messages to the first message middleware or the second message middleware according to the configuration. The consumer module is used to listen to messages from both the first and second message middleware simultaneously, and dynamically switch the message middleware to consume based on the configuration. The message format conversion module is used to convert message formats between different message middleware.

3. The API adaptation method for supporting non-stop switching of inter-message middleware according to claim 1, characterized in that, The adaptation layer dynamically obtains the configuration information of the message middleware through the configuration management system, and automatically switches the message middleware according to the configuration information. The first message middleware is RabbitMQ, and the second message middleware is RocketMQ.

4. The API adaptation method for supporting non-stop switching of heterogeneous message middleware according to claim 1, characterized in that, The adaptation layer also includes a monitoring module, which is used to monitor the running status of the message middleware and the sending and consumption of messages.

5. The API adaptation method for supporting non-stop switching of heterogeneous message middleware according to claim 1, characterized in that, The switching method also includes the following preliminary preparation steps: Review the existing application systems and clarify the message middleware and business logic used by each application system; Design a message middleware adaptation layer architecture to encapsulate operations on the first and second message middleware; The adaptation layer implements client integration for the first and second message middleware and provides a unified API interface. Add message format conversion logic to the adaptation layer to ensure message format compatibility between different message middleware; Design a configuration management system that supports dynamic modification of message middleware configuration information.

6. The API adaptation method for supporting non-stop switching of heterogeneous message middleware according to claim 1, characterized in that, According to claim 2, the API adaptation method for supporting non-stop switching of heterogeneous message middleware is characterized in that the message format conversion module performs message format conversion using the following steps: (1) Parse the message format of the first message middleware: Receive a message from the first message middleware, the message including a message header, message body, timestamp, and message type; Use regular expressions or predefined parsing templates to parse the message and extract key fields; The extracted key fields are stored as intermediate data structures for subsequent transformation processing; (2) Perform field conversion according to the preset mapping rules; Load a preset mapping rule table, which defines the correspondence between the key fields of the first message middleware and the fields supported by the second message middleware. The mapping rule table can be a configuration file, a database table, or a data structure in memory. Traverse each key field in the intermediate data structure, use a hash table or search tree algorithm to quickly match the field name, and convert it into a field format supported by the second message middleware according to the mapping rules; For fields that cannot be directly mapped, the following algorithm is used for processing: Check if the field is an optional field. If it is an optional field and there is no corresponding mapping, fill it with the default value. If a field is required and there is no corresponding mapping, the error handling mechanism is triggered, the log is recorded and the error information is returned; For fields that require data type conversion, use a type conversion algorithm to ensure that the converted fields meet the format requirements of the second message middleware. (3) The message after serialization: The transformed fields are organized into a message structure supported by the second message middleware, which can be JSON, XML, Protobuf or other custom binary formats; The message is serialized using a serialization algorithm, which includes, but is not limited to: For JSON format, a recursive traversal algorithm is used to convert the intermediate data structure into a JSON string; For the Protobuf format, the serialization code generated by the Protobuf compiler is used to serialize the intermediate data structure into binary format; During the serialization process, the message is verified to ensure its integrity and correctness. The verification algorithm includes, but is not limited to, checksum algorithm, hash algorithm or custom verification logic. The serialized message is sent to the second message middleware, completing the entire message format conversion process.

7. An API adaptation system that supports non-stop switching of heterogeneous message middleware, characterized in that, include: The adaptation layer is set up between the application system and the message middleware, encapsulating the operations on the first and second message middleware, and providing a unified API interface to the upper layer. A configuration management system is used to dynamically manage the configuration information of the message middleware; The monitoring module is used to monitor the running status of the message middleware and the sending and consumption of messages.

8. An API adaptation system supporting non-stop switching of heterogeneous message middleware according to claim 7, characterized in that, The adapter layer includes: The producer module is used to send messages to the first message middleware or the second message middleware according to the configuration. The consumer module is used to listen to messages from both the first and second message middleware simultaneously, and dynamically switch the message middleware to consume based on the configuration. The message format conversion module is used to convert message formats between different message middleware.

9. An API adaptation system supporting non-stop switching of heterogeneous message middleware according to claim 7, characterized in that, The configuration management system supports dynamically modifying the configuration information of the message middleware and triggering the adaptation layer to automatically switch the message middleware.

10. An API adaptation system supporting non-stop switching of heterogeneous message middleware according to claim 7, characterized in that, The monitoring module includes: The message queue monitoring unit is used to monitor the message queue status of the message middleware. The performance monitoring unit is used to monitor performance metrics such as latency and throughput for message sending and consumption. The alarm unit is used to issue alarm notifications when monitoring indicators are abnormal.

Images