A microservice architecture optimization method based on intelligent orchestration

Abstract

The present invention discloses a microservice architecture optimization method based on intelligent arrangement, comprising the following steps: the system realizes the flexibility of microservice combination through message queue, and uses Kafka message queue to realize data communication between microservices; the service combination strategy is described uniformly, and each microservice is intelligently arranged according to the service combination strategy; the microservice combination hot update control platform provides a platform where users can define the service combination strategy by themselves; the daemon thread monitors the change of the combination strategy in real time, and realizes the change of the data flow direction between microservices by changing the message channel Topic subscribed by the microservice. The present invention realizes the automatic configuration of microservice combination and the hot update method without stopping and restarting, and improves the efficiency of changing the data flow direction between services.

Description

Technical Field

[0001] The present invention belongs to the field of microservice architecture, and in particular relates to a microservice architecture optimization method based on intelligent orchestration. Background Art

[0002] Microservices is a new software architecture style. In the microservice architecture, applications can be decomposed into multiple smaller services, each of which can be developed and deployed by an independent team. With the emergence of microservice architecture, more and more large applications are beginning to be deployed and run in a microservices manner, and traditional monolithic applications are split into multiple functionally independent microservices. Microservices have a high degree of platform interoperability, dynamic expansion capabilities, and strong fault tolerance, and have received great attention from the engineering and academic communities.

[0003] In a microservice architecture, a single microservice can usually only implement a simple function. To implement complex business functions, it is necessary to flexibly combine the independent microservices in the system to meet more diverse business needs. The microservice architecture solves the problem of poor scalability and maintainability of the monolithic architecture, but how to effectively coordinate and combine multiple microservices has become a new problem. Microservice combination has also become a research focus that cannot be ignored in the microservice architecture.

[0004] The traditional orchestration model can describe the automated arrangement, coordination and management between middleware and various businesses in complex computer systems, which is a typical centralized model. This method can better control the global state of the business process, but the coupling between services is high, the control center is fragile, and the system fault tolerance is low. In some relatively complex business scenarios, there may be problems such as mathematical confusion and blurred boundaries between functional modules, which will reduce the service quality and cause a lot of redundant code. The emerging distributed orchestration model adopts a decentralized model, and each service involved knows when to execute its own operations and when to interact with other services. At present, the microservice composition method is basically based on the workflow model, which predefines the order of service composition interaction. Therefore, it is particularly important to study a microservice intelligent orchestration with automated service composition configuration. Summary of the invention

[0005] Purpose of the invention: In view of the high cost of changing service orchestration combinations and the need for restart in a complex microservice environment, this study proposes an intelligent orchestration solution with automatic configuration of data flow and non-stop configuration updates, which realizes automated service combination configuration and improves the efficiency of changing the direction of data flow between services.

[0006] Technical solution: The present invention provides a microservice architecture optimization method based on intelligent orchestration, comprising:

[0007] (1) Data flow control;

[0008] (2) Unified description of service composition strategy;

[0009] (3) Microservice combination hot update control platform;

[0010] (4) Intelligent orchestration;

[0011] Furthermore, the data flow control design in step (1) includes using Kafka message queues to implement data communication between microservices, and then changing the message channel Topic subscribed by the microservices to implement changes in data flow between microservices.

[0012] Furthermore, the unified description of the service composition strategy in step (2) includes a unified description of the service composition strategies involved, so as to facilitate subsequent software developers or policy makers to participate in system development. The design of the data structure in the microservice composition strategy includes two aspects: service composition information and system functions. The service composition information is used to concretize the static properties of the microservice composition strategy, including the microservice name, microservice message channel information, microservice publishing and subscription topic information, etc., while the system functions include the function of exception handling and the function of return result processing.

[0013] Furthermore, the design of the microservice combination hot update control platform in step (3) includes the corresponding configuration of the combination strategy, and the microservice combination information after the configuration takes effect can also be seen. The hot update control platform abstracts the microservices into node information, abstracts the data link relationship between microservices into a linked list, and generates a corresponding topology diagram to clearly display the link call relationship between each microservice.

[0014] Furthermore, the intelligent arrangement design in step (4) includes the following steps:

[0015] (4-1) Setting up a common module to judge the rationality of the microservice combination strategy change request issued by the microservice hot update control platform;

[0016] (4-2) Set up a daemon thread to monitor all configuration policy information of the microservice and update the message channel topic subscribed by the microservice in real time;

[0017] (4-3) Changes to the service data link are implemented through the Kafka message queue.

[0018] Beneficial effect: Compared with the prior art, the present invention proposes a microservice architecture optimization solution with automatic configuration of data flow and non-stop configuration update, realizes service requests and calls between microservices through message queues, and realizes changes in data flow between microservices by changing the message channel topics subscribed by microservices, realizes automated service orchestration configuration, and improves the efficiency of changing the direction of data flow between services. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A logical design block diagram for the microservice architecture optimization method based on intelligent orchestration.

[0020] Figure 2 Hot update control platform for microservices.

[0021] Figure 3 This is a system block diagram of the microservice architecture optimization method based on intelligent orchestration. DETAILED DESCRIPTION

[0022] The present invention will be further described below. A microservice architecture optimization method based on intelligent orchestration of the present invention comprises the following steps:

[0023] Step 1: Data flow control.

[0024] like Figure 1 As shown in the figure, in the microservice architecture optimization method based on intelligent orchestration, we first use the Kafka message queue to realize data communication between microservices, and then change the message channel Topic subscribed by the microservice to realize the change of data flow between microservices. The communication method between all microservices in the system is changed from the traditional "HTTP-type API" to the message queue communication based on the subscription and publishing mode, and the Kafka message queue, which is currently the most widely used in the industry, is used as the communication channel between microservices. As a distributed, partitioned, multi-copy distributed MQ system, Kafka can provide message persistence capabilities in a time complexity of O(1), and can also support the transmission of 100K messages per second on a single machine, thus laying the foundation for the robustness and high availability of communication between microservices.

[0025] All microservices are connected to Kafka's server node broker. The broker is responsible for ensuring the survival, data communication and data consistency of the microservices. At the same time, the microservices open their own message channels by selecting different message topics and divide the data in the topic into one or more partitions. The data under N partitions is stored in multiple brokers to ensure high availability. At the same time, the index files corresponding to multiple segments are used to ensure the consumption order of each partition.

[0026] In the data communication solution based on Kafka message queue, each microservice will be assigned a corresponding topic, and each microservice will only publish data to its corresponding topic. When other microservices subscribe to a topic, the data flow of the microservice corresponding to the topic can be realized to the microservice subscribing to this topic. For example, there are microservices A, B, and C. When microservice B subscribes to the topic of microservice A, and microservice C subscribes to the topic of microservice B, the data flow from A to B to C can be realized.

[0027] Step 2: Unified description of service portfolio strategy.

[0028] After using Kafka message queues to implement data communication between microservices, we can change the way microservices are combined by changing the topics that microservices subscribe to and publish. The way microservices are combined needs to comply with certain specifications, so first of all, a unified description of the service combination strategy involved is made to facilitate subsequent software developers or policy makers to participate in system development.

[0029] In order to unify the code style of the project and make the program readable, the system adopts the internationally accepted identifier naming convention: 1) The character range that can be used in identifiers is English letters (A~Z a~z) and numbers (0~9) and the only punctuation marks that can be used; 2) Identifiers must start with an English letter; 3) Identifiers are strictly case-sensitive and must follow the camel case format; 4) There is no strict length limit for identifiers, but the length of identifiers will be limited according to actual needs; 5) Identifier groups need to be self-explanatory and use a subject-verb structure.

[0030] There are eight basic data types in Java, namely char, byte, short, int, long, float, double, and boolean, which can be used to describe the data model in a standardized manner. The enumeration type Enum in Java is concise, safe, and convenient. It is implemented by inheriting the java.lang.Enum class and is suitable for describing constants in the data model.

[0031] The design of data structure in microservice composition strategy includes two aspects: service composition information and system functions. Service composition information is used to materialize the static properties of microservice composition strategy, including microservice name, microservice message channel information, microservice publishing and subscription topic information, etc., while system functions include exception handling and return result processing.

[0032] The microservice composition information includes JSONSchema, subsTopic and sendTopic, as described in Table 1. microName represents the specific name of the microservice in the system; subsTopic is the message topic in the message channel that describes the microservice subscribes to; sendTopic is the message topic in the message channel that describes the microservice publishes.

[0033] Table 1 Microservice composition information description

[0034]

[0035] The corresponding table of the hot update control platform is shown in Table 2. The microservice name and the message topic of subscription and publication are in Json format, and the list of subscribed / published message topics can be obtained from the system.

[0036] Table 2 User information description

[0037]

[0038] The system function is the exception handling function set for system stability and the processing function for the service combination strategy result return. The exception handling data structure design is shown in Table 3 below, and the return result data structure design is shown in Table 4.

[0039] In the exception handling data structure, ExceptionEnum uses the Enum enumeration data type in the Java language to materialize the service exception into a specific constant. Code is used to represent the exception type as a code number, and MSG is used to describe the exception information context to facilitate locating the location where the exception occurs.

[0040] Table 3 Exception handling function description

[0041]

[0042] In the returned result data structure, status indicates the status value of the returned result. Generally, 0 indicates success, and other values ​​indicate failure. errorMsg is used to describe the error message in the returned result. When the status field is 0, errorMsg is success. data uses generics in the Java language and is compatible with different types. It records the output parameters of the return body message in English.

[0043] Table 4 Return result description

[0044]

[0045] Step 3: Microservice combination hot update control platform.

[0046] After uniformly describing the service composition strategy, we can combine each microservice according to the formulated strategy. As a visual operation platform, the hot update control platform can configure the corresponding service composition strategy and send the service composition strategy to the system background. The thread monitoring module in the system background will monitor the changes in the service composition relationship in real time. It will switch the data flow link between microservices and change the data flow direction according to the updated microservice information and the corresponding subscription topic information.

[0047] In addition, the hot update control platform abstracts microservices into node information, abstracts the data link relationship between microservices into a linked list, records the parent node ID of each node information, and draws the topology diagram of the microservice data link by identifying the parent node ID of all nodes. By obtaining the microservice subscription information, the parent-child relationship of the microservice subscription is generated to form an object array as the data input of the topology diagram. When the data link between microservices changes, the topology diagram will be redrawn. The topology diagram can clearly show the link call relationship between each microservice. Figure 2 In the microservice hot update control platform, users can formulate corresponding microservice combination strategies. Microservices are combined according to the strategies configured by users. The topology diagram in the figure is the topology diagram of the data links between microservices.

[0048] Step 4: Intelligent orchestration.

[0049] like Figure 3 As shown in the figure, the backend system is mainly responsible for receiving new microservice combination strategies issued by the microservice combination hot update control platform and realizing the change of data flow direction. The backend system mainly includes three parts: common module, daemon thread and kafka message queue.

[0050] (1) Common modules

[0051] As the common module of the system design hub, it is responsible for the request processing and distribution of the entire system, including but not limited to the verification of request parameters, the capture of exceptions, the processing of returned results, etc. At the same time, it provides common tool modules for other microservices to ensure the stable operation of the system. During the service composition strategy update process, the common module is mainly responsible for receiving service composition strategy update requests and judging the rationality of the requests, forwarding the service composition strategy update requests that meet the requirements to the daemon thread module, and the daemon thread implements the modification of the data flow direction.

[0052] (2) Daemon Thread

[0053] The daemon thread plays the role of a "sentinel" in the system, monitoring the occurrence of events and implementing changes in the data flow between services. In this system, an event refers to a change in the microservice combination strategy. Since Kafka does not support automatic switching of subscribed topics, we use the daemon thread to monitor in real time to dynamically change the topic subscribed by the microservice. The specific process is as follows: In the program, a thread is started to monitor the topic variable subscribed by the microservice. When the topic variable changes, the original consumer is closed in the thread and a consumer is reinitialized (when initializing the consumer, the topic to be subscribed by the consumer will be configured), thereby switching the subscription topic. The microservice combination strategy issued by the hot update control platform and judged by the public management module for rationality will be monitored and captured by the daemon thread. After that, the daemon thread will notify the microservice to update its message subscription topic according to the combination strategy, and all this does not require restarting the machine or modifying the system code. It is a hot update implementation method.

[0054] (3) Kafka message queue

[0055] When the daemon thread receives the request to change the service composition strategy and completes the processing, Kafka can change the direction of data flow according to the modified subscription topic relationship. For example, there are microservices A, B, and C, and the original data link relationship is A to B to C; when we let microservice C subscribe to the topic of microservice A, and microservice B subscribe to the topic of microservice C, the data flow direction can be changed to A to C to B, thereby changing the data flow link between microservices. As the cornerstone of communication between microservices, the high throughput, high availability, and high performance of the Kafka message queue ensure the stability of communication between microservices; the message topic Topic and subscription and publishing mode of the message queue make the data flow between microservices more flexible and can meet any service composition strategy.

[0056] In the design of the microservice architecture optimization method based on intelligent orchestration, the common module is responsible for judging the rationality of the microservice combination strategy change request issued by the microservice hot update control platform; the daemon thread under the microservice is responsible for monitoring the combination strategy update and hot update without downtime; the Kafka message queue is responsible for data communication between services. Together, these realize the microservice architecture optimization method based on intelligent orchestration.

[0057] The technical means disclosed in the present invention are not limited to the technical means disclosed in the above technical means, but also include technical solutions composed of equivalent replacements of the above technical features. Matters not covered in the present invention belong to the common knowledge of those skilled in the art.

Claims

1. A microservice architecture optimization method based on intelligent orchestration, characterized in that: The method comprises the following steps: Step (1) data flow control: the data flow control in step (1) includes realizing service requests and calls between microservices through message queues, and realizing changes in data flow between microservices by changing the message channel topics subscribed by the microservices; Step (2) unified description of service composition strategy; the unified description of service composition strategy in step (2) includes a unified description of the service composition strategy involved, including service composition information and system functions, the service composition information is used to concretize the static attributes of the microservice composition strategy, including the microservice name, microservice message channel information and microservice publishing and subscription topic information, and the system functions include exception handling functions and return result processing functions; Step (3) The microservice combination hot update control platform configures the service combination strategy accordingly, and sees the microservice combination information after the configuration takes effect. The microservice combination hot update control platform abstracts the microservices into node information, abstracts the data link relationship between microservices into a linked list, and generates a corresponding topology diagram to clearly display the link call relationship between each microservice; Step (4) Intelligent orchestration: The intelligent orchestration in step (4) includes the following steps: (4-1) Set up a public module to determine the rationality of the microservice combination strategy change request issued by the microservice combination hot update control platform; (4-2) Set up a daemon thread to monitor all configuration policy information of the microservice and update the message channel Topic subscribed by the microservice in real time; (4-3) The Kafka message queue is used to implement changes to the service data link, realize automatic configuration of data flow, and hot update of combined configuration without stopping the machine.

Images