A high-performance microservices system combining software and hardware

Abstract

The present invention provides a high-performance microservice execution environment for hybrid operation of software microservices and hardware microservices. Various microservices implemented based on software and hardware can be integrated through microservice orchestration to achieve a high-performance application system. In particular, by integrating modules implemented based on hardware, the data processing ability of the application can be improved, and the difficulty of building a high-performance application can be simplified. In application fields such as smart cities, compute-intensive and data-intensive tasks have become common computing requirements. Application systems based on the microservice architecture are simple, flexible, and highly available. However, for compute-intensive applications, development still needs to be carried out for dedicated acceleration devices. To effectively utilize the hardware acceleration capability, the hardware needs to be abstracted into microservice modules and can interact with other services to achieve application construction and simplify the application customization process. This invention proposes a microservice operation environment that combines software and hardware. By defining a common interaction interface, it provides service discovery, service proxy, concurrent access control, and solves the management problem of hardware microservices compared to software microservices, realizing a general microservice operation environment and improving the overall operation efficiency of the system for building a computing power platform for smart cities.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a high-performance microservice system computing that combines software and hardware, which is mainly used to manage computing resources in complex environments of cloud computing and edge computing. It effectively integrates various types of computing acceleration hardware systems and application systems in a unified microservice manner, thereby improving the operating efficiency of applications, data processing throughput and reducing energy consumption. Background Art

[0002] Cloud-Edge Collaborative Computing 1 As a computing platform for large-scale artificial intelligence applications serving smart cities, it needs to support data-intensive and computing-intensive applications, and therefore needs to integrate a large number of hardware acceleration devices. 2 Application construction, providing modular implementation and high availability of applications, has become a way to quickly and efficiently build high-availability large-scale application systems. How to effectively utilize hardware acceleration capabilities in a microservice-based application deployment environment, improve software development and deployment efficiency, and avoid consuming too much manpower and time in the process of system debugging and error checking is a challenge faced by improving artificial intelligence applications. Existing application systems usually use software microservices to call hardware acceleration capabilities through APIs, which easily increases the complexity of each application development and debugging. At the same time, multiple software microservices access or occupy the same hardware acceleration device through APIs, which requires data synchronization and device exclusivity. It relies on the implementation of APIs or device drivers, which is prone to errors that are difficult to find and debug. In actual application deployment, this end-to-end encapsulation of hardware devices will also affect the soul of system resource management and scheduling.

[0003] This invention establishes a microservice management system that supports the mixed operation of software and hardware microservices, and forms the ability of software and hardware microservices to communicate with each other based on hardware-oriented microservice packaging standards, thereby realizing hardware microservice packaging that is transparent to users. Similar to the existing software microservice system, users can transparently implement microservice-based application customization, and at the same time, they can obtain the computing and I / O acceleration capabilities brought by hardware microservices, becoming an efficient solution for computing and I / O intensive applications while taking into account production efficiency. Summary of the invention

[0004] The present invention provides a high-performance microservice execution environment for hybrid operation of software microservices and hardware microservices. It can integrate various microservices implemented based on software and hardware through microservice orchestration to achieve a high-performance application system. In particular, by integrating modules implemented based on hardware, it can improve the data processing ability of the application and simplify the difficulty of building a high-performance application. In application fields such as smart cities, computing-intensive and data-intensive tasks are basic application requirements. Application systems based on the microservice architecture are characterized by simplicity, flexibility, high availability, and high development and deployment efficiency. However, for computing and I / O-intensive applications, customized development needs to be carried out for dedicated acceleration devices. To effectively utilize the hardware acceleration ability, the hardware is abstracted as a microservice module, which can interact with other services to realize application construction and simplify the application customization process. Through a microservice operation environment that combines software and hardware, a common interaction interface is defined to provide service discovery, service proxy, concurrent access control, and solve the management problem of hardware microservices compared to software microservices, realizing a general microservice operation environment and improving the overall operation efficiency of the system for building a computing power platform for smart cities.

[0005] To achieve the above object, the present invention proposes a microservice operation environment that integrates software and hardware microservices, including several basic requirements of microservices such as service discovery, service encapsulation, service proxy, and service interaction between services, realizing the ability of user-transparent integration of software and hardware resources, and can be used as a solution for a computing power platform for smart cities to provide a service system for applications to improve operation efficiency.

[0006] The present invention includes 4 parts: 1. Hardware microservice encapsulation, proxy, and external interface definition; 2. Hardware microservice discovery algorithm; 3. Software and hardware microservice interaction technology. By integrating the above 3 technologies, a high-performance microservice execution environment for hybrid operation of software microservices and hardware microservices is realized. Description of the Drawings

[0007] Figure 1 Data structure supporting hardware microservice proxy;

[0008] Figure 2 Mapping table of microservices with local devices and microservice proxies;

[0009] Figure 3 Tree-shaped hardware microservice proxy architecture;

[0010] Figure 4 Hardware microservice interaction based on shared data cache;

[0011] Figure 5 Is the algorithm flowchart;

[0012] Figure 6 Is the algorithm execution flowchart;

[0013] Figure 7It is a flowchart of the algorithm execution effect. Detailed implementation mode

[0014] The implementation method of this invention is to encapsulate hardware microservices with software agents in a tree - like hierarchical structure, and perform microservice management, discovery and registration. The software agent itself provides functions of interacting with hardware, interacting with low - level software agents, and providing service discovery to the microservice management center. The interaction between software microservices and hardware microservices can be achieved through hardware drivers, and the exchange between hardware microservices can be directly interacted through software agents or hardware protocols.

[0015] Hardware microservice encapsulation and external interface definition:

[0016] The encapsulation of hardware microservices is implemented in the way of software drivers, that is, the call and return processing of hardware microservices are encapsulated through software processes. The steps of calling hardware microservices are as follows:

[0017] 1. Process the call request and convert the call parameters into data types recognizable by the hardware device driver;

[0018] 2. Call and execute the hardware microservice through the hardware device driver;

[0019] 3. Obtain the return result. The running end status information can be obtained by polling or registering hardware interrupts, and the return result can be obtained by reading the hardware buffer;

[0020] 4. Return the microservice call, convert the hardware device return data into a data type recognizable by the software call and return it to the caller.

[0021] The proxy of hardware microservices is implemented in a tree - like structure through the software agent agent, that is, the agent itself represents the hardware microservices on the leaf nodes in a tree - like structure. In a distributed computing cluster, hardware devices (especially computing acceleration devices) usually access the data exchange medium (such as PCIe bus, TCP, RDMA network) in a tree - like structure, so the hardware microservices are encapsulated in a tree - like structure. Recursively integrate the supported hardware microservice interfaces from the leaf nodes to the top nodes of the tree - like structure.

[0022] Agent nodes include two types: leaf nodes and non - leaf nodes. Leaf nodes provide the following functions:

[0023] 1. Provide an interface proxy for hardware microservices, that is, provide software interface encapsulation for interface calls to assist its interaction with other microservices;

[0024] 2. In the scenario of interaction between multiple hardware microservices, act as a running container, that is, support the interaction between hardware microservices corresponding to two hardware devices on the same physical server to run in the same process space in an aggregatable manner;

[0025] 3. Receive the agency of the upper-layer agent and perform hardware microservice calls;

[0026] 4. Receive the registration / deregistration commands of the upper-layer agent and register / deregister with the microservice management center;

[0027] 5. Accept microservice agency requests and perform hardware microservice calls.

[0028] Non-leaf nodes provide the following functions:

[0029] 1. Manage the hardware microservice structure managed by all lower-layer agents below it;

[0030] 2. Receive the agency of the upper-layer agent and perform lower-layer agent calls;

[0031] 3. Receive the registration / deregistration commands of the upper-layer agent and register / deregister with the microservice management center;

[0032] 4. Accept microservice agency requests and perform lower-layer microservice calls;

[0033] 5. If the agent has no upper-layer agent, it needs to register with the microservice management center.

[0034] The agent nodes in the tree structure are constructed as Figure 1 shown and contain the following:

[0035] 1. Basic information: the name, IP, and port of the microservice agent;

[0036] 2. Microservice list: the microservices under the jurisdiction of this agent;

[0037] 3. Local device list: the list of computing devices under the jurisdiction of the computing nodes where this agent runs;

[0038] 4. Sub-microservice agent list: the sub-microservice agents contained in this agent;

[0039] 5. Mapping table: the mapping relationship between the microservices under the jurisdiction of this agent and local devices and sub-microservices.

[0040] Here, the computing node represents a server node, that is, a physical server. The computing devices under the jurisdiction of the computing node include the CPU of the server itself and acceleration devices such as GPUs and FPGAs managed through the PCIe bus. The agent node itself is encapsulated in a container and runs as a resident service on the server node where the agent is located. The mapping table manages the mapping relationship between microservices and local devices or sub-microservices managed by the agent, as Figure 2As shown in the figure, the mapping table provides the mapping relationship between hardware microservice discovery and microservice call proxy service. Each agent node also maintains the association information with the parent node.

[0041] Hardware microservice discovery algorithm:

[0042] Hardware microservice discovery is performed through the interaction between the microservice management center and the agent. The management center maintains a global registry to establish the association between microservices and agents. Agents are implemented in a tree structure, and service discovery needs to implement two auxiliary functions: microservice agent registration and microservice registration. Each top-level agent is responsible for registering with the microservice management center, and then recursively registering its lower-level agents. The process is as follows: Figure 5 shown.

[0043] Given input information: r represents the top-level agent, that is, the root agent.

[0044] As can be seen above, the agent registration process is a top-down recursive process, and each microservice agent in the tree structure is registered. The registration process itself registers the microservice agent and the microservices it proxies to the global information table. The deregistration process is similar, clearing the agent information recursively.

[0045] The microservice registration process is as follows:

[0046] Given input information: ms represents the microservice, msd represents the agent corresponding to the microservice, ld represents the local device corresponding to the microservice, and n represents the top-level agent. The microservice registration process is a top-down recursive process, and all upper-level microservice agents in the tree structure are registered. The registration process itself registers the microservice agent and the microservices it proxies to the global information table. The deregistration process is similar, clearing the microservice information recursively.

[0047] Microservice discovery process:

[0048] Based on the microservice and agent registration, microservice discovery can be obtained by querying the global registry. Here, the query can return multiple proxy nodes (upper nodes in the tree structure) of a given hardware microservice, and the caller of the microservice can select the lowest-level microservice proxy to implement the call. For the leaf node agent, the microservice of its proxy itself corresponds to its local device, and the microservice call can be implemented directly through the hardware proxy driver. For non-leaf nodes, Figure 7 The procedure is located and called.

[0049] Given input information: ms represents the microservice, and n represents the current agent.

[0050] Hardware and software microservice interaction:

[0051] As mentioned above, the hardware and software microservice interaction can achieve standard microservice interaction through the software encapsulation corresponding to the hardware. The interaction between hardware microservices can be further improved in efficiency through communication optimization. The ways of hardware microservice interaction include:

[0052] 1. Sharing data cache and implementing calls and interactions in the way of direct storage access, such as Figure 4 ;

[0053] 2. Conducting data interaction through the PCIe bus, such as Figure 4 。

[0054] The interaction between hardware devices needs to exchange data through a preset interaction protocol, and the operation of the interaction protocol (initiation and control of data reading and writing) needs to be controlled by the corresponding software encapsulation.

Claims

1. A high-performance microservices system combining software and hardware, characterized in that Establish a software-implemented encapsulation and proxy system for hardware-implemented microservices. Encapsulation of hardware microservices and definition of external interfaces: The encapsulation of hardware microservices is implemented in a software-driven manner, that is, the invocation and return processing of hardware microservices are encapsulated through software processes. The steps for invoking hardware microservices are as follows: S1 Process the invocation request and convert the invocation parameters into a data type recognizable by the hardware device driver; S2 Invoke and execute the hardware microservice through the hardware device driver; S3 Obtain the return result. Obtain the status information of the end of operation by polling or registering a hardware interrupt, and obtain the return result by reading the hardware buffer; S4 Return the microservice invocation, convert the hardware device return data into a data type recognizable by the software invocation and return it to the invoker; The proxy of the hardware microservice is implemented in a tree structure by a software agent, that is, the agent itself represents the hardware microservices on the leaf nodes in a tree structure. The software agent nodes include two types: leaf nodes and non-leaf nodes; the leaf nodes provide interface proxies for the hardware microservices; The non-leaf nodes manage the hardware microservice structures managed by all lower-level agents below them. In a distributed computing cluster, the hardware devices are connected to the data exchange medium in a tree structure. Therefore, the hardware microservices are encapsulated in a tree structure, and the supported hardware microservice interfaces are recursively integrated from the leaf nodes to the top nodes of the tree structure.

2. The high-performance microservices system combining software and hardware according to claim 1, characterized in that A service discovery system for hardware-implemented microservices is established.

3. A high-performance microservices system combining software and hardware according to claim 1, characterized in that Establish a service invocation and invocation transfer system for hardware-implemented microservices.

4. A high-performance microservices system combining software and hardware according to claim 1, characterized in that Establish an interaction system between software- and hardware-implemented microservices.

5. A high-performance microservice system combining software and hardware according to claim 1, characterized in that Establish a system for mutual invocation between hardware-implemented microservices.

Images