Data plane optimization for microservices

Abstract

Optimization of an API software layer (e.g., an API framework library) is provided to avoid API traffic data serialization overhead in feasible scenarios. The optimization enables the API framework to make a decision whether to serialize an API request or response prior to sending the API request or response by utilizing information from a control plane platform (e.g., orchestration system) and by exchanging information through existing preamble processes performed by the API framework and protocol. Eliminating the need for serialization and deserialization for each request and response reduces data transfer latency and results in processing cycle and power savings. The modification to the API framework is transparent to the microservice application, and only changes to the API framework are required.

Description

Technical Field

[0001] The present disclosure generally relates to cloud-based communication networks, and more particularly, to optimization of the data plane for cloud-based communication networks. Background Art

[0002] Modern best practice cloud-native application architectures typically include a large number of loosely coupled software entities (referred to as "microservices") packaged as containers or functions as a service (FaaS), which are deployed and orchestrated by an orchestration system (e.g., Kubernetes) on small or large data centers. These microservices interact using application programming interfaces (APIs) defined over one or more technologies (e.g., Hypertext Transfer Protocol (HTTP) and Representational State Transfer (REST), Remote Procedure Call (RPC) (e.g., Google RPC (gRPC))). User requests and external requests are serviced by front-end microservices, which make such API calls to various self-contained back-end microservices as needed to satisfy the requests.

[0003] The benefits of this cloud-native architecture include: 1) independent deployment and scaling, which results in better scalability; 2) fault isolation, which results in higher resilience; 3) faster and continuous development and deployment cycles; 4) better organizational mapping; and 5) the ability to have independent programming languages on different microservices.

[0004] Data plane API interactions (requests and responses) between microservices typically involve some form of data serialization. That is, the request is serialized by the caller and "sent" to the callee, where it is deserialized. The callee processes the request and serializes the response, which is then sent to the caller. The caller then deserializes the response.

[0005] Data serialization provides many benefits. Two interacting microservices may exist on the same operating system, the same virtual / physical server, or two completely different servers with heterogeneous hardware. API requests and responses are typically prepared for the worst-case scenario for transmission over the network, and thus, they are serialized at one end and deserialized at the other end. Moreover, microservices can be implemented in different programming languages, and thus, their representations of common API request and response data structures may be different. Language-agnostic data serialization formats (such as Protocol Buffers (Protobuf)) are used as a common intermediate language. For these reasons, data serialization is performed regardless of where the API caller and callee reside in the data center, even if they reside on the same operating system instance and use the same programming language.

[0006] Due to the large number of internal API request / response operations in a data center, even for a single request / response, the data serialization overhead increases quite significantly. Additionally, the designed self - contained microservices only address specific aspects of the application and business logic (hence the term "micro"), and thus, the actual processing time for API requests is typically very low, on the millisecond or even microsecond time scale, which has led to the creation of the term "microsecond RPC". Consequently, in the end - to - end API request - response cycle, the data serialization overhead becomes significant compared to the actual business logic processing.

[0007] It has been reported that data serialization and RPC in Google's data centers account for 12% of all fleet cycles across all applications. Thus, clearly, this data serialization adds significant overhead, leading to increased latency and wasted processing cycles, and consequently increased power. Summary of the Invention

[0008] This disclosure introduces optimizations to the API software layer (e.g., API framework library) to avoid API service data serialization overhead in feasible scenarios. The optimizations enable the API framework to make a decision on whether to serialize an API request or response before sending the API request or response by leveraging information from the control - plane platform (e.g., an orchestration system) and existing pre - process procedures performed by the API framework and protocols to exchange information. Eliminating the need for serialization and deserialization for each request and response reduces data transfer latency and results in savings in processing cycles and power. The modifications to the API framework are transparent to microservices applications and only require changes to the API framework.

[0009] A first aspect of this disclosure includes a method implemented by a client in a microservice - based communication network for exchanging API requests and responses with a server. The method includes: receiving information indicating the data transfer capabilities supported by the server API framework for API request / response communication between the client and the server. The method further includes: based on the received information, determining that the client API framework and the server API framework support the same data transfer mode and the same in - memory representation of the API request / response. The method further includes: in response to the determination, establishing a direct memory access channel for API request / response communication between the client and the server using the same in - memory representation of the API request / response. The method further includes: using the direct memory access channel to exchange API messages with the server.

[0010] A second aspect of the present disclosure includes a client in a microservices-based communication network, the client being configured to exchange API requests and responses with a server. The client is configured to receive information indicating a data transfer capability for communication of API requests / responses between the client and the server supported by a remote API framework in the server. The client is further configured to determine, based on the received information, that the client API framework and the server API framework support the same data transfer mode and the same memory representation of the API requests / responses. The client is further configured to, in response to the determination, establish a direct memory access channel for communication of API requests / responses between the client and the server using the same memory representation of the API requests / responses. The client is further configured to use the direct memory access channel to exchange API messages with the server.

[0011] A third aspect of the present disclosure includes a computing device in a microservices-based communication network, the computing device being configured to exchange API requests and responses with a server. The computing device includes interface circuitry for communicating with the server and processing circuitry operably coupled to the interface circuitry. The processing circuitry is configured to receive information indicating a data transfer capability for communication of API requests / responses between the client and the server supported by a server API framework. The processing circuitry is further configured to determine, based on the received information, that the client API framework and the server API framework support the same data transfer mode and the same memory representation of the API requests / responses. The processing circuitry is further configured to, in response to the determination, establish a direct memory access channel for communication of API requests / responses between the client and the server using the same memory representation of the API requests / responses. The processing circuitry is further configured to use the direct memory access channel to exchange API messages with the server.

[0012] A fourth aspect of the present disclosure includes a computer program for a client running on a computing device. The computer program includes executable instructions that, when executed by processing circuitry in the computing device, cause the computing device to perform the method according to the first aspect.

[0013] A fifth aspect of the present disclosure includes a carrier containing the computer program according to the fourth aspect. The carrier is one of an electronic signal, an optical signal, a radio signal, or a non-transitory computer-readable storage medium.

[0014] The sixth aspect of the present disclosure includes a method for optimizing data exchange between a client and a server implemented by an orchestrator or other management function in a microservices-based communication network. The method includes: deploying a client and a server, the server being configured to use direct memory access for the exchange of API requests / responses with the server. The method further includes: receiving, from the client, a query regarding the data transfer capabilities of the API server framework of the server. The method further includes: in response to the query, providing the client with information indicating the data transfer capabilities supported by the server API framework for the communication of API requests / responses between the client and the server.

[0015] The seventh aspect of the present disclosure includes a management function in a microservices-based communication network for optimizing data exchange between a client and a server. The management function is configured to deploy a client and a server, the server being configured to use direct memory access for the exchange of API requests / responses with the server. The management function is further configured to receive, from the client, a query regarding the data transfer capabilities of the API server framework of the server. The management function is further configured to, in response to the query, provide the client with information indicating the data transfer capabilities supported by the server API framework for the communication of API requests / responses between the client and the server.

[0016] The eighth aspect of the present disclosure includes a computing device for optimizing data exchange between a client and a server in a microservices-based communication network. The computing device includes interface circuitry for communicating with the server and processing circuitry operably coupled to the interface circuitry. The processing circuitry is configured to deploy a client and a server, the server being configured to use direct memory access for the exchange of API requests / responses with the server. The processing circuitry is further configured to receive, from the client, a query regarding the data transfer capabilities of the API server framework of the server. The processing circuitry is further configured to, in response to the query, provide the client with information indicating the data transfer capabilities supported by the server API framework for the communication of API requests / responses between the client and the server.

[0017] The ninth aspect of the present disclosure includes a computer program for an orchestrator or management function to run on a computing device. The computer program includes executable instructions that, when executed by the processing circuitry in the orchestrator or management function, cause the orchestrator or management function to perform the method according to the sixth aspect.

[0018] The tenth aspect of the present disclosure includes a carrier containing the computer program according to the ninth aspect. The carrier is one of an electronic signal, an optical signal, a radio signal, or a non-transitory computer-readable storage medium. Description of the Drawings

[0019] Figure 1Shows the data plane in a traditional microservices-based architecture that uses data serialization for the exchange of API requests and responses.

[0020] Figure 2 Shows a microservices-based architecture with an optimized data plane for the exchange of API requests and responses.

[0021] Figure 3 Shows a microservices-based architecture including a load balancer.

[0022] Figure 4 Is a sequence diagram for data plane optimization according to the first embodiment.

[0023] Figure 5 Is a sequence diagram for data plane optimization according to the first embodiment.

[0024] Figure 6 Shows a method for data plane optimization implemented by a client.

[0025] Figure 7 Shows an exemplary client configured for data plane optimization.

[0026] Figure 8 Shows a method for data plane optimization implemented by an orchestrator.

[0027] Figure 9 Shows an exemplary orchestrator configured for data plane optimization.

[0028] Figure 10 Shows the main functional components of a computing device in a microservices-based architecture that can be configured to implement a client and / or a server. Detailed Description

[0029] The present disclosure provides techniques for optimizing communication between microservices to reduce overhead and latency. Traditionally, data plane API interactions (i.e., requests and responses) between microservices involve some form of data serialization (which is added to the overhead), increasing data transfer latency and adding required processing cycles and power. In embodiments of the present disclosure, an API framework obtains information about available servers, such as server location, available data transfer modes, and in-memory representations of supported API request / response structures (i.e., API data structures). Information about the server location is obtained from a cloud control plane or a load balancer. Information about the supported data transfer modes and API data structures can be obtained from a cloud control plane or a load balancer, or by exchanging information between a client-side API framework and a server-side API framework. Based on this information, the client-side API framework selects a transport mechanism for communication of API requests / responses between the client and the server. If the client and the server reside on the same operating system and support the same API data structure, shared memory can be used for the transfer of API requests / responses. If the client and the server reside on different physical servers but support the same API data structure, remote direct memory access (RDMA) can be used to establish an optimized channel. RDMA enables direct memory access from the memory of one computer to the memory of another computer without involving the processors, caches, or operating systems of either computer. If these options are not available, the default channel using HTTP / REST can be used for API request / response communication.

[0030] Figure 1An exemplary cloud computing environment is shown that has three cloud servers 120 (represented as Server-1 to Server-3) supported by a common control plane 110. The cloud servers 120 can include physical servers or virtual machines (VMs) that form a cluster according to an orchestration system such as Kubernetes. Each server 120 runs an operating system that can support multiple containers 130 (represented as Container-1 to Container-5). Each container 130 implements an instance of a microservice 140 that enables rapid scaling of services by adding or removing containers 130 as needed. In this example, Container-1, Container-3, and Container-5 implement instances of a client application (Microservice-X), and Container-2 and Container-4 implement instances of a corresponding server application (Microservice-Y). The microservice binary includes a specific API framework 150, depending on the API protocol / language used. The API framework 150 enables communication between microservices 140 to be implemented by different containers 130 on the same or different servers 130. In the example, the API framework implements remote procedure call (RPC), also referred to herein as the RPC framework. RPC is a software communication protocol that a program can use to request services from a program in another computer located on a network without having to understand the details of the network. Thus, the RPC framework is an example of an API framework that implements the RPC protocol.

[0031] In Figure 1 In the example shown, an instance of Microservice-X in Container-1 has established a connection with an instance of Microservice-Y in Container-2 on the same server 120. In this case, API requests are serviced from the local server. Instances of Microservice-X in Container-3 and Container-5 have both established connections with an instance of Microservice-Y in Container-4 on a different server 120. In this case, API requests are serviced from a remote server. Even though an API request from an instance of Microservice-X in Container-1 is serviced locally, the request and response are serialized, which adds to the signaling overhead. Also, even in the case where instances of Microservice-Y support the same in-memory representation of the API request / response structure, API requests from instances of Microservice-X in Container-3 and Container-5 are serialized.

[0032] One aspect of the present disclosure is to provide a mechanism for enabling unserialized communication between microservices running on the same or different servers in a scenario where direct memory transfer for API requests / responses is supported and the client application and the server application support the same memory representation of the API request / response structure. Generally, the client-side API obtains information about the server location and capabilities, i.e., the supported data transfer modes and API data structures, by exchanging messages with the cloud control plane and possibly the server-side API framework during the establishment of the initial channel for the exchange of API requests / responses using a default protocol such as HTTP / REST. Based on this information, the client-side API framework selects a transport mechanism for the communication of API requests / responses between the client and the server. If the client and the server reside on the same operating system and support the same API data structure, shared memory can be used for the transfer of API requests / responses. If the client and the server reside on different physical servers but support the same API data structure, an optimized channel can be established using Remote Direct Memory Access (RDMA). If these options are not available, the default channel using HTTP / REST can be used for the communication of API requests / responses.

[0033] Figure 2 A cloud computing environment with data plane optimization for the exchange of API requests / responses is shown. The cloud computing environment includes three cloud servers 120 supported by a common control plane 110, represented as Server-1 to Server-3. An instance of microservice-X runs in container-1 on Server-1. Instances of microservice-Y run in containers-2, -4, and -5 on Server-1, Server-2, and Server-3, respectively. In this example, it is assumed that the API frameworks for containers-2 and -4 support the same memory representation of the API request / response structure as the API framework for container-1, while the API framework for container-5 does not. It is also assumed that the instance of microservice-Y in container-4 supports RDMA.

[0034] Since Container-1 and Container-2 run on the same operating system and can access the same memory, a shared memory space is set up for the instance of Microservice-X in Container-1 and the instance of Microservice-Y in Container-2 to exchange API requests / responses. Further, the instance of Microservice-X implemented in Container-1 and the instance of Microservice-Y in Container-4 both support RDMA and can use the same memory representation of the API request / response structure. In this case, an RDMA connection is established between the instance of Microservice-X implemented in Container-1 and the instance of Microservice-Y in Container-4. In contrast, the instance of Microservice-Y in Container-5 does not support RDMA. In this case, a default HTTP / REST connection is established between the instance of Microservice-X implemented in Container-1 and the instance of Microservice-Y in Container-5.

[0035] To optimize application-level API interactions (e.g., microservice RPC), certain steps need to be taken when compiling microservice binary files to enable runtime inference of the best data transfer mechanism by the client API framework. Based on the deployment configuration parameters, the cloud orchestrator sets up the necessary environment at container instantiation to support alternative faster transfer mechanisms such as shared memory (e.g., via shared inter-process communication (IPC) namespaces) or the RDMA protocol. For Kubernetes, the configuration of the environment is achieved via device plugins and the container network interface (CNI). If the orchestrator or other management functions allow on-demand requests for establishing faster transfer modes (shared memory or RDMA support) at runtime, the API framework can utilize this feature.

[0036] When the API client and server binary files for microservices are compiled, the language-specific API framework library (or code generator) is modified to store language-specific information (such as programming language, compiler name, distribution / version, etc.) into the generated client and server binary files. This compile-time information (metadata) is exported to the control plane and stored together with the microservice binary files / images. This metadata can be used by the client-side API framework at runtime to determine whether the client and server can support the same memory representation of the API request / response structure.

[0037] The language-specific information can include any combination of the following non-limiting information:

[0038] · The programming language of the server API framework;

[0039] · The type of compiler used for the server API framework;

[0040] · The version of the compiler used for the server API framework;

[0041] · ABI of the server API framework;

[0042] · Supported hardware platforms (e.g., x86, ARM, etc.).

[0043] This information can be collected and embedded as part of the program, or provided as metadata along with the program or through the API.

[0044] In some embodiments, the program can be compiled to support more than one format.

[0045] In some embodiments, additional information can be included to indicate compatibility or equivalence. Table 1 below shows an example of compatibility information.

[0046]

[0047] Table 1: Example of compatibility information

[0048] In some embodiments, the application developer can specify some possible targets for the server microservices, e.g., which language / version or architecture the server can be in the deployment, and the compiler can generate a server target-specific representation for optimization. Another option is to store information indicating whether to use a language-agnostic representation such as Apache Arrow.

[0049] Regardless of the deployment method used, compile-time optimization enables the decision on whether the client and the server can avoid data serialization by using the same memory API request / response representation or pre-generated stubs.

[0050] At runtime, the API client typically requests the API framework layer to establish an API request / response channel or connection with a specific API server. This request can be an explicit leading request to the API framework. Alternatively, the request-response channel can be automatically established in response to the first API request sent by the client.

[0051] When the client - side API framework receives an explicit or implicit request to establish an API request / response channel, it can determine whether the API client and server can avoid serialization costs and use an optimized transport mechanism for message exchange between the client and the server. Generally, establishing an optimized channel for API request / response requires two conditions to be met. First, the client - side API framework needs to determine the supported data transport modes for the server. As will be described below, this step involves new interactions between the data plane and the control plane. Second, the client - side API framework needs to determine whether the client and server support the same memory representation of the API request / response structure. This step can involve interactions between the data plane and the control plane or the exchange of messages between the client - side API framework and the server - side API framework. If the server supports the same optimized data transport mode and the same memory representation of the API request / response structure, an optimized channel can be established for API request / response exchange.

[0052] In the first step, when the client microservice triggers a connection request, the client - side API framework queries the control - plane orchestration system (e.g., Kubernetes orchestrator) to obtain details of the corresponding server microservice, such as the server location (e.g., whether the client and server are co - located on the same operating system instance) and available data transport modes (e.g., shared memory between co - located client and server, RDMA between client and server on different nodes, etc.). In a non - limiting example of a Kubernetes environment, the API framework can query the Kubernetes API server. The orchestration system or control plane already has information about the API service name, (one or more) service IP addresses, ports, service location between cluster nodes, and possible communication modes (e.g., traditional HTTP - HTTP / 2, RDMA, shared memory, etc.). The cloud control plane resolves the query by responding with details of the server microservice, including location information (service IP and port addresses, server details, etc.) and available data transport modes, which can include shared memory (if on the same operating system); RDMA (if the server and client can communicate via InfiniBand or RDMA over Converged Ethernet (RoCE)), or HTTP / TCP (default). As an example, to use shared memory, the containers must have a shared IPC namespace, which may also be a deployment configuration known to the orchestration system.

[0053] The second step is to determine whether the same memory representation is supported, which can involve interaction with the control plane or can be based on the preamble information exchanged between the client-side API framework and the server-side API framework. In the control plane approach, the client-side API framework initially queries the control plane to retrieve server-side language identifier specific information (such as programming language, version, server-side byte order, etc.). The control plane responds with server identifier specific information that indicates a service with the ability to support the same memory representation of the API request / response structure. In some embodiments, the server identifier specific information can be a separate query-response between the client and the control plane. Alternatively, the control plane can provide the server identifier specific information to answer the initial query, so no additional query is needed.

[0054] In the preamble exchange approach, the client-side API framework uses a default protocol (e.g., HTTP, HTTP / 2, etc.) to initiate and negotiate an initial connection with the server-side API framework. The client and the server exchange preamble information, which includes language specific information and API related information. During this negotiation, the client-side API framework sends language identifier specific information about the language identifier of the API client to the server. The server-side API framework responds with similar server-side information. Thus, each side knows whether the same memory representation is supported. Based on the exchanged preamble information, the client-side RPC framework will decide the best possible mode of the data transfer mechanism.

[0055] If the server is co-located on the same operating system and can support the same memory representation, the client-side API framework can choose "shared memory" as the preferred transport mode. As an illustrative mechanism, if the client-side API framework determines that the client and the server have the same IPC namespace for shared memory and support the same memory representation for requests and responses, the client API framework can request the server to create and share the details of the shared memory region. In response, the server-side API framework creates a shared memory segment and shares the shared memory region key or identifier with the client-side API framework via an existing (default) communication channel. Both the client-side and the server-side API frameworks can map the shared memory region to their address spaces and then can use the shared memory to exchange API requests and responses, which is faster without the need for serialization.

[0056] Note that the server can create a common large shared memory pool (with appropriate locking) among multiple clients or can set up the shared memory for each client. Data encryption between the client and the server is not affected; the client and the server can choose to encrypt the memory data if they so choose.

[0057] If the server and the client are not co-located but support the same memory representation, the client API framework selects "RDMA" as the preferred transport mode. The client API framework may request the server API framework to set up the server-side RDMA. In response to this request, the server-side API framework creates an RDMA memory region, a send-receive queue pair, and a completion queue, and listens for connection requests from the client. The server-side API framework exchanges RDMA-related details with the client via an existing (default) communication channel. Meanwhile, the client sets up the RDMA channel and connects and establishes a connection with the RDMA server. Once the channel is established, RDMA can be used to transfer API calls exchanged between the corresponding server and client without serialization.

[0058] If no alternative faster transport mechanism is available, the client and the server continue to pass API calls via the default mechanism (e.g., HTTP, HTTP / 2).

[0059] Once the initial setup is complete, the API framework is ready to use an alternative transport mechanism (e.g., shared memory) for direct sharing of API requests and responses without serialization and uses locks as needed for appropriate protection.

[0060] The optimized data described in this document has minimal impact on or imposes additional requirements on existing fault handling mechanisms in such an API framework or control plane. For example, in the case of a failure of a microservice or a container, the cloud orchestrator will typically reinstantiate the microservice (not necessarily on the same node). Additionally, existing API framework libraries of the prior art typically attempt to rediscover peers / servers and / or API retries. Therefore, in such a scenario, the client API framework will simply repeat the process of establishing a connection with the server (or establishing a faster transport mode) as described previously, for which they will query the control plane, etc. to obtain the best peer / server details and establish a faster communication channel step by step if feasible.

[0061] Figure 3 A scenario where microservices communicate via a load balancer is shown. The control plane provides a list of potential remote services to which the client can connect. The client selects a suitable candidate from the list and sends a request to establish a connection. The request includes an indication of the selected server microservice. After receiving the request, the load balancer transparently forwards the message to the destination based on the destination information present in the client request. If no destination is specified in the request, the load balancer can use appropriate information and algorithms to infer the remote service and forward the connection request.

[0062] In cases where there is a load balancer in the deployment to spread client RPC requests across a pool of RPC server instances, the API client framework can query the load balancer and control plane to connect to the most likely instance of the API server microservice. The criteria for deciding the best instance can be the client and server locations and other metadata about the client and server available to the control plane and load balancer (e.g., metadata tags for client and server language / release information).

[0063] In some embodiments, the API framework can influence the decisions of the control plane and load balancer: unlike the client - side API framework library that only queries information, certain optimizations for low - latency interactions may be influential in nature. For low - latency scenarios, the API framework / library can recommend to, or otherwise influence, the control plane and load - balancing decisions to connect to the server instance preferred by the client API framework.

[0064] Figure 4 An exemplary process of using a control - plane query to obtain information for setting up optimized data transfer is shown. The cloud control plane instantiates client and server microservices with the required binaries to enable shared memory and RDMA and stores the necessary information about the deployment configuration (S1). After deployment, the client application initiates connection establishment with the server application by sending an explicit request to the client API framework or by sending a client request (S2). The client - side API queries the control plane to obtain information about server details (S3). The control plane responds with the service location (e.g., IP address and port) and available data - transfer modes for one or more servers (S4). The client - side API framework sends a second query to the control plane to obtain information about the server identities of the services that indicate the same memory representation capable of supporting API requests / responses (S4). In response, the control plane provides server - specific information such as programming language, byte order, etc. (S5). Based on the information obtained, the client - side API framework infers the preferred server instance and uses a default protocol (such as HTTP / REST) to establish an initial connection with the preferred server instance (S6). Once the default connection is established, the client - side API initiates the establishment of an optimized channel and invites the selected server instance to join (S7). Then, the selected server instance can join and switch to the optimized channel (S8). Thereafter, the optimized channel can be used by the client and server to exchange API requests and responses (S9).

[0065] Figure 5An exemplary process of using leading exchange to obtain information for establishing optimized data transfer is shown. The cloud control plane instantiates client and server microservices with the required binaries to enable shared memory and RDMA, and stores the necessary information about the deployment configuration (S11). After deployment, the client application initiates connection establishment with the server application by sending an explicit request to the client API framework or by sending a client request (S12). The client-side API queries the control plane to obtain information about server details (S13). The control plane responds with the service location (e.g., IP address and port) and the available data transfer modes for one or more servers (S14). The client-side API framework uses a default protocol (such as HTTP / REST) to establish an initial connection with the preferred server instance (S15). During the negotiation of the initial connection, the client API framework and the server API framework exchange information about server-specific and language-specific details, such as programming language, byte order, etc. (S16). Based on the information received from the control plane and during the exchange with the server-side API framework, the client-side API framework infers the preferred data transfer method and establishes an optimized channel for the exchange of API requests / responses (S17). Then, the selected server instance can join and switch to the optimized channel. Thereafter, the optimized channel can be used by the client and the server to exchange API requests and responses (S18).

[0066] Figure 3 and Figure 4 The basic process shown can be easily modified to allow the server-side API framework to select the preferred data transfer mode.

[0067] Figure 6 An exemplary method 200 for transferring data between a client and a server is shown. As used herein, the client is an instance of a client or server microservice implemented in a cloud server, which may include a physical server, a VM, or a container. The client receives information indicating the data transfer capabilities for the communication of API requests / responses between the client and the server supported by a remote API framework in the server, and the client and the server may reside on the same server or different servers (block 210). The server may reside on the server as a client. The client determines that the client API framework and the server API framework support the same data transfer mode and the same memory representation of the API requests / responses based on the received information (block 220). In response to this determination, the client uses the same memory representation of the API requests / responses to establish a direct memory access channel for the communication of API requests / responses between the client and the server (block 230). The client uses the direct memory access channel to exchange API messages with the server (block 240).

[0068] Some embodiments of method 200 also include using data serialization to establish an initial connection for communication of API requests / responses between a client and a server.

[0069] In some embodiments of method 200, receiving information indicating available data transfer capabilities for communication of API requests / responses between a client and a server supported by a remote API framework further includes: receiving from a control plane entity information indicating available data transfer modes supported by the remote API framework.

[0070] In some embodiments of method 200, receiving information indicating available data transfer capabilities for communication of API requests / responses between a client and a server supported by a remote API framework further includes: receiving from a control plane entity information indicating available languages supported by the remote API framework.

[0071] In some embodiments of method 200, receiving information indicating available data transfer capabilities for communication of API requests / responses between a client and a server supported by a remote API framework further includes: receiving from the remote API framework information indicating available languages supported by the remote API framework.

[0072] Some embodiments of method 200 also include: receiving location information indicating the location of the server.

[0073] In some embodiments of method 200, establishing a direct memory access channel for communication of API requests / responses between a client and a server includes: based on the location information, determining that the client and the server can access a shared memory; and using the shared memory to establish the direct memory access channel.

[0074] In some embodiments of method 200, using the shared memory to establish the direct memory access channel includes: sending a request to the remote API framework via the initial connection to establish shared memory access; receiving from the remote API framework via the initial connection configuration information indicating a shared memory area for shared memory access; and mapping the shared memory area to the address space of the local API framework.

[0075] In some embodiments of method 200, establishing a direct memory access channel for communication of API requests / responses between a client and a server includes: based on the location information, determining that the client and the server cannot access the same shared memory; and using the remote memory of the server to establish the direct memory access channel.

[0076] In some embodiments of method 200, using a shared memory to establish a direct memory access channel includes: sending a request via an initial connection to a remote API framework to establish a remote direct memory access (RDMA); receiving, via the initial connection, configuration information for the RDMA connection from the remote API framework; and using the configuration information to establish the RDMA connection.

[0077] Figure 7 A client 300 according to an embodiment is shown. The client 300 includes a receiving unit 310, a determining unit 320, an establishing unit 330, and a swapping unit 340. Each of the units 310 - 340 may include one or more processors, hardware, firmware, software, or a combination thereof. The receiving unit 310 is configured to receive information indicating a data transfer capability for communication of API requests / responses between the client and the server supported by the server API framework. The determining unit 320 is configured to determine, based on the received information, that the client API framework and the server API framework support the same data transfer mode and the same memory representation of the API requests / responses. The establishing unit 330 is configured to, in response to the determination, establish a direct memory access channel for communication of API requests / responses between the client and the server using the same memory representation of the API requests / responses. The swapping unit 340 is configured to exchange API messages with the server using the direct memory access channel.

[0078] Figure 8 A method 250 for optimizing data transfer between a client and a server implemented by an orchestrator or other management function in a microservices-based communication network is shown. The orchestrator deploys the client and the server, and the server is configured to use direct memory transfer for the exchange of API requests / responses with the server (block 260). The orchestrator also receives a first query from the client regarding the data transfer capability of the API server framework of the server (block 270). In response to the first query, the orchestrator provides the client with information indicating the data transfer capability for communication of API requests / responses between the client and the server supported by the server API framework (block 280).

[0079] In some embodiments of method 250, providing the client with information indicating the data transfer capability supported by the server API framework in response to the first query includes: providing the client with the data transfer mode supported by the server API framework.

[0080] Some embodiments of method 250 further include: receiving, from a client, a second query for language-specific server API information indicating a memory representation of an API request / response supported by a server API framework (block 290). In this embodiment, in response to the second query, the management function provides the client with language-specific server API information indicating a memory representation of an API request / response supported by the server API framework (block 295).

[0081] In some embodiments of method 250, providing the client with information indicating data transfer capabilities supported by a server API framework in response to a first query includes: providing the client with: a data transfer mode supported by the server API framework, and language-specific server API information indicating a memory representation of an API request / response supported by the server API framework.

[0082] In some embodiments of method 250, the language-specific server API information includes one or more of the following: the programming language of the server API framework; the type of compiler for the server API framework; the version of the compiler for the server API framework; the ABI of the server API framework; the supported hardware platforms.

[0083] Figure 9 A management function 350 according to an embodiment is shown. The management function 350 includes a deployment unit 360, a receiving unit 370, and a response unit 380. Each of the units 360 - 380 may include one or more processors, hardware, firmware, software, or a combination thereof. The deployment unit 360 is configured to deploy a client and a server, the server being configured to use direct memory transfer for the exchange of API requests / responses with the server. The receiving unit 370 is configured to receive, from the client, a first query for the data transfer capabilities of a server API framework of the server. The response unit 380 is configured to receive, from the client, a first query for the data transfer capabilities of a server API framework of the server.

[0084] In some embodiments, the receiving unit is further configured to receive, from the client, a second query for language-specific server API information indicating a memory representation of an API request / response supported by the server API framework. In this embodiment, the providing unit 370 is further configured to, in response to the second query, provide the client with language-specific server API information indicating a memory representation of an API request / response supported by the server API framework.

[0085] Figure 10Shows a host device 400 according to an embodiment. The host device 400 can be configured to function as a client, server, or management function as described herein. The host device 400 includes: an interface circuit 420 to enable communication with other network nodes in a communication network; a processing circuit 430 to control the operation of the network node 400; and a memory 440 to store computer programs and data required for operation.

[0086] The interface circuit 420 couples the host device 400 to a communication network for communicating with other network nodes in a wireless communication network. The interface circuit 420 can include a wired or wireless interface operating according to any standard, such as Ethernet, Wi-Fi (Wireless Fidelity), and SONET (Synchronous Optical Network) standards.

[0087] The processing circuit 430 controls the overall operation of the data analysis system 300. The processing circuit 430 can include one or more microprocessors, hardware, firmware, or a combination thereof. In some embodiments, the host device is configured to function as a client. When configured as a client, the processing circuit 430 is configured to query the control plane and exchange data with a server, as described herein. In one embodiment, the processing circuit 430 is configured to execute Figure 6 Method 200. In other embodiments, the host device 400 is configured to function as an orchestrator or other management function. When configured as an orchestrator, the processing circuit 430 is configured to deploy clients and servers and respond to queries from clients, as described herein. In one embodiment, the processing circuit 430 is configured to execute Figure 8 Method 250.

[0088] The memory 440 includes both volatile and non-volatile memory for storing computer program code and data required for operation by the processing circuit 430. The memory 440 can include any tangible non-transitory computer-readable storage medium for storing data, including electronic, magnetic, optical, electromagnetic, or semiconductor data storage. The memory 440 stores a computer program 450 including executable instructions that configure the processing circuit 430 to implement Method 200 (for client functionality) as described herein or Figure 6 or Figure 8Method 250 (for management functions). In this regard, computer program 450 may include one or more code modules corresponding to the devices or units described above. Generally, computer program instructions and configuration information are stored in non-volatile memory (such as ROM, erasable programmable read-only memory (EPROM), or flash memory). Temporary data generated during operation may be stored in volatile memory, such as random access memory (RAM). In some embodiments, computer program 450 for configuring processing circuit 430 as described herein may be stored in removable memory, such as a portable optical disc, a portable digital video disc, or other removable media. Computer program 450 may also be embodied in a carrier, such as an electronic signal, an optical signal, a radio signal, or a computer-readable storage medium.

[0089] Those skilled in the art will also understand that the embodiments herein also include corresponding computer programs. The computer program includes instructions that, when executed on at least one processor of the device, cause the device to perform any of the corresponding processes described above. The computer program in this regard may include one or more code modules corresponding to the devices or units described above.

[0090] The embodiments also include a carrier containing such a computer program. The carrier may include one of an electronic signal, an optical signal, a radio signal, or a computer-readable storage medium.

[0091] In this regard, the embodiments herein also include a computer program product stored on a non-transitory computer-readable (storage or recording) medium and including instructions that, when executed by a processor of the device, cause the device to perform as described above.

[0092] The embodiments also include a computer program product that includes a program code portion for performing the steps of any of the embodiments herein when the computer program product is executed by a computing device. The computer program product may be stored on a computer-readable recording medium.

Claims

1. A method implemented by a client for exchanging API messages with a server, the method comprising: Receiving information indicating data transfer capabilities for API request / response communication between the client and the server supported by a remote API framework in the server; Based on the received information, determining that the client API framework and the server API framework support the same data transfer mode and the same memory representation of the API request / response structure; In response to the determination, using the same memory representation of the API request / response to establish a direct memory access channel for the API request / response communication between the client and the server; And Using the direct memory access channel and the same memory representation of the API request / response structure to exchange API messages with the server.

2. The method according to claim 1 further comprises: Using data serialization to establish an initial connection for API request / response communication between the client and the server.

3. The method according to claim 1 or 2, wherein Receiving information indicating available data transfer capabilities for API request / response communication between the client and the server supported by the remote API framework includes: Receiving from a control plane entity information indicating available data transfer modes supported by the remote API framework.

4. The method according to claim 3, wherein, Receiving information indicating the available data transfer capabilities for API request / response communication between the client and the server supported by the remote API framework further includes: Receiving from the control plane entity information indicating available languages supported by the remote API framework.

5. The method according to claim 1 or 2, wherein Receiving information indicating the available data transfer capabilities for API request / response communication between the client and the server supported by the remote API framework further includes: Receiving from the remote API framework information indicating available languages supported by the remote API framework.

6. The method according to any one of claims 2 to 5, further comprising: Receiving location information indicating the location of the server.

7. The method according to claim 6, wherein, Establishing a direct memory access channel for the API request / response communication between the client and the server includes: Based on the location information, determining that the client and the server can access a shared memory; and Using the shared memory to establish the direct memory access channel.

8. The method according to claims 2 and 7, wherein, Using the shared memory to establish the direct memory access channel includes: Sending a request to the remote API framework on the initial connection to establish shared memory access; and Receiving from the remote API framework on the initial connection configuration information indicating a shared memory area for shared memory access; and Mapping the shared memory area to the address space of the local API framework.

9. The method according to claim 6, wherein Establishing a direct memory access channel for the API request / response communication between the client and the server includes: Based on the location information, determining that the client and the server cannot access the same shared memory; and Using the remote memory of the server to establish the direct memory access channel.

10. The method according to claims 2 and 9, wherein, Using the shared memory to establish the direct memory access channel includes: Send a request on the initial connection to the remote API framework to establish Remote Direct Memory Access (RDMA); and Receive, on the initial connection, configuration information for the RDMA connection from the remote API framework; and Use the configuration information to establish the RDMA connection.

11. A client in a microservices-based communication network, the client being configured to: Receive information indicating the data transfer capabilities for API request / response communication between the client and the server supported by an API framework in the server; Based on the received information, determine that the client API framework and the server API framework support the same data transfer mode and the same memory representation of the API request / response; In response to the determination, use the same memory representation of the API request / response to establish a direct memory access channel for the API request / response communication between the client and the server; And Use the direct memory access channel and the same memory representation of the API request / response structure to exchange API messages with the server.

12. The client according to claim 11, further configured to perform the method according to any one of claims 2 to 10.

13. A computing device in a microservices-based communication network, the client comprising: Interface circuitry for communicating with a server; And Processing circuitry operably coupled to the interface circuitry and configured to: Receive information indicating the data transfer capabilities for API request / response communication between the client and the server supported by an API framework in the server; Based on the received information, determine that the client API framework and the server API framework support the same data transfer mode and the same memory representation of the API request / response; In response to the determination, use the same memory representation of the API request / response to establish a direct memory access channel for the API request / response communication between the client and the server; And Use the direct memory access channel to exchange API messages with the server.

14. The computing device according to claim 13, wherein, The processing circuitry is further configured to perform the method according to any one of claims 2 to 10.

15. A computer program comprising executable instructions which, when executed by processing circuitry in a computing device, cause it to perform the method according to any one of claims 1 to 10.

16. A carrier, comprising the computer program according to claim 15, wherein, The carrier is one of an electronic signal, an optical signal, a radio signal, or a computer-readable storage medium.

17. A non-transitory computer-readable storage medium containing a computer program, the computer program comprising executable instructions which, when executed by processing circuitry in a client, cause it to perform the method according to any one of claims 1 to 10.

18. A method implemented by a management function in a microservices-based communication network, the method comprising: Deploy a client and a server, the server being configured to use direct memory access for the exchange of API requests / responses with the server; Receive a first query from the client regarding the data transfer capabilities of the server's API server framework; And In response to the first query, provide the client with information indicating the data transfer capabilities for the communication of API requests / responses between the client and the server supported by the server API framework.

19. The method according to claim 18, wherein, In response to the first query, providing the client with information indicating the data transfer capabilities supported by the server API framework includes: providing the client with the data transfer modes supported by the server API framework.

20. The method according to claim 19, further comprising: Receive a second query from the client regarding language-specific server API information indicating the memory representation of API requests / responses supported by the server API framework; And In response to the second query, provide the client with language-specific server API information indicating the memory representation of API requests / responses supported by the server API framework.

21. The method according to claim 18, wherein, In response to the first query, providing the client with information indicating the data transfer capabilities supported by the server API framework includes providing the client with: The data transfer modes supported by the server API framework; And Language-specific server API information indicating the memory representation of API requests / responses supported by the server API framework.

22. The method according to claim 20 or 21, wherein The language-specific server API information includes one or more of the following: The programming language of the server API framework; The type of compiler for the server API framework; The version of the compiler for the server API framework; The ABI of the server API framework; The supported hardware platforms.

23. A management function in a microservices-based communication network for optimizing data exchange between a client and a server, the management function being configured to: Deploy a client and a server, the server being configured to use direct memory access for the exchange of API requests / responses with the server; Receive a first query from the client regarding the data transfer capabilities of the server's API server framework; And In response to the first query, provide the client with information indicating the data transfer capabilities for the communication of API requests / responses between the client and the server supported by the server API framework.

24. The management function according to claim 23 is further configured to perform the method according to any one of claims 19 to 22.

25. A computing device in a microservices-based communication network for optimizing data exchange between a client and a server, the management device being configured to: An interface circuit for communicating with the client; And A processing circuit operably coupled to the interface circuit and configured to: Deploy a client and a server, the server being configured to use direct memory transfer for the exchange of API requests / responses with the server; Receive a first query from the client regarding the data transfer capabilities of the API server framework of the server; And In response to the first query, provide the client with information indicating the data transfer capabilities for the communication of API requests / responses between the client and the server supported by the server API framework.

26. The computing device according to claim 25, wherein, The processing circuit is further configured to execute the method according to any one of claims 19 to 22.

27. A computer program comprising executable instructions which, when executed by a processing circuit in a computing device, cause it to execute the method according to any one of claims 18 to 22.

28. A carrier, comprising the computer program according to claim 27, wherein, The carrier is one of an electronic signal, an optical signal, a radio signal, or a computer-readable storage medium.

29. A non-transitory computer-readable storage medium containing a computer program, the computer program comprising executable instructions which, when executed by a processing circuit in a computing device, cause it to execute the method according to any one of claims 18 to 22.

Images