Self-orchestrating applications
By integrating a service mesh layer directly into runtime instances of distributed computing services, the challenges of resource consumption, complexity, and security in existing solutions are addressed, enabling efficient and scalable service mesh management for distributed computing environments.
Patent Information
- Application Number
- PCT/EP2024/062142
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-10
- Filing Date
- 2024-05-02
- Publication Date
- 2024-11-14
AI Technical Summary
Existing service mesh integration solutions in distributed computing environments, such as those using proxy sidecars, face issues like significant hardware resource consumption, implementation complexities, debugging challenges, latency, security concerns, and compatibility problems, especially in resource-intensive applications.
Integrating a service mesh layer directly into runtime instances of distributed computing services, eliminating the need for proxy sidecars by incorporating a control plane and data plane component within the application itself, enabling self-orchestrating applications that manage their deployment and operation autonomously.
This approach reduces hardware resource consumption, simplifies implementation and debugging, minimizes latency, enhances security, and improves compatibility, allowing for efficient and scalable service mesh management without the overhead of external proxy sidecars.
Smart Images

Figure EP2024062142_14112024_PF_FP_ABST
Abstract
Description
SELF-ORCHESTRATING APPLICATIONSTECHNICAL FIELD
[0001] The disclosure relates generally to distributed computing. More specifically, the disclosure relates to a computer-implemented method for managing service mesh integration of a distributed computing environment. The disclosure also relates to an associated computer-implemented system, a non-transitory computer-readable storage medium, and a computer program product.BACKGROUND
[0002] In a distributed computing environment data is shared among a plurality of distributed units. These units are commonly referred to as computing nodes or services (e.g. microservices or nanoservices), and the following disclosure will adhere to either one of these terms. Despite the data being shared and distributed across multiple locations, actions or tasks associated with said data are still executed as if they are run on a computing environment of a single computer or node, thus making it possible to decentralize the execution of, for instance, a monolithic application program. Distributed computing environments provide many advantageous technical effects over conventional non-distributed computing environments relating to performance, resilience, robustness and scalability, to name a few advantages. Distributed computing models are hence becoming a very popular computing model in, e.g., database or application design, especially in line with the constant desire for new computing opportunities, more data and computationally demanding technical areas of applicability.
[0003] A service mesh is a configurable infrastructure layer for distributed computing services that facilitates flexible, reliable and fast communication between a plurality of such distributed computing services. Service mesh integration refers to the process of incorporating a service mesh into an application program, i.e., the application program to be executed in a distributed way by the distributed computing services. Service mesh integration typically involves configuring a control plane component and a data plane component of the service mesh layer to function properly in conjunction with the tasks to be carried out by the application program. Some service mesh integration configurations also rely on adaptations to the software code of the application program such that the integration is compatible with the particular application program to be executed. Service mesh integration aims to solve problems relating to, for instance, service discovery, load balancing, failure recovery, metrics,and monitoring and management of network loads across the services of the distributed computing environment.
[0004] The prior art suggests various solutions for managing service mesh integration, such as the software tools Istio, Linkerd, Consul, AWS App Mesh, Kuma, Traefik Mesh and Open Service Mesh. These tools use a proxy sidecar, and more specifically an Envoy proxy server, to manage and secure communication between the distributed computing services. Other proxy sidecars exist in the art, such as NGINX reverse proxy or HAProxy. A proxy sidecar is a container (or “application container”) that runs alongside an associated service, for instance in a pod of a cluster. The cluster may be orchestrated using programs such as Kubemetes, Docker Swarm or Amazon ECS. The sidecar of these tools, and corresponding sidecar proxy patterns of other similar tools, include features that are responsible for handling the above-mentioned problems of service discovery, load balancing, failure recovery, metrics, and monitoring and management of network loads. Moreover, alternative tools known in the art uses other types of external applications or services with a similar purpose as the proxy sidecar. It is understood that said other types of external applications or services share similar disadvantages as the proxy sidecar.
[0005] The present inventors have identified a plurality of undesired implications relating to using the proxy sidecar pattern for managing service mesh integration, implications which involve, for instance, significant hardware resource consumption, implementation and debugging complexities, system latencies, security compromises and compatibility issues. The proxy sidecar pattern adds a proxy to each instance of a distributed computing service, thus significantly increasing the hardware resource consumption. This is especially the case for application programs inherently requiring large amounts of computing power in order to execute, such as video editing software, 3D modelling / animation software, virtualization software or machine learning / Al software, to name a few examples. Moreover, implementing and managing a sidecar proxy for each service instance incur complexities, especially for larger distributed computing environments having many services. Additionally, the use of sidecar proxies is unreliable when it comes to debugging as the traffic has to flow through an additional layer. Said additional layer also introduces an additional network hop which can negatively affect latencies such that the performance of the application is impacted. In addition, the additional layer of the sidecar proxies introduces additional attack surfaces for a person or software with malicious intent. Hence, it involves careful management of e.g., keys and security certificates. Further, some application programs or network protocols areincompatible with the sidecar proxy pattern due to architectural design choices or other related requirements.SUMMARY
[0006] It is accordingly an object of the present disclosure to manage service mesh integration of a distributed computing environment that can solve, or at least mitigate, the above-mentioned deficiencies of the prior art.
[0007] According to a first aspect of the disclosure, a computer-implemented method for managing service mesh integration of a distributed computing environment is provided. The method comprises configuring a service mesh layer of one or more distributed computing services being orchestrated by the distributed computing environment, wherein the service mesh layer comprises a control plane component and a data plane component, the control plane component being configured to control forwarding of application data and networking data between said one or more distributed computing services, the data plane component being configured to carry out said forwarding of said application data and networking data between said one or more distributed computing services; and integrating the control plane component and the data plane component of the service mesh layer into runtime instances of said one or more distributed computing services.
[0008] According to a second aspect, a computer-implemented system for managing service mesh integration of a distributed computing environment is provided. The system comprises one or more processing units being configured to provide a service mesh layer of one or more distributed computing services being orchestrated by the distributed computing environment, wherein the service mesh layer comprises a control plane component and a data plane component, the control plane component being configured to control forwarding of application data and networking data between said one or more distributed computing services, the data plane component being configured to carry out said forwarding of said application data and networking data between said one or more distributed computing services; and integrate the control plane component and the data plane component of the service mesh layer into runtime instances of said one or more distributed computing services.
[0009] According to a third aspect, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium comprises instructions, which when executed by one or more processing units, cause the processing unitsto perform the method of the first aspect.
[0010] According to a fourth aspect, a computer program product is provided. The computer program product comprises instructions which, when executed by one or more processing units, cause the processing units to perform the method of the first aspect.
[0011] In the present disclosure, the service mesh layer is integrated and thereby built directly into the runtime instances, which allows for execution of the application program directly within the runtime instances and enabling the functionality of the service mesh layer integrated therein. A type of all-in-one service mesh orchestrator and scheduler is thus obtained, where there is no need to include the proxy sidecar, or any other type of external service / application for managing the functionality of a service mesh orchestrator / scheduler, as used in the prior art solutions. Applications using this type of all-in-one service mesh orchestrator and scheduler are so-called self-orchestrating applications, i.e., capable of automatically managing and coordinating their own deployment and operation. The functionality of a traditional service mesh is built into the self-orchestrating applications, the individual applications thereby in practice replacing the functionality of the traditional service mesh without requiring an extra layer of complexity as is the case with prior art using solutions similar to a proxy sidecar. Hence, integrating the service mesh layer into the runtime instances shall in this context be interpreted as additionally or alternatively doing so into generated networking code as they interact with one another, and consequently also into the application itself. One or more of the undesired implications as indicated in the Background section are accordingly prevented, or at least mitigated.
[0012] Further technical advantages of the teachings herein are given by the appended embodiments / claims as well as the respective detailed descriptions of the embodiments.
[0013] The above inventive aspects, accompanying claims / clauses, and / or examples disclosed herein above and later below may be suitably combined with each other as would be apparent to anyone of ordinary skill in the art.
[0014] Additional features and advantages are disclosed in the following description, claims, and drawings, and in part will be readily apparent therefrom to those skilled in the art or recognized by practicing the disclosure as described herein. There are also disclosed herein control units, computer readable media, and computer program products associated with the above discussed technical benefits.BRIEF DESCRIPTION OF THE DRAWINGS
[0015] With reference to the appended drawings, below follows a more detailed description of aspects of the disclosure cited as examples.
[0016] FIG. 1 shows an exemplary computer-implemented system configured for managing service mesh integration.
[0017] FIG. 2 shows an exemplary resource group deployment approach.
[0018] FIG. 3 is an exemplary schematic block diagram of a meshed networking model.
[0019] FIG. 4 is an exemplary schematic block diagram of an internal networking model.
[0020] FIG. 5 is an exemplary schematic block diagram of a traditional networking model.
[0021] FIG. 6 is an exemplary schematic block diagram of a hybrid networking model.
[0022] FIG. 7 is schematic block diagram of some exemplary platform components.
[0023] FIG. 8 is an exemplary schematic block diagram of a computer-implemented system.
[0024] FIG. 9 is an exemplary schematic block diagram of a computer-implemented method.
[0025] FIG. 10 is an exemplary visualization of a non-transitory computer-readable storage mediumDETAILED DESCRIPTION
[0026] The present invention will now be described more fully hereinafter. The invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those persons skilled in the art. Like reference numbers refer to like elements or method steps throughout the description.
[0027] FIG. 1 shows an exemplary schematic block diagram of how service mesh integration is managed according to some of the teachings of the claimed solution.
[0001] The schematic block diagram depicts a computer-implemented system 200. The system 200 may operate in a production mode or a preparation mode. The preparation mode may include development, testing and staging modes. In the production mode, the computer- implemented system 200 refers to the state or condition where the system is running live and in use by end-users to perform real-world tasks, and the system 200 is expected to operate at optimal performance, with high availability, stability, and security. In the preparation mode,the computer-implemented system 200 is not yet ready for live use by end-users, but is instead being developed, sandboxed, tested, or prepared for deployment. The preparation mode may be interpreted as any stage before said deployment, with ongoing coding and design changes, subjected to various tests to validate its functionality, performance, and reliability, and so forth. The switch from one mode to another usually involves a formal process of testing, validation, and release management to ensure the smooth and error-free operation of the system in its intended environment.
[0028] The system 200 is configured to execute a step of providing a software application program 30 (which is herein interchangeably referred to as an “application program”, “application” or “program”, for reasons of brevity). The software application program 30 comprises software application code which, when executed by a processing device or unit, cause said processing device or unit to carry out one or more functionalities. The application program 30 is a type of computer software designed to perform specific computerized tasks. The application program 30 can range from simple, single-purpose tools like a calculating program, to complex, multi-faceted and data demanding programs like video editing software, word processors, search engines, artificial intelligence-powered services, and so forth. The application program 30 may be designed to run on a variety of different operating systems, devices, apparatuses, servers, etc., including personal computers, laptops, smartphones, tablets, data centers, loT devices, microprocessors, or other exemplary devices known to the skilled person. The application program 30 may be developed by third-party companies and are designed to solve specific problems or perform specific functions for the user. In alternative examples, a plurality of application programs 30 may be provided, each one of said plurality of application programs 30 having respective software application code.
[0029] The application program 30 comprises software application code. The software application code may be stored in a package storage resource. The package storage resource may be used in the production mode of the system 200 to store data not yet being used for networking code generation. The package storage resource may include associated package files (for instance in TOML, YAML, JSON, XML, INI, HCL formats, but not limited to such formats), package metadata and / or modules including respective programming languages, dependencies, capabilities and / or policies. In one example, the package storage resource is a package manager for an arbitrary runtime environment, such as node package manager (npm) for the JavaScript runtime environment Node.js, that manages and shares packages including libraries, tools, frameworks, etc., required for a project associated with the applicationprogram 30. The skilled person will appreciate that various other types of package managers may be equally or alternatively suitable for acting as the package storage resource.
[0030] The application program 30 may be a monolithic application program, i.e., a self- contained application being designed as a single service and independent from other applications. The present disclosure makes it possible to automatically distribute the execution of the application program 30 being a monolithic application program.
[0031] The software application code of the application program 30 is not limited to a particular abstraction level. To this end, the software application code may be provided in a high-level programming language. Alternatively, the software application code may be defined in any other level of abstraction, including but not limited to an intermediate representation format being any type of compiled language or conceivably even a machine level language. For example, the intermediate representation format may be in the form of a binary code such as Web Assembly Bytecode, a syntax tree, such as an abstract syntax tree, ELF code or machine code. Where the intermediate representation format is in the form of Web Assembly Bytecode, the providing of the software application code in the intermediate representation format is performed in compile-time or start-up operation, or by analyzing the software application code or the intermediate representation format thereof of the application program 30.
[0032] The application program 30 is configured to be executed by one or more distributed computing services 22 orchestrated by a distributed computing environment 20. To this end, the distributed computing services 22 are configured to provide a distributed execution of the application program 30 to cause control of one or more functionalities of the system 200, as discussed above. In the exemplary illustration, there are m number of distributed computing services 22a-m, where m is any arbitrary integer value adaptable depending on the type of environment 20 and / or application program 30 to be executed therein. An initial set-up of the distributed computing services 22a-n may be based on applying techniques readily known by the skilled person. The present disclosure is not limited to any such readily known orchestration scheme, since the service mesh integration is advantageously adaptable to function in conjunction with any suitable known tool applied in the prior art. For purely exemplary purposes, the skilled person may consider the following tools for orchestration of one or more distributed computing services 22a-m: container orchestration (e.g. using tools such as Kubernetes or Docker), cluster management (e.g. using tools such as Apache Mesos, DC / OS or Amazon ECS), workflow management (e.g. using tools such as Apache Airflow or AWS Step Functions), service discovery (e.g. using toolssuch as Consul, etcd or ZooKeeper) and load balancing (e.g. using tools such as HAProxy, NGINX or Amazon ELB), to name some examples. However, the sidecar proxy additions of these tools would not be used because of the inventive service mesh integration approach as is described in the present disclosure.
[0033] The distributed computing environment 20 may comprise one or more computing platforms. Such computing platforms may be cloud platforms (such as Amazon Web Services, Microsoft Azure or Google Cloud Platform), virtualized environments (such as Docker containers), on-premise servers, data centers, bare-metal servers or any other external device.
[0034] The distributed computing services 22a-m and a runtime instance 24 are two distinct, albeit related terms. Further, a node may be interpreted as either one of a distributed computing service 22a-m and a runtime instance 24. In the exemplary illustration, each distributed computing service 22a-m is associated with one or more runtime instances 24al- n; 24bl-n; 24ml-n, where n is an arbitrary integer value similar to the m value as discussed above. The values m and n are, however, not necessarily the same. Typically, each distributed computing service 22 is associated with one respective runtime instance 24, but this may vary over the course of the execution procedure of the application program 30, for instance depending on varying resource requirements. To this end, each one of the distributed computing services 22 may be configured to run multiple runtime instances, thereby allowing horizontal scaling for handling increase in traffic and workloads as well as a large number of simultaneously received requests. Alternatively or additionally, the distributed computing services 22a-m running in the distributed computing environment 20 can run on and across multiple runtime instances 24. A runtime is a software environment being configured to provide relevant resources for the execution of the application program 30, including e.g. memory management, input / output handling and threading. A distributed computing service 22, on the other hand, is a modular and independently deployable unit or software being configured to perform a particular function of the application program 30. In the exemplary illustration, two or more distributed computing services 22a-m are configured to communicate with one another to carry out the execution of a plurality of functions associated with the application program 30. Moreover, they are designed to be highly scalable, fault- tolerant and replaceable. While the distributed computing services 22 are runnable on a variety of runtime instances 24, they may be deployed in the distributed computing environment 20 using various cloud-related tools as discussed above, thus taking advantage of fault tolerance, scalability, and other features offered by the distributed computingenvironment 20. Alternatively or additionally, the (self-orchestrating) applications may have built-in fault tolerance, scaling and redundancy functionalities, etc., based on customized settings configured for the individual applications.
[0035] The distributed computing services 22 may be nanoservices or microservices. The distributed computing services 22 may be queryable. It is thus possible to query a service’s runtime instance to identify what capabilities are associated therewith such that data can be quickly retrieved.
[0036] Further shown in FIG. 1 is a service mesh layer 10. The service mesh layer 10 comprises a control plane component 12 and a data plane (or alternatively a forwarding plane) component 14. A “plane” is, as such, a concept known in the art which is to be interpreted as an entity where computer-implemented processes take place. Hence, a “plane component” is a computerized component being configured to carry out computerized processes. Although control planes and data planes are known concepts that are employed in conventional service mesh layers, the service mesh layer 10 and associated components 12; 14 according to the present disclosure are configured in an entirely different way which at least alleviates one or more of the disadvantages of the prior art, as discussed in the Background section.
[0037] The control plane component 12 is configured to control forwarding of application data and networking data. The application data refers to data used associated with application programs being executed in the computing environment 20, such as the application program 30. The networking data refers to data used to enable flexible, reliable and fast communication between the distributed computing services 22, such as communication between runtime instances 24al-n of a particular distributed computing service 22a or communication between runtime instances 24al-n; 24bl-n; 24ml-n of different distributed computing services 22a-m. To this end, the networking data may include network loads for managing load balancing between runtime instances 24. By “controlling” the forwarding of application and networking data, the control plane component 12 is configured for determining how packages of data are to be sent between the runtime instances 24, e.g. using routine tables, protocols, or other techniques readily known by the skilled person for controlling the forwarding of data.
[0038] The data plane component 14 is configured to carry out forwarding of the application data and networking data between the distributed computing services 22, i.e. by communicating the application and networking data between runtime instances 24. For instance, such forwarding may be carried out through netlink interfaces, as will be further discussed later on with reference to FIG. 3.
[0039] The control plane component 12 and the data plane component 14 are integrated into the runtime instances 24 of the distributed computing services 22. To this end, each one of the runtime instance 24 that are executing at least a portion of the application program 30 comprise the functionality of the service mesh layer 10. The runtime instances 24 are thus configured to act as respective hosts for workloads of the application program 30 running on the computing platform(s) of the distributed computing environment 20.
[0040] In some examples, the integration of the service mesh layer 10 into the respective runtime instances 24 comprises placing packages of software code of the service mesh layer 10 within said runtime instances 24. The packages of software code of the service mesh layer 10 may correspond to the functionality of the control plane component 12 and the data plane component 14. This can be interpreted as directly placing the computational logic of the service mesh layer 10 with the computational logic of the runtime instances 24. The packages of software code may be structured generally according to what is shown in FIG. 2, i.e., in several software components, each component having several software modules. Software modules will be described later on in this disclosure with further reference to FIG. 2.
[0041] Said packages of software code of the service mesh layer 10 may be configured to interact with networking code that is generated by the system 200. The networking code comprises the networking data that defines respective communication protocols at least of said one or more distributed computing services 22. A communication protocol is defined as a set of rules and standards that govern the exchange of data between entities in at least the distributed computing environment 20. Such rules and standards involve format and structure of the data, error detection and correction methods, and procedures for carrying out the transfer of data, for allowing said entities to effectively and reliably communication.Communication protocols may include both short-range or long-range standards known in the art. The following list of examples are some exemplary network / radio communication protocols that may defined by the networking code: HTTP(S), TCP / IP, UDP, FTP, SMTP, DNS, DHCP, SSH, POP3, SCP, NFS, SFTP, ICMP, ARP, RTP, RTCP IEEE 802.11, IEEE 802.15, ZigBee, WirelessHART, WiFi, Bluetooth®, BLE, RFID, WLAN, MQTT loT, CoAP, DDS, NFC, AMQP, LoRaWAN, Z-Wave, Sigfox, Thread, EnOcean, mesh communication, any form of proximity-based device-to-device radio communication, LTE Direct, W- CDMA / HSPA, GSM, UTRAN, LTE, IPv4, IPv6, 6L0WPAN, IrDA, or 5G NR. Other radio / network-based communication protocols may alternatively be realized.
[0002] The following list of examples are some exemplary light-based communication protocols that may be defined by the networking code: IrDA, VLC, Li-Fi(-X), FSOC, OCC,OWC, OBS, PON, or FSO. Other light-based communication protocols may alternatively be realized.
[0003] The following list of examples are some exemplary wired communication protocols that may be defined by the networking code: A2B, AFDX, ARINC 429, Byteflights, CAN, D2B, Ethernet, FlexRay, IDB-1394, lEBus, I2C, ISO 9141-1 / -2, J1708, J1587, J1850, J1939, ISO 11783, KWP2000, LIN, MOST, Multifunction Vehicle Bus, SMARTwireX, SPI, VAN or UAVCAN. Other wired communication protocols may alternatively be realized.
[0004] The following list of examples are some exemplary communication protocols based on quantum computing that may be defined by the networking code: QKD, Quantum Teleportation, QIP or Entanglement-based quantum cryptography. Other quantum-based communication protocols may alternatively be realized.
[0042] As clearly indicated by the above examples, the networking code may define practically any type of communication protocol based on various different technologies, none of which are to be interpreted as limiting to the scope of the present disclosure. In fact, this is an advantageous technical effect provided by the disclosed automatic networking code generation scheme, since its compatibility and adaptability is superior to known prior art technologies. The generated networking code, being configured to support multiple networking protocols, typically generates the protocol handling as individual modules that are only loaded if that protocol is being used.
[0043] The interaction between the packages of software code of the service mesh layer 10 and the networking code may provide access to underlying hardware resources of the runtime instances 24 in which the service mesh layer 10 is integrated into. The underlying hardware resources of the runtime instances 24 may depend on a plurality of different aspects including but not limited to capabilities, network loads, access rights, computing requirements, memory requirements, scheduling and balancing aspects. For example, a certain execution of the application program 30 may require access to a Bluetooth receiver or transmitter unit in order to effect data forwarding between two or more distributed computing services 22a-m. In this example, the software code of the service mesh layer 10 interact with the networking code such that access to said Bluetooth receiver or transmitter unit is provided. Other underlying hardware resources that may be accessed may be processing units, microcontrollers, loT devices, memories, FPGAs or sensors, to name a few examples. To this end, the access to underlying hardware resources may depend on the specific needs of the runtime instance 24, the distributed computing service 22, or the application program 30, allof which may be controlled by the interaction between the packages of software code of the service mesh layer 10 and the networking code.
[0044] The integration of the service mesh layer 10 into a runtime instance 24 may be triggered in response to a deployment of said runtime instance 24. A deployment of the runtime instance 24 involves the procedure of setting up the environment where at least a portion of the application program 30 is to be executed, an example of which can be seen in FIG. 2. Steps associated with the deployment may be installation of dependencies, set-up of storage resources and providing network protocols, etc. The deployment may be based on any known deployment strategies for setting up runtime instances 24, such as a static deployment, a dynamic deployment, a blue-green deployment, a canary deployment or a rollback deployment.
[0045] The service mesh layer 10 may be agnostic to prevailing network protocols of the distributed computing services 22. This means that the service mesh layer 10 is configurable and compatible with networking code of any and / or one or more different computing environments, operating systems, communication protocols, programming languages, networking mechanisms, computing platforms, services, underlying hardware resources, and so forth. To this end, at least one of the prevailing network protocols, e.g. of a particular distributed computing service 22, may be one of the protocols as described herein in conjunction with the networking code. For example, the networking protocol may be a nonTCP / IP -based network communication protocol. Such non-TCP / IP -based network communication protocols may be used in communication technologies such as Bluetooth, Zigbee, Z-Wave, Wi-Fi Direct, NFC, EnOcean or Thread. Alternatively or additionally, nonTCP / IP -based network communication protocols may be used in vehicles, for instance by a wired communication protocol as discussed above. Alternatively or additionally, non-TCP / IP- based network communication protocols may be a quantum-based communication protocol as discussed above. Other communication protocols not based on TCP / IP may be realized.
[0046] As discussed herein, the teachings of the present disclosure, and more specifically the integration of the service mesh layer 10 into runtime instances 24, may be realized in container-based orchestration tools. Alternatively or additionally, the distributed computing environment 20 may be a non-contained based computing environment, which makes the system 200 significantly more versatile compared to prior art solutions. To this end, the service mesh integration within the runtime instances may eliminate the need for containers. The present service mesh integration strategy is thus applicable to other type of computing technologies that are not based on containerization technologies, such as one or more ofvirtual machines, serverless computing, bare-metal provisioning, cluster computing, peer-to- peer computing, and so forth. No matter if container-based solutions or non-container based solutions are used, there is still no need to include a proxy sidecar or any similar type of module since this functionality is integrated directly into the runtime instances 24.
[0047] By instead placing the logic of the service mesh layer 10 directly in the runtime instance 24, the control plane component 12 may carry out the functionality of the conventional platform scheduler as well as further advantageous actions relating to scheduling. To this end, scheduling actions may be carried out on-demand on each node of the distributed computing environment 20. The control performed by the control plane component 12 may be a variety of different actions to control other runtime instances 24, e.g., for the purposes of load balancing, scalability, fault tolerance, and so forth. The control may be a scheduling of computational resources for a distributed computing service 22. In conventional platform schedulers, the logic thereof is typically placed in the operating system’s kernel or a separate component (such as the proxy sidecar), which determines timings of processed based on e.g., priorities, buffer times and amounts of utilized processing power in order to optimize the overall system performance.
[0048] The control plane component 12 may be configured to manage scheduling of workloads of the application program 30. This may be done using one or more scheduling settings. The scheduling settings may be configured to prioritize locations where code of the application program is to be placed. A location is to be interpreted as nodes (e.g. distributed computing services 22 or runtime instances 24) of the distributed computing environment 20 where the application program 30 is to be executed. The scheduling of workloads may be realized between runtime instances 24a-n of a particular distributed computing service 22a, between runtime instances 24al-n; 24bl-n; 24ml-n of different computing services 22a-m, or a combination thereof. To this end, managing scheduling of workloads may involve implementing a workload placement strategy, the workload placement strategy being configured to prioritize workloads with respect to execution locations. For example, the workload placement strategy may involve configuring the scheduling setting such that execution of the application program 30 is carried out on one particular distributed computing service 22 or by a particular runtime instance 24. In another example, the workload placement strategy may involve configuring the scheduling setting such that specific runtime instances 24 of a distributed computing service 22 are locally called upon the distributed computing service 22 being available. In yet another example, the workload placement strategy may involve configuring the scheduling setting such that specific runtime instances 24 of adistributed computing service 22 are remotely called upon no resources, or an insufficient amount of resources, being available on said computing service 22 where the application program 30 is running. These calls (remote or local) referred to above may be further configurable depending on what type of request is performed (for instance, for processing power, storage or connectivity).
[0049] Another advantage of integrating the service mesh layer 10 into the runtime instance 24 is that the control plane component 12 may have access to and thus comprise data capabilities of the runtime instance 24 (or “runtime capabilities”) in which the service mesh layer 10 is integrated into. With this knowledge, the control plane component 12 is configured to, based on data capabilities, control one or more distributed computing services 22 having at least one respective runtime instance 24. Such control actions by the control plane component 12 based on the data capabilities may include, for instance, an instantiation of a runtime instance 24, a duplication of a runtime instance 24, a shutdown of a runtime instance 24, or an access provisioning to underlying hardware resources / routines of the associated machine running the runtime instance 24. By way of example, certain devices, e.g. iOS, Desktop or Android devices, may require certain data capabilities to access a routine, e.g. a camera routine. The service mesh layer 10 being integrated into the runtime instance 24 may provide the necessary data capabilities such that the control plane component 12 may give access to such routine as required by the device.
[0050] The term “data capabilities” shall in this context be interpreted as technical functions that are related to data types, data sources, data technologies, data stores and data operations of the runtime instances 24. The service mesh layer 10 being integrated into the runtime instance 24 may have access to and thus comprise data capabilities including host capabilities, networking capabilities and user-provided capabilities.
[0051] The host capabilities are related to the distributed computing services 22 being configured to host the runtime instances 24. The host capabilities may be in the form of machine code that can run directly on a computing platform’s hardware. The host capabilities of the distributed computing services 22 depend on a variety of factors related to the platform on which it is hosted, i.e. the computing platforms associated with the distributed computing environment 20 (or other environments being in communication with) as discussed above. The host capabilities may include scalability data. The scalability data includes data pertaining to how a service manages an increased workload by scaling. The host capabilities may include availability data. The availability data includes data pertaining to assurance of availability of a service in the event of a failure, a switch, a request for availability, or othersuch scenarios. The host capabilities may include isolation data. The isolation data includes data pertaining to a service’s ability to isolate itself from other processes / tasks running on the same host. The host capabilities may include resource management data. The resource management data includes data pertaining to a service’s ability to manage and allocate necessary resources, such as CPU, GPU, memory and storage for the service to run as intended. The host capabilities may include monitoring and management data. The monitoring and management data includes data pertaining to a service’s ability to performance and health monitoring, as well as management thereof and potential configuration updates. Other related host capabilities known to the skilled person may alternatively be realized.
[0052] Networking capabilities are related to the networking code as discussed herein, and may include security-related data. The security-related data determines means for establishing secure communication between computing services, environments and external entities in a network, through mechanisms such as encryption schemes, firewalls, access controls schemes, etc. The networking capabilities may include load balancing data. The loadbalancing data manages a service’s ability to distribute traffic or workloads across a plurality of runtime instances to obtain an improved performance and availability. The networking capabilities may include service discovery data which manages a service’s ability to discover and communication with other services within or external to the environment without requiring hardcoding of e.g. hostnames or IP addresses. The networking capabilities may include gateway data which pertains to proxies, triggers, authentication, rate limits, routing of information, etc. The networking capabilities may include troubleshooting and monitoring data which pertains to network traffic, latency, errors, vulnerabilities, etc.
[0053] User-provided capabilities are related to the application data of the application program 30, including the software application code, and comprise data pertaining to specific functionalities or features. Such functionalities or features may correspond to those being implemented by developers of the application program 30. The user-provided capabilities may include storage data which is related to the program’s 30 ability to store, retrieve and manipulate data using e.g. databases, data models or other data access mechanisms. The user- provided capabilities may include interface data which is related to the program’s 30 ability to provide interfaces for a user to interact with the program 30 through e.g. GUIs, CLIs, webbased interfaces, and so forth. The user-provided capabilities may include logic data pertaining to the program’s 30 ability to manage business logic and rules including calculations, validations and workflow management. The user-provided capabilities may alsoinclude other data related to the application program 30, including but not limited to communication data, security data, reporting data and customization data.
[0054] The workload placement strategy may be configured to prioritize workloads based on data capabilities. The data capability requirement may refer to one or more of the networking capabilities, user-provided capabilities or host capabilities. In the case of host capabilities, the workload prioritization may require certain host capabilities of a particular distributed computing service 22 for the application program 30 to be able to run, such as how the distributed computing service 22 manages scaling once the application program 30 is executed therein. In the case of user-provided capabilities, the workload prioritization may require certain user-provided capabilities for the application program 30 to be able to run, such as favoring a low latency execution. In the case of networking capabilities, the workload prioritization may require certain networking capabilities of the networking code for the application program 30 to be able to run, such as a certain security level as determined by the security-related data. Other exemplary placement strategies utilizing workload prioritizations based on data capabilities may be realized.
[0055] The workload placement strategy may be configured to prioritize workloads based on regional data. Regional data may indicate a region, e.g., country, area or otherwise geographically delimited locations, where distributed computing services 22 and / or runtimes 24 are hosted from. Regional data may include information pertaining to data sovereignty regulations / laws for said regions. To this end, the regional data may specify whether executions are allowed to take place within or outside the regions. Having a workload placement strategy utilizing prioritization based on regional data may increase the reliability of the execution. This is because even if certain regions were to e.g., lose or be associated with poor connectivity for some reason, or other similar reasons that negatively affects the execution based on regional data, the execution could continue to run in other regions or be rescheduled on other regions. The prioritization based on regional data may be further adapted for taking into account a level of redundancy needed for a particular application program 30, for instance as determined by any of the data capabilities. By way of example, certain (more sensitive) executions of an application program 30 may be associated with a higher level of redundancy, and the workload placement strategy based on regional workload prioritization can be configured accordingly, e.g., by adding one or more additional regions. The same but opposite can also be realized, i.e., removing one or more additional regions where the application program 30 requires less redundancy.
[0056] The workload placement strategy may be configured for fan-out messaging. To this end, networking code associated with the application program 30 may be distributed to any number of targets, where the targets corresponds to nodes such as runtime instances 24 or distributed computing services 22. The targets can then be configured and changed independently of the code of the application program 30 by just changing the scheduling settings. Optionally, the code of the application program 30 may dynamically configure the targets. The fan-out messaging approach may be layered, which enables fanning out messages to a large number of targets. The layered fan-out messaging may be configured as a tree graph, wherein the tree graph comprises a parent node and a plurality of child nodes in a hierarchical tree structure, wherein the parent node is the source of a message and the child nodes are the destinations. By way of example, a message may be sent from the parent node to multiple child nodes being interconnected by the tree graph, wherein the message is transmitted along the edges of the tree with each child node receiving a copy thereof. Such layered tree graph-based fan-out messaging may be efficient for hierarchically allowing a message to be broadcasted to multiple destinations simultaneously, e.g., to certain regions.
[0057] The workload placement strategy may be configured for batch scheduling. To this end, the one or more scheduling settings may be a batch scheduling setting. The batch scheduling setting enables a parent component (e.g., a first distributed computing service 22 or runtime instance 24) to call a function parameter of the application program 30 a large number of times, up to as much as ten thousands of function parameter calls, and subsequently await said function parameter calls to finish their respective execution before returning. The workload placement strategy configured for batch scheduling involves batching a plurality of function parameter calls together into chunks that can be placed at a particular runtime 24. The batch scheduling setting may also include dynamic chunk sizing such that the size of the chunk (i.e., the number of batched function parameter calls) is adapted to be dynamically adjusted based on underlying hardware resource availability. This could thus result in differing sizes of batches depending on said underlying hardware resource availability, which is beneficial for optimizing the execution to the specific available underlying hardware resources.
[0058] The workload placement strategy may be configured for scheduling based on a workload characteristic. A workload characteristic may be a long running workload or a short running workload. The terms “short” and “long” shall be interpreted in relation to each other, i.e., the short running workloads run for a shorter period of time compared to a corresponding long running workload. To this end, long running workloads refer to tasks of applicationprograms 30 that run continuously or for an extended period of time, such as database servers, web servers, or cloud-based applications. The long running workloads typically require a significant amount of resources and may have a constant stream of incoming requests or data to process. Conversely, short running workloads refer to application programs 30 that run quickly and complete their tasks in a shorter amount of time compared to long running workloads. Some exemplary short running workloads may include text editors, image viewers or other utility programs. Alternatively, it is possible to spin up multiple instances of short running workloads to carry out functionalities corresponding to a long running workload. Short or long running workloads may be started manually or automatically in response to a trigger. The trigger may also be manually activated.
[0059] The workload placement strategy being configured for scheduling based on a workload characteristic may involve marking runtime instances 24 as draining. If short running workloads are scheduled on a runtime instance 24 being marked as draining, said marked runtime instance 24 may stop new workloads from being scheduled thereon. As soon as the latest running workload has completed its execution on the marked runtime instance 24, the marked runtime instance 24 will shut itself down. If long running workloads are scheduled on runtime instance 24 being marked as draining, a state of said marked runtime instance 24 may be copied and streamed to another available runtime instance 24. Once the state is synchronized with said another available runtime instance 24, said marked runtime instance 24 will be shut down and said another available runtime instance 24 will resume and thus take over the execution. The marking procedure may be achieved by the control plane component 12 of said marked runtime instance 24 being configured to send a workload migration request to said another available runtime instance 24 having enough available resources, which, once having accepted the request, migrates the running workload. In addition, the control plane component 12 of said marked runtime instance 24 may be configured to notify connected clients of the workload migration, and further information sent or received from or to the connected clients will occur with respect to said another available runtime instance 24.
[0060] Short running workloads, by design, provide high availability of runtime instances 24 due to their relatively short execution times. However, this is not the case for long running workloads due to said long running workloads often requiring migration between a plurality of runtime instances 24 before it has finished its execution. Migration between runtime instances 24 may be necessary due to e.g., timeouts, crashes, interrupts, or other such situations. During the plurality of migration phases, the long running workloads may, at least occasionally, have to wait for additional runtime instances 24 to be respectively instantiatedsuch that the workload can be migrated. This can be somewhat alleviated by providing a startup time of the runtime instances 24 of as low as a few microseconds up to milliseconds in the order of magnitude, which provides a minimum of waiting time for the long running workloads. Certain long running workloads maintain critical information that cannot be lost, because a recreation of said critical information would provide potential attack vectors and / or take a significant amount of time. To this end, the control plane component 12 may be configured to provide long running workload resilience with automatic failover of a runtime instance 24, which would not impact the workload or user experience. The long running workload resilience with automatic failover may be provided by running two or more instances of the long running workload in a fallback mode, said fallback mode replicating all the incoming network requests to both of the long running workload instances, and the requests are not considered processed until said two or more instances of the long running workload have acknowledged the respective requests. The acknowledgement may be an “OK” response or an “ERROR” response. In the case where different instances sent different responses, this is an indication that the application program 30 may be faulty, or non- deterministic. The long running workload resilience with automatic failover guarantees fallback without data loss with a minimum impact to latency. The long running workload resilience with automatic failover may be particularly useful for certain application programs 30, such as databases which that natively do not support high availability. By way of example, a database such as PostgreSQL may have a significant downtime or several seconds to a couple of minutes before switching over to a fallback, even if it is being ran as a managed service, e.g., Amazon RDS. Using the long running workload resilience with automatic failover approach as described above, there would be practically zero downtime between switchovers and consequently no impact to the end user.
[0061] In addition to or as an alternative to the above, short running workloads may be configured for workload resilience with automatic failover. This can be done by spinning up multiple instances of short running workloads.
[0062] An alternative or addition to the long running workload resilience with automatic failover approach may involve the control plane component 12 being configured to copy the long running workloads in a memory state and write it back to a deployment storage unit. The copying may be performed once a request has been processed or at a predetermined interval. To this end, where workload migration is needed, the control plane component 12 of an available runtime instance 24 may then, instead of requesting the long running workload fromthe first runtime instance 24 (which no longer exists), request the long running workload from the deployment storage unit.
[0063] The control plane component 12 may carry out migration of long running workloads using domain name assignations to the long running workloads using a Domain Name Server (DNS). Domain names may be assigned to single instances of long running workloads or be shared amongst multiple instances of the same type of long running workload.
[0064] In some examples, or more middleware components may be defined. For example, the middleware components may be defined by the control plane component 12. The middleware components is configured between application programs 30 and is configured to enable communication and data exchange therebetween. Generally, a common layer of functionality and abstraction is provided to simplify the development, deployment and maintenance of more complex software systems. The middleware components may be executed in parallel. The middleware components may be executed locally and / or distributed. The middleware components may be read-only, mutable or partially mutable (i.e., only some fields thereof being mutable). In order to avoid data races or overwriting data of, only one of the middleware components is preferably configured to be able to mutate one field at a given time. This approach is similar to Rust’s ownership model combined with GraphQL, and optionally a pipeline with the ability to have synchronization points, but with the added advantage of multiple servers instead of just one machine. In this example, the middleware components may define one request but have a large number (theoretically an unlimited amount) of servers processing said request simultaneously. Some advantages of defining middleware components include being able to batch data or handle one item at a time in an array or data map per middleware component instance, handle one item in an object per middleware component instance, run the same task in parallel for redundancy reasons, and run automatic retries of a failed task (which may be useful when failures occur to only have a single performance impact at the cost of additional latency added with each failure).
[0065] The middleware components may be separated from the application programs 30. This involves being able to easily switch out one or more middleware components without requiring code changes or recompilation of the application program 30. Hence, parallel execution of one or more middleware components can be carried out to obtain a faster execution of a request.
[0066] The middleware components may represent any computerized module of the distributed computing environment 20. For example, adding logging or tracing to the requestsmay be carried out by adding a middleware with read access to the relevant data being placed in a suitable location of a stack of the middleware components. In this example, turning on or off the logging in production may be as simple as changing the configuration for which middleware component among the middleware components handles the request. This can be done with no extra code or redeployment of the application program 30. In another example, another authentication provider may be added in the stack of the middleware components. Other exemplary middleware components may be configured for parsing, trust and validation. The parse middleware component may be configured to manage parsing of data into a working format. The trust middleware component may be configured to manage authentication of a client which submitted the request. The validation middleware component may be configured to manage data validation in a format being ready for additional steps in the process. By dividing this functionality into three separate middleware components, subsequent processing steps may only require carrying out data processing. In some examples, the authentication may be processed in parallel. The parallel authentication may be carried out with respect to an OAuth provider at the same time as user roles being (i.e., whether the user is allowed to perform an action). If these actions succeed, the application program 30 may run the request knowing that the user is trusted. At any time during this process, a policy system or authentication provider may be switched out without any changes to the application program 30 being required.
[0067] If the application program 30 is built to process one object at a time, batch processing could be added by introducing a pipeline component in the stack of middleware components. The pipeline may be configured to obtain a request comprising a list of objects and start one application for each object in the list for parallel batch processing of data. The results of the batch processing may be returned as a response to the request. The pipeline component may be executed in a local and / or distributed way. The pipeline component may comprise synchronization points capable of storing the state of the pipeline component to resume the execution at any later stage without data loss in the event of, for instance, an outage. The pipeline component may be multilayered. The pipeline component may be configured to start so called “Fire and forget” middleware components for some of the data. The fire and forget middleware components are components that process a message or request without waiting for a response. In other words, once the middleware component receives the message, it sends the message to the next component in the pipeline and does not wait for a response from that component before proceeding. This can, for example, be a log message sent at different parts of the process, tracing or metrics.
[0068] The middleware components may be configured for local sequential execution. This may be configured as default in case no other execution scheme has been defined for the application program 30. As the name suggests, sequential execution is the same as sequentially executing the middleware components one after the other and waiting for the first one to execute before calling a second one. Alternatively, the middleware components may be configured for distributed sequential execution. Distributed sequential execution may involve executing a middleware component such that one part of the middleware stack runs on a separate runtime instance. In addition, if multiple runtime instances are run for resilience, the execution may occur across multiple instances of a middleware component across multiple server nodes. Depending on scheduler settings for the middleware component, it may be combined with local execution, and optionally remove one or more steps of networking, serialization, deserialization, trust, etc. The local node that executes the middleware component typically requires available resources, and the optional removal may be seen as an optimization step.
[0069] With reference to FIG. 2, an exemplary resource group deployment is shown. Resource group deployment refers to the process of creating and configuring a resource group 440. The resource group 440 is a collection of providers, regions and instance types that will be used to provide the underlying hardware for running the application program 30. The deployment unit 410 is configured for providing what and where the hardware will be deployed. The deployment is based on information provided by at least one policy setting 412. The deployment unit 410 may include information pertaining to e.g., a minimum number of underlying hardware instances that the application program 30 is to run at, or a maximum value which is used to set a cap of the automatic scaling in order to reduce potentially unwanted costs. The deployment unit 410 may also allow specification of a scaling strategy which can be used to influence the behavior of the scaling. Scaling strategies may include a reliable strategy. The reliable strategy is configured to scale the underlying hardware to be spread out as much as possible on the underlying providers, regions and instances to minimize the risk of an outage of the application programs 30 running on top of the hardware. Scaling strategies may alternatively include a cost reduction strategy. The cost reduction strategy involves performing scaling on the cheapest underlying hardware as a priority. Only in the case where the deployment unit 410 is unable to scale more on the cheapest hardware of a particular provider or region, the deployment unit 410 will perform scaling using the second cheapest option available, and so forth. The cost reduction strategy may be especially usefulin testing scenarios where the application program 30 is generally not sensitive to an outage as no end users would be affected.
[0070] Generally, the deployment unit 410 may be configured to carry out deployment of the application program 30 into a resource group 440 as shown in FIG. 2. The deployment involves dividing the application program 30 into software packages 420; 430, each software package 420, 430 having respective components 422, 426; 432, 436, each component 422, 426, 432, 436 having respective software modules 423, 424; 427, 428; 433, 434; 437, 438 of software application code. The respective software modules 423, 424; 427, 428; 433, 434; 437, 438 are assigned to the resource group 440 where a plurality of runtime instances 24al, 24a2, 24bn, 24mn are hosted. As defined throughout this disclosure, the service mesh layer is integrated into runtime instances where the application program 30 is to be executed (and hence into the networking code and the application program 30 itself, as discussed herein). The service mesh layer may thus directly interact with the application program 30 within the runtime instances without having to pass any network layer, such as a proxy sidecar, as opposed to prior art solutions. Clearly, in other examples, the deployment may involve any number of software packages, components or modules, as well as any number of resource groups having any number of runtime instances.
[0071] A software module corresponds to software application code in intermediate representation format being compiled and bundled together in the form of modules. The software modules may thus be generated for some particular intermediate representation formats, for instance Web Assembly bytecode. Other examples may be dynamic libraries and / or ELF code. The software modules may be stored in a module storage resource until a loading condition has been triggered. The module storage resource may be a database, cache, repository, or other known storage structure capable of storing software modules which can be loaded and executed by at least the distributed computing environment where the application program 30 is to be executed. The software module storage may be disposed in several different formats, such as locally, in memory, remotely, and have one or more (different) layers for fallback in case of system failure or to reduce latency. The loading condition may be a start-up of the application program 30, an initiation of a particular program function of the application program 30, or any such similar condition that triggers loading of the associated software module. The module storage resource may be configured to store a respective state of the one or more software modules. This may serve as a failsafe measurement for handling interrupts or otherwise crashing distributed computing services. The states may be stored in memory and copied over to a second distributed computingservice whenever a failure has occurred in a first distributed computing service for continued execution of said software module. For this particular purpose, the state may be replicated such that said second distributed computing service configured for continued execution can carry on where the first distributed computing service crashed, got interrupted or any other reason for failure. In addition, the memory state may be stored such that the memory of the crashed software modules may be loaded and reused in a local runtime instance with enabled debugging functionality. In some cases, this is not necessarily a failsafe measurement since a similar procedure may be carried out where some distributed computing services need to be switched out for some reasons, such as testing reasons. For example, Web Assembly which is a stack-based virtual machine uses a stack data structure to store and manage memory during program execution. It is thus possible to actively move the application program 30 when it is already running. This corresponds to migration of workloads, as discussed herein. To this end, the software modules may embodied as short running modules or long running modules, i.e., corresponding to short or long running workloads as discussed herein. The software modules may be assigned to accessible hardware components. The assignation may include a specific and / or growable amount of memory that can be dynamically resized up to a set amount of memory. The assignation may alternatively or additionally include a specific and / or growable amount of CPU / GPU resources that can be dynamically resized up to a set amount of GPU / CPU resources, or storage, Al processing or other types of hardware. By doing so, automatic vertical and horizontal scaling of said software modules is achieved such that they may communicate with distributed computing services (e.g. services of different computing environments) different from the distributed computing services orchestrated by the distributed computing environment.
[0072] FIGS. 3-6 show different exemplary networking models 300; 400; 500; 600 that can be implemented by a distributed computing environment 20. In any of these networking models 300; 400; 500; 600, the distributed computing environment 20 may be a server-side computing environment, a client-side computing environment, or a combination thereof. The distributed computing environment 20 may be configured to communicate with one or more remote services, the one or more remote services being external devices 72, data centers 74 or additional distributed computing environments 76 having respective distributed computing services 78. For these four examples, the skilled person will appreciate that networking code may be generated for any of the remote services, networking code may define protocols of any of the remote services, networking information may be associated with any of the remote services, and code may be sent to / from among any of the remote services. This may beperformed such that the distributed computing environment 20 can communicate with the remote services and distribute the execution of the application program also to any of the remote services.
[0073] Although not shown in FIGS. 3-6, the control plane component 12 may comprise network egress and ingress data policies. These policies determine rules and configurations of how network traffic is to be transmitted within, into, and out from the distributed computing environment 20 and the remote services. Egress traffic is related to data leaving a network, which may occur e.g. upon a user sending an email or downloading a file. Ingress traffic is related to data entering the network, which may occur e.g. upon the user receiving an email or accessing a website. The network egress and ingress data policies are configured to control and manage incoming and outgoing traffic depending on various criteria, such as the type of traffic, the source and destination of the traffic, the security level of the traffic, and so forth. The control plane component 12 may configure the egress and ingress data policies at different network layers, from devices to subnets or domains. By way of example, a network egress data policy may configure a particular type of sensitive data only to be sent out from the network if it is encrypted, and a network ingress data policy may configure that incoming traffic from a certain IP address range is blocked for prevention of unauthorized access.
[0074] In any of the exemplary networking models 300; 400; 500; 600, the distributed computing environment 20 may be configured to orchestrate the plurality of distributed computing services 22 as a capability-based security model. This is related to the data capabilities as discussed with reference to the linking procedure. An application program running in the capability-based security model will only do what it is explicitly allowed to do, as set by the capabilities. Different portions of the code in the intermediate representation format will thus be scheduled to run at a service having the desired capabilities. In situations where there are no services currently available or all available services are already being occupied by other running portions of the code, the code will be scheduled at a service without the desired capabilities. This may be useful in case the application program will perform or function better on services having the desired capabilities, but it is still relevant for running the program if there are no available services, or if all services are occupied. Such scenario may be realized where the computing environment has a small set of services that are enough to handle normal load conditions, although sometimes heavy load spikes causes other services to temporarily assist in the execution of the program. An exemplary scenario may be an application program directed at graphics rendering where the related services have the desired capabilities of a GPU. However, if all these are occupied or non-available, thegraphics rendering may be executed on another service. The skilled person will appreciate that other such scenarios may be realized where services without the desired capabilities may step in and still execute functionality of the application program, for instance upon features being omitted or the code running at a reduced performance (in the case of replacing GPU rendering with CPU). The capability-based security model is thus a very flexible model.
[0075] FIG. 3 shows an exemplary meshed networking model 300. Each entity shown in the model 300 acts as a transceiver, which allows for multiple paths for data to flow between said entities. This provides a greater redundancy and resiliency, as data can be transmitted along multiple paths to the various entities in the event of failure or congestion on one particular path. The meshed networking model 300 is commonly used for wireless networks, particularly in ad-hoc networks where a central infrastructure is unavailable. The meshed networking model 300 is implemented by the distributed computing environment 20 being configured to orchestrate the distributed computing services 22a-m. The distributed computing environment 20 is configured to be able to communicate, in runtime operation or prior to runtime operation (e.g. build-compile time) with an additional distributed computing environment 76 being configured to orchestrate the additional distributed computing services 78-1, 78-2, 78-3, 78-4, data centers 74-1, 74-2, 74-3, 74-4, a remote device 72 (such as a user phone), and a module storage resource 60. The skilled person will appreciate that the exemplary meshed networking model 300 may have any additional or fewer such entities, and it may be dynamically scalable (both increased and decreased) in response to incoming traffic or workload. The communication within entities may be established based on any protocol defined by the networking code and discussed in conjunction therewith.
[0076] In the meshed networking model 300, connections and thus communication between runtime instances are open to all other runtimes within the same resource group through one or more netlink interfaces. To this end, the control plane component of a runtime instance may maintain information relating to each connectable runtime instance to monitor current available resources, latency and capabilities. This information may then be used in conjunction with the workload placement strategy for scheduling new workloads. Various different networking approaches can be used for communication via said netlink interfaces, such as a DAG (direct acyclic graph) for reduced resource consumption when there is a high number of available runtime instances, or any other reasonable approach for network discovery, such as DNS. This procedure may allow the data plane component of a runtime instance to carry out forwarding of information based on data capabilities rather than DNS / IP routing, as is typically the case for conventional solutions.
[0077] FIG. 4 shows an exemplary internal networking model 400. The internal networking model 400 comprises similar aspects as the meshed networking model, with the difference that the distributed computing services 22a-m; 78-1, 78-2, 78-3, 78-4 are respectively configured to, in runtime operation, only be able to communicate with other services orchestrated by the same distributed computing environment 20; 76. The internal networking model may take the form of a star network, a bus network, a ring network or a meshed network, depending on different specifications as determined by the networking code.
[0078] FIG. 5 shows an exemplary traditional networking model 500. The traditional networking model 500 refers to models that resemble earlier models of computer networks. The traditional networking model 500 involves that the distributed computing environments 20; 76 are adapted to communicate with remote devices 72, e.g., via an application, through a respective API Gateway API-GW1, API-GW2. The API gateways may be based on technologies such as Amazon API Gateway, Kong, Tyk, Apigee, Microsoft Azure API or NGINX Plus. Alternatively, the API Gateway of the traditional networking mode 500 may in some examples be replaced with reverse proxies, routers, other gateways, service meshes, or other such means known to the skilled person.
[0079] FIG. 6 shows an exemplary hybrid networking model 600. The hybrid networking model 600 combines the aspects of a meshed networking model and a traditional networking model.
[0080] With reference to FIG. 7 some exemplary platform components 700 of the computer-implemented system 200 as discussed herein, for instance with reference to FIG. 1, are shown. The platform components 200 comprises a plurality of hardware components, in this particular example being based on x86, x64, arm and RISC V architectures Other examples may be realized, for instance quantum computing hardware components. These hardware components are in operative communication with an external device 72 comprising software functionality for orchestrating at least one distributed computing service 22 being operable to provide respective runtime instances. A plurality of application programs 30-1, 30-2, 30-n may be configured to be executed by said at least one distributed computing service 22 such that the hardware device 72 and respective hardware components carry out the intended functionality of the respective application programs 30-1, 30-2, 30-n.
[0081] As shown in FIG. 8, the system 200 may include a number of units known to the skilled person for implementing the functionalities as described in the present disclosure. The system 200 may comprise one or more computing units capable of including firmware, hardware, and / or executing software instructions to implement the functionality describedherein. The system 200 may comprise one or more processor devices (may also be referred to as a control unit) 230, one or more memories 235 and one or more buses 240. The system 200 may include at least one computing device having the processor device 230. A system bus 240 may provide an interface for system components including, but not limited to, the memories 235 and the processor devices 230. The processor device 230 may include any number of hardware components for conducting data or signal processing or for executing computer code stored in the memories. The processor device 230 may, for example, include a general-purpose processor, an application specific processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), a circuit containing processing components, a group of distributed processing components, a group of distributed computers configured for processing, or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The processor device 230 may further include computer executable code that controls operation of the programmable device.
[0082] The system bus 240 may be any of several types of bus structures that may further interconnect to a memory bus (with or without a memory controller), a peripheral bus, and / or a local bus using any of a variety of bus architectures. The memories 235 may be one or more devices for storing data and / or computer code for completing or facilitating methods described herein. The memories 235 may include database components, object code components, script components, or other types of information structure for supporting the various activities herein. Any distributed or local memory device may be utilized with the systems and methods of this description. The memories 235 may be communicably connected to the processor device 230 (e.g., via a circuit or any other wired, wireless, or network connection) and may include computer code for executing one or more processes described herein. The memories may include non-volatile memories (e.g., read-only memory (ROM), erasable programmable read-only memories (EPROM), electrically erasable programmable read-only memories (EEPROM), etc.), and volatile memories (e.g., random-access memory (RAM)), or any other medium which can be used to carry or store desired program code in the form of machine-executable instructions or data structures and which can be accessed by a computer or other machine with a processor device. A basic input / output system (BIOS) may be stored in the non-volatile memories and can include the basic routines that help to transfer information between elements within the computer system.
[0083] A storage 245 may be operably connected to the system 200 via, for example, I / O interfaces (e.g., card, device) 250 and I / O ports 255. The storage 245 can include, but is not limited to, devices like a magnetic disk drive, a solid state drive, an optical drive, a flash memory card, a memory stick, etc. The storage 245 may also include a cloud-based server implemented using any commonly known cloud-computing platform, as described above. The storage 245 or memory 235 can store an operating system that controls and allocates resources of the system 200.
[0084] The system 200 may interact with network devices 260 via the I / O interfaces 250, or the I / O ports 255. Through the network devices 260, the system 200 may interact with a network. Through the network, the system 200 may be logically connected to remote computers. Through the network, a server-side platform may communicate with a client-side platform. The networks with which the system 200 may interact include, but are not limited to, a local area network (LAN), a wide area network (WAN), and other networks.
[0085] FIG. 9 shows an exemplary method 100 for managing service mesh integration of a distributed computing environment 20. The method 100 comprises a step of configuring 110 a service mesh layer 10 of one or more distributed computing services 22 being orchestrated by the distributed computing environment 20, wherein the service mesh layer 10 comprises a control plane component 12 and a data plane component 14. The control plane component 12 is configured to control forwarding of application data and networking data between said one or more distributed computing services 22, and the data plane component 14 is configured to carry out said forwarding of said application data and networking data between said one or more distributed computing services 22. The method 100 further comprises a step of integrating 120 the control plane component and the data plane component of the service mesh layer 10 into runtime instances of said one or more distributed computing services 22.
[0086] With reference to FIG. 10, a schematic illustration of a (non-transitory) computer- readable (storage) medium 300 is shown according to one exemplary embodiment. The computer-readable medium 300 may be associated with or connected to the system 200 as described herein, and is capable of storing a computer program product 310. The computer- readable medium 300 in the disclosed embodiment is a memory stick, such as a Universal Serial Bus (USB) stick. The USB stick 300 comprises a housing 330 having an interface, such as a connector 340, and a memory chip 320. In the disclosed embodiment, the memory chip 320 is a flash memory, i.e., a non-volatile data storage that can be electrically erased and reprogrammed. The memory chip 320 stores the computer program product 310 which is programmed with computer program code (instructions) that when loaded into a processordevice, will perform a method, for instance the method 100 explained with reference to FIG. 9. The USB stick 300 is arranged to be connected to and read by a reading device for loading the instructions into the processor device. It should be noted that a computer-readable medium can also be other mediums such as compact discs, digital video discs, hard drives or other memory technologies commonly used. The computer program code (instructions) can also be downloaded from the computer-readable medium via a wireless interface to be loaded into the processing device.
[0087] The operational steps described in any of the exemplary aspects herein are described to provide examples and discussion. The steps may be performed by hardware components, may be embodied in machine-executable instructions to cause a processor to perform the steps, or may be performed by a combination of hardware and software. Although a specific order of method steps may be shown or described, the order of the steps may differ. In addition, two or more steps may be performed concurrently or with partial concurrence.
[0088] The terminology used herein is for the purpose of describing particular aspects only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items. It will be further understood that the terms "comprises," "comprising," "includes," and / or "including" when used herein specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.
[0089] It will be understood that, although the terms first, second, etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element without departing from the scope of the present disclosure.
[0090] Relative terms such as "below" or "above" or "upper" or "lower" or "horizontal" or "vertical" may be used herein to describe a relationship of one element to another element as illustrated in the Figures. It will be understood that these terms and those discussed above are intended to encompass different orientations of the device in addition to the orientation depicted in the Figures. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or intervening elements may be present. In contrast, when an element isreferred to as being "directly connected" or "directly coupled" to another element, there are no intervening elements present.
[0091] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It will be further understood that terms used herein should be interpreted as having a meaning consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
[0092] It is to be understood that the present disclosure is not limited to the aspects described above and illustrated in the drawings; rather, the skilled person will recognize that many changes and modifications may be made within the scope of the present disclosure and appended claims. In the drawings and specification, there have been disclosed aspects for purposes of illustration only and not for purposes of limitation, the scope of the inventive concepts being set forth in the following claims.
Claims
CLAIMS1. A computer-implemented method (100) for managing service mesh integration of a distributed computing environment (20), the method (100) comprising: configuring (110) a service mesh layer (10) of one or more distributed computing services (22) being orchestrated by the distributed computing environment (20), wherein the service mesh layer (10) comprises a control plane component (12) and a data plane component (14), the control plane component (12) being configured to control forwarding of application data and networking data between said one or more distributed computing services (22), the data plane component (14) being configured to carry out said forwarding of said application data and networking data between said one or more distributed computing services (22); and integrating (120) the control plane component (12) and the data plane component (14) of the service mesh layer (10) into runtime instances (24) of said one or more distributed computing services (22).
2. The computer-implemented method (100) of claim 1, wherein said integrating (120) comprises placing packages of software code of the service mesh layer (10) within said runtime instances (24) of said one or more distributed computing services (22).
3. The computer-implemented method (100) of claim 2, wherein said packages of software code are configured to interact with generated networking code comprising the networking data defining respective communication protocols at least of said one or more distributed computing services (22), said interaction providing access to underlying hardware resources of the runtime instances (24) in which the service mesh layer (10) is integrated into.
4. The computer-implemented method (100) of any preceding claim, wherein said integrating (120) is performed in response to a deployment of said runtime instances (24) of said one or more distributed computing services (22).
5. The computer-implemented method (100) of any preceding claim, wherein said runtime instances (24) of the one or more distributed computing services (22) are configured to provide a distributed execution of an application program (30) comprising the application data to cause control of one or more functionalities of a computer-implemented system (200).
6. The computer-implemented method (100) of claim 5, wherein the control plane component (12) is configured to implement a workload placement strategy, the workload placement strategy being configured to manage scheduling of workloads of the application program (30) using one or more scheduling settings, the one or more scheduling settings being indicative of prioritization of locations where code of the application program (30) is to be placed.
7. The computer-implemented method (100) of claim 6, wherein the workload placement strategy is configured to prioritize workloads based on data capabilities.
8. The computer-implemented method (100) of claim 7, wherein said workload prioritization based on the data capabilities involves one or more of a scheduling of computational resources, an instantiation of a runtime instance (24), a duplication of a runtime instance (24), a shutdown of a runtime instance (24), an interruption of a runtime instance (24), and a provisioning of access for a runtime instance (24) to one or more hardware resources.
9. The computer-implemented method (100) of any one of claims 6-8, wherein the workload placement strategy is configured to prioritize workloads based on regional data.
10. The computer-implemented method (100) of any one of claims 6-9, wherein the workload placement strategy is configured for fan-out messaging.
11. The computer-implemented method (100) of any one of claims 6-10, wherein the workload placement strategy is configured for batch scheduling.
12. The computer-implemented method (100) of any one of claims 6-11, wherein the workload placement strategy is configured for scheduling based on a workload characteristic, the workload characteristic being a short running workload or a long running workload.
13. The computer-implemented method (100) of claim 12, wherein said short running workload or long running workload is configured for workload resilience with automatic failover.
14. The computer-implemented method (100) of any preceding claim, wherein the distributed computing environment (20) is a non-container based computing environment.
15. The computer-implemented method (100) of any preceding claim, wherein the service mesh layer (10) is agnostic to prevailing network protocols of said one or more distributed computing services (22) of the distributed computing environment (20).
16. The computer-implemented method (100) of claim 15, wherein at least one of said prevailing network protocols is a non-TCP / IP -based network communication protocol.
17. The computer-implemented method (100) of any preceding claim, wherein the control plane component (12) comprises network egress and ingress data policies.
18. The computer-implemented method (100) of any preceding claim, wherein the service mesh layer (10) integrated into said runtime instances (24) is configured to communicate said application data and networking data to other runtime instances (24) through one or more netlink interfaces.
19. A computer-implemented system (200) for managing service mesh integration of a distributed computing environment (20), the system (200) comprising one or more processing units being configured to:provide a service mesh layer (10) of one or more distributed computing services (22) being orchestrated by the distributed computing environment (20), wherein the service mesh layer (10) comprises a control plane component (12) and a data plane component (14), the control plane component (12) being configured to control forwarding of application data and networking data between said one or more distributed computing services (22), the data plane component (14) being configured to carry out said forwarding of said application data and networking data between said one or more distributed computing services (22); and integrate the control plane component (12) and the data plane component (14) of the service mesh layer (10) into runtime instances (24) of said one or more distributed computing services (22).
20. A non-transitory computer-readable storage medium comprising instructions which, when executed by one or more processing units, cause the processing units to perform the method (100) of any of claims 1-18.
21. A computer program product comprising instructions which, when executed by one or more processing units, cause the processing units to perform the method (100) of any of claims 1-18.
Citation Information
Cited By
Data processing method and device of service grid platform and electronic equipment
CN121098872A
