Using dependency graph scaling for microservices

Abstract

A method is disclosed for reduced latency between software services running in a service mesh, where the software services are instantiated on demand to fulfill requests. The method includes providing a plurality of services that fulfill requests, where communication between the services is based on supporting components. The method also includes creating a directed dependency graph of the plurality of services by tracing request flows between the plurality of services, such that nodes of the directed dependency graph represent services and edges of the directed dependency graph represent communication paths used between selected ones of the services; determining, based on the directed dependency graph, a dependent service for an incoming request to a selected one of the plurality of services; and launching the selected one of the plurality of services together with an instance of the dependent service.

Description

Background Technology

[0001] Invention Field Various aspects of the present invention generally relate to a method for optimizing operations in a service mesh, and more specifically, to a computer implementation method for reducing latency between software services running in a service mesh, wherein the software services are instantiated when a request is fulfilled. Various aspects of the present invention also relate to a distributed multi-service system for reducing latency associations between software services running in a service mesh, and an associated computer program product. Related technologies

[0002] Software development has undergone significant changes in recent years. Starting with monolithic legacy applications, its evolution has been slow, progressing through client / server applications, service-oriented architecture (SOA), and then through virtualization and virtual machines to containers, service meshes, and finally low-code / no-code paradigms. Regardless of its form or underlying framework, the concept of services—connecting core functionalities—continues to play a vital role in software applications.

[0003] In a microservices architecture, an application typically comprises multiple components. When the system performs a specific task (such as processing an incoming request), these components interact and work together to fulfill that request. Typically, these interactions occur over a network connection. Furthermore, the components providing the service are placed in a topology that spans multiple physical machines, time zones, geographical regions, etc. Both of these aspects are the root causes of latency overhead, which affects the overall processing time for a particular request. Therefore, the overall performance of the system is impacted, and resource efficiency needs improvement.

[0004] In most modern cloud computing platforms, software components are typically scaled up dynamically and shrunk to zero when not in use. Both dimensions of scaling require the system to scale quickly to process incoming requests with low latency. If the system cannot manage the scaling of application components fast enough, related service requests will have to queue, resulting in additional latency. This can be amplified when multiple microservices need to interact to process requests, and each microservice needs to scale up first. Continuing this initial example, each scaling up of a service component introduces latency overhead, negatively impacting the overall request processing time and thus degrading the performance of the entire computing system.

[0005] The opposite occurs when scaling down a system (i.e., reducing the number of active software components). Scaling down latency can also cause unnecessary delays for the entire system. Therefore, scaling up and down typically occur in virtualized environments, where instances of software components required for a specific software service are instantiated only when needed.

[0006] Against this backdrop, several documents have been published: CN 2020 / 210 732 879 A describes a dependency graph-based microservice performance diagnostic method. As output, it generates a list to identify potential performance bottlenecks. Furthermore, US 2018 / 0 131 764 A1 describes how to dynamically scale application components using microservices. Therefore, the introduced technique monitors component usage during application execution and determines whether to migrate components to microservices based on this usage. Thus, components can be migrated to microservices by initially initiating the microservice on a remote computing device.

[0007] However, in a microservices architecture, previously required services are either insufficient, completed too late, or scaled down, so in both cases the entire system cannot run at its best performance.

[0008] Therefore, an improved method is needed to scale services up and down in the service mesh to optimize overall system performance. Summary of the Invention

[0009] According to one aspect of the invention, a computer-implemented method can be provided for reducing latency between software services running in a service mesh, wherein the software services are instantiated when processing a request. The method may include: providing multiple services that at least partially and collaboratively fulfill requests, wherein communication between the services is based on supporting components, such that each service can be linked to an associated supporting component.

[0010] The method may also include: creating a directed dependency graph of multiple services by tracing request flows between multiple services, such that nodes in the directed dependency graph represent services and edges in the directed dependency graph represent used communication paths between selected services; determining dependent services for incoming requests to selected services among the multiple services based on the directed dependency graph; and starting one of the selected services among the multiple services together with an instance of the dependent service.

[0011] According to another aspect of the invention, a distributed multi-service system can be provided for reduced latency between software services running in a service mesh, wherein the software services are instantiated when processing a request. The system may include one or more processors and a memory operatively coupled to the one or more processors, wherein the memory stores portions of program code that, when executed by the one or more processors, enable the one or more processors to: provide multiple services that at least partially and cooperatively fulfill requests, wherein communication between services is based on supporting components, and each service is linked to one of the supporting components.

[0012] It also enables one or more processors to create a directed dependency graph of multiple services by tracing request flows between multiple services, where nodes in the directed dependency graph represent services and edges represent used communication paths between selected services; based on the directed dependency graph, to determine dependent services for incoming requests to selected services among multiple services; and to start one of the selected services along with instances of the dependent services.

[0013] Furthermore, embodiments may take the form of related computer program products, accessible from a computer-usable or computer-readable medium that provides program code for use by or linked to a computer or any instruction execution system. For purposes of description, a computer-usable or computer-readable medium may be any means that may include components for storing, communicating, propagating, or transmitting programs for use by or linked to an instruction execution system, apparatus, or device. Brief description of the attached figures

[0014] It should be noted that embodiments of the present invention are described with reference to different themes. In particular, some embodiments are described with reference to method-type claims, while others are described with reference to apparatus-type claims. However, those skilled in the art will understand from the foregoing and following description that, unless otherwise stated, any combination of features belonging to one theme, as well as any combination of features from different themes, particularly combinations of features from method-type claims and features from apparatus-type claims, are also considered to be disclosed in this document.

[0015] The foregoing limitations and other aspects of the invention will be apparent from the examples of embodiments described below, and will be explained with reference to these examples, but the invention is not limited to these examples.

[0016] Preferred embodiments of the inventive concept will be described by way of example only and with reference to the following figures, but the inventive concept (with variations and at least partial substitutions) is not limited to these figures:

[0017] Figure 1 A block diagram illustrating an embodiment of a computer implementation method of the present invention for reducing latency between software services running in a service mesh is shown.

[0018] Figure 2 A block diagram of an embodiment of a service mesh is shown, illustrating the core dependencies between different components.

[0019] Figure 3A block diagram of an embodiment of a distributed multi-service system for reduced latency between software services running in a service mesh is shown.

[0020] Figure 4 It shows including according to Figure 3 An example of a computing system. Detailed Implementation

[0021] Embodiments of the present invention can be described as follows:

[0022] According to one aspect of the invention, a computer-implemented method can be provided for reducing latency between software services running in a service mesh, wherein the software services are instantiated when processing a request. The method may include: providing multiple services that at least partially and cooperatively fulfill requests, wherein communication between the services is based on supporting components, and each service is linked to an associated supporting component.

[0023] The method may also include: creating a directed dependency graph of multiple services by tracing request flows between multiple services, such that nodes in the directed dependency graph represent services and edges in the directed dependency graph represent used communication paths between selected services among the multiple services; determining dependent services for incoming requests to selected services among the multiple services based on the directed dependency graph; and starting instances of the selected services among the multiple services together with instances of dependent services.

[0024] According to another aspect of the invention, a distributed multi-service system can be provided for reduced latency between software services running in a service mesh, wherein the software services are instantiated when processing a request. The system may include one or more processors and a memory operatively coupled to the one or more processors, wherein the memory stores portions of program code that, when executed by the one or more processors, enable the one or more processors to: provide multiple services that at least partially and cooperatively fulfill requests, wherein communication between services is based on supporting components, and wherein each service is linked to an associated supporting component.

[0025] It also enables one or more processors to: create a directed dependency graph of multiple services by tracing request flows between multiple services, where nodes in the directed dependency graph represent services and edges in the directed dependency graph represent used communication paths between selected services among the multiple services; determine dependent services for incoming requests to selected services among the multiple services based on the directed dependency graph; and start one of the selected services among the multiple services together with instances of dependent services.

[0026] The proposed computer implementation method and related system for reducing latency between software services running in a service mesh, wherein software services are instantiated when a request is fulfilled, can provide several advantages, technical effects, contributions and / or improvements:

[0027] It can meet the need for improved methods to scale services up and down within the service mesh, thereby optimizing overall system performance. The end result is that available computing resources can be utilized more effectively. This also has environmental implications, as less energy may be required to deliver the same results to computer system users.

[0028] Therefore, the solution proposed in this paper not only observes and analyzes system behavior but also leverages knowledge of interactions between components to dynamically scale services up and down as early as possible. To this end, a directed dependency graph—which can be considered an experiential background system—can be used to dynamically learn system behavior (i.e., the dependent processes / services required to complete, for example, a user request). This approach enables the underlying system to adapt to different needs over time. This knowledge, condensed into the directed dependency graph, can be used to co-locate certain services on the same server or within the same virtual machine to minimize communication overhead.

[0029] In particular, a directed dependency graph structure using weighted (or weighted) factors between nodes (which can represent services) can derive specific affinity factors between services, or conversely, anti-affinity factors between services. This, in turn, can be used to deploy different services in different time zones, different underlying physical machines, and / or different geographical regions, provided they do not belong to the same dependency graph or are managed and optimized according to other systems.

[0030] Additional embodiments of the inventive concept applicable to the method and system will be described below.

[0031] According to another advantageous embodiment, the method may further include: starting the selected service along with multiple dependent services based on the number of incoming requests to the selected service. Therefore, if the number of triggered services (particularly the service receiving incoming requests) is higher than normal, the dependent services may also be instantiated at a higher frequency to be ready when the selected service is invoked. Thus, the multiplication factor between the selected service and one or more dependent services can be based on weights in the directed dependency graph.

[0032] To facilitate this, according to another advantageous embodiment of the method, the directed dependency graph can be a weighted directed dependency graph, where the weights can be related to the edges of the weighted directed dependency graph, representing the ratio between inbound and outbound requests for the service. Therefore, the natural characteristics and advantages of directed dependency graphs can be directly utilized without additional enhancements to the multiplicative factors between the selected service and other dependent services. The service can be a direct dependency (also known as an immediate dependency) or an indirect dependency (also known as a transitional dependency).

[0033] According to another preferred embodiment of the method, the ratio between the initiated service and the services that depend on the initiated service can be determined based on the corresponding weights of the edges between the relevant services. For example, if the weight factor value of the edge between service A and another service B is 5, then when a request arrives at an instantiated service A, the other service B can be instantiated. Fortunately, if service A is shut down, the five other dependent services B can also be shut down immediately, thereby reducing unnecessary overhead in the entire system.

[0034] According to one practical embodiment, the method may further include storing a directed dependency graph in a database. This database may be populated with messages from microservice sidecar elements or interpreted messages, where the sidecar elements act as communication components between services and other components. Furthermore, the method may include accessing records in the database by a scheduler suitable for instantiating services in a service mesh. This establishes a closed loop: during the observation phase, microservices can send “I’m alive / I’m needed / I’m running” messages to the database via communication components (or sidecar components), where weighting factors among nodes in the directed dependency graph can also be determined; during the usage phase, scaling up and down certain services can be based on the weighting factors still stored in the database. Therefore, a self-learning and self-adaptive system can be created, which can modify its behavior over time, thereby saving valuable system resources and self-optimizing throughout its lifecycle.

[0035] According to a further embodiment of the method, starting the selected service together with multiple dependent services may further include: co-proposing instances of the invoked dependent service together with instances of services from the multiple services (particularly those in the same framework component / the same virtual machine). Additionally, in one embodiment, the method may further include co-proposing services of multiple services that have a predetermined affinity value with each other. This may also be based on a weighting factor of the directed dependency graph. There may be affinity between services that are part of the same dependency graph. If they belong to different dependency graphs, anti-affinity may be assumed.

[0036] Therefore, according to a subsequent embodiment of the method, co-position can depend on the weight values ​​(e.g., can be interpreted as affinity values) of the edges between services in a directed dependency graph and the invoked dependent services.

[0037] According to a high-level embodiment of the method, the directed dependency graph can also be multiple disconnected directed dependency graphs. Thus, services related to different directed dependency graphs can be instantiated on different compute nodes at the service mesh layer. Therefore, interference or negative dependencies between different service groups can be eliminated.

[0038] According to a further enhanced and useful embodiment, the method may also include scaling down one of the multiple services along with the number of instances of its dependent services if the number of incoming requests decreases. Thus, a "breathing" system capable of adapting to varying usage intensities can be established. This can be particularly accomplished by a scheduler or autoscaler for the infrastructure framework used by the service mesh.

[0039] According to an alternative embodiment of the method, the scaling factor can depend on a ratio determined based on the corresponding weight values ​​of the edges between the respective services. Similarly, the characteristics of the directed dependency graph can also be used to scale down instantiated dependent services.

[0040] In the context of this specification, the following technical conventions, terms and / or expressions may be used:

[0041] The term "software service" (or simply service or microservice) refers to a function in a computing system environment that delivers a specific result when given certain input values. Software services can be implemented using different frameworks and architectural concepts, such as service-oriented architecture (SOA) or containerized architecture (like Docker). In a virtualized environment, software services can be instantiated only when needed.

[0042] The term "service mesh" can refer to multiple (software) services orchestrated to satisfy multiple incoming requests. The result of this service collaboration can be delivered to further consuming elements. Various frameworks are known for operating these services.

[0043] The term "request" or "incoming request" can refer to input values ​​received by a service to facilitate a specific result defined by the parameters of the request. For example, a request might query the number of maintenance cycles for a production machine, where the corresponding maintenance tasks and their results are stored in different tables in a database. This result may require coordinating different services that read data from different database tables or sensors, and combining portions of the results into a final response to the initial request.

[0044] The term "supporting component" (which can also refer to "sidecar component") refers to the software functionality responsible for the inbound and outbound communication of a component that includes a (micro)service. Therefore, as part of the concepts presented in this paper, a microservice never exists in isolation, but is always accompanied by a corresponding supporting component or sidecar component.

[0045] The term "dependency graph" can refer to a directed graph representing the dependencies between multiple objects. In this article, objects can be independent services, so a dependency graph can also refer to a "directed dependency graph" (DDG). A dependency graph allows us to deduce whether an evaluation order follows a given dependency relationship or whether such an evaluation order does not exist. The input to constructing a dependency graph can come from the instantiated services themselves, which use supporting components or sidecar components to transmit each instantiation of dependent services to a database storing information about the dependency graph.

[0046] The term "weighted directed dependency graph" can refer to a dependency graph that also stores weighted values ​​representing the dependencies between different nodes (i.e., instantiation multiplication factors). For example, if component or microservice A is instantiated and has a weighting factor of five (in the dependency graph) for component or microservice B, then microservice B can be instantiated with a factor of 5 compared to the number of instances of microservice A.

[0047] The term "request flow" can refer to a sequence of requests that are activated to satisfy a specific task based on an initial incoming request.

[0048] The term "node in a directed dependency graph" can represent a (micro)service in a service mesh. Meanwhile, edges or links between nodes can represent weighting factors or affinity factors between different services (i.e., different nodes in the dependency graph).

[0049] The term "dependent service" refers to a service that is only activated when invoked by another service it depends on (e.g., the service that receives the initial request). Dependencies between different services among multiple services can be represented by a dependency graph.

[0050] The term "incoming request" can refer to a task that will be fulfilled by the service mesh.

[0051] The accompanying drawings will now be described in detail. All illustrations in the drawings are schematic. First, a block diagram of an embodiment of a computer-implemented method of the present invention for reducing latency between software services running in a service mesh is given. Further embodiments, and related embodiments of distributed multi-service systems for reducing latency between software services running in a service mesh, will then be described.

[0052] Figure 1A block diagram of a preferred embodiment of a computer-implemented method 100 for reduced latency between software services running in a service mesh is shown. Thus, the software services of the service mesh are instantiated only when a service is invoked and a request is fulfilled. The method includes providing, 102, and using multiple services that can be organized as a network of dependent services. Typically, a management framework can be used to orchestrate the collaboration of multiple services. One or more services can at least partially and collaboratively fulfill a request; that is, not every request requires the complete set of services. Additionally, when discussing containerized service meshes, communication between services or microservices is based on specific supporting components or sidecar components or sidecar containers. Therefore, each component including microservices also includes supporting components for communication between services.

[0053] Additionally, method 100 includes creating, 104, a (weighted) directed dependency graph of multiple services by tracing request flows between them. The results of these tracings can be stored in a database or any other persistent storage device, such as a file system. Thus, nodes in the directed dependency graph represent services, and edges in the directed dependency graph represent used communication paths between selected (micro)services. Furthermore, edges may also be associated with weighting factors representing the multiplication factor between the calling service and the called service.

[0054] In addition, method 100 includes determining, 106, (or more preferably, at least one) dependent service (typically, determining multiple dependent services) for an incoming request to a selected service among a plurality of services based on a directed dependency graph.

[0055] Last but not least, method 100 includes starting one of the selected services along with instances of the dependent services, 108. Typically, multiple dependent services or microservices can be started based on weight factors between relevant nodes in a directed dependency graph.

[0056] Figure 2A block diagram 200 of a partial embodiment of the service mesh is shown, illustrating exemplary core dependencies between different components. Request 204 can be received by an application operated by user 202 and can also be received by gateway 206. Here, the request is sent to communication component 208, namely the sidecar component of component A 210, which represents (micro)service A in the service mesh of observation system 212. In this simplified example, observation system 212 includes component A 210, component B 214, component C 216, and component D 218. Thus, component B 214 includes microservice 220 along with sidecar component 222; component C 216 includes microservice 224 along with sidecar component 226; and component D 218 includes microservice 228 along with sidecar component 230. They collaborate to satisfy the incoming request 204 in order to deliver the result to an external component or external system 232.

[0057] To construct the dependency graph, sidecar components 208, 222, 226, and 230 provide information about the invocation state to the dependency graph builder (not shown), used to store data about the dependency graph or to store data stored in dependency graph storage 234. Over time, a weighted dependency graph can be constructed by tracking the invocation and activity events of components A, B, C, and D. This is characterized by "write" operations to dependency graph storage 234.

[0058] To manage multiple services or components, upon receiving a request from gateway 206, the scheduler or autoscaler 236 is activated, represented by line 238, shown as the dashed line behind the box of observation system 212. Scheduler 236 reads data from dependency graph storage 234 to activate or instantiate the corresponding component A, component B, component C, component D, or the corresponding microservices 211, 220, 224, and 228. This is represented by the "scaling" of arrows from scheduler 236 pointing to different services in the service mesh.

[0059] Additionally, a weighting factor “w” (or weight or weighting factor) is also included in the observed system 212, which indicates the weighting factor—for example, in the form of a:b—where “a” may indicate the number of instantiations of the invoked service, and “b” may indicate the number of instantiations of the invoked service. These weights “w” can be extracted from the dependency graph storage 234. Thus, the weighting factor represents the ratio of scaling down for instantiations of a service and the ratio of scaling up for instantiations of a service.

[0060] In the manner described, the number of instantiations of dependent services can be advantageously scaled based on the weight factors associated with the edges in the directed dependency graph and the frequency of incoming requests 204 to gateway 206.

[0061] Additionally, because it understands the dependencies between different components of a service or microservice, the scheduler 236 considers not only the number of instances but also the location where the service is executed. Therefore, related services are co-located within the same virtual machine, the same container framework, or the same physical compute node.

[0062] If the request rate on gateway 206 changes, scheduler 236 will traverse the dependency graph again and easily adjust the scaling of components based on the weight factors of the edges in the dependency graph. Thus, autoscalor 236 can handle multiple disconnected dependency graphs and components simultaneously, where autoscalor 236 introduces an anti-affinity factor between two disconnected dependency graphs; that is, if component A scales up, component A and its dependent component B are placed on different nodes.

[0063] Additionally, the term "affinity" is shown between component A 210 and component B 214. Affinity indicates that the two components belong to the same dependency graph. Exemplarily, the term "anti-affinity" is shown next to the dashed line between component B 214 and component D 218, indicating, for understandability reasons only, that the two services do not belong to the same dependency graph. In the example shown, this is not entirely realistic, as the relevant sidecar components are shown communicating with each other. The anti-affinity feature is more realistic when there is no communication path between the relevant sidecar components. However, for load balancing and performance optimization reasons, the anti-affinity feature can lead to the instantiation of the relevant services on different physical nodes at the service mesh underlying layer.

[0064] Figure 3 A block diagram of an embodiment of a distributed multi-service system 300 for reduced latency between software services running in a service mesh is shown. Also herein, software services are instantiated when a request is fulfilled. System 300 includes one or more processors 302 and a memory 304 operatively coupled to the one or more processors 302, wherein the memory stores portions of program code that, when executed by the one or more processors 302, enable the one or more processors 302 to provide multiple services—particularly in the form of a service mesh system 306—that at least partially and cooperatively fulfill requests. Therefore, communication between services is based on supporting components, wherein each service is linked to one of the supporting components.

[0065] This enables the one or more processors 302 to—in particular, using a direct dependency graph (DDG) creator and storage device 308—create a directed dependency graph of multiple services by tracing request flows between them. Thus, nodes in the directed dependency graph represent services, and edges represent used communication paths between selected services.

[0066] Furthermore, the one or more processors 302 are also able—in particular using the determining unit 310—to determine the dependent service for an incoming request for a selected service among a plurality of services based on a directed dependency graph, and—in particular by the starting unit 312—to start the selected service among the plurality of services together with an instance of the dependent service.

[0067] It should also be noted that all functional units, modules, and function blocks can be communicatively coupled to each other in a selected 1:1 manner for signal or message exchange. Alternatively, functional units, modules, and function blocks can be linked to the system internal bus system 314 for selective signal or message exchange. Therefore, the functional units, modules, and function blocks are one or more processors 302, memory 304, service mesh system 306, DDG creator and storage device 308, determination unit 310, and startup unit 312.

[0068] Various aspects of this disclosure are described by narrative text, flowcharts, block diagrams of computer systems, and / or machine logic block diagrams included in embodiments of a computer program product (CPP). With respect to any flowchart, operations may be performed in a different order than shown in a given flowchart, depending on the art involved. For example, again according to the art involved, two operations shown in consecutive flowchart blocks may be performed in reverse order, as a single integrated step, concurrently, or with at least partial temporal overlap.

[0069] Computer program product embodiment (CPP embodiment or CPP) is a term used in this disclosure to describe a collection of storage media (also referred to as media) included in a set of storage devices (one or more) that collectively comprise machine-readable code corresponding to instructions and / or data specified in a given CPP claim for performing computer operations. A storage device is any tangible device capable of holding and storing instructions for use by a computer processor. Without limitation, a computer-readable storage medium can be an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, a mechanical storage medium, or any suitable combination thereof. Some known types of storage devices that include these media include floppy disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), optical disc read-only memory (CDROM), digital versatile optical disc (DVD), memory sticks, floppy disks, mechanical encoding devices (such as punched cards or pits / planes formed on the main surface of the disc), or any suitable combination thereof. As used in this disclosure, the term computer-readable storage medium should not be construed as storage of transient signals, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides, optical pulses transmitted through fiber optic cables, electrical signals transmitted through wires, and / or other transmission media. As will be understood by those skilled in the art, during normal operation of a storage device, data is typically moved at certain points in time, such as during access, defragmentation, or garbage collection, but this does not render the storage device transient, as data is not transient at the time of storage.

[0070] Figure 4 A computing environment 400 is shown, which includes an example of an environment for executing at least some computer code relating to performing methods of the present invention (e.g., code block 450), such as code block 450 as a computer implementation of a method for reducing latency between software services running in a service mesh, wherein the software service is instantiated when a request is fulfilled.

[0071] In addition to frame 450, the computing environment 400 also includes, for example, a computer 401, a wide area network (WAN) 402, an end-user equipment (EUD) 403, a remote server 404, a public cloud 405, and a private cloud 406. In this embodiment, the computer 401 includes a processor set 410 (including processing circuitry 420 and a cache 421), a communication infrastructure 411, volatile memory 412, persistent storage 413 (including an operating system 422 and frame 450 as identified above), a peripheral device set 414 (including a user interface (UI), a device set 423, a storage device 424, and an Internet of Things (IoT) sensor set 425), and a network module 415. The remote server 404 includes a remote database 430. The public cloud 405 includes a gateway 440, a cloud orchestration module 441, a host physical machine set 442, a virtual machine set 443, and a container set 444.

[0072] Computer 401 can take the form of a desktop computer, laptop computer, tablet computer, smartphone, smartwatch, or other wearable computer, mainframe computer, quantum computer, or any other computer or mobile device now known or to be developed in the future capable of running programs, accessing networks, or querying databases (such as remote database 430). As is well known in the field of computer technology, and depending on the technology, the execution of a computer implementation method can be distributed across multiple computers and / or multiple locations. On the other hand, in this presentation of computing environment 400, to keep the presentation as simple as possible, the detailed discussion focuses on a single computer, specifically computer 401. Computer 401 can be located in the cloud, even... Figure 4 It is not shown in the cloud. On the other hand, unless there is any explicit instruction, computer 401 does not need to be located in the cloud.

[0073] Processor assembly 410 includes one or more computer processors of any type, including those currently known or to be developed in the future. Processing circuitry 420 may be distributed across multiple packages, such as multiple coordinated integrated circuit chips. Processing circuitry 420 may implement multiple processor threads and / or multiple processor cores. Cache 421 is memory located within the processor chip package(s) and is typically used to store data or code that should be quickly accessed by the threads or cores running on processor assembly 410. Cache memory is typically organized into multiple levels based on its relative proximity to the processing circuitry. Alternatively, some or all of the cache for the processor assembly may be located "off-chip". In some computing environments, processor assembly 410 may be designed to process qubits and perform quantum computing.

[0074] Computer-readable program instructions are typically loaded onto computer 401 to cause the processor set 410 of computer 401 to perform a series of operational steps to implement a computer-implemented method, such that the instructions, when executed, instantiate the method specified in the flowcharts and / or the descriptive description of the computer-implemented method included in this document (collectively, "the method of the present invention"). These computer-readable program instructions are stored in various types of computer-readable storage media, such as cache 421 and other storage media discussed below. The processor set 410 accesses these program instructions, along with associated data, to control and direct the execution of the method of the present invention. In computing environment 400, at least some of the instructions for performing the method of the present invention may be stored in block 450 of persistent storage device 413.

[0075] Communication structure 411 is a signal transmission path that allows the various components of computer 401 to communicate with each other. Typically, this structure includes switches and conductive paths, such as those forming buses, bridges, physical input / output ports, etc. Other types of signal communication paths, such as fiber optic communication paths and / or wireless communication paths, may also be used.

[0076] Volatile memory 412 is any type of volatile memory currently known or developed in the future. Examples include dynamic random access memory (RAM) or static RAM. Typically, volatile memory is characterized by random access, but is not required unless explicitly indicated. In computer 401, volatile memory 412 is located within a single package and is an internal component of computer 401; however, alternatively or additionally, volatile memory may be distributed across multiple packages and / or located externally to computer 401.

[0077] Persistent storage device 413 is any form of non-volatile storage known or developed for a computer. The non-volatility of this storage device means that the stored data is retained regardless of whether power is supplied to the computer 401 and / or directly to the persistent storage device 413. Persistent storage device 413 may be a read-only memory (ROM), but typically at least a portion of the persistent storage device allows data to be written, deleted, and rewritten. Some common forms of persistent storage devices include hard disks and solid-state storage devices. Operating system 422 can take many forms, such as various known proprietary operating systems or operating systems employing an open-source portable operating system interface type with a kernel. The code included in block 450 typically includes at least a portion of the computer code involved in performing the methods of the present invention.

[0078] Peripheral device set 414 includes a collection of peripheral devices of computer 401. Data communication connections between peripheral devices and other components of computer 401 can be achieved in various ways, such as Bluetooth connectivity, near field communication (NFC) connectivity, connections established via cables (such as Universal Serial Bus (USB) type cables), plug-in connections (e.g., Secure Digital (SD) cards), connections established via local area networks (LANs), and even connections established via wide area networks (WANs) (such as the Internet). In various embodiments, UI device set 423 may include components such as displays, speakers, microphones, wearable devices (such as goggles and smartwatches), keyboards, mice, printers, touchpads, game controllers, and haptic devices. Storage device 424 is an external storage device, such as an external hard drive, or a pluggable storage device, such as an SD card. Storage device 424 may be persistent and / or volatile. In some embodiments, storage device 424 may take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where computer 401 requires significant storage (e.g., where computer 401 locally stores and manages a large database), this storage can be provided by peripheral storage devices designed specifically for storing massive amounts of data, such as a Storage Area Network (SAN) shared by multiple geographically distributed computers. The IoT sensor set 425 includes sensors that can be used for Internet of Things (IoT) applications. For example, one sensor could be a thermometer, and another sensor could be a motion detector.

[0079] Network module 415 is a collection of computer software, hardware, and firmware that allows computer 401 to communicate with other computers via WAN 402. Network module 415 may include hardware such as a modem or WiFi transceiver, software for transmitting packetized and / or unpacked data for the communication network, and / or web browser software for communicating data over the Internet. In some embodiments, the network control functions and network forwarding functions of network module 415 are performed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing Software Defined Networking (SDN), the control functions and forwarding functions of network module 415 are performed on physically separate devices, such that the control functions manage multiple different network hardware devices. Computer-readable program instructions for performing the methods of the present invention can typically be downloaded to computer 401 from an external computer or external storage device via a network adapter card or network interface included in network module 415.

[0080] A WAN (402) is any wide area network (e.g., the Internet) capable of transmitting computer data over non-local distances using any computer data communication technology currently known or to be developed in the future. In some embodiments, a WAN can be replaced by and / or supplemented by a local area network (LAN), which is designed to transmit data between devices located in a local area (such as a WiFi network). WANs and / or LANs typically include computer hardware such as copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and edge servers.

[0081] End User Device (EUD) 403 is any computer system used and controlled by an end user (e.g., a customer of the enterprise operating computer 401), and can take any of the forms of discussion related to computer 401 above. EUD 403 typically receives useful and helpful data from the operation of computer 401. For example, in one hypothetical scenario, computer 401 is designed to provide recommendations to the end user, which are typically transmitted from network module 415 of computer 401 to EUD 403 via WAN 402. Thus, EUD 403 can display or otherwise present the recommendations to the end user. In some embodiments, EUD 403 can be a client device, such as a thin client, a thick client, a mainframe computer, a desktop computer, etc.

[0082] Remote server 404 is any computer system that supplies at least some data and / or functionality to computer 401. Remote server 404 may be controlled and used by the same entity operating computer 401. Remote server 404 represents multiple machines that collect and store useful data for use by other computers, such as computer 401. For example, in a hypothetical scenario, computer 401 is designed and programmed to provide recommendations based on historical data, which can then be provided to computer 401 from a remote database 430 of remote server 404.

[0083] Public cloud 405 is any computer system used by multiple entities, providing on-demand computer system resources and / or other computing capabilities, particularly data storage (cloud storage) and computing power, without the need for direct active management by the user. Cloud computing typically leverages resource sharing to achieve consistency and economies of scale. Direct active management of the computing resources of public cloud 405 is performed by the computer hardware and / or software of cloud orchestration module 441. The computing resources provided by public cloud 405 are typically implemented using virtual computing environments, which run on computers including a set of host physical machines 442, which is global to the physical computers within public cloud 405 and / or available to public cloud 405. Virtual computing environments (VCEs) typically take the form of virtual machines in set of virtual machines 443 and / or containers in set of containers 444. It should be understood that these VCEs can be stored as images and can be transferred between various physical machine hosts, either as images or after instantiation of the VCE. The cloud orchestration module 441 manages the transfer and storage of images, deploys new VCE instances, and manages active instances of VCE deployments. Gateway 440 is a collection of computer software, hardware, and firmware that allows the public cloud 405 to communicate via WAN 402.

[0084] We will now elaborate further on Virtualized Computing Environments (VCEs). A VCE can be stored as an "image." A new, active instance of a VCE can be instantiated from an image. Two common types of VCEs are virtual machines and containers. A container is a type of VCE that uses operating system-level virtualization. This refers to an operating system feature where the kernel allows the existence of multiple isolated user-space instances (called containers). From the perspective of a program running on an isolated user-space instance, these isolated user-space instances typically behave like a real computer. A computing program running on a regular operating system can utilize all the resources of that computer, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, a program running inside a container can only use the contents of the container and the devices assigned to the container; this characteristic is called containerization.

[0085] Private cloud 406 is similar to public cloud 405, except that its computing resources are available only to a single enterprise. While private cloud 406 is depicted communicating with WAN 402 in the diagram, in other embodiments, private cloud may be completely disconnected from the internet and accessible only via a local / private network. A hybrid cloud is a combination of multiple clouds of different types (e.g., private, community, or public cloud types), typically implemented by different vendors. Each of the multiple clouds remains an independent and discrete entity, but the larger hybrid cloud architecture is joined together through standardization or proprietary technologies that enable orchestration, management, and / or data / application portability across the multiple component clouds. In this embodiment, both public cloud 405 and private cloud 406 are part of a larger hybrid cloud.

[0086] It should also be noted that the distributed multi-service system 300 for reducing latency between software services running in a service mesh, wherein software services are instantiated when a request is fulfilled, may be an operating subsystem of computer 401 and may be attached to the computer’s internal bus system.

[0087] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used herein, the singular forms “a,” “an,” and “the” are intended to also include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprises” and / or “comprising”, when used in this specification, indicate the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0088] All the structures, materials, actions, and equivalents of the means or steps plus functional elements in the following claims are intended to include any structure, material, or action combined with other claim elements to perform the function, as described in the specific claims. The description of the invention is for illustrative purposes and is not intended to be exhaustive or to limit the invention to the forms disclosed herein. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the invention. The embodiments were chosen and described in order to best explain the principles of the invention and its practical application, and to enable others skilled in the art to understand that the invention is applicable to various embodiments with various modifications for the intended particular use. Finally, the inventive concept can be summarized by the following terms: 1. A computer-implemented method for reducing latency between software services running in a service mesh, wherein a software service is instantiated when a request is fulfilled, the method comprising: - Provides multiple services that at least partially and collaboratively fulfill requests, where communication between services is based on supporting components, and each service is linked to one of the supporting components. - By tracing the request flow between multiple services, a directed dependency graph is created, where nodes in the directed dependency graph represent services, and edges represent used communication paths between selected services. - Based on a directed dependency graph, determine the dependent services for an incoming request to a selected service among multiple services, and - Start the selected service from multiple services together with instances of the dependent services. 2. The method according to Clause 1 also includes The selected service, along with its dependent services, will be started based on the number of incoming requests to that service. 3. The method according to Clause 1 or 2, wherein the directed dependency graph is a weighted directed dependency graph, wherein the weights are related to the edges of the weighted directed dependency graph, and the weights represent the ratio between inbound and outbound requests for the service. 4. The method of Clause 3, wherein the ratio between the initiated service and the dependent initiated service is determined based on the corresponding weights of the edges between the relevant services. 5. The method according to any of the foregoing clauses also includes - Store the directed dependency graph in the database, and - Records in the database are accessed by the scheduler, which is applicable to the instantiation of services in the service mesh. 6. The method according to Clause 2, wherein starting the selected service together with multiple dependent services also includes - Co-locate instances of the invoked dependent services along with instances of services from multiple services, and - Co-locate multiple services that have a predetermined affinity value with each other. 7. The method according to Clause 6, wherein the co-location depends on the weight values ​​of the edges between a service in a directed dependency graph and the invoked dependent service. 8. The method according to any of the preceding clauses, wherein the directed dependency graph is a plurality of disconnected directed dependency graphs, and wherein services related to different directed dependency graphs are instantiated on different nodes at the service mesh underlying layer. 9. The method according to any of the foregoing clauses further includes: - If the number of incoming requests is decreasing, the selected service from multiple services, along with the number of instances of the dependent services, will be scaled down. 10. The method of claim 9, further comprising wherein the scaling factor depends on a ratio determined based on the respective weights of the edges between the respective services. 11. A distributed multi-service system for reduced latency between software services running in a service mesh, wherein software services are instantiated when a request is fulfilled, the system comprising: - One or more processors and a memory operatively coupled to the one or more processors, wherein the memory stores portions of program code that, when executed by the one or more processors, enable the one or more processors to - Provides multiple services that at least partially and collaboratively fulfill requests, where communication between services is based on supporting components, and each service is linked to a supporting component. - A directed dependency graph of multiple services is created by tracing the request flow between them. Nodes in the directed dependency graph represent services, and edges represent used communication paths between selected services. - Based on a directed dependency graph, determine the dependent services for an incoming request to a selected service among multiple services, and Start the selected service from multiple services, along with instances of the dependent services. 12. The system pursuant to Clause 11 also includes - Start the selected service along with its dependent services based on the number of incoming requests to the selected service. 13. A system pursuant to Clause 11 or 12, wherein the directed dependency graph is a weighted directed dependency graph, wherein the weights are related to the edges of the weighted directed dependency graph, and the weights represent the ratio between inbound and outbound requests for a service. 14. In a system pursuant to Clause 13, the ratio between initiated services and dependent initiated services is determined based on the corresponding weights of the edges between the relevant services. 15. A system according to any one of clauses 11 to 13, wherein one or more processors are also capable of - Store the directed dependency graph in the database, and - Records in the database are accessed by the scheduler, which is applicable to the instantiation of services in the service mesh. 16. A system pursuant to Clause 12, wherein one or more processors are also able to, during the startup of a selected service along with multiple dependent services, - Colocate instances of the invoked dependent services together with instances of services from multiple services. - Co-locate multiple services that have a predetermined affinity value with each other. 17. A system pursuant to Clause 16, wherein the weight of the edge between a service in a directed dependency graph and the invoked dependent service is co-located. 18. A system according to any of Clauses 11 to 17, wherein the directed dependency graphs are multiple disconnected directed dependency graphs, and wherein services relating to different directed dependency graphs are instantiated on different nodes at the service mesh underlying layer. 19. A system according to any one of clauses 11 to 18, wherein one or more processors are also capable of - If the number of incoming requests is decreasing, then scale down one of the selected services along with the number of instances of the dependent services. 20. A computer program product for reducing latency between software services running in a service mesh, wherein a software service is instantiated upon fulfilling a request, the computer program product including a computer-readable storage medium embodying program instructions, the program instructions being executable by one or more computing systems or controllers, causing one or more computing systems to... - Provides multiple services that at least partially and collaboratively fulfill requests, where communication between services is based on supporting components, and each service is linked to one of the supporting components. - A directed dependency graph of multiple services is created by tracing the request flow between them. Nodes in the directed dependency graph represent services, and edges represent used communication paths between selected services. - Based on a directed dependency graph, determine the dependent services for an incoming request to a selected service among multiple services, and - Start the selected services in the service list along with instances of the dependent services.

Claims

1. A computer-implemented method for reducing latency between software services running in a service mesh, wherein the software services are instantiated when a request is fulfilled, the method comprising: - Provides multiple services that at least partially and collaboratively fulfill requests, wherein communication between the services is based on supporting components, and each service is linked to one of the supporting components. - A directed dependency graph of the multiple services is created by tracing the request flow between the services, whereby the nodes of the directed dependency graph represent services, and the edges of the directed dependency graph represent used communication paths between selected services. - Based on the directed dependency graph, determine the dependent services for an incoming request to a selected service among the plurality of services, and - Start the selected service from the plurality of services together with instances of the dependent services.

2. The method according to claim 1, further comprising: - Based on the number of incoming requests to the selected service, start the selected service together with multiple dependent services.

3. The method according to claim 1 or 2, wherein the directed dependency graph is a weighted directed dependency graph, wherein the weights are related to the edges of the weighted directed dependency graph, and the weights represent the ratio between inbound and outbound requests for the service.

4. The method of claim 3, wherein the ratio between the initiated service and the dependent initiated service is determined based on the corresponding weights of the edges between the relevant services.

5. The method according to any of the preceding claims, further comprising: - Store the directed dependency graph in a database, and - Records in the database accessed by a scheduler, which is suitable for instantiating services in the service mesh.

6. The method of claim 2, wherein starting the selected service together with the plurality of dependent services further includes - Colocate instances of the invoked dependent services together with instances of the services from the multiple services, and - Co-locate multiple services that have a predetermined affinity value with each other.

7. The method of claim 6, wherein the co-location depends on the weight value of the edge between the service and the invoked dependent service in the directed dependency graph.

8. The method according to any of the preceding claims, wherein the directed dependency graph is a plurality of disconnected directed dependency graphs, and wherein services related to different directed dependency graphs are instantiated on different nodes of the service mesh underlying layer.

9. The method according to any of the preceding claims, further comprising: - If the number of incoming requests is decreasing, then the selected service among the plurality of services, along with the number of instances of the dependent services, will be scaled down.

10. The method of claim 9, further comprising the fact that the scaling factor depends on a ratio determined based on the respective weights of the edges between the respective services.

11. A distributed multi-service system for reduced latency between software services running in a service mesh, wherein the software services are instantiated when a request is fulfilled, the system comprising... - One or more processors and a memory, the memory being operatively coupled to the one or more processors, wherein the memory stores a portion of program code that, when executed by the one or more processors, causes the one or more processors to... - Provides multiple services that at least partially and collaboratively fulfill requests, wherein communication between the services is based on supporting components, and each service is linked to a supporting component of the supporting components. - By tracing the request flow between the multiple services, a directed dependency graph is created for the multiple services, where the nodes of the directed dependency graph represent services, and the edges of the directed dependency graph represent used communication paths between selected services. - Based on the directed dependency graph, determine the dependent services for an incoming request to a selected service among the plurality of services, and - Start the selected service from the plurality of services together with instances of the dependent services.

12. The system of claim 11, further comprising: - Based on the number of incoming requests to the selected service, start the selected service together with multiple dependent services.

13. The system of claim 11 or 12, wherein the directed dependency graph is a weighted directed dependency graph, wherein the weights are related to the edges of the weighted directed dependency graph, and the weights represent the ratio between inbound and outbound requests for the service.

14. The system of claim 13, wherein the ratio between the initiated service and the dependent initiated service is determined based on the corresponding weights of the edges between the relevant services.

15. The system according to any one of claims 11 to 13, wherein the one or more processors are further capable of - Store the directed dependency graph in a database, and - Records in the database accessed by a scheduler, which is suitable for instantiating services in the service mesh.

16. The system of claim 12, wherein the one or more processors are also capable of, during the period when the selected service is started together with the plurality of dependent services, - Colocate the instances of the invoked dependent services together with the instances of the services from the multiple services. - Co-locate multiple services that have a predetermined affinity value with each other.

17. The system of claim 16, wherein the co-location depends on the weight value of the edge between the service and the invoked dependent service in the directed dependency graph of the plurality of services.

18. The system according to any one of claims 11 to 17, wherein the directed dependency graph is a plurality of disconnected directed dependency graphs, and wherein services related to different directed dependency graphs are instantiated on different nodes of the service mesh underlying layer.

19. The system according to any one of claims 11 to 18, wherein the one or more processors are further capable of - If the number of incoming requests is decreasing, then the selected service among the plurality of services, along with the number of instances of the dependent services, will be scaled down.

20. A computer program product for reducing latency between software services running in a service mesh, wherein the software services are instantiated upon fulfillment of a request, the computer program product comprising a computer-readable storage medium on which program instructions are embodied, the program instructions being executable by one or more computing systems or controllers, causing the one or more computing systems to... - Provides multiple services that at least partially and collaboratively fulfill requests, wherein communication between the services is based on supporting components, and each service is linked to one of the supporting components. - A directed dependency graph of the multiple services is created by tracing the request flow between the services, whereby the nodes of the directed dependency graph represent services, and the edges of the directed dependency graph represent used communication paths between selected services. - Based on the directed dependency graph, determine the dependent service for an incoming request to one of the selected services from the plurality of services, and - Start the selected service from the plurality of services together with instances of the dependent services.

Images