In-situ triggered functions as a service within a service mesh

Abstract

The present technology relates to a system and method for automatically triggering related code segments corresponding to a series of code segments or function codes with a preferred execution order. The automatic triggering action is based on listening to the response generated due to the execution of the previous code segment. Information about the next code segment in the preferred execution order can be obtained by instructing a network proxy such as Envoy to listen to the uniform resource identifier (URI) field of the response packet being forwarded to the client entity. In this way, the network proxy can preemptively generate and instantiate the subsequent function code (pointed to by the monitored uniform resource identifier) ​​before receiving the corresponding client request. In this way, when a client request for the subsequent function code is received, the code is ready to execute.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of and priority to U.S. non-provisional patent application No. 16 / 102,395, filed on August 13, 2018, entitled “IN SITU TRIGGERED FUNCTION ASA SERVICE WITHIN A SERVICE MESH,” which is expressly incorporated herein by reference in its entirety. Technical Field

[0003] The present technology relates to serverless computing. More specifically, the present technology aims to improve the performance of serverless architecture in a service mesh environment. Background Art

[0004] Real-world implementations of microservices-based architectures must be able to implement most of the functionality supported by the central bus (i.e., the Enterprise Service Bus used in service-oriented architecture implementations) at the microservice level. Therefore, although the same basic set of problems are solved, these basic problems are solved by microservices in different dimensions. For example, the Enterprise Service Bus architecture can be used to easily handle scenarios involving calling multiple downstream services in a resilient manner and exposing functionality as another (composite) service by leveraging the architecture's built-in capabilities to build virtual / composite services and functions that are useful during inter-service communication (e.g., circuit breakers, timeouts, service discovery, etc.). However, implementing the same scenario in a microservices-based architecture must be implemented using a set of (composite and atomic) microservices and cannot benefit from the centralized integration layer of the Enterprise Service Bus. Therefore, in a microservices-based architecture, all required functionality must be implemented at the microservice level.

[0005] Microservices include business logic fragments that implement one or more functions, service composition / integration logic, or perform calculations. In addition to business logic fragments, microservices also include network functions to facilitate inter-service communication mechanisms (and enable basic service calls through given protocols, application resilience and stability patterns, service discovery, etc.). These network functions can be built on top of the network stack at the underlying operating system level. In such a microservice-based architecture, it will be very complex to implement functions related to service-to-service communication from scratch. A lot of time / energy / resources must be spent on building service-to-service communication functions instead of implementing business logic. If multiple technologies are used to build microservices (for example, multiple programming languages), the situation will be even worse because the same time and energy must be repeated between different languages. Therefore, the most complex challenge in implementing a microservice architecture is not building the service itself, but building communication between services. Since most inter-service communication requirements are very common in all microservice implementations, a remedial solution may involve offloading all such tasks to different layers so that the service code can be kept independent. This can be achieved using a service mesh.

[0006] In short, a service mesh is an inter-service communication infrastructure used to make service-to-service communication secure, reliable, observable, and configurable. In a service mesh environment, a given microservice does not communicate directly with other microservices, but all service-to-service communication will be based on a software component called a service mesh. The service mesh can also provide built-in support for some network functions such as fault tolerance (resilience) and service discovery.

[0007] The concept of Function as a Service (FaaS) associated with serverless computing is closely related to the microservice-based architecture. Software developers can use this approach to deploy a single function (which can be represented as a single or a series of one or more microservices), actions, or business logic fragments directly into the cloud network. These functions are expected to start within a few milliseconds when called and process each request after the process ends. The function-as-a-service framework implements a complete abstraction of the server away from the developer, resulting in, for example, a billing system based on consumption and execution and event-driven and instantly scalable services. Function as a service is particularly suitable for large-volume transactions. For example, when a transaction or code snippet is executed hundreds of times per second, it will be very beneficial to isolate the corresponding logic into easily scalable functions. Functions may have many potential uses, for example, they provide an efficient model to perform scheduled tasks (running specific code snippets according to a predefined schedule) or manually run code snippets when necessary. Another suitable use case may involve processing network requests and / or queue messages.

[0008] Elasticity and scalability are fundamental attributes of any application architecture. Utilizing a function-as-a-service model provides a flexible way to add scalability and elasticity to application architecture. It can also reduce costs because resources are allocated based on the needs of the application. Function-as-a-service architecture is auto-scalable, and when traffic or load increases, auto-scalar adds resources to handle additional activities, and removes resources to help reduce costs when traffic or load decreases. It also eliminates the need to manage servers on the cloud and replaces servers on the cloud by widely using function computing resources.

[0009] In a serverless architecture, short-lived functions are called on each request, and the provider can charge customers for running each individual function or business logic (hence the name Function as a Service or FaaS). Functions in a serverless or Function as a Service architecture are small, discrete, and reusable blocks of code that can be represented as a single or a series of one or more microservice operations. These functions can also be chained together. For example, a network request can be written to a queue, which is then picked up by a different function. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to describe the manner in which the above and other advantages and features of the present disclosure can be obtained, the principles briefly described above will be described in more detail by reference to specific embodiments shown in the accompanying drawings. It should be understood that these drawings depict only exemplary embodiments of the present disclosure and are therefore not to be considered limiting of its scope. The principles of this document are described and explained with additional specificity and detail through the use of the accompanying drawings, in which:

[0011] Figure 1 A basic operational flow chart of serverless computing in a standard service mesh environment is shown.

[0012] Figure 2 An operational flow chart of an enhanced serverless framework based on in-situ triggering of service code to improve service grid response time according to an embodiment of the present technology is shown.

[0013] Figure 3 The basic operation steps for implementing serverless computing in a service grid environment based on in-situ triggering of service codes according to an embodiment of the present technology are shown.

[0014] Figure 4 An overview of a system for implementing cloud-based serverless computing in a service mesh environment according to an embodiment of the present technology is shown. DETAILED DESCRIPTION

[0015] Various aspects of the invention are set out in the independent claims and preferred features are set out in the dependent claims. Features of one aspect may be applicable to each aspect alone or in combination with the other aspects.

[0016] Additional features and advantages of the present disclosure will be set forth in the following description, and in part will be apparent from the description, or may be learned by practicing the principles disclosed herein. The features and advantages of the present disclosure may be realized and obtained by the means and combinations particularly indicated in the appended claims. These and other features of the present disclosure may become more apparent from the following description and the appended claims, or may be learned by practicing the principles set forth herein.

[0017] Overview

[0018] Disclosed are systems, methods, and devices that relate to automatically calling relevant function code or microservice instances in a serverless architecture. In addition to facilitating the invocation and execution of function code in response to a trigger such as an initial client request and forwarding the corresponding (one or more) response packets back to the client entity, the controller entity or network agent can also perform additional operations on the forwarded / routed packets. For example, (one or more) client-specified response packets can be snooped to reveal information about any function code that may be the target of subsequent client requests / queries. Snooping return traffic can include checking the uniform resource identifier field for any information about identification, location, and related access / communication protocols, as well as other relevant details associated with the next (micro) service or code segment. As disclosed in some embodiments of the present technology, based on the intercepted information, a network agent such as Envoy can preemptively generate and instantiate (pointed to by the intercepted uniform resource identifier) ​​subsequent function code before receiving the corresponding client request. The triggering event can be a client request for the initial function code in the service chain that implements the required function. Aspects of the present technology also relate to automatically instantiating / calling subsequent (micro)service codes after calling and executing a first (micro)service code among multiple (micro)service codes having a preferred execution order (triggered by an initial event).

[0019] Example Embodiments

[0020] Disclosed are systems, methods, and non-transient computer-readable storage media for automatically triggering function codes associated with a service chain. Embodiments of the present technology also relate to implementing a network agent with enhanced functionality so as to improve the response time of the underlying service grid architecture. A network agent such as ENVOY can be configured to inspect data traffic generated in response to a client request. The inspection is intended to discover the next action or the next set of actions that may be required to be initiated by the client. If the discovered action involves executing a code snippet such as a function or a (micro) service instance, the network agent will facilitate the setup and instantiation of the intercepted function code so that it is ready to execute upon receiving a corresponding request from a client entity. Various embodiments of the present disclosure are discussed in detail below. Although specific implementations are discussed, it should be understood that this is for illustrative purposes only. Those skilled in the relevant art will recognize that other components and configurations may be used without departing from the spirit and scope of the present disclosure.

[0021] A service is an application unit with a unique name that other services use to refer to the called function. A service instance can be a pod / VM / container that implements the service. From a serverless computing perspective, a function is an abstract version of a service that may not have any computing resources associated with it. A function can be represented by a code snippet on a host device that encodes some functionality or business logic. In response to a trigger such as the occurrence of a specific event, the function code can be called and executed by an external entity.

[0022] Serverless computing allows service code to be implemented as a function without allocating any computing resources. Currently, Function as a Service (FaaS) works more on demand, so that whenever there is a trigger (e.g., a group of interest or a request for a specific function), the associated controller or network agent will execute the corresponding software code. The trigger for executing the code may also involve, for example, scheduling tasks, Hypertext Terminal Protocol (HTTP) requests, responding to queue messages, or manually calling code snippets when necessary. From the perspective of the end user, dedicated computing resources associated with managing server-based services may not be required. Alternatively, it is less costly to implement application-related functions or business logic fragments as a set of functions (abstract versions of services without associated computing resources) executed by external entities (such as computing services, such as AWS Lambda). The external entity will monitor certain triggers and call and execute the function code in response to certain triggering events. After execution, the function can be shut down. This feature operation is a distinguishing feature between server-based architectures and serverless (function as a service) architectures.

[0023] In a standard implementation of a serverless model in a service mesh environment, a triggering event (e.g., a client request / query) is required to generate an instance of the corresponding function code, which can then be executed to generate a response, which is then forwarded to the client. In the case where the requested function requires multiple function codes to be called in sequence, the required function codes must be individually triggered by a directed client request to the (micro)service. The execution of each function generates a response that can be forwarded to the client. The response can also include information about the subsequent function code to which the next client request will be directed. An example of a standard implementation of Functions as a Service in a service mesh environment is provided below.

[0024] A web service application implemented in a serverless architecture can be divided into multiple functions. For example, the front-end function can handle incoming network requests (i.e., http requests), and the back-end function can handle processes such as user authentication and access authentication. In this case, the incoming client http request will be directed to the front-end function. After receiving the http request, the external entity (e.g., AWS Lambda network proxy) will execute the front-end function and forward the result to the client. However, if there is a back-end authentication function associated with the web service application, the user making the request must first establish appropriate credentials before being granted access to the requested website. In order to properly identify the user, the front-end function may include an appropriate uniform resource identifier and the required details provided by the user in its response to the client's http request (e.g., a request for a username and password).

[0025] A Uniform Resource Identifier is a string designed to unambiguously identify a resource and is used to interact with a representation of the resource over a network using a specific protocol. In the case of the above example, the Uniform Resource Identifier sent back to the client by the front-end function of the network service / application may point to the back-end function of the application (i.e., the authentication service). Therefore, the next set of packets or the next traffic flow from the client will point to a completely different service based on the Uniform Resource Identifier, and the next round of negotiation will essentially occur between the client and the authentication function, which has to be cold-started when triggered by the client request.

[0026] When implementing the Function as a Service (FaaS) model in a service grid, each function may have one or more (micro) services corresponding to one or more code snippets, each of which needs to be cold started (instantiated / initialized) and executed when triggered by an event such as a directed client request. Cold starting a function in a serverless platform can take a considerable time to load, which can range from about a few milliseconds to several minutes. In addition, this operation will need to wait for a directed client request before it is started. Currently, it is not possible to instantiate the pointed code segment in the return traffic to the client (which is generated as a result of executing the previous function code) before receiving the corresponding trigger (i.e., the client request). Therefore, the described scheme associated with the standard operation of the associated technology results in an operational delay corresponding to the generation and instantiation of the request (micro) service or function code generated by each service code request of the client.

[0027] Some embodiments of the present technology are directed to improving the latency and delay associated with standard approaches to implementing microservices-based architectures. To this end, embodiments of the present technology disclose a method of data plane triggering for automatically identifying and instantiating relevant function code within a service mesh after an initial trigger. Even when a service mesh is implemented according to a function-as-a-service model, embodiments of the present technology provide improvements in operational response time.

[0028] Figure 1 An example scenario is shown illustrating the inherent operational latency and response time delays associated with a serverless-based service mesh environment.

[0029] Figure 1 A serverless implementation 100 based on a service grid is shown, wherein a function-as-a-service model is implemented in a service grid environment. Figure 1 As shown, functionality 102 is implemented as a series of three (micro) services 104, 106 and 108. Figure 1 , in response to detecting an initial query / request 110 from a client entity, a (micro)service 104 (which is the first in the series) is triggered. The (micro)service 104 corresponds to a code segment or business logic segment that performs some operations when executed. Upon receiving the initial request 110, the code segment corresponding to the (micro)service 104 is instantiated / initialized. This process causes a delay 112, after which at 114 the code segment is ready to be executed. The execution may also cause a delay 116, after which at 118, a response 120 is generated and forwarded to the client entity. The response 120 may include details about the (micro)service 106 so as to instruct the client entity to send the next request / query 124 to the (micro)service 106.

[0030] Upon receiving the request / query 124, the code segment corresponding to the (micro)service 106 is instantiated / initialized. Again, this process causes a delay 126, after which at 128 the code segment is ready to be executed. This execution may also cause a delay 130, after which at 132 a response 134 is generated and forwarded to the client entity. Similar to the previous stage response 120, the response 134 may include detailed information about the next (micro)service (108 in the case of the client response 134), which instructs the client entity to send the next request / query 138 to the (micro)service 108.

[0031] Upon receiving the request / query 138, the code segment corresponding to the (micro)service 108 is instantiated / initialized. This process generates a delay 140, after which the code segment is ready to be executed at 142. The execution may also cause a delay 144, after which a response 148 is generated and forwarded to the client entity at 146, at which point the operation of the function 102 is completed.

[0032] like Figure 1 As shown, in the standard solution, a client request for each code segment must be directed to each code segment in order to instantiate / initialize and execute the code segment. Some embodiments of the present technology are intended to improve and implement reference Figure 1 The latencies and delays associated with the standard scheme of a microservices-based architecture shown and described. To this end, embodiments of the present technology disclose a method of data plane triggering for automatically identifying and instantiating related functional code within a service mesh after an initial trigger. Even when the service mesh is implemented according to a function-as-a-service model, embodiments of the present technology provide improvements in operational response time. According to some embodiments, this can be achieved by having an initial request (i.e., an http request in the aforementioned example) trigger another set of related functions, so that the related functional code is instantiated and ready to execute when a client request directed to a specific code segment is received.

[0033] As described above, when an event (e.g., an http request) triggers an associated service function (expressed by a function code), return traffic corresponding to the execution result of the function code is sent back to the client. The response packet (s) will carry all relevant information (e.g., the uniform resource identifier of the next service function) and any related descriptions, as well as relevant cookie details required by the client to carry additional details associated with the invocation and execution of the next service.

[0034] According to some embodiments of the present technology, in addition to the required layer 7 routing and forwarding of response messages (i.e., http responses) back to the client, the network communication agent (e.g., the data plane portion of the service mesh, i.e., Envoy) can also perform additional functions. According to some embodiments, the additional functions performed by the network communication agent (i.e., Envoy) include listening to response traffic to the client. For example, Envoy will listen to the packet (i.e., look at the payload of the http packet) to discover whether there is a uniform resource identifier within the packet, and if so, identify and locate the uniform resource identifier.

[0035] If the requested web page is a simple web page that does not require authentication (or if the authentication requirement is avoided due to the nature of the user making the request), then there may be no need to call the backend authentication function. Therefore, in these cases, there may be no next service function to which the return response points, and therefore, the response traffic will not contain a uniform resource identifier. However, according to some embodiments, if the associated network proxy (i.e., Envoy) discovers a uniform resource identifier in the response packet when listening to the return traffic, it will identify the associated service function of the discovered URI, and according to some embodiments, a notification can be sent to the appropriate service for the incoming client request before the corresponding request from the client is received to facilitate the instantiation / initialization of the associated function code. Service call.

[0036] In a serverless architecture implemented according to some embodiments of the present technology, the instantiation of all function codes after the initial function code can be triggered by executing the previous function code. In some embodiments, this can be implemented as follows: listen to the response traffic to the client generated by the execution of the previous function code, thereby checking and analyzing the uniform resource identifier (URI) that may exist in the payload of the response traffic. By checking the uniform resource identifier, the next function code in the chain can be identified and located. Therefore, a local communication management entity such as Envoy can appropriately trigger the instantiation of one or more subsequent function codes when forwarding the response from the previous function code to the client. In this way, when the corresponding request / query for the subsequent function code is received from the client, the function code is ready to be executed, so any delay associated with establishing (instantiating / initializing) all function codes after the initial function code (the first in the series) can be avoided or at least reduced.

[0037] Figure 2 An exemplary implementation 200 corresponding to serverless computing of a service grid architecture implemented according to some embodiments of the present technology is shown. In the serverless computing implementation 200, the function 202 is appropriately represented as a series of three (micro) services 204, 206 and 208 (in the same way as Figure 1The initial function code or (micro)service 204 is processed in the same way as the function 102 in FIG. Figure 1 The processing of the initial function code 104 of the illustrated function 102 proceeds in a similar manner. The process involves the instantiation and execution of the initial function code 204 triggered by the detection of an initial query / request 210 originating from a client entity.

[0038] refer to Figure 2 In example 200, time span 212 corresponds to the duration of the instantiation process associated with function code 204, and time span 216 corresponds to the duration of the execution operation associated with function code 204. When the execution operation at 218 is completed, a client response 220 is generated and forwarded to the client via a local communication controller or network proxy such as Envoy. However, in addition to performing the required Layer 7 routing and forwarding operations, the local network proxy or communication controller in exemplary embodiment 200 also performs additional actions such as listening operation 222 on return traffic 220 to the client to detect whether additional function code needs to be instantiated for function 202. According to some embodiments, the listening operation retrieves information included in the uniform resource identifier field of (one or more) return packets to determine the identity, location, and associated protocol corresponding to the next code segment (or function code) that must be called and executed as part of function 202, as well as other related information.

[0039] Back to Figure 2 , the network proxy identifies, locates, and triggers the instantiation of subsequent function code 206 based on the uniform resource identifier information intercepted from the return traffic 220 to the client. This function is performed in conjunction with or in addition to the appropriate routing and forwarding of the response traffic 220. Therefore, the instantiation of the function code 206 is triggered before a request for the function code 206 is received from the client. Figure 2 In the exemplary case shown in the operational flow chart of FIG. 2 , when a client request 224 is received for function code 206 (as pointed to by a uniform resource identifier intercepted from return traffic 220), the delay 226 associated with instantiation of function code 206 has elapsed (the instantiation operation is complete), and the function code is ready for execution (as Figure 2The improved response time can be attributed in part to the fact that the function code 206 in the embodiment 200 is invoked earlier (based on intercepting the return traffic 220 generated by the previous function code 204) than the corresponding function code 106 in the embodiment 100. Therefore, the execution preparation phase 228 associated with the function code 206 in the exemplary embodiment 200 occurs before the corresponding execution preparation phase 128 associated with the function code 106 in the embodiment 100. This is due to the deployment of the interception-based triggering mechanism in the embodiment 200.

[0040] Return again Figure 2 , after a delay period 230 associated with the execution of the function code 206, at 234, (one or more) response packets 232 are generated by the associated proxy service and forwarded to the client. In addition to generating and forwarding the response traffic 232 to the client, at step 234, the network proxy is instructed to perform a listening operation 236 on the response traffic. The listening operation may involve inspecting a uniform resource identifier field to retrieve information about a subsequent function code to which the next client query should be directed. Figure 2 , the uniform resource identifier field of the (one or more) return packets 232 points to the function code 208, which is the next function code of the function 202 in the series. Based on the information retrieved from the intercepted uniform resource identifier, the network agent identifies, locates and calls the function code 208 before receiving the corresponding client request 235. When the client request 235 for the function code 208 is received, the instantiation time period 238 has passed and the function code is ready to execute. After the execution time period 240 expires, at 244, the final client response 242 is generated and forwarded to the client. Since the function code or (micro) service 208 constitutes the last part of the function 202, no uniform resource identifier is returned after intercepting the response traffic 242, and the execution of the function 202 is completed.

[0041] In some embodiments, the serverless architecture described above is implemented in a service mesh environment. In the service mesh environment described above, data plane functions may be provided by a network communication agent such as Envoy, which may perform all operations related to forwarding requests and messages to (micro) services or function codes, forwarding requests and messages from (micro) services or function codes, and forwarding requests and messages between (micro) services or function codes. Envoy may be characterized as an independent process that is designed to run with each application server. All Envoys form a transparent communication mesh in which each application sends and receives messages to and from the local host, and is unaware of the network topology. In essence, Envoy mediates and controls all network communications between services in the service mesh. According to some embodiments, in addition to routing / forwarding of request / response packets, Envoy may also be configured to perform a set of actions on packets being forwarded, such as listening / sniffing operations.

[0042] Some embodiments of the present technology utilize Envoy to listen to return packets from the initial function code instance and all subsequent function code, thereby detecting whether any other code needs to be instantiated for a given function. According to some embodiments, this may involve dynamically updating the routing rules (RouteRule) used by Envoy. Routing rules provide instructions on how to forward messages (e.g., http requests) between different (micro) services and facilitate communication between different services in a service mesh. According to some embodiments, in addition to layer 7 routing and forwarding (one or more) response packets, routing rules can also be statically or dynamically configured to listen to (one or more) reply packets and perform additional activities, such as triggering the instantiation of subsequent functions pointed to in the uniform resource identifier field of the listened (one or more) reply packets, for example, in a service chain.

[0043] Once the Uniform Resource Identifier field is intercepted, the network communication agent (i.e., Envoy) can use a local mapping table to identify whether the intercepted Uniform Resource Identifier points to an executable code snippet or a standard container / virtual machine. If the intercepted Uniform Resource Identifier points to an executable function code or a (micro) service, and if the function code or (micro) service is connected to the same Envoy or Pod, the local Envoy instance (attached (sidecar) agent) can use internal signaling to trigger the instantiation of the function code. If the identified function code is connected to a different / remote Envoy or Pod, the local Envoy instance can use an in-band mechanism (i.e., iOAM) to send a signal to trigger the instantiation of the function code. Therefore, when a subsequent client request for the function code is received, the function code has been called out, instantiated, and ready to be executed. The Envoy routing rules can then be updated again to ensure that the subsequent function code in the series is ready to execute in advance, and so on.

[0044] According to some embodiments of the present technology, Figure 3 The flowchart 300 shown provides a basic sequence of operational steps for implementing automatic instantiation of related code segments in a service grid environment. At 302, a client generates a request for a service. The request is sent over the Internet and is routed to an appropriate cloud-based computing / communication service. At 304, when the appropriate computing / communication service receives the client request, it locates and instantiates the corresponding function code. At 306, the computing / communication service executes the function code and generates a response to the client request. At 308, the generated response is forwarded to the client. In addition to forwarding the generated response to the client (308), at 310, the computing / communication service also performs a listening action on the generated response. The listening action performed in step 310 is intended to identify subsequent function codes that may be required in the service requested by the client. If a reference to another function code is found in the generated client response through the listening operation, an instance of the referenced function code is generated preemptively at 312 before receiving an actual client request for the referenced function code. In this way, when the client generates and distributes a request for the referenced function code at step 314, the referenced function code has been instantiated and is ready to execute. Therefore, when a directed client request is received, at 316, the target function code intercepted from the previous client response is executed and a response is generated. Then at 318, the generated response is forwarded to the client.

[0045] The response generated at 316, in addition to being forwarded to the client (318), is also intercepted for a reference to a subsequent service code, as indicated by arrow 319 pointing back to step 310. If the interception of the return traffic generated at 316 reveals a reference to another service code, then steps 312 through 318 are repeated. Otherwise, the process may terminate after completion of step 318, as it will constitute the final response to complete the client request generated at 302.

[0046] Figure 4 An exemplary service mesh-based serverless computing system 400 according to some embodiments of the present technology is shown. In the serverless computing system 400, each function (function code) 402 uploaded to the service provider cloud network 403 can be paired with a local proxy instance 404. This will enable communication between different (micro) services / functions and facilitate other service mesh data plane functions.

[0047] refer to Figure 4 , the client 408 generates a request for a specific service (i.e., an http request for accessing a website). The request is routed to the entry / incoming point 410 of the appropriate service provider cloud network 403 via the Internet, and is routed to the service grid-based computing platform 412. If the requested service code (function) has not yet been uploaded to the computing / execution module 414, the service code (function) can be located and accessed from the storage service 416, encapsulated into an appropriate runtime environment 418 (i.e., a container, a virtual machine, etc.), and then executed using the computing / execution module 414. The generated response 420 can then be forwarded by the service grid data plane 421 through the exit point 422 and reach the client.

[0048] However, in addition to the forwarding operation, the exemplary system 400 implemented according to some embodiments of the present technology may also perform additional actions on the generated response. Figure 4 , the generated response 420 to the exit point 422 is intercepted (424) by the service mesh data plane controller or agent software 421 to obtain relevant information. For example, the content of the uniform resource identifier field in the generated response (return traffic) 420 can be intercepted to obtain information related to the next function code 425 that needs to be called and executed as the next stage to complete the client request. Once the subsequent function code 425 related to the client request and pointed to by the intercepted return response 420 is identified and located by the system, the appropriate service is alerted to initiate preparation and establishment of the function code 425 referenced in the intercepted uniform resource identifier before directing the client request.

[0049] According to some embodiments, the listening operation may be followed by one or more actions defined by a set of listening rules. For example, the listening rules may instruct a network agent or communication controller (e.g., Envoy) to perform a set of specified operations on the listened uniform resource identifier. For example, the listening rules may instruct Envoy to generate a service call and forward it to the service code specified in the listened uniform resource identifier. In response, Envoy may check the routing rules to identify the local Envoy instance or attached agent associated with the listened service code instance, at which point it will send a service call to the appropriate software component, notifying the client that the request is for the function code associated with the listened uniform resource identifier, thereby calling out the appropriate function code.

[0050] In summary, embodiments of the present technology relate to a system and method for automatically triggering related code segments corresponding to a series of code segments or function codes having a preferred execution order. The automatic triggering action is based on listening to the response generated due to the execution of the previous code segment. Information about the next code segment in the preferred execution order can be obtained by instructing a network proxy such as Envoy to listen to the uniform resource identifier (URI) field of the response packet being forwarded to the client entity. In this way, the network proxy can preemptively generate and instantiate subsequent function code (pointed to by the listened uniform resource identifier) ​​before receiving the corresponding client request. In this way, when a client request for the subsequent function code is received, the code is ready to execute.

[0051] For clarity of explanation, in some cases, the technology may be presented as including separate functional blocks including functional blocks having devices, device components, method steps or routines embodied in software, or a combination of hardware and software.

[0052] In some embodiments, computer-readable storage devices, media, and memories may include cables or wireless signals containing bit streams, etc. However, when referred to, non-transitory computer-readable storage media explicitly exclude media such as energy, carrier signals, electromagnetic waves, and signals themselves.

[0053] The method according to the above example can be implemented using computer executable instructions stored in or otherwise available from a computer readable medium. Such instructions may include, for example, instructions and data that enable or otherwise configure a general-purpose computer, a special-purpose computer, or a special-purpose processing device to perform a specific function or group of functions. The various computer resources used can be accessed through a network. Computer executable instructions can be, for example, binary intermediate format instructions, such as assembly language, firmware, or source code. Examples of computer readable media that can be used to store instructions, information used, and / or information created during the method according to the example include disks or optical disks, flash memory, USB devices equipped with non-volatile memory, network storage devices, etc.

[0054] Devices implementing methods according to these disclosures may include hardware, firmware, and / or software, and may take any of a variety of form factors. Typical examples of such form factors include laptop computers, smart phones, small form factor personal computers, personal digital assistants, etc. The functions described herein may also be embodied in peripheral devices or add-on cards. As a further example, such functions may also be implemented on circuit boards between different chips, or on different processes in a single device.

[0055] Instructions, media for communicating such instructions, computing resources for executing the instructions, and other structures for supporting such computing resources are means for providing the functionality described in these disclosures.

[0056] Although various examples and other information are used to explain various aspects within the scope of the appended claims, limitations to the claims should not be implied based on specific features or arrangements in such examples, as those of ordinary skill in the art will be able to use these claim examples to derive various implementations. In addition, although some subject matter may be described in language specific to examples of structural features and / or method steps, it should be understood that the subject matter defined in the appended claims is not necessarily limited to these described features or actions. For example, such functionality may be distributed or performed differently in components other than those identified herein. In contrast, the described features and steps are disclosed as examples of components of systems and methods within the scope of the appended claims.

Claims

1. A method for automatically instantiating related code segments in a service grid environment, comprising: listening for at least one uniform resource identifier of one or more response packets generated by executing a code segment of a plurality of code segments having a preferred execution sequence, wherein the at least one uniform resource identifier identifies at least a next code segment in the preferred execution sequence; In response to listening to the at least one uniform resource identifier and before receiving a request to instantiate the next code segment, preemptively instantiating the next code segment in the preferred execution sequence, wherein An initial code segment among the plurality of code segments is instantiated in response to a trigger.

2. The method according to claim 1, wherein: The plurality of code segments associated with the preferred execution sequence correspond to a plurality of microtasks involved in executing a macrotask.

3. The method according to claim 1 or 2, wherein: The trigger includes a specified request for the initial code segment.

4. The method according to claim 1 or 2, wherein: The trigger corresponds to a packet of interest or a request for a specific function.

5. The method according to claim 1 or 2, wherein: The trigger corresponds to a scheduling task, a network request, a queue message or a manual call.

6. The method according to claim 1 or 2, wherein: Each code segment corresponds to a service, and the plurality of code segments having the preferred execution sequence corresponds to a service chain.

7. The method according to claim 1 or 2, wherein: Each code segment in the plurality of code segments corresponds to a function in a serverless computing architecture.

8. The method according to claim 7, wherein: The serverless computing architecture is implemented within a service grid.

9. A method for improving the response time of a service grid, comprising: preemptively instantiating a next code segment identified based on listening to at least one uniform resource identifier of one or more response packets generated as a result of executing a previous code segment, wherein the at least one uniform resource identifier identifies at least the next code segment in a preferred execution sequence; and For multiple code segments having the preferred execution sequence, after triggering execution of an initial code segment, the listening and instantiating operations are repeated for each code segment in the preferred execution sequence.

10. The method according to claim 9, wherein: Each code segment corresponds to a service code, and the plurality of code segments having a preferred execution sequence corresponds to a service chain.

11. The method according to claim 10, wherein: The service chain is implemented according to a serverless model.

12. The method according to any one of claims 9 to 11, wherein: The triggering execution of the initial code segment is caused by a specified request for the initial code segment.

13. The method according to any one of claims 9 to 11, wherein: The triggered execution of the initial code segment is caused by the detection of a packet of interest or a request for a specific function.

14. The method according to any one of claims 9 to 11, wherein: The triggering execution of the initial code segment is caused by any one of a scheduling task, a network request, a queue message or a manual call.

15. A non-transitory computer readable medium comprising computer executable instructions stored thereon, which when executed can cause a computer to perform the following operations: listening for at least one uniform resource identifier of one or more response packets generated by executing a code segment from among a plurality of code segments having a preferred execution sequence, wherein, The at least one uniform resource identifier identifies at least a next code segment in the preferred execution sequence; In response to listening to the at least one uniform resource identifier and before receiving a request to instantiate the next code segment, preemptively instantiating the next code segment in the preferred execution sequence, wherein An initial code segment among the plurality of code segments is instantiated in response to a trigger.

16. The non-transitory computer readable medium of claim 15, wherein: Each code segment corresponds to a service code, and the plurality of code segments having the preferred execution sequence corresponds to a service chain.

17. The non-transitory computer readable medium of claim 15 or 16, wherein: The trigger includes any one of the following: a specified request for the initial code segment, detection of a packet of interest, detection of a request for a specific function, a scheduled task, a network request, a queue message, and a manual call.

18. A device for implementing automatic instantiation of related code segments in a service grid environment, comprising: means for listening for at least one uniform resource identifier of one or more response packets generated by executing a code segment of a plurality of code segments having a preferred execution sequence, wherein the at least one uniform resource identifier identifies at least a next code segment in the preferred execution sequence; Means for preemptively instantiating the next code segment in the preferred execution sequence in response to listening to the at least one uniform resource identifier and before receiving a request to instantiate the next code segment, wherein An initial code segment among the plurality of code segments is instantiated in response to a trigger.

19. The apparatus according to claim 18, further comprising: An apparatus for implementing the method according to any one of claims 2 to 14.

20. A computer program product comprising instructions which, when executed by a computer, cause the computer to perform the steps of the method of any one of claims 1 to 14.

Images