Local routing method and system for optimizing throughput of non-perceptual computing workflow of server

Abstract

The invention provides a local routing method and a local routing system for optimizing throughput of a non-perceptual computing workflow of a server. The method comprises the following steps of: distributing a routing function to each function instance by a global controller to support high system throughput; enabling the function instance to automatically obtain addresses of other function instances and perform direct handshake, meanwhile, storing a local routing table in the instance, and dynamically selecting a routing instance for each request; a stable routing mechanism and a panic routing mechanism are designed to guarantee normal execution of the workflow; partial functions of function capacity expansion and contraction are also put down to each instance, and a capacity expansion and contraction mechanism based on a local routing group is constructed. According to the invention, the technical problem that the improvement of the throughput performance of the system is limited due to routing contradiction caused by locking among instances and lack of global controller coordination is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of cloud computing, and in particular to data transmission between functions in a workflow in a server-unaware computing scenario. Background Art

[0002] In recent years, server-aware computing has become an increasingly popular cloud computing development paradigm due to its advantages such as high abstraction of resources and programming, on-demand billing, and dynamic expansion. To implement complex practical applications, users usually orchestrate a series of fine-grained functions into a workflow in the form of a directed acyclic graph, which defines the order of functions and their data dependencies. For example, the existing invention patent application document "Direct Data Transfer Method between Functions for Server-Aware Computing Scenarios" with publication number CN117834709A includes: a gateway used as a front-end API endpoint receives external function requests, performs load balancing, and forwards them to the engine within the node; the engine within the node dispatches the request to the function, and decides whether to establish DTCs with the downstream function based on the QPS; if DTCs cannot be established between functions, IPC and Fabric transmission methods are used for data transmission between functions on the same node and between functions across nodes, respectively; if DTCs can be established between functions, DTC_over_IPC and DTC_over_Fabric transmission methods are used to establish stateful connections between functions on the same node and between functions across nodes, respectively, to achieve direct data transmission between functions. The aforementioned current mainstream server-aware computing platform deploys each function in a separate instance. Due to the stateless characteristics of server-aware computing and the dynamic scaling of instances, the platform does not provide the actual address of the function instance to the user. Therefore, point-to-point direct communication cannot be established between instances whose addresses are not aware of each other, and data transmission between functions can only be completed through third-party routing. Specifically, the result of the upstream function is first received by the global controller through the API provided by the platform. The controller then searches the global routing table that stores the status of all function instances, finds the address of the idle downstream function instance, and finally sends the received data. During the entire transmission process, the data undergoes multiple copies, and the communication delay is significantly increased compared to direct transmission between instances. In addition, the platform has restrictions on the size of a single request, and needs to rely on third-party storage services to access big data. Functions can only pass their actual locations, which further increases communication overhead. Obviously, as the system throughput increases, the burden on the global controller will become higher and higher, and it can easily become a bottleneck for the entire system, ultimately making it difficult to further increase throughput. Therefore, how to improve the communication efficiency of workflows is an important challenge in the field of server-aware computing.

[0003] Most existing work accelerates the transmission of intermediate data between functions by optimizing the efficiency of data forwarding, such as using faster in-memory databases instead of disk databases, or deploying functions on the same node, using efficient inter-process communication technology instead of expensive network communication with protocol stack overhead, etc. Although these solutions have achieved certain improvements in latency, they have not changed the nature of third-party routing, and all data transmission still needs to pass through the global controller. Therefore, the arrival of system bottlenecks is only delayed, not avoided.

[0004] In order to optimize the throughput of server-aware computing workflows, most existing systems focus on optimizing communication methods, using faster databases, or using local communication instead of network communication. These solutions reduce communication delays to a certain extent by accelerating the data forwarding rate, thereby improving system throughput. However, each request in these third-party routing-based solutions still passes through a centralized controller, and system bottlenecks still occur under high throughput. To overcome the limitations of direct connections between functions, some methods first complete the handshake between two instances through a third-party router, and then reuse the established direct connection to transmit subsequent intermediate data. However, the handshake based on third-party routing will bring high overhead to the critical path. At the same time, the static routing strategy used in the reused connection leads to mutual locking between instances, which in turn triggers the expansion of synchronous instances of all functions, resulting in serious waste of resources.

[0005] Some works have also tried to break through the limitation of direct communication between functions through connection reuse schemes. Specifically, this scheme first completes the handshake between function instances through third-party routing, that is, first exchange address information, and then establish a direct connection channel. After the handshake is completed, the function instance can continuously reuse the channel to transmit intermediate data. Although this scheme achieves the best communication efficiency through direct transmission, it will lead to significant handshake overhead and low resource efficiency, which will affect the system throughput. First, the handshake between function instances still relies on third-party routing to complete, and the first intermediate data transmission needs to wait until the handshake is completed before it can start, so the first request will experience a higher end-to-end delay than the original one based on third-party routing. In addition, the upstream function only shakes hands with a single instance of each downstream function and continuously reuses the same direct connection channel to transmit data. This static routing strategy causes the function instances to lock each other, that is, the newly expanded instance can no longer directly connect to the original instance. Therefore, the system can only expand all other functions synchronously and re-handshake between these new instances, resulting in serious resource waste, which in turn limits the improvement of system throughput.

[0006] In summary, the existing technology has technical problems such as inter-instance locking, routing conflicts caused by lack of global controller coordination, and restrictions on improving system throughput performance. Summary of the invention

[0007] The technical problem to be solved by the present invention is: how to solve the technical problems in the prior art of locking between instances, routing conflicts caused by lack of global controller coordination, and limiting the improvement of system throughput performance.

[0008] The present invention adopts the following technical solution to solve the above technical problem: a local routing method for optimizing server-unaware computing workflow throughput includes:

[0009] S1. Use the global controller to delegate the function information of the function routing to each function instance. The function instance obtains the address of the function instance with dependencies based on the function information, establishes a direct connection with the function instance with dependencies, dynamically selects the applicable routing object for each workflow request, and performs capacity expansion operations.

[0010] S2, enabling the function instance to obtain the addresses of other function instances and perform a direct handshake operation, and dynamically selecting a routing instance for each workflow request according to the local routing table in the function instance memory;

[0011] S3. Design a stable and panic routing mechanism so that each function instance can independently make dynamic routing decisions and select the destination instance to which data is sent, wherein the workflow includes: a first function A, a second function B, and a third function C, and the first function A, the second function B, and the third function C are executed using a preset fan-in structure;

[0012] S4. Decentralize some of the functions of function scaling to each instance, build a scaling mechanism based on the local routing group, use the function instance to monitor the arrival frequency and request execution time of workflow requests in real time, coordinate with the centralized controller to decide on the scaling strategy, and design the local routing group as the scaling management unit.

[0013] The present invention realizes direct data transmission supporting direct handshake and dynamic routing through local routing, and supports high system throughput by delegating routing functions from the global controller to each function instance, thereby reducing additional handshake overhead and resource waste. The present invention designs mechanisms such as stable / panic mode and local routing group expansion and contraction to ensure the correct operation and expansion and contraction of workflows under the local routing method, and solves the problem of low system throughput in server-unaware computing workflows.

[0014] Since the global controller is removed from the intermediate data transmission, the present invention also delegates part of the function scaling functions to each instance, and constructs a scaling mechanism based on the local routing group to meet different system throughput requirements.

[0015] In a more specific technical solution, S1 includes:

[0016] S11. Using the system gateway, the workflow request is directly sent to the function instance of the entry function to execute the function code;

[0017] S12. In the current function instance, use the local controller to consult the local storage routing table, select a suitable downstream function instance as an applicable routing object, and directly transmit the intermediate data to the applicable routing object;

[0018] S13: Enable the function instance to monitor the arrival frequency of workflow requests and its own processing capacity in real time, and request the centralized controller to perform capacity expansion when the workflow request is in a queued state.

[0019] In a more specific technical solution, S2 includes:

[0020] S21. Using a centralized controller, send routing table update information to the new function instance;

[0021] S22: The function instance performs a handshake operation with the new function instance to serve as a direct transfer candidate for subsequent workflow requests.

[0022] The present invention supports function instances to obtain addresses of other function instances by themselves and perform direct handshakes. Meanwhile, a local routing table is stored in the instance memory, and a routing instance can be dynamically selected for each request to avoid locking between instances.

[0023] The present invention delegates the routing capability to each function instance, thereby realizing direct handshake and dynamic routing capabilities of the instance. While realizing high-speed direct data transmission between instances, the present invention avoids high handshake overhead and ensures fine-grained scaling capabilities at the function level, and can achieve the highest system throughput with the same resources.

[0024] In a more specific technical solution, in S22, for function instances deployed on different nodes, a TCP connection is established between the function instances based on the IP addresses stored in the local routing table, and direct data transmission is completed through a preset network connection method, wherein the preset network connection method includes: RDMA hardware-accelerated cross-machine communication, and shared memory and unilateral RDMA-accelerated large data transmission.

[0025] In more specific technical solutions, S3 includes:

[0026] S31, when the first function instance C-1 of the third function C is started, the address of the first function instance C-1 is notified to the first function instance and the second function instance by using the global controller, and a handshake is performed to establish a direct connection;

[0027] S32. When the function instance is in a default stable state, a stable mode is adopted, and a differential routing algorithm is used to process a fan-in structure and a non-fan-in structure;

[0028] S33. When the function instance is expanded, switch to panic mode and perform rerouting.

[0029] In order to solve the routing contradiction problem that may be caused by the lack of global controller coordination, the present invention designs two routing mechanisms, stable and panic, to ensure the normal execution of the workflow.

[0030] In a more specific technical solution, S32 also includes:

[0031] S321. For a non-fan-in structure, a polling algorithm is used in the upstream function instance, and each downstream function instance takes turns as a routing target;

[0032] S322. For the fan-in structure, a consistent hashing algorithm is used, with the ID of the workflow request as the key value, and the hash value of each workload request is mapped in the ring structure formed by the local routing table to determine the routing result.

[0033] In a more specific technical solution, S33 includes:

[0034] S331, when the function instance is notified that it needs to update the local routing table, it switches from stable mode to panic mode;

[0035] S332, assuming that the critical path data arrives at the latest, when the critical path data of the function instance has not all arrived, determining that a routing conflict occurs;

[0036] S333, causing the current function instance to send a rerouting notification to the function instance of the data missing function;

[0037] S334. When the upstream function instance receives the rerouting notification, if the upstream function instance has the required data locally, the upstream function instance resends the required data to the function instance that sent the rerouting notification, and requests the function instance to delete the required data.

[0038] S335. When it is determined that all workflow requests sent during the routing table update are completely aggregated in the downstream function instance, exit the panic mode.

[0039] The present invention supports dynamic routing for each request, and designs stable and panic modes to ensure the normal operation of the workflow. The stable mode ensures load balancing of downstream instances and minimizes the sending of routing conflicts, while the panic mode monitors the occurrence of routing conflicts and requires upstream instances to reroute when necessary.

[0040] In a more specific technical solution, in S334, the function instance is made to continuously check whether subsequent workflow requests meet the preset conditions, and when the required data is generated, the re-routed data is routed to the corresponding downstream function instance; when the downstream function instance receives the re-routed data, it notifies the remaining upstream instances to stop checking.

[0041] In a more specific technical solution, in S4, the local routing group includes: a function instance with a direct connection, and each local routing group also includes: a workflow function; and the function instance is preferentially expanded or reduced in the local routing group.

[0042] In view of the lack of real-time monitoring of global instance status in local routing scenarios, the present invention designs an instance scaling mechanism based on local routing groups, where each function instance is responsible for monitoring its own request arrival frequency and execution time, and collaborates with the centralized controller to complete the scaling work when necessary. The present invention only requires users to submit a workflow directed acyclic graph containing dependencies between functions, and can automatically deploy functions to the server's non-aware computing platform and perform various tasks without user participation, effectively reducing labor costs.

[0043] In a more specific technical solution, the local routing system for server-aware computing workflow throughput optimization includes:

[0044] The routing function decentralization module is used to use the global controller to decentralize the function information of the function routing to each function instance. The function instance obtains the address of the function instance with dependency according to the function information, establishes a direct connection with the function instance with dependency, dynamically selects the applicable routing object for each workflow request, and performs capacity expansion operations;

[0045] A routing instance dynamic selection module is used to enable a function instance to obtain the addresses of other function instances and perform a direct handshake operation. A routing instance is dynamically selected for each workflow request according to a local routing table in the memory of the function instance. The routing instance dynamic selection module is connected to the routing function decentralization module.

[0046] A stable panic routing mechanism module is used to set stable and panic routing mechanisms, so that each function instance independently makes dynamic routing decisions and selects a destination instance for data transmission, wherein the workflow includes: a first function A, a second function B and a third function C, and the first function A, the second function B and the third function C are executed using a preset fan-in structure, and the stable panic routing mechanism module is connected to the routing instance dynamic selection module;

[0047] The scaling module is used to delegate some of the functions of function scaling to each instance, build a scaling mechanism based on the local routing group, use the function instance to monitor the arrival frequency and request execution time of workflow requests in real time, coordinate with the centralized controller to decide on the scaling strategy, and design the local routing group as the scaling management unit. The scaling module is connected to the stable panic routing mechanism module.

[0048] Compared with the prior art, the present invention has the following advantages:

[0049] The present invention realizes direct data transmission supporting direct handshake and dynamic routing through local routing, and supports high system throughput by delegating routing functions from the global controller to each function instance, thereby reducing additional handshake overhead and resource waste. The present invention designs mechanisms such as stable / panic mode and local routing group expansion and contraction to ensure the correct operation and expansion and contraction of workflows under the local routing method, and solves the problem of low system throughput in server-unaware computing workflows.

[0050] Since the global controller is removed from the intermediate data transmission, the present invention also delegates part of the function scaling functions to each instance, and constructs a scaling mechanism based on the local routing group to meet different system throughput requirements.

[0051] The present invention supports function instances to obtain addresses of other function instances by themselves and perform direct handshakes. Meanwhile, a local routing table is stored in the instance memory, and a routing instance can be dynamically selected for each request to avoid locking between instances.

[0052] The present invention delegates the routing capability to each function instance, thereby realizing direct handshake and dynamic routing capabilities of the instance. While realizing high-speed direct data transmission between instances, the present invention avoids high handshake overhead and ensures fine-grained scaling capabilities at the function level, and can achieve the highest system throughput with the same resources.

[0053] In order to solve the routing contradiction problem that may be caused by the lack of global controller coordination, the present invention designs two routing mechanisms, stable and panic, to ensure the normal execution of the workflow.

[0054] The present invention supports dynamic routing for each request, and designs stable and panic modes to ensure the normal operation of the workflow. The stable mode ensures load balancing of downstream instances and minimizes the sending of routing conflicts, while the panic mode monitors the occurrence of routing conflicts and requires upstream instances to reroute when necessary.

[0055] In view of the lack of real-time monitoring of global instance status in local routing scenarios, the present invention designs an instance scaling mechanism based on local routing groups, where each function instance is responsible for monitoring its own request arrival frequency and execution time, and collaborates with the centralized controller to complete the scaling work when necessary. The present invention only requires users to submit a workflow directed acyclic graph containing dependencies between functions, and can automatically deploy functions to the server's non-aware computing platform and perform various tasks without user participation, effectively reducing labor costs.

[0056] The present invention solves the technical problems existing in the prior art of inter-instance locking, routing conflicts caused by lack of global controller coordination, and restrictions on improving system throughput performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1A schematic diagram of basic steps of a local routing method for optimizing server-aware computing workflow throughput according to Embodiment 1 of the present invention;

[0058] Figure 2 A schematic diagram of data flow processing steps of a local routing method for optimizing server-aware computing workflow throughput according to Embodiment 1 of the present invention;

[0059] Figure 3 This is an example diagram of routing contradictions in Embodiment 1 of the present invention;

[0060] Figure 4 This is an example diagram of the panic mode of Embodiment 1 of the present invention;

[0061] Figure 5a This is a schematic diagram of an example of a local routing group expansion function according to Embodiment 1 of the present invention;

[0062] Figure 5b This is a schematic diagram of the principle of expanding the local routing group to avoid full connection according to Embodiment 1 of the present invention.

[0063] Figure 6 This is a schematic diagram of data transmission efficiency according to Embodiment 1 of the present invention;

[0064] Figure 7 This is a schematic diagram of system throughput of Example 1 of the present invention;

[0065] Figure 8a A schematic diagram of system overhead of different modules in Embodiment 1 of the present invention;

[0066] Figure 8b This is a schematic diagram of panic routing mode overhead according to Embodiment 1 of the present invention. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] Example 1

[0069] like Figure 1 As shown in FIG. 1 , the local routing method for optimizing the throughput of server-aware computing workflows includes the following basic steps:

[0070] S1, supports high system throughput by delegating routing functions from the global controller to each function instance;

[0071] In this embodiment, the function routing function is decentralized from the global controller to each function instance. The function instance can directly obtain the address of the function instance with which it has a dependency and establish a direct connection, and dynamically select the appropriate routing object for each request.

[0072] like Figure 1 As shown, in this embodiment, in the overall architecture of the local routing system for server-aware computing workflow throughput optimization, the workflow request will be sent directly to an instance of the entry function by the system gateway. After the function code is executed, the local controller on the edge of the instance will consult the locally stored routing table, and then select a suitable downstream function instance to directly transmit the intermediate data; the function instance will monitor the frequency of request arrival and its own processing capacity in real time, and once it is found that request queuing will occur, it will request the centralized controller to expand capacity;

[0073] In this embodiment, multiple direct connection modes are supported according to the different locations of the function instances, including but not limited to: direct connection of in-thread communication within the same node and TCP and RDMA direct connection across nodes. In this embodiment, for functions deployed in the same node, the routing table stores the Linux named pipe path, and the function instance can realize direct data transmission based on inter-process communication by reading and writing the pipe.

[0074] S2, enabling the function instance to obtain the address of other function instances and perform direct handshakes. At the same time, a local routing table is stored in the instance to dynamically select a routing instance for each request.

[0075] In this embodiment, after the new instance is expanded, the centralized controller will send the updated information of the routing table to all the function instances involved, and then the function instance can shake hands with the new instance and use it as a direct transmission alternative for subsequent requests. Note that the centralized controller in the present invention still exists, but is only responsible for synchronizing the routing tables between the instances, starting new function instances, etc., and is no longer involved in handshakes and inter-function communication.

[0076] In this embodiment, for function instances deployed on different nodes, the IP addresses are stored in the routing table, a TCP connection is established between the function instances, and data is directly transmitted through the network. In this embodiment, cross-machine communication is also supported through RDMA hardware acceleration. The function instances first transmit the GID of the network card, the queue QP for sending and receiving requests, and the MR for accessing actual data, and then shake hands to establish a direct channel. In addition, the present invention supports the acceleration of large data transmission within and between nodes through shared memory and unilateral RDMA.

[0077] S3,designed two routing mechanisms, stable and panic, to ensure the normal execution of the workflow;

[0078] In the stable and panic routing mechanisms of this embodiment, each function instance independently makes dynamic routing decisions and selects the destination instance to which data is sent. This asynchronous communication paradigm may lead to routing conflicts, that is, the results of the same request processed by parallel functions in the fan-in mode may be routed to different downstream instances, which in turn causes the downstream function to be unable to obtain complete data to continue execution. Therefore, in this embodiment, stable and panic modes are designed to avoid routing conflicts as much as possible and to perform rerouting when routing conflicts occur.

[0079] like Figure 3 As shown, in this embodiment, the workflow consists of three functions A, B, and C, among which there is a fan-in structure, that is, function C needs to obtain the results of A and B before it can start to execute. After the first instance C-1 of function C is started, the controller will inform the instances of function A and B of the address, and then shake hands to establish a direct connection. At this time, there is only C-1 in the local routing table of the instances of A and B, so the execution results of the requests of both can definitely be routed to C-1, and the workflow can be executed normally. However, after the system expands the capacity of instance C-2, the time when the instances of A and B receive the notification of updating the routing table may be inconsistent, and then the two may select the target instance according to different routing tables. As shown in the figure, the instance of A needs to choose between C-1 and C-2 when making routing decisions, and finally routes the data to the newly expanded C-2, while the instance of B still only knows the existence of C-1 when making decisions, so the data is still routed to C-1. Therefore, the results of the same workflow request are held by C-1 and C-2 respectively, and neither can continue to execute the request. The present invention refers to this problem as a routing contradiction.

[0080] By default in this embodiment, the function instance is in a stable state, and two different routing algorithms are used to cope with different workflow structures. For a non-fan-in structure, that is, a scenario in which a downstream instance only needs to obtain one upstream data to start execution, since no routing conflicts will occur, it is only necessary to ensure the load balancing of the downstream instance. Therefore, the upstream instance uses a polling algorithm, and each downstream instance takes turns as a routing target. For fan-in structures where routing conflicts may occur, this embodiment uses a consistent hashing algorithm, with the requested ID as the key value, and determines the routing result based on the mapping of its hash value on the ring structure formed by the local routing table. Through the consistent hashing algorithm, not only can the function instances storing the same routing table be guaranteed to have the same routing results for the same request, but the ring structure also reduces the offset of the hash value mapping when the routing table is updated, thereby reducing the probability of routing conflicts when the routing tables between instances are not completely consistent.

[0081] In this embodiment, when the function instance is expanded, the local routing table of each instance will be updated successively. At this time, the routing tables between the instances will be temporarily different, which will cause the consistent hashing algorithm in the stable state to no longer guarantee that all routing results are the same. Therefore, the present invention designs a panic mode to reroute when routing conflicts occur.

[0082] like Figure 4 As shown, in this embodiment, based on Figure 3 The example of illustrates how panic mode resolves routing conflicts. In this example, function A is on the critical path. When instance C-2 receives data from A, it checks whether it has received data from B. If not, it considers that a routing conflict may have occurred, and then C-2 gives feedback to the instance of B. After receiving the feedback, B reroutes the request to C-2 and notifies C-1 to delete the originally transmitted data. At this point, C-2 obtains complete data and can continue to execute, and the routing conflict is resolved.

[0083] See also Figure 4 As shown on the right, it is possible that the routing conflict has not yet occurred. At this time, after receiving the feedback, B will continuously check whether the generated data meets the requirements. If yes, it will be sent directly to C-2.

[0084] First, when the instance is notified that it needs to update the local routing table, it switches from stable mode to panic mode. Then, the present invention assumes that the data of the critical path should arrive last. If there is still data that has not arrived after the instance receives the critical path data, it is considered that a routing conflict may have occurred, and then the instance will send a rerouting notification to all instances of the function where the data is missing. In this embodiment, after the upstream instance receives the rerouting notification, if the required data exists locally, it will be resent to the instance of the notification source, and the originally routed instance will be required to delete the data. In this embodiment, considering that the required data may not have been generated, the instance will continuously check whether subsequent requests meet the conditions. If the required data is generated subsequently, it will be directly routed to the corresponding downstream instance. To avoid redundant inspection overhead, the downstream function will notify other upstream instances to stop checking after receiving the rerouted data.

[0085] In this embodiment, in order to ensure that all requests can be executed correctly, the panic mode is exited only after it is determined that all requests sent during the update of the routing table are fully aggregated in the downstream instance. Specifically, the centralized controller informs each function instance after determining that all local routing tables have been updated that the routed requests may have routing conflicts during the period from entering the panic mode to receiving the notification, and the function instance will record these requests. In panic mode, after receiving all upstream data for the same request, the downstream instance will notify all upstream functions that the request can be executed correctly. When all recorded requests that may send routing conflicts are confirmed to be ok, the function instance will notify the centralized controller, and then notify all instances to exit panic mode.

[0086] In this embodiment, in order to achieve rerouting in panic mode, the function instance will temporarily save the data locally for retransmission after completing the direct data transmission. After the workflow request is executed, it will continuously notify the upstream function instance to delete the result of the request starting from the most downstream function instance.

[0087] S4. Decentralize some of the functions of function scaling to each instance and build a scaling mechanism based on the local routing group.

[0088] In the local routing group-based scaling mechanism of this embodiment, the function instance monitors the frequency of request arrival and the execution time of the request in real time, and then collaborates with the centralized controller to decide on the scaling strategy, and designs the local routing group as the unit of scaling management.

[0089] In this embodiment, the set of function instances with direct connections is called a local routing group. Each local routing group contains all functions of the workflow, and each workflow may be supported by multiple local routing groups. Different routing groups are not aware of each other. Function instances are scaled up or down in the local routing group first, and local routing groups are scaled up or down when necessary.

[0090] In this embodiment, the function instance monitors the arrival frequency λ of requests per unit time and the average execution time t of requests in real time. If it is found that This means that the load of the current instance has exceeded its own processing capacity, and the function needs to expand the capacity of a new instance. Then, the function instance sends a capacity expansion request containing λ and t data to the centralized controller, and the controller calculates the number of instances required to support the current system throughput. And call the relevant interface to create a new instance. Specifically, the centralized controller may receive expansion requests from different instances of the same function. At this time, the controller will update the total arrival frequency λ and the average execution time t and recalculate α, and then expand the number of instances required. In addition, in order to avoid instance failure due to excessive request concurrency, a threshold is set for λ. When the request frequency of a single instance exceeds the threshold, the instance expansion will also be triggered. By default, the centralized controller will add the expanded instance to the local routing group where the expansion request comes from. However, there may be hundreds or thousands of function instances in the workflow at the same time. Adding all instances to the same routing group and establishing direct connections can easily cause scalability problems. Therefore, expansion conditions are set for the local routing group. When all functions in the routing group need to be expanded, that is, when the last function also initiates an expansion request, the centralized controller will create a new routing group and add the newly expanded instance to it.

[0091] In this embodiment, to achieve the scaling function, the centralized controller will set a timer of, for example, 10 minutes, requiring each function instance to report λ ​​and t data, and then calculate the total request arrival frequency ∑λ and the average execution time according to the collected Calculate the number of instances required by the system, and finally determine the number of instances that need to be recycled. In addition, if the function instance detects that no requests have arrived for a period of time, it will directly notify the controller to remove it. When all instances of a local routing group are released, the routing group will be removed. After determining the function instance to be recycled, the centralized scheduler will first notify the upstream instance directly connected to it to stop sending data to it, and then wait for the instance to confirm that all requests it has issued can be correctly executed on the downstream instance, and finally release all direct connections and recycle the instance.

[0092] In this embodiment, it does not rely on any specific scheduling system, and only updates the local routing table and establishes a direct connection according to the result of system scheduling, so it can be seamlessly adapted to various scheduling frameworks.

[0093] As shown in Figure 5(a), in the capacity expansion operation based on the local routing group in this embodiment, the workflow consists of two functions A and B. Function instance A-1 detects that the current request arrival rate is 6 requests per second, and the time it takes to execute each request is 0.5 seconds, which satisfies Therefore, the expansion mechanism is triggered. According to λ and t, the controller confirms that A needs 3 instances to meet the current request frequency, so it expands the two instances A-2 and A-3. As for function B, a single instance can process 6 requests per second without expansion. Since only function A is currently being expanded, the present invention adds the newly expanded A-2 and A-3 to the current local routing group and establishes a direct connection with B-1.

[0094] As shown in Figure 5(b), in this embodiment, as the request arrival rate further increases to 8 requests per second, function A expands a new instance A-4, and then instance B-1 also encounters a processing bottleneck, triggering the expansion mechanism. In order to avoid full connection between multiple instances of A and B, the present invention expands a new local routing group, and adds B-2 and A-4 to it. The two instances shake hands and transmit directly. The workflow requests are evenly distributed by the gateway to each instance of the entry function A, and the routing groups are not aware of each other. Note that A-4 in the figure has originally joined routing group 2 and is connected to B-1. In order to avoid switching routing groups to further complicate the routing contradiction, the present invention will start a new A-4 instance to join local routing group 2 and destroy the original A-4 in routing group 1.

[0095] Compared with the existing work, the present invention has significant effects. Figure 6 As shown, in this embodiment, the efficiency of different methods for transmitting data of various sizes between two functions is tested. The data direct transmission of the present invention is 1.4-15.6 times faster than the third-party forwarding, and the local and RDMA direct transmission are 1.2-3.5 times and 1.1-2.1 times faster than the TCP direct transmission respectively. Figure 7 As shown in the figure, in this embodiment, the end-to-end delay of each system in three workflows is tested. In terms of throughput, the present invention improves 5.7-93.8 times compared with the third-party routing solution, and improves the throughput by 1.8-5.4 times compared with the existing connection multiplexing solution while maintaining the original performance. Figure 8a As shown, in this embodiment, based on connection multiplexing, the modules designed by the present invention are added in sequence, and the workflow delay is tested to measure the module overhead. Dynamic routing and real-time monitoring of requests only cause an additional overhead of about 20 microseconds. Figure 8b As shown in Figure 2, workflows with different degrees of parallelism are selected to compare the delays in stable and panic routing modes. The results show that the panic routing mode only causes the 99th percentile delay of the workflow to increase by 1.9-11.1% in a short period of time.

[0096] In summary, the present invention realizes direct data transmission supporting direct handshake and dynamic routing through local routing, and reduces extra handshake overhead and resource waste by delegating routing functions from the global controller to each function instance to support high system throughput. The present invention designs mechanisms such as stable / panic mode and local routing group expansion and contraction to ensure the correct operation and expansion and contraction of workflows under the local routing method, and solves the problem of low system throughput in server-unaware computing workflows.

[0097] Since the global controller is removed from the intermediate data transmission, the present invention also delegates part of the function scaling functions to each instance, and constructs a scaling mechanism based on the local routing group to meet different system throughput requirements.

[0098] The present invention supports function instances to obtain addresses of other function instances by themselves and perform direct handshakes. Meanwhile, a local routing table is stored in the instance memory, and a routing instance can be dynamically selected for each request to avoid locking between instances.

[0099] The present invention delegates the routing capability to each function instance, thereby realizing direct handshake and dynamic routing capabilities of the instance. While realizing high-speed direct data transmission between instances, the present invention avoids high handshake overhead and ensures fine-grained scaling capabilities at the function level, and can achieve the highest system throughput with the same resources.

[0100] In order to solve the routing contradiction problem that may be caused by the lack of global controller coordination, the present invention designs two routing mechanisms, stable and panic, to ensure the normal execution of the workflow.

[0101] The present invention supports dynamic routing for each request, and designs stable and panic modes to ensure the normal operation of the workflow. The stable mode ensures load balancing of downstream instances and minimizes the sending of routing conflicts, while the panic mode monitors the occurrence of routing conflicts and requires upstream instances to reroute when necessary.

[0102] In view of the lack of real-time monitoring of global instance status in local routing scenarios, the present invention designs an instance scaling mechanism based on local routing groups, where each function instance is responsible for monitoring its own request arrival frequency and execution time, and collaborates with the centralized controller to complete the scaling work when necessary. The present invention only requires users to submit a workflow directed acyclic graph containing dependencies between functions, and can automatically deploy functions to the server's non-aware computing platform and perform various tasks without user participation, effectively reducing labor costs.

[0103] The present invention solves the technical problems existing in the prior art of inter-instance locking, routing conflicts caused by lack of global controller coordination, and restrictions on improving system throughput performance.

[0104] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A local routing method for optimizing server-aware computing workflow throughput, characterized in that: The method comprises: S1. Use the global controller to delegate the function information of the function routing to each function instance. The function instance obtains the address of the function instance with dependency according to the function information, establishes a direct connection with the function instance with dependency, dynamically selects an applicable routing object for each workflow request, and performs expansion operation. S2, enabling the function instance to obtain addresses of other function instances and perform a direct handshake operation, and dynamically selecting a routing instance for each workflow request according to a local routing table in the memory of the function instance; S3, design a stable and panic routing mechanism, so that each function instance independently makes a dynamic routing decision and selects a destination instance for data transmission, wherein the workflow includes: a first function A, a second function B and a third function C, and the first function A, the second function B and the third function C are executed by using a preset fan-in structure; S4. Part of the functions of function scaling are also delegated to each instance, and a scaling mechanism based on the local routing group is constructed, so that the function instance can be used to monitor the arrival frequency and request execution time of the workflow request in real time, and the scaling strategy can be decided in collaboration with the centralized controller, and the local routing group can be designed as the scaling management unit.

2. The local routing method for optimizing server-aware computing workflow throughput according to claim 1, characterized in that: The S1 includes: S11, using the system gateway, directly sending the workflow request to the function instance of the entry function to execute the function code; S12, in the current function instance, using the local controller to consult the local storage routing table, selecting a suitable downstream function instance as the applicable routing object, and directly transmitting the intermediate data to the applicable routing object; S13, enabling the function instance to monitor the arrival frequency of the workflow request and its own processing capability in real time, and requesting a centralized controller to perform the capacity expansion operation when the workflow request is in a queued state.

3. The local routing method for optimizing server-aware computing workflow throughput according to claim 1, characterized in that: The S2 includes: S21. Using a centralized controller, send routing table update information to the new function instance; S22: The function instance performs a handshake operation with the new function instance to serve as a candidate for direct transmission of the subsequent workflow request.

4. The local routing method for optimizing server-aware computing workflow throughput according to claim 3, characterized in that: In S22, for the function instances deployed on different nodes, a TCP connection is established between the function instances according to the IP addresses stored in the local routing table, and direct data transmission is completed through a preset network connection method, wherein the preset network connection method includes: RDMA hardware-accelerated cross-machine communication, and shared memory and unilateral RDMA-accelerated large data transmission.

5. The local routing method for optimizing server-aware computing workflow throughput according to claim 1, characterized in that: The S3 includes: S31, when the first function instance C-1 of the third function C is started, the global controller is used to inform the first function instance and the second function instance of the address of the first function instance C-1, and a handshake is performed to establish a direct connection; S32, when the function instance is in a default stable state, adopting a stable mode, and using a differential routing algorithm to process a fan-in structure and a non-fan-in structure; S33: When the function instance is expanded, switch to panic mode and perform rerouting.

6. The local routing method for optimizing server-aware computing workflow throughput according to claim 5, characterized in that: The S32 further includes: S321: For the non-fan-in structure, the function instance at the upstream uses a polling algorithm, and each of the function instances at the downstream takes turns as a routing target; S322. For the fan-in structure, a consistent hashing algorithm is used, with the ID of the workflow request as the key value, and according to the hash value of each workload request, a ring structure mapping is formed in the local routing table to determine the routing result.

7. The local routing method for optimizing server-aware computing workflow throughput according to claim 5, characterized in that: The S33 includes: S331, when the function instance is notified that the local routing table needs to be updated, the function instance switches from the stable mode to the panic mode; S332, assuming that the critical path data arrives at the latest, when the critical path data of the function instance has not all arrived, determining that a routing conflict occurs; S333, causing the current function instance to send a rerouting notification to the function instance of the data missing function; S334: When the function instance at the upstream receives the rerouting notification, if the requested data exists locally in the function instance at the upstream, the requested data is resent to the function instance that sent the rerouting notification, and the function instance is requested to delete the requested data; S335: When it is determined that all the workflow requests sent during the updating of the routing table are completely aggregated in the downstream function instance, exit the panic mode.

8. The local routing method for optimizing server-aware computing workflow throughput according to claim 7, characterized in that: In S334, the function instance is made to continuously check whether the subsequent workflow requests meet the preset conditions, and when the required data is generated, the re-routing data is routed to the corresponding downstream function instance; when the downstream function instance receives the re-routing data, it notifies the remaining upstream instances to stop checking.

9. The local routing method for optimizing server-aware computing workflow throughput according to claim 1, characterized in that: In S4, the local routing group includes: the function instance with direct connection, and each local routing group also includes: a workflow function; and the function instance is preferentially expanded or reduced in the local routing group.

10. A local routing system for optimizing server-aware computing workflow throughput, characterized in that: The system comprises: The routing function decentralization module is used to use the global controller to decentralize the function information of the function routing to each function instance. The function instance obtains the address of the function instance with dependency relationship according to the function information, establishes a direct connection with the function instance with dependency relationship, dynamically selects the applicable routing object for each workflow request, and performs capacity expansion operation; A routing instance dynamic selection module, used to enable the function instance to obtain the addresses of the remaining function instances and perform a direct handshake operation, and dynamically select a routing instance for each workflow request according to a local routing table in the memory of the function instance, the routing instance dynamic selection module being connected to the routing function decentralization module; A stable panic routing mechanism module is used to set stable and panic routing mechanisms, so that each of the function instances independently makes dynamic routing decisions and selects a destination instance for data transmission, wherein the workflow includes: a first function A, a second function B and a third function C, and the first function A, the second function B and the third function C are executed using a preset fan-in structure, and the stable panic routing mechanism module is connected to the routing instance dynamic selection module; The capacity expansion and contraction module is used to delegate part of the function expansion and contraction functions to each instance, and to build a capacity expansion and contraction mechanism based on the local routing group, so as to use the function instance to monitor the arrival frequency and request execution time of the workflow request in real time, coordinate with the centralized controller to decide the capacity expansion and contraction strategy, and design the local routing group as the capacity expansion and contraction management unit. The capacity expansion and contraction module is connected to the stable panic routing mechanism module.

Images