Local routing method and system for server-unaware computing workflow throughput optimization

Abstract

The application provides a local routing method and system for server non-aware computing workflow throughput optimization. The method comprises: supporting high system throughput by decentralizing routing functions from a global controller to each function instance; enabling the function instance to obtain the address of other function instances and directly handshake, and having a local routing table in the instance memory to dynamically select a routing instance for each request; designing stable and panic routing mechanisms to ensure normal execution of the workflow; and decentralizing part of the function expansion and contraction function to each instance to build an expansion and contraction mechanism based on a local routing group. The application solves the technical problems of instance locking, routing conflicts caused by the lack of global controller coordination, and the limitation of system throughput performance improvement.

Description

Technical Field

[0001] This invention relates to the field of cloud computing technology, specifically to data transmission between functions in a workflow under server-insensitive computing scenarios. Background Technology

[0002] In recent years, server-insensitive computing has become an increasingly popular cloud computing development paradigm due to its advantages such as high abstraction of resources and programming, pay-as-you-go billing, and dynamic scaling. To implement complex practical applications, users typically orchestrate a series of fine-grained functions into workflows in the form of directed acyclic graphs, defining the order of the functions and their data dependencies. For example, the existing invention patent application document CN117834709A, entitled "A Method for Direct Data Transfer Between Functions in Server-Insensitive Computing Scenarios," describes a method that includes: a gateway acting as a front-end API endpoint receiving external function requests, performing load balancing, and forwarding them to the engine within the node; the engine within the node assigning requests to functions and determining whether to establish Direct Communication Terminals (DTCs) with downstream functions based on the QPS; if DTCs cannot be established between functions, data is transmitted between functions within the same node and between functions across nodes using IPC and Fabric transmission methods, respectively; if DTCs can be established between functions, stateful connections are established between functions within the same node and between functions across nodes using DTC_over_IPC and DTC_over_Fabric transmission methods, respectively, to achieve direct data transfer between functions. Currently, mainstream server-sensitive computing platforms deploy each function in a separate instance. Due to the stateless nature of server-sensitive computing and the dynamic scaling of instances, the platform does not provide users with the actual addresses of function instances. Therefore, instances with mutually unaware addresses cannot establish direct point-to-point communication and can only complete data transfer between functions through third-party routing. Specifically, the results of the upstream function are first received by the global controller via the platform's API. The controller then looks up the global routing table, which stores the states of all function instances, to find the address of an available downstream function instance, and finally sends the received data there. Throughout this transmission process, data undergoes multiple copies, significantly increasing communication latency compared to direct transmission between instances. Furthermore, the platform has limitations on the size of individual requests, requiring reliance on third-party storage services to access large datasets. Functions can only exchange their actual locations, further increasing communication overhead. Clearly, as system throughput increases, the global controller's burden grows heavier, easily becoming a bottleneck and ultimately hindering further throughput improvements. Therefore, improving workflow communication efficiency is a significant challenge in the field of server-insensitive computing.

[0003] Existing work mostly accelerates data transfer between functions by optimizing data forwarding efficiency. For example, this involves using faster in-memory databases instead of disk-based databases, deploying functions on the same node, or replacing expensive network communication with efficient inter-process communication techniques. While these solutions improve latency to some extent, they don't change the fundamental nature of third-party routing; all data transmission still needs to pass through the global controller. Therefore, the system bottleneck is only postponed, not avoided.

[0004] To optimize the throughput of server-insensitive computing workflows, most existing systems focus on optimizing communication methods, using faster databases, or replacing network communication with local communication. These solutions reduce communication latency to some extent by accelerating data forwarding rates, thus improving system throughput. However, each request in these third-party routing-based solutions still passes through a centralized controller, and system bottlenecks still appear under high throughput. To overcome the limitations of direct connections between functions, some methods first complete the handshake between two instances through a third-party route, and then reuse the established direct connection to transmit subsequent intermediate data. However, handshakes based on third-party routes incur high overhead on the critical path, and the static routing strategy used for reusing connections leads to mutual locking between instances, which in turn triggers synchronous instance expansion for all functions, resulting in serious resource waste.

[0005] Some works have attempted to overcome the limitations of direct communication between functions by using connection reuse. Specifically, this scheme first completes the handshake between function instances through a third-party route, exchanging address information before establishing a direct connection channel. After the handshake is complete, the function instances can continuously reuse this channel to transmit intermediate data. Although this scheme achieves optimal communication efficiency through direct transmission, it leads to significant handshake overhead and low resource efficiency, thus affecting system throughput. First, the handshake between function instances still relies on a third-party route, and the first intermediate data transmission waits for the handshake to complete before starting, so the first request experiences a higher end-to-end latency than the original third-party route-based approach. Furthermore, the upstream function only handshakes with a single instance of each downstream function and continuously reuses the same direct connection channel to transmit data. This static routing strategy causes mutual locking between function instances; newly expanded instances cannot directly connect to existing instances. Therefore, the system can only simultaneously expand all other functions and re-handshake between these new instances, resulting in severe resource waste and limiting the improvement of system throughput.

[0006] In summary, existing technologies suffer from technical problems such as inter-instance locking, routing conflicts caused by the lack of a global controller for coordination, and limitations on improving system throughput performance. Summary of the Invention

[0007] The technical problem to be solved by this invention is: how to solve the technical problems of inter-instance locking, routing contradictions caused by lack of global controller coordination, and limitations on the improvement of system throughput performance in the prior art.

[0008] This invention solves the above-mentioned technical problems by employing the following technical solution: a local routing method for optimizing server-insensitive computing workflow throughput includes:

[0009] S1. Using the global controller, the function routing information is distributed to each function instance. The function instance obtains the address of the function instance with which it has a dependency relationship based on the function information, establishes a direct transmission connection with the function instance with which it has a dependency relationship, dynamically selects the appropriate routing object for each workflow request, and performs expansion operation.

[0010] S2. Enables the function instance to obtain the addresses of other function instances and perform a direct handshake operation. Based on the local routing table in the function instance's memory, dynamically selects a routing instance for each workflow request.

[0011] S3. Design stable and panic routing mechanisms to enable each function instance to make independent dynamic routing decisions and select the destination instance for data transmission. The workflow includes: first function A, second function B, and third function C, which are executed using a pre-defined fan-in structure.

[0012] S4. Decentralize some of the function scaling functionality to each instance, build a scaling mechanism based on local routing groups, and use function instances to monitor the arrival frequency and execution time of workflow requests in real time, collaborate with the centralized controller to make scaling decisions, and design local routing groups as the scaling management unit.

[0013] This invention enables direct data transmission supporting both direct handshakes and dynamic routing through local routing. By delegating routing functionality from the global controller to each function instance, it supports high system throughput and reduces additional handshake overhead and resource waste. This invention incorporates mechanisms such as stable / panic modes and local routing group scaling to ensure the correct operation and scaling of workflows under the local routing method, thus solving the problem of low system throughput in server-insensitive computing workflows.

[0014] Since the global controller is removed from intermediate data transmission, this invention also decentralizes some of the functions for scaling up and down to each instance, constructing a scaling up and down mechanism based on local routing groups to meet different system throughput needs.

[0015] In a more specific technical solution, S1 includes:

[0016] S11. Using the system gateway, the workflow request is sent directly to the function instance of the entry function to execute the function code;

[0017] S12. In the current function instance, use the local controller to look up the local storage routing table, select the appropriate downstream function instance as the applicable routing object, and directly transmit the intermediate data to the applicable routing object.

[0018] S13 enables the function instance to monitor the arrival frequency of workflow requests and its own processing capacity in real time, and requests the centralized controller to perform expansion operations when workflow requests are in a queue.

[0019] In a more specific technical solution, S2 includes:

[0020] S21. Using the centralized controller, send routing table update information to the new function instance;

[0021] S22. The function instance and the new function instance perform a handshake operation to provide a direct transmission alternative for subsequent workflow requests.

[0022] This invention supports function instances to obtain the addresses of other function instances and perform direct handshakes. At the same time, it has a local routing table in the instance memory, which can dynamically select the routing instance for each request and avoid locking between instances.

[0023] This invention decentralizes routing capabilities to each function instance, thereby enabling direct handshakes and dynamic routing between instances. While achieving high-speed direct data transmission between instances, this invention avoids high handshake overhead and ensures fine-grained scaling at the function level, achieving maximum system throughput with the same resources.

[0024] In a more specific technical solution, in S22, for function instances deployed on different nodes, TCP connections are established between function instances based on the IP addresses stored in the local routing table, and data is directly transmitted through preset network connection methods. These preset network connection methods include: RDMA hardware-accelerated cross-machine communication, and shared memory and one-sided RDMA-accelerated large data transmission.

[0025] In a more specific technical solution, S3 includes:

[0026] S31. When the first function instance C-1 of the third function C starts, the global controller is used to inform the first function instance and the second function instance of the address of the first function instance C-1, and a handshake is performed to establish a direct transmission connection.

[0027] S32. When the function instance is in the default stable state, the stable mode is adopted, and the differential routing algorithm is used to handle fan-in structure and non-fan-in structure.

[0028] S33. When a function instance is expanded, switch to panic mode and reroute.

[0029] To address routing conflicts that may result from the lack of a global controller for coordination, this invention designs two routing mechanisms: stable and panic, to ensure the normal execution of the workflow.

[0030] In a more specific technical solution, S32 also includes:

[0031] S321. For non-fan-in structures, a round-robin algorithm is used for upstream function instances, and each downstream function instance takes turns as the routing target.

[0032] S322. For the fan-in structure, use the consistent hashing algorithm, with the ID of the workflow request as the key, and map the routing result in the ring structure formed by the local routing table according to the hash value of each workflow request.

[0033] In a more specific technical solution, S33 includes:

[0034] S331. When a function instance is notified that it needs to update its local routing table, it switches from stable mode to panic mode.

[0035] S332. Assume that the critical path data arrives last. If the critical path data of the function instance has not all arrived, determine that a routing conflict has occurred.

[0036] S333: Cause the current function instance to send a rerouting notification to the function instance of the function with missing data;

[0037] S334. When an upstream function instance receives a rerouting notification, if the upstream function instance has the requested data locally, it resends the requested data to the function instance that sent the rerouting notification and requests that function instance to delete the requested data.

[0038] S335. Exit panic mode when it is determined that all workflow requests sent during all routing table updates have been fully aggregated in downstream function instances.

[0039] This invention supports dynamic routing for each request and incorporates stable and panic modes to ensure the workflow operates normally. The stable mode ensures load balancing of downstream instances and minimizes routing conflicts, while the panic mode monitors for routing conflicts and, if necessary, requests upstream instances to reroute.

[0040] In a more specific technical solution, S334 enables function instances to continuously check whether subsequent workflow requests meet preset conditions. When generating the required data, the data is rerouted to the corresponding downstream function instance. When the downstream function instance receives the rerouted data, it notifies the other upstream instances to stop checking.

[0041] In a more specific technical solution, in S4, the local routing group includes: function instances with direct transmission connections, and each local routing group also includes: workflow functions; the function instances are preferentially scaled up or down within the local routing group.

[0042] To address the lack of real-time monitoring of global instance status in local routing scenarios, this invention designs an instance scaling mechanism based on local routing groups. Each function instance is responsible for monitoring its own request arrival frequency and execution time, and, when necessary, collaborates with a centralized controller to complete scaling operations. This invention only requires the user to submit a directed acyclic graph of workflows containing inter-function dependencies. It can automatically deploy functions to a server-insensitive computing platform and execute various tasks without user intervention, effectively reducing labor costs.

[0043] In more specific technical solutions, the local routing system for server-insensitive computing workflow throughput optimization includes:

[0044] The routing function decentralization module is used to leverage the global controller to decentralize the function routing function information to each function instance. The function instance obtains the address of the function instance with which it has a dependency relationship based on the function information, establishes a direct transmission connection with the function instance with which it has a dependency relationship, dynamically selects the appropriate routing object for each workflow request, and performs expansion operations.

[0045] The dynamic selection module for routing instances enables function instances to obtain the addresses of other function instances and perform direct handshake operations. Based on the local routing table in the function instance's memory, it dynamically selects a routing instance for each workflow request. The dynamic selection module for routing instances is connected to the routing function decentralization module.

[0046] The stable and panic routing mechanism module is used to set up stable and panic routing mechanisms, enabling each function instance to make independent dynamic routing decisions and select the destination instance for data transmission. The workflow includes: first function A, second function B, and third function C. First function A, second function B, and third function C are executed using a preset fan-in structure. The stable and panic routing mechanism module is connected to the dynamic selection module for routing instances.

[0047] The scaling module is used to decentralize some of the functions' scaling functionality to each instance, build a scaling mechanism based on local routing groups, and use function instances to monitor the arrival frequency and execution time of workflow requests in real time. It collaborates with the centralized controller to make scaling decisions on scaling strategies and designs local routing groups as the scaling management unit. The scaling module is connected to the stable panic routing mechanism module.

[0048] The present invention has the following advantages over the prior art:

[0049] This invention enables direct data transmission supporting both direct handshakes and dynamic routing through local routing. By delegating routing functionality from the global controller to each function instance, it supports high system throughput and reduces additional handshake overhead and resource waste. This invention incorporates mechanisms such as stable / panic modes and local routing group scaling to ensure the correct operation and scaling of workflows under the local routing method, thus solving the problem of low system throughput in server-insensitive computing workflows.

[0050] Since the global controller is removed from intermediate data transmission, this invention also decentralizes some of the functions for scaling up and down to each instance, constructing a scaling up and down mechanism based on local routing groups to meet different system throughput needs.

[0051] This invention supports function instances to obtain the addresses of other function instances and perform direct handshakes. At the same time, it has a local routing table in the instance memory, which can dynamically select the routing instance for each request and avoid locking between instances.

[0052] This invention decentralizes routing capabilities to each function instance, thereby enabling direct handshakes and dynamic routing between instances. While achieving high-speed direct data transmission between instances, this invention avoids high handshake overhead and ensures fine-grained scaling at the function level, achieving maximum system throughput with the same resources.

[0053] To address routing conflicts that may result from the lack of a global controller for coordination, this invention designs two routing mechanisms: stable and panic, to ensure the normal execution of the workflow.

[0054] This invention supports dynamic routing for each request and incorporates stable and panic modes to ensure the workflow operates normally. The stable mode ensures load balancing of downstream instances and minimizes routing conflicts, while the panic mode monitors for routing conflicts and, if necessary, requests upstream instances to reroute.

[0055] To address the lack of real-time monitoring of global instance status in local routing scenarios, this invention designs an instance scaling mechanism based on local routing groups. Each function instance is responsible for monitoring its own request arrival frequency and execution time, and, when necessary, collaborates with a centralized controller to complete scaling operations. This invention only requires the user to submit a directed acyclic graph of workflows containing inter-function dependencies. It can automatically deploy functions to a server-insensitive computing platform and execute various tasks without user intervention, effectively reducing labor costs.

[0056] This invention solves the technical problems of inter-instance locking, routing conflicts caused by lack of global controller coordination, and limitations on system throughput performance in the prior art. Attached Figure Description

[0057] Figure 1This is a schematic diagram of the basic steps of the local routing method for optimizing server-insensitive computing workflow throughput in Embodiment 1 of the present invention.

[0058] Figure 2 This is a schematic diagram of the data flow processing steps of the local routing method for optimizing server-insensitive computing workflow throughput in Embodiment 1 of the present invention.

[0059] Figure 3 This is an example diagram of routing conflicts in Embodiment 1 of the present invention;

[0060] Figure 4 This is an example diagram of the panic mode in Embodiment 1 of the present invention;

[0061] Figure 5a This is a schematic diagram of an example of a local routing group expansion function in Embodiment 1 of the present invention;

[0062] Figure 5b This is a schematic diagram illustrating the principle of expanding the local routing group to avoid full connectivity in Embodiment 1 of the present invention.

[0063] Figure 6 This is a schematic diagram illustrating the data transmission efficiency of Embodiment 1 of the present invention;

[0064] Figure 7 This is a schematic diagram of the system throughput of Embodiment 1 of the present invention;

[0065] Figure 8a This is a schematic diagram of the system overhead of different modules in Embodiment 1 of the present invention;

[0066] Figure 8b This is a schematic diagram of the panic routing mode overhead in Embodiment 1 of the present invention. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0068] Example 1

[0069] like Figure 1 As shown, the local routing method for optimizing throughput in server-insensitive computing workflows includes the following basic steps:

[0070] S1. Support high system throughput by delegating routing functionality from the global controller to each function instance;

[0071] In this embodiment, the function routing functionality is delegated from the global controller to each function instance. The function instance can directly obtain the addresses of function instances that it depends on and establish direct transmission connections, and dynamically select the appropriate routing object for each request.

[0072] like Figure 1 As shown in this embodiment, in the overall architecture of the local routing system for optimizing throughput of server-insensitive computing workflows, workflow requests are directly sent by the system gateway to an instance of the entry function. After the function code is executed, the local controller in the instance will consult the locally stored routing table and then select a suitable downstream function instance to directly transmit the intermediate data. The function instance will monitor the frequency of request arrivals and its own processing capacity in real time. Once it finds that a request queue will occur, it will request the centralized controller to expand its capacity.

[0073] In this embodiment, various direct connection methods are supported depending on the location of the function instance, including but not limited to: direct connection for inter-process communication within the same node and direct TCP and RDMA connections across nodes. In this embodiment, for functions deployed within the same node, the routing table stores the Linux named pipe path, and the function instance can achieve direct data transmission based on inter-process communication by reading and writing to this pipe.

[0074] S2 enables function instances to obtain the addresses of other function instances and perform direct handshakes, while maintaining a local routing table in the instance memory to dynamically select a routing instance for each request;

[0075] In this embodiment, after expanding to include new instances, the centralized controller sends updated routing table information to all involved function instances. The function instances can then handshake with the new instances and be used as a backup for subsequent direct transmission requests. Note that the centralized controller in this invention still exists, but it is only responsible for synchronizing routing tables between instances and starting new function instances, and is no longer involved in handshakes and inter-function communication.

[0076] In this embodiment, for function instances deployed on different nodes, the routing table stores IP addresses. TCP connections are established between function instances, and data is directly transmitted over the network. This embodiment also supports cross-machine communication accelerated by RDMA hardware. Function instances first transmit network interface card (NIC) GID, request queues (QP), and data mapping (MR) information via the network, and then establish a direct channel through a handshake. Furthermore, this invention supports accelerating large data transmission within and between nodes using shared memory and one-sided RDMA, respectively.

[0077] S3 incorporates both stable and panic routing mechanisms to ensure the normal execution of the workflow;

[0078] In the stable and panic routing mechanisms of this embodiment, each function instance independently makes dynamic routing decisions and selects the destination instance for data transmission. This asynchronous communication paradigm may lead to routing conflicts, meaning that the result of the same request processed by parallel functions in fan-in mode may be routed to different downstream instances, causing downstream functions to be unable to obtain complete data to continue execution. Therefore, in this embodiment, stable and panic modes are designed to minimize routing conflicts and to perform rerouting when routing conflicts occur, respectively.

[0079] like Figure 3 As shown, in this embodiment, the workflow consists of three functions: A, B, and C. A fan-in structure exists, meaning function C needs to obtain the results from A and B before it can begin execution. After the first instance C-1 of function C starts, the controller informs the instances of functions A and B of the address, thus establishing a direct connection. At this time, the local routing tables of instances A and B only contain C-1, so the execution results of both requests can be routed to C-1, and the workflow can execute normally. However, after the system expands to include instance C-2, the notification times for updating the routing tables of instances A and B may differ, leading them to select the target instance based on different routing tables. As shown, instance A needs to choose between C-1 and C-2 during routing decisions and ultimately routes the data to the newly expanded C-2, while instance B still only knows of the existence of C-1 during decision-making, so the data is still routed to C-1. Therefore, the results of the same workflow request are held by C-1 and C-2 respectively, and neither can continue executing the request. This problem is referred to as a routing conflict in this invention.

[0080] In this embodiment, by default, function instances are in a stable state, employing two different routing algorithms to handle different workflow structures. For non-fan-in structures, where downstream instances only need to obtain data from one upstream instance to begin execution, since routing conflicts do not occur, only load balancing of downstream instances needs to be ensured. Therefore, upstream instances use a round-robin algorithm, with each downstream instance taking turns as the routing target. For fan-in structures where routing conflicts may occur, this embodiment uses a consistent hashing algorithm, using the request ID as the key and determining the routing result based on the mapping of its hash value on a ring structure formed by the local routing tables. Through consistent hashing, not only can the routing result for the same request be guaranteed for function instances storing the same routing table, but the ring structure also reduces the offset of hash value mapping during routing table updates, thereby reducing the probability of routing conflicts when the routing tables between instances are not completely consistent.

[0081] In this embodiment, when a function instance is expanded, the local routing tables of each instance are updated successively. At this time, the routing tables between instances will temporarily differ, causing the consistent hashing algorithm in a stable state to no longer guarantee that all routing results are the same. Therefore, this invention designs a panic mode to re-route when routing conflicts occur.

[0082] like Figure 4 As shown, in this embodiment, based on Figure 3 The example illustrates how panic mode resolves routing conflicts. In this example, function A is on the critical path. When instance C-2 receives data from A, it checks if it has already received data from B. If not, it assumes a routing conflict may have occurred and then C-2 sends a feedback to instance B. Upon receiving the feedback, B reroutes the request to C-2 and notifies C-1 to delete the originally transmitted data. At this point, C-2 receives the complete data and can continue execution, thus resolving the routing conflict.

[0083] See Figure 4 As shown on the right, it's also possible that the routing conflict hasn't occurred yet. In this case, after receiving the feedback, B will continuously check whether the generated data meets the requirements; if so, it will directly send it to C-2.

[0084] First, when an instance is notified that its local routing table needs updating, it switches from stable mode to panic mode. Then, this invention assumes that data on the critical path should arrive last. If an instance receives critical path data but data is still missing, it assumes a routing conflict may have occurred and sends a rerouting notification to all instances of the function with missing data. In this embodiment, after receiving the rerouting notification, if the upstream instance has the required data locally, it resends the notification to the instance that originated it and requests the original routing instance to delete the data. In this embodiment, considering that the required data may not yet be generated, the instance continuously checks whether subsequent requests meet the conditions. If the required data is generated later, it directly routes the request to the corresponding downstream instance. To avoid redundant checking overhead, the downstream function, upon receiving the rerouting data, notifies other upstream instances to stop checking.

[0085] In this embodiment, to ensure that all requests are executed correctly, the panic mode is exited only after it is confirmed that all requests sent during the routing table update period have been fully aggregated in the downstream instances. Specifically, after the centralized controller confirms that all local routing tables have been updated, it informs each function instance that during the period from entering panic mode to receiving the notification, routed requests may cause routing conflicts, and the function instances will record these requests. In panic mode, after receiving all upstream data for the same request, the downstream instance will notify all upstream functions that the request can be executed correctly. Once all recorded requests that may cause routing conflicts have been confirmed to be without problems, the function instance will notify the centralized controller, which in turn notifies all instances to exit panic mode.

[0086] In this embodiment, to achieve rerouting in panic mode, the function instance will temporarily save the data locally after completing the direct data transmission for retransmission. After the workflow request is completed, it will continuously notify the upstream function instance to delete the result of the request, starting from the downstream function instance.

[0087] S4. Delegate some of the functions' scaling capabilities to each instance, and build a scaling mechanism based on local routing groups.

[0088] In the scaling mechanism based on local routing groups in this embodiment, function instances monitor the frequency of request arrivals and the execution time of requests in real time, and then work with the centralized controller to make scaling decisions on scaling strategies, and design local routing groups as the unit of scaling management.

[0089] In this embodiment, the set of function instances with direct connections is called a local routing group. Each local routing group contains all the functions of the workflow, and each workflow may be supported by multiple local routing groups. Different routing groups are unaware of each other. Function instances are scaled up or down within the local routing group first, and the local routing group is scaled up or down as necessary.

[0090] In this embodiment, the function instance monitors the arrival frequency λ of requests and the average execution time t of requests per unit time in real time. If it detects... This indicates that the current instance's load has exceeded its processing capacity, and the function needs to scale up with new instances. The function instance then sends a scaling request containing λ and t data to the centralized controller, which then calculates the number of instances required to support the current system throughput. The controller then calls the relevant interfaces to create new instances. Specifically, the centralized controller may receive scaling requests from different instances of the same function. In this case, the controller updates the total arrival frequency λ and average execution time t, recalculates α, and then determines the number of instances required for scaling. Furthermore, to prevent instance failures due to excessive request concurrency, a threshold is set for λ. When the request frequency of a single instance exceeds this threshold, instance scaling is also triggered. By default, the centralized controller adds the scaled-up instances to the local routing group from which the scaling request originated. However, workflows may contain hundreds or thousands of function instances simultaneously. Adding all instances to the same routing group and establishing direct connections can easily lead to scalability issues. Therefore, scaling conditions are set for local routing groups. When all functions within a routing group need scaling (i.e., the last function also initiates a scaling request), the centralized controller creates a new routing group and adds the newly scaled-up instances to it.

[0091] In this embodiment, to achieve the scaling-down function, the centralized controller will periodically, for example, every 10 minutes, require each function instance to report λ ​​and t data, and then calculate the data based on the collected total request arrival frequency ∑λ and average execution time. The system calculates the required number of instances and then determines the number of instances to be reclaimed. Furthermore, if a function instance detects that no requests have arrived for a period of time, it will directly notify the controller to remove it. Once all instances in a local routing group have been released, that routing group will be removed. After identifying the function instances to be reclaimed, the centralized scheduler first notifies its directly connected upstream instances to stop sending data to it, then waits for the instance to confirm that all its requests can be correctly executed by downstream instances, and finally releases all direct connections and reclaims the instance.

[0092] In this embodiment, it does not rely on any specific scheduling system, but only updates the local routing table and establishes direct transmission connections based on the system scheduling results, thus it can be seamlessly adapted to various scheduling frameworks.

[0093] As shown in Figure 5(a), in the local routing group-based expansion operation of this embodiment, the workflow consists of two functions, A and B. Function instance A-1 detects that the current request arrival rate is 6 requests per second, while its execution time for each request is 0.5 seconds, which satisfies... Therefore, the scaling mechanism is triggered. The controller determines, based on λ and t, that three instances of function A are needed to meet the current request frequency, and thus scales up instances A-2 and A-3. As for function B, a single instance can handle 6 requests per second, so scaling is not required. Since only function A is currently scaling up, this invention adds the newly scaled-up instances A-2 and A-3 to the current local routing group and establishes a direct connection with B-1.

[0094] As shown in Figure 5(b), in this embodiment, as the request arrival rate further increases to 8 requests per second, function A expands to create a new instance A-4, which in turn causes instance B-1 to encounter a processing bottleneck, triggering the expansion mechanism. To avoid full connectivity between multiple instances of A and B, this invention expands to create a new local routing group and adds B-2 and A-4 to it. The two instances handshake and transmit directly. Workflow requests are evenly distributed by the gateway to each instance of the entry function A, and the routing groups are unaware of each other. Note that A-4 in the figure was originally added to routing group 2 and connected to B-1. To avoid further complicating the routing conflict situation by switching routing groups, this invention will start a new A-4 instance to join local routing group 2 and destroy the original A-4 in routing group 1.

[0095] Compared to existing work, this invention has significant advantages. For example... Figure 6 As shown, in this embodiment, the efficiency of different methods for transferring data of various sizes between two functions was tested. The direct data transmission of the present invention is 1.4-15.6 times faster than third-party forwarding, and local and RDMA direct transmissions are 1.2-3.5 times and 1.1-2.1 times faster than TCP direct transmissions, respectively. Figure 7 As shown, in this embodiment, the end-to-end latency of each system in three workflows was tested. In terms of throughput, this invention improves throughput by 5.7-93.8 times compared to third-party routing solutions, and while maintaining the original performance, it improves throughput by 1.8-5.4 times compared to existing connection multiplexing solutions. Figure 8a As shown, in this embodiment, based on connection multiplexing, the various modules designed in this invention are added sequentially, and the workflow latency is tested to measure module overhead. Dynamic routing and real-time request monitoring only resulted in an additional overhead of about 20 microseconds. Figure 8b As shown, the latency of workflows with different degrees of parallelism was compared between stable and panic routing modes. The results showed that panic routing mode only caused a 1.9-11.1% increase in the 99th percentile latency of the workflow in a short period of time.

[0096] In summary, this invention enables direct data transmission supporting both direct handshakes and dynamic routing through local routing. By delegating routing functionality from the global controller to each function instance, it supports high system throughput while reducing additional handshake overhead and resource waste. This invention incorporates mechanisms such as stable / panic modes and local routing group scaling to ensure the correct operation and scaling of workflows under the local routing method, thus solving the problem of low system throughput in server-insensitive computing workflows.

[0097] Since the global controller is removed from intermediate data transmission, this invention also decentralizes some of the functions for scaling up and down to each instance, constructing a scaling up and down mechanism based on local routing groups to meet different system throughput needs.

[0098] This invention supports function instances to obtain the addresses of other function instances and perform direct handshakes. At the same time, it has a local routing table in the instance memory, which can dynamically select the routing instance for each request and avoid locking between instances.

[0099] This invention decentralizes routing capabilities to each function instance, thereby enabling direct handshakes and dynamic routing between instances. While achieving high-speed direct data transmission between instances, this invention avoids high handshake overhead and ensures fine-grained scaling at the function level, achieving maximum system throughput with the same resources.

[0100] To address routing conflicts that may result from the lack of a global controller for coordination, this invention designs two routing mechanisms: stable and panic, to ensure the normal execution of the workflow.

[0101] This invention supports dynamic routing for each request and incorporates stable and panic modes to ensure the workflow operates normally. The stable mode ensures load balancing of downstream instances and minimizes routing conflicts, while the panic mode monitors for routing conflicts and, if necessary, requests upstream instances to reroute.

[0102] To address the lack of real-time monitoring of global instance status in local routing scenarios, this invention designs an instance scaling mechanism based on local routing groups. Each function instance is responsible for monitoring its own request arrival frequency and execution time, and, when necessary, collaborates with a centralized controller to complete scaling operations. This invention only requires the user to submit a directed acyclic graph of workflows containing inter-function dependencies. It can automatically deploy functions to a server-insensitive computing platform and execute various tasks without user intervention, effectively reducing labor costs.

[0103] This invention solves the technical problems of inter-instance locking, routing conflicts caused by lack of global controller coordination, and limitations on system throughput performance in the prior art.

[0104] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A local routing method for optimizing throughput in server-insensitive computing workflows, characterized in that: The method includes: S1. Using the global controller, the function routing information is distributed to each function instance. The function instance obtains the address of the function instance with which it has a dependency relationship according to the function information, establishes a direct transmission connection with the function instance with which it has a dependency relationship, dynamically selects the appropriate routing object for each workflow request, and performs expansion operation. S2. Enable the function instance to obtain the addresses of the other function instances and perform a direct handshake operation, and dynamically select a routing instance for each workflow request according to the local routing table in the memory of the function instance. S3. Design stable and panic routing mechanisms so that each function instance can independently make dynamic routing decisions and select the destination instance for data transmission. The workflow includes: a first function A, a second function B, and a third function C. The first function A, the second function B, and the third function C are executed using a preset fan-in structure. When the function instance is expanded, switching to panic mode and rerouting includes: S331. When the function instance is notified that the local routing table needs to be updated, it switches from stable mode to panic mode. S332. Assuming the critical path data arrives last, if not all the critical path data of the function instance arrives, a routing conflict is determined to have occurred. S333, cause the current function instance to send a rerouting notification to the function instance of the data-missing function; S334. When the upstream function instance receives the rerouting notification, if the upstream function instance has the required data locally, the required data is retransmitted to the function instance that sent the rerouting notification, and the function instance is asked to delete the required data. S335. When it is determined that all workflow requests sent during all routing table updates have been fully aggregated in the downstream function instances, exit the panic mode; S4. Delegate some of the functions' scaling up and down functionality to each instance, build a scaling up and down mechanism based on local routing groups, so as to use the function instances to monitor the arrival frequency and execution time of the workflow requests in real time, coordinate with the centralized controller to make scaling up and down strategies, and design local routing groups as the scaling up and down management units. In the stable mode, for non-fan-in structures, the upstream instance uses a round-robin algorithm, and each downstream instance takes turns as the routing target. For fan-in structures, a one-time hash algorithm is used for routing. The local routing group includes: the function instance with a direct connection, and each local routing group also includes: a workflow function.

2. The local routing method for server-insensitive computing workflow throughput optimization according to claim 1, characterized in that, S1 includes: S11. Using the system gateway, the workflow request is directly sent to the function instance of the entry function to execute the function code; S12. In the current function instance, the local controller is used to consult the local storage routing table, select a suitable downstream function instance as the applicable routing object, and directly transmit the intermediate data to the applicable routing object. S13. The function instance monitors the arrival frequency of the workflow requests and its own processing capacity in real time, and requests the centralized controller to perform the expansion operation when the workflow requests are in a queue.

3. The local routing method for server-insensitive computing workflow throughput optimization according to claim 1, characterized in that, S2 includes: S21. Using the centralized controller, send routing table update information to the new function instance; S22. The function instance and the new function instance perform a handshake operation to provide a direct transmission alternative for subsequent workflow requests.

4. The local routing method for server-insensitive computing workflow throughput optimization according to claim 3, characterized in that, In step S22, for the function instances deployed on different nodes, a TCP connection is established between the function instances based on the IP address stored in the local routing table, and data is directly transmitted through a preset network connection method. The preset network connection method includes: RDMA hardware-accelerated cross-machine communication, and shared memory and one-sided RDMA-accelerated large data transmission.

5. The local routing method for server-insensitive computing workflow throughput optimization according to claim 1, characterized in that, S3 includes: S31. When the first function instance C-1 of the third function C starts, the global controller is used to inform the first function instance C-1 and the second function instance of the address of the first function instance C-1, and a handshake is performed to establish a direct transmission connection. S32. When the function instance is in the default stable state, the stable mode is adopted, and the differential routing algorithm is used to handle fan-in structure and non-fan-in structure.

6. The local routing method for server-insensitive computing workflow throughput optimization according to claim 5, characterized in that, S32 further includes: S321. For the non-fan-in structure, a round-robin algorithm is used for the upstream function instance, and each downstream function instance takes turns as the routing target. S322. For the fan-in structure, a consistent hashing algorithm is used, with the ID of the workflow request as the key value, and the routing result is determined by mapping the hash value of the workflow request in the ring structure formed by the local routing table.

7. The local routing method for server-insensitive computing workflow throughput optimization according to claim 1, characterized in that, In step S334, the function instance continuously checks whether subsequent workflow requests meet preset conditions. When generating required data, it routes the data in the rerouting notification to the corresponding downstream function instance. When the downstream function instance receives the data in the rerouting notification, it notifies the other upstream instances to stop checking.

8. The local routing method for server-insensitive computing workflow throughput optimization according to claim 1, characterized in that, In step S4, the function instance is first scaled up or down within the local routing group.

9. A local routing system for server-insensitive computing workflow throughput optimization, used to execute the local routing method for server-insensitive computing workflow throughput optimization as described in any one of claims 1 to 8, characterized in that, The system includes: The routing function decentralization module is used to decentralize the function routing function information to each function instance using the global controller. The function instance obtains the address of the function instance with which it has a dependency relationship based on the function information, establishes a direct transmission connection with the function instance with which it has a dependency relationship, dynamically selects the appropriate routing object for each workflow request, and performs expansion operation. The dynamic routing instance selection module enables the function instance to obtain the addresses of the other function instances and perform a direct handshake operation. Based on the local routing table in the memory of the function instance, the module dynamically selects a routing instance for each workflow request. The dynamic routing instance selection module is connected to the routing function decentralization module. The stable panic routing mechanism module is used to set up stable and panic routing mechanisms, enabling each function instance to independently make dynamic routing decisions and select the destination instance for data transmission. The workflow includes: a first function A, a second function B, and a third function C. The first function A, the second function B, and the third function C are executed using a preset fan-in structure. The stable panic routing mechanism module is connected to the dynamic selection module for routing instances. The scaling module is used to decentralize some of the function scaling functionality to each instance, build a scaling mechanism based on local routing groups, and use the function instances to monitor the arrival frequency and execution time of the workflow requests in real time, coordinate with the centralized controller to make scaling decisions on scaling strategies, and design local routing groups as the scaling management unit. The scaling module is connected to the stable panic routing mechanism module.

Images