Task deployment method and apparatus, task management system, controller, and storage medium

Abstract

The present application belongs to the field of distributed data processing. Disclosed are a task deployment method and apparatus, a task management system, a controller, and a storage medium. The method comprises: acquiring an aggregation resource utilization rate and bandwidth resource utilization rate of each network device, and a processing resource utilization rate of each server; according to task running resources required by a first data processing task to be deployed, the aggregation resource utilization rates and bandwidth resource utilization rates of a plurality of network devices and the processing resource utilization rates of a plurality of servers, determining a task deployment strategy and an aggregation resource allocation strategy, wherein the task deployment strategy indicates a first server from among the plurality of servers that is used for executing the first data processing task, and the aggregation resource allocation strategy indicates an aggregator in a first network device among the plurality of network devices, which aggregator is used for performing an aggregation operation on data to be processed in said first data processing task; and on the basis of the task deployment strategy and the aggregation resource allocation strategy, deploying said first data processing task in the first server.

Description

Task deployment method, device, task management system, controller and storage medium

[0001] This application claims priority to Chinese patent application No. 202310797785.9, filed on June 30, 2023, entitled “Task deployment method, device, task management system, controller and storage medium,” the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of distributed data processing, and in particular to a task deployment method, device, task management system, controller and storage medium. Background Art

[0003] With the expansion of the network structure of machine learning models and the increase in training data, distributed training has gradually become the mainstream training method for machine learning models. As a distributed architecture, the parameter server (PS) architecture divides the servers in the server cluster into two types: parameter servers (also called PS nodes) and computing servers (also called worker nodes). A model training task is usually deployed on a parameter server and multiple computing servers. The parameter server stores the model parameters of the model to be trained. The multiple computing servers obtain the model parameters from the parameter server and train the model locally based on the model parameters. The gradient data obtained from the training is sent to the parameter server, so that the parameter server updates the model parameters by aggregating the gradient data sent by the multiple computing servers.

[0004] In the related art, when a server cluster under the PS architecture deploys a model training task, the parameter server and multiple computing servers corresponding to the model training task are determined based on the task running resources required for the model training task, the processing resource utilization of each server, and the bandwidth resources between the parameter server and the computing server, so that the model training task is deployed in the server cluster. For any model training task, the multiple computing servers that deploy the model training task will send all gradient data to the parameter server for aggregation operations. This "many-to-one" communication mode will cause network congestion and even model training task timeouts. In order to speed up the speed of distributed training, a programmable switch is introduced into the above-mentioned PS architecture. The programmable switch can not only realize the data forwarding function between the computing server and the parameter server, but also perform aggregation operations on the gradient data obtained by the multiple computing servers, and then send the aggregation results to the parameter server, reducing the data processing volume of the parameter server.

[0005] Since the computing resources of programmable switches are introduced, and multiple model training tasks can share the computing resources of a programmable switch, when the multiple model training tasks are deployed unevenly and occupy the same programmable switch at the same time, it will seriously affect the overall performance of the programmable switch. Therefore, a new task deployment method is urgently needed to fully consider the computing resources of the programmable switch and the processing resources of the server when deploying model training tasks.

[0006] Summary of the Invention

[0007] This application provides a task deployment method, device, task management system, controller, and storage medium that can integrate the aggregate resource utilization and bandwidth resource utilization of network devices, as well as the processing resource utilization of servers, to deploy data processing tasks in a server cluster. The technical solution is as follows:

[0008] In a first aspect, a task deployment method is provided, which is applied to a controller in a task management system, wherein the task management system also includes a server cluster and multiple network devices, the server cluster includes multiple servers, and each network device includes multiple aggregators obtained by dividing the memory resources of the network device; the method includes: obtaining the aggregate resource utilization of each network device, the bandwidth resource utilization of each network device, and the processing resource utilization of each server; determining a task deployment strategy and an aggregate resource allocation strategy based on the task running resources required for the first data processing task to be deployed, the aggregate resource utilization and bandwidth resource utilization of multiple network devices, and the processing resource utilization of multiple servers, the task deployment strategy indicating a first server among multiple servers for executing the first data processing task, and the aggregate resource allocation strategy indicating an aggregator in the first network device among multiple network devices for performing aggregate operations on the processed data of the first data processing task; and deploying the first data processing task in the first server based on the task deployment strategy and the aggregate resource allocation strategy.

[0009] Thus, when deploying the first data processing task in the task management system, the controller can comprehensively analyze the aggregate resource utilization of each network device, the bandwidth resource utilization of each network device, the processing resource utilization of each server, and the task computing resources required for the first data processing task, thereby determining the task deployment strategy and aggregate resource allocation strategy for the first data processing task. By fully considering the usage of the server's processing resources, the network device's bandwidth resources, and the aggregate resources, task deployment and resource allocation are more reasonable, thereby improving the overall execution performance of the first data processing task and reducing the task completion time.

[0010] Optionally, obtaining the aggregation resource utilization of each network device includes: sending a measurement message to the network device, the measurement message carrying period information of a first measurement cycle, the first measurement cycle being the measurement cycle in which the deployment time of the first data processing task is located; receiving a measurement reply message sent by the network device, the measurement reply message carrying the number of occupied aggregators of the network device within the first measurement cycle; and determining the aggregation resource utilization of the network device based on the number of occupied aggregators.

[0011] Specifically, the controller sends measurement messages to obtain the number of aggregators occupied by a network device during a measurement period. Based on this number, the controller analyzes and calculates the aggregate resource utilization of the network device. This improves the efficiency of measuring aggregate resource utilization, allowing the controller to monitor aggregate resource utilization of network devices on demand at any time.

[0012] Optionally, sending a measurement message to a network device includes: determining a target server from multiple servers, where the target server refers to a server connected to the network device, and the network device is located on a communication link between the controller and the target server; sending a measurement message to the target server through the communication link between the controller and the target server, so that the network device receives and intercepts the measurement message.

[0013] Since the controller sends the measurement message to the network device via the communication link between the controller and the server, all network devices on the communication link can receive the measurement message, thereby comprehensively and effectively measuring the aggregate resource utilization of each network device.

[0014] Optionally, based on the number of occupied aggregators, the aggregation resource utilization of the network device is determined, including: determining the maximum number of aggregations of the aggregators included in the network device within the first measurement period based on the total number of aggregators included in the network device, the single occupancy time of the aggregators of the network device within the first measurement period, and the period length of the first measurement period; determining the aggregation resource utilization of the network device based on the number of occupied aggregators and the maximum number of aggregations.

[0015] Task messages occupy the aggregator's aggregation resources. After the aggregation operation is completed on the processed data contained in the task message, the aggregator's aggregation resources are released, and the aggregator can now process task messages for other data processing tasks. Therefore, for an aggregator, its single occupation duration is the time it takes for the network device to receive a task message and complete the aggregation operation on the processed data in the task message.

[0016] Specifically, the controller determines the number of times each aggregator in the network device is occupied during the first measurement period based on the length of the first measurement period and the duration of each aggregator's single occupation during the first measurement period. The controller then determines the maximum number of aggregations for the network device during the first measurement period based on the number of times each aggregator is occupied during the first measurement period and the total number of aggregators in the network device. This allows the controller to analyze the aggregation resource utilization of the network device based on the occupation of each aggregator in the network device during a single measurement period, simplifying operations and providing more accurate results.

[0017] Optionally, the first data processing task refers to a data processing task that requires an exclusive aggregator; the method also includes: determining a deployed second data processing task, the processing data of the second data processing task and the first data processing task both need to be aggregated in the first network device, and the second data processing task refers to a data processing task that can share an aggregator with other data processing tasks; sending a task pause message to the second server to instruct the second server to suspend sending task messages after the target sequence number of the second data processing task, the second server is a server among multiple servers that executes the second data processing task; receiving a pause response message sent by the second server, and sending an addressing reconfiguration message to the second server, the addressing reconfiguration message indicating the aggregator reallocated for the second data processing task in the first network device.

[0018] As can be seen from this, if the first data processing is a data processing task that exclusively occupies an aggregator, when the first data processing task is deployed in the task management system and the processing data of the first data processing task is aggregated through the first network device, it is necessary to reallocate the aggregator to the second data processing task that shares the first network device aggregator with other data processing tasks. This is to avoid the situation where the task messages of the first and second data processing tasks occupy the same aggregator, resulting in the inability to effectively aggregate task messages with the same sequence number and the inability to release the aggregation resources of the aggregator. In this way, through aggregator reallocation, it is possible to ensure that the first network device effectively aggregates the processing data of each data processing task, and each data processing task can operate normally. Secondly, by setting the target sequence number, the task message sending progress of the second data processing task can be unified, so that multiple computing servers executing the second data processing task can pause at the task message with the same sequence number, and restart the second data processing task based on the task message with the same sequence number, so that the task messages of each sequence number of the second data processing task before and after the pause can be normally aggregated.

[0019] Optionally, the second server includes a parameter server and multiple computing servers, the parameter server is used to store task parameters of the second data processing task, and each computing server is used to execute the second data processing task based on the task parameters; sending a task pause message to the second server includes: sending a task pause message to the parameter server so that the parameter server controls multiple computing servers to pause sending task messages after the target sequence number of the second data processing task; receiving a pause response message sent by the second server includes: receiving a pause response message sent by the parameter server, the pause response message indicating that multiple computing servers have paused sending task messages after the target sequence number of the second data processing task.

[0020] The task pause message is used to instruct the second server to suspend sending task messages following the target sequence number of the second data processing task, to prevent the processing data carried in the task messages sent by the second server from being effectively aggregated by the aggregator after the first network device allocates an exclusive aggregator for the first data processing task. After the second server receives the task pause message and sends the task message with the target sequence number of the second data processing task, it suspends sending task messages for the second data processing task to the first network device and sends a pause response message to the controller.

[0021] That is, when the second data processing task is deployed on multiple computing servers, the controller interacts with the parameter server to control the multiple computing servers to pause at the task message with the same sequence number, and subsequently restart the second data processing task based on the task message with the same sequence number.

[0022] Optionally, based on the task running resources required for the first data processing task to be deployed, the aggregate resource utilization and bandwidth resource utilization of multiple network devices, and the processing resource utilization of multiple servers, a task deployment strategy and an aggregate resource allocation strategy are determined, including: determining the deployment constraints of the first data processing task based on the task running resources, the aggregate resource utilization and bandwidth resource utilization of multiple network devices, and the processing resource utilization of multiple servers; and determining the task deployment strategy and the aggregate resource allocation strategy based on the deployment constraints.

[0023] Optionally, deployment constraints include:

[0024] The task execution resources are less than or equal to the available processing resources of the server capable of executing the first data processing task;

[0025] The processing resource usage of the server capable of executing the first data processing task is less than or equal to the processing resource threshold;

[0026] The bandwidth resource between the server capable of executing the first data processing task and the network device capable of performing aggregation calculations on the processed data of the first data processing task is less than or equal to the bandwidth threshold;

[0027] The memory resource occupancy of the network device capable of performing an aggregation operation on the processing data of the first data processing task is less than or equal to the memory resource threshold.

[0028] Among them, the server's available processing resources are used to indicate whether the server can continue to deploy data processing tasks, the processing resource threshold is used to indicate whether the server is in an overloaded state, the bandwidth threshold is used to indicate the bandwidth resources where data loss may occur, and the memory resource threshold is used to indicate whether the network device is in an overloaded state.

[0029] Since the deployment constraints fully limit the conditions that the server that can execute the first data processing task should meet, and the conditions that the network equipment that can perform aggregation operations on the processed data of the first data processing task should meet, the task deployment strategy and aggregation resource allocation strategy determined based on the deployment constraints are more accurate and reasonable.

[0030] On the second aspect, another task deployment method is provided, which is applied to a task management system, wherein the task management system includes a controller, a server cluster and multiple network devices, the server cluster includes multiple servers, and each network device includes multiple aggregators obtained by dividing the memory resources of the network device; the method includes: the controller obtains the aggregate resource utilization of each network device, the bandwidth resource utilization of each network device, and the processing resource utilization of each server; the controller determines the task deployment strategy and the aggregate resource allocation strategy based on the task running resources required for the first data processing task to be deployed, the aggregate resource utilization and bandwidth resource utilization of multiple network devices, and the processing resource utilization of multiple servers, the task deployment strategy indicates the first server among the multiple servers for executing the first data processing task, and the aggregate resource allocation strategy indicates the aggregator in the first network device among the multiple network devices for performing aggregate operations on the processed data of the first data processing task; the controller deploys the first data processing task in the first server based on the task deployment strategy and the aggregate resource allocation strategy; the first server stores the task data of the first data processing task and the resource information of the aggregate resources allocated to the first data processing task.

[0031] Optionally, the controller obtains the aggregation resource utilization of each network device, including: the controller sends a measurement message to the network device, the measurement message carries the period information of a first measurement cycle, and the first measurement cycle is the measurement cycle in which the deployment time of the first data processing task is located; the network device receives the measurement message, determines the number of occupied aggregators of the network device within the first measurement cycle based on the period information of the first measurement cycle, and sends a measurement reply message to the controller, the measurement reply message carries the number of occupied aggregators; the controller receives the measurement reply message, and determines the aggregation resource utilization of the network device based on the number of occupied aggregators.

[0032] Optionally, the period information of the first measurement cycle is the version number corresponding to the first measurement cycle, and a first state counter is provided in the network device, and the first state counter has a first version number; the network device determines the number of occupied aggregators of the network device within the first measurement cycle based on the period information of the first measurement cycle, including: in response to the version number corresponding to the first measurement cycle being the same as the first version number, the network device determines the number of occupied aggregators of the network device within the first measurement cycle based on the count value of the first state counter.

[0033] The first state counter is used to record the number of occupied aggregators in the network device during the current measurement period. After receiving the measurement message, in response to the version number corresponding to the first measurement period being the same as the first version number, the network device determines the number of occupied aggregators during the first measurement period based on the count value of the first state counter.

[0034] Optionally, the first state counter also has first state information, and a second state counter is also provided in the network device, and the second state counter has second state information and a second version number; after the network device sends a measurement reply message to the controller, the method also includes: in response to the first state information indicating the current measurement cycle and the second state information indicating the previous measurement cycle, the network device switches the first state information and the second state information, updates the second version number to the version number corresponding to the second measurement cycle, clears the count value of the second state counter, and records the number of occupied aggregators in the second measurement cycle through the second state counter, and the second measurement cycle is the next measurement cycle adjacent to the first measurement cycle.

[0035] Specifically, the network device uses two counters, one in each state, to record the number of occupied aggregators in the current measurement cycle and the other in the previous measurement cycle. By switching the states of the two counters and updating their version numbers, the device ensures that only one state is used to record the number of occupied aggregators during each measurement cycle, and that the states of the counters used to record the number of occupied aggregators in consecutive measurement cycles are different.

[0036] Optionally, the first data processing task refers to a data processing task that requires an exclusive aggregator; the method also includes: the controller determines a deployed second data processing task, and the processing data of the second data processing task and the first data processing task both need to be aggregated in the first network device, and the second data processing task refers to a data processing task that can share the aggregator with other data processing tasks; the controller sends a task pause message to the second server, and the second server is a server among multiple servers that executes the second data processing task; the second server receives the task pause message, and after sending the task message with the target sequence number of the second data processing task, it suspends sending the task message of the second data processing task, and sends a pause response message to the controller; the controller receives the pause response message, and sends an addressing reconfiguration message to the second server, and the addressing reconfiguration message indicates the aggregator reallocated for the second data processing task in the first network device.

[0037] Optionally, the second server includes a parameter server and multiple computing servers, the parameter server is used to store task parameters of the second data processing task, and each computing server is used to execute the second data processing task based on the task parameters; the controller sends a task pause message to the second server, including: the controller sends a task pause message to the parameter server; the second server receives the task pause message, suspends sending the task message of the second data processing task after sending the task message with the target sequence number of the second data processing task, and sends a pause response message to the controller, including: the parameter server receives the task pause message, determines the maximum message sequence number that has completed the aggregation operation at the current moment, and sends the maximum message sequence number to multiple computing servers; the computing server determines the target sequence number based on the maximum message sequence number and the size of the message sending window, and suspends sending the task message of the second data processing task after sending the task message with the target sequence number of the second data processing task; after the parameter server receives the task message with the target sequence number of the second data processing task sent by multiple computing servers, it sends a pause response message to the controller.

[0038] Among them, the message sending window is used to limit the number of task messages sent by the computing server at one time. That is, within this message sending window, the computing server does not need to wait for the parameter server to send a task reply message, and can continue to send task messages until the number of task messages corresponding to the message sending window has been sent.

[0039] Therefore, to ensure that the compute servers respond to the same maximum message sequence number and stop at the same task message, after receiving the maximum message sequence number, the compute server sends the task message with the target sequence number and then pauses sending the task message for the second data processing task. In other words, even if there are time differences between the compute servers sending task messages, they will ultimately maintain consistency and stop at the task message with the same sequence number.

[0040] Optionally, the task reply message with the maximum message sequence number sent by the parameter server to the computing server carries the maximum message sequence number and a pause identifier; the method also includes: if there is a target computing server among multiple computing servers that has not received the task reply message with the maximum message sequence number, but has received a task reply message with the next message sequence number of the maximum message sequence number, the target computing server re-sends the task message with the maximum message sequence number to the parameter server; the parameter server re-sends the task reply message with the maximum message sequence number to the target computing server.

[0041] By having the target compute server resend the task message with the maximum message sequence number, we ensure that each compute server can execute the task pause operation based on the same maximum message sequence number. This way, even if there is a time difference between the compute servers receiving the task reply message with the maximum message sequence number, they can still execute the task pause operation based on the maximum message sequence number determined by the parameter server at the same time. This ensures that the message sequence numbers of the task messages sent at the final pause time are the same, allowing the parameter server to effectively aggregate messages before that time in sequence.

[0042] In a third aspect, a task deployment device is provided, which has the function of implementing the task deployment method described in the first aspect. The task deployment device includes one or more modules, which are used to implement the task deployment method described in the first aspect.

[0043] In a fourth aspect, a task management system is provided, which includes a controller, a server cluster and multiple network devices. The server cluster includes multiple servers, and each network device includes multiple aggregators obtained by dividing the memory resources of the network device.

[0044] Among them, the controller is used to obtain the aggregate resource utilization of each network device, the bandwidth resource utilization of each network device, and the processing resource utilization of each server, and determine the task deployment strategy and the aggregate resource allocation strategy based on the task running resources required for the first data processing task to be deployed, the aggregate resource utilization and bandwidth resource utilization of multiple network devices, and the processing resource utilization of multiple servers. The task deployment strategy indicates the first server among the multiple servers for executing the first data processing task, and the aggregate resource allocation strategy indicates the aggregator in the first network device among the multiple network devices for performing aggregate operations on the processing data of the first data processing task. Based on the task deployment strategy and the aggregate resource allocation strategy, the first data processing task is deployed in the first server; the first server is used to store the task data of the first data processing task and the resource information of the aggregate resources allocated to the first data processing task; the first network device is used to perform aggregate operations on the processing data of the first data processing task.

[0045] In a fifth aspect, a controller is provided, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor calls and executes the computer program, the steps of the task deployment method shown in the first aspect are implemented.

[0046] In a sixth aspect, a computer-readable storage medium is provided, in which a computer program is stored. When a controller executes the computer program, the steps of the task deployment method shown in the first aspect are implemented.

[0047] In a seventh aspect, a computer program product is provided, which includes a computer program. When a controller executes the computer program, the steps of the task deployment method shown in the first aspect are implemented.

[0048] The technical effects obtained in the above-mentioned second to seventh aspects are similar to the technical effects achieved by the corresponding technical features in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] FIG1 is a system architecture diagram of a task management system provided by an embodiment of the present application;

[0050] FIG2 is a schematic diagram of the structure of a task message provided in an embodiment of the present application;

[0051] FIG3 is a schematic structural diagram of a computer device provided in an embodiment of the present application;

[0052] FIG4 is a flow chart of a task deployment method provided in an embodiment of the present application;

[0053] 5 is a schematic diagram of a controller sending a measurement message to a network device according to an embodiment of the present application;

[0054] FIG6 is a schematic diagram of the structure of a measurement message provided in an embodiment of the present application;

[0055] 7 is a schematic diagram of a count value of a network device update status counter provided by an embodiment of the present application;

[0056] FIG8 is a schematic diagram of the structure of a measurement reply message provided in an embodiment of the present application;

[0057] FIG9 is a schematic diagram of determining a single occupancy duration of an aggregator according to an embodiment of the present application;

[0058] 10 is a schematic diagram of state information of a network device switching state counter provided in an embodiment of the present application;

[0059] 11 is a schematic diagram of a network device responding to a retransmitted measurement message according to an embodiment of the present application;

[0060] FIG12 is a schematic diagram of aggregator allocation in a network device provided in an embodiment of the present application;

[0061] FIG13 is a schematic diagram of aggregator allocation in another network device provided in an embodiment of the present application;

[0062] FIG14 is a flow chart of another task deployment method provided in an embodiment of the present application;

[0063] FIG15 is a schematic diagram of the structure of a task reply message provided in an embodiment of the present application;

[0064] FIG16 is a schematic diagram of pausing a data processing task provided in an embodiment of the present application;

[0065] FIG17 is a schematic diagram of another method of pausing a data processing task provided by an embodiment of the present application;

[0066] FIG18 is a schematic diagram of the structure of an addressing reconfiguration message provided in an embodiment of the present application;

[0067] FIG19 is a schematic structural diagram of a task deployment device provided in an embodiment of the present application;

[0068] Figure 20 is a structural diagram of another task deployment device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0070] In environments such as public clouds, data processing services are provided externally through multiple servers in a server cluster. Typically, multiple data processing tasks for multiple users are deployed in a server cluster. After the introduction of network devices, multiple data processing tasks can share the aggregated resources of the network devices. That is, when the network device forwards the task message of the data processing task between multiple servers, it performs aggregation operations on the processing data of the data processing task carried in the task message, thereby reducing the amount of data that the end-side server performs the aggregation operation on, so as to alleviate the network congestion caused by the "many-to-one" transmission of the processing data of the data processing task between multiple servers.

[0071] In one exemplary scenario, the data processing task can be a model training task for a machine learning model trained using a distributed training approach. When training a machine learning model, the gradient data obtained by training the model locally on multiple distributed servers needs to be sent to a single server for aggregation to update the model parameters. During this process, network devices can be used to aggregate the forwarded gradient data.

[0072] In another exemplary scenario, the data processing task can be an IoT data processing task. After multiple servers obtain monitoring data from front-end IoT devices, they need to send the monitoring data to a single server for aggregation and analysis. During this process, network devices can be used to perform aggregation operations on the monitoring data forwarded by the servers.

[0073] It should be understood that data processing tasks can also be other tasks that require the use of a "many-to-one" communication mode to transmit and process data. The embodiments of this application only explain the data processing task as a model training task as an example, and do not constitute a limitation on the type of data processing tasks.

[0074] However, since the aggregation resources used to perform aggregation operations in network devices are limited, when the aggregation resources of network devices are unevenly distributed among multiple data processing tasks, multiple data processing tasks may occupy the same network device, causing the aggregation resources of the network device to be unable to meet the aggregation operation requirements of the processing data of multiple data processing tasks, affecting the overall computing performance of the network device and increasing the completion time of the data processing tasks.

[0075] Based on this, an embodiment of the present application provides a task deployment method that can fully consider the processing resources of each server in the server cluster, the aggregated resources of the network devices, and the bandwidth resources of the network devices when deploying data processing tasks in a server cluster, thereby optimizing the deployment method of data processing tasks, so that each data processing task can make full use of the aggregated resources of the network devices, improve resource utilization, and shorten the task completion time.

[0076] Before explaining in detail the task deployment method provided in the embodiment of the present application, the system architecture and implementation environment involved in the embodiment of the present application are first introduced.

[0077] Please refer to Figure 1, which is an architectural diagram of a task management system provided in an embodiment of the present application. The task management system includes: a controller 110, a server cluster 120 and multiple network devices 130. The server cluster 120 includes multiple servers, and each network device 130 includes multiple aggregators obtained by dividing the memory resources of the network device 130.

[0078] It should be noted that the controller manages all devices in the task management system and can communicate with the servers and network devices in the server cluster. The controller can send instructions, data, and information to the network devices, send instructions, data, and information to the servers through the network devices, send instructions, data, and information directly to the servers, and receive data and information from the servers and / or network devices.

[0079] Figure 1 merely illustrates a server cluster comprising eight servers and three network devices, and does not limit the number of devices that may be included in the task management system. Furthermore, each network device may be connected to one or more servers, and this is not a limitation in the present embodiment. The same server may be connected to one or more network devices, and this is not a limitation in the present embodiment.

[0080] In this task management system, when a user submits a data processing task, the controller determines a server in the server cluster capable of deploying the task and a network device capable of performing aggregate operations on the data processed by the task, based on the processing resource usage of each server in the server cluster, as well as the aggregate resource usage and bandwidth resource usage of each network device. The controller then deploys the user-submitted data processing task on the corresponding server and instructs the server to send the processed data obtained from executing the data processing task to the corresponding network device for aggregate operations.

[0081] In some embodiments, the controller obtains the aggregate resource utilization of each network device, the bandwidth resource utilization of each network device, and the processing resource utilization of each server; determines a task deployment strategy and an aggregate resource allocation strategy based on the task running resources required for the first data processing task to be deployed, the aggregate resource utilization and bandwidth resource utilization of multiple network devices, and the processing resource utilization of multiple servers, wherein the task deployment strategy indicates a first server among multiple servers for executing the first data processing task, and the aggregate resource allocation strategy indicates an aggregator in a first network device among multiple network devices for performing aggregate operations on the processed data of the first data processing task; deploys the first data processing task in the first server based on the task deployment strategy and the aggregate resource allocation strategy. After the controller completes the deployment of the first data processing task, the first server is used to store the task data of the first data processing task and the resource information of the aggregate resources allocated to the first data processing task; and the first network device is used to perform aggregate operations on the processed data of the first data processing task.

[0082] In one possible implementation, after determining the task deployment strategy and the aggregate resource allocation strategy, the controller sends task data for a first data processing task and resource information of aggregate resources of a first network device allocated to the first data processing task to a first server. The first server receives and stores the task data and resource information, executes the first data processing task based on the task data, encapsulates the obtained processing data and resource information into a task message, and sends the task message to the first network device, instructing the aggregator assigned to the first data processing task in the first network device to perform an aggregation operation on the processing data carried in the task message.

[0083] The aggregation resource is at least one aggregator in the first network device that can be used to process the processing data of the first data processing task, and the resource information includes the number of aggregators and the first address of the aggregator.

[0084] As an example, see Figure 2. A task message includes message information such as a message header, task identifier, message sequence number, number of aggregators, aggregator first address, and processed data. The message header can carry information such as the protocol type and Internet Protocol (IP) address. The task identifier can be a task identity document (ID) or a task number. For some data processing tasks, when the data to be processed is large or the message carries limited data, the processing data for executing a data processing task can be divided into multiple task messages for transmission. The message sequence number is the sequence number of the currently transmitted task message. The number of aggregators can be one or more. The aggregator first address is the address of the first aggregator.

[0085] In some embodiments, after receiving a task message, the network device uses hash addressing to determine an aggregator that performs aggregation operations on the processed data in the task message.

[0086] As an example, the network device addresses the aggregator through the following formula (1): index = hash (taskID, seqN) % size + addr (1)

[0087] Among them, index is the aggregator determined by addressing, taskID is the task ID of the data processing task, seqN is the message sequence number, size is the number of aggregators, and addr is the first address of the aggregator.

[0088] In other words, the network device does not need to store the aggregator information corresponding to each data processing task. After receiving the task message, it can address the aggregator based on the message information carried in the task message. Among them, the number of aggregators and the aggregator's first address are used to limit the aggregator area that can be used by a data processing task, and the task ID and message sequence number are used to determine an aggregator in the aggregator area corresponding to the data processing task that performs aggregation operations on the task message of the message sequence number. In this way, while ensuring that the processing data of the data processing task is normally aggregated by the first network device, it can also be ensured that the processing data in the task message with the same sequence number can complete the aggregation operation in one aggregator, thereby improving the aggregation flexibility and the accuracy of the aggregation results.

[0089] In some embodiments, for the above-mentioned task management system, the embodiments of the present application do not limit the number of data processing tasks that a user can submit at one time, nor the frequency of task submission. A user can submit one data processing task at a time, or multiple data processing tasks at a time. For ease of explanation, the embodiments of the present application only use the deployment of a first data processing task as an example for explanation, and do not constitute a limit on the number of data processing tasks that the controller can deploy at one time.

[0090] As an example, if the user submits multiple data processing tasks, the controller can determine the aggregate resource utilization and bandwidth resource utilization of multiple network devices and the processing resource utilization of multiple servers. It can then determine the task deployment strategy and aggregate resource allocation strategy for each data processing task in turn based on the task running resources required for each data processing task, thereby deploying the multiple data processing tasks in the server cluster in turn; it can also combine the task running resources required for each data processing task in the multiple data processing tasks to uniformly coordinate the deployment of multiple data processing tasks, thereby deploying multiple data processing tasks in batches to the server cluster.

[0091] In some embodiments, in addition to traditional forwarding and routing functions, the network devices in the above-mentioned task management system also need to have certain programmable and caching capabilities, be able to identify messages transmitted within the network (including data messages, control messages and other messages), calculate and modify fields in the messages, and have certain caching capabilities.

[0092] As an example, the network device may be a programmable switch, which may be a top-of-rack switch (TOR) or a spine switch (also known as a spine switch). The present embodiment of the application does not limit the type of programmable switch. Furthermore, the network device that performs the aggregation operation may be a TOR switch or a spine switch, which is also not limited in the present embodiment.

[0093] It should be noted that when the TOR switch is connected to multiple servers in the cabinet, the Spine switch can be directly connected to multiple TOR switches to form a two-layer network architecture including a cabinet layer and a Spine layer. Based on the two-layer network architecture, the processed data transmitted between multiple servers in the server cluster is aggregated in the network. A leaf layer (also called a Leaf layer) can also be added between the cabinet layer and the Spine layer. The Leaf layer includes multiple leaf switches, each leaf switch is connected to multiple TOR switches in the cabinet layer, and multiple leaf switches are connected to a Spine switch, that is, a three-layer network architecture including a cabinet layer, a Leaf layer, and a Spine layer is constructed. Based on the three-layer network architecture, the processed data transmitted between multiple servers in the server cluster is aggregated in the network. The embodiment of the present application does not limit the network hierarchy formed by the Spine switch and the TOR switch.

[0094] Furthermore, the embodiments of the present application do not restrict the number of aggregators in each network device, nor the amount of memory resources corresponding to each aggregator. In other words, the number of aggregators in multiple network devices can be the same or different, and the amount of memory resources corresponding to each aggregator in a network device can be the same or different.

[0095] In some embodiments, the controller may be any computer device with communication, data calculation and storage capabilities. Referring to FIG. 3 , the computer device includes at least one processor 301 , a communication bus 302 , a memory 303 and at least one communication interface 304 .

[0096] The processor 301 may be a general-purpose central processing unit (CPU), a network processor (NP), a microprocessor, or one or more integrated circuits for implementing the solution of the present application, such as an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0097] Communication bus 302 is used to transmit information between the above components. Communication bus 302 can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or one type of bus.

[0098] The memory 303 may be a read-only memory (ROM), a random access memory (RAM), an electrically erasable programmable read-only memory (EEPROM), an optical disc (including a compact disc read-only memory (CD-ROM), a compact disc, a laser disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0099] The memory 303 may exist independently and be connected to the processor 301 via the communication bus 302. The memory 303 may also be integrated with the processor 301.

[0100] The communication interface 304 uses any device, such as a transceiver, for communicating with other devices or communication networks. The communication interface 304 includes a wired communication interface and may also include a wireless communication interface. For example, the wired communication interface may be an Ethernet interface. The Ethernet interface may be an optical interface, an electrical interface, or a combination thereof. The wireless communication interface may be a wireless local area network (WLAN) interface, a cellular network communication interface, or a combination thereof.

[0101] As an example, the processor 301 may include one or more CPUs, such as CPU0 and CPU1 shown in FIG. 3 .

[0102] As an example, a computer device may include multiple processors, such as processor 301 and processor 305 shown in FIG3 . Each of these processors may be a single-core processor or a multi-core processor. A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0103] In some embodiments, the computer device may further include an output device and an input device (not shown in FIG3 ). The output device communicates with the processor 301 and can display information in a variety of ways. For example, the output device can be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector. The input device communicates with the processor 301 and can receive user input in a variety of ways. For example, the input device can be a mouse, a keyboard, a touch screen device, or a sensor device.

[0104] In some embodiments, the memory 303 is used to store program code for executing the solution of the present application, and the processor 301 can execute the program code stored in the memory 303. The program code may include one or more software modules, and the computer device can implement the task deployment method provided in any of the following embodiments through the processor 301 and the program code in the memory 303.

[0105] FIG4 is a flowchart of a task deployment method provided by an embodiment of the present application, which is applied to the controller 110 shown in FIG1 above. Referring to FIG4 , the method includes the following steps:

[0106] Step 401: The controller obtains the aggregate resource utilization of each network device, the bandwidth resource utilization of each network device, and the processing resource utilization of each server.

[0107] Among them, the aggregate resource utilization rate of the network device can be expressed as the ratio between the aggregate resources of the occupied aggregators in the network device and the total memory resources of the network device (that is, the aggregate resources of all aggregators contained in the network device). By analyzing the aggregate resource utilization rate of the network device, the available aggregate resources of the unoccupied aggregators in the network device (that is, the resources that can be used for other data processing tasks) can be determined; similarly, the processing resource utilization rate of the server can be expressed as the ratio between the occupied processing resources and the total processing resources in the server. By analyzing the processing resource utilization rate of the server, the available processing resources in the server (that is, the processing resources that can be used to deploy other data processing tasks) can be determined. Each network device includes multiple ports, each of which can be connected to a server to transmit messages with the server through the port. Therefore, based on the data traffic of each port in the network device, the utilization rate of the bandwidth resources allocated to each port by the network device can be determined.

[0108] In some embodiments, the controller may obtain the processing resource utilization of the server by: the controller stores the total processing resources of each server in the server cluster, and records and updates the processing resource utilization of each server according to the deployment of data processing tasks in the task management system.

[0109] For example, when a server in a server cluster is newly added and no data processing tasks are deployed, the controller records the server's processing resource utilization as 100%. After a data processing task is deployed on the server, the controller calculates and updates the server's processing resource utilization based on the task execution resources required by the data processing task and the server's total processing resources.

[0110] In some embodiments, the controller may obtain the bandwidth resource utilization of the network device by setting a counter at each port of the network device, and using the counter to count the total number of packets and the total length of the packets passing through the port over a period of time. The controller reads the total number of packets and the total length of the packets counted by the counters of each port, and calculates the bandwidth resource utilization of each port by dividing the total length of the packets by the length of the statistical period. Based on the bandwidth resource utilization of each port, the controller then determines the overall bandwidth resource utilization of the network device.

[0111] It should be noted that when the controller obtains the processing resource utilization of the server and the bandwidth resource utilization of the network device, it can be achieved through the above method, or it can be achieved by referring to other related technologies. The embodiments of the present application do not limit this.

[0112] In some embodiments, the controller may obtain the aggregate resource utilization of the network device through the following steps (1)-(3):

[0113] (1) The controller sends a measurement message to the network device. The measurement message carries period information of a first measurement period. The first measurement period is the measurement period in which the deployment time of the first data processing task is located.

[0114] The period length of the measurement period may be any set value, such as 5 minutes, 10 minutes, 20 minutes, etc. In different situations, the period length of the measurement period may be different.

[0115] Typically, the controller obtains the aggregate resource utilization of each network device, the bandwidth resource utilization of each network device, and the processing resource utilization of each server based on task deployment requirements, that is, when there is a data processing task to be deployed. Therefore, the controller obtains the aggregate resource utilization of the network device through steps (1)-(3), which must be completed within the same measurement cycle as the deployment process of the first data processing task.

[0116] That is, if a first data processing task to be deployed exists in the task management system within the first measurement period, the controller obtains the aggregate resource utilization of the network device, the bandwidth resource utilization of the network device, and the processing resource utilization of the server, and then deploys the first data processing task within the first measurement period. In this case, the deployment time of the first data processing task falls within the first measurement period.

[0117] Based on this, when the controller needs to deploy the first data processing task within the first measurement period, the period information carried in the measurement message sent by the controller to the network device is the period information of the first measurement period.

[0118] In this way, when the deployment time of the first data processing task is within the measurement period in which the controller executes the above step 401, the resource utilization obtained by the controller can reflect the actual usage of resources of multiple servers and multiple network devices within the measurement period. The task deployment strategy and aggregated resource allocation strategy determined based on the resource utilization within the measurement period are also more accurate, and the task deployment effect is better.

[0119] It should be understood that when there is no task deployment requirement, the controller may also periodically perform step 401 above, i.e., obtain and record the aggregate resource utilization of each network device, the bandwidth resource utilization of each network device, and the processing resource utilization of each server during each measurement period. Specifically, the controller only counts and stores the resource utilization of each device and updates the stored resource information after obtaining new data information in the next measurement period.

[0120] It's important to note that when the controller sends measurement messages to network devices, it doesn't know the address information or communication links of each network device. However, the controller stores the address information of each server in the server cluster, as well as the network devices connected to each server. Furthermore, the fundamental function of network devices is to forward data between servers in the server cluster. Therefore, based on the communication link between the controller and the server, the controller can send measurement messages to network devices in that communication link.

[0121] In one possible implementation, the implementation process of the controller sending a measurement message to the network device can be: the controller determines a target server from multiple servers, where the target server refers to a server connected to the network device, and the network device is located on the communication link between the controller and the target server; the controller sends a measurement message to the target server through the communication link between the controller and the target server, so that the network device receives and intercepts the measurement message.

[0122] For multiple servers connected to a network device, the controller can select a target server from the multiple servers based on the length of the communication link between the controller and the server, the operating status of the server, etc.; or it can randomly select a server from the multiple servers as the target server. This embodiment of the application does not limit the target server selection strategy.

[0123] It should be noted that the controller may send measurement messages to multiple network devices through the above step (1), or may send measurement messages only to the network device that performs aggregation operations among the multiple network devices. This embodiment of the present application does not impose any restrictions on this.

[0124] Assuming that the multiple network devices in the task management system include spine switches and TOR switches, and that the network device performing the aggregation operation is a TOR switch, the controller needs to send a measurement message to each TOR switch when obtaining the aggregated resource utilization of the network devices. Taking the first TOR switch as an example, the controller first selects server a as the target server from among servers a, b, and c connected to the first TOR switch. The controller writes server a's IP address as the destination IP address into the measurement message header and sends the measurement message to server a via the communication link between the controller and server a.

[0125] It should be understood that when multiple network devices exist on the communication link between the controller and the target server, upon receiving a measurement message, each network device will determine, based on the IP address carried in the message header, whether the target node indicated by the IP address is a directly connected node to itself. If the target node is a directly connected node, the measurement message will be intercepted; if not, the measurement message will be forwarded to the next node based on the communication link between the controller and the target server.

[0126] Continuing with Figure 5, upon receiving the measurement message, the Spine switch determines that the destination IP address carried in the measurement message header is not its directly connected node, and then forwards the measurement message to the first TOR switch. Upon receiving the measurement message, the first TOR switch determines that the destination IP address is the IP address of server a, to which it is directly connected, and then intercepts the measurement message.

[0127] In some embodiments, the network device that performs aggregation operations can also be a Spine switch. When the Spine switch receives the above-mentioned measurement message, it determines that the server indicated by the destination IP carried in the message header of the measurement message is connected to any TOR switch to which it is connected, and then intercepts the measurement message.

[0128] (2) The network device receives the measurement message, determines the number of occupied aggregators of the network device in the first measurement period based on the period information of the first measurement period, and sends a measurement reply message to the controller, where the measurement reply message carries the number of occupied aggregators.

[0129] The period information of the first measurement period is a version number corresponding to the first measurement period, and the network device is provided with a first state counter having the first version number. After the network device receives the measurement message, in response to the version number corresponding to the first measurement period being the same as the first version number, the network device determines the number of occupied aggregators of the network device during the first measurement period based on the count value of the first state counter.

[0130] As an example, a measurement message includes a message header, a version number (ver) corresponding to the measurement period, and an empty count value field. The count value field is used for the network device to determine the number of occupied aggregators, fill the number of occupied aggregators into the count value field, and then generate a measurement reply message.

[0131] Referring to FIG. 6 , in this embodiment, the version number in the measurement message is version=2 corresponding to the first measurement period, and the value of the count value field can be expressed as cnt=0.

[0132] In other words, the first state counter is used to record the number of occupied aggregators in the network device during the current measurement period and to update the first version number based on changes in the measurement period. During the first measurement period, whenever a task message is transmitted to the network device and an aggregator in the network device is required to perform an aggregation operation on the processed data carried in the task message, the count value of the first state counter is automatically incremented by one.

[0133] As an example, referring to FIG7 , assuming that the version number corresponding to the first measurement cycle is ver.2, and the first version number of the first state counter is ver.2, for the first task message of task 3 (i.e., the task ID is: task3, and the message sequence number is seq1), after the network device receives the task message pkt(task3, seq1), it is mapped to aggregator No. 4 in the network device by the method of “hash(task3, seq1)%”. At this time, aggregator No. 4 is empty (i.e., the aggregation resources are in an idle state and no aggregation operation is performed), then the task message pkt(task3, seq1) can occupy aggregator No. 4 to perform aggregation operations on the processed data carried in the task message pkt(task3, seq1) through the aggregation resources of aggregator No. 4. At this time, the count value cnt of the first state counter is automatically increased by one, that is, the count value is updated from 4 to 5.

[0134] It should be understood that if the addressed aggregator is not empty, it means that the aggregator is currently performing aggregation operations, and its aggregation resources are occupied by processing data of other data processing tasks, and the counter of the first state counter does not increment.

[0135] When the first state counter reaches 5, it indicates that five aggregators in the network device are occupied. In this case, after receiving the measurement message, the network device reads the first state counter value of 5 based on the first measurement cycle period information ver. 2 and feeds the count value back to the controller via a measurement reply message.

[0136] 8 , the measurement reply message includes a message header, a version number corresponding to the first measurement cycle (version=2), and the number of occupied aggregators (cnt=5) written into the count value field.

[0137] (3) The controller receives the measurement reply message and determines the aggregation resource utilization of the network device based on the number of occupied aggregators.

[0138] In one possible implementation, the controller may determine the aggregation resource utilization of a network device by: determining the maximum number of aggregations of the aggregators included in the network device within the first measurement period based on the total number of aggregators included in the network device, the single occupancy duration of the aggregators of the network device within the first measurement period, and the period length of the first measurement period; and determining the aggregation resource utilization of the network device based on the number of occupied aggregators and the maximum number of aggregations.

[0139] Task messages occupy the aggregator's aggregation resources. After the aggregation operation is completed on the processed data contained in the task message, the aggregator's aggregation resources are released, and the aggregator can now process task messages for other data processing tasks. Therefore, for an aggregator, its single occupation duration is the time it takes for the network device to receive a task message and complete the aggregation operation on the processed data in the task message.

[0140] Taking the end-to-end communication of the server as an example, the implementation process of determining the single occupancy time of the aggregator can be as follows: for any server, the moment when the server sends the task message to the network device is taken as the starting time, and the moment when the server receives the task reply message fed back by the network device is taken as the ending time. Based on the time length between the start time and the end time, the single occupancy time of the aggregator is determined.

[0141] That is, since the aggregator in the network device needs to perform aggregation operations on the processed data in the task message, the aggregator is in an occupied state when processing the task message. Therefore, the single occupancy time of the aggregator can also be expressed as the end-to-end round-trip time (RTT) of the server.

[0142] As an example, referring to Figure 9, server p sends a task message pkt(task1, seq1) to server q via a network device at 10:00. After hashing, the network device uses aggregator #1 to perform an aggregation operation on the processed data contained in task message pkt(task1, seq1) and sends the intermediate result to server q. After receiving the intermediate result of task pkt(task1, seq1), server q sends a task reply message ack(task1, seq1) corresponding to task message pkt(task1, seq1) to server p via the network device. Server p receives the task reply message ACK(task1, seq1) at 10:01, indicating that server q has accurately received task message pkt(task1, seq1). In this case, the single occupancy duration of aggregator #1 in the network device is the end-to-end round-trip time between server p and server q, which is 1 minute.

[0143] It should be noted that since the network device includes multiple aggregators, the single occupancy time of the aggregator included in the network device can be the average of the single occupancy time of multiple aggregators, or the maximum value of the single occupancy time of multiple aggregators. The embodiments of the present application do not impose any restrictions on this.

[0144] In one possible implementation, the implementation process of determining the maximum number of aggregation times of the aggregators included in the network device within the first measurement period may be: determining the number of occupancy times of each aggregator within the first measurement period based on the period length of the first measurement period and the single occupancy duration of the aggregators of the network device within the first measurement period; determining the maximum number of aggregation times of the network device within the first measurement period based on the number of occupancy times of each aggregator within the first measurement period and the total number of aggregators included in the network device.

[0145] As an example, the maximum number of aggregators of the network device in the first measurement period can be calculated using the following formula (2).

[0146] Wherein, T is the period length of the first measurement period, t is the average single occupancy duration of multiple aggregators included in the network device during the first measurement period, and N is the total number of aggregators included in the network device; T, t, and N are all non-zero numbers.

[0147] The number of occupied aggregators can reflect the actual number of aggregations of the network device. Therefore, based on the number of occupied aggregators of the network device in the first measurement period and the maximum number of aggregations that the aggregators included in the network device can achieve in the first measurement period, the aggregation resource utilization of the network device in the first measurement period is determined.

[0148] As an example, the aggregate resource utilization of the network device in the first measurement period can be calculated using the following formula (3).

[0149] Where n is the number of occupied aggregators of the network device in the first measurement period, t is the average single occupancy duration of multiple aggregators included in the network device in the first measurement period, N is the total number of aggregators included in the network device, and T is the period length of the first measurement period; n, t, N and T are all non-zero numbers.

[0150] In some embodiments, after executing step (2), the controller may not receive a measurement reply message within the measurement duration, i.e., the measurement reply message sent by the network device to the controller is lost. In this case, the controller may resend the measurement message to the network device, i.e., repeat steps (1) and (2) to obtain the number of aggregators occupied by the network device during the first measurement period.

[0151] When a network device receives a measurement message, it must determine whether the measurement reply message is the first message sent within the first measurement cycle or a retransmission. The number of occupied aggregators carried in the measurement reply message sent back by the network device should be the same for all measurement messages sent by the controller within the same measurement cycle.

[0152] That is, during the first measurement cycle, the measurement message sent by the controller to the network device carries the period information of the first measurement cycle. However, after determining that the measurement reply message is lost, the controller resends the measurement message to the network device, that is, the resent measurement message still carries the period information of the first measurement cycle. However, since the number of occupied aggregators of the network device may change during the interval between the two measurement messages sent by the controller, the count value of the first state counter that counts the number of occupied aggregators in the current measurement cycle also changes. If the network device continues to read the count value of the first state counter and feeds it back to the controller, the aggregate resource utilization of the network device obtained by the controller, the bandwidth resource utilization of each network device, and the processing resource utilization of each server will be out of sync in time, affecting the task deployment effect and the aggregate resource allocation effect.

[0153] To avoid situations where a network device determines different numbers of occupied aggregators for the same measurement message, the network device can set up two counters in different states, one for recording the number of occupied aggregators in the current measurement cycle and the other for the previous measurement cycle. By switching the states of the two counters and updating the version numbers of the counters, only one state is used to record the number of occupied aggregators in each measurement cycle, and the states of the counters recording the number of occupied aggregators in consecutive measurement cycles are different.

[0154] In one possible implementation, a first state counter in the network device further includes first state information, and a second state counter is further provided in the network device, the second state counter including second state information and a second version number. After the network device sends a measurement reply message to the controller, in response to the first state information indicating the current measurement cycle and the second state information indicating the previous measurement cycle, the network device switches the first state information to the second state information, updates the second version number to the version number corresponding to the second measurement cycle, and clears the count value of the second state counter, thereby recording the number of occupied aggregators in the second measurement cycle via the second state counter. The second measurement cycle is the next measurement cycle adjacent to the first measurement cycle.

[0155] Referring to Figure 10, assuming that the cycle information of the first measurement cycle is ver2, the first state information of the first state counter in the network device is the current cycle state, and the corresponding first version number is ver2, that is, the first state counter records the number of occupied aggregators in the network device during the first measurement cycle; the second state information of the second state counter is the previous cycle state, and the second version number is ver1, that is, the second state counter records the number of occupied aggregators in the network device during the previous measurement cycle (the cycle information is ver1).

[0156] After receiving a measurement message carrying the ver2 version number, the network device compares the version number ver2 carried in the measurement message with the first version number of the first state counter and the second version number of the second state counter, and reads the count value cnt=5 of the first state counter which is the same as the version number ver2 in the measurement message, so as to feedback to the controller through a measurement reply message that the number of occupied aggregators of the network device in the first measurement cycle is 5.

[0157] After the network device sends a measurement reply message to the controller, it switches between the first state information and the second state information. Specifically, the first state counter is switched to the previous cycle state, so that the first state counter records the number of occupied aggregators in the previous measurement cycle (i.e., the first measurement cycle in which the measurement reply message was sent). The version number of the first state counter remains ver2. The second state counter is switched to the current cycle state. At the same time, the second version number of the second state counter is updated to ver3, and the value of the second state counter is cleared. The second state counter records the number of occupied aggregators in the network device in the next measurement cycle.

[0158] In this case, if the controller does not receive the measurement reply message, it resends the measurement message carrying the ver2 version number. Referring to Figure 11, when the network device receives the measurement message carrying the ver2 version number, it determines that the measurement message is the same as the version number of the first state counter, and at this time the state information of the first state counter is the state of the previous cycle, then it determines that the measurement message is a retransmission message. According to the version number, the network device reads the count value cnt=5 of the first state counter to feedback to the controller through the measurement reply message that the number of occupied aggregators of the network device in the first measurement cycle is 5. That is, regardless of whether it is the first measurement message or the retransmission of the measurement message, the number of occupied aggregators in the first measurement cycle counted by the network device is 5.

[0159] In this way, the network device can effectively identify whether the measurement message is the first message or the retransmission message based on the counters in two states, solving the problem of counting the number of occupied aggregators in the network device in the case of packet loss and retransmission, so that for the same measurement message, the number of occupied aggregators carried in the measurement reply message fed back by the network device is the same.

[0160] Step 402: The controller determines a task deployment strategy and an aggregate resource allocation strategy based on the task running resources required for the first data processing task to be deployed, the aggregate resource utilization and bandwidth resource utilization of multiple network devices, and the processing resource utilization of multiple servers. The task deployment strategy indicates a first server among the multiple servers for executing the first data processing task, and the aggregate resource allocation strategy indicates an aggregator in the first network device among the multiple network devices for performing aggregate operations on the processed data of the first data processing task.

[0161] In some embodiments, the implementation process of step 402 can be: determining the deployment constraints of the first data processing task based on the task running resources, the aggregate resource utilization and bandwidth resource utilization of multiple network devices, and the processing resource utilization of multiple servers; and determining the task deployment strategy and the aggregate resource allocation strategy based on the deployment constraints.

[0162] That is, the controller determines the first server on which to deploy the first data processing task and the aggregator in the first network device that performs aggregation operations on the processed data of the first data processing task from the multiple servers in the server cluster based on the task running resources required for the first data processing task to be deployed, the aggregate resource utilization and bandwidth resource utilization of multiple network devices, and the processing resource utilization of multiple servers.

[0163] It should be understood that once networking is complete, the communication routes between the network devices and the server are determined. Therefore, after determining the first server, the first network device connected to the first server can be further determined. The aggregation resource allocation strategy then selects an aggregator from among the multiple aggregators included in the first network device to perform aggregation operations on the processed data of the first data processing task.

[0164] It should be noted that the first data processing task can be deployed on a single server or on multiple servers for parallel execution. This embodiment of the present application does not limit the number of first servers on which the first data processing task can be deployed. Similarly, this embodiment of the present application does not limit the number of aggregators in the first network device that perform aggregation operations on the processed data of the first data processing task.

[0165] Deployment constraints include:

[0166] (1) The task execution resources are less than or equal to the available processing resources of the server capable of executing the first data processing task.

[0167] The server's available processing resources indicate whether the server can continue to deploy data processing tasks. When the task running resources of a data processing task are less than or equal to the server's available processing resources, it means that the server can meet the data processing task's running resource requirements, and the data processing task can be deployed on the server. If the task running resources of a data processing task are greater than the server's available processing resources, it means that the server cannot meet the data processing task's running resource requirements, and the data processing task cannot be deployed on the server.

[0168] That is, for any server in the server cluster, if the available processing resources of the server can meet the running resource requirements of the first data processing task, the first data processing task can be considered to be deployed on the server.

[0169] (2) The processing resource usage of the server capable of executing the first data processing task is less than or equal to the processing resource threshold.

[0170] The processing resource threshold is used to indicate whether the server is in an overloaded state. When the processing resource usage of the server capable of executing the first data processing task is less than or equal to the processing resource threshold, it means that even if the first data processing task is deployed on the server, the server can still operate normally and there is no possibility of downtime due to overload operation; when the processing resource usage of the server capable of executing the first data processing task is greater than the processing resource threshold, it means that after deploying the first data processing task on the server, the server may be overloaded, making it difficult to effectively execute the data processing task deployed therein, and there may even be a situation where the server downtime due to overload operation.

[0171] That is, after determining that the server meets the operating resource requirements of the first data processing task based on the above deployment constraint (1), it is also necessary to ensure that after the first data processing task is deployed on the server, the server will not be overloaded. In other words, the processing resource usage of the server where the first data processing task is deployed must not exceed the processing resource threshold of the server.

[0172] It should be noted that the above deployment constraints (1) and (2) are used to screen servers from the server cluster that can execute the first data processing task, and then further screen servers that meet the requirements of the first data processing task through the following deployment constraints (3) and (4), as well as network devices that can perform the processing data aggregation operation obtained by executing the first data processing task on the server.

[0173] (3) The bandwidth resource between the server capable of executing the first data processing task and the network device capable of performing aggregation operations on the processed data of the first data processing task is less than or equal to the bandwidth threshold.

[0174] The bandwidth threshold is used to indicate bandwidth resources where data packet loss may occur. When the bandwidth resource is less than or equal to the bandwidth threshold, it indicates that the amount of data transmitted on the communication link between the server and the network device is small, and the bandwidth resources configured for the communication link can meet the actual data transmission requirements of the first data processing task, and the processed data of the first data processing task can be transmitted normally between the server and the network device; when the bandwidth resource is greater than the bandwidth threshold, it indicates that the amount of data transmitted on the communication link between the server and the network device is large, and the bandwidth resources configured for the communication link cannot meet the actual data transmission requirements of the first data processing task, and data packet loss may occur when transmitting the processed data of the first data processing task based on the communication link.

[0175] In other words, since the server that deploys the first data processing task needs to send the processed data to the corresponding network device for aggregation calculation after executing the first data processing task, when the bandwidth resources for transmitting the processed data between the server and the network device are less than or equal to the bandwidth threshold, the server that deploys the first data processing task can communicate normally with the network device that performs aggregation calculation on the processed data of the first data processing task.

[0176] (4) The memory resource usage of the network device capable of performing aggregation operations on the processing data of the first data processing task is less than or equal to the memory resource threshold.

[0177] The memory resource threshold indicates whether the network device is overloaded. When the network device's memory resource usage is less than or equal to the threshold, the device's memory resources can aggregate all the processed data sent by the server, and there is no overload. When the network device's memory resource usage is greater than the threshold, the device's memory resources are insufficient to aggregate all the processed data sent by the server. Some data will be returned to the server for aggregation, or the computation will be delayed, affecting the task's runtime.

[0178] That is, when the server capable of executing the first data processing task is connected to multiple network devices, it is necessary to determine which network device to send the processed data of the first data processing task to for aggregation calculation based on the memory resource usage of each network device.

[0179] In one possible implementation, the implementation process of determining the task deployment strategy and the aggregate resource allocation strategy based on the deployment constraints may be: based on the deployment constraints, determining at least one candidate deployment scheme that can execute the first data processing task, the candidate deployment scheme including a candidate server and a candidate network device; based on preset scheme screening conditions, determining the first server for deploying the first data processing task from the candidate servers included in the at least one candidate deployment scheme, and determining the first network device for performing aggregation operations on the processed data of the first data processing task from the candidate network devices connected to the first server; and determining an aggregator in the first network device for performing aggregation operations on the processed data of the first data processing task based on the task running resources of the first data processing task.

[0180] The solution screening condition is that the candidate deployment solution has the shortest task completion time. That is, for the candidate servers and candidate network devices included in the candidate deployment solution, if the total time it takes for the first data processing task to be deployed on the candidate server, the candidate server to execute the first data processing task, transmit the processed data of the first data processing task to the candidate network device, and the candidate network device to perform aggregation operations on the processed data is the shortest, the candidate server in the candidate deployment solution is determined as the first server, and the candidate network device is determined as the first network device.

[0181] It should be understood that, by determining the candidate deployment schemes based on the above deployment constraints (1)-(4), when the number of candidate deployment schemes is 1, the candidate server in the candidate deployment scheme is used as the first server for executing the first data processing task, and the candidate network device in the candidate deployment scheme is used as the first network device for performing an aggregation operation on the processed data of the first data processing task. When the number of candidate deployment schemes is not 1, that is, when there are multiple candidate deployment schemes, the first server for executing the first data processing task and the aggregator in the first network device for performing an aggregation operation on the processed data of the first data processing task are determined based on the scheme screening conditions and the multiple candidate deployment schemes.

[0182] Optionally, during implementation, the above-mentioned solution screening conditions may be used as optimization objectives. Then, based on the optimization objectives and the above-mentioned deployment constraints, a modeling approach may be employed to determine the task deployment strategy and the aggregate resource allocation strategy. For example, the task deployment strategy and the aggregate resource allocation strategy for deploying the first data processing task may be determined based on integer linear programming (ILP).

[0183] Step 403: The controller deploys the first data processing task in the first server based on the task deployment strategy and the aggregated resource allocation strategy.

[0184] In some embodiments, step 403 may be implemented as follows: the controller sends the task data of the first data processing task and the resource information of the aggregated resources allocated to the first data processing task to the first server, wherein the resource information of the aggregated resources includes the number of aggregators and the first address of the aggregator.

[0185] Based on this, the first server stores the task data and resource information of the first data processing task. In response to the task execution instruction, the first data processing task is executed to obtain processed data. The processed data, the number of aggregators for the first data processing task, and the aggregator's first address are encapsulated into a task message, which is then sent to the first network device. After receiving the task message, the first network device determines the aggregator within the first network device based on the number of aggregators and the aggregator's first address for the first data processing task, and the aggregator performs aggregation operations on the processed data carried in the task message.

[0186] In an embodiment of the present application, when deploying a first data processing task in a task management system, the controller can comprehensively analyze the aggregate resource utilization of each network device, the bandwidth resource utilization of each network device, the processing resource utilization of each server, and the task computing resources required for the first data processing task, thereby determining the task deployment strategy and aggregate resource allocation strategy for the first data processing task. Because the server's processing resources, the network device's bandwidth resources, and the aggregate resources are fully considered, task deployment and resource allocation are more reasonable, improving the overall execution performance of the first data processing task and reducing the task completion time.

[0187] Based on the above task deployment method, since multiple data processing can share the aggregator of the network device, that is, the aggregator of a network device may need to perform aggregation operations on the processing data of multiple data processing tasks, for low-priority data processing tasks, the aggregation resources of the aggregator are used in a preemptive manner; for high-priority data processing tasks, the aggregation resources of the aggregator are used in an exclusive manner.

[0188] Based on this, the aggregators contained in the network equipment can be divided into two areas: the isolation area and the shared area. The aggregation resources of the aggregators in the isolation area are exclusively used by high-priority data processing tasks and are only used to process the processing data of the high-priority data processing tasks; if no high-priority data processing tasks are deployed in the task management system at the current moment, then the aggregators in the network equipment belong to the shared area to be preempted by multiple low-priority data processing tasks, that is, the low-priority data processing tasks use the aggregator addressing method to preempt the aggregators in the shared area, so as to perform aggregation operations on the processing data of the low-priority data processing tasks through the preempted aggregators. The processing data of the low-priority data processing tasks that have not preempted the aggregators are returned to the server for aggregation operations.

[0189] It's important to note that isolated zones are dynamic. Whether a network device has an aggregator located within an isolated zone depends on whether the task management system to which the network device belongs has deployed data processing tasks that require exclusive use of an aggregator. In other words, a network device can include aggregators in isolated zones or only aggregators in shared zones. Aggregators in a network device are divided into isolated and shared zones only when a data processing task requiring exclusive use of an aggregator exists in the task management system.

[0190] Moreover, here we only use the priority of the data processing task to illustrate the aggregator that can be used by the data processing task, but in actual applications, other methods can also be used to determine whether the data processing task requires an exclusive aggregator or a shared aggregator. The embodiments of this application do not limit this.

[0191] In addition, the server that performs data processing tasks is unaware of the isolation area and shared area in the network device. It only needs to send the task message to the network device according to the resource information of the allocated aggregated resources, and the network device will map the task message to the corresponding aggregator.

[0192] As an example, referring to FIG12 , it is assumed that task1 is a data processing task that exclusively occupies the aggregator, and it exclusively occupies aggregator No. 0 and aggregator No. 1 in the shared area, and uses the aggregation resources of aggregator No. 0 and aggregator No. 1 to perform aggregation operations on the processed data of task1; task2 and task3 are data processing tasks that share the aggregator, and they use aggregator addressing to seize the aggregator from the shared area, that is, they use the aggregation resources of aggregator No. 2 to aggregator No. 7 to perform aggregation operations on the processed data of task2 and task3.

[0193] It should be noted that Figure 12 only takes task2 deployed on server 1 and server 2 and sending three task messages to the network device in sequence as an example, and does not limit the task running order of task2 and task3, nor the specific preempted aggregator.

[0194] When a new data processing task, task4, is added to the network device, the aggregator in the shared zone will be occupied by task4. When task4 is a data processing task that shares an aggregator, tasks2, task3, and task4 preemptively use the aggregator in the shared zone. When task4 is a data processing task that exclusively uses an aggregator, task4 needs to determine at least one aggregator that it exclusively owns from the aggregators in the shared zone so that the aggregator it exclusively owns can perform aggregation operations on task4's processed data. Task2 and task3 can no longer preempt the aggregator exclusively used by task4, resulting in fewer aggregators originally available to tasks2 and task3.

[0195] That is, when the data processing task task4 that needs an exclusive aggregator needs to use the aggregation resources of the network device, it will determine its own exclusive aggregator from the shared area of ​​the network device, causing the aggregators of other data processing tasks that use the shared area aggregators in the network device to change, and the corresponding aggregator addressing method also needs to change.

[0196] As shown in Figure 13, if task 4 occupies aggregators 2 and 3 in the shared zone originally belonging to tasks 2 and 3, the aggregators available to tasks 2 and 3 in the shared zone will become aggregators 4 through 7. When multiple servers executing tasks 2 and 3 send task messages to the network device, if the aggregator switching occurs at different times, task messages with the same sequence number may be addressed to different aggregators.

[0197] For example, for servers 1 and 2 deploying task 2, server 1 switches slowly and still uses the old addressing, selecting aggregator 3 to aggregate the processed data carried in message 3. Server 2 switches quickly and uses the new addressing to select aggregator 5 to aggregate the processed data carried in message 3. At this point, one message 3 from task 2 is mapped to aggregator 3 and the other to aggregator 5. The same task message occupies two different aggregators, resulting in the data in these two task messages never being aggregated, and the aggregation resources of the corresponding aggregators cannot be released.

[0198] Therefore, in order to ensure the accuracy of the aggregation operation and the aggregation resource utilization of the aggregator in the network device, when a new data processing task that exclusively uses the aggregator is added to the network device, the aggregator must be reallocated for the data processing task of the aggregator using the shared area, and the servers corresponding to the data processing tasks of the aggregator using the shared area must be instructed to perform the aggregator re-addressing operation at the same time.

[0199] Referring to FIG. 14 , when the first data processing task is a data processing task that requires exclusive use of an aggregator, the task deployment method further includes the following steps:

[0200] Step 404: The controller determines the deployed second data processing task. The processing data of the second data processing task and the first data processing task both need to be aggregated in the first network device, and the second data processing task refers to a data processing task that can share an aggregator with other data processing tasks.

[0201] That is, the second data processing task is a data processing task that uses the aggregation resources of the aggregator in the shared area of ​​the first network device. When the first data processing task needs to exclusively occupy at least one aggregator from the shared area of ​​the first network device, it will cause the aggregator that the second data processing task can seize in the shared area to change.

[0202] It should be noted that, before the first data processing task is deployed, the aggregator in the shared area of ​​the first network device can be used to perform aggregation operations on the processing data of a single data processing task, or can also perform aggregation operations on the processing data of multiple data processing tasks, and this embodiment of the present application does not limit this. That is, the number of second data processing tasks can be one or more, and this embodiment of the present application only uses one as an example for explanation. When there are multiple data processing tasks, steps 404-406 can also be used to perform a one-time batch execution of the aggregator addressing reconfiguration operation.

[0203] Step 405: The controller sends a task suspension message to the second server, where the second server is a server among the multiple servers that executes the second data processing task.

[0204] Among them, the task pause message is used to instruct the second server to suspend sending task messages after the target sequence number of the second data processing task, so as to avoid the situation where an exclusive aggregator is allocated to the first data processing task in the shared area of ​​the first network device, resulting in the processing data carried in the task message sent by the second server being unable to be effectively aggregated by the aggregator.

[0205] After receiving the task suspension message and sending the task message of the target sequence number of the second data processing task, the second server suspends sending the task message of the second data processing task to the first network device and sends a suspension response message to the controller.

[0206] In some embodiments, for a distributed deployment task, the second server includes a parameter server and multiple computing servers. The parameter server is used to store task parameters of the second data processing task, and each computing server is used to execute the second data processing task based on the task parameters. Based on this, the implementation process of pausing the second data processing task may include the following steps (1)-(4):

[0207] (1) The controller sends a task pause message to the parameter server, so that the parameter server controls the multiple computing servers to pause sending task messages after the target sequence number of the second data processing task.

[0208] The pause message carries the task identifier of the second data processing task, which is used to indicate that the data processing task that the parameter server currently needs to pause is the second data processing task.

[0209] (2) The parameter server receives the task pause message, determines the maximum message sequence number that has completed the aggregation operation at the current moment, and sends the maximum message sequence number to multiple computing servers.

[0210] After receiving the task suspension message, the parameter server determines, based on the task identifier carried in the task suspension message, multiple computing servers that execute the second data processing task from the computing servers communicating with the parameter server.

[0211] Typically, each computing server executes the second data processing task based on the task parameters of the second data processing task to obtain processed data, encapsulates the processed data in a task message, and sends it to the parameter server via the first network device, so that the parameter server updates the task parameters based on the processed data and feeds back a task reply message to the computing server. During this process, the first network device receives and intercepts the task messages sent by each computing server, performs aggregation operations on the processed data carried in the task messages, obtains an intermediate calculation result, and sends the intermediate calculation result to the parameter server. After receiving and aggregating the intermediate calculation results of the second data processing task sent by the first network device, the parameter server feeds back a task reply message to the computing server that executes the second data processing task via the first network device.

[0212] Among them, the task reply message with the maximum message sequence number sent by the parameter server to the computing server carries the maximum message sequence number and a pause identifier, and the pause identifier is used to indicate that the computing server shall perform an operation of pausing the second data processing task based on the task message with the maximum message sequence number.

[0213] In other words, the maximum message sequence number can be carried in the task reply message corresponding to the maximum message sequence number. When the task reply message includes a pause identifier, it means that the task reply message is not only used to feedback to the computing server that the task message has been completely received, but also used to instruct the computing server to perform a task pause operation.

[0214] Referring to Figure 15 , in response to the task pause message, after aggregating the task messages with the largest message sequence number, the parameter server sends a task reply message to each compute server, including a message header, a task identifier, the largest message sequence number, and a pause identifier. In this embodiment, the task identifier in the task reply message is the task ID or task number of the second data processing task.

[0215] Therefore, after receiving the task reply message carrying the pause identifier, the computing server can perform the task pause operation through the following step (3) based on the maximum message sequence number carried in the task reply message.

[0216] (3) The computing server determines the target sequence number based on the maximum message sequence number and the size of the message sending window, and suspends sending the task message of the second data processing task after sending the task message of the target sequence number of the second data processing task.

[0217] Among them, the message sending window is used to limit the number of task messages sent by the computing server at one time. That is, within this message sending window, the computing server does not need to wait for the parameter server to send a task reply message, and can continue to send task messages until the number of task messages corresponding to the message sending window has been sent.

[0218] In some embodiments, the size of the message sending window can be flexibly set and adjusted. For example, if the message sending window is set large and too many messages are sent at one time, resulting in excessive data processing and data transfer pressure on the first network device, the size of the message sending window can also be adjusted to reduce the number of task messages sent at one time by the computing server. This embodiment of the present application does not impose any restrictions on this.

[0219] As an example, referring to FIG16 , assuming that the message sending window is 5, that is, 5 task messages can be sent continuously at one time. For computing server 1 and computing server 2 executing the second data processing task, after sending task message No. 9 to the parameter server through the network device, if the parameter server receives the task pause message indicating the suspension of the second data processing task at this time, and determines that the maximum message sequence number currently completed is 9, then the task reply message fed back to each computing server includes the maximum message sequence number 9 and the suspension identifier (i.e., the ACK9 message in FIG16 ). The task reply message ACK9 indicates that the task suspension operation is performed with task message No. 9 as the suspension starting point.

[0220] After receiving the ACK9 message, compute server 1 and compute server 2 use task message No. 9 as the pause starting point, send 5 task messages in sequence until task message No. 13, and then stop sending task messages. At this time, after the network device processes the processing data carried by each task message, it feeds back the intermediate processing results to the parameter server, and the parameter server completes the aggregation of task messages No. 10 to No. 13 in sequence.

[0221] It should be noted that since the maximum message sequence number is carried in the task reply message, when the task reply message is lost, it will cause a time difference between the task reply messages received by each computing server for the same task message, and the maximum message sequence number carried in the task reply message will also be different. In this case, if the second data processing task is directly suspended based on the maximum message sequence number, it may cause some computing servers to stop at the maximum message sequence number, and some computing servers to stop at the message sequence number corresponding to the task reply message received because they did not receive the task reply message, which in turn causes the task message sequence numbers sent by the multiple computing servers corresponding to the second data processing task to be inconsistent. When the message sequence numbers sent by multiple computing servers are not synchronized, the first network device cannot effectively aggregate them according to the message sequence number.

[0222] Based on this, to ensure that the compute servers can respond to the same maximum message sequence number and stop at the task message with the same sequence number, after receiving the maximum message sequence number, the compute server sends the task message with the target sequence number, and then stops sending the task message for the second data processing task. In other words, even if there is a time difference between the compute servers sending task messages, they will ultimately maintain consistency and stop at the task message with the same sequence number.

[0223] Based on this, in some embodiments, if there is a target computing server among multiple computing servers that has not received the task reply message with the maximum message sequence number, but has received a task reply message with the next message sequence number of the maximum message sequence number, the target computing server re-sends the task message with the maximum message sequence number to the parameter server; the parameter server re-sends the task reply message with the maximum message sequence number to the target computing server.

[0224] By having the target compute server resend the task message with the maximum message sequence number, we ensure that each compute server can execute the task pause operation based on the same maximum message sequence number. This way, even if there is a time difference between the compute servers receiving the task reply message with the maximum message sequence number, they can still execute the task pause operation based on the maximum message sequence number determined by the parameter server at the same time. This ensures that the message sequence numbers of the task messages sent at the final pause time are the same, allowing the parameter server to effectively aggregate messages before that time in sequence.

[0225] As an example, referring to FIG17, after computing server 1 receives the task reply message, i.e., the ACK9 message, it takes task message No. 9 as the pause starting point, sends 5 task messages in sequence until task message No. 13, and then stops sending task messages. However, computing server 2 does not receive the ACK9 message, but receives ACK10 after sending task message No. 10. At this time, computing server 2 determines that the task reply message is lost, and resends task message No. 9 to the parameter server. The parameter server carries the maximum message sequence number 9 and the pause flag after the feedback task reply message. In this way, although computing server 1 and computing server 2 receive ACK9 at different times, they can eventually suspend the execution of the second data processing task after sending task message No. 13, and their pause time is consistent.

[0226] (4) After receiving the task messages of the target sequence number of the second data processing task sent by the multiple computing servers, the parameter server sends a pause response message to the controller.

[0227] That is, after receiving the task message with the target sequence number from the multiple compute servers executing the second data processing task, the parameter server determines that the multiple compute servers have paused execution of the second data processing task and will no longer send task messages through the first network device. At this point, the parameter server sends a pause response message to the controller to inform the controller that the second data processing task has been paused.

[0228] Step 406: The controller receives the pause response message sent by the second server, and sends an addressing reconfiguration message to the second server, where the addressing reconfiguration message indicates the aggregator reallocated for the second data processing task in the first network device.

[0229] The addressing reconfiguration message carries the task identifier of the second data processing task, the number of reallocated aggregators, and the first address of the reallocated aggregators.

[0230] As an example, referring to Figure 18, the addressing reconfiguration message includes a message header, a task identifier, the number of reallocated aggregators, and the reallocated aggregator first address. In this embodiment, the task identifier in the addressing reconfiguration message is the task ID or task number of the second data processing task.

[0231] In this way, after receiving the addressing reconfiguration message, the second server can continue to send task messages after the target sequence number to the first network device according to the reallocated aggregator.

[0232] In an embodiment of the present application, when the first data processing task to be deployed is a data processing task that exclusively occupies an aggregator, the first data processing task is deployed in the task management system, and when the processing data of the first data processing task is aggregated through the first network device, it is necessary to reallocate the aggregator to the second data processing task that shares the aggregator of the first network device with other data processing tasks, so as to avoid the situation where the task messages of the first data processing task and the second data processing task occupy the same aggregator, resulting in the inability to effectively aggregate the task messages of the same sequence number and the inability to release the aggregation resources of the aggregator. In this way, through the reallocation of the aggregation resources of the aggregator, it is possible to ensure that the first network device effectively aggregates the processing data of each data processing task, and each data processing task can operate normally. Secondly, by setting the target sequence number, the task message sending progress of the second data processing task can be unified, so that multiple computing servers executing the second data processing task can pause at the task message with the same sequence number, and restart the second data processing task based on the task message with the same sequence number, so that the task messages of each sequence number of the second data processing task before and after the pause can be normally aggregated.

[0233] FIG19 is a schematic diagram of the structure of a task deployment apparatus provided in an embodiment of the present application. The task deployment apparatus can be implemented as part or all of a computer device by software, hardware, or a combination of both. The computer device can be controller 110 shown in FIG1 . Referring to FIG19 , the task deployment apparatus 1900 includes a resource measurement module 1901, a policy determination module 1902, and a task deployment module 1903.

[0234] Resource measurement module 1901, used to obtain the aggregate resource utilization of each network device, the bandwidth resource utilization of each network device, and the processing resource utilization of each server;

[0235] A policy determination module 1902 is configured to determine a task deployment policy and an aggregate resource allocation policy based on task running resources required by a first data processing task to be deployed, aggregate resource utilization and bandwidth resource utilization of multiple network devices, and processing resource utilization of multiple servers, wherein the task deployment policy indicates a first server among the multiple servers for executing the first data processing task, and the aggregate resource allocation policy indicates an aggregator in the first network device among the multiple network devices for performing an aggregation operation on processed data of the first data processing task.

[0236] The task deployment module 1903 is configured to deploy the first data processing task in the first server based on the task deployment strategy and the aggregated resource allocation strategy.

[0237] Optionally, the resource measurement module 1901 includes:

[0238] A sending unit, configured to send a measurement message to the network device, where the measurement message carries period information of a first measurement period, where the first measurement period is the measurement period in which the deployment time of the first data processing task is located;

[0239] a receiving unit, configured to receive a measurement reply message sent by the network device, where the measurement reply message carries the number of aggregators occupied by the network device in the first measurement period;

[0240] The first determining unit is configured to determine an aggregation resource utilization rate of the network device based on the number of occupied aggregators.

[0241] Optionally, the sending unit is specifically configured to:

[0242] determining a target server from the plurality of servers, where the target server is a server connected to a network device, and the network device is located on a communication link between the controller and the target server;

[0243] The measurement message is sent to the target server via the communication link between the controller and the target server, so that the network device receives and intercepts the measurement message.

[0244] Optionally, the first determining unit is specifically configured to:

[0245] Determine, based on the total number of aggregators included in the network device, a single occupancy duration of the aggregator of the network device in the first measurement period, and the period length of the first measurement period, a maximum number of aggregation times of the aggregator included in the network device in the first measurement period;

[0246] Determine the aggregation resource utilization of the network device based on the number of occupied aggregators and the maximum aggregation times.

[0247] Optionally, referring to FIG. 20 , the task deployment apparatus 1900 further includes:

[0248] A task determination module 1904 is configured to determine a deployed second data processing task, wherein the processed data of the second data processing task and the first data processing task both need to be aggregated in the first network device, and the second data processing task is a data processing task that can share an aggregator with other data processing tasks;

[0249] The task suspension module 1905 is configured to send a task suspension message to the second server, instructing the second server to suspend sending task messages after the target sequence number of the second data processing task, where the second server is a server among the multiple servers that executes the second data processing task;

[0250] The reconfiguration module 1906 is configured to receive the pause response message sent by the second server, and send an addressing reconfiguration message to the second server, where the addressing reconfiguration message indicates the aggregator reallocated for the second data processing task in the first network device.

[0251] Optionally, the second server includes a parameter server and multiple computing servers, the parameter server is used to store task parameters of the second data processing task, and each computing server is used to execute the second data processing task based on the task parameters;

[0252] The task suspension module 1905 is specifically configured to: send a task suspension message to the parameter server, so that the parameter server controls the plurality of computing servers to suspend sending task messages subsequent to the target sequence number of the second data processing task;

[0253] The reconfiguration module 1906 is further configured to receive a pause response message sent by the parameter server, where the pause response message indicates that the plurality of computing servers have paused sending task messages subsequent to the target sequence number of the second data processing task.

[0254] Optionally, the policy determination module 1902 includes:

[0255] a second determining unit, configured to determine a deployment constraint condition of the first data processing task based on task running resources, aggregate resource utilization and bandwidth resource utilization of multiple network devices, and processing resource utilization of multiple servers;

[0256] The third determining unit is configured to determine a task deployment strategy and an aggregated resource allocation strategy based on the deployment constraint conditions.

[0257] Optionally, deployment constraints include:

[0258] The task execution resources are less than or equal to the available processing resources of the server capable of executing the first data processing task;

[0259] The processing resource usage of the server capable of executing the first data processing task is less than or equal to the processing resource threshold;

[0260] The bandwidth resource between the server capable of executing the first data processing task and the network device capable of performing aggregation calculations on the processed data of the first data processing task is less than or equal to the bandwidth threshold;

[0261] The memory resource occupancy of the network device capable of performing an aggregation operation on the processing data of the first data processing task is less than or equal to the memory resource threshold.

[0262] In an embodiment of the present application, when deploying the first data processing task in the task management system via the task deployment device, the task deployment device can comprehensively analyze the aggregate resource utilization rate of each network device, the bandwidth resource utilization rate of each network device, the processing resource utilization rate of each server, and the task computing resources required for the first data processing task, thereby determining the task deployment strategy and aggregate resource allocation strategy for the first data processing task. Because the usage of the server's processing resources, the network device's bandwidth resources, and the aggregate resources is fully considered when determining the task deployment strategy and aggregate resource allocation strategy, task deployment and resource allocation are more reasonable, thereby improving the overall execution performance of the first data processing task and reducing the task completion time.

[0263] It should be noted that the task deployment device provided in the above embodiment only uses the division of the above functional modules as an example to illustrate when deploying data processing tasks in the task management system. In actual applications, the above functional distribution can be completed by different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the task deployment device provided in the above embodiment and the task deployment method embodiment are of the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0264] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, or a magnetic tape), an optical medium (e.g., a digital versatile disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)). It is worth noting that the computer-readable storage medium mentioned in the embodiments of the present application may be a non-volatile storage medium, in other words, a non-transient storage medium.

[0265] It should be understood that the "plurality" mentioned herein refers to two or more. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is merely a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0266] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the user information involved in the embodiments of this application and the task information of the data processing tasks submitted by the user are all obtained with full authorization.

[0267] The above description is an embodiment provided for this application and is not intended to limit this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application should be included in the scope of protection of this application.

Claims

1. A task deployment method, characterized in that: A controller applied to a task management system, wherein the task management system further comprises a server cluster and a plurality of network devices, wherein the server cluster comprises a plurality of servers, and each of the network devices comprises a plurality of aggregators obtained by dividing the memory resources of the network device; the method comprises: Obtaining the aggregate resource utilization of each of the network devices, the bandwidth resource utilization of each of the network devices, and the processing resource utilization of each of the servers; Determine a task deployment strategy and an aggregate resource allocation strategy according to the task running resources required for the first data processing task to be deployed, the aggregate resource utilization and bandwidth resource utilization of the multiple network devices, and the processing resource utilization of the multiple servers, wherein the task deployment strategy indicates a first server among the multiple servers for executing the first data processing task, and the aggregate resource allocation strategy indicates an aggregator in a first network device among the multiple network devices for performing an aggregate operation on the processed data of the first data processing task; Based on the task deployment strategy and the aggregate resource allocation strategy, the first data processing task is deployed in the first server.

2. The method according to claim 1, characterized in that The obtaining of the aggregate resource utilization of each of the network devices includes: Sending a measurement message to the network device, where the measurement message carries period information of a first measurement period, where the first measurement period is the measurement period in which the deployment time of the first data processing task is located; receiving a measurement reply message sent by the network device, wherein the measurement reply message carries the number of aggregators occupied by the network device in the first measurement period; Based on the number of occupied aggregators, an aggregation resource utilization rate of the network device is determined.

3. The method according to claim 2, characterized in that The sending the measurement message to the network device includes: Determining a target server from the multiple servers, the target server being a server connected to the network device, and the network device being located on a communication link between the controller and the target server; The measurement message is sent to the target server through a communication link between the controller and the target server, so that the network device receives and intercepts the measurement message.

4. The method according to claim 2, characterized in that The determining, based on the number of occupied aggregators, the aggregation resource utilization of the network device comprises: Determine a maximum number of aggregation times of the aggregator included in the network device in the first measurement period according to the total number of aggregators included in the network device, a single occupancy duration of the aggregator of the network device in the first measurement period, and a period length of the first measurement period; Based on the number of occupied aggregators and the maximum aggregation times, an aggregation resource utilization rate of the network device is determined.

5. The method according to any one of claims 1 to 4, characterized in that: The first data processing task refers to a data processing task that requires an exclusive aggregator; the method further includes: Determine a deployed second data processing task, where both the processed data of the second data processing task and the first data processing task need to be aggregated in the first network device, and the second data processing task refers to a data processing task that can share an aggregator with other data processing tasks; Sending a task suspension message to a second server to instruct the second server to suspend sending task messages after the target sequence number of the second data processing task, where the second server is a server among the multiple servers that executes the second data processing task; Receive a pause response message sent by the second server, and send an addressing reconfiguration message to the second server, wherein the addressing reconfiguration message indicates an aggregator reallocated in the first network device for the second data processing task.

6. The method according to claim 5, characterized in that The second server includes a parameter server and a plurality of computing servers, the parameter server is used to store task parameters of the second data processing task, and each computing server is used to execute the second data processing task based on the task parameters; The sending of the task suspension message to the second server includes: Sending the task suspension message to the parameter server, so that the parameter server controls the multiple computing servers to suspend sending task messages after the target sequence number of the second data processing task; The receiving a pause response message sent by the second server includes: A pause response message is received from the parameter server, where the pause response message indicates that the plurality of computing servers have all paused sending task messages after the target sequence number of the second data processing task.

7. The method according to any one of claims 1 to 4, characterized in that: The step of determining the task deployment strategy and the aggregate resource allocation strategy according to the task running resources required by the first data processing task to be deployed, the aggregate resource utilization and bandwidth resource utilization of the plurality of network devices, and the processing resource utilization of the plurality of servers includes: Determining deployment constraints of the first data processing task based on the task running resources, the aggregate resource utilization and bandwidth resource utilization of the multiple network devices, and the processing resource utilization of the multiple servers; Based on the deployment constraints, the task deployment strategy and the aggregate resource allocation strategy are determined.

8. The method according to claim 7, characterized in that The deployment constraints include: The task running resources are less than or equal to the available processing resources of the server capable of executing the first data processing task; The processing resource usage of the server capable of executing the first data processing task is less than or equal to the processing resource threshold; The bandwidth resource between the server capable of executing the first data processing task and the network device capable of performing aggregation calculation on the processed data of the first data processing task is less than or equal to the bandwidth threshold; The memory resource occupancy of the network device capable of performing aggregation operations on the processing data of the first data processing task is less than or equal to the memory resource threshold.

9. A task deployment method, characterized in that: Applied to a task management system, the task management system includes a controller, a server cluster and multiple network devices, the server cluster includes multiple servers, and each of the network devices includes multiple aggregators obtained by dividing the memory resources of the network device; the method includes: The controller obtains the aggregate resource utilization of each of the network devices, the bandwidth resource utilization of each of the network devices, and the processing resource utilization of each of the servers; The controller determines a task deployment strategy and an aggregate resource allocation strategy according to task running resources required for a first data processing task to be deployed, aggregate resource utilization and bandwidth resource utilization of the multiple network devices, and processing resource utilization of the multiple servers, wherein the task deployment strategy indicates a first server among the multiple servers for executing the first data processing task, and the aggregate resource allocation strategy indicates an aggregator in a first network device among the multiple network devices for performing an aggregate operation on processing data of the first data processing task; The controller deploys the first data processing task in the first server based on the task deployment strategy and the aggregate resource allocation strategy; The first server stores task data of the first data processing task and resource information of aggregate resources allocated to the first data processing task.

10. The method according to claim 9, characterized in that The controller obtains the aggregate resource utilization of each of the network devices, including: The controller sends a measurement message to the network device, where the measurement message carries period information of a first measurement period, where the first measurement period is the measurement period in which the deployment time of the first data processing task is located; The network device receives the measurement message, determines the number of occupied aggregators of the network device in the first measurement period based on the period information of the first measurement period, and sends a measurement reply message to the controller, where the measurement reply message carries the number of occupied aggregators; The controller receives the measurement reply message and determines the aggregation resource utilization of the network device based on the number of occupied aggregators.

11. The method according to claim 10, characterized in that The period information of the first measurement period is a version number corresponding to the first measurement period, and a first state counter is provided in the network device, and the first state counter has a first version number; The network device determines, based on the period information of the first measurement period, the number of occupied aggregators of the network device in the first measurement period, including: In response to the version number corresponding to the first measurement period being the same as the first version number, the network device determines the number of occupied aggregators of the network device in the first measurement period based on the count value of the first state counter.

12. The method according to claim 11, characterized in that The first state counter also has first state information, and the network device is further provided with a second state counter, wherein the second state counter has second state information and a second version number; After the network device sends a measurement reply message to the controller, the method further includes: In response to the first state information indicating the current measurement cycle and the second state information indicating the previous measurement cycle, the network device switches the first state information and the second state information, updates the second version number to the version number corresponding to the second measurement cycle, and clears the count value of the second state counter to record the number of occupied aggregators in the second measurement cycle through the second state counter. The second measurement cycle is the next measurement cycle adjacent to the first measurement cycle.

13. The method according to any one of claims 9 to 12, characterized in that: The first data processing task refers to a data processing task that requires an exclusive aggregator; the method further includes: The controller determines a deployed second data processing task, where both the processed data of the second data processing task and the first data processing task need to be aggregated in the first network device, and the second data processing task refers to a data processing task that can share an aggregator with other data processing tasks; The controller sends a task suspension message to a second server, where the second server is a server among the multiple servers that executes the second data processing task; The second server receives the task suspension message, suspends sending the task message of the second data processing task after sending the task message of the target sequence number of the second data processing task, and sends a suspension response message to the controller; The controller receives the pause response message and sends an addressing reconfiguration message to the second server, where the addressing reconfiguration message indicates the aggregator reallocated for the second data processing task in the first network device.

14. The method according to claim 13, characterized in that The second server includes a parameter server and a plurality of computing servers, the parameter server is used to store task parameters of the second data processing task, and each computing server is used to execute the second data processing task based on the task parameters; The controller sends a task suspension message to the second server, including: The controller sends the task suspension message to the parameter server; The second server receives the task suspension message, suspends sending the task message of the second data processing task after sending the task message of the target sequence number of the second data processing task, and sends a suspension response message to the controller, including: The parameter server receives the task suspension message, determines the maximum message sequence number that has completed the aggregation operation at the current moment, and sends the maximum message sequence number to the multiple computing servers; The computing server determines the target sequence number based on the maximum message sequence number and the size of the message sending window, and suspends sending the task message of the second data processing task after sending the task message of the target sequence number of the second data processing task; After receiving the task message of the target sequence number of the second data processing task sent by the multiple computing servers, the parameter server sends the pause response message to the controller.

15. The method according to claim 14, characterized in that The task reply message with the maximum message sequence number sent by the parameter server to the computing server carries the maximum message sequence number and the pause identifier; the method further includes: If there is a target computing server among the multiple computing servers that has not received the task reply message with the maximum message sequence number, but has received a task reply message with a message sequence number next to the maximum message sequence number, the target computing server resends the task message with the maximum message sequence number to the parameter server; The parameter server resends the task reply message with the maximum message sequence number to the target computing server.

16. A task deployment device, characterized in that: The device is applied to a task management system, the task management system includes a server cluster and multiple network devices, the server cluster includes multiple servers, each of the network devices includes multiple aggregators obtained by dividing the memory resources of the network device; the device includes: A resource measurement module, used to obtain the aggregate resource utilization of each of the network devices, the bandwidth resource utilization of each of the network devices, and the processing resource utilization of each of the servers; a strategy determination module, configured to determine a task deployment strategy and an aggregate resource allocation strategy according to task running resources required for a first data processing task to be deployed, aggregate resource utilization and bandwidth resource utilization of the plurality of network devices, and processing resource utilization of the plurality of servers, wherein the task deployment strategy indicates a first server among the plurality of servers for executing the first data processing task, and the aggregate resource allocation strategy indicates an aggregator in a first network device among the plurality of network devices for performing an aggregate operation on processing data of the first data processing task; A task deployment module is used to deploy the first data processing task in the first server based on the task deployment strategy and the aggregate resource allocation strategy.

17. A task management system, characterized in that: The task management system includes a controller, a server cluster and a plurality of network devices, wherein the server cluster includes a plurality of servers, and each of the network devices includes a plurality of aggregators obtained by dividing the memory resources of the network device; The controller is used to obtain the aggregate resource utilization of each of the network devices, the bandwidth resource utilization of each of the network devices, and the processing resource utilization of each of the servers, and determine a task deployment strategy and an aggregate resource allocation strategy according to the task running resources required for the first data processing task to be deployed, the aggregate resource utilization and bandwidth resource utilization of the multiple network devices, and the processing resource utilization of the multiple servers, wherein the task deployment strategy indicates a first server among the multiple servers for executing the first data processing task, and the aggregate resource allocation strategy indicates an aggregator in a first network device among the multiple network devices for performing an aggregate operation on the processed data of the first data processing task, and deploys the first data processing task in the first server based on the task deployment strategy and the aggregate resource allocation strategy; The first server is used to store task data of the first data processing task and resource information of aggregate resources allocated to the first data processing task; The first network device is used to perform aggregation operations on the processed data of the first data processing task.

18. A controller, characterized in that: The controller includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 8 when calling and executing the computer program.

19. A computer-readable storage medium, characterized in that: The storage medium stores instructions, and when the instructions are executed on the computer, the computer executes the method according to any one of claims 1 to 8.

20. A computer program product, characterized in that The program product stores instructions, and when the instructions are executed on the computer, a controller is caused to execute the method according to any one of claims 1 to 8.

Images