Task deployment method and device, task management system, controller and storage medium

Abstract

The invention discloses a task deployment method and device, a task management system, a controller and a storage medium, and belongs to the field of distributed data processing. The method comprises the following steps: acquiring an aggregation resource utilization rate and a bandwidth resource utilization rate of each network device, and a processing resource utilization rate of each server; determining a task deployment strategy and an aggregation resource allocation strategy according to task operation resources required by a to-be-deployed first data processing task, the aggregation resource utilization rate and the bandwidth resource utilization rate of the plurality of network devices, and the processing resource utilization rate of the plurality of servers, the task deployment strategy indicates a first server used for executing a first data processing task in the plurality of servers, and the aggregation resource allocation strategy indicates an aggregator in a first network device used for performing aggregation operation on processing data of the first data processing task in the plurality of network devices; and deploying the first data processing task in the first server based on the task deployment strategy and the aggregation resource allocation strategy.

Description

Technical Field

[0001] The present application relates to the field of distributed data processing, and in particular to a task deployment method, device, task management system, controller and storage medium. Background Art

[0002] With the expansion of the network structure of machine learning models and the increase of training data, distributed training has gradually become the mainstream training method for machine learning models. As a distributed architecture, the parameter server (PS) architecture divides the servers in the server cluster into two types: parameter servers (also called PS nodes) and computing servers (also called worker nodes). A model training task is usually deployed on a parameter server and multiple computing servers. The parameter server stores the model parameters of the model to be trained. The multiple computing servers obtain the model parameters from the parameter server and train the model locally based on the model parameters. The gradient data obtained from the training is sent to the parameter server, so that the parameter server updates the model parameters by aggregating the gradient data sent by the multiple computing servers.

[0003] In the related art, when a server cluster under the PS architecture deploys a model training task, the parameter server and multiple computing servers corresponding to the model training task are determined according to the task running resources required for the model training task, the processing resource utilization of each server, and the bandwidth resources between the parameter server and the computing server, so that the model training task is deployed in the server cluster. For any model training task, the multiple computing servers that deploy the model training task send all gradient data to the parameter server for aggregation calculation. This "many-to-one" communication mode will cause network congestion and even model training task timeout. In order to speed up the speed of distributed training, a programmable switch is introduced in the above-mentioned PS architecture. The programmable switch can not only realize the data forwarding function between the computing server and the parameter server, but also perform aggregation calculations on the gradient data obtained by the multiple computing servers, and then send the aggregation results to the parameter server, reducing the data processing volume of the parameter server.

[0004] Since the computing resources of programmable switches are introduced, and multiple model training tasks can share the computing resources of a programmable switch, but when the multiple model training tasks are deployed unevenly and occupy the same programmable switch at the same time, it will seriously affect the overall performance of the programmable switch. Therefore, a new task deployment method is urgently needed to fully consider the computing resources of the programmable switch and the processing resources of the server when deploying model training tasks. Summary of the invention

[0005] The present application provides a task deployment method, device, task management system, controller and storage medium, which can comprehensively consider the aggregate resource utilization and bandwidth resource utilization of network devices and the processing resource utilization of servers to deploy data processing tasks in a server cluster. The technical solution is as follows:

[0006] In a first aspect, a task deployment method is provided, which is applied to a controller in a task management system, wherein the task management system also includes a server cluster and multiple network devices, the server cluster includes multiple servers, and each network device includes multiple aggregators obtained by dividing the memory resources of the network device; the method includes: obtaining the aggregate resource utilization of each network device, the bandwidth resource utilization of each network device, and the processing resource utilization of each server; determining a task deployment strategy and an aggregate resource allocation strategy based on the task running resources required for a first data processing task to be deployed, the aggregate resource utilization and bandwidth resource utilization of multiple network devices, and the processing resource utilization of multiple servers, the task deployment strategy indicating a first server among multiple servers for executing the first data processing task, and the aggregate resource allocation strategy indicating an aggregator in the first network device among multiple network devices for performing an aggregation operation on the processed data of the first data processing task; and deploying the first data processing task in the first server based on the task deployment strategy and the aggregate resource allocation strategy.

[0007] It can be seen that when deploying the first data processing task in the task management system, the controller can comprehensively analyze the aggregate resource utilization of each network device, the bandwidth resource utilization of each network device, the processing resource utilization of each server, and the task computing resources required for the first data processing task, thereby determining the task deployment strategy and aggregate resource allocation strategy for the first data processing task. Since the processing resources of the server, the bandwidth resources of the network device, and the usage of aggregate resources are fully considered, the task deployment and resource allocation are more reasonable, thereby improving the overall execution performance of the first data processing task and reducing the task completion time.

[0008] Optionally, the aggregation resource utilization of each network device is obtained, including: sending a measurement message to the network device, the measurement message carrying period information of a first measurement cycle, the first measurement cycle being the measurement cycle in which the deployment time of the first data processing task is located; receiving a measurement reply message sent by the network device, the measurement reply message carrying the number of occupied aggregators of the network device within the first measurement cycle; and determining the aggregation resource utilization of the network device based on the number of occupied aggregators.

[0009] That is, the controller obtains the number of occupied aggregators of the network device in a measurement cycle by sending measurement messages, and then analyzes and calculates the aggregate resource utilization of the network device based on the number of occupied aggregators. In this way, the measurement efficiency of the aggregate resource utilization of the network device is improved, making it easier for the controller to monitor the aggregate resource utilization of the network device at any time as needed.

[0010] Optionally, sending a measurement message to a network device includes: determining a target server from multiple servers, where the target server refers to a server connected to the network device, and the network device is located on a communication link between a controller and the target server; sending a measurement message to the target server through the communication link between the controller and the target server, so that the network device receives and intercepts the measurement message.

[0011] Since the controller sends the measurement message to the network device via the communication link between the controller and the server, all network devices on the communication link can receive the measurement message, thereby comprehensively and effectively measuring the aggregate resource utilization of each network device.

[0012] Optionally, based on the number of occupied aggregators, the aggregation resource utilization of the network device is determined, including: determining the maximum number of aggregation times of the aggregators included in the network device within the first measurement cycle according to the total number of aggregators included in the network device, the single occupancy duration of the aggregators of the network device within the first measurement cycle, and the cycle length of the first measurement cycle; based on the number of occupied aggregators and the maximum number of aggregation times, determining the aggregation resource utilization of the network device.

[0013] Among them, the task message will occupy the aggregation resources of the aggregator. After the aggregation operation is completed for the processing data carried in the task message, the aggregation resources of the aggregator will be released, and the aggregator can be used to process the task messages of other data processing tasks. Therefore, for an aggregator, its single occupancy time is the time from the network device receiving the task message to completing the aggregation operation of the processing data in the task message.

[0014] That is, the controller determines the number of occupancy times of each aggregator in the first measurement cycle according to the cycle length of the first measurement cycle and the single occupancy duration of the aggregator of the network device in the first measurement cycle, and then determines the maximum number of aggregation times of the network device in the first measurement cycle according to the number of occupancy times of each aggregator in the first measurement cycle and the total number of aggregators included in the network device. In this way, the controller can analyze the aggregation resource utilization of the network device based on the occupancy of each aggregator in the network device in a single measurement cycle, with simple operation and more accurate results.

[0015] Optionally, the first data processing task refers to a data processing task that requires an exclusive aggregator; the method also includes: determining a deployed second data processing task, the processing data of the second data processing task and the first data processing task both need to be aggregated in the first network device, and the second data processing task refers to a data processing task that can share the aggregator with other data processing tasks; sending a task pause message to the second server to instruct the second server to pause sending task messages after the target sequence number of the second data processing task, the second server being a server among multiple servers that executes the second data processing task; receiving a pause response message sent by the second server, and sending an addressing reconfiguration message to the second server, the addressing reconfiguration message indicating the aggregator reallocated for the second data processing task in the first network device.

[0016] It can be seen from this that if the first data processing is a data processing task that exclusively occupies an aggregator, the first data processing task is deployed in the task management system, and when the processing data of the first data processing task is aggregated through the first network device, it is necessary to reallocate the aggregator to the second data processing task that shares the first network device aggregator with other data processing tasks, so as to avoid the situation where the task messages of the first data processing task and the second data processing task occupy the same aggregator, resulting in the inability to effectively aggregate the task messages with the same sequence number and the aggregation resources of the aggregator cannot be released. In this way, through the reallocation of the aggregator, it can be ensured that the first network device effectively aggregates the processing data of each data processing task, and each data processing task can run normally. Secondly, by setting the target sequence number, the task message sending progress of the second data processing task can be unified, so that multiple computing servers executing the second data processing task can be suspended at the task message with the same sequence number, and the second data processing task can be restarted based on the task message with the same sequence number, so that the task messages of each sequence number of the second data processing task before and after the suspension can be aggregated normally.

[0017] Optionally, the second server includes a parameter server and multiple computing servers, the parameter server is used to store task parameters of the second data processing task, and each computing server is used to execute the second data processing task based on the task parameters; sending a task pause message to the second server includes: sending a task pause message to the parameter server so that the parameter server controls multiple computing servers to pause sending task messages after the target sequence number of the second data processing task; receiving a pause response message sent by the second server includes: receiving a pause response message sent by the parameter server, the pause response message indicating that multiple computing servers have paused sending task messages after the target sequence number of the second data processing task.

[0018] The task pause message is used to instruct the second server to suspend sending task messages after the target sequence number of the second data processing task, so as to avoid the situation where the processing data carried in the task message sent by the second server cannot be effectively aggregated by the aggregator after the first network device allocates an exclusive aggregator for the first data processing task. After the second server receives the task pause message and sends the task message with the target sequence number of the second data processing task, it suspends sending the task message of the second data processing task to the first network device and sends a pause response message to the controller.

[0019] That is, when the second data processing task is deployed on multiple computing servers, the controller interacts with the parameter server through communication to control the multiple computing servers to pause at the task message with the same sequence number, and subsequently restart the second data processing task based on the task message with the same sequence number.

[0020] Optionally, based on the task running resources required for the first data processing task to be deployed, the aggregate resource utilization and bandwidth resource utilization of multiple network devices, and the processing resource utilization of multiple servers, a task deployment strategy and an aggregate resource allocation strategy are determined, including: determining the deployment constraints of the first data processing task based on the task running resources, the aggregate resource utilization and bandwidth resource utilization of multiple network devices, and the processing resource utilization of multiple servers; determining the task deployment strategy and the aggregate resource allocation strategy based on the deployment constraints.

[0021] Optionally, the deployment constraints include:

[0022] The task running resources are less than or equal to the available processing resources of the server capable of executing the first data processing task;

[0023] The processing resource usage of the server capable of executing the first data processing task is less than or equal to the processing resource threshold;

[0024] The bandwidth resource between the server capable of executing the first data processing task and the network device capable of performing aggregation calculation on the processed data of the first data processing task is less than or equal to the bandwidth threshold;

[0025] The memory resource occupancy of the network device capable of performing aggregation calculations on the processing data of the first data processing task is less than or equal to the memory resource threshold.

[0026] Among them, the server's available processing resources are used to indicate whether the server can continue to deploy data processing tasks, the processing resource threshold is used to indicate whether the server is in an overloaded state, the bandwidth threshold is used to indicate the bandwidth resources where data packet loss may occur, and the memory resource threshold is used to indicate whether the network device is in an overloaded state.

[0027] Since the deployment constraints fully limit the conditions that the server that can execute the first data processing task should meet, and the conditions that the network equipment that can perform aggregation operations on the processed data of the first data processing task should meet, the task deployment strategy and aggregation resource allocation strategy determined based on the deployment constraints are more accurate and reasonable.

[0028] In a second aspect, another task deployment method is provided, which is applied to a task management system, wherein the task management system includes a controller, a server cluster and multiple network devices, the server cluster includes multiple servers, and each network device includes multiple aggregators obtained by dividing the memory resources of the network device; the method includes: the controller obtains the aggregate resource utilization of each network device, the bandwidth resource utilization of each network device, and the processing resource utilization of each server; the controller determines the task deployment strategy and the aggregate resource allocation strategy according to the task running resources required for the first data processing task to be deployed, the aggregate resource utilization and bandwidth resource utilization of multiple network devices, and the processing resource utilization of multiple servers, the task deployment strategy indicates the first server among the multiple servers for executing the first data processing task, and the aggregate resource allocation strategy indicates the aggregator in the first network device among the multiple network devices for performing aggregate operations on the processed data of the first data processing task; the controller deploys the first data processing task in the first server based on the task deployment strategy and the aggregate resource allocation strategy; the first server stores the task data of the first data processing task, and the resource information of the aggregate resources allocated to the first data processing task.

[0029] Optionally, the controller obtains the aggregation resource utilization of each network device, including: the controller sends a measurement message to the network device, the measurement message carries the period information of a first measurement cycle, the first measurement cycle is the measurement cycle in which the deployment time of the first data processing task is located; the network device receives the measurement message, determines the number of occupied aggregators of the network device within the first measurement cycle based on the period information of the first measurement cycle, and sends a measurement reply message to the controller, the measurement reply message carries the number of occupied aggregators; the controller receives the measurement reply message, and determines the aggregation resource utilization of the network device based on the number of occupied aggregators.

[0030] Optionally, the period information of the first measurement cycle is a version number corresponding to the first measurement cycle, and a first state counter is provided in the network device, and the first state counter has a first version number; the network device determines the number of occupied aggregators of the network device within the first measurement cycle based on the period information of the first measurement cycle, including: in response to the version number corresponding to the first measurement cycle being the same as the first version number, the network device determines the number of occupied aggregators of the network device within the first measurement cycle based on the count value of the first state counter.

[0031] The first state counter is used to record the number of occupied aggregators in the network device in the current measurement period. After the network device receives the measurement message, in response to the version number corresponding to the first measurement period being the same as the first version number, the network device can determine the number of occupied aggregators in the first measurement period based on the count value of the first state counter.

[0032] Optionally, the first state counter also has first state information, and a second state counter is also provided in the network device, and the second state counter has second state information and a second version number; after the network device sends a measurement reply message to the controller, the method also includes: in response to the first state information indicating the current measurement cycle and the second state information indicating the previous measurement cycle, the network device switches the first state information and the second state information, updates the second version number to the version number corresponding to the second measurement cycle, clears the count value of the second state counter, and records the number of occupied aggregators in the second measurement cycle through the second state counter, and the second measurement cycle is the next measurement cycle adjacent to the first measurement cycle.

[0033] That is, two counters are set in the network device, which are used to record the number of occupied aggregators in the current measurement cycle and the number of occupied aggregators in the previous measurement cycle. By switching the state information of the two counters and updating the version number of the counters, it is ensured that only one counter is used to record the number of occupied aggregators in each measurement cycle, and the states of the counters recording the number of occupied aggregators in adjacent measurement cycles are different.

[0034] Optionally, the first data processing task refers to a data processing task that requires an exclusive aggregator; the method also includes: the controller determines a deployed second data processing task, the processing data of the second data processing task and the first data processing task both need to be aggregated in the first network device, and the second data processing task refers to a data processing task that can share the aggregator with other data processing tasks; the controller sends a task pause message to the second server, and the second server is a server among multiple servers that executes the second data processing task; the second server receives the task pause message, suspends sending the task message of the second data processing task after sending the task message with the target sequence number of the second data processing task, and sends a pause response message to the controller; the controller receives the pause response message, and sends an addressing reconfiguration message to the second server, and the addressing reconfiguration message indicates the aggregator reallocated for the second data processing task in the first network device.

[0035] Optionally, the second server includes a parameter server and multiple computing servers, the parameter server is used to store task parameters of the second data processing task, and each computing server is used to execute the second data processing task based on the task parameters; the controller sends a task pause message to the second server, including: the controller sends a task pause message to the parameter server; the second server receives the task pause message, suspends sending the task message of the second data processing task after sending the task message with the target sequence number of the second data processing task, and sends a pause response message to the controller, including: the parameter server receives the task pause message, determines the maximum message sequence number that has completed the aggregation operation at the current moment, and sends the maximum message sequence number to multiple computing servers; the computing server determines the target sequence number based on the maximum message sequence number and the size of the message sending window, and suspends sending the task message of the second data processing task after sending the task message with the target sequence number of the second data processing task; after the parameter server receives the task message with the target sequence number of the second data processing task sent by multiple computing servers, it sends a pause response message to the controller.

[0036] Among them, the message sending window is used to limit the number of task messages sent by the computing server at one time. That is, within this message sending window, the computing server does not need to wait for the parameter server to send a task reply message, and can continue to send task messages until the corresponding number of task messages in the message sending window have been sent.

[0037] Therefore, to ensure that the computing server can respond to the same maximum message sequence number and stop at the same task message, after receiving the maximum message sequence number, the computing server sends the task message of the target sequence number and then stops sending the task message of the second data processing task. That is, even if there is a time difference in sending task messages among the computing servers, they can all eventually remain consistent and stop at the task message with the same sequence number.

[0038] Optionally, the task reply message with the maximum message number sent by the parameter server to the computing server carries the maximum message number and a pause identifier; the method also includes: if there is a target computing server among multiple computing servers that has not received the task reply message with the maximum message number, but has received a task reply message with the next message number of the maximum message number, the target computing server re-sends the task message with the maximum message number to the parameter server; the parameter server re-sends the task reply message with the maximum message number to the target computing server.

[0039] By having the target computing server resend the task message with the maximum message sequence number, it can be ensured that each computing server can perform the task pause operation based on the same maximum message sequence number. In this way, even if there is a time difference in the task reply message with the maximum message sequence number received by each computing server, the task pause operation can be performed based on the maximum message sequence number determined by the parameter server at the same time, so that the message sequence numbers of the task messages sent at the final stop time are the same, and the parameter server can effectively aggregate the messages before this time in sequence.

[0040] In a third aspect, a task deployment device is provided, which has the function of implementing the task deployment method in the first aspect. The task deployment device includes one or more modules, which are used to implement the task deployment method provided in the first aspect.

[0041] In a fourth aspect, a task management system is provided, which includes a controller, a server cluster and multiple network devices, the server cluster includes multiple servers, and each network device includes multiple aggregators obtained by dividing the memory resources of the network device.

[0042] Among them, the controller is used to obtain the aggregate resource utilization of each network device, the bandwidth resource utilization of each network device, and the processing resource utilization of each server, and determine the task deployment strategy and the aggregate resource allocation strategy according to the task running resources required for the first data processing task to be deployed, the aggregate resource utilization and bandwidth resource utilization of multiple network devices, and the processing resource utilization of multiple servers. The task deployment strategy indicates the first server among multiple servers used to execute the first data processing task, and the aggregate resource allocation strategy indicates the aggregator in the first network device among multiple network devices used to perform aggregate operations on the processed data of the first data processing task. Based on the task deployment strategy and the aggregate resource allocation strategy, the first data processing task is deployed in the first server; the first server is used to store the task data of the first data processing task and the resource information of the aggregate resources allocated to the first data processing task; the first network device is used to perform aggregate operations on the processed data of the first data processing task.

[0043] In a fifth aspect, a controller is provided, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor calls and executes the computer program, the steps of the task deployment method shown in the first aspect are implemented.

[0044] In a sixth aspect, a computer-readable storage medium is provided, in which a computer program is stored. When a controller executes the computer program, the steps of the task deployment method shown in the first aspect are implemented.

[0045] In a seventh aspect, a computer program product is provided, which includes a computer program. When a controller executes the computer program, the steps of the task deployment method shown in the first aspect are implemented.

[0046] The technical effects obtained in the above-mentioned second to seventh aspects are similar to the technical effects achieved by the corresponding technical features in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a system architecture diagram of a task management system provided by an embodiment of the present application;

[0048] Figure 2 It is a structural diagram of a task message provided in an embodiment of the present application;

[0049] Figure 3 is a structural schematic diagram of a computer device provided in an embodiment of the present application;

[0050] Figure 4 It is a flowchart of a task deployment method provided in an embodiment of the present application;

[0051] Figure 5 It is a schematic diagram of a controller sending a measurement message to a network device provided in an embodiment of the present application;

[0052] Figure 6 It is a schematic diagram of the structure of a measurement message provided in an embodiment of the present application;

[0053] Figure 7 It is a schematic diagram of a count value of a network device updating status counter provided by an embodiment of the present application;

[0054] Figure 8 This is a schematic diagram of the structure of a measurement reply message provided by an embodiment of the present application;

[0055] Fig. 9 It is a schematic diagram of determining a single occupancy time of an aggregator provided in an embodiment of the present application;

[0056] Fig.10 It is a schematic diagram of state information of a network device switching state counter provided by an embodiment of the present application;

[0057] Fig.11 It is a schematic diagram of a network device responding to a retransmitted measurement message provided by an embodiment of the present application;

[0058] Fig.12 It is a schematic diagram of aggregator allocation in a network device provided in an embodiment of the present application;

[0059] Fig.13It is a schematic diagram of aggregator allocation in another network device provided in an embodiment of the present application;

[0060] Fig.14 It is a flowchart of another task deployment method provided in an embodiment of the present application;

[0061] Fig.15 This is a schematic diagram of the structure of a task reply message provided in an embodiment of the present application;

[0062] Fig.16 is a schematic diagram of pausing a data processing task provided by an embodiment of the present application;

[0063] Fig.17 is a schematic diagram of another method of pausing a data processing task provided in an embodiment of the present application;

[0064] Fig.18 It is a structural diagram of an addressing reconfiguration message provided in an embodiment of the present application;

[0065] Fig.19 It is a structural diagram of a task deployment device provided in an embodiment of the present application;

[0066] Fig. 20 It is a structural diagram of another task deployment device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0067] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0068] In public cloud environments, multiple servers in a server cluster provide data processing services. Usually, multiple data processing tasks of multiple users are deployed in a server cluster. After the introduction of network devices, multiple data processing tasks can share the aggregated resources of the network devices. That is, when the network device forwards the task message of the data processing task between multiple servers, the processing data of the data processing task carried in the task message is aggregated, thereby reducing the amount of data that the server on the end side performs the aggregation operation, so as to alleviate the network congestion caused by the "many-to-one" transmission of the processing data of the data processing task between multiple servers.

[0069] In an exemplary scenario, the data processing task may be a model training task corresponding to a machine learning model trained in a distributed training manner. When training a machine learning model, the gradient data obtained by training the model locally on multiple distributed servers needs to be sent to a server for aggregation operations to update the model parameters. In this process, the forwarded gradient data can be aggregated with the help of network devices.

[0070] In another exemplary scenario, the data processing task may be an IoT data processing task. After multiple servers obtain monitoring data from front-end IoT devices, they need to send the monitoring data to a server for summary analysis. In this process, the monitoring data forwarded by the server can be aggregated with the help of network devices.

[0071] It should be understood that the data processing task can also be other tasks that require the use of a "many-to-one" communication mode to transmit and process data. The embodiments of the present application only explain the data processing task as a model training task, and do not constitute a limitation on the type of data processing tasks.

[0072] However, since the aggregation resources used to perform aggregation operations in network devices are limited, when the aggregation resources of network devices are unevenly distributed among multiple data processing tasks, multiple data processing tasks may occupy the same network device, causing the aggregation resources of the network device to be unable to meet the aggregation operation requirements of processing data of multiple data processing tasks, affecting the overall computing performance of the network device and increasing the completion time of the data processing tasks.

[0073] Based on this, an embodiment of the present application provides a task deployment method, which can fully consider the processing resources of each server in the server cluster, the aggregated resources of the network devices, and the bandwidth resources of the network devices when deploying data processing tasks in a server cluster, thereby optimizing the deployment method of data processing tasks, so that each data processing task can make full use of the aggregated resources of the network devices, improve resource utilization, and shorten the task completion time.

[0074] Before explaining in detail the task deployment method provided in the embodiment of the present application, the system architecture and implementation environment involved in the embodiment of the present application are first introduced.

[0075] Please refer to Figure 1 , Figure 1 This is an architecture diagram of a task management system provided in an embodiment of the present application, wherein the task management system includes: a controller 110, a server cluster 120 and multiple network devices 130, wherein the server cluster 120 includes multiple servers, and each network device 130 includes multiple aggregators obtained by dividing the memory resources of the network device 130.

[0076] It should be noted that the controller is used to manage all devices in the task management system, and it can communicate with the servers and network devices in the server cluster respectively. The controller can send instructions, data and information to the network devices, send instructions, data and information to the servers through the network devices, send instructions, data and information directly to the servers, and receive data and information sent by the servers and / or network devices.

[0077] Figure 1 The example of a server cluster including 8 servers and 3 network devices does not constitute a limitation on the number of devices that can be included in the task management system. Moreover, for each network device, there can be one or more servers connected thereto, which is not limited in the embodiments of the present application. For the same server, it can be connected to one or more network devices, which is not limited in the embodiments of the present application.

[0078] In the task management system, when a user submits a data processing task, the controller determines the server that can deploy the data processing task and the network device that can perform aggregate calculations on the processed data of the data processing task from the server cluster based on the processing resource usage of each server in the server cluster, as well as the aggregate resource usage and bandwidth resource usage of each network device. The data processing task submitted by the user is deployed on the corresponding server, and the server is instructed to send the processed data obtained by executing the data processing task to the corresponding network device for aggregate calculations.

[0079] In some embodiments, the controller obtains the aggregate resource utilization of each network device, the bandwidth resource utilization of each network device, and the processing resource utilization of each server; according to the task running resources required for the first data processing task to be deployed, the aggregate resource utilization and bandwidth resource utilization of multiple network devices, and the processing resource utilization of multiple servers, the task deployment strategy and the aggregate resource allocation strategy are determined, the task deployment strategy indicates the first server among multiple servers for executing the first data processing task, and the aggregate resource allocation strategy indicates the aggregator in the first network device among multiple network devices for performing aggregate operations on the processed data of the first data processing task; based on the task deployment strategy and the aggregate resource allocation strategy, the first data processing task is deployed in the first server. After the controller completes the deployment of the first data processing task, the first server is used to store the task data of the first data processing task and the resource information of the aggregate resources allocated to the first data processing task; the first network device is used to perform aggregate operations on the processed data of the first data processing task.

[0080] In a possible implementation, after determining the task deployment strategy and the aggregate resource allocation strategy, the controller sends the task data of the first data processing task and the resource information of the aggregate resources of the first network device allocated to the first data processing task to the first server. The first server receives and stores the task data and resource information, executes the first data processing task according to the task data, encapsulates the obtained processing data and resource information into a task message, and sends the task message to the first network device to instruct the aggregator allocated to the first data processing task in the first network device to perform an aggregation operation on the processing data carried in the task message.

[0081] Among them, the aggregation resource is at least one aggregator in the first network device that can be used to process the processing data of the first data processing task, and the resource information includes the number of aggregators and the first address of the aggregator.

[0082] As an example, see Figure 2 , the task message includes message information such as message header, task identifier, message sequence number, number of aggregators, aggregator first address and processing data. The message header can carry protocol type, Internet protocol (IP) address, etc.; the task identifier can be a task identity document (ID) or a task number; for some data processing tasks, when the data to be processed of the data processing task is large or the message carries limited data, the processing data of executing a data processing task can be divided into multiple task messages for transmission, and the message sequence number is the sequence number of the currently transmitted task message; the number of aggregators can be one or more; the first address of the aggregator is the address of the first aggregator.

[0083] In some embodiments, after receiving a task message, the network device uses hash addressing to determine an aggregator that performs aggregation operations on the processed data in the task message.

[0084] As an example, the network device addresses the aggregator via the following formula (1).

[0085] index=hash(taskID,seqN)%size+addr(1)

[0086] Among them, index is the aggregator determined by addressing, taskID is the task ID of the data processing task, seqN is the message sequence number, size is the number of aggregators, and addr is the first address of the aggregator.

[0087] In other words, the network device itself does not need to store the aggregator information corresponding to each data processing task. After receiving the task message, it can address the aggregator based on the message information carried in the task message. Among them, the number of aggregators and the aggregator head address are used to limit the aggregator area that can be used by a data processing task, and the task ID and message sequence number are used to determine an aggregator in the aggregator area corresponding to the data processing task that performs aggregation operations on the task message with the message sequence number. In this way, while ensuring that the processing data of the data processing task is normally aggregated by the first network device, it can also be ensured that the processing data in the task message with the same sequence number can complete the aggregation operation in one aggregator, thereby improving the aggregation flexibility and the accuracy of the aggregation results.

[0088] In some embodiments, for the above-mentioned task management system, the embodiment of the present application does not limit the number of data processing tasks submitted by the user at one time, and the frequency of task submission. One data processing task can be submitted at one time, or multiple data processing tasks can be submitted at one time. For ease of explanation, the embodiment of the present application only takes the deployment of a first data processing task as an example for explanation, and does not constitute a limit on the number of data processing tasks that can be deployed by the controller at one time.

[0089] As an example, if the user submits multiple data processing tasks, the controller can determine the aggregate resource utilization and bandwidth resource utilization of multiple network devices and the processing resource utilization of multiple servers. It can then determine the task deployment strategy and aggregate resource allocation strategy for each data processing task in turn based on the task running resources required for each data processing task, thereby deploying the multiple data processing tasks in the server cluster in turn; it can also combine the task running resources required for each data processing task in the multiple data processing tasks to uniformly coordinate the deployment of multiple data processing tasks, thereby deploying multiple data processing tasks in batches to the server cluster.

[0090] In some embodiments, in addition to traditional forwarding and routing functions, the network devices in the above-mentioned task management system also need to have certain programmability and caching capabilities, be able to identify messages transmitted within the network (including data messages, control messages and other messages), calculate and modify fields in the messages, and have certain caching capabilities.

[0091] As an example, the network device may be a programmable switch, which may be a top-of-rack switch (TOR) or a spine switch (also called a Spine switch). The embodiment of the present application does not limit the type of programmable switch. Moreover, the network device that performs the aggregation operation may be a TOR switch or a Spine switch, which is also not limited in the embodiment of the present application.

[0092] It should be noted that when the TOR switch is connected to multiple servers in the cabinet, the Spine switch can be directly connected to multiple TOR switches to form a two-layer network architecture including a cabinet layer and a Spine layer. Based on the two-layer network architecture, the processed data transmitted between multiple servers in the server cluster is aggregated in the network. A leaf layer (also called a Leaf layer) can also be added between the cabinet layer and the Spine layer. The Leaf layer includes multiple leaf switches, each leaf switch is connected to multiple TOR switches in the cabinet layer, and multiple leaf switches are connected to a Spine switch, that is, a three-layer network architecture including a cabinet layer, a Leaf layer, and a Spine layer is constructed. Based on the three-layer network architecture, the processed data transmitted between multiple servers in the server cluster is aggregated in the network. The embodiment of the present application does not limit the network hierarchy formed by the Spine switch and the TOR switch.

[0093] In addition, the embodiments of the present application do not limit the number of aggregators divided in each network device, and the amount of memory resources corresponding to the aggregators. That is, the number of aggregators in multiple network devices can be the same or different; the amount of memory resources corresponding to each aggregator in a network device can be the same or different.

[0094] In some embodiments, the controller may be any computer device with communication, data computing and storage capabilities. Figure 3 The computer device includes at least one processor 301 , a communication bus 302 , a memory 303 and at least one communication interface 304 .

[0095] The processor 301 may be a general-purpose central processing unit (CPU), a network processor (NP), a microprocessor, or may be one or more integrated circuits for implementing the solution of the present application, such as an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above-mentioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0096] The communication bus 302 is used to transmit information between the above components. The communication bus 302 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0097] The memory 303 may be a read-only memory (ROM), a random access memory (RAM), an electrically erasable programmable read-only memory (EEPROM), an optical disc (including a compact disc read-only memory (CD-ROM), a compressed optical disc, a laser disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0098] The memory 303 may exist independently and be connected to the processor 301 via the communication bus 302. The memory 303 may also be integrated with the processor 301.

[0099] The communication interface 304 uses any transceiver-like device for communicating with other devices or communication networks. The communication interface 304 includes a wired communication interface and may also include a wireless communication interface. Among them, the wired communication interface may be, for example, an Ethernet interface. The Ethernet interface may be an optical interface, an electrical interface, or a combination thereof. The wireless communication interface may be a wireless local area network (WLAN) interface, a cellular network communication interface, or a combination thereof, etc.

[0100] As an example, the processor 301 may include one or more CPUs, such as Figure 3 CPU0 and CPU1 are shown in the figure.

[0101] As an example, a computer device may include multiple processors, such as Figure 3 301 and processor 305 shown in FIG. Each of these processors may be a single-core processor or a multi-core processor. A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0102] In some embodiments, the computer device may also include an output device and an input device ( Figure 3The output device communicates with the processor 301 and can display information in a variety of ways. For example, the output device can be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector. The input device communicates with the processor 301 and can receive user input in a variety of ways. For example, the input device can be a mouse, a keyboard, a touch screen device, or a sensor device.

[0103] In some embodiments, the memory 303 is used to store program codes for executing the solution of the present application, and the processor 301 can execute the program codes stored in the memory 303. The program code may include one or more software modules, and the computer device may implement the task deployment method provided in any of the following embodiments through the processor 301 and the program codes in the memory 303.

[0104] Figure 4 is a flowchart of a task deployment method provided in an embodiment of the present application, which is applied to the above Figure 1 The controller 110 is shown.

[0105] Please refer to Figure 4 , the method comprises the following steps:

[0106] Step 401: The controller obtains the aggregate resource utilization of each network device, the bandwidth resource utilization of each network device, and the processing resource utilization of each server.

[0107] Among them, the aggregate resource utilization rate of the network device can be expressed as the ratio between the aggregate resources of the occupied aggregators in the network device and the total memory resources of the network device (that is, the aggregate resources of all aggregators contained in the network device). By analyzing the aggregate resource utilization rate of the network device, the available aggregate resources of the unoccupied aggregators in the network device (that is, the available aggregate resources that can be used for other data processing tasks) can be determined; similarly, the processing resource utilization rate of the server can be expressed as the ratio between the occupied processing resources in the server and the total processing resources. By analyzing the processing resource utilization rate of the server, the available processing resources in the server (that is, the processing resources that can be used to deploy other data processing tasks) can be determined. Each network device includes multiple ports, each of which can be connected to a server to transmit messages with the server through the port. Therefore, based on the data traffic of each port in the network device, the utilization rate of the bandwidth resources allocated to each port by the network device can be determined.

[0108] In some embodiments, the controller may obtain the processing resource utilization of the server by: the controller stores the total processing resources of each server in the server cluster, and records and updates the processing resource utilization of each server according to the deployment of data processing tasks in the task management system.

[0109] As an example, for any server in the server cluster, when the server just joins the server cluster and no data processing task is deployed, the controller records the processing resource utilization of the server as 100%. After the data processing task is deployed on the server, the controller calculates and updates the processing resource utilization of the server based on the task running resources required by the data processing task and the total processing resources of the server.

[0110] In some embodiments, the controller may obtain the bandwidth resource utilization of the network device by setting a counter at each port of the network device, and using the counter to count the total number of messages and the total length of messages flowing through the port within a period of time. The controller reads the total number of messages and the total length of messages counted by the counter of each port, and calculates the bandwidth resource utilization of each port by dividing the total length of the messages by the length of the statistical time period, and then determines the overall bandwidth resource utilization of the network device based on the bandwidth resource utilization of each port.

[0111] It should be noted that when the controller obtains the processing resource utilization of the server and the bandwidth resource utilization of the network device, it can be implemented in the above manner, or it can be implemented by referring to other related technologies. The embodiments of the present application do not limit this.

[0112] In some embodiments, the implementation process of the controller obtaining the aggregate resource utilization of the network device may include the following steps (1)-(3):

[0113] (1) The controller sends a measurement message to the network device. The measurement message carries period information of a first measurement period. The first measurement period is the measurement period in which the deployment time of the first data processing task is located.

[0114] The period length of the measurement period may be any set value, such as 5 minutes, 10 minutes, 20 minutes, etc. In different situations, the period length of the measurement period may be different.

[0115] Typically, the controller obtains the aggregate resource utilization of each network device, the bandwidth resource utilization of each network device, and the processing resource utilization of each server based on the task deployment requirements, that is, when there is a data processing task to be deployed. Therefore, the controller obtains the aggregate resource utilization of the network device through steps (1)-(3), which must be completed within one measurement cycle with the deployment process of the first data processing task.

[0116] That is, if there is a first data processing task to be deployed in the task management system within the first measurement period, the controller will deploy the first data processing task within the first measurement period after acquiring the aggregate resource utilization of the network device, the bandwidth resource utilization of the network device, and the processing resource utilization of the server. At this time, the deployment time of the first data processing task is within the first measurement period.

[0117] Based on this, when the controller needs to deploy the first data processing task within the first measurement period, the period information carried in the measurement message sent by the controller to the network device is the period information of the first measurement period.

[0118] In this way, when the deployment time of the first data processing task is within the measurement cycle of the controller executing the above step 401, the resource utilization obtained by the controller can reflect the actual usage of resources of multiple servers and multiple network devices within the measurement cycle. The task deployment strategy and aggregate resource allocation strategy determined based on the resource utilization within the measurement cycle are also more accurate, and the task deployment effect is better.

[0119] It should be understood that when there is no task deployment requirement, the controller may also periodically perform the above step 401, that is, according to the measurement cycle, obtain and record the aggregate resource utilization of each network device, the bandwidth resource utilization of each network device, and the processing resource utilization of each server within the measurement cycle. That is, the controller only counts and stores the resource utilization of each device, and updates the stored resource information after obtaining new data information in the next measurement cycle.

[0120] It should be noted that when the controller sends a measurement message to the network device, the controller is not aware of the address information of each network device and the communication link it is in, but the controller stores the address information of each server in the server cluster and the network device connected to each server. In addition, the basic function of the network device is to implement data forwarding between servers in the server cluster. Therefore, the controller can send a measurement message to the network device in the communication link based on the communication link between the controller and the server.

[0121] In one possible implementation, the implementation process of the controller sending a measurement message to the network device can be: the controller determines a target server from multiple servers, where the target server refers to a server connected to the network device, and the network device is located on the communication link between the controller and the target server; the controller sends a measurement message to the target server through the communication link between the controller and the target server, so that the network device receives and intercepts the measurement message.

[0122] For multiple servers connected to a network device, the controller can select a target server from the multiple servers based on the length of the communication link between the controller and the server, the working status of the server, etc.; or randomly select a server from the multiple servers as the target server. The embodiment of the present application does not limit the selection strategy of the target server.

[0123] It should be noted that the controller may send measurement messages to multiple network devices through the above step (1), or may send measurement messages only to the network device that performs aggregation operations among the multiple network devices, and the embodiment of the present application does not impose any limitation on this.

[0124] See also Figure 5 , assuming that multiple network devices in the task management system include Spine switches and TOR switches, and the network device that performs aggregation operations is a TOR switch, then when the controller obtains the aggregate resource utilization of the network device, it needs to send a measurement message to each TOR switch. Taking the first TOR switch as an example, the controller first selects server a as the target server from server a, server b, and server c connected to the first TOR switch. The controller writes the IP address of server a as the destination IP address into the message header of the measurement message, and sends the measurement message to server a through the communication link between the controller and server a.

[0125] It should be understood that when there are multiple network devices on the communication link between the controller and the target server, each network device will determine whether the target node indicated by the IP address is its own directly connected node based on the IP address carried in the message header after receiving the measurement message. If the target node is a directly connected node, the measurement message is intercepted; if it is not a directly connected node, the measurement message is forwarded to the next node based on the communication link between the controller and the target server.

[0126] Continue to see Figure 5 When the Spine switch receives the measurement message, it determines that the destination IP address carried in the header of the measurement message is not its own directly connected node, and then continues to forward the measurement message to the first TOR switch. After receiving the measurement message, the first TOR switch determines that the destination IP address is the IP address of server a to which it is directly connected, and then intercepts the measurement message.

[0127] In some embodiments, the network device performing the aggregation operation may also be a Spine switch. When the Spine switch receives the above-mentioned measurement message, it determines that the server indicated by the destination IP carried in the message header of the measurement message is connected to any TOR switch to which it is connected, and then intercepts the measurement message.

[0128] (2) The network device receives the measurement message, determines the number of occupied aggregators of the network device in the first measurement period based on the period information of the first measurement period, and sends a measurement reply message to the controller, where the measurement reply message carries the number of occupied aggregators.

[0129] The period information of the first measurement period is a version number corresponding to the first measurement period, and a first state counter is provided in the network device, and the first state counter has a first version number. After the network device receives the measurement message, in response to the version number corresponding to the first measurement period being the same as the first version number, the network device determines the number of occupied aggregators of the network device in the first measurement period based on the count value of the first state counter.

[0130] As an example, a measurement message includes a message header, a version number (ver) corresponding to the measurement period, and an empty count value field. The count value field is used for the network device to determine the number of occupied aggregators, fill the number of occupied aggregators into the count value field, and then generate a measurement reply message.

[0131] See also Figure 6 In this embodiment, the version number in the above measurement message is the version number version=2 corresponding to the first measurement period, and the value of the count value field can be expressed as cnt=0.

[0132] In other words, the first state counter is used to record the number of occupied aggregators in the network device during the current measurement period, and to update the first version number according to changes in the measurement period. During the first measurement period, whenever a task message is transmitted to the network device and an aggregator in the network device is required to perform aggregation operations on the processed data carried in the task message, the count value of the first state counter is automatically increased by one.

[0133] As an example, see Figure 7 , assuming that the version number corresponding to the first measurement cycle is ver.2, and the first version number of the first state counter is ver.2, for the first task message of task 3 (that is, the task ID is: task3, and the message sequence number is seq1), after the network device receives the task message pkt(task3, seq1), it is mapped to aggregator No. 4 in the network device by "hash(task3, seq1)%". At this time, aggregator No. 4 is empty (that is, the aggregation resources are in an idle state and no aggregation operation is performed), then the task message pkt(task3, seq1) can occupy aggregator No. 4 to perform aggregation operations on the processed data carried in the task message pkt(task3, seq1) through the aggregation resources of aggregator No. 4. At this time, the count value cnt of the first state counter is automatically increased by one, that is, the count value is updated from 4 to 5.

[0134] It should be understood that if the addressed aggregator is not empty, it means that the aggregator is currently performing aggregation operations, and its aggregation resources are occupied by processing data of other data processing tasks, and the counter of the first state counter does not increment.

[0135] When the count value of the first state counter is 5, it means that 5 aggregators in the network device are occupied. In this case, after receiving the measurement message, the network device reads the count value 5 of the first state counter according to the period information ver.2 of the first measurement period, and feeds back the count value to the controller through the measurement reply message.

[0136] See also Figure 8 The measurement reply message includes a message header, a version number version=2 corresponding to the first measurement cycle, and the number of occupied aggregators cnt=5 written into the count value field.

[0137] (3) The controller receives the measurement reply message and determines the aggregation resource utilization of the network device based on the number of occupied aggregators.

[0138] In one possible implementation, the controller may determine the aggregation resource utilization of a network device by: determining the maximum number of aggregations of the aggregators included in the network device within the first measurement period based on the total number of aggregators included in the network device, the single occupancy duration of the aggregators of the network device within the first measurement period, and the period length of the first measurement period; and determining the aggregation resource utilization of the network device based on the number of occupied aggregators and the maximum number of aggregations.

[0139] Among them, the task message will occupy the aggregation resources of the aggregator. After the aggregation operation is completed for the processing data carried in the task message, the aggregation resources of the aggregator will be released, and the aggregator can be used to process the task messages of other data processing tasks. Therefore, for an aggregator, its single occupancy time is the time from the network device receiving the task message to completing the aggregation operation of the processing data in the task message.

[0140] Taking the end-to-end communication of the server as an example, the implementation process of determining the single occupancy time of the aggregator can be as follows: for any server, the moment when the server sends the task message to the network device is taken as the starting time, and the moment when the server receives the task reply message fed back by the network device is taken as the ending time. Based on the length of time between the start time and the end time, the single occupancy time of the aggregator is determined.

[0141] That is, since the aggregator in the network device needs to perform aggregation operations on the processed data in the task message, the aggregator is in an occupied state when processing the task message. Therefore, the single occupancy time of the aggregator can also be expressed as the end-to-end round-trip time (RTT) of the server.

[0142] As an example, see Fig. 9 , server p sends a task message pkt(task1, seq1) to server q via the network device at 10:00. After hash addressing, the network device uses aggregator No. 1 to perform aggregation operations on the processed data carried in the task message pkt(task1, seq1), and sends the intermediate operation results to server q. After server q receives the intermediate operation results of task pkt(task1, seq1), it sends the task reply message ack(task1, seq1) corresponding to the task message pkt(task1, seq1) to server p via the network device. Server p receives the task reply message ACK(task1, seq1) at 10:01. The task reply message indicates that server q has accurately received the task message pkt(task1, seq1). In this case, the single occupancy time of aggregator No. 1 in the network device is the end-to-end round-trip time between server p and server q, that is, 1 minute.

[0143] It should be noted that since the network device includes multiple aggregators, the single occupancy time of the aggregator included in the network device can be the average value of the single occupancy time of multiple aggregators, or the maximum value of the single occupancy time of multiple aggregators. The embodiment of the present application does not impose any restrictions on this.

[0144] In one possible implementation, the implementation process of determining the maximum number of aggregation times of the aggregators included in the network device within the first measurement cycle may be: determining the number of occupancy times of each aggregator within the first measurement cycle based on the cycle length of the first measurement cycle and the single occupancy duration of the aggregators of the network device within the first measurement cycle; determining the maximum number of aggregation times of the network device within the first measurement cycle based on the number of occupancy times of each aggregator within the first measurement cycle and the total number of aggregators included in the network device.

[0145] As an example, the maximum number of aggregators of the network device in the first measurement period can be calculated by the following formula (2).

[0146]

[0147] In the formula, T is the cycle length of the first measurement cycle, t is the average single occupancy time of multiple aggregators included in the network device in the first measurement cycle, and N is the total number of aggregators included in the network device; T, t and N are all non-zero numbers.

[0148] The number of occupied aggregators can reflect the actual number of aggregations of the network device. Therefore, based on the number of occupied aggregators of the network device in the first measurement period and the maximum number of aggregations that the aggregators included in the network device can achieve in the first measurement period, the aggregation resource utilization of the network device in the first measurement period is determined.

[0149] As an example, the aggregate resource utilization of the network device in the first measurement period may be calculated using the following formula (3).

[0150]

[0151] In the formula, n is the number of occupied aggregators of the network device in the first measurement period, t is the average value of the single occupancy time of multiple aggregators included in the network device in the first measurement period, N is the total number of aggregators included in the network device, and T is the period length of the first measurement period; n, t, N and T are all non-zero numbers.

[0152] In some embodiments, after executing the above step (2), the controller may not receive a measurement reply message within the measurement duration, that is, the measurement reply message sent by the network device to the controller is lost. In this case, the controller can resend the measurement message to the network device, that is, repeat the above steps (1) and (2) to obtain the number of occupied aggregators of the network device in the first measurement period.

[0153] For the network device, when it receives the measurement message, it also needs to determine whether the measurement reply message is the first message in the first measurement cycle or a retransmission message. For the measurement messages sent by the controller in the same measurement cycle, the number of occupied aggregators carried in the measurement reply message fed back by the network device should be the same.

[0154] That is, in the first measurement cycle, the measurement message sent by the controller to the network device carries the period information of the first measurement cycle, but after determining that the measurement reply message is lost, the controller resends the measurement message to the network device, that is, the resent measurement message still carries the period information of the first measurement cycle. However, since the number of occupied aggregators of the network device may change during the interval between the two measurement messages sent by the controller, the count value of the first state counter that counts the number of occupied aggregators in the current measurement cycle also changes. If the network device continues to read the count value of the first state counter and feeds it back to the controller, the aggregate resource utilization of the network device obtained by the controller, the bandwidth resource utilization of each network device, and the processing resource utilization of each server will be out of sync in time, affecting the task deployment effect and the aggregate resource allocation effect.

[0155] Based on this, in order to avoid the situation where the network device determines different numbers of occupied aggregators for the same measurement message, two counters in two states can be set in the network device, which are used to record the number of occupied aggregators in the current measurement cycle and the number of occupied aggregators in the previous measurement cycle. By switching the state information of the two counters and updating the version number of the counters, it is ensured that only one state of the counter is used to record the number of occupied aggregators in each measurement cycle, and the states of the counters recording the number of occupied aggregators in adjacent measurement cycles are different.

[0156] In a possible implementation, the first state counter in the network device also has the first state information, and the network device is also provided with a second state counter, and the second state counter has the second state information and the second version number. After the network device sends a measurement reply message to the controller, in response to the first state information indicating the current measurement cycle and the second state information indicating the previous measurement cycle, the network device switches the first state information and the second state information, updates the second version number to the version number corresponding to the second measurement cycle, and clears the count value of the second state counter, so as to record the number of occupied aggregators in the second measurement cycle through the second state counter. The second measurement cycle is the next measurement cycle adjacent to the first measurement cycle.

[0157] See also Fig.10 , assuming that the cycle information of the first measurement cycle is ver2, the first state information of the first state counter in the network device is the current cycle state, and the corresponding first version number is ver2, that is, the first state counter is used to record the number of occupied aggregators in the network device during the first measurement cycle; the second state information of the second state counter is the previous cycle state, and the second version number is ver1, that is, the second state counter is used to record the number of occupied aggregators in the network device during the previous measurement cycle (the cycle information is ver1).

[0158] After receiving a measurement message carrying version number ver2, the network device compares the version number ver2 carried in the measurement message with the first version number of the first state counter and the second version number of the second state counter, and reads the count value cnt=5 of the first state counter which is the same as the version number ver2 in the measurement message, so as to feedback to the controller through a measurement reply message that the number of occupied aggregators of the network device in the first measurement cycle is 5.

[0159] After the network device sends a measurement reply message to the controller, the first state information and the second state information are switched. That is, the state information of the first state counter is switched to the state of the previous cycle, so as to record the number of occupied aggregators in the previous measurement cycle (i.e., the first measurement cycle when the measurement reply message is sent) through the first state counter, and the version number of the first state counter is still ver2; the state information of the second state counter is switched to the current cycle state, and at the same time, the second version number of the second state counter is updated to ver3, and the value of the second state counter is cleared, so as to record the number of occupied aggregators in the network device in the next measurement cycle through the second state counter.

[0160] In this case, if the controller does not receive the measurement reply message, it resends the measurement message carrying the ver2 version number. Fig.11 , when the network device receives a measurement message carrying the ver2 version number, it determines that the measurement message is the same as the version number of the first state counter, and the state information of the first state counter is the state of the previous cycle at this time, then it determines that the measurement message is a retransmission message. According to the version number, the network device reads the count value cnt=5 of the first state counter to feedback to the controller through a measurement reply message that the number of occupied aggregators of the network device in the first measurement cycle is 5. That is, regardless of whether it is the first measurement message or the retransmission measurement message, the number of occupied aggregators in the first measurement cycle counted by the network device is 5.

[0161] In this way, the network device can effectively identify whether the measurement message is the first message or the retransmission message based on the counters in two states, which solves the problem of counting the number of occupied aggregators in the network device in the case of packet loss and retransmission, so that for the same measurement message, the number of occupied aggregators carried in the feedback measurement reply message of the network device is the same.

[0162] Step 402: The controller determines a task deployment strategy and an aggregate resource allocation strategy based on the task running resources required for the first data processing task to be deployed, the aggregate resource utilization and bandwidth resource utilization of multiple network devices, and the processing resource utilization of multiple servers. The task deployment strategy indicates a first server among the multiple servers for executing the first data processing task, and the aggregate resource allocation strategy indicates an aggregator in a first network device among the multiple network devices for performing aggregate operations on the processed data of the first data processing task.

[0163] In some embodiments, the implementation process of step 402 may be: determining the deployment constraints of the first data processing task based on the task running resources, the aggregate resource utilization and bandwidth resource utilization of multiple network devices, and the processing resource utilization of multiple servers; and determining the task deployment strategy and the aggregate resource allocation strategy based on the deployment constraints.

[0164] That is, the controller determines the first server on which to deploy the first data processing task and the aggregator in the first network device that performs aggregation operations on the processed data of the first data processing task from the multiple servers in the server cluster based on the task running resources required for the first data processing task to be deployed, the aggregate resource utilization and bandwidth resource utilization of multiple network devices, and the processing resource utilization of multiple servers.

[0165] It should be understood that after the networking is completed, the communication route between the network device and the server is determined. Therefore, after determining the first server, the first network device connected to the first server can be further determined, and the aggregation resource allocation strategy is to select an aggregator from the multiple aggregators included in the first network device to perform aggregation operations on the processed data of the first data processing task.

[0166] It should be noted that, for the first data processing task, it can be deployed on one server or on multiple servers for parallel execution, and the embodiment of the present application does not limit the number of first servers on which the first data processing task can be deployed. Similarly, the embodiment of the present application does not limit the number of aggregators in the first network device for performing aggregation operations on the processed data of the first data processing task.

[0167] The deployment constraints include:

[0168] (1) The task running resources are less than or equal to the available processing resources of the server capable of executing the first data processing task.

[0169] The available processing resources of the server are used to indicate whether the server can continue to deploy data processing tasks. When the task running resources of a data processing task are less than or equal to the available processing resources of the server, it means that the server can meet the running resource requirements of the data processing task, so the data processing task can be deployed on the server; and when the task running resources of a data processing task are greater than the available processing resources of the server, it means that the server cannot meet the running resource requirements of the data processing task, so the data processing task cannot be deployed on the server.

[0170] That is, for any server in the server cluster, if the available processing resources of the server can meet the running resource requirements of the first data processing task, then the first data processing task can be considered to be deployed on the server.

[0171] (2) The processing resource usage of the server capable of executing the first data processing task is less than or equal to the processing resource threshold.

[0172] The processing resource threshold is used to indicate whether the server is in an overloaded state. When the processing resource usage of the server capable of executing the first data processing task is less than or equal to the processing resource threshold, it means that even if the first data processing task is deployed on the server, the server can still work normally and there is no downtime due to overload operation; when the processing resource usage of the server capable of executing the first data processing task is greater than the processing resource threshold, it means that after the first data processing task is deployed on the server, the server may be overloaded and it is difficult to effectively execute the data processing task deployed therein, and there may even be a situation where the server crashes due to overload operation.

[0173] That is, after determining that the server meets the operating resource requirements of the first data processing task based on the above deployment constraint condition (1), it is also necessary to ensure that after the first data processing task is deployed on the server, the server will not be overloaded. That is, the processing resource usage of the server where the first data processing task is deployed must not exceed the processing resource threshold of the server.

[0174] It should be noted that the above deployment constraints (1) and (2) are used to screen servers from the server cluster that can execute the first data processing task, and then further screen servers that meet the requirements of the first data processing task through the following deployment constraints (3) and (4), as well as network devices that can perform the processing data aggregation operations obtained by executing the first data processing task on the servers.

[0175] (3) The bandwidth resource between the server capable of executing the first data processing task and the network device capable of performing aggregation operations on the processed data of the first data processing task is less than or equal to the bandwidth threshold.

[0176] The bandwidth threshold is used to indicate bandwidth resources where data packet loss may occur. When the bandwidth resource is less than or equal to the bandwidth threshold, it indicates that the amount of data transmitted on the communication link between the server and the network device is small, and the bandwidth resources configured for the communication link can meet the actual data transmission requirements of the first data processing task, and the processed data of the first data processing task can be normally transmitted between the server and the network device; when the bandwidth resource is greater than the bandwidth threshold, it indicates that the amount of data transmitted on the communication link between the server and the network device is large, and the bandwidth resources configured for the communication link cannot meet the actual data transmission requirements of the first data processing task, and data packet loss may occur when the processed data of the first data processing task is transmitted based on the communication link.

[0177] In other words, since the server that deploys the first data processing task needs to send the processed data to the corresponding network device for aggregation calculations after executing the first data processing task, when the bandwidth resources for transmitting the processed data between the server and the network device are less than or equal to the bandwidth threshold, the server that deploys the first data processing task can communicate normally with the network device that performs aggregation calculations on the processed data of the first data processing task.

[0178] (4) The memory resource occupancy of the network device capable of performing aggregation operations on the processing data of the first data processing task is less than or equal to the memory resource threshold.

[0179] The memory resource threshold is used to indicate whether the network device is in an overload state. When the memory resource usage of the network device is less than or equal to the memory resource threshold, it means that the memory resources of the network device can perform aggregate operations on all the processed data sent by the server, and there is no overload operation; when the memory resource usage of the network device is greater than the memory resource threshold, it means that the memory resources of the network device cannot perform aggregate operations on all the processed data sent by the server, and some data will be returned to the server for aggregate operations, or delayed operations, affecting the task running time.

[0180] That is, when the server capable of executing the first data processing task is connected to multiple network devices, it is necessary to determine which network device the processing data of the first data processing task is to be sent to for aggregation calculation based on the memory resource usage of each network device.

[0181] In one possible implementation, the implementation process of determining the task deployment strategy and the aggregate resource allocation strategy based on the deployment constraints may be: based on the deployment constraints, determine at least one candidate deployment scheme that can execute the first data processing task, the candidate deployment scheme including candidate servers and candidate network devices; based on preset scheme screening conditions, determine the first server for deploying the first data processing task from the candidate servers included in the at least one candidate deployment scheme, and determine the first network device for performing aggregate operations on the processed data of the first data processing task from the candidate network devices connected to the first server; and determine, in the first network device, an aggregator for performing aggregate operations on the processed data of the first data processing task based on the task running resources of the first data processing task.

[0182] The scheme screening condition is that the task completion time of the candidate deployment scheme is the shortest. That is, for the candidate server and the candidate network device included in the candidate deployment scheme, if the first data processing task is deployed on the candidate server, the candidate server executes the first data processing task, sends the processed data of the first data processing task to the candidate network device, and the candidate network device performs the aggregation operation on the processed data, the candidate server in the candidate deployment scheme is determined as the first server, and the candidate network device is determined as the first network device.

[0183] It should be understood that, by determining the candidate deployment schemes through the above deployment constraints (1)-(4), when the number of candidate deployment schemes is 1, the candidate server in the candidate deployment scheme is used as the first server for executing the first data processing task, and the candidate network device in the candidate deployment scheme is used as the first network device for performing aggregation operations on the processed data of the first data processing task. When the number of candidate deployment schemes is not 1, that is, when there are multiple candidate deployment schemes, the first server for executing the first data processing task and the aggregator in the first network device for performing aggregation operations on the processed data of the first data processing task are determined based on the scheme screening conditions and the multiple candidate deployment schemes.

[0184] Optionally, in a specific implementation, the above scheme screening conditions may be used as optimization targets, and then based on the optimization targets and the above deployment constraints, the task deployment strategy and the aggregate resource allocation strategy may be solved by modeling. For example, the task deployment strategy and the aggregate resource allocation strategy for deploying the first data processing task may be solved based on integer linear programming (ILP).

[0185] Step 403: The controller deploys the first data processing task in the first server based on the task deployment strategy and the aggregate resource allocation strategy.

[0186] In some embodiments, the implementation process of step 403 may be: the controller sends the task data of the first data processing task and the resource information of the aggregated resources allocated to the first data processing task to the first server, wherein the resource information of the aggregated resources includes the number of aggregators and the first address of the aggregator.

[0187] Based on this, the first server stores the task data and resource information of the first data processing task. In response to the task running instruction, the first data processing task is executed to obtain the processed data; and the processed data, the number of aggregators of the first data processing task and the first address of the aggregator are encapsulated into a task message, and then the task message is sent to the first network device. After receiving the task message, the first network device determines the aggregator in the first network device according to the number of aggregators and the first address of the aggregator of the first data processing task, and the aggregator performs aggregation operations on the processed data carried in the task message.

[0188] In an embodiment of the present application, when deploying the first data processing task in the task management system, the controller can comprehensively analyze the aggregate resource utilization of each network device, the bandwidth resource utilization of each network device, the processing resource utilization of each server, and the task computing resources required for the first data processing task, thereby determining the task deployment strategy and aggregate resource allocation strategy for the first data processing task. Since the processing resources of the server, the bandwidth resources and aggregate resources of the network device are fully considered, the task deployment and resource allocation are more reasonable, which improves the overall execution performance of the first data processing task and reduces the task completion time.

[0189] Based on the above task deployment method, since multiple data processing can share the aggregator of the network device, that is, the aggregator of a network device may need to perform aggregation operations on the processing data of multiple data processing tasks, for low-priority data processing tasks, the aggregation resources of the aggregator are used in a preemptive manner; for high-priority data processing tasks, the aggregation resources of the aggregator are used in an exclusive manner.

[0190] Based on this, the aggregators included in the network device can be divided into two areas: the isolation area and the shared area. The aggregation resources of the aggregators in the isolation area are exclusively used by high-priority data processing tasks and are only used to process the processing data of the high-priority data processing tasks; if the high-priority data processing tasks are not deployed in the task management system at the current moment, the aggregators in the network device belong to the shared area for multiple low-priority data processing tasks to preempt, that is, the low-priority data processing tasks use the aggregator addressing method to preempt the aggregators in the shared area, so as to perform aggregation operations on the processing data of the low-priority data processing tasks through the preempted aggregators, and the processing data of the low-priority data processing tasks that have not preempted the aggregators are returned to the server for aggregation operations.

[0191] It should be noted that the isolation zone is a dynamic area. Whether an aggregator in a network device is located in the isolation zone depends on whether a data processing task that requires exclusive use of the aggregator is deployed in the task management system to which the network device belongs. In other words, the network device may include aggregators in the isolation zone or only include aggregators in the shared zone. Only when there is a data processing task that requires exclusive use of the aggregator in the task management system, the aggregators in the network device will be divided into the isolation zone and the shared zone.

[0192] Moreover, here only the priority of the data processing task is used to illustrate the aggregator that can be used by the data processing task, but in actual applications, other methods can also be used to determine whether the data processing task requires an exclusive aggregator or a shared aggregator, and the embodiments of the present application do not limit this.

[0193] In addition, the server that performs data processing tasks is unaware of the isolation area and shared area in the network device. It only needs to send the task message to the network device according to the resource information of the allocated aggregated resources, and the network device will map the task message to the corresponding aggregator.

[0194] As an example, see Fig.12 Assume that task1 is a data processing task that exclusively occupies the aggregator. It exclusively occupies aggregator No. 0 and aggregator No. 1 in the shared area. It uses the aggregation resources of aggregator No. 0 and aggregator No. 1 to perform aggregation operations on the processed data of task1. Task2 and task3 are data processing tasks that share the aggregator. They use aggregator addressing to seize the aggregator from the shared area, that is, they use the aggregation resources of aggregator No. 2 to aggregator No. 7 to perform aggregation operations on the processed data of task2 and task3.

[0195] It should be noted that Fig.12 Taking task2 deployed on server 1 and server 2 and sending three task messages to the network device in sequence as an example, the task running order of task2 and task3 and the specific preempted aggregator are not restricted.

[0196] When a new data processing task task4 is added to the network device, the aggregator in the shared area will be occupied by task4. When task4 is a data processing task that shares an aggregator, task2, task3 and task4 use the aggregator in the shared area in a preemptive manner; when task4 is a data processing task that exclusively occupies an aggregator, task4 needs to determine at least one aggregator that it exclusively occupies from the aggregators in the shared area, so that the exclusive aggregator can perform aggregation operations on the processing data of task4, and task2 and task3 can no longer preempt the aggregator exclusively occupied by task4, resulting in fewer aggregators originally available to task2 and task3.

[0197] That is, when the data processing task task4 that needs an exclusive aggregator needs to use the aggregation resources of the network device, it will determine its own exclusive aggregator from the shared area of ​​the network device, causing the aggregators of other data processing tasks that use the shared area aggregators in the network device to change, and the corresponding aggregator addressing method also needs to change.

[0198] See also Fig.13 If task4 occupies aggregators 2 and 3 in the shared area that originally belonged to task2 and task3, the aggregators available to task2 and task3 in the shared area will become aggregators 4 to 7. When multiple servers executing task2 and task3 send task messages to the network device, if the aggregator switching time is different, the aggregator addressing of the task message with the same sequence number may be different.

[0199] For example, for server 1 and server 2 that deploy task2, server 1 switches slowly and still uses the old addressing, selecting aggregator No. 3 to aggregate the processed data carried in message No. 3; server 2 switches quickly and uses the new addressing to select aggregator No. 5 to aggregate the processed data carried in message No. 3. At this time, for message No. 3 of task2, one is mapped to aggregator No. 3 and the other is mapped to aggregator No. 5. The same task message occupies two different aggregators, resulting in the fact that the processed data in the two task messages can never be aggregated, and the aggregation resources of the corresponding aggregators cannot be released.

[0200] Therefore, in order to ensure the accuracy of the aggregation operation and the aggregation resource utilization of the aggregator in the network device, when a new data processing task that exclusively occupies the aggregator is added to the network device, the aggregator must be reallocated for the data processing task of the aggregator using the shared area, and the servers corresponding to the data processing tasks of the aggregator using the shared area must be instructed to perform the aggregator re-addressing operation at the same time.

[0201] See also Fig.14 When the first data processing task is a data processing task that requires an exclusive aggregator, the task deployment method further includes the following steps:

[0202] Step 404: The controller determines the deployed second data processing task, and the processing data of the second data processing task and the first data processing task both need to be aggregated in the first network device, and the second data processing task refers to a data processing task that can share an aggregator with other data processing tasks.

[0203] That is, the second data processing task is a data processing task that uses the aggregation resources of the aggregator in the shared area of ​​the first network device. When the first data processing task needs to exclusively occupy at least one aggregator from the shared area of ​​the first network device, it will cause changes in the aggregator that the second data processing task can occupy in the shared area.

[0204] It should be noted that, before the first data processing task is deployed, the aggregator in the shared area of ​​the first network device can be used to perform aggregation operations on the processing data of one data processing task, or can be used to perform aggregation operations on the processing data of multiple data processing tasks, and the embodiment of the present application does not limit this. That is, the number of second data processing tasks can be one or more, and the embodiment of the present application only takes one as an example for explanation. When there are multiple data processing tasks, steps 404-406 can also be used to perform a one-time batch execution of the aggregator addressing reconfiguration operation.

[0205] Step 405: The controller sends a task suspension message to the second server, where the second server is a server among the multiple servers that executes the second data processing task.

[0206] Among them, the task pause message is used to instruct the second server to suspend sending the task message after the target sequence number of the second data processing task, so as to avoid the situation where an exclusive aggregator is allocated to the first data processing task in the shared area of ​​the first network device, resulting in the processing data carried in the task message sent by the second server cannot be effectively aggregated by the aggregator.

[0207] After receiving the task suspension message and sending the task message of the target sequence number of the second data processing task, the second server suspends sending the task message of the second data processing task to the first network device and sends a suspension response message to the controller.

[0208] In some embodiments, for a distributed deployment task, the second server includes a parameter server and multiple computing servers, the parameter server is used to store the task parameters of the second data processing task, and each computing server is used to execute the second data processing task based on the task parameters. Based on this, the implementation process of pausing the second data processing task may include the following steps (1)-(4):

[0209] (1) The controller sends a task suspension message to the parameter server, so that the parameter server controls the multiple computing servers to suspend sending task messages after the target sequence number of the second data processing task.

[0210] The pause message carries the task identifier of the second data processing task, which is used to indicate that the data processing task that the parameter server currently needs to pause is the second data processing task.

[0211] (2) The parameter server receives the task pause message, determines the maximum message sequence number that has completed the aggregation operation at the current moment, and sends the maximum message sequence number to multiple computing servers.

[0212] After receiving the task suspension message, the parameter server determines, based on the task identifier carried in the task suspension message, multiple computing servers that execute the second data processing task from computing servers that communicate with the parameter server.

[0213] Normally, each computing server executes the second data processing task according to the task parameters of the second data processing task to obtain processing data, and encapsulates the processing data in a task message, and sends it to the parameter server via the first network device, so that the parameter server updates the task parameters based on the processing data, and feeds back a task reply message to the computing server. In this process, the first network device receives and intercepts the task messages sent by each computing server, performs aggregation operations on the processing data carried in the task message, obtains an intermediate calculation result, and sends the intermediate calculation result to the parameter server. After receiving and aggregating the intermediate calculation result of the second data processing task sent by the first network device, the parameter server feeds back a task reply message to the computing server that executes the second data processing task through the first network device.

[0214] Among them, the task reply message with the maximum message sequence number sent by the parameter server to the computing server carries the maximum message sequence number and a pause identifier, and the pause identifier is used to indicate that based on the task message of the computing server based on the maximum message sequence number, an operation of pausing the second data processing task is performed.

[0215] In other words, the maximum message sequence number can be carried in the task reply message corresponding to the maximum message sequence number. When the task reply message includes a pause identifier, it means that the task reply message is not only used to feedback to the computing server that the task message has been completely received, but also to instruct the computing server to perform a task pause operation.

[0216] See also Fig.15 In response to the task pause message, after the parameter server aggregates the task message with the largest message sequence number, the task reply message sent to each computing server includes a message header, a task identifier, a maximum message sequence number and a pause identifier. In this embodiment, the task identifier in the task reply message is the task ID or task number of the second data processing task.

[0217] Therefore, after receiving the task reply message carrying the pause identifier, the computing server can perform the task pause operation through the following step (3) based on the maximum message sequence number carried in the task reply message.

[0218] (3) The computing server determines the target sequence number based on the maximum message sequence number and the size of the message sending window, and suspends sending the task message of the second data processing task after sending the task message of the target sequence number of the second data processing task.

[0219] Among them, the message sending window is used to limit the number of task messages sent by the computing server at one time. That is, within this message sending window, the computing server does not need to wait for the parameter server to send a task reply message, and can continue to send task messages until the corresponding number of task messages in the message sending window have been sent.

[0220] In some embodiments, the size of the message sending window can be flexibly set and adjusted. For example, when the message sending window is set to be large, too many messages are sent at one time, resulting in excessive pressure on data processing and data transfer of the first network device, the size of the message sending window can also be adjusted to reduce the number of task messages sent by the computing server at one time, and the embodiments of the present application do not limit this.

[0221] As an example, see Fig.16 , assuming that the message sending window is 5, that is, 5 task messages can be sent continuously at one time. For computing server 1 and computing server 2 executing the second data processing task, after sending task message No. 9 to the parameter server through the network device, if the parameter server receives the task pause message indicating the suspension of the second data processing task at this time, and determines that the maximum message sequence number of the currently completed aggregation is 9, then the task reply message fed back to each computing server includes the maximum message sequence number 9 and the pause identifier (i.e. Fig.16 The task reply message ACK9 indicates that the task pause operation is performed with task message No. 9 as the pause starting point.

[0222] After receiving the ACK9 message, computing server 1 and computing server 2 use task message No. 9 as the pause starting point, send 5 task messages in sequence until task message No. 13, and then stop sending task messages. At this time, after the network device processes the processing data carried by each task message, it feeds back the intermediate processing results to the parameter server, and the parameter server completes the aggregation of task messages No. 10 to No. 13 in sequence.

[0223] It should be noted that, since the maximum message sequence number is carried in the task reply message, when the task reply message is lost, it will cause the time difference between the task reply messages received by each computing server for the same task message, and the maximum message sequence number carried in the task reply message will also be different. In this case, if the second data processing task is directly suspended based on the maximum message sequence number, some computing servers may stop at the maximum message sequence number, and some computing servers may stop at the message sequence number corresponding to the task reply message received because they did not receive the task reply message, which will lead to inconsistency in the task message sequence numbers sent by multiple computing servers corresponding to the second data processing task. When the message sequence numbers sent by multiple computing servers are not synchronized, the first network device cannot effectively aggregate them according to the message sequence number.

[0224] Based on this, in order to ensure that the computing server can respond to the same maximum message sequence number and stop at the task message with the same sequence number, after receiving the maximum message sequence number, the computing server sends the task message with the target sequence number and then stops sending the task message of the second data processing task. That is, even if there is a time difference in sending task messages among the computing servers, they can all be consistent in the end and stop at the task message with the same sequence number.

[0225] Based on this, in some embodiments, if there is a target computing server among multiple computing servers that has not received the task reply message with the maximum message sequence number, but has received a task reply message with the next message sequence number of the maximum message sequence number, the target computing server re-sends the task message with the maximum message sequence number to the parameter server; the parameter server re-sends the task reply message with the maximum message sequence number to the target computing server.

[0226] By having the target computing server resend the task message with the maximum message sequence number, it can be ensured that each computing server can perform the task pause operation based on the same maximum message sequence number. In this way, even if there is a time difference in the task reply message with the maximum message sequence number received by each computing server, the task pause operation can be performed based on the maximum message sequence number determined by the parameter server at the same time, so that the message sequence numbers of the task messages sent at the final stop time are the same, and the parameter server can effectively aggregate the messages before this time in sequence.

[0227] As an example, see Fig.17, after computing server 1 receives the task reply message, i.e., the ACK9 message, it takes task message No. 9 as the pause starting point, sends 5 task messages in sequence to task message No. 13, and then stops sending task messages. However, computing server 2 did not receive the ACK9 message, but received ACK10 after sending task message No. 10. At this time, computing server 2 determines that the task reply message is lost, and then resends task message No. 9 to the parameter server. The parameter server carries the maximum message sequence number 9 and the pause flag after the feedback task reply message. In this way, although computing server 1 and computing server 2 receive ACK9 at different times, they can eventually suspend the execution of the second data processing task after sending task message No. 13, and their pause time is consistent.

[0228] (4) After receiving the task messages of the target sequence number of the second data processing task sent by the multiple computing servers, the parameter server sends a pause response message to the controller.

[0229] That is, after the parameter server receives the task message of the target sequence number sent by the multiple computing servers executing the second data processing task, it determines that the multiple computing servers have all suspended the execution of the second data processing task and will no longer send the task message through the first network device. At this time, the parameter server sends a pause response message to the controller to inform the controller that the second data processing task has been suspended.

[0230] Step 406: The controller receives the pause response message sent by the second server, and sends an addressing reconfiguration message to the second server, where the addressing reconfiguration message indicates the aggregator reallocated for the second data processing task in the first network device.

[0231] The addressing reconfiguration message carries the task identifier of the second data processing task, the number of reallocated aggregators and the first address of the reallocated aggregator.

[0232] As an example, see Fig.18 The addressing reconfiguration message includes a message header, a task identifier, the number of reallocated aggregators, and the first address of the reallocated aggregator. In this embodiment, the task identifier in the addressing reconfiguration message is the task ID or task number of the second data processing task.

[0233] In this way, after receiving the addressing reconfiguration message, the second server can continue to send task messages after the target sequence number to the first network device according to the reallocated aggregator.

[0234] In an embodiment of the present application, when the first data processing task to be deployed is a data processing task that exclusively occupies an aggregator, the first data processing task is deployed in the task management system, and the processing data of the first data processing task is aggregated through the first network device, it is necessary to reallocate the aggregator to the second data processing task that shares the first network device aggregator with other data processing tasks, so as to avoid the situation where the task messages of the first data processing task and the second data processing task occupy the same aggregator, resulting in the inability to effectively aggregate the task messages of the same sequence number and the aggregation resources of the aggregator cannot be released. In this way, through the reallocation of the aggregation resources of the aggregator, it can be ensured that the first network device effectively aggregates the processing data of each data processing task, and each data processing task can run normally. Secondly, by setting the target sequence number, the task message sending progress of the second data processing task can be unified, so that multiple computing servers executing the second data processing task can be suspended at the task message with the same sequence number, and the second data processing task can be restarted based on the task message with the same sequence number, so that the task messages of each sequence number of the second data processing task before and after the suspension can be aggregated normally.

[0235] Fig.19 is a schematic diagram of a task deployment device provided in an embodiment of the present application. The task deployment device can be implemented as part or all of a computer device by software, hardware, or a combination of both. The computer device can be Figure 1 Controller 110 is shown. Fig.19 The task deployment device 1900 includes: a resource measurement module 1901, a strategy determination module 1902 and a task deployment module 1903.

[0236] Resource measurement module 1901, used to obtain the aggregate resource utilization of each network device, the bandwidth resource utilization of each network device, and the processing resource utilization of each server;

[0237] The strategy determination module 1902 is used to determine the task deployment strategy and the aggregate resource allocation strategy according to the task running resources required by the first data processing task to be deployed, the aggregate resource utilization and bandwidth resource utilization of multiple network devices, and the processing resource utilization of multiple servers, wherein the task deployment strategy indicates a first server among the multiple servers for executing the first data processing task, and the aggregate resource allocation strategy indicates an aggregator in the first network device among the multiple network devices for performing an aggregate operation on the processed data of the first data processing task;

[0238] The task deployment module 1903 is used to deploy the first data processing task in the first server based on the task deployment strategy and the aggregate resource allocation strategy.

[0239] Optionally, the resource measurement module 1901 includes:

[0240] A sending unit, configured to send a measurement message to a network device, wherein the measurement message carries period information of a first measurement period, where the first measurement period is the measurement period in which the deployment time of the first data processing task is located;

[0241] A receiving unit, configured to receive a measurement reply message sent by a network device, wherein the measurement reply message carries the number of aggregators occupied by the network device in a first measurement cycle;

[0242] The first determining unit is used to determine the aggregation resource utilization rate of the network device based on the number of occupied aggregators.

[0243] Optionally, the sending unit is specifically configured to:

[0244] Determining a target server from a plurality of servers, the target server being a server connected to a network device, and the network device being located on a communication link between the controller and the target server;

[0245] The measurement message is sent to the target server through the communication link between the controller and the target server, so that the network device receives and intercepts the measurement message.

[0246] Optionally, the first determining unit is specifically configured to:

[0247] Determine the maximum number of aggregation times of the aggregators included in the network device in the first measurement period according to the total number of aggregators included in the network device, the single occupation duration of the aggregators of the network device in the first measurement period, and the period length of the first measurement period;

[0248] Based on the number of occupied aggregators and the maximum number of aggregations, the aggregation resource utilization of the network device is determined.

[0249] Alternatively, see Fig. 20 , the task deployment device 1900 further includes:

[0250] The task determination module 1904 is used to determine the deployed second data processing task, where both the processed data of the second data processing task and the first data processing task need to be aggregated in the first network device, and the second data processing task refers to a data processing task that can share an aggregator with other data processing tasks;

[0251] The task suspension module 1905 is used to send a task suspension message to the second server to instruct the second server to suspend sending task messages after the target sequence number of the second data processing task, and the second server is a server that executes the second data processing task among the multiple servers;

[0252] The reconfiguration module 1906 is configured to receive a pause response message sent by the second server, and send an addressing reconfiguration message to the second server, wherein the addressing reconfiguration message indicates an aggregator reallocated in the first network device for the second data processing task.

[0253] Optionally, the second server includes a parameter server and a plurality of computing servers, the parameter server is used to store task parameters of the second data processing task, and each computing server is used to execute the second data processing task based on the task parameters;

[0254] The task suspension module 1905 is specifically used to: send a task suspension message to the parameter server, so that the parameter server controls the multiple computing servers to suspend sending the task messages after the target sequence number of the second data processing task;

[0255] The reconfiguration module 1906 is further used to: receive a pause response message sent by the parameter server, where the pause response message indicates that the multiple computing servers have all paused sending task messages after the target sequence number of the second data processing task.

[0256] Optionally, the policy determination module 1902 includes:

[0257] A second determining unit is used to determine the deployment constraint condition of the first data processing task based on the task running resources, the aggregate resource utilization and bandwidth resource utilization of the plurality of network devices, and the processing resource utilization of the plurality of servers;

[0258] The third determining unit is used to determine the task deployment strategy and the aggregate resource allocation strategy based on the deployment constraint condition.

[0259] Optionally, the deployment constraints include:

[0260] The task running resources are less than or equal to the available processing resources of the server capable of executing the first data processing task;

[0261] The processing resource usage of the server capable of executing the first data processing task is less than or equal to the processing resource threshold;

[0262] The bandwidth resource between the server capable of executing the first data processing task and the network device capable of performing aggregation calculation on the processed data of the first data processing task is less than or equal to the bandwidth threshold;

[0263] The memory resource occupancy of the network device capable of performing aggregation calculations on the processing data of the first data processing task is less than or equal to the memory resource threshold.

[0264] In an embodiment of the present application, when the first data processing task is deployed in the task management system through the task deployment device, the task deployment device can comprehensively analyze the aggregate resource utilization rate of each network device, the bandwidth resource utilization rate of each network device, the processing resource utilization rate of each server, and the task computing resources required for the first data processing task, thereby determining the task deployment strategy and aggregate resource allocation strategy for the first data processing task. Since the usage of the processing resources of the server, the bandwidth resources of the network device, and the aggregate resources are fully considered when determining the task deployment strategy and the aggregate resource allocation strategy, the task deployment and resource allocation are more reasonable, which improves the overall execution performance of the first data processing task and reduces the task completion time.

[0265] It should be noted that when the task deployment device provided in the above embodiment deploys data processing tasks in the task management system, only the division of the above functional modules is used as an example. In actual applications, the above functional distribution can be completed by different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the task deployment device provided in the above embodiment and the task deployment method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0266] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital versatile disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)). It is worth noting that the computer-readable storage medium mentioned in the embodiment of the present application may be a non-volatile storage medium, in other words, a non-transient storage medium.

[0267] It should be understood that the "multiple" mentioned herein refers to two or more. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate the clear description of the technical solution of the embodiments of the present application, in the embodiments of the present application, the words "first", "second" and the like are used to distinguish between the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not limit them to be necessarily different.

[0268] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the user information involved in the embodiments of this application and the task information of the data processing tasks submitted by the user are all obtained with full authorization.

[0269] The above-mentioned embodiments are provided for the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A task deployment method, characterized in that: A controller applied to a task management system, wherein the task management system further comprises a server cluster and a plurality of network devices, wherein the server cluster comprises a plurality of servers, and each of the network devices comprises a plurality of aggregators obtained by dividing the memory resources of the network device; the method comprises: Obtaining the aggregate resource utilization of each of the network devices, the bandwidth resource utilization of each of the network devices, and the processing resource utilization of each of the servers; Determine a task deployment strategy and an aggregate resource allocation strategy according to the task running resources required for the first data processing task to be deployed, the aggregate resource utilization and bandwidth resource utilization of the multiple network devices, and the processing resource utilization of the multiple servers, wherein the task deployment strategy indicates a first server among the multiple servers for executing the first data processing task, and the aggregate resource allocation strategy indicates an aggregator in a first network device among the multiple network devices for performing an aggregate operation on the processed data of the first data processing task; Based on the task deployment strategy and the aggregate resource allocation strategy, the first data processing task is deployed in the first server.

2. The method according to claim 1, characterized in that The obtaining of the aggregate resource utilization of each of the network devices includes: Sending a measurement message to the network device, where the measurement message carries period information of a first measurement period, where the first measurement period is the measurement period in which the deployment time of the first data processing task is located; receiving a measurement reply message sent by the network device, wherein the measurement reply message carries the number of aggregators occupied by the network device in the first measurement period; Based on the number of occupied aggregators, an aggregation resource utilization rate of the network device is determined.

3. The method according to claim 2, characterized in that The sending the measurement message to the network device includes: Determine a target server from the multiple servers, the target server being a server connected to the network device, and the network device being located on a communication link between the controller and the target server; The measurement message is sent to the target server through a communication link between the controller and the target server, so that the network device receives and intercepts the measurement message.

4. The method according to claim 2, characterized in that The determining, based on the number of occupied aggregators, the aggregation resource utilization of the network device comprises: Determine a maximum number of aggregation times of the aggregators included in the network device in the first measurement period according to the total number of aggregators included in the network device, a single occupancy duration of the aggregators of the network device in the first measurement period, and a period length of the first measurement period; Based on the number of occupied aggregators and the maximum aggregation times, an aggregation resource utilization rate of the network device is determined.

5. The method according to any one of claims 1 to 4, characterized in that: The first data processing task refers to a data processing task that requires an exclusive aggregator; the method further includes: Determine a deployed second data processing task, where both the processed data of the second data processing task and the first data processing task need to be aggregated in the first network device, and the second data processing task refers to a data processing task that can share an aggregator with other data processing tasks; Sending a task suspension message to a second server to instruct the second server to suspend sending task messages after the target sequence number of the second data processing task, where the second server is a server among the multiple servers that executes the second data processing task; Receive a pause response message sent by the second server, and send an addressing reconfiguration message to the second server, wherein the addressing reconfiguration message indicates an aggregator reallocated in the first network device for the second data processing task.

6. The method according to claim 5, characterized in that The second server includes a parameter server and a plurality of computing servers, the parameter server is used to store task parameters of the second data processing task, and each computing server is used to execute the second data processing task based on the task parameters; The sending of the task suspension message to the second server includes: Sending the task suspension message to the parameter server, so that the parameter server controls the multiple computing servers to suspend sending task messages after the target sequence number of the second data processing task; The receiving a pause response message sent by the second server includes: A pause response message is received from the parameter server, where the pause response message indicates that the plurality of computing servers have all paused sending task messages after the target sequence number of the second data processing task.

7. The method according to any one of claims 1 to 4, characterized in that: The step of determining the task deployment strategy and the aggregate resource allocation strategy according to the task running resources required by the first data processing task to be deployed, the aggregate resource utilization and bandwidth resource utilization of the plurality of network devices, and the processing resource utilization of the plurality of servers includes: Determining deployment constraints of the first data processing task based on the task running resources, the aggregate resource utilization and bandwidth resource utilization of the multiple network devices, and the processing resource utilization of the multiple servers; Based on the deployment constraints, the task deployment strategy and the aggregate resource allocation strategy are determined.

8. The method according to claim 7, characterized in that The deployment constraints include: The task running resources are less than or equal to the available processing resources of the server capable of executing the first data processing task; The processing resource usage of the server capable of executing the first data processing task is less than or equal to the processing resource threshold; The bandwidth resource between the server capable of executing the first data processing task and the network device capable of performing aggregation calculation on the processed data of the first data processing task is less than or equal to the bandwidth threshold; The memory resource occupancy of the network device capable of performing aggregation operations on the processing data of the first data processing task is less than or equal to the memory resource threshold.

9. A task deployment method, characterized in that: Applied to a task management system, the task management system includes a controller, a server cluster and multiple network devices, the server cluster includes multiple servers, and each of the network devices includes multiple aggregators obtained by dividing the memory resources of the network device; the method includes: The controller obtains the aggregate resource utilization of each of the network devices, the bandwidth resource utilization of each of the network devices, and the processing resource utilization of each of the servers; The controller determines a task deployment strategy and an aggregate resource allocation strategy according to task running resources required for a first data processing task to be deployed, aggregate resource utilization and bandwidth resource utilization of the multiple network devices, and processing resource utilization of the multiple servers, wherein the task deployment strategy indicates a first server among the multiple servers for executing the first data processing task, and the aggregate resource allocation strategy indicates an aggregator in a first network device among the multiple network devices for performing an aggregate operation on processing data of the first data processing task; The controller deploys the first data processing task in the first server based on the task deployment strategy and the aggregate resource allocation strategy; The first server stores task data of the first data processing task and resource information of aggregate resources allocated to the first data processing task.

10. The method according to claim 9, characterized in that The controller obtains the aggregate resource utilization of each of the network devices, including: The controller sends a measurement message to the network device, where the measurement message carries period information of a first measurement period, where the first measurement period is the measurement period in which the deployment time of the first data processing task is located; The network device receives the measurement message, determines the number of occupied aggregators of the network device in the first measurement period based on the period information of the first measurement period, and sends a measurement reply message to the controller, where the measurement reply message carries the number of occupied aggregators; The controller receives the measurement reply message and determines the aggregation resource utilization of the network device based on the number of occupied aggregators.

11. The method according to claim 10, characterized in that The period information of the first measurement period is a version number corresponding to the first measurement period, and a first state counter is provided in the network device, and the first state counter has a first version number; The network device determines, based on the period information of the first measurement period, the number of occupied aggregators of the network device in the first measurement period, including: In response to the version number corresponding to the first measurement period being the same as the first version number, the network device determines the number of occupied aggregators of the network device in the first measurement period based on the count value of the first state counter.

12. The method according to claim 11, characterized in that The first state counter also has first state information, and the network device is further provided with a second state counter, wherein the second state counter has second state information and a second version number; After the network device sends a measurement reply message to the controller, the method further includes: In response to the first state information indicating the current measurement cycle and the second state information indicating the previous measurement cycle, the network device switches the first state information and the second state information, updates the second version number to the version number corresponding to the second measurement cycle, and clears the count value of the second state counter to record the number of occupied aggregators in the second measurement cycle through the second state counter. The second measurement cycle is the next measurement cycle adjacent to the first measurement cycle.

13. The method according to any one of claims 9 to 12, characterized in that: The first data processing task refers to a data processing task that requires an exclusive aggregator; the method further includes: The controller determines a deployed second data processing task, where both the processed data of the second data processing task and the first data processing task need to be aggregated in the first network device, and the second data processing task refers to a data processing task that can share an aggregator with other data processing tasks; The controller sends a task suspension message to a second server, where the second server is a server among the multiple servers that executes the second data processing task; The second server receives the task suspension message, suspends sending the task message of the second data processing task after sending the task message of the target sequence number of the second data processing task, and sends a suspension response message to the controller; The controller receives the pause response message and sends an addressing reconfiguration message to the second server, where the addressing reconfiguration message indicates the aggregator reallocated for the second data processing task in the first network device.

14. The method according to claim 13, characterized in that The second server includes a parameter server and a plurality of computing servers, the parameter server is used to store task parameters of the second data processing task, and each computing server is used to execute the second data processing task based on the task parameters; The controller sends a task suspension message to the second server, including: The controller sends the task suspension message to the parameter server; The second server receives the task suspension message, suspends sending the task message of the second data processing task after sending the task message of the target sequence number of the second data processing task, and sends a suspension response message to the controller, including: The parameter server receives the task suspension message, determines the maximum message sequence number that has completed the aggregation operation at the current moment, and sends the maximum message sequence number to the multiple computing servers; The computing server determines the target sequence number based on the maximum message sequence number and the size of the message sending window, and suspends sending the task message of the second data processing task after sending the task message of the target sequence number of the second data processing task; After receiving the task message of the target sequence number of the second data processing task sent by the multiple computing servers, the parameter server sends the pause response message to the controller.

15. The method according to claim 14, characterized in that The task reply message with the maximum message sequence number sent by the parameter server to the computing server carries the maximum message sequence number and the pause identifier; the method further includes: If there is a target computing server among the multiple computing servers that has not received the task reply message with the maximum message sequence number, but has received a task reply message with a message sequence number next to the maximum message sequence number, the target computing server resends the task message with the maximum message sequence number to the parameter server; The parameter server resends the task reply message with the maximum message sequence number to the target computing server.

16. A task deployment device, characterized in that: The device is applied to a task management system, the task management system includes a server cluster and multiple network devices, the server cluster includes multiple servers, each of the network devices includes multiple aggregators obtained by dividing the memory resources of the network device; the device includes: A resource measurement module, used to obtain the aggregate resource utilization of each of the network devices, the bandwidth resource utilization of each of the network devices, and the processing resource utilization of each of the servers; a strategy determination module, configured to determine a task deployment strategy and an aggregate resource allocation strategy according to task running resources required for a first data processing task to be deployed, aggregate resource utilization and bandwidth resource utilization of the plurality of network devices, and processing resource utilization of the plurality of servers, wherein the task deployment strategy indicates a first server among the plurality of servers for executing the first data processing task, and the aggregate resource allocation strategy indicates an aggregator in a first network device among the plurality of network devices for performing an aggregate operation on processing data of the first data processing task; A task deployment module is used to deploy the first data processing task in the first server based on the task deployment strategy and the aggregate resource allocation strategy.

17. A task management system, characterized in that: The task management system includes a controller, a server cluster and a plurality of network devices, wherein the server cluster includes a plurality of servers, and each of the network devices includes a plurality of aggregators obtained by dividing the memory resources of the network device; The controller is used to obtain the aggregate resource utilization of each of the network devices, the bandwidth resource utilization of each of the network devices, and the processing resource utilization of each of the servers, and determine a task deployment strategy and an aggregate resource allocation strategy according to the task running resources required for the first data processing task to be deployed, the aggregate resource utilization and bandwidth resource utilization of the multiple network devices, and the processing resource utilization of the multiple servers, wherein the task deployment strategy indicates a first server among the multiple servers for executing the first data processing task, and the aggregate resource allocation strategy indicates an aggregator in a first network device among the multiple network devices for performing an aggregate operation on the processed data of the first data processing task, and deploys the first data processing task in the first server based on the task deployment strategy and the aggregate resource allocation strategy; The first server is used to store task data of the first data processing task and resource information of aggregate resources allocated to the first data processing task; The first network device is used to perform aggregation operations on the processed data of the first data processing task.

18. A controller, characterized in that: The controller includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 8 when calling and executing the computer program.

19. A computer-readable storage medium, characterized in that: The storage medium stores instructions, and when the instructions are executed on the computer, the computer executes the method according to any one of claims 1 to 8.

Images