Message processing method and device, electronic equipment, storage medium and computer program product

Abstract

The application provides a message processing method and device, electronic equipment, storage medium and computer program product, relates to the technical field of distributed training, and the method comprises the following steps: a plurality of first service nodes in an aggregated path respectively send first messages to obtain second messages; wherein the path is determined based on path information sent by a control node, the path information is obtained by the control node based on the service capabilities of each node in a network topology; the first service nodes and the first forwarding nodes in different paths are different; and the second message is sent to a second service node in the path. Through the scheme in the embodiment of the application, the efficiency of the second service node for model parameter updating can be improved.

Description

Technical Field

[0001] This application relates to the field of distributed training technology, and in particular to a message processing method, apparatus, electronic device, storage medium, and computer program product. Background Technology

[0002] In distributed machine learning, a parameter server (PS) is used to manage and share model parameters. The basic idea is to store model parameters on one or more central servers and share these parameters with worker servers via messages. Each worker server retrieves the current model parameters from the parameter server to calculate its local gradient and returns the calculated local gradient to the parameter server via a message for parameter updates. However, when multiple worker servers send local gradient messages to the parameter server, they face bandwidth bottlenecks, including the INCAST bandwidth bottleneck in the last-hop link and insufficient computing resources on the parameter server itself. This results in low message transmission efficiency from worker servers to the parameter server, or low processing efficiency of the parameter server for multiple local gradient messages. Consequently, the parameter server cannot quickly utilize local gradients for parameter updates, reducing the efficiency of model parameter updates. Summary of the Invention

[0003] The message processing method, apparatus, electronic device, storage medium, and computer program product provided in this application embodiment can improve the parameter update efficiency of the parameter server for the model.

[0004] The technical solution of this application is implemented as follows:

[0005] This application provides a message processing method, including:

[0006] The aggregation path consists of multiple first service nodes sending first messages to obtain a second message; wherein the path is determined based on path information sent by the control node, and the path information is obtained by the control node through path planning based on the service capabilities of each node in the network topology; the first service nodes and the first forwarding nodes are different in different paths;

[0007] The second message is sent to the second service node in the path.

[0008] In the above scheme, before the second message is obtained from the first message sent by the multiple first service nodes in the aggregation path, the method further includes:

[0009] Receive local bitmap information and local node identifier sent by the control node.

[0010] In the above scheme, the first message includes: a message header and a message payload;

[0011] The message header includes at least one of the following: communication address information, first target bitmap information of the target forwarding node, and multiple protocol fields; the message payload includes data to be aggregated.

[0012] In the above scheme, the first message sent by multiple first service nodes in the aggregation path results in a second message, including:

[0013] Based on the matching results of multiple first target bitmap information and local bitmap information, and the current computing resources, the aggregated detection results of multiple first messages are determined; wherein, each of the multiple first target bitmap information belongs to the corresponding first message;

[0014] If the aggregation detection result indicates that it supports the aggregation of multiple first messages, then the data to be aggregated in the payloads of multiple messages is aggregated and combined with the message header to obtain the second message.

[0015] In the above scheme, the data to be aggregated includes: N sub-data in a certain order; N is an integer greater than 0; the aggregation of the data to be aggregated in multiple message payloads and its combination with the message header to obtain a second message includes:

[0016] By aggregating the sub-data that correspond to the same order from multiple data to be aggregated, an aggregated message payload is obtained;

[0017] Modify the first protocol field in the message header to obtain the aggregated message header; wherein, the modified first protocol field is used to indicate that the corresponding first message has been aggregated.

[0018] The aggregated message payload and the aggregated message header are combined to obtain the second message.

[0019] In the above scheme, after determining the aggregated detection result of multiple first packets based on the matching results of multiple first target bitmap information and local bitmap information, and the current computing resources, the method further includes:

[0020] If the aggregation detection result indicates that aggregation of multiple first packets is not supported, then the second protocol field in the header of each first packet is modified to obtain the corresponding third packet; wherein, the modified second protocol field is used to indicate that the corresponding first forwarding node does not support aggregation.

[0021] Each of the third packets is sent to the second service node based on the next hop in the path information. The second service node then aggregates the multiple third packets and updates the model parameters based on the aggregated packets.

[0022] In the above scheme, after sending the second message to the second service node in the path, the method further includes:

[0023] The system receives a fourth message sent by the second service node and determines a message detection result for the fourth message; wherein the message detection result is used to characterize whether the fourth message is a one-to-many message sent to multiple first service nodes.

[0024] If the message detection result indicates that the fourth message is a one-to-many message sent to multiple first service nodes, then the fourth message is copied for each first service node to obtain a fifth message corresponding to each first service node.

[0025] The fifth message is sent to each of the corresponding first service nodes.

[0026] In the above scheme, determining the message detection result for the fourth message includes:

[0027] Based on the matching result of the second target bitmap information and the local bitmap information in the header of the fourth message, and the third protocol field in the header of the fourth message, the message detection result is determined; wherein, the third protocol field is used to characterize whether the fourth message is a one-to-many message sent to multiple first service nodes.

[0028] This application also provides a message processing method applied to a control node, including:

[0029] Path planning is performed based on the service capabilities of each node in the acquired network topology to determine path information; wherein, the path information is used to characterize the first service node, the first forwarding node, and the second service node included in each path; the first service node and the first forwarding node are different in different paths;

[0030] The path information is sent to each of the nodes, so that the first forwarding node in each path can aggregate the first messages sent by the multiple first service nodes respectively, and send the aggregated second message to the second service node.

[0031] In the above scheme, before determining the path information by performing path planning based on the service capabilities of each node in the acquired network topology, the method further includes:

[0032] A set of correspondences is formed by configuring bitmap information and node identifiers for each node.

[0033] The set of correspondences is sent to each service node in the network topology, and the bitmap information and node identifier corresponding to each forwarding node are sent to the corresponding forwarding node; wherein, the service nodes include: the first service node and the second service node; the forwarding nodes include the first forwarding node.

[0034] In the above scheme, the step of determining path information based on the service capabilities of each node in the acquired network topology includes:

[0035] Based on the service capabilities of each node obtained through processing using a preset model, initial path information between each first service node and its corresponding second service node is determined; wherein, the initial path information includes at least one of the following: port information and address information of each node in the path, and sending task information of each first service node;

[0036] Based on the service capability representation data of each forwarding node in the path represented by the initial path information, the first forwarding node with the strongest aggregation capability is determined among the forwarding nodes.

[0037] The initial path information and the bitmap information of the first forwarding node are combined to determine the path information.

[0038] In the above scheme, after determining the first forwarding node with the strongest aggregation capability among the forwarding nodes based on the service capabilities of each forwarding node in the path represented by the initial path information, the method further includes:

[0039] The service capabilities of each forwarding node in the path information are obtained at predetermined intervals.

[0040] Based on the service capabilities of each forwarding node obtained each time, the first forwarding node with the strongest aggregation capability is determined among the forwarding nodes.

[0041] New path information is formed based on the newly determined first forwarding node and sent to each of the nodes.

[0042] This application embodiment also provides a message processing apparatus, applied to a first forwarding node, including:

[0043] An aggregation unit is used to aggregate first messages sent by multiple first service nodes in a path to obtain a second message; wherein the path is determined based on path information sent by the control node, and the path information is obtained by the control node through path planning based on the service capabilities of each node in the network topology; the first service nodes and the first forwarding nodes are different in different paths;

[0044] The first sending unit is used to send the second message to the second service node in the path.

[0045] This application also provides a message processing apparatus for use in a control node, including:

[0046] The path planning unit is used to perform path planning based on the service capabilities of each node in the acquired network topology and determine path information; wherein, the path information is used to characterize the first service node, the first forwarding node and the second service node included in each path; the first service node and the first forwarding node are different in different paths;

[0047] The second sending unit is used to send the path information to each of the nodes, so that the first forwarding node in each path can aggregate the first messages sent by the multiple first service nodes respectively, and send the aggregated second message to the second service node.

[0048] This application also provides an electronic device, including a first memory and a first processor. The first memory stores a computer program that can run on the first processor. When the first processor executes the computer program, it implements the steps in the method on the first forwarding node side.

[0049] This application also provides an electronic device, including a second memory and a second processor. The second memory stores a computer program that can run on the second processor. When the second processor executes the computer program, it implements the steps in the method on the control node side.

[0050] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a first processor, implements the steps in the method on the first forwarding node side.

[0051] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a second processor, implements the steps in the method on the control node side.

[0052] This application also provides a computer program product, including a computer program that, when executed by a first processor, implements the steps in a method on the first forwarding node side.

[0053] This application also provides a computer program product, including a computer program that, when executed by a second processor, implements the steps of a method on the control node side.

[0054] In this embodiment, multiple first service nodes in the aggregation path send first messages to obtain a second message. The path is determined based on path information sent by the control node, which is obtained by the control node through path planning based on the service capabilities of each node in the network topology. Different paths have different first service nodes and first forwarding nodes. The second message is then sent to the second service node in the path. Thus, because multiple first messages are aggregated at the first forwarding node, a second message with a smaller data volume is obtained, reducing the total communication volume between the first and second service nodes. This reduces the computational resource consumption of the second service node when updating model parameters, thereby improving the efficiency of the second service node in updating model parameters.

[0055] Furthermore, since the path is planned based on the service capabilities of each node in the network topology, such path planning can make the most of the bandwidth capabilities of all nodes. In other words, it can utilize all nodes with forwarding functions in the network topology to upload the first packet, thus dispersing the bandwidth pressure and solving the bandwidth bottleneck problem.

[0056] Furthermore, the different first service nodes and first forwarding nodes in different paths, compared to the scheme of uploading packets through the default route in related technologies, can distribute the packets of each first service node in the network topology to different forwarding nodes, reduce the latency when processing packets through the first forwarding device in the default route, and quickly send packets to the second service node, thereby improving the efficiency of the second service node in updating model parameters. Attached Figure Description

[0057] Figure 1 This is an optional effect diagram of the related technology provided in the embodiments of this application;

[0058] Figure 2 This is an optional effect diagram of the related technology provided in the embodiments of this application;

[0059] Figure 3 An optional flowchart illustrating the message processing method provided in an embodiment of this application;

[0060] Figure 4 An optional flowchart illustrating the message processing method provided in an embodiment of this application;

[0061] Figure 5 An optional flowchart illustrating the message processing method provided in an embodiment of this application;

[0062] Figure 6 An optional flowchart illustrating the message processing method provided in an embodiment of this application;

[0063] Figure 7 An optional effect diagram of the message processing method provided in the embodiments of this application;

[0064] Figure 8 An optional flowchart illustrating the message processing method provided in an embodiment of this application;

[0065] Figure 9 An optional flowchart illustrating the message processing method provided in an embodiment of this application;

[0066] Figure 10 An optional flowchart illustrating the message processing method provided in an embodiment of this application;

[0067] Figure 11 An optional effect diagram of the message processing method provided in the embodiments of this application;

[0068] Figure 12 An optional effect diagram of the message processing method provided in the embodiments of this application;

[0069] Figure 13 An optional flowchart illustrating the message processing method provided in an embodiment of this application;

[0070] Figure 14 An optional flowchart illustrating the message processing method provided in an embodiment of this application;

[0071] Figure 15 An optional flowchart illustrating the message processing method provided in an embodiment of this application;

[0072] Figure 16 An optional flowchart illustrating the message processing method provided in an embodiment of this application;

[0073] Figure 17 An optional flowchart illustrating the message processing method provided in an embodiment of this application;

[0074] Figure 18 An optional flowchart illustrating the message processing method provided in an embodiment of this application;

[0075] Figure 19 An interactive schematic diagram of the message processing method provided in the embodiments of this application;

[0076] Figure 20 Schematic diagram of the message processing apparatus provided in the embodiments of this application Figure 1 ;

[0077] Figure 21 A hardware entity illustration of an electronic device provided in the embodiments of this application. Figure 1 ;

[0078] Figure 22 Schematic diagram of the message processing apparatus provided in the embodiments of this application Figure 2 ;

[0079] Figure 23 A hardware entity illustration of an electronic device provided in the embodiments of this application. Figure 2 . Detailed Implementation

[0080] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0081] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0082] If the application documents contain similar descriptions such as "first / second", the following explanation shall be added: In the following description, the terms "first / second / third" are used only to distinguish similar objects and do not represent a specific order of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0083] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0084] Among related technologies, existing distributed machine learning solutions have the following problems:

[0085] 1. Routing and Scheduling: Through pre-planning, the bandwidth imbalance between the working server and the parameter server is alleviated, effectively utilizing all available bandwidth. However, the total amount of message communication between the working server and the parameter server remains unchanged. The INCAST multi-to-one bandwidth bottleneck faced by the last-hop link of the parameter server still exists, and the computational resource consumption of the parameter server itself has not decreased.

[0086] Combination Figure 1In this context, the working server is represented by W, the switch by S, and the parameter server by ps. With an ingress bandwidth of 1Gbps from the working server to the switch, the aggregate bandwidth processed by PS is 9Gbps, and the bandwidth of each link (S1->S3, S2->S3, S1->S2) is 3Gbps.

[0087] Set the routing policy based on the path bandwidth constraints between the worker server and the parameter server:

[0088] W4->S1->S2->S3->PS (to prevent congestion on the S1->S3 path)

[0089] W1->S1->S3->PS

[0090] W2->S1->S3->PS

[0091] W5, W6->S2->S3->PS

[0092] W7, W8->S3->PS

[0093] If the amount of gradient data sent exceeds 1G, messages exceeding the aggregate bandwidth of PS will cause INCAST congestion from S3 to PS. Therefore, the transmission gradient bandwidth of W1-W6 is 1G.

[0094] 2. On-Network Aggregation: Without specifying a forwarding device, on-network computation is performed on the first forwarding device based on the default route. This prevents the use of multiple forwarding devices to collaboratively offload the same computation group and the same computational task, hindering aggregation operations. This will be combined with... Figure 2 Explanation:

[0095] S1 aggregates W1, W2, W3, and W4.

[0096] S2 aggregates W5, W6

[0097] S3 aggregates W7, W8

[0098] With an ingress bandwidth of 1Gbps from the working server to the switch, the aggregate bandwidth processed by the PS is 9Gbps, and the bandwidth of each link (S1->S3, S2->S3, S1->S2) is 3Gbps.

[0099] Even when the gradient processing bandwidth of S is 9G, gradients exceeding the bandwidth are still sent to PS for processing.

[0100] Therefore, the gradient transmission bandwidth of W1 to W4 is 9 / 4 = 2.25G.

[0101] The link bandwidth is 3, and the remaining bandwidth is 3 - 2.25 = 0.75G. From the bandwidth to be aggregated sent to PS, 0.75 / 4 = 0.18G. Therefore, the calculated transmission gradient bandwidth for W1-W4 is 2.53G.

[0102] The transmit gradient bandwidth of W5, W6, W7, and W8 is 3G. W1-W4 become the weakest points in the transmit gradient within the computing group.

[0103] This application provides a message processing method; please refer to [link / reference]. Figure 3 This is an optional flowchart illustrating a message processing method provided in an embodiment of this application, which will be combined with... Figure 3 The steps shown are explained below:

[0104] S101, the first messages sent by multiple first service nodes in the aggregation path are used to obtain the second message; wherein, the path is determined based on the path information sent by the control node, and the path information is obtained by the control node through path planning based on the service capabilities of each node in the network topology; the first service nodes and the first forwarding nodes in different paths are different.

[0105] In this embodiment, after receiving the path information sent by the control node, the first forwarding node determines its local node as the forwarding node for aggregating and forwarding the first packets based on the path information. After obtaining the first packets sent by multiple first service nodes in the corresponding path, the multiple first packets are aggregated to obtain the second packet.

[0106] The first message includes a message header and a message payload. The message header includes at least one of the following: communication address information, first target bitmap information of the target forwarding node, and multiple protocol fields. The message payload includes data to be aggregated. The first service node can be a working server, the first forwarding node can be a switch, and the second service node can be a parameter server.

[0107] In this process, the control node determines path information by acquiring the service capabilities of each node in the network topology (including at least one of the following: egress bandwidth, link bandwidth, aggregate bandwidth, and task to be computed) and performing path planning for each node. The first service node and the first forwarding node in each path are different. For example, the control node acquires the egress bandwidth, link bandwidth, and aggregate bandwidth of each node in the network topology, models these parameters, and calculates the optimal solution to determine the path, planning the local data computation task for the first service node. The control node then distributes the path information to the first service node, the first forwarding node, and the second service node in each path.

[0108] In this embodiment of the application, after the first forwarding node receives multiple first messages, it can aggregate the data to be aggregated (gradient data) in each first message in a certain order to obtain a second message.

[0109] In this embodiment of the application, if after receiving multiple first messages, the first forwarding node detects that the local device does not support the aggregation of multiple first messages, then it forwards the multiple first messages to the next-hop node according to the next hop in the path.

[0110] S102, Send the second message to the second service node in the path.

[0111] In this embodiment, after the first forwarding node obtains the corresponding path information, it can obtain the port and address information of the second service node in the corresponding path. Based on the port and address information of the second service node, the second packet is sent to the second service node. After receiving the second packet, the second service node can update the model using the gradient parameters in the second packet to obtain a new model.

[0112] Because multiple first packets are aggregated in the first forwarding node to obtain a second packet with a smaller data volume, the total communication volume between the first and second service nodes is reduced, thereby reducing the computational resource consumption of the second service node when updating model parameters and improving the efficiency of the second service node in updating model parameters.

[0113] Furthermore, since the path is planned based on the service capabilities of each node in the network topology, such path planning can make the most of the bandwidth capabilities of all nodes. In other words, it can utilize all nodes with forwarding functions in the network topology to upload the first packet, thus dispersing the bandwidth pressure and solving the bandwidth bottleneck problem.

[0114] Furthermore, the different first service nodes and first forwarding nodes in different paths, compared to the scheme of uploading packets through the default route in related technologies, can distribute the packets of each first service node in the network topology to different forwarding nodes, reduce the latency when processing packets through the first forwarding device in the default route, and quickly send packets to the second service node, thereby improving the efficiency of the second service node in updating model parameters.

[0115] Please see Figure 4 This is a schematic diagram of an optional message processing method provided in an embodiment of this application. Figure 3 S101 to S102 can also be achieved through steps 1 to 6. The steps are explained below:

[0116] 1. Calculation task distribution.

[0117] In this embodiment, the control node obtains the outbound bandwidth (related to server computing power), link bandwidth, and aggregate bandwidth of on-network computing forwarding devices of each node, and models the above parameters to find the optimal solution, and plans the local data computing tasks of the working server.

[0118] 2. Routing and scheduling.

[0119] In this embodiment, the control node obtains the outbound bandwidth (related to server computing power), link bandwidth, and aggregate bandwidth of on-network computing forwarding devices of each node, models the above parameters to find the optimal solution to determine the path, obtains path information, and sends the path information to each node to realize path scheduling of each node in the network topology.

[0120] 3. Calculate the gradient and send it to the forwarding device.

[0121] In this embodiment of the application, the first service node 101 performs gradient calculation based on the parameters issued by the second service node 103 to determine the first message, and sends the first message to the first forwarding node 102.

[0122] 4. Link bandwidth statistics.

[0123] In this embodiment, the first forwarding node 102 performs device link bandwidth statistics for nodes on the path.

[0124] 5. Gradient aggregation.

[0125] In this embodiment of the application, after receiving multiple first messages, the first forwarding node 102 aggregates the gradient data included in the multiple first messages to obtain a second message.

[0126] In this embodiment, the communication protocol between the first service node 101 and the second service node 103, and the local identifier of the first forwarding node 102, support the selection of the first forwarding node 102 for parameter aggregation calculation of many-to-one messages. The first forwarding node 102 completes device link bandwidth statistics and periodically synchronizes with the control node. The control node calculates the scheduling strategy for new tasks based on the latest link status. When the first forwarding node 102 cannot complete the gradient aggregation calculation task, a fallback mechanism is designed, and the second service node 103 implements gradient aggregation and parameter updates.

[0127] 6. Gradient aggregation / update.

[0128] In this embodiment, the second service node 103 completes gradient set update and task orchestration update based on the second message (aggregated gradients or unaggregated local gradients) from the first forwarding node 102. The second service node 103 sends the messages that need to be sent to the first service node 101 to the first forwarding node 102 that needs to participate in the on-network computation in a one-to-one manner, and the first forwarding node 102 performs one-to-many message copying and sending.

[0129] Please see Figure 5 This is a schematic diagram of an optional message processing method provided in an embodiment of this application. Figure 3 S101 can also include S201, which will explain the steps:

[0130] S201, Receive local bitmap information and local node identifier sent by the control node.

[0131] In this embodiment, the control node can configure bitmap information and node identifier for each node, and establish a set of correspondences between the bitmap information and the corresponding node identifier for each node. This set of correspondences is sent to each service node in the network topology, and the bitmap information and node identifier corresponding to each forwarding node are sent to the corresponding forwarding node. The first forwarding node receives the local bitmap information and local node identifier corresponding to the first forwarding node sent by the control node.

[0132] The local bitmap information can be a single bit in the bitmap table. The local node identifier can be a string corresponding to the first forwarding node.

[0133] In this embodiment, the local bitmap information and local node identifier sent by the control node are received. Since the first message includes the first target bitmap information, the aggregation of multiple first messages can be achieved by matching the local bitmap information with the first target bitmap information in the interaction messages between the first service node and the second service node. This allows the node that aggregates the messages to be explicitly specified, making the interaction message scheme between the first service node and the second service node adaptable to more complex network topologies and more scalable.

[0134] Please see Figure 6 This is a schematic diagram of an optional message processing method provided in an embodiment of this application. Figure 3 S101 in the above can also be implemented through S301 to S302. The steps are explained below:

[0135] S301. Based on the matching results of multiple first target bitmap information and local bitmap information, and the current computing resources, determine the aggregated detection results of multiple first messages.

[0136] In this embodiment, the first forwarding node matches the first target bitmap information in each first packet with its local bitmap information to obtain a matching result. Based on the matching result and current computing resources, an aggregated detection result for multiple first packets is determined. The aggregated detection result characterizes whether the first forwarding node supports the aggregation of multiple first packets. Each of the multiple first target bitmap information belongs to the corresponding first packet.

[0137] For example, in combination Figure 7 Based on the Internet Protocol (IP) protocol stack, the first message designed includes a message header and a message payload. The message header enables the first forwarding node to participate in distributed machine learning computation tasks in a transparent manner, and specifies the on-network computing forwarding devices participating in aggregation computation through a bitmap. The first forwarding node can match the first target bitmap information with the local bitmap information to obtain the matching result.

[0138] The message header contains: the first bit information, used to identify the device requiring parameter aggregation, is represented by a bitmap set to 1; multiple protocol fields include: gID, jID, tID, FIN, RES, and DI. Specifically: gID: groupID (computation group ID); jID: jobID (computation job ID); tID: taskID (task ID); FIN: finish (0: computation not finished; 1: computation finished); RES: Resend (1: computation completed by the PS when the switch has no resources); DI: Direction (0: message sent from worker to PS; 1: message sent from PS to worker).

[0139] In the message payload section: gi1, gi2, ..., gin are the calculation data to be aggregated. When Di = 0; W1, W2, ..., Wn in the message payload section are the parameter data to be distributed. When Di = 1.

[0140] S302. If the aggregation detection result indicates that it supports the aggregation of multiple first messages, then the data to be aggregated in the payloads of multiple messages is aggregated and combined with the message header to obtain the second message.

[0141] In this embodiment, if the first target bitmap information in multiple first packets matches the local bitmap information, and the current computing resources meet predetermined conditions, then it is determined that the first forwarding node supports the aggregation detection results of multiple first packets. Then, the first forwarding node aggregates the data to be aggregated in the payloads of the multiple packets and combines it with the packet header to obtain the second packet.

[0142] In this embodiment of the application, the first forwarding node can combine multiple data to be aggregated in a certain order, and then combine them with the header of any first message to obtain a second message.

[0143] For example, in combination Figure 7 Each first message includes a message payload section, which aggregates multiple computational data to be aggregated (gi1, gi2, ..., gin) and combines them with the message header of any message to obtain a second message.

[0144] In this embodiment, based on the matching results of the first target bitmap information and local bitmap information in multiple first messages, and the current computing resources, the aggregation detection result of multiple first messages is determined. If the aggregation detection result indicates that aggregation of multiple first messages is supported, the data to be aggregated in the payloads of multiple messages is aggregated and combined with the message header to obtain the second message. In this way, multiple first messages with large data volumes can be transformed into second messages with smaller data volumes, thereby reducing the total communication volume between the first and second service nodes, solving the INCAST multi-call bandwidth bottleneck problem faced by the last-hop link of the second service node, and also reducing the computing resource consumption of the second service node when processing received messages.

[0145] Please see Figure 8 This is a schematic diagram of an optional message processing method provided in an embodiment of this application. Figure 6 S302 in the above can also be implemented through S303 to S304. The steps are explained below:

[0146] S303. If the aggregation detection result indicates that it does not support the aggregation of multiple first messages, then modify the second protocol field in the message header of each first message to obtain the corresponding third message.

[0147] In this embodiment, if the first target bitmap information in multiple first packets matches the local bitmap information, and the current computing resources do not meet the predetermined conditions, then it is determined that the aggregation detection result of multiple first packets is not supported. The first forwarding node modifies the second protocol field in the packet header of each first packet to obtain a third packet corresponding to each first packet. The modified second protocol field is used to indicate that the corresponding first forwarding node does not support the aggregation of first packets.

[0148] For example, in combination Figure 7 The first forwarding node can modify the RES field in the header of the first message to 1 to obtain the third message, which is used to indicate that the first forwarding node does not support the aggregation of the first message.

[0149] S304. Based on the next hop in the path information, each of the third messages is sent to the second service node, so that the second service node can aggregate the multiple third messages and update the model parameters based on the aggregated messages.

[0150] In this embodiment, the first forwarding node sends the third packet to the next-hop indicating node based on the port and address information of the next hop represented in the path information, and then sends the third packet to the second serving node through the next-hop indicating node. The second serving node aggregates multiple third packets and updates the model parameters based on the aggregated packets.

[0151] In this embodiment, if the aggregated detection result does not support the aggregation of multiple first packets, the second protocol field in the header of each first packet is modified to obtain the corresponding third packet. Each third packet is sent to the second service node based on the next hop in the path information. The second service node then aggregates the multiple third packets and updates the model parameters based on the aggregated packets. In this way, even if the first service node receives packets aggregated on its local device but lacks sufficient computing resources, it can still forward multiple first packets without affecting the packet interaction between the first and second service nodes in the path.

[0152] Please see Figure 9 This is a schematic diagram of an optional message processing method provided in an embodiment of this application. Figure 6 S302 in the above can also be implemented through S401 to S403. The steps are explained below:

[0153] S401. Aggregate the sub-data corresponding to the same order from multiple data to be aggregated to obtain the aggregated message payload.

[0154] In this embodiment of the application, the data to be aggregated in each first message includes N sub-data in a certain order; N is an integer greater than 0. The first forwarding node can aggregate the first sub-data in the multiple data to be aggregated to obtain the first aggregated sub-data, and then aggregate them in order to finally obtain N aggregated sub-data. The N aggregated sub-data are combined in order to obtain the aggregated message payload.

[0155] For example, the data to be aggregated in the first message 1 includes N sub-data: g11, g12, ..., g1n; the data to be aggregated in the first message 1 includes N sub-data: g21, g22, ..., g2n; the data to be aggregated in the first message N includes N sub-data: gN1, gN2, ..., gNn. The first forwarding node can aggregate g11, g21 to gN1 to obtain the first aggregated sub-data, and continue to aggregate g1n, g2n to gNn to obtain N aggregated sub-data.

[0156] S402. Modify the first protocol field in the message header to obtain the aggregated message header; wherein, the modified first protocol field is used to indicate that the corresponding first message has been aggregated.

[0157] In this embodiment, the first forwarding node can modify the first protocol field in the header of any first message to obtain an aggregated message header. The modified first protocol field indicates that the corresponding first message has undergone aggregation processing.

[0158] For example, in combination Figure 7 The first forwarding node can modify the FIN field in the header of the first packet to 1 to obtain the aggregated packet header, which is used to indicate that the corresponding first packet has been aggregated.

[0159] S403. Combine the aggregated message payload and the aggregated message header to obtain the second message.

[0160] In this embodiment, the first forwarding node combines the aggregated packet payload and the aggregated packet header to obtain a second packet. The determined second packet is then sent to the second serving node via the next hop.

[0161] In this embodiment, sub-data corresponding to the same order from multiple data to be aggregated are aggregated to obtain an aggregated message payload. The first protocol field in the message header is modified to obtain the aggregated message header; the modified first protocol field indicates that the corresponding first message has undergone aggregation processing. The aggregated message payload and the aggregated message header are combined to obtain a second message. Thus, by aggregating N sub-data from multiple first messages in order, not only is the data volume of the multiple first messages reduced, but this method of aggregating N sub-data in order does not disrupt the order of gradients in the data to be aggregated, resulting in a second message with smaller data volume and no data distortion. This further reduces the total communication volume between the first and second service nodes, thereby reducing the computational resource consumption of the second service node when updating model parameters and improving the efficiency of the second service node in updating model parameters.

[0162] Please see Figure 10 This is a schematic diagram of an optional message processing method provided in an embodiment of this application. Figure 3 S102 can be followed by S501 to S503, and the steps will be explained below:

[0163] S501. Receive the fourth message sent by the second service node, and determine the message detection result for the fourth message; wherein, the message detection result is used to characterize whether the fourth message is a one-to-many message sent to multiple first service nodes.

[0164] In this embodiment of the application, the first forwarding node receives the fourth message sent by the second service node and determines the message detection result for the fourth message; wherein, the message detection result is used to characterize whether the fourth message is a one-to-many message sent to multiple first service nodes.

[0165] In this embodiment, the message structure of the fourth message is the same as that of the first message. The fourth message includes a message header and a message payload. The message header has multiple protocol fields, communication address information, and second target bitmap information. The message payload includes data to be distributed. The first forwarding node determines the message detection result based on the matching result between the second target bitmap information in the message header of the fourth message and the local bitmap information, and the third protocol field in the message header of the fourth message; wherein, the third protocol field is used to characterize whether the fourth message is a one-to-many message sent to multiple first service nodes.

[0166] For example, the first forwarding node can determine that the fourth message is a one-to-many message sent to multiple first service nodes based on the fact that the DI=1 field in multiple protocol fields in the fourth message and the matching of the second destination location information with the local bitmap information.

[0167] S502. If the message detection result indicates that the fourth message is a one-to-many message sent to multiple first service nodes, then the fourth message is copied for each first service node to obtain a fifth message corresponding to each first service node.

[0168] In this embodiment of the application, if the message detection result indicates that the fourth message is a one-to-many message sent to multiple first service nodes, then the fourth message is copied for each first service node, and the destination address information in the message copied for each first service node is modified to the first information of the corresponding first service node, thereby obtaining the fifth message corresponding to each first service node.

[0169] For example, the second service node sends a fourth packet W0 (destination IP: 1.1.1.S, ***) to the first forwarding node (IP: 1.1.1.S). The first forwarding node determines, based on the bitmap, that it needs to be copied and distributed within its own device. After copying the fourth packet, it sends parameter W to each of the first service nodes. The first forwarding node modifies the destination IP in the fourth packet W0 using the IP address of the first service node 1, obtaining a fifth packet (destination IP: 1.1.1.1, ***) destined for the first service node 1. This scheme can be used to obtain the fifth packet corresponding to each first service node.

[0170] S503, The fifth message is sent to each of the corresponding first service nodes.

[0171] In this embodiment of the application, the first forwarding node sends each fifth message to the corresponding first service node.

[0172] In this embodiment, a fourth message sent by a second service node is received, and a message detection result is determined for the fourth message. If the message detection result indicates that the fourth message is a one-to-many message sent to multiple first service nodes, the fourth message is copied for each first service node to obtain a fifth message corresponding to each first service node. The fifth message is then sent to each corresponding first service node. In this way, by forwarding the second message obtained by aggregating multiple first messages through the first forwarding node, and forwarding the received fourth message in a one-to-many manner, the first forwarding node can simultaneously offload and share the many-to-one aggregation calculation and the one-to-many multicast calculation, thereby reducing the computational resource consumption of the forwarding path and computing device.

[0173] The following embodiments will describe the solution in this application in detail:

[0174] Combination Figure 11There are N worker servers (first service nodes) and one parameter server (second service node), interconnected through forwarding devices X and Y (forwarding nodes). Worker servers 1 (IP: 1.1.1.1) to N (IP: 1.1.1.N) obtain model parameters W0 from the parameter server (IP: 1.1.1.P). The bitmap corresponding to forwarding device X (first forwarding node) is 00000001, and the bitmap corresponding to forwarding device Y is 01000000.

[0175] Worker servers 1 through N each send the calculated local gradient gi (first message) to the parameter server.

[0176] The message sent by worker server 1 to parameter server:

[0177] Destination IP: 1.1.1.P; Source IP: 1.1.1.1;

[0178] 01000000;gId,jId,tId;FIN:0;RS:0;DI:0,g11,g12,...,g1n

[0179] The message sent by worker server N to parameter server:

[0180] Destination IP: 1.1.1.P; Source IP: 1.1.1.N;

[0181] 01000000;gId,jId,tId;FIN:0;RS:0;DI:0,gN1,gN2,...,gNn

[0182] The messages sent from worker server 2 to worker server N-1 to the parameter server are similar.

[0183] If forwarding device X determines that the packet does not need to be aggregated on this device based on the bitmap, it will forward it to the next-hop forwarding device Y.

[0184] Forwarding device Y performs gradient aggregation on the multiple received packet payloads and sends the aggregation parameters to the parameter server.

[0185] Message sent to the parameter server:

[0186] Destination IP: 1.1.1.P; Source IP: 1.1.1.S; 01000000;

[0187] gId, jId, tId; FIN: 1; RS: 0, DI: 0, gr1, gr2,..., grn.

[0188] The parameter server sends model parameters W0 (destination IP: 1.1.1.S, source IP: 1.1.1.P, 01000000, gId, jId, tId, FIN:0, RS:0, DI:1, w1, w2, ..., wn) to the forwarding device Y (IP: 1.1.1.S).

[0189] Forwarding device Y determines that the message is a one-to-many message sent from the parameter server to the worker server based on DI=1. Based on the bit map, it determines that it needs to be copied and distributed on this device. After copying the original message, it sends model parameter W0 to worker server 1 to worker server N respectively.

[0190] If forwarding device X determines that it does not need to copy and distribute the packet on its own device based on the bitmap, it will forward the packet directly.

[0191] Message sent to worker server 1:

[0192] Destination IP: 1.1.1.1, Source IP, 1.1.1.P, 01000000, gId, jId, tId, FIN: 0, RS: 0, DI: 1, w1, w2, ..., wn

[0193] N messages destined for the working server:

[0194] Destination IP: 1.1.1.N, Source IP: 1.1.1.P, 01000000, gId, jId, tId, FIN: 0, RS: 0, DI: 1, w1, w2, ..., wn

[0195] The messages sent to worker servers 2 through N-1 are similar.

[0196] Combination Figure 12 There are N worker servers (first service nodes) and one parameter server (second service node), interconnected through forwarding devices X and Y (forwarding nodes). Worker servers 1 (IP: 1.1.1.1) to N (IP: 1.1.1.N) obtain model parameters W0 from the parameter server (IP: 1.1.1.P). The bitmap corresponding to forwarding device X is 00000001, and the bitmap corresponding to forwarding device Y is 01000000.

[0197] Worker servers 1 (IP: 1.1.1.1) to N (IP: 1.1.1.N) obtain model parameters W0 from the parameter server (IP: 1.1.1.P). Forwarding device X has a bitmap of 00000001, and forwarding device Y has a bitmap of 01000000. Worker servers 1 to N each send their calculated local gradient gi to the parameter server.

[0198] The message sent by worker server 1 to parameter server:

[0199] Destination IP: 1.1.1.P; Source IP: 1.1.1.1;

[0200] 01000000;gId,jId,tId;FIN:0;RS:0;DI:0,g11,g12,...,g1n

[0201] The message sent by worker server N to parameter server:

[0202] Destination IP: 1.1.1.P; Source IP: 1.1.1.N;

[0203] 01000000;gId,jId,tId;FIN:0;RS:0;DI:0,gN1,gN2,...,gNn

[0204] The messages sent from worker server 2 to worker server N-1 to the parameter server are similar.

[0205] Forwarding device X determines that the packet does not need to be aggregated on this device based on the bitmap, and then forwards it to the next-hop forwarding device Y. Forwarding device Y checks and finds that it does not have aggregation resources and cannot support aggregation calculation for the current gradient data. Therefore, it sets the RES of the packet in step 4 to 1, indicating that this packet is a data packet to be aggregated and retransmitted to the parameter server.

[0206] Message 1 sent to the parameter server:

[0207] Destination IP: 1.1.1.P; Source IP: 1.1.1.S; 01000000;

[0208] gId, jId, tId; FIN: 1; RS: 1, DI: 0, gr1, gr2,..., grn.

[0209] Message N sent to the parameter server:

[0210] Destination IP: 1.1.1.P; Source IP: 1.1.1.N;

[0211] gId,jId,tId;FIN:0;RS:1;DI:0,gN1,gN2,...,gNn

[0212] Messages 2 through N-1 are similar.

[0213] Once the parameter server receives a gradient data message with RS set to 1, it performs parameter aggregation calculations on the server and then sends the updated parameter data back to all worker servers. The specific process is the same as that for W0.

[0214] The parameter server sends model parameters W0 (destination IP: 1.1.1.S, source IP: 1.1.1.P, 01000000, gId, jId, tId, FIN:1, RS:1, DI:1, w1, w2, ..., wn) to the forwarding device (IP: 1.1.1.S).

[0215] Forwarding device Y determines that the message is a one-to-many message sent from the parameter server to the worker server based on DI=1. Based on the bit map, it determines that it needs to be copied and distributed on this device. Therefore, after copying the original message, it sends parameter W to worker server 1 to worker server N respectively.

[0216] If forwarding device X determines that it does not need to copy and distribute the packet on its own device based on the bitmap, it will forward the packet directly.

[0217] Forwarding device Y sends parameter W0 to working servers 1 through N respectively.

[0218] Message sent to worker server 1:

[0219] Destination IP: 1.1.1.1, Source IP, 1.1.1.P, 01000000, gId, jId, tId, FIN: 0, RS: 0, DI: 0, w1, w2, ..., wn

[0220] N messages destined for the working server:

[0221] Destination IP: 1.1.1.N, Source IP: 1.1.1.P, 01000000, gId, jId, tId, FIN: 0, RS: 0, DI: 0, w1, w2, ..., wn

[0222] The messages sent to worker servers 2 through N-1 are similar.

[0223] Please see Figure 13 The following is an optional flowchart illustrating the message processing method provided in the embodiments of this application, and the steps are explained as follows:

[0224] S11, Input port reception.

[0225] In this embodiment of the application, the forwarding packet ingress port of the first forwarding node receives the first packet, which is the same as the function of a normal forwarding device.

[0226] S12, Message parsing.

[0227] In this embodiment of the application, the first forwarding node extracts the Differentiated Services Code Point (DSCP) flag of the packet.

[0228] S13. Whether the message is to be aggregated.

[0229] In this embodiment, the first forwarding node determines whether the packet is to be aggregated based on DSCP. If it is, S17 is executed; otherwise, S14 is executed.

[0230] S14, Forward table lookup.

[0231] In this embodiment, the first forwarding node determines the next-hop destination and outgoing port of the packet based on the destination IP of the packet, which is the same as the function of a normal forwarding device.

[0232] S15, Reverse parsing of messages.

[0233] In this embodiment, the first forwarding node completes the corresponding packet filling and encapsulation operation according to the next-hop destination and the outgoing port, just like a normal forwarding device.

[0234] S16, Send from the output port.

[0235] In this embodiment, the first forwarding node forwards the packet outgoing port packet sending process, which is the same as the function of a normal forwarding device.

[0236] S17. Whether to aggregate on this device.

[0237] In this embodiment of the application, if it is a message to be aggregated, the bitmap is extracted, and it is determined whether it is aggregated on this device. If yes, S18 is executed; otherwise, S14 is executed.

[0238] S18. Find aggregated resources.

[0239] In this embodiment of the application, the first forwarding node searches for the aggregated resource based on the information (gID, jID, tID, defined in "System Design - Protocol Design") in the data packet to be aggregated.

[0240] S19. Does it support aggregation?

[0241] In this embodiment, if the first forwarding node does not find a corresponding aggregation resource, it assumes that the current device does not have the capability to perform calculations on this aggregation data packet. Therefore, it modifies the Res protocol field of the packet and sends it to the destination server for aggregation calculation. If a corresponding aggregation resource is found, the data aggregation operation is performed.

[0242] S20, Perform gradient aggregation calculation.

[0243] In this embodiment, the first forwarding node performs aggregation calculations on the received data packets to be aggregated and the current aggregated data, and determines whether the aggregation calculation for this round has been completed.

[0244] S21. Modify the RS flag of the message.

[0245] In this embodiment of the application, after the first forwarding node completes the aggregation and the aggregation data for this round has been collected, the DSCP of the packet is modified to make it a normal forwarding packet, and it is sent back to the ingress port packet parsing process for normal forwarding.

[0246] S22. Has the aggregation been completed?

[0247] In this embodiment, the first forwarding node determines whether to complete the aggregation of multiple packets based on the path information sent by the control node. The path information includes the packets that the first forwarding node needs to aggregate and the source node information of the packets. If the aggregation is complete, step S23 is executed; otherwise, step S24 is executed.

[0248] S23. Modify the FIN flag of the message.

[0249] In this embodiment, the first forwarding node modifies the FIN field in the aggregated message.

[0250] S24, Message discarded.

[0251] In this embodiment, if the aggregated data for this message is not fully collected in the current round after data aggregation, the message is discarded and the forwarding process ends. The above modules work together to complete the forwarding and calculation functions of the forwarding device's message processing pipeline.

[0252] Please see Figure 14 The following is an optional flowchart illustrating the message processing method provided in the embodiments of this application, and the steps are explained as follows:

[0253] S701. Based on the service capabilities of each node in the acquired network topology, perform path planning to determine path information; wherein, the path information is used to characterize the first service node, the first forwarding node, and the second service node included in each path; the first service node and the first forwarding node are different in different paths.

[0254] In this embodiment, the control node acquires the service capabilities of each node in the network topology. Based on the service capabilities of each node, path planning is performed to determine paths that can transmit packets through different forwarding nodes, and the path information of each path is sent to each node. The path information may include port information, IP address information, etc., corresponding to the first service node, second forwarding node, and second service node in each path. The first service node and first forwarding node are different in different paths.

[0255] The service capabilities of each node may include at least one of the following: egress bandwidth, link bandwidth, aggregate bandwidth, and tasks to be computed.

[0256] S702. The path information is sent to each of the nodes, so that the first forwarding node in each path can aggregate the first messages sent by the multiple first service nodes respectively, and send the aggregated second message to the second service node.

[0257] In this embodiment, the first forwarding node sends path information to each node. After obtaining the path information, each node can determine its role in the corresponding path and obtain port and IP address information corresponding to at least one of the first service node, the first forwarding node, and the second service node in the same path. The first forwarding node in the same path can send a first packet to itself. The first forwarding node aggregates multiple first packets to obtain a second packet and then sends the aggregated second packet to the second service node.

[0258] Path planning is performed based on the service capabilities of each node in the acquired network topology to determine path information. This path information characterizes the first service node, first forwarding node, and second service node included in each path; the first service node and first forwarding node differ in different paths. The path information is sent to each node, allowing the first forwarding node in each path to aggregate multiple first packets sent by the first service nodes and send the aggregated second packet to the second service node. This aggregation of multiple first packets at the first forwarding node results in a smaller second packet, reducing the total communication volume between the first and second service nodes. This, in turn, reduces the computational resource consumption of the second service node when updating model parameters, improving the efficiency of model parameter updates.

[0259] Furthermore, since the path is planned based on the service capabilities of each node in the network topology, such path planning can make the most of the bandwidth capabilities of all nodes. In other words, it can utilize all nodes with forwarding functions in the network topology to upload the first packet, thus dispersing the bandwidth pressure and solving the bandwidth bottleneck problem.

[0260] Furthermore, the different first service nodes and first forwarding nodes in different paths, compared to the scheme of uploading packets through the default route in related technologies, can distribute the packets of each first service node in the network topology to different forwarding nodes, reduce the latency when processing packets through the first forwarding device in the default route, and quickly send packets to the second service node, thereby improving the efficiency of the second service node in updating model parameters.

[0261] Please see Figure 15 This is a schematic diagram of an optional message processing method provided in an embodiment of this application. Figure 14 The steps may also include S801 to S802 before S701, which will be explained in detail below:

[0262] S801. Configure the correspondence between bitmap information and node identifier for each node to form a correspondence set.

[0263] In this embodiment of the application, after the control node obtains the service capabilities of each node in the network topology, it can configure the correspondence between bitmap information and node identifier for each node, and combine the correspondence of each node to obtain a correspondence set.

[0264] In this embodiment of the application, the control node can obtain the address information of all nodes (forwarding devices and servers) in the computing network topology through the network management protocol, assign a unique identifier to each node, and establish a set of correspondences between a 64-bit bitmap table and the unique identifiers of forwarding devices, wherein each bit in the bitmap table corresponds to a forwarding device.

[0265] S802. Send the set of correspondences to each service node in the network topology, and send the bitmap information and node identifier corresponding to each forwarding node to the corresponding forwarding node.

[0266] In this embodiment, the control node sends the set of correspondences to each service node (the first service node and the second service node), and sends the bitmap information and node identifier of the forwarding node to the corresponding forwarding node.

[0267] In this embodiment, the control node simultaneously distributes the full bitmap table (correspondence set) to all computing servers (including the first service node and the second service node) on the network computing control plane. The control node also distributes its unique identifier and corresponding bit bits from the bitmap to each forwarding device (forwarding node). The control node collaboratively completes the distribution and routing of computing tasks through two modules: "computing task distribution" and "routing scheduling strategy."

[0268] In this embodiment, a correspondence between bitmap information and node identifiers is configured for each node, forming a correspondence set. This correspondence set is sent to each service node in the network topology, and the bitmap information and node identifier corresponding to each forwarding node are sent to the corresponding forwarding node. In this way, the first service node can also configure the first packet based on the bitmap information of the determined forwarding node. After receiving the first packet, the forwarding node can determine whether aggregation is needed at this node based on the matching result between its local bitmap information and the bitmap information in the first packet. This bitmap information-based matching and aggregation scheme adds a forwarding device that participates in the calculation of bitmap information in the interaction packets between the first and second service nodes, explicitly specifying the device that performs the calculation on the packet. This makes the on-network computing scheme adaptable to more complex network topologies and more scalable.

[0269] For example, in combination Figure 16 The working logic of the control node's "computation task distribution" and "routing scheduling strategy" can be implemented through steps S25 to S32, which will be explained in conjunction with the steps:

[0270] S25. Calculate the task, obtain the server list and gradient transmission bandwidth, and parameter server list.

[0271] In this embodiment of the application, the control node obtains the server IP list and gradient transmission bandwidth, and the parameter is the server IP list.

[0272] S26. Obtain the topology from the computing server to the parameter server, and the bitmap of the switch numbers within the topology.

[0273] In this embodiment, the control node obtains the service capabilities from the computing server to the parameter server and the service capabilities of the forwarding device in the network topology, and configures the correspondence between the node identifier and the corresponding bitmap information for each node.

[0274] S27, Switch bitmap, the switch's uplink and downlink IP addresses.

[0275] In this embodiment of the application, the control node sends the "computing task distribution" based on the bitmap information of the switch and the IP address information of the uplink and downlink nodes of the switch.

[0276] S28. Obtain the aggregation capability of the switch.

[0277] In this embodiment of the application, the "computation task distribution" module in the control node obtains the aggregate bandwidth of each switch in the network topology.

[0278] S29, switch bitmap, aggregated bandwidth.

[0279] In this embodiment of the application, the "computation task distribution" module in the control node transmits the bitmap information of the switch and the corresponding aggregated bandwidth to the corresponding "routing scheduling" module.

[0280] S30. Task planning and modeling: Solve for the next hop from the working server to the parameter server and the aggregation switch bitmap.

[0281] In this embodiment of the application, the "routing scheduling" module in the control node performs task planning modeling based on the acquired switch bitmap information and aggregated bandwidth, and determines the next hop of the message sent by the working server to the parameter server and the bitmap information of the switch in the path.

[0282] S31, working server, switch bitmap.

[0283] S32. Send gradient calculation tasks, parameter server IP, bitmap, and next hop to parameter server to the working server; send aggregation tasks to the switches in the bitmap; send aggregation tasks to the parameter server.

[0284] In this embodiment, the working server uses the communication protocol in the path information of this application to send local gradients to the parameter server, supporting the selection of on-network computing forwarding devices for parameter aggregation. The forwarding devices complete device link bandwidth statistics and periodically synchronize with the on-network computing control plane. The control node calculates the scheduling strategy for new tasks based on the latest link status. The forwarding devices aggregate the received local gradients and send the aggregated parameters to the parameter server. This application includes a fallback mechanism: when a forwarding device cannot complete gradient aggregation, it carries an uncalculated flag, and the parameter server performs gradient aggregation and parameter updates. The parameter server receives the gradient aggregation result from the forwarding device and completes the parameter update; when the parameter server receives an unaggregated gradient, it performs gradient aggregation and parameter updates.

[0285] Please see Figure 17 This is a schematic diagram of an optional message processing method provided in an embodiment of this application. Figure 14 S701 in the diagram can also be implemented through S901 to S903. The steps are explained below:

[0286] S901. Based on the service capability representation data of each node obtained by processing the preset model, determine the initial path information between each first service node and the corresponding second service node; wherein, the initial path information includes at least one of the following: port information and address information of each node in the path, and sending task information of each first service node.

[0287] In this embodiment, after obtaining the service capability representation data of each node, the control node performs task planning modeling based on the service capability representation data of each node to determine the initial path information between each first service node and the corresponding second service node. The modeling constraints can be solved using formulas (1) to (4) in combination, ultimately solving for the constraints. and Maximize λ to determine the initial path information.

[0288] Formula (1) indicates that the gradient emitted by each worker n can only be aggregated on the switch s or the parameter server α.

[0289] Where λ is the sum of the gradient rates sent by all workers n to the exchange s.

[0290] The sum of the gradient rates sent by all workers to the switch must be less than the aggregation capacity of the switch.

[0291] The sum of the gradient rates sent by all workers to parameter server α and the gradient rates sent by all switches to parameter server α must be less than the uplink bandwidth B from the switch to the parameter server. α .

[0292] Formula explanation and constraints:

[0293] This indicates whether the gradient data emitted by worker n is aggregated by exchange s.

[0294] This indicates whether the gradient data emitted by worker n is aggregated by parameter server α.

[0295] This represents the rate at which worker n sends gradient data to parameter server α.

[0296] This represents the rate at which worker n sends gradient data to exchange s. Once the exchange performs gradient aggregation, the result should be sent to parameter server α.

[0297] y s, representing the sending rate of the switch to the parameter server α.

[0298] C s The ability to perform gradient aggregation for switches.

[0299] S902. Based on the service capabilities of each forwarding node in the path represented by the initial path information, determine the first forwarding node with the strongest aggregation capability among the forwarding nodes.

[0300] In this embodiment of the application, since the path may include multiple forwarding nodes, the control node analyzes the service capabilities of multiple forwarding nodes and determines the first forwarding node with the strongest aggregation capability.

[0301] S903. Combine the initial path information and the bitmap information of the first forwarding node to determine the path information.

[0302] In this embodiment, the control node combines initial path information with bitmap information of the first forwarding node determined based on the path represented by the initial path information to obtain path information. The control node then sends the determined path information to each node in the path.

[0303] In this embodiment, based on the service capabilities of each node obtained through a preset model processing, initial path information between each first service node and its corresponding second service node is determined. Based on the service capabilities of each forwarding node in the path represented by the initial path information, the first forwarding node with the strongest aggregation capability is determined among all forwarding nodes. The initial path information and the bitmap information of the first forwarding node are combined to determine the path information. Since the path is obtained through path planning based on the service capabilities of each node in the network topology, such path planning can maximize the utilization of the bandwidth capabilities of all current nodes. This means that all nodes with forwarding functions in the network topology can be used to upload the first packet, dispersing bandwidth pressure and solving the bandwidth bottleneck problem. Furthermore, the first forwarding node in the path is the forwarding node with the strongest aggregation capability. Aggregating multiple first packets through the forwarding node with the strongest aggregation capability can also improve the packet interaction efficiency between the first and second service nodes, while simultaneously improving the efficiency of the second service node in updating model parameters.

[0304] Please see Figure 18 The following is an optional flowchart illustrating the message processing method provided in the embodiments of this application, and the steps are explained as follows:

[0305] S904. Obtain the service capabilities of each forwarding node in the path information at predetermined intervals.

[0306] In this embodiment, the control node acquires the service capabilities of each forwarding node in each path information at predetermined intervals.

[0307] The predetermined interval can be one day or one week; however, in this embodiment, the specific duration of the predetermined interval is not limited. The control node can obtain the corresponding aggregated bandwidth from each forwarding node as the corresponding service capability.

[0308] S905. Based on the service capabilities of each of the forwarding nodes obtained each time, determine the first forwarding node with the strongest aggregation capability among the forwarding nodes.

[0309] In this embodiment, after the control node obtains the service capabilities (aggregated bandwidth) of each forwarding node, it determines the forwarding node with the largest aggregated bandwidth as the new first forwarding node with the strongest aggregation capability.

[0310] S906. Based on the newly determined first forwarding node, new path information is formed and sent to each of the nodes.

[0311] In this embodiment of the application, the control node can combine the initial path information with the bitmap information of the new first forwarding node to determine the new path information, and send the new path information to each node in the path.

[0312] In this embodiment, the service capabilities of each forwarding node in the path information are acquired at predetermined intervals. Based on the service capabilities of each forwarding node acquired each time, a new first forwarding node with the strongest aggregation capability is determined among the forwarding nodes. New path information is formed based on the newly determined first forwarding node and sent to each node. Thus, since the service capabilities of each node change, by determining a new first forwarding node with the strongest aggregation capability at predetermined intervals, the aggregation efficiency of the forwarding nodes used for aggregation during message interaction between the first and second service nodes can be guaranteed, thereby ensuring the efficiency of message interaction between the first and second service nodes, and also ensuring the efficiency of the second service node in updating model parameters.

[0313] Please see Figure 19 This is an interactive schematic diagram of the message processing method provided in the embodiments of this application, and the steps will be explained as follows:

[0314] S100, the first message is sent by multiple first service nodes in the aggregation path of the first forwarding node, and a second message is obtained; wherein, the path is determined based on the path information sent by the control node, and the path information is obtained by the control node through path planning based on the service capabilities of each node in the network topology; the first service nodes and the first forwarding nodes in different paths are different.

[0315] In this embodiment of the application, the implementation steps of step S100 can be implemented with reference to S101, and will not be described in detail here.

[0316] S200, the first forwarding node sends the second message to the second service node in the path.

[0317] In this embodiment of the application, the implementation steps of step S200 can be referred to S102, and will not be described in detail here.

[0318] Please see Figure 20 The diagram below illustrates the structure of the message processing apparatus provided in this embodiment. Figure 1 .

[0319] This application also provides a message processing device 600, applied to a first forwarding node, including: an aggregation unit 601 and a first sending unit 602.

[0320] Aggregation unit 601 is used to aggregate first messages sent by multiple first service nodes in a path to obtain a second message; wherein, the path is determined based on path information sent by the control node, and the path information is obtained by the control node through path planning based on the service capabilities of each node in the network topology; the first service nodes and the first forwarding nodes in different paths are different.

[0321] The first sending unit 602 is used to send the second message to the second service node in the path.

[0322] In this embodiment of the application, the message processing device 600 is used to receive local bitmap information and local node identifier sent by the control node.

[0323] In this embodiment of the application, the first message includes: a message header and a message payload;

[0324] The message header includes at least one of the following: communication address information, first target bitmap information of the target forwarding node, and multiple protocol fields; the message payload includes data to be aggregated.

[0325] In this embodiment of the application, the aggregation unit 601 in the message processing device 600 is used to determine the aggregation detection result of multiple first messages based on the matching result of multiple first target bitmap information and local bitmap information, as well as the current computing resources; wherein, each of the multiple first target bitmap information belongs to the corresponding first message;

[0326] If the aggregation detection result indicates that aggregation of multiple first messages is supported, then the data to be aggregated in the multiple message payloads is aggregated and combined with the message header to obtain the second message.

[0327] In this embodiment of the application, the data to be aggregated includes: N sub-data in a certain order; N is an integer greater than 0; the aggregation unit 601 in the message processing device 600 is used to aggregate the sub-data corresponding to the same order in multiple data to be aggregated to obtain an aggregated message payload.

[0328] Modify the first protocol field in the message header to obtain the aggregated message header; wherein, the modified first protocol field is used to indicate that the corresponding first message has been aggregated.

[0329] The aggregated message payload and the aggregated message header are combined to obtain the second message.

[0330] In this embodiment of the application, the aggregation unit 601 in the message processing device 600 is used to modify the second protocol field in the header of each first message to obtain the corresponding third message if the aggregation detection result indicates that the aggregation of multiple first messages is not supported; wherein, the modified second protocol field is used to indicate that the corresponding first forwarding node does not support aggregation.

[0331] Each of the third packets is sent to the second service node based on the next hop in the path information. The second service node then aggregates the multiple third packets and updates the model parameters based on the aggregated packets.

[0332] In this embodiment of the application, the message processing device 600 is used to receive a fourth message sent by the second service node and determine a message detection result for the fourth message; wherein, the message detection result is used to characterize whether the fourth message is a one-to-many message sent to multiple first service nodes;

[0333] If the message detection result indicates that the fourth message is a one-to-many message sent to multiple first service nodes, then the fourth message is copied for each first service node to obtain a fifth message corresponding to each first service node; the first sending unit 602 in the message processing device 600 is used to send the fifth message to each corresponding first service node.

[0334] In this embodiment of the application, the message processing device 600 is used to determine the message detection result based on the matching result of the second target bitmap information and the local bitmap information in the message header of the fourth message, and the third protocol field in the message header of the fourth message; wherein, the third protocol field is used to characterize whether the fourth message is a one-to-many message sent to multiple first service nodes.

[0335] It should be noted that, in the embodiments of this application, if the above-described message processing method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a message processing device (which may be a personal computer, etc.) to execute all or part of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0336] Correspondingly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps in the method on the first forwarding node side.

[0337] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0338] It should be noted that, Figure 21 A hardware entity illustration of an electronic device provided in the embodiments of this application. Figure 1 ,like Figure 21As shown, this application embodiment provides an electronic device 700, including a first memory 702 and a first processor 701. The first memory 702 stores a computer program that can run on the first processor 701. When the processor 701 executes the program, it implements the steps in the above-described method, wherein;

[0339] The first processor 701 typically controls the overall operation of the electronic device 700.

[0340] The first memory 702 is configured to store instructions and applications executable by the first processor 701, and can also cache data to be processed or already processed by the first processor 701 and various modules in the electronic device 700 (e.g., image data, audio data, voice communication data and video communication data), which can be implemented by flash memory or random access memory (RAM).

[0341] Correspondingly, this application also provides a computer program product, including a computer program that can be executed by the processor 701 of the electronic device 700 to complete the steps in the method of the message processing device 600.

[0342] Please see Figure 22 The diagram below illustrates the structure of the message processing apparatus provided in this embodiment. Figure 2 .

[0343] This application also provides a message processing device 800, applied to a control node, including: a path planning unit 801 and a second sending unit 802.

[0344] The path planning unit 801 is used to perform path planning based on the service capabilities of each node in the acquired network topology and determine path information; wherein, the path information is used to characterize the first service node, the first forwarding node and the second service node included in each path; the first service node and the first forwarding node are different in different paths.

[0345] The second sending unit 802 is used to send the path information to each of the nodes, so that the first forwarding node in each path can aggregate the first messages sent by multiple first service nodes respectively, and send the aggregated second message to the second service node.

[0346] In this embodiment of the application, the second sending unit 802 in the message processing device 800 is used to configure the correspondence between bitmap information and node identifier for each of the nodes, forming a correspondence set; send the correspondence set to each service node in the network topology, and send the bitmap information and node identifier corresponding to each forwarding node to the corresponding forwarding node; wherein, the service node includes: the first service node and the second service node; the forwarding node includes the first forwarding node.

[0347] In this embodiment of the application, the path planning unit 801 in the message processing device 800 is used to process the service capability representation data of each of the nodes obtained based on a preset model, and determine the initial path information between each first service node and the corresponding second service node; wherein, the initial path information includes at least one of the following: port information and address information of each of the nodes in the path, and sending task information of each first service node.

[0348] Based on the service capabilities of each forwarding node in the path represented by the initial path information, the first forwarding node with the strongest aggregation capability is determined among the forwarding nodes.

[0349] The initial path information and the bitmap information of the first forwarding node are combined to determine the path information.

[0350] In this embodiment of the application, the path planning unit 801 in the message processing device 800 is used to obtain the service capabilities of each forwarding node in the path information at predetermined intervals.

[0351] Based on the service capabilities of each forwarding node obtained each time, the first forwarding node with the strongest aggregation capability is determined among the forwarding nodes.

[0352] New path information is formed based on the newly determined first forwarding node and sent to each of the nodes.

[0353] Correspondingly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps in the method on the control node side.

[0354] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0355] It should be noted that, Figure 23A hardware entity illustration of an electronic device provided in the embodiments of this application. Figure 2 ,like Figure 23 As shown, this application embodiment provides an electronic device 900, including a second memory 902 and a second processor 901. The second memory 902 stores a computer program that can run on the second processor 901. When the processor 901 executes the program, it implements the steps in the above-described method, wherein;

[0356] The second processor 901 typically controls the overall operation of the electronic device 900.

[0357] The second memory 902 is configured to store instructions and applications executable by the second processor 901, and can also cache data to be processed or already processed by the second processor 901 and various modules in the electronic device 900 (e.g., image data, audio data, voice communication data, and video communication data), which can be implemented by flash memory or random access memory (RAM).

[0358] Correspondingly, this application also provides a computer program product, including a computer program that can be executed by the processor 901 of the electronic device 900 to complete the steps in the method of the message processing device 800.

[0359] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0360] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0361] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the apparatus or units can be electrical, mechanical, or other forms.

[0362] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0363] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0364] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0365] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0366] The above are merely embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method of processing a packet, the method comprising: The method is applied to a first forwarding node, and comprises the following steps of: obtaining a second message by aggregating first messages respectively sent by a plurality of first service nodes in an aggregated path; wherein the path is determined based on path information sent by a control node, and the path information is obtained by the control node based on service capabilities of each node in a network topology; the first service nodes in different paths and the first forwarding node are different; sending the second message to a second service node in the path.

2. The packet processing method of claim 1, wherein, Before the step of obtaining the second message by aggregating the first messages respectively sent by the plurality of first service nodes in the aggregated path, the method further comprises the following steps of: receiving local bitmap information and a local node identifier sent by the control node.

3. The packet processing method of claim 1, wherein, The first message comprises a message header and a message payload. The message header comprises at least one of the following: communication address information, first target bitmap information of a target forwarding node, and a plurality of protocol fields; and the message payload comprises to-be-aggregated data.

4. The packet processing method of any of claims 1 to 3, wherein, The step of obtaining the second message by aggregating the first messages respectively sent by the plurality of first service nodes in the aggregated path comprises the following steps of: determining an aggregation detection result of the plurality of first messages based on a matching result of a plurality of first target bitmap information and local bitmap information, and current computing resources; wherein each first target bitmap information in the plurality of first target bitmap information belongs to a corresponding first message; if the aggregation detection result represents that aggregation of the plurality of first messages is supported, then combining the to-be-aggregated data in the plurality of message payloads with the message header after aggregation, to obtain the second message.

5. The packet processing method of claim 4, wherein, The to-be-aggregated data comprises N sub-data in a certain order; N is an integer greater than 0; and the step of combining the to-be-aggregated data in the plurality of message payloads with the message header after aggregation, to obtain the second message, comprises the following steps of: aggregating the sub-data corresponding to the same order in the plurality of to-be-aggregated data, to obtain an aggregated message payload; modifying a first protocol field in the message header, to obtain an aggregated message header; wherein the modified first protocol field is used to represent that the corresponding first message has been subjected to aggregation processing; combining the aggregated message payload and the aggregated message header, to obtain the second message.

6. The packet processing method of claim 4, wherein, After the step of determining the aggregation detection result of the plurality of first messages based on the matching result of the plurality of first target bitmap information and the local bitmap information, and the current computing resources, the method further comprises the following steps of: if the aggregation detection result represents that aggregation of the plurality of first messages is not supported, then modifying a second protocol field in the message header of each first message, to obtain a corresponding third message; wherein the modified second protocol field is used to represent that the corresponding first forwarding node does not support aggregation; sending each third message to the second service node based on a next hop in the path information, so that the second service node aggregates the plurality of third messages, and then completes model parameter updating based on the aggregated message.

7. The packet processing method of any of claims 1 to 3, wherein, After the step of sending the second message to the second service node in the path, the method further comprises the following steps of: receive a fourth message sent by the second service node, and determine a message detection result for the fourth message; the message detection result is used to represent whether the fourth message is a one-to-many message sent to multiple first service nodes; if the message detection result represents that the fourth message is a one-to-many message sent to multiple first service nodes, the fourth message is copied for each first service node to obtain a fifth message corresponding to each first service node; send the fifth message to each corresponding first service node.

8. The packet processing method of claim 7, wherein, The method further comprises: determine the message detection result based on the matching result of the second target bitmap information in the message header of the fourth message and the local bitmap information, and the third protocol field in the message header of the fourth message; the third protocol field is used to represent whether the fourth message is a one-to-many message sent to multiple first service nodes.

9. A method of processing a packet, the method comprising: The method is applied to a control node, and comprises: perform path planning based on the service capabilities of the nodes in the obtained network topology, and determine path information; the path information is used to represent the first service node, the first forwarding node and the second service node included in each path; the first service node and the first forwarding node in different paths are different; send the path information to each node, so that the first forwarding node in each path aggregates first messages sent by multiple first service nodes respectively, and sends a second message obtained by aggregation to the second service node.

10. The packet processing method of claim 9, wherein, Before the path planning based on the service capabilities of the nodes in the obtained network topology, the method further comprises: configure a corresponding relationship between bitmap information and node identifiers for each node to form a corresponding relationship set; send the corresponding relationship set to each service node in the network topology, and send the bitmap information and the node identifier corresponding to each forwarding node to the corresponding forwarding node; the service nodes include the first service nodes and the second service nodes; the forwarding nodes include the first forwarding nodes.

11. The packet processing method of claim 10, wherein, The path planning based on the service capabilities of the nodes in the obtained network topology comprises: determine initial path information between each first service node and a corresponding second service node based on a preset model processing data represented by the service capabilities of each node; the initial path information includes at least one of the following: port information and address information of each node in the path, and sending task information of each first service node; determine the first forwarding node with the strongest aggregation capability in each forwarding node based on the service capabilities of each forwarding node in the path represented by the initial path information; combine the initial path information and the bitmap information of the first forwarding node to determine the path information.

12. The packet processing method of claim 11, wherein, The method further comprises the following steps after determining the first forwarding node with the strongest aggregation capability among the forwarding nodes in the path based on the service capabilities of the forwarding nodes in the path represented by the initial path information: acquiring the service capabilities of the forwarding nodes in the path information every predetermined time interval; determining a new first forwarding node with the strongest aggregation capability among the forwarding nodes based on the service capabilities of the forwarding nodes acquired each time; forming new path information based on the newly determined first forwarding node and sending the new path information to each node.

13. A packet processing device, characterized by The application is applied to a first forwarding node and comprises: an aggregation unit configured to aggregate first packets sent by a plurality of first service nodes in a path to obtain a second packet, wherein the path is determined based on path information sent by a control node, and the path information is obtained by the control node based on service capabilities of nodes in a network topology; the first service nodes and the first forwarding nodes in different paths are different; a first sending unit configured to send the second packet to a second service node in the path.

14. A packet processing device, characterized by The application is applied to a control node and comprises: a path planning unit configured to plan a path based on acquired service capabilities of nodes in a network topology to determine path information, wherein the path information is used to represent first service nodes, first forwarding nodes and second service nodes included in each path; the first service nodes and the first forwarding nodes in different paths are different; a second sending unit configured to send the path information to each node, so that the first forwarding node in each path aggregates first packets sent by a plurality of first service nodes and sends a second packet obtained by aggregation to the second service node.

15. An electronic device, comprising: The computer program is executed by the processor to implement the steps in the method of any one of claims 1 to 8 or the steps in the method of any one of claims 9 to 12.

16. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps in the method of any one of claims 1 to 8 or the steps in the method of any one of claims 9 to 12.

17. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps in the method of any one of claims 1 to 8 or the steps in the method of any one of claims 9 to 12.

Images