Data transmission method and device, network node and medium
By transmitting data packets between network nodes and offloading computing tasks using SFC and NSH protocols, network congestion and delay problems in distributed computing are solved, and computing efficiency is improved.
Patent Information
- Application Number
- CN202310497953.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-05
- Publication Date
- 2024-11-05
AI Technical Summary
The network congestion and communication delay caused by distributed computing in the data parallel collaboration mode reduces the computing efficiency.
By transmitting data packets between network nodes, using the Service Function Chain (SFC) protocol and the Network Service Header (NSH) protocol, offloading computing tasks to the network nodes, using the computing power of the network nodes to participate in computing, and reducing data between computing nodes Interaction.
It effectively avoids network congestion and communication delays, improves computing efficiency, and makes the training process of distributed machine learning models more efficient.
Smart Images

Figure CN118900264A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data transmission, and in particular to a data transmission method, device, network node and medium. Background Art
[0002] At present, machine learning is developing rapidly and has strong learning ability, and has been widely used in various fields. However, this powerful learning ability depends on complex neural network models and massive training data sets. The training process of machine learning is to continuously iterate on the training data set. Complex neural network models mean that the time consumption of a single calculation process increases; and massive training data sets mean an increase in the number of iterations. Therefore, although the reasoning ability of machine learning is improving, the training time is also rising rapidly.
[0003] In order to reduce the training time, distributed computing can be used to improve the parallelism of the training process. Distributed computing includes model parallelism and data parallelism collaborative training modes. For the data parallelism collaborative training mode, each computing node in the cluster is only responsible for completing its own part of the training data, and then the training models of all computing nodes are aggregated and merged through the training model parameters, and finally the machine learning training model is output.
[0004] However, in the data parallel collaborative mode, due to the excessive communication traffic, there are problems of network congestion and communication delay, which leads to low computing efficiency. Summary of the invention
[0005] The technical solution of the present invention aims to provide a data transmission method, device, network node and medium to solve the problem that the prior art distributed computing easily causes network congestion and communication delay, resulting in low computing efficiency.
[0006] To achieve the above object, an embodiment of the present invention provides a data transmission method, which is applied to a first network node. The method includes:
[0007] Acquire a first data packet sent by a first computing node on a first transmission path; wherein the first data packet includes a computing task identifier;
[0008] Calculate the calculation data carried by the first data packet according to the calculation task identifier to obtain a calculation result;
[0009] Generate a second data packet according to the calculation result and the first data packet;
[0010] The second data packet is sent to the second computing node through a second transmission path.
[0011] Optionally, in the data transmission method, the first data packet further includes computing operation information;
[0012] The step of calculating the calculation data carried by the first data packet according to the calculation task identifier to obtain a calculation result includes:
[0013] According to the computing task identifier, the operation calculation indicated by the computing operation information is performed on the computing data carried by the first data packet to obtain the computing result.
[0014] Optionally, in the data transmission method, generating a second data packet according to the calculation result and the first data packet includes:
[0015] Obtaining first encapsulation strategy information;
[0016] Determine the transmission protocol indicated by the service function chain SFC protocol according to the first encapsulation strategy information;
[0017] The calculation result and the first data packet are encapsulated using a transmission protocol indicated by an SFC protocol to generate the second data packet.
[0018] Optionally, in the data transmission method, the first data packet further includes path index information;
[0019] The step of generating a second data packet according to the calculation result and the first data packet includes:
[0020] Updating the path index information included in the first data packet;
[0021] The second data packet is generated according to the calculation result and the first data packet carrying the updated path index information.
[0022] Optionally, in the data transmission method, the step of sending the second data packet to the second computing node through the second transmission path includes:
[0023] Obtaining first transmission strategy information;
[0024] Determining the second transmission path according to the first transmission strategy information;
[0025] The second data packet is sent to the second computing node through the second transmission path.
[0026] Optionally, in the data transmission method, the first data packet further includes at least one of the following:
[0027] Path identification information; path index information; reserved bit; total number of calculation data blocks corresponding to the calculation task identification; calculation data block identification; calculation data sending sequence number; calculation data processing confirmation sequence number; calculation operation information; calculation data operation offset processing information.
[0028] To achieve the above object, an embodiment of the present invention further provides a data transmission method, which is applied to a second network node. The method includes:
[0029] Obtaining a service data packet sent by the first computing node;
[0030] Encapsulating the business data packet to generate a first data packet; wherein the first data packet includes a computing task identifier;
[0031] The first data packet is sent to a first network node; wherein the first network node is located on a first transmission path, and the first data packet is transmitted to the first computing node through the first transmission path.
[0032] Optionally, in the data transmission method, the step of encapsulating the service data packet to generate a first data packet includes:
[0033] Obtaining second encapsulation strategy information;
[0034] Determine the transmission protocol indicated by the SFC protocol according to the second encapsulation strategy information;
[0035] The service data packet is encapsulated using the transmission protocol indicated by the SFC protocol to generate the first data packet.
[0036] Optionally, in the data transmission method, the step of sending the first data packet to the first network node comprises:
[0037] Obtaining second transmission strategy information;
[0038] determining the first transmission path according to the second transmission strategy information;
[0039] The first data packet is sent to the first network node through the first transmission path.
[0040] Optionally, in the data transmission method, the first data packet further includes at least one of the following:
[0041] Path identification information; path index information; reserved bit; total number of calculation data blocks corresponding to the calculation task identification; calculation data block identification; calculation data sending sequence number; calculation data processing confirmation sequence number; calculation operation information; calculation data operation offset processing information.
[0042] To achieve the above object, an embodiment of the present invention further provides a data transmission method, which is applied to a third network node. The method includes:
[0043] Obtaining a second data packet sent by the first network node on the second transmission path; wherein the second data packet is generated according to a calculation result and the first data packet, and the calculation result is obtained by calculating the calculation data carried by the first data packet;
[0044] The second data packet is sent to a second computing node; wherein the second computing node is located on a second transmission path, and the second data packet is transmitted to the second computing node through the second transmission path.
[0045] Optionally, in the data transmission method, the step of sending the second data packet to the second computing node includes:
[0046] Get unblocking policy information;
[0047] Decapsulating the second data packet according to the decapsulation policy information to generate a third data packet;
[0048] The third data packet is sent to the second computing node.
[0049] Optionally, in the data transmission method, the second data packet includes path index information;
[0050] The step of sending the second data packet to the second computing node includes:
[0051] determining, according to the path index information, whether the third network node is a previous node of the second computing node on the second transmission path;
[0052] In a case where it is determined according to the path index information that the third network node is a previous node of the second computing node on the second transmission path, the second data packet is sent to the second computing node.
[0053] To achieve the above object, an embodiment of the present invention further provides a data transmission device, applied to a first network node, the device comprising:
[0054] A first acquisition module, configured to acquire a first data packet sent by a first computing node on a first transmission path; wherein the first data packet includes a computing task identifier;
[0055] an obtaining module, configured to calculate the calculation data carried by the first data packet according to the calculation task identifier to obtain a calculation result;
[0056] A generating module, configured to generate a second data packet according to the calculation result and the first data packet;
[0057] The first sending module is used to send the second data packet to the second computing node through a second transmission path.
[0058] To achieve the above object, an embodiment of the present invention further provides a data transmission device, which is applied to a second network node, and the device includes:
[0059] A second acquisition module, used to acquire a service data packet sent by the first computing node;
[0060] A second generating module is used to encapsulate the business data packet to generate a first data packet; wherein the first data packet includes a computing task identifier;
[0061] The second sending module is used to send the first data packet to a first network node; wherein the first network node is located on a first transmission path, and the first data packet is transmitted to the first computing node through the first transmission path.
[0062] To achieve the above object, an embodiment of the present invention further provides a data transmission device, which is applied to a third network node, and the device includes:
[0063] A third acquisition module is used to acquire a second data packet sent by the first network node on the second transmission path; wherein the second data packet is generated according to the calculation result and the first data packet, and the calculation result is obtained by calculating the calculation data carried by the first data packet;
[0064] The third sending module is used to send the second data packet to the second computing node; wherein the second computing node is located on a second transmission path, and the second data packet is transmitted to the second computing node through the second transmission path.
[0065] To achieve the above objectives, an embodiment of the present invention further provides a network node, comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the data transmission method as described in any one of the above items.
[0066] To achieve the above object, an embodiment of the present invention further provides a readable storage medium, on which a program is stored, and when the program is executed by a processor, the data transmission method as described in any one of the above items is implemented.
[0067] The beneficial effects of the above technical solution of the present invention are as follows:
[0068] The data transmission method described in the embodiment of the present invention is used to obtain a first data packet sent by a first computing node on a first transmission path; wherein the first data packet includes a computing task identifier; according to the computing task identifier, the computing data carried by the first data packet is calculated to obtain a computing result; according to the computing result and the first data packet, a second data packet is generated; and the second data packet is sent to the second computing node via a second transmission path. In this way, the computing task of the first computing node is offloaded to the first network node, so that the computing power of the first network node can participate in the computing task, avoiding network congestion and communication delays, and improving computing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0069] Figure 1 A schematic diagram of an application system architecture of the data transmission method according to an embodiment of the present invention;
[0070] Figure 2 A schematic diagram of a data transmission method according to one embodiment of the present invention;
[0071] Figure 3 A schematic diagram of a flow chart of a data transmission method according to another embodiment of the present invention;
[0072] Figure 4 A schematic diagram of a flow chart of a data transmission method according to another embodiment of the present invention;
[0073] Figure 5 It is a structural schematic diagram of a data transmission device according to one embodiment of the present invention;
[0074] Figure 6 It is a structural schematic diagram of a data transmission device according to another embodiment of the present invention;
[0075] Figure 7 It is a structural schematic diagram of a data transmission device according to another embodiment of the present invention;
[0076] Figure 8 A schematic diagram of the structure of a network node according to an embodiment of the present invention. DETAILED DESCRIPTION
[0077] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.
[0078] The application system architecture of the data transmission method described in the embodiment of the present invention is as follows: Figure 1As shown, it includes a control device, a first computing node, a first network node, a second network node, a third network node and a second computing node. The number of first network nodes is one or more, each first computing node can be connected to multiple first network nodes respectively, and each first network node can also be connected to other multiple first network nodes respectively, that is, multiple first network nodes are arranged in a mesh. Of course, the number of second network nodes and third network nodes can also be multiple.
[0079] This application system architecture can be applied to multi-computing node collaborative distributed machine learning training scenarios. It can convert data interaction between computing nodes into data interaction between network nodes, and data interaction between computing nodes and network nodes, so that the computing power of network nodes can participate in computing tasks, and computing nodes and network nodes can collaborate to complete computing tasks. The computing data of computing nodes is greatly reduced, network congestion and communication delays are avoided, and computing efficiency is improved.
[0080] Embodiments of the present invention Figure 1 For example, the number of the first network nodes is three, the number of the second network node is one, and the number of the third network node is one.
[0081] It should be noted that, the first network node 2 includes a service function forwarder (Service Function Forwarder, SFF for short) and a service processing function (Service Function, SF for short). The first network node 1 and the first network node 3 may only include the SFF.
[0082] In order to solve the problem that the existing distributed computing technology is prone to cause network congestion and communication delays, resulting in low computing efficiency, an embodiment of the present invention provides a data transmission method, in which a first network node obtains a first data packet sent by a first computing node on a first transmission path, and the first data packet includes a computing task identifier. According to the computing task identifier, the computing data carried by the first data packet is calculated to obtain a computing result. According to the computing result and the first data packet, a second data packet is generated, and the second data packet is sent to the second computing node through the second transmission path. Therefore, by adopting this implementation, the computing task of the first computing node is unloaded to the first network node, so that the computing power of the first network node can participate in the computing task, avoiding network congestion and communication delays, and improving computing efficiency.
[0083] One embodiment of the present invention, as Figure 2 As shown, a data transmission method is provided, which is applied to a first network node, and the method includes:
[0084] Step 201: Acquire a first data packet sent by a first computing node on a first transmission path; wherein the first data packet includes a computing task identifier.
[0085] In this embodiment of the present invention, the transmission path from the first computing node to the first network node is called the first transmission path, which includes the first computing node and the first network node.
[0086] The computing task identifier is used to distinguish different computing tasks and provide the possibility for decomposition and combination of computing tasks.
[0087] Step 202: Calculate the calculation data carried by the first data packet according to the calculation task identifier to obtain a calculation result.
[0088] In this embodiment of the present invention, the first network node can obtain the historical calculation data and / or historical calculation results corresponding to the calculation task identifier based on the calculation task identifier, and then calculate the calculation data carried by the first data packet in combination with the historical calculation data and / or historical calculation results to obtain the calculation result.
[0089] Step 203: Generate a second data packet according to the calculation result and the first data packet.
[0090] Step 204: Send the second data packet to the second computing node through a second transmission path.
[0091] In this embodiment of the present invention, the transmission path from the first network node to the second computing node is called the second transmission path, including the second computing node but not the first network node. The first network node is located on the first transmission path, and the first transmission path is connected to the second transmission path.
[0092] It should be noted that in the machine learning training model scenario, the first computing node and the second computing node in this embodiment of the present invention can both be terminals or servers responsible for the calculation of the training model. For data parallelism, different computing nodes are responsible for the calculation of all training models, but each server is only responsible for calculating part of the learning data. The training models of all computing nodes need to be aggregated and merged through the parameters of the training models of each computing node before the machine learning training model can be output.
[0093] In one implementation manner, optionally, the first data packet also includes computing operation information;
[0094] Wherein, step 202 includes:
[0095] According to the computing task identifier, the operation calculation indicated by the computing operation information is performed on the computing data carried by the first data packet to obtain the computing result.
[0096] It should be noted that the metadata carried by the first data packet is referred to as calculation data.
[0097] The computing operation information is used to indicate that at least one of the following is performed on the computing data carried by the first data packet:
[0098] Shaping (Shap) operation calculation; convolution operation (Conv) calculation; full connection (Fc) operation calculation; pooling (Pool) operation calculation; flattening (Flat) operation calculation.
[0099] In this embodiment of the present invention, historical calculation data and / or historical calculation results corresponding to the calculation task identifier are obtained according to the calculation task identifier, and then the operation calculation indicated by the calculation operation information is performed on the historical calculation data and / or historical calculation results, as well as the calculation data carried by the first data packet, to obtain the calculation result.
[0100] It should also be noted that the first network node is Figure 1 The first network node 2 includes a SFF and a SF.
[0101] The SFF of the first network node 2 obtains the first data packet sent by the first computing node on the first transmission path. Since the first transmission path is the transmission path between the first computing node and the first network node 2, including the first computing node, the second network node, and the first network node 1, the first data packet is sent by the first computing node, forwarded by the second network node and the first network node 1, and reaches the SFF of the first network node 2.
[0102] After the SFF of the first network node 2 obtains the first data packet, the SFF of the first network node 2 determines the operation instruction information of the first network node 2 according to the path identification information and the path index information carried by the first data packet, and the association relationship between the operation instruction information and the path identification information and the path index information respectively; then, the SFF of the first network node 2 determines whether the first network node 2 performs a computing operation according to the operation instruction information. Here, if it is determined that the first network node 2 does not perform a computing operation, the first network node 2 performs a forwarding operation, and the SFF of the first network node 2 forwards the first data packet to the first network node 3.
[0103] The association relationship between the operation instruction information and the path identification information and the path index information may be sent by the control device.
[0104] The SFF of the first network node 2 determines, according to the operation instruction information, that the SFF of the first network node 2 forwards the first data packet to the SF of the first network node 2 when the first network node 2 performs the computing operation.
[0105] Next, the SF of the first network node 2 performs the operation calculation indicated by the calculation operation information on the calculation data carried by the first data packet according to the calculation task identifier, obtains the calculation result, and then forwards the calculation result to the SFF of the first network node 2.
[0106] In one implementation mode, optionally, step 203 includes:
[0107] Obtaining first encapsulation strategy information;
[0108] Determine, according to the first encapsulation strategy information, a transmission protocol indicated by a service function chain (SFC) protocol;
[0109] The calculation result and the first data packet are encapsulated using a transmission protocol indicated by an SFC protocol to generate the second data packet.
[0110] Optionally, the obtaining the first encapsulation strategy information includes:
[0111] Receive first encapsulation strategy information sent by a control device, where the control device includes at least one of the following:
[0112] Terminals; servers; network management and orchestration systems.
[0113] Further, determining the transmission protocol indicated by the SFC protocol according to the first encapsulation strategy information includes:
[0114] According to the first encapsulation strategy information, it is determined that the transmission protocol indicated by the SFC protocol is the NSH (Network Service Header) protocol.
[0115] It should be noted that the NSH protocol is an extensible protocol. When the NSH protocol is used, extended information is added to the header of the first data packet. The extended information indicates that the first network node participates in the computing task, and the computing task of the first computing node is offloaded to the first network node, thereby reducing processing delay and improving computing efficiency.
[0116] Furthermore, the adopting of the transmission protocol indicated by the SFC protocol to encapsulate the calculation result and the first data packet to generate the second data packet includes:
[0117] The calculation result and the first data packet are encapsulated using the NSH protocol to generate the second data packet.
[0118] It should be noted that, after receiving the calculation result sent by the SF of the first network node 2, the SFF of the first network node 2 encapsulates the calculation result and the first data packet using the NSH protocol to generate a second data packet.
[0119] In one implementation manner, optionally, the first data packet also includes path index information;
[0120] Wherein, step 203 includes:
[0121] Updating the path index information included in the first data packet;
[0122] The second data packet is generated according to the calculation result and the first data packet carrying the updated path index information.
[0123] In this embodiment of the present invention, after obtaining the calculation result, the first network node updates the path index information included in the first data packet, and then encapsulates the calculation result and the first data packet carrying the updated path index information to generate a second data packet.
[0124] Optionally, updating the path index information included in the first data packet includes:
[0125] The path index value indicated by the path index information is subtracted by one.
[0126] In one implementation mode, optionally, step 204 includes:
[0127] Obtaining first transmission strategy information;
[0128] Determining the second transmission path according to the first transmission strategy information;
[0129] The second data packet is sent to the second computing node through the second transmission path.
[0130] Optionally, the acquiring the first transmission strategy information includes:
[0131] Receive first transmission strategy information sent by a control device, where the control device includes at least one of the following:
[0132] Terminals; servers; network management and orchestration systems.
[0133] In the embodiment of the present invention, optionally, the first data packet further includes at least one of the following:
[0134] Path identification information; path index information; reserved bit; total number of calculation data blocks corresponding to the calculation task identification; calculation data block identification; calculation data sending sequence number; calculation data processing confirmation sequence number; calculation operation information; calculation data operation offset processing information.
[0135] Specifically, the header of the first data packet is shown in Table 1 below.
[0136]
[0137] Table 1
[0138] Below, the English in the above Table 1 is explained and illustrated.
[0139] The first row in Table 1 is the basic header (Base Header), which is used to provide information about the service header and payload protocol, and is not described in detail here.
[0140] The second row in Table 1 is the Service Path Header, which includes the Service Path Identifier and the Service Index.
[0141] Lines 3 to 7 in Table 1 may be referred to as an on-line computing service context header, including:
[0142] Reserved bit (Reserved); computing task ID (Group ID); total number of computing data blocks corresponding to the computing task ID (Total MetaData Numbers); computing data block ID (MetaData ID); computing data sending sequence number (Seq Number); computing data processing confirmation sequence number (Ack Seq Number); computing operation information (DataOperation); computing data operation offset processing information (Data Offset).
[0143] Among them, the reserved bits provide the possibility for subsequent additional information.
[0144] The path identification information and the path index information can both be in table form, i.e., a path identification table and a path index table. It should be noted that the control device manages and arranges according to the service request of the first computing node and the registration information of all managed nodes (e.g., the first network node, the second network node, and the third network node), and determines the nodes involved in data transmission and data calculation, i.e., determines the second network node, at least one first network node, and the third network node.
[0145] Then, the control device formulates the transmission path of the data packet and forms a table. The first network node passed through is recorded in the first row of the path identification table. Then, the identifications of all other network nodes participating in the data transmission are recorded in the path identification table in sequence.
[0146] Correspondingly, the path index value of the first network node is recorded in the path index table in sequence. Of course, the path index values of all network nodes participating in the data transmission can also be recorded in the path index table in sequence. The path index values are assigned in order from large to small. The path index value of the last network node participating in the data transmission can be assigned 1. After each network node executes data transmission, the corresponding path index value needs to be reduced by 1 until the path index value of the network node is 0, which means that the network node is the last network node participating in the data transmission. The network node can send the data packet to the second computing node to complete the data transmission.
[0147] In addition, only one path index value can be recorded in the path index table. After each network node executes data transmission, the path index value needs to be reduced by 1 until the path index value is 0, which means that the current network node is the last network node participating in the data transmission. The network node can send the data packet to the second computing node to complete the data transmission.
[0148] Furthermore, the total number of computing data blocks corresponding to the computing data block identifier and the computing task identifier is used to transfer data by sequence number, thereby realizing sequence number identification, mapping, splitting and merging between large data units and small data units in distributed computing tasks.
[0149] For the calculation data sending sequence number and the calculation data processing confirmation sequence number, after executing data transmission, the network node fills in the calculation data processing confirmation sequence number according to the calculation data sending sequence number. In this way, if the first calculation node obtains the calculation data processing confirmation sequence number, it can be determined that no data packet loss has occurred and no retransmission is required. If the first calculation node does not obtain the calculation data processing packet loss, it can be determined that the sending data packet is lost and retransmission is required, that is, the service data packet is sent again.
[0150] The calculation data operation offset processing information is used to instruct the first network node to perform offset processing during the calculation process of the calculation data carried by the first data packet.
[0151] In addition, the computing task identifier and computing operation information have been described in other embodiments and will not be repeated here.
[0152] Therefore, the first network node described in the embodiment of the present invention not only realizes the data transmission function, but also realizes the on-line computing function, presenting a customizable, schedulable and programmable on-line computing function to the outside world, completing data transmission, data encapsulation and instruction interaction, thereby solving the problem that distributed computing easily causes network congestion and communication delays, resulting in low computing efficiency.
[0153] Another embodiment of the present invention, as Figure 3As shown, a data transmission method is provided, which is applied to a second network node, and the method includes:
[0154] Step 301, obtaining a service data packet sent by a first computing node;
[0155] The second network node may be Figure 1 The first computing node sends the service data packet that needs to be calculated to the second network node.
[0156] Step 302, encapsulating the business data packet to generate a first data packet; wherein the first data packet includes a computing task identifier;
[0157] In the process of encapsulating the service data packets, the second network node needs to associate the service data packets into a group, and the first data packets of the same type that need to be calculated by the first network node can be encapsulated to carry the same calculation task identifier.
[0158] For TCP and UDP packets, flow identification is performed based on the packet quintuple.
[0159] Step 303: Send the first data packet to a first network node; wherein the first network node is located on a first transmission path, and the first data packet is transmitted to the first computing node through the first transmission path.
[0160] In this embodiment of the present invention, the second network node is located on the first transmission path, and the transmission path from the first computing node through the second network node to the first network node is called the first transmission path, including the first computing node, the second network node and the first network node.
[0161] That is to say, the first computing node sends the business data packet to the second network node through the first transmission path; the second network node encapsulates the business data packet to generate a first data packet, and sends the first data packet to the first network node through the first transmission path; the first network node calculates the first data packet to obtain a calculation result, and generates a second data packet based on the calculation result and the first data packet, and then sends the second data packet to the second computing node through the second transmission path.
[0162] In one implementation mode, optionally, step 302 includes:
[0163] Obtaining second encapsulation strategy information;
[0164] Determine the transmission protocol indicated by the SFC protocol according to the second encapsulation strategy information;
[0165] The service data packet is encapsulated using the transmission protocol indicated by the SFC protocol to generate the first data packet.
[0166] Optionally, the acquiring the second encapsulation strategy information includes:
[0167] Receiving second encapsulation strategy information sent by a control device, wherein the control device includes at least one of the following:
[0168] Terminals; servers; network management and orchestration systems.
[0169] Further, determining the transmission protocol indicated by the SFC protocol according to the second encapsulation strategy information includes:
[0170] According to the second encapsulation strategy information, it is determined that the transmission protocol indicated by the SFC protocol is the NSH protocol.
[0171] Furthermore, the adopting the transmission protocol indicated by the SFC protocol to encapsulate the service data packet to generate the first data packet includes:
[0172] The service data packet is encapsulated using the NSH protocol to generate the first data packet.
[0173] In one implementation mode, optionally, step 303 includes:
[0174] Obtaining second transmission strategy information;
[0175] determining the first transmission path according to the second transmission strategy information;
[0176] The first data packet is sent to the first network node through the first transmission path.
[0177] Optionally, the acquiring the second transmission strategy information includes:
[0178] Receive second transmission strategy information sent by a control device, wherein the control device includes at least one of the following:
[0179] Terminals; servers; network management and orchestration systems.
[0180] In one implementation manner, optionally, the first data packet further includes at least one of the following:
[0181] Path identification information; path index information; reserved bit; total number of calculation data blocks corresponding to the calculation task identification; calculation data block identification; calculation data sending sequence number; calculation data processing confirmation sequence number; calculation operation information; calculation data operation offset processing information.
[0182] Another embodiment of the present invention, as Figure 4 As shown, a data transmission method is provided, which is applied to a third network node, and the method includes:
[0183] Step 401, obtaining a second data packet sent by a first network node on a second transmission path; wherein the second data packet is generated according to a calculation result and a first data packet, and the calculation result is obtained by calculating the calculation data carried by the first data packet;
[0184] Step 402, sending the second data packet to a second computing node; wherein the second computing node is located on a second transmission path, and the second data packet is transmitted to the second computing node via the second transmission path.
[0185] In this embodiment of the present invention, the third network node is located on the second transmission path, and the transmission path from the first network node through the third network node to the second computing node is called the second transmission path, including the third network node and the second computing node, but not including the first network node. The first network node is located on the first transmission path, and the first transmission path and the second transmission path are connected.
[0186] That is to say, the first computing node sends the business data packet to the second network node through the first transmission path; the second network node encapsulates the business data packet to generate a first data packet, and sends the first data packet to the first network node through the first transmission path; the first network node calculates the first data packet to obtain a calculation result, and generates a second data packet based on the calculation result and the first data packet, and then sends the second data packet to the third network node through the second transmission path; the third network node sends the second data packet to the second computing node through the second transmission path.
[0187] It should be noted that the first transmission path may also include other first network nodes, but these first network nodes do not calculate the calculation data carried by the first data packet, and only forward the first data packet. Of course, the second transmission path may include other first network nodes, but these first network nodes also do not calculate the second data packet, and only forward the second data packet.
[0188] In addition, the embodiment of the present invention may also use at least two first network nodes to calculate the calculation data carried by the first data packet according to requirements.
[0189] In one implementation mode, optionally, step 402 includes:
[0190] Get unblocking policy information;
[0191] Decapsulating the second data packet according to the decapsulation policy information to generate a third data packet;
[0192] The third data packet is sent to the second computing node.
[0193] Optionally, obtaining the unblocking policy information includes:
[0194] Receiving unblocking policy information sent by a control device, wherein the control device includes at least one of the following:
[0195] Terminals; servers; network management and orchestration systems.
[0196] In this embodiment of the present invention, the third network node decapsulates the acquired second data packet according to the decapsulation policy information, generates a third data packet, and sends the third data packet to the second computing node.
[0197] In one implementation manner, optionally, the second data packet includes path index information;
[0198] Wherein, step 402 includes:
[0199] determining, according to the path index information, whether the third network node is a previous node of the second computing node on the second transmission path;
[0200] In a case where it is determined according to the path index information that the third network node is a previous node of the second computing node on the second transmission path, the second data packet is sent to the second computing node.
[0201] In this embodiment of the present invention, when the path index value indicated by the path index information included in the second data packet is equal to the first value, the third network node determines that the third network node is located at a position before the position of the second computing node on the second transmission path, that is, the third network node is the previous node of the second computing node on the second transmission path.
[0202] It should be noted that the data packets (including service data packets, first data packets, second data packets and third data packets) of the embodiment of the present invention are sent by the first computing node during the transmission process, and pass through the second network node, at least one first network node, and the third network node in sequence to reach the second computing node. Wherein, each first network node needs to update the path index information carried by the data packet when performing a computing operation or forwarding operation on the data packet, and optionally, reduce the path index value indicated by the path index information by one, until the path index value indicated by the path index information carried by the data packet received by the third network node is the first value, which means that the third network node needs to send the data packet to the second computing node.
[0203] The first network node, the second network node and the third network node mentioned above may all be switches or routers. The switch is a programmable switch and the router is a programmable router.
[0204] In summary, by adopting the data transmission method described in the embodiment of the present invention, the computing tasks of computing nodes including GPUs (such as terminals and servers) are offloaded to network nodes (such as switches and routers). On the network plane, data transmission is achieved while completing data calculation (such as aggregation processing), thereby greatly improving the efficiency of distributed machine learning model training.
[0205] One embodiment of the present invention further provides a data transmission device, which is applied to a first network node, such as Figure 5 As shown, the device comprises:
[0206] A first acquisition module 501 is used to acquire a first data packet sent by a first computing node on a first transmission path; wherein the first data packet includes a computing task identifier;
[0207] An obtaining module 502 is used to calculate the calculation data carried by the first data packet according to the calculation task identifier to obtain a calculation result;
[0208] A generating module 503, configured to generate a second data packet according to the calculation result and the first data packet;
[0209] The first sending module 504 is configured to send the second data packet to the second computing node through a second transmission path.
[0210] Optionally, in the data transmission device, the first data packet further includes computing operation information;
[0211] The obtaining module 502 is specifically used for:
[0212] According to the computing task identifier, the operation calculation indicated by the computing operation information is performed on the computing data carried by the first data packet to obtain the computing result.
[0213] Optionally, in the data transmission device, the generating module 503 is specifically configured to:
[0214] Obtaining first encapsulation strategy information;
[0215] Determine the transmission protocol indicated by the service function chain SFC protocol according to the first encapsulation strategy information;
[0216] The calculation result and the first data packet are encapsulated using a transmission protocol indicated by an SFC protocol to generate the second data packet.
[0217] Optionally, in the data transmission device, the first data packet further includes path index information;
[0218] The generating module 503 is specifically used for:
[0219] Updating the path index information included in the first data packet;
[0220] The second data packet is generated according to the calculation result and the first data packet carrying the updated path index information.
[0221] Optionally, in the data transmission device, the first sending module 504 is specifically configured to:
[0222] Obtaining first transmission strategy information;
[0223] Determining the second transmission path according to the first transmission strategy information;
[0224] The second data packet is sent to the second computing node through the second transmission path.
[0225] Optionally, in the data transmission device, the first data packet further includes at least one of the following:
[0226] Path identification information; path index information; reserved bit; total number of calculation data blocks corresponding to the calculation task identification; calculation data block identification; calculation data sending sequence number; calculation data processing confirmation sequence number; calculation operation information; calculation data operation offset processing information.
[0227] Another embodiment of the present invention further provides a data transmission device, which is applied to a second network node, such as Figure 6 As shown, the device comprises:
[0228] The second acquisition module 601 is used to acquire the service data packet sent by the first computing node;
[0229] The second generating module 602 is used to encapsulate the business data packet to generate a first data packet; wherein the first data packet includes a computing task identifier;
[0230] The second sending module 603 is used to send the first data packet to a first network node; wherein the first network node is located on a first transmission path, and the first data packet is transmitted to the first computing node through the first transmission path.
[0231] Optionally, in the data transmission device, the second generating module 602 is specifically configured to:
[0232] Obtaining second encapsulation strategy information;
[0233] Determine the transmission protocol indicated by the SFC protocol according to the second encapsulation strategy information;
[0234] The service data packet is encapsulated using the transmission protocol indicated by the SFC protocol to generate the first data packet.
[0235] Optionally, in the data transmission device, the second sending module 603 is specifically configured to:
[0236] Obtaining second transmission strategy information;
[0237] determining the first transmission path according to the second transmission strategy information;
[0238] The first data packet is sent to the first network node through the first transmission path.
[0239] Optionally, in the data transmission device, the first data packet further includes at least one of the following:
[0240] Path identification information; path index information; reserved bit; total number of calculation data blocks corresponding to the calculation task identification; calculation data block identification; calculation data sending sequence number; calculation data processing confirmation sequence number; calculation operation information; calculation data operation offset processing information.
[0241] Another embodiment of the present invention further provides a data transmission device, which is applied to a third network node, such as Figure 7 As shown, the device comprises:
[0242] A third acquisition module 701 is used to acquire a second data packet sent by the first network node on the second transmission path; wherein the second data packet is generated according to the calculation result and the first data packet, and the calculation result is obtained by calculating the calculation data carried by the first data packet;
[0243] The third sending module 702 is used to send the second data packet to a second computing node; wherein the second computing node is located on a second transmission path, and the second data packet is transmitted to the second computing node through the second transmission path.
[0244] Optionally, in the data transmission device, the third sending module 702 is specifically configured to:
[0245] Get unblocking policy information;
[0246] Decapsulating the second data packet according to the decapsulation policy information to generate a third data packet;
[0247] The third data packet is sent to the second computing node.
[0248] Optionally, in the data transmission device, the second data packet includes path index information;
[0249] The third sending module 702 is specifically configured to:
[0250] determining, according to the path index information, whether the third network node is a previous node of the second computing node on the second transmission path;
[0251] In a case where it is determined according to the path index information that the third network node is a previous node of the second computing node on the second transmission path, the second data packet is sent to the second computing node.
[0252] like Figure 8 As shown, an embodiment of the present invention also provides a network node, which is a first network node, including: a processor 801; and a memory 802 connected to the processor 801 through a bus interface, the memory 802 is used to store programs and data used by the processor 801 when performing operations, and the processor 801 calls and executes the programs and data stored in the memory 802.
[0253] The network node further includes a transceiver 803, which is connected to the bus interface and is used to receive and send data under the control of the processor 801;
[0254] Specifically, the transceiver 803 performs the following process:
[0255] Acquire a first data packet sent by a first computing node on a first transmission path; wherein the first data packet includes a computing task identifier;
[0256] The processor 801 executes the following process:
[0257] Calculate the calculation data carried by the first data packet according to the calculation task identifier to obtain a calculation result;
[0258] Generate a second data packet according to the calculation result and the first data packet;
[0259] The transceiver 803 further performs the following process:
[0260] The second data packet is sent to the second computing node through a second transmission path.
[0261] Optionally, in the network node, the first data packet also includes computing operation information;
[0262] The processor 801 is specifically configured to:
[0263] According to the computing task identifier, the operation calculation indicated by the computing operation information is performed on the computing data carried by the first data packet to obtain the computing result.
[0264] Optionally, in the network node, the processor 801 is specifically configured to:
[0265] Obtaining first encapsulation strategy information;
[0266] Determine the transmission protocol indicated by the service function chain SFC protocol according to the first encapsulation strategy information;
[0267] The calculation result and the first data packet are encapsulated using a transmission protocol indicated by an SFC protocol to generate the second data packet.
[0268] Optionally, the network node, wherein the first data packet also includes path index information;
[0269] The processor 801 is specifically configured to:
[0270] Updating the path index information included in the first data packet;
[0271] The second data packet is generated according to the calculation result and the first data packet carrying the updated path index information.
[0272] Optionally, in the network node, the transceiver 803 is specifically configured to:
[0273] Obtaining first transmission strategy information;
[0274] Determining the second transmission path according to the first transmission strategy information;
[0275] The second data packet is sent to the second computing node through the second transmission path.
[0276] Optionally, in the network node, the first data packet further includes at least one of the following:
[0277] Path identification information; path index information; reserved bit; total number of calculation data blocks corresponding to the calculation task identification; calculation data block identification; calculation data sending sequence number; calculation data processing confirmation sequence number; calculation operation information; calculation data operation offset processing information.
[0278] Among them, Figure 8In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 801 and memory represented by memory 802. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides a user interface 804. The transceiver 803 may be a plurality of components, i.e., including a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium. The processor 801 is responsible for managing the bus architecture and general processing, and the memory 802 may store data used by the processor 801 when performing operations.
[0279] An embodiment of the present invention also provides a network node, which is a second network node, comprising: a processor; and a memory connected to the processor via a bus interface, the memory being used to store programs and data used by the processor when performing operations, and the processor calling and executing the programs and data stored in the memory.
[0280] Wherein, the network node further comprises a transceiver, the transceiver being connected to the bus interface and configured to receive and send data under the control of the processor;
[0281] It should be noted that the second network node provided in the embodiment of the present invention is Figure 8 The structure of the first network node shown is similar and will not be described in detail here.
[0282] Specifically, the transceiver performs the following process:
[0283] Obtaining a service data packet sent by the first computing node;
[0284] The processor executes the following processes:
[0285] Encapsulating the business data packet to generate a first data packet; wherein the first data packet includes a computing task identifier;
[0286] The transceiver further performs the following process:
[0287] The first data packet is sent to a first network node; wherein the first network node is located on a first transmission path, and the first data packet is transmitted to the first computing node through the first transmission path.
[0288] Optionally, in the network node, the processor is specifically configured to:
[0289] Obtaining second encapsulation strategy information;
[0290] Determine the transmission protocol indicated by the SFC protocol according to the second encapsulation strategy information;
[0291] The service data packet is encapsulated using the transmission protocol indicated by the SFC protocol to generate the first data packet.
[0292] Optionally, in the network node, the transceiver is specifically used for:
[0293] Obtaining second transmission strategy information;
[0294] determining the first transmission path according to the second transmission strategy information;
[0295] The first data packet is sent to the first network node through the first transmission path.
[0296] Optionally, in the network node, the first data packet further includes at least one of the following:
[0297] Path identification information; path index information; reserved bit; total number of calculation data blocks corresponding to the calculation task identification; calculation data block identification; calculation data sending sequence number; calculation data processing confirmation sequence number; calculation operation information; calculation data operation offset processing information.
[0298] An embodiment of the present invention also provides a network node, which is a third network node, including: a processor; and a memory connected to the processor through a bus interface, the memory being used to store programs and data used by the processor when performing operations, and the processor calling and executing the programs and data stored in the memory.
[0299] Wherein, the network node further comprises a transceiver, the transceiver being connected to the bus interface and configured to receive and send data under the control of the processor;
[0300] It should be noted that the third network node provided in the embodiment of the present invention is Figure 8 The structure of the first network node shown is similar and will not be described in detail here.
[0301] Specifically, the transceiver performs the following process:
[0302] Obtaining a second data packet sent by the first network node on the second transmission path; wherein the second data packet is generated according to a calculation result and the first data packet, and the calculation result is obtained by calculating the calculation data carried by the first data packet;
[0303] The second data packet is sent to a second computing node; wherein the second computing node is located on a second transmission path, and the second data packet is transmitted to the second computing node through the second transmission path.
[0304] Optionally, in the network node, the transceiver is specifically used for:
[0305] Get unblocking policy information;
[0306] Decapsulating the second data packet according to the decapsulation policy information to generate a third data packet;
[0307] The third data packet is sent to the second computing node.
[0308] Optionally, the network node, wherein the second data packet includes path index information;
[0309] The transceiver is specifically used for:
[0310] determining, according to the path index information, whether the third network node is a previous node of the second computing node on the second transmission path;
[0311] In a case where it is determined according to the path index information that the third network node is a previous node of the second computing node on the second transmission path, the second data packet is sent to the second computing node.
[0312] In addition, a specific embodiment of the present invention further provides a readable storage medium on which a computer program is stored, wherein the program, when executed by a processor, implements the steps in any one of the data transmission methods described above.
[0313] Specifically, the computer-readable storage medium is applied to the above-mentioned first network node, second network node or third network node. When applied to the first network node, second network node or third network node, the execution steps in the corresponding data transmission method are described in detail above and will not be repeated here.
[0314] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0315] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may be physically included separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.
[0316] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform some steps of the sending and receiving methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.
[0317] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary personnel in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A data transmission method, characterized in that: Applied to a first network node, the method comprises: Acquire a first data packet sent by a first computing node on a first transmission path; wherein the first data packet includes a computing task identifier; Calculate the calculation data carried by the first data packet according to the calculation task identifier to obtain a calculation result; Generate a second data packet according to the calculation result and the first data packet; The second data packet is sent to the second computing node through a second transmission path.
2. The data transmission method according to claim 1, characterized in that: The first data packet also includes computing operation information; The step of calculating the calculation data carried by the first data packet according to the calculation task identifier to obtain a calculation result includes: According to the computing task identifier, the operation calculation indicated by the computing operation information is performed on the computing data carried by the first data packet to obtain the computing result.
3. The data transmission method according to claim 1, characterized in that: The step of generating a second data packet according to the calculation result and the first data packet includes: Obtaining first encapsulation strategy information; Determine the transmission protocol indicated by the service function chain SFC protocol according to the first encapsulation strategy information; The calculation result and the first data packet are encapsulated using a transmission protocol indicated by an SFC protocol to generate the second data packet.
4. The data transmission method according to claim 1, characterized in that: The first data packet also includes path index information; The step of generating a second data packet according to the calculation result and the first data packet includes: Updating the path index information included in the first data packet; The second data packet is generated according to the calculation result and the first data packet carrying the updated path index information.
5. The data transmission method according to claim 1, characterized in that: The sending the second data packet to the second computing node through the second transmission path includes: Obtaining first transmission strategy information; Determining the second transmission path according to the first transmission strategy information; The second data packet is sent to the second computing node through the second transmission path.
6. The data transmission method according to claim 1, characterized in that: The first data packet also includes at least one of the following: Path identification information; Path index information; Reserved bit; total number of calculation data blocks corresponding to the calculation task identifier; calculation data block identifier; calculation data sending sequence number; calculation data processing confirmation sequence number; calculation operation information; calculation data operation offset processing information.
7. A data transmission method, characterized in that: Applied to a second network node, the method comprises: Obtaining a service data packet sent by the first computing node; Encapsulating the business data packet to generate a first data packet; wherein the first data packet includes a computing task identifier; The first data packet is sent to a first network node; wherein the first network node is located on a first transmission path, and the first data packet is transmitted to the first computing node through the first transmission path.
8. The data transmission method according to claim 7, characterized in that: The step of encapsulating the service data packet to generate a first data packet includes: Obtaining second encapsulation strategy information; Determine the transmission protocol indicated by the SFC protocol according to the second encapsulation strategy information; The service data packet is encapsulated using the transmission protocol indicated by the SFC protocol to generate the first data packet.
9. The data transmission method according to claim 7, characterized in that: The sending the first data packet to the first network node includes: Obtaining second transmission strategy information; determining the first transmission path according to the second transmission strategy information; The first data packet is sent to the first network node through the first transmission path.
10. The data transmission method according to claim 7, characterized in that: The first data packet also includes at least one of the following: Path identification information; Path index information; Reserved bit; total number of calculation data blocks corresponding to the calculation task identifier; calculation data block identifier; calculation data sending sequence number; calculation data processing confirmation sequence number; calculation operation information; calculation data operation offset processing information.
11. A data transmission method, characterized in that: Applied to a third network node, the method comprises: Obtaining a second data packet sent by the first network node on the second transmission path; wherein the second data packet is generated according to a calculation result and the first data packet, and the calculation result is obtained by calculating the calculation data carried by the first data packet; The second data packet is sent to a second computing node; wherein the second computing node is located on a second transmission path, and the second data packet is transmitted to the second computing node through the second transmission path.
12. The data transmission method according to claim 11, characterized in that: The sending the second data packet to the second computing node includes: Get unblocking policy information; Decapsulating the second data packet according to the decapsulation policy information to generate a third data packet; The third data packet is sent to the second computing node.
13. The data transmission method according to claim 11, characterized in that: The second data packet includes path index information; The step of sending the second data packet to the second computing node includes: determining, according to the path index information, whether the third network node is a previous node of the second computing node on the second transmission path; In a case where it is determined according to the path index information that the third network node is a previous node of the second computing node on the second transmission path, the second data packet is sent to the second computing node.
14. A data transmission device, characterized in that: Applied to a first network node, the device includes: A first acquisition module, configured to acquire a first data packet sent by a first computing node on a first transmission path; wherein the first data packet includes a computing task identifier; an obtaining module, configured to calculate the calculation data carried by the first data packet according to the calculation task identifier to obtain a calculation result; A generating module, configured to generate a second data packet according to the calculation result and the first data packet; The first sending module is used to send the second data packet to the second computing node through a second transmission path.
15. A data transmission device, characterized in that: Applied to a second network node, the device comprises: A second acquisition module, used to acquire a service data packet sent by the first computing node; A second generating module, used to encapsulate the business data packet to generate a first data packet; wherein the first data packet includes a computing task identifier; The second sending module is used to send the first data packet to a first network node; wherein the first network node is located on a first transmission path, and the first data packet is transmitted to the first computing node through the first transmission path.
16. A data transmission device, characterized in that: Applied to a third network node, the device includes: A third acquisition module is used to acquire a second data packet sent by the first network node on the second transmission path; wherein the second data packet is generated according to the calculation result and the first data packet, and the calculation result is obtained by calculating the calculation data carried by the first data packet; The third sending module is used to send the second data packet to the second computing node; wherein the second computing node is located on a second transmission path, and the second data packet is transmitted to the second computing node through the second transmission path.
17. A network node, characterized in that: include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the program implements the data transmission method according to any one of claims 1 to 6, or the data transmission method according to any one of claims 7 to 10, or the data transmission method according to any one of claims 11 to 13.
18. A readable storage medium, characterized in that: The readable storage medium stores a program, and when the program is executed by the processor, it implements the data transmission method according to any one of claims 1 to 6, or implements the data transmission method according to any one of claims 7 to 10, or implements the data transmission method according to any one of claims 11 to 13.
Citation Information
Patent Citations
Method and apparatus for realizing task follow-up
CN107547471A
Data sending method, processing method and receiving method, equipment thereof and storage medium
CN112788074A
Method and apparatus for determining network node
CN113347208A
Task information processing method and device, equipment and storage medium
CN114399230A
Data processing method and device
CN114448889A
