Communication system, communication method and apparatus

Abstract

The application provides a communication system, a communication method and a device, and relates to the field of communication. The application is used for improving resource use efficiency in the process of performing a coordinated flow service. The communication system comprises a sending node, a first receiving node and a second receiving node. The sending node is used for sending a first sub-flow to the first receiving node, and the first sub-flow comprises a first priority identifier. The sending node is also used for sending a second sub-flow to the second receiving node, and the second sub-flow comprises a second priority identifier. The first sub-flow and the second sub-flow belong to the same coordinated flow. The first receiving node is used for preferentially processing the first sub-flow according to a first priority corresponding to the first priority identifier. The second receiving node is used for postponing processing of the second sub-flow according to a second priority corresponding to the second priority identifier.

Description

[0001] This application is a divisional application. The original application has the application number 202311863142.6 and the original application date is December 29, 2023. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of communications, and more particularly to a communication system, communication method and apparatus. Background Technology

[0003] A coflow is a set of data flows that have a cooperative relationship. Each independent data flow within a coflow is called a subflow of that coflow. The coflow completion time (CCT) depends on the latest completion time of each of its subflows.

[0004] Compared to the transmission of a single data stream, a co-stream is more complex and resource-intensive because it involves multiple cooperating sub-streams. Therefore, improving resource utilization efficiency is a key challenge when implementing co-stream services. Summary of the Invention

[0005] This application provides a communication system, communication method, and apparatus for improving resource utilization efficiency during the execution of cooperative streaming services.

[0006] A first aspect provides a communication system comprising a transmitting node, a first receiving node, and a second receiving node. The transmitting node is configured to transmit a first sub-stream to the first receiving node, the first sub-stream including a first priority identifier. The transmitting node is also configured to transmit a second sub-stream to the second receiving node, the second sub-stream including a second priority identifier. The first sub-stream and the second sub-stream belong to the same coflow. The first receiving node is configured to process the first sub-stream with priority based on the first priority identifier. The second receiving node is configured to process the second sub-stream with deferred priority based on the second priority identifier. The time difference between the first receiving node completing processing the first sub-stream and the second receiving node completing processing the second sub-stream satisfies at least one of the following: less than the time difference between the first receiving node and the second receiving node completing processing the first sub-stream according to the first priority, and less than the time difference between the first receiving node and the second receiving node completing processing the first sub-stream according to the second priority.

[0007] In the method described above in this application, when a sending node sends a sub-stream to each receiving node, it can carry a priority identifier in the sub-stream. Then, the receiving nodes can process the sub-stream according to the priority corresponding to the priority identifier. On one hand, some receiving nodes can process the sub-stream with higher priority (i.e., the first receiving node processes the sub-stream with higher priority according to the first priority identifier); on the other hand, other receiving nodes can postpone processing the sub-stream with lower priority (i.e., the second receiving node postpones processing the sub-stream according to the second priority identifier). This ensures that the time difference between the first receiving node completing processing the first sub-stream and the second receiving node completing processing the second sub-stream satisfies at least one of the following: less than the time difference between the first and second receiving nodes completing processing the first and second sub-streams according to the first priority, and less than the time difference between the first and second receiving nodes completing processing the first and second sub-streams according to the second priority. Therefore, compared to the first and second receiving nodes processing the sub-streams with the same priority, the method described above in this application can reduce the time difference between different receiving nodes completing processing the sub-streams, thereby improving the resource utilization efficiency of the receiving nodes.

[0008] In one implementation, the communication system described above is a distributed storage system, wherein the first receiving node and the second receiving node are storage nodes in the distributed storage system, respectively.

[0009] In the above implementation, it is considered that in a distributed storage system, a co-streaming approach can be used to store data across multiple storage nodes. For example, when storing data using a multi-replica method, a co-streaming approach can be used to send multiple data replicas to different storage nodes for storage. Similarly, when storing data using erasure codes (EC), a co-streaming approach can be used to send the data blocks and check blocks of the EC code block to different storage nodes for storage. Therefore, when the first receiving node and the second receiving node are storage nodes in the distributed storage system, the time difference between the first receiving node and the second receiving node when storing data to storage nodes including the first and second receiving nodes using a co-streaming approach can be reduced, thereby improving the resource utilization efficiency of the storage nodes.

[0010] In one implementation, the aforementioned distributed storage system includes a hard disk. The hard disk includes a network interface card (NIC), a processor, and a storage medium. The first receiving node and / or the second receiving node are hard disks in the distributed storage system. When the first receiving node is a first hard disk in the distributed storage system, the NIC in the first hard disk is used to receive a first sub-stream, the processor in the first hard disk is used to prioritize processing the first sub-stream according to a first priority identifier, and the storage medium in the first hard disk is used to store data in the first sub-stream. When the second receiving node is a second hard disk in the distributed storage system, the NIC in the second hard disk is used to receive a second sub-stream, the processor in the second hard disk is used to postpone processing the second sub-stream according to a second priority identifier, and the storage medium in the second hard disk is used to store data in the first sub-stream.

[0011] By implementing the above method, the distributed storage system can reduce the time difference between hard disks and other storage nodes when storing data to storage nodes, including hard disks, in a cooperative manner, thereby improving the resource utilization efficiency of hard disks and other storage nodes.

[0012] In one implementation, the communication system described above is a distributed storage system, where the first receiving node and the second receiving node are transmission nodes connected to the storage nodes in the distributed storage system.

[0013] In the above implementation, it is considered that when a storage node accesses the distributed storage system through a transmission node, after receiving the sub-stream sent to each storage node, each transmission node can process the sub-stream according to the priority corresponding to the priority identifier in the sub-stream (for example, send the sub-stream to the storage node according to the priority corresponding to the priority identifier in the sub-stream), thereby improving the utilization efficiency of the scheduling resources of each transmission node and each storage node.

[0014] In one implementation, the aforementioned sending node is further configured to determine a first priority identifier based on the historical flow completion time (FCT) of the n receiving nodes corresponding to the co-flow when they process the data flow from the sending node, where n is a positive integer greater than 1.

[0015] By implementing the above method, the priority identifier included in the sub-stream (specifically, the first priority identifier included in the first sub-stream) is determined based on the historical FCT (Functional Time Transmission) of the n receiving nodes corresponding to the co-stream K when processing the data stream from the sending node. This allows the completion time of the first receiving node processing the first sub-stream according to the priority corresponding to the first priority identifier to be close to the completion time of other receiving nodes among the n receiving nodes processing their sub-streams, thereby reducing the time difference in processing each sub-stream. This, in turn, improves the resource utilization efficiency of each receiving node.

[0016] In one implementation, the sending node is further configured to determine a first priority identifier based on the historical FCTs of the n receiving nodes corresponding to the co-flow when processing the data stream from the sending node, including: the sending node is further configured to determine a reference duration based on the historical FCTs of the n receiving nodes corresponding to the co-flow when processing the data stream from the sending node, and this reference duration is used to indicate the CCT of the co-flow. The sending node is further configured to determine the first FCT in the first FCT set that is less than the reference duration and closest to the reference duration. The first FCT set includes: the historical FCTs corresponding to the first receiving nodes when processing the data stream from the sending node according to different priorities. The sending node is further configured to determine the first priority identifier based on the priority corresponding to the first FCT.

[0017] In the above implementation, considering the following: Firstly, the historical FCTs of the n receiving nodes corresponding to the co-flow when processing data streams from the sending node can be used to predict the CCT of the co-flow, thereby determining a reference duration for indicating the CCT of the co-flow. Secondly, the FCTs of each receiving node processing sub-streams can be made to be less than and as close as possible to the reference duration. This reduces the time difference in processing time between sub-streams in co-flow K. Furthermore, in the above implementation, firstly, the reference duration is determined based on the historical FCTs of the n receiving nodes corresponding to the co-flow when processing data streams from the sending node; secondly, the first FCT, which is less than and closest to the reference duration, is selected from the first FCT set, and a first priority identifier is determined based on the priority corresponding to the first FCT. In this way, when the first receiving node processes the first sub-flow according to the first priority identifier, the FCT of the first sub-flow can be made as close as possible to the reference duration. When the FCTs of multiple sub-streams are all made as close as possible to the reference duration using the above method, the time difference in processing time between sub-streams can be reduced. This can improve the resource utilization efficiency of each receiving node.

[0018] Secondly, a communication method is provided, applied to a communication system including a sending node, a first receiving node, and a second receiving node. The method includes: the sending node sending a first sub-stream to the first receiving node, the first sub-stream including a first priority identifier; the sending node sending a second sub-stream to the second receiving node, the second sub-stream including a second priority identifier; the first sub-stream and the second sub-stream belonging to the same co-stream; the first receiving node processing the first sub-stream preferentially according to the first priority corresponding to the first priority identifier; and the second receiving node delaying the processing of the second sub-stream according to the second priority corresponding to the second priority identifier, such that the time difference between the first receiving node completing processing the first sub-stream and the second receiving node completing processing the second sub-stream satisfies at least one of the following: less than the time difference between the first receiving node and the second receiving node completing processing the first sub-stream according to the first priority, and less than the time difference between the first receiving node and the second receiving node completing processing the first sub-stream according to the second priority.

[0019] In one implementation, the method further includes: the sending node determines a first priority identifier based on the historical FCTs of the n receiving nodes corresponding to the co-flow when they process the data stream from the sending node, where n is a positive integer greater than 1.

[0020] In one implementation, the sending node determines a first priority identifier based on the historical FCTs (Final Cross-Checks) of the data streams from the sending node processed by the n receiving nodes corresponding to the co-flow. This includes: the sending node determining a reference duration based on the historical FCTs of the data streams from the sending node processed by the n receiving nodes corresponding to the co-flow; the reference duration is used to indicate the CCT (Combined Cross-Check) of the co-flow. The sending node determines the first FCT in the first FCT set that is less than and closest to the reference duration; wherein the first FCT set includes: the historical FCTs corresponding to the data streams from the sending node processed by the first receiving node according to different priorities. The sending node determines the first priority identifier based on the priority corresponding to the first FCT.

[0021] In one implementation, the communication system described above is a distributed storage system, where the first receiving node and the second receiving node are storage nodes within the distributed storage system.

[0022] In one implementation, the communication system described above is a distributed storage system, where the first receiving node and the second receiving node are transmission nodes connected to the storage nodes in the distributed storage system.

[0023] Thirdly, a communication system is provided, comprising a first receiving node and a second receiving node, wherein: the first receiving node is configured to receive a first sub-stream from a sending node, the first sub-stream including a first priority identifier; the second receiving node is configured to receive a second sub-stream from the sending node, the second sub-stream including a second priority identifier, the first sub-stream and the second sub-stream belonging to the same co-stream; the first receiving node is further configured to prioritize processing the first sub-stream according to the first priority corresponding to the first priority identifier; the second receiving node is further configured to postpone processing the second sub-stream according to the second priority corresponding to the second priority identifier, such that the time difference between the first receiving node completing processing the first sub-stream and the second receiving node completing processing the second sub-stream satisfies at least one of the following: less than the time difference between the first receiving node and the second receiving node completing processing the first sub-stream and the second sub-stream according to the first priority, and less than the time difference between the first receiving node and the second receiving node completing processing the first sub-stream and the second sub-stream according to the second priority.

[0024] In one implementation, the communication system described above is a distributed storage system, wherein the first receiving node and the second receiving node are storage nodes in the distributed storage system, respectively.

[0025] In one implementation, the aforementioned distributed storage system includes a hard disk. The hard disk includes a network interface card (NIC), a processor, and a storage medium. The first receiving node and / or the second receiving node are hard disks in the distributed storage system. When the first receiving node is a first hard disk in the distributed storage system, the NIC in the first hard disk is used to receive a first sub-stream, the processor in the first hard disk is used to prioritize processing the first sub-stream according to a first priority identifier, and the storage medium in the first hard disk is used to store data in the first sub-stream. When the second receiving node is a second hard disk in the distributed storage system, the NIC in the second hard disk is used to receive a second sub-stream, the processor in the second hard disk is used to postpone processing the second sub-stream according to a second priority identifier, and the storage medium in the second hard disk is used to store data in the first sub-stream.

[0026] In one implementation, the communication system described above is a distributed storage system, where the first receiving node and the second receiving node are transmission nodes connected to the storage nodes in the distributed storage system.

[0027] Fourthly, a communication method is provided, applied to a communication system including a first receiving node and a second receiving node. The method includes: the first receiving node receiving a first sub-stream from a sending node, the first sub-stream including a first priority identifier; and the second receiving node receiving a second sub-stream from the sending node, the second sub-stream including a second priority identifier, the first and second sub-streams belonging to the same co-stream. The first receiving node processes the first sub-stream preferentially according to the first priority corresponding to the first priority identifier. The second receiving node processes the second sub-stream later according to the second priority corresponding to the second priority identifier, such that the time difference between the first receiving node completing processing the first sub-stream and the second receiving node completing processing the second sub-stream satisfies at least one of the following: less than the time difference between the first receiving node and the second receiving node completing processing the first and second sub-streams according to the first priority, and less than the time difference between the first receiving node and the second receiving node completing processing the first and second sub-streams according to the second priority.

[0028] In one implementation, the communication system described above is a distributed storage system, wherein the first receiving node and the second receiving node are storage nodes in the distributed storage system, respectively.

[0029] In one implementation, the aforementioned distributed storage system includes a hard disk. The hard disk includes a network interface card (NIC), a processor, and a storage medium. The first receiving node and / or the second receiving node are hard disks in the distributed storage system. When the first receiving node is a first hard disk in the distributed storage system, the NIC in the first hard disk is used to receive a first sub-stream, the processor in the first hard disk is used to prioritize processing the first sub-stream according to a first priority identifier, and the storage medium in the first hard disk is used to store data in the first sub-stream. When the second receiving node is a second hard disk in the distributed storage system, the NIC in the second hard disk is used to receive a second sub-stream, the processor in the second hard disk is used to postpone processing the second sub-stream according to a second priority identifier, and the storage medium in the second hard disk is used to store data in the first sub-stream.

[0030] In one implementation, the communication system described above is a distributed storage system, where the first receiving node and the second receiving node are transmission nodes connected to the storage nodes in the distributed storage system.

[0031] Fifthly, a computer-readable storage medium is provided, wherein a computer program is stored therein, which, when executed by a transmitting node and a receiving node in a communication system, implements the method as described in the second aspect or any implementation thereof, or the fourth aspect or any implementation thereof.

[0032] Sixthly, a computer program product is provided, the computer program product including instructions that, when executed on a transmitting node and a receiving node in a communication system, implement a method as described in the second aspect or any implementation of the second aspect or the fourth aspect or any implementation of the fourth aspect. Attached Figure Description

[0033] Figure 1 This is one of the structural schematic diagrams of a communication system provided in an embodiment of this application.

[0034] Figure 2 This is one of the schematic diagrams illustrating the operation flow of a communication system provided in an embodiment of this application.

[0035] Figure 3 This is a second schematic diagram of the operation flow of a communication system provided in an embodiment of this application.

[0036] Figure 4 This is the third schematic diagram of the operation flow of a communication system provided in the embodiments of this application.

[0037] Figure 5 This is the fourth schematic diagram of the operation flow of a communication system provided in the embodiments of this application.

[0038] Figure 6 This is a second schematic diagram of a communication system provided in an embodiment of this application.

[0039] Figure 7 This is one of the structural diagrams of a distributed storage system provided in an embodiment of this application.

[0040] Figure 8 This is a second schematic diagram of a distributed storage system provided in an embodiment of this application.

[0041] Figure 9 This is the third schematic diagram of a distributed storage system provided in an embodiment of this application.

[0042] Figure 10 This is the fifth schematic diagram of the operation flow of a communication system provided in the embodiments of this application.

[0043] Figure 11 This is the sixth schematic diagram of the operation flow of a communication system provided in the embodiments of this application. Detailed Implementation

[0044] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0045] Currently, with the development of computer technology, many business operations are implemented using coflow. For example, in cloud computing, coflow can be used to achieve collaborative computing among multiple computing nodes. Specifically, in a cloud computing system, a sending node (e.g., the management node in the cloud computing system) sends sub-streams to multiple computing nodes. These computing nodes then process the sub-streams and send feedback messages to the sending node upon completion. After receiving feedback messages from each computing node, the sending node determines that the coflow is complete and releases memory resources. The coflow completion time (CCT) depends on the latest time among the multiple computing nodes to complete processing.

[0046] The following is based on Figure 1 Taking the communication system shown as an example, this paper introduces the implementation process of cooperative streaming in related technologies. Figure 1 As shown, the communication system 10 includes a transmitting node 110 and multiple receiving nodes (receiving node 121, receiving node 122 and receiving node 123 are examples in the figure).

[0047] When communication system 10 is applied in a cloud computing scenario, the sending node 110 can be a management node in the cloud computing system used to distribute computing tasks to various computing nodes, and each receiving node can be a computing node in the cloud computing system. The execution process of this co-stream is as follows: Figure 2 As shown, it includes: S201. The sending node 110 occupies memory resources and sends the sub-streams (i.e., sub-stream a, sub-stream b, and sub-stream c) to each receiving node respectively.

[0048] Each substream can consist of one or more data packets.

[0049] S202. Each receiving node processes the received sub-stream according to the priority in the scheduling policy.

[0050] Specifically, when a receiving node needs to process multiple tasks simultaneously, each receiving node can determine the priority of data packets in each task according to the corresponding scheduling strategy (e.g., strict priority (SP) scheduling strategy, weighted round-robin (WRR) scheduling strategy, or differential weighted round-robin (DWRR) scheduling strategy, and process the data packets according to the priority).

[0051] When each receiving node processes the received sub-stream according to the same priority in the scheduling strategy, for different sub-streams in the co-stream, due to factors such as the different load sizes of each receiving node, there may be a situation where some receiving nodes process the sub-stream earlier and others process the sub-stream later.

[0052] For example, in Figure 2 In the scenario shown, assume that receiving node 123 has the highest load, receiving node 122 has the lowest load, and receiving node 121's load is in between. Then, when the three receiving nodes process sub-streams a, b, and c according to the same priority in the scheduling policy, the processing order of each receiving node for the sub-streams is as follows: 1. Receiving node 121 processes the first two data packets (represented by shaded squares in the diagram), then processes sub-stream a; 2. Receiving node 122 can directly process sub-stream b without waiting; 3. Receiving node 123 processes the first four data packets (represented by shaded squares in the diagram), then processes sub-stream c.

[0053] S203. After completing data processing, each receiving node sends a feedback message to the sending node 110.

[0054] For example, after processing sub-stream a, receiving node 121 sends a feedback message to sending node 110; after processing sub-stream b, receiving node 122 sends a feedback message to sending node 110; and after processing sub-stream c, receiving node 123 sends a feedback message to sending node 110. Specifically, in Figure 2 In the example, receiving node 122 will complete data processing the earliest, and therefore receiving node 122 will send a feedback message to sending node 110 the earliest; receiving node 123 will complete data processing the latest, and therefore receiving node 123 will send a feedback message to sending node 110 the latest; the time when receiving node 121 sends a feedback message to sending node 110 is between the two.

[0055] S204. After receiving feedback messages from each receiving node, the sending node 110 determines that the co-flow execution is complete and releases memory resources.

[0056] It can be seen that, Figure 2 During the execution of the co-stream shown, there is a time difference in the completion time of each receiving node processing its sub-stream. In this case, there may be a waste of scheduling resources for the receiving node that completes data processing earlier.

[0057] Taking receiving node 122 as an example: Since sending node 110 only releases memory resources after determining that all receiving nodes have completed data processing (i.e., the CCT of the co-flow depends on the latest completion time of each sub-flow), the CCT of the co-flow will not decrease because receiving node 122 can process sub-flow b earlier. In other words, if receiving node 122 can process other service data packets first (e.g., receiving node 122 processes data packets of other services first...), the CCT of the co-flow will not decrease because receiving node 122 can process sub-flow b earlier. Figure 2 The data packets 1, 2, and 3 in the data packet are processed, and the sub-stream b is processed when other receiving nodes process the sub-stream (e.g., when receiving node 123 processes sub-stream c). In this way, the speed at which receiving node 122 processes other services can be improved without increasing the CCT of the co-flow, thereby improving the utilization efficiency of the scheduling resources in receiving node 122.

[0058] To address the aforementioned problems, this application provides a technical solution. In this solution, when executing a co-stream service, the sending node can carry a priority identifier in the sub-stream when sending it to each receiving node. Then, the receiving nodes can process the sub-stream according to the priority corresponding to the priority identifier. On one hand, some receiving nodes can process sub-streams with higher priorities (for example, receiving nodes with high loads can process sub-streams with higher priorities); on the other hand, other receiving nodes can postpone processing sub-streams with lower priorities (for example, receiving nodes with low loads can postpone processing sub-streams with lower priorities). This reduces the time difference between different receiving nodes completing sub-stream processing, thereby improving resource utilization efficiency.

[0059] For example, such as Figure 3 As shown in (a), when the co-stream service includes sub-stream a sent by sending node 110 to receiving node 121 and sub-stream b sent by sending node 110 to receiving node 122, as follows: Figure 3 As shown in (b), assume that the FCT when sending data from sending node 110 to receiving node 121 and receiving node 121 processes the data according to priority 1 in the scheduling policy is 10us (which can be understood as: the time from sending data from sending node 110 to receiving node completing data processing is 10us); the FCT when sending node 110 to receiving node 121 and receiving node 121 processes the data according to priority 2 in the scheduling policy is 16us; the FCT when sending node 110 to receiving node 122 and receiving node 122 processes the data according to priority 1 in the scheduling policy is 20us; and the FCT when sending node 110 to receiving node 121 and receiving node 122 processes the data according to priority 2 in the scheduling policy is 26us.

[0060] Therefore, in this embodiment, the sub-stream a sent by the sending node 110 to the receiving node 121 may include an identifier of priority 2, and the sub-stream b sent by the sending node 110 to the receiving node 122 may include an identifier of priority 1. Thus, after receiving sub-stream a, the receiving node 121 may prioritize processing sub-stream a according to priority 2; and after receiving sub-stream b, the receiving node 122 may postpone processing sub-stream b according to priority 1.

[0061] It can be seen that, compared to the time difference of 10us (i.e., 20us-10us) when both receiving nodes complete the processing of sub-streams a and b according to priority 1 and the time difference of 10us (i.e., 26us-16us) when both receiving nodes complete the processing of sub-streams a and b according to priority 2, the above process of the embodiments of this application can reduce the time difference of the two receiving nodes completing the processing of sub-streams to 4us (i.e., 20us-16us).

[0062] For example, such as Figure 4 As shown in (a), in a co-stream service, there are sub-stream c sent by sending node 110 to receiving node 121, sub-stream d sent by sending node 110 to receiving node 122, and sub-stream e sent by sending node 110 to receiving node 123. Assume that the FCT of each receiving node processing data according to different priorities in the scheduling policy is as follows: Figure 4 As shown in (b).

[0063] Therefore, in this embodiment, the sub-stream c sent by the sending node 110 to the receiving node 121 may include an identifier of priority 3; the sub-stream d sent by the sending node 110 to the receiving node 122 may include an identifier of priority 2; and the sub-stream e sent by the sending node 110 to the receiving node 123 may include an identifier of priority 1. Furthermore, after receiving sub-stream c, the receiving node 121 can process sub-stream c according to priority 3; after receiving sub-stream d, the receiving node 122 can process sub-stream d according to priority 2; and after receiving sub-stream e, the receiving node 123 can process sub-stream e according to priority 1.

[0064] and Figure 3 The example shown is similar, in Figure 4 In the example, compared to each receiving node processing the substream with the same priority, the above process of the embodiments of this application can reduce the time difference in processing substreams by different receiving nodes.

[0065] Based on the above, this application provides a communication system. The structure of this communication system can be referred to... Figure 1 The communication system 10 is shown. This communication system 10 may include a transmitting node 110 and multiple receiving nodes (such as...). Figure 1(Taking three receiving nodes as an example: receiving node 121, receiving node 122, and receiving node 123).

[0066] The sending node 110 can send the sub-streams in the co-stream to each receiving node in accordance with the methods provided below in the embodiments of this application. Each receiving node can process the received sub-streams in accordance with the methods provided below in the embodiments of this application.

[0067] In practical applications, the functions of the transmitting node 110 and each receiving node in the communication system 10 provided in this application embodiment can be implemented by electronic devices such as personal computers (including desktop computers, laptop computers, handheld computers, and notebook computers), ultra-mobile personal computers (UMPCs), network cards, or servers. Alternatively, the functions of the transmitting node 110 and each receiving node can be implemented by hardware devices (e.g., chips) or software devices (e.g., virtual machines) running in the aforementioned electronic devices.

[0068] Furthermore, the sending node 110 and each receiving node can be connected via a transmission network. The structure and type of the transmission network are not limited in this embodiment.

[0069] The following is a detailed description of the operation process of the communication system 10 performing cooperative streaming services in the embodiments of this application. Figure 5 As shown, the operation of the communication system 10 may include: S301, Sending node 110 obtains each sub-stream included in the co-stream K.

[0070] For example, sending node 110 can obtain data that requires collaborative processing by multiple receiving nodes from relevant applications in the application layer, and process the data to obtain the sub-streams in the co-stream K.

[0071] For example, sending node 110 can also receive multiple sub-streams included in the co-stream K from other nodes. For instance, when sending node 110 is a control node in a cloud computing system, sending node 110 can receive multiple sub-streams included in the co-stream K from clients that have established a connection with the cloud computing system.

[0072] S302, Sending node 110 sends a substream to multiple receiving nodes.

[0073] Each substream includes a priority identifier. This priority identifier is used to instruct the receiving node to process the substream according to the priority corresponding to the priority identifier.

[0074] For example Figure 3In the example shown, the substream a sent by the sending node 110 to the receiving node 121 includes an identifier of priority 2, which is used to instruct the receiving node 121 to process the substream according to priority 2; the substream b sent by the sending node 110 to the receiving node 122 includes an identifier of priority 1, which is used to instruct the receiving node 122 to process the substream according to priority 1.

[0075] For example Figure 4 In the example shown, the sub-stream c sent by the sending node 110 to the receiving node 121 includes an identifier of priority 3, the sub-stream d sent by the sending node 110 to the receiving node 122 includes an identifier of priority 2, and the sub-stream e sent by the sending node 110 to the receiving node 123 includes an identifier of priority 1.

[0076] Furthermore, the process by which the sending node 110 determines the priority identifiers included in each substream will be described in detail below through S601-S602. It will not be elaborated upon here.

[0077] S303. Each receiving node processes the substream according to the priority identifier included in the substream.

[0078] For example Figure 3 In the example shown, receiving node 121 processes substream a according to priority 2 (i.e., the priority corresponding to the priority identifier included in substream a); receiving node 122 processes substream b according to priority 1 (i.e., the priority corresponding to the priority identifier included in substream b).

[0079] For example Figure 4 In the example shown, receiving node 121 processes substream c according to priority 3 (i.e., the priority corresponding to the priority identifier included in substream c); receiving node 122 processes substream d according to priority 2 (i.e., the priority corresponding to the priority identifier included in substream d); and receiving node 123 processes substream e according to priority 1 (i.e., the priority corresponding to the priority identifier included in substream e).

[0080] In this embodiment of the application, depending on the scheduling strategy adopted by the receiving node, the priority corresponding to the priority identifier can be the priority in different scheduling strategies.

[0081] For example, when each receiving node uses the SP scheduling strategy to determine the order in which data is processed, the priority corresponding to the above priority identifier can be the priority in the SP scheduling strategy. For instance, Figure 3 Priority 1 and priority 2 in the example shown Figure 4 In the example shown, priority 1, priority 2, and priority 4 can each be a priority in the SP scheduling policy. Therefore, each receiving node can process the substream according to the priority identifier in the SP scheduling policy.

[0082] Similarly, when the receiving node adopts a WRR scheduling strategy or other scheduling strategies such as DWRR, the priority corresponding to the above priority identifier can be the priority in other scheduling strategies such as DWRR. In this embodiment, there are no restrictions on the scheduling strategy adopted by each receiving node.

[0083] In addition, in this embodiment of the application, processing a sub-stream can be understood as follows: the receiving node determines the order of processing the sub-stream according to the corresponding priority (i.e., the priority corresponding to the priority identifier included in the sub-stream received by the receiving node) and the scheduling strategy, and performs subsequent operations on the sub-stream according to the scheduling order (e.g., sending the data in the sub-stream to the application layer for calculation, storage and other operations).

[0084] In this embodiment of the application, the process of each receiving node processing a sub-stream may include at least two receiving nodes processing a sub-stream in accordance with the following processes S3031-S3032: a first receiving node processing a first sub-stream and a second receiving node processing a second sub-stream. S3031. The first receiving node processes the first sub-stream first according to the first priority corresponding to the first priority identifier (i.e., the priority identifier included in the first sub-stream).

[0085] S3032. The second receiving node processes the second sub-stream later according to the second priority corresponding to the second priority identifier (i.e., the priority identifier included in the second sub-stream).

[0086] The time difference T1 between the first receiving node completing the processing of the first sub-stream and the second receiving node completing the processing of the second sub-stream satisfies the target condition.

[0087] The target conditions include at least one of the following: less than the time difference T2 between the first receiving node and the second receiving node in processing the first sub-stream and the second sub-stream according to the first priority, and less than the time difference T3 between the first receiving node and the second receiving node in processing the first sub-stream and the second sub-stream according to the second priority.

[0088] For example, the first receiving node can be Figure 3 The receiving node 122 in the middle, the second receiving node can be Figure 3The receiving node is 121. Receiving node 122 processes sub-stream b (the first sub-stream) first according to priority 1; receiving node 121 processes sub-stream a (the second sub-stream) later according to priority 2. The time difference T1 between receiving node 122 completing the processing of sub-stream b and receiving node 121 completing the processing of sub-stream a is 4µs (i.e., 20µs - 16µs). This time difference T1 is less than the time difference T2 (20µs - 10µs = 10µs) between both receiving nodes completing the processing of sub-streams b and a according to priority 1, and the time difference T3 (26µs - 16µs = 10µs) between both receiving nodes completing the processing of sub-streams b and a according to priority 2.

[0089] For example, the first receiving node can be Figure 4 The receiving node 123 in the middle, the second receiving node can be Figure 4 The receiving node is 122. Receiving node 123 processes sub-stream e (the first sub-stream) first according to priority 1; receiving node 122 processes sub-stream d (the second sub-stream) later according to priority 2. The time difference between receiving node 123 completing the processing of sub-stream e and receiving node 122 completing the processing of sub-stream d is 4µs (30µs - 26µs). This time difference T1 is less than the time difference T2 (30µs - 20µs = 10µs) when both receiving nodes complete the processing of sub-streams e and d according to priority 1, and the time difference T3 (36µs - 26µs = 10µs) when both receiving nodes complete the processing of sub-streams e and d according to priority 2.

[0090] For example, the first receiving node can be Figure 4 The receiving node 122 in the middle, the second receiving node can be Figure 4 The receiving node is 121. Receiving node 122 processes sub-stream d (the first sub-stream) first according to priority 2; receiving node 121 processes sub-stream c (the second sub-stream) later according to priority 3. The time difference between receiving node 122 completing the processing of sub-stream d and receiving node 121 completing the processing of sub-stream c is 0µs (i.e., 26µs - 26µs). This time difference T1 is less than the time difference T2 (26µs - 16µs = 10µs) between both receiving nodes completing the processing of sub-streams d and c according to priority 2, and the time difference T3 (36µs - 26µs = 10µs) between both receiving nodes completing the processing of sub-streams d and c according to priority 2.

[0091] For example, the first receiving node can be Figure 4 The receiving node 123 in the middle, the second receiving node can be Figure 4The receiving node is 121. Receiving node 123 processes sub-stream e (the first sub-stream) first according to priority 1; receiving node 121 processes sub-stream c (the second sub-stream) later according to priority 3. The time difference between receiving node 121 completing the processing of sub-stream e and receiving node 121 completing the processing of sub-stream c is 4µs (30µs - 26µs). This time difference T1 is less than the time difference T2 (30µs - 10µs = 20µs) when both receiving nodes complete the processing of sub-streams e and c according to priority 1, and the time difference T3 (46µs - 26µs = 20µs) when both receiving nodes complete the processing of sub-streams e and c according to priority 2.

[0092] In one possible design, on the one hand, the time difference T2 between the first receiving node and the second receiving node in processing the first sub-stream and the second sub-stream according to the first priority can be: the time difference T2 between the first receiving node in processing the first sub-stream and the second receiving node in processing the first data.

[0093] The first data is the data carrying the second priority identifier received by the first receiving node before or after receiving the first sub-stream.

[0094] by Figure 3 Taking receiving node 122 as the first receiving node and receiving node 121 as the second receiving node as an example: the time difference T2 between receiving node 122 and receiving node 121 in processing sub-stream a and sub-stream b according to priority 1 can be specifically: the time difference between receiving node 122 in processing sub-stream a according to priority 1 and receiving node 121 in processing the first data (which can be the service data including priority 1 received by receiving node 121 before or after receiving sub-stream b) according to priority 1.

[0095] On the other hand, the time difference T3 between the first receiving node and the second receiving node in processing the first sub-stream and the second sub-stream according to the second priority can be specifically: the time difference T2 between the first receiving node in processing the second data and the second receiving node in processing the second sub-stream.

[0096] The second data refers to the data carrying the first priority identifier received by the second receiving node before or after receiving the second sub-stream.

[0097] Continue with Figure 3Taking receiving node 122 as the first receiving node and receiving node 121 as the second receiving node as an example: the time difference T2 between receiving node 122 and receiving node 121 in processing sub-stream a and sub-stream b according to priority 2 can be: the time difference between receiving node 122 in processing the second data (which can be the service data including priority 2 received by receiving node 122 before or after receiving sub-stream a) and receiving node 121 in processing sub-stream b according to priority 2.

[0098] In the above design, it is considered that in practical applications, when processing sub-streams, the receiving node only needs to process them according to one priority (i.e., the priority corresponding to the priority identifier carried in the sub-stream). For example, when the first receiving node processes the first sub-stream according to the first priority (i.e., S3031), the first receiving node does not need to process the first sub-stream according to the second priority. Similarly, when the second receiving node processes the second sub-stream according to the second priority (i.e., S3032), the second receiving node does not need to process the second sub-stream according to the first priority.

[0099] Therefore, when the first receiving node processes the first sub-stream according to the first priority, the time when the first receiving node completes processing other service data according to the second priority (for example, the other service data can be service data carrying the second priority identifier received by the first receiving node before or after receiving the first sub-stream) can be used to reflect the time when the first receiving node completes processing the first sub-stream according to the second priority.

[0100] by Figure 3 Taking receiving node 122 as the first receiving node and receiving node 121 as the second receiving node as an example: When receiving node 122 prioritizes processing sub-stream a according to priority 1, the time it takes for receiving node 122 to complete processing other service data according to priority 2 (this service data can be service data including priority 2 received before or after receiving sub-stream a) can be used to reflect the time it takes for receiving node 122 to complete processing sub-stream a according to priority 2.

[0101] Similarly, when the second receiving node processes the second sub-stream according to the second priority, the time it takes for the second receiving node to complete processing other service data according to the first priority (for example, the other service data can be service data carrying the first priority identifier received by the second receiving node before or after receiving the second sub-stream) can be used to reflect the time it takes for the second receiving node to complete processing the first sub-stream according to the first priority.

[0102] Continue with Figure 3Taking the example where receiving node 122 is the first receiving node and receiving node 121 is the second receiving node: When receiving node 121 processes sub-stream a according to priority 2, the time it takes for receiving node 121 to complete processing other service data according to priority 1 (this service data can be service data including priority 1 received before or after receiving sub-stream b) can be used to reflect the time it takes for receiving node 122 to complete processing sub-stream a according to priority 2.

[0103] After each receiving node has completed processing its respective substream, it may also include: S304. After each receiving node completes processing of its sub-stream, it sends a feedback message to the sending node 110.

[0104] For example Figure 3 In the example shown, after receiving nodes 121 and 122 have completed processing substreams a and b, they can send a feedback message to sending node 110 to indicate that the substreams have been processed. For example... Figure 4 In the example shown, after receiving nodes 121, 122, and 123 have completed processing sub-streams c, d, and e, they can send feedback messages to sending node 110 to indicate that the sub-streams have completed processing.

[0105] S305. After receiving the feedback messages from each receiving node, the sending node releases the resources corresponding to the co-current K.

[0106] For example, after receiving feedback messages from all receiving nodes, the sending node determines that the co-flow K has been completed and then releases the memory resources occupied by the co-flow K.

[0107] The above Figure 5 China mainly Figure 1 Taking the structure of the communication system 10 shown as an example, the operation process of the communication system provided in this application embodiment will be introduced. Figure 1 In the communication system 10 shown, the transmitting node 110 and multiple receiving nodes can each be components of the communication system 10. In another implementation, the communication system provided in this embodiment may not include the software / hardware device that performs the functions of the transmitting node 110. For example, Figure 6 The diagram shown is a structural schematic of another communication system provided in an embodiment of this application.

[0108] The communication system 40 includes multiple receiving nodes (receiving node 401, receiving node 402 and receiving node 403 are examples in the figure).

[0109] The functions of each receiving node can be implemented by electronic devices such as personal computers (including desktop computers, laptop computers, handheld computers, and notebook computers), ultra-mobile personal computers (UMPCs), network cards, or servers. Alternatively, the functions of each receiving node can be implemented by hardware devices (e.g., chips) and software devices (e.g., virtual machines) running in the aforementioned electronic devices.

[0110] Each receiving node can interact with the sending node 41 outside the communication system 40 via the transmission network. For example, the sending node 41 can be a client with access to the communication system 40, which can interact with each receiving node via the transmission network.

[0111] When the communication system 40 is used to perform a co-stream service, the sending node 41 can obtain each sub-stream in the co-stream by referring to the content of S301 above, and send the sub-stream including priority identifiers to each receiving node in the communication system 40 by referring to the content of S302 above. In addition, the sending node 110 can also determine the priority identifiers included in each sub-stream by referring to the content of S601-S602 below.

[0112] After receiving a substream from the sending node 41, each receiving node in the communication system 40 can process the substream according to the priority corresponding to the priority identifier included in the substream, referring to the content of S303 above. Each receiving node in the communication system 40 may include at least two receiving nodes, and the process of processing the substream conforms to the content of S3031 and S3032.

[0113] In addition, after processing the substream, each receiving node can send a feedback message to the sending node 41 as described in S304 above. After receiving the feedback message from the receiving node, the sending node 41 can release resources as described in S305 above.

[0114] In one implementation, it is considered that in a distributed storage system, a co-streaming approach can be used to store data across multiple storage nodes. For example, when storing data using a multi-replica approach, a co-streaming approach can be used to send multiple data replicas to different storage nodes for storage. As another example, when storing data using erasure codes (EC), a co-streaming approach can be used to send the data blocks and checksum blocks within the EC code block to different storage nodes for storage.

[0115] Therefore, the above text can be used as an example. Figure 5 The operation process of the communication system shown is applied to a distributed storage system, thereby improving resource utilization efficiency when storing data in a distributed storage system through a co-streaming method.

[0116] like Figure 7 The diagram shown is a schematic representation of a distributed storage system according to an embodiment of this application. The distributed storage system 50 includes: multiple storage nodes for storing data (four storage nodes are shown in the diagram: storage node 501, storage node 502, storage node 503, and storage node 504), and one or more computing nodes for providing computing power (one computing node is shown in the diagram: computing node 505). Each storage node and computing node can be connected via a transmission network 506. In practical applications, each storage node and computing node in the distributed storage system 50 can be a server, desktop computer, or other similar device.

[0117] Understandable, Figure 7 This application merely provides an exemplary system architecture for a distributed storage system. In specific implementations, the distributed storage system applying the technical solutions provided in this application may also employ other system architectures. For example, when a storage node in the distributed storage system is a device with both computing and storage capabilities, the storage node can implement the functions of the computing node 505; in this case, the distributed storage system may not include the computing node 505. Furthermore, when a storage node in the distributed storage system is a device with both computing and storage capabilities, the computing node 505 may also run on the storage node as a virtual machine; in this case, the distributed storage system may not include a computing node independent of the storage node. The technical solutions provided in this application can be applied to distributed storage systems with different system architectures.

[0118] Furthermore, when the distributed storage system 50 is running, on the one hand, the computing node 505 can send data generated during the computation process to the storage nodes for storage, and the computing node 505 can also read the data needed for computation from the storage nodes. On the other hand, each storage node can also receive access requests from devices outside the computing node and read and write data according to the access requests. For example, each storage node can receive access requests from other storage nodes in the distributed storage system and read and write data according to the access requests. As another example, each storage node can receive access requests from devices outside the distributed storage system and read and write data according to the access requests.

[0119] The following describes three implementation processes for the above text. Figure 5 The operation process of the communication system shown is described in the following section on its implementation in a distributed storage system: In the first implementation, when in the distributed storage system 50, the computing node 505 sends data to multiple storage nodes and the data is stored by the multiple storage nodes, the computing node 505 can be used to implement... Figure 5The function of the sending node 110 can be implemented by each storage node. Figure 5 The functions of each receiving node in the process.

[0120] Specifically, compute node 505 can refer to S301 above to obtain each sub-stream in the co-stream (for example, when the storage node uses a multi-replica method for data storage, each sub-stream can include replica data; or, for example, when the storage node uses EC method for data storage, each sub-stream can include partial data blocks or parity blocks) and send the sub-streams including priority identifiers to each storage node in the distributed storage system 50, referring to S302 above. Additionally, compute node 505 can also refer to S601-S602 below to determine the priority identifiers included in each sub-stream.

[0121] After receiving a substream from the computing node 505, each storage node in the distributed storage system 50 can process the substream according to the priority corresponding to the priority identifier included in the substream, referring to the content of S303 above. (Processing the substream may include: the storage node determining the order of the substreams according to the priority corresponding to the priority identifier included in the substream, and writing the data in the substream to the storage medium in that order). Each storage node in the distributed storage system 50 may include at least two storage nodes processing the substream in accordance with the content of S3031 and S3032. Furthermore, after completing the data writing, each storage node can send a feedback message to the computing node 505, referring to the content of S304 above. After receiving the feedback message from the storage node, the computing node 505 can release resources, referring to the content of S305 above.

[0122] In the second implementation, it is considered that in a distributed storage system, one storage node can send data to multiple other storage nodes, and the data can be stored by multiple storage nodes. For example, when a storage node has computing capabilities, it can send the data generated during computation to multiple storage nodes for storage; or, for another example, when a computing node needs to store data, it can first send the data to one storage node in the distributed storage system, and then that storage node can send the data to multiple storage nodes for storage.

[0123] Therefore, in the embodiments of this application, reference can be made to Figure 5 The execution process shown involves sending data from one storage node to multiple other storage nodes, where it is then stored.

[0124] Taking the example of storage node 501 sending data to storage nodes 502, 503, and 504, and the data being stored by storage nodes 502, 503, and 504: On one hand, storage node 501 can refer to the content of S301-S302 above to determine each sub-stream in the co-flow and send the sub-stream including priority identifiers to each storage node in the distributed storage system 50 (for example, when the storage node uses a multi-replica method for data storage, each sub-stream can include a priority identifier and replica data respectively; as another example, when the storage node uses an EC method for data storage, each sub-stream can include a priority identifier and data blocks or parity blocks respectively). On the other hand, storage node 501 can also refer to the content of S601-S602 below to determine the priority identifiers included in each sub-stream.

[0125] On the other hand, after receiving the substream from storage node 501, storage nodes 502, 503, and 504 can process the substream according to the priority corresponding to the priority identifier included in the substream, referring to the content of S303 above. (Processing the substream may include: the storage nodes determining the order of the substreams according to the priority corresponding to the priority identifier included in the substream, and writing the data in the substreams to the storage medium in that order.) The process of at least two storage nodes processing the substreams may conform to the content of S3031 and S3032. Furthermore, after completing the data writing, each storage node can send a feedback message to storage node 501, referring to the content of S304 above. After receiving the feedback message from the storage node, storage node 501 can release resources, referring to the content of S305 above.

[0126] In the third implementation, it is considered that data can be sent from a device outside the distributed storage system to multiple other storage nodes and stored by these nodes. Therefore, in the embodiments of this application, reference can be made to... Figure 5 The execution process shown involves sending data from a device outside the distributed storage system to multiple other storage nodes for storage. The specific process can be referenced from the process described above, where compute node 505 sends data to multiple storage nodes and the multiple storage nodes store the data, or where storage node 501 sends data to multiple storage nodes and the multiple storage nodes store the data. Repeated content will not be elaborated upon here.

[0127] Additionally, in one possible design, the functionality of some or all storage nodes in the distributed storage system of this application embodiment can be implemented by hard drives with data processing and communication capabilities. For example, such as... Figure 8 As shown, in the distributed storage system 50, storage node 502 and storage node 503 can be hard disk 502 and hard disk 503, respectively.

[0128] The hard disk 502 may include a network interface card 5021 for communicating with other nodes in the distributed storage system, a processor 5022 for data processing, and a storage medium 5023 for storing data. Similarly, the hard disk 503 may include a network interface card 5031 for communicating with other nodes in the distributed storage system, a processor 5032 for data processing, and a storage medium 5033 for storing data.

[0129] It should be noted that, in Figure 8 In the original design, hard disks 502 and 503 each include independent processors (i.e., processor 5022 and processor 5032). In practical applications, when the network interface cards (NICs 5021 and 5031) in hard disks 502 and 503 have built-in processors, data processing tasks can also be performed by NICs 5021 and 5031. In this case, processor 5022 can be integrated into NIC 5021, and processor 5032 can be integrated into NIC 5031.

[0130] When the above text Figure 5 When the operation process of the communication system shown is applied to a distributed storage system, hard disks 502 and 503 can be used to implement... Figure 5 The function of receiving nodes.

[0131] For example, in the first implementation described above, after receiving a substream from computing node 505, hard disks 502 and 503 can respectively process the substream according to the priority identifier included in the substream, referring to the content of S303 above. As another example, in the second implementation described above, after receiving a substream from storage node 501, hard disks 502 and 503 can respectively process the substream according to the priority identifier included in the substream, referring to the content of S303 above. As yet another example, in the third implementation described above, after receiving a substream from a device outside the distributed storage system 50, hard disks 502 and 503 can respectively process the substream according to the priority identifier included in the substream, referring to the content of S303 above.

[0132] On one hand, when the first receiving node in S3031 is a hard disk that serves as a storage node in a distributed storage system, for example, when the first receiving node is hard disk 502: The network interface card 5021 in the hard disk 502 is used to receive a first substream including a first priority identifier from the sending node (at this time, the sending node can specifically be the computing node 505, other storage nodes in the distributed storage system 50, or devices outside the distributed storage system 50).

[0133] Processor 5022 is used to process substreams according to the first priority identifier corresponding to the first priority. For example, processor 5023 is used to determine the processing order of the first substream according to the first priority, and write the data in the first substream into storage medium 5023 in that order.

[0134] Storage medium 5023 is used to store data in the first substream.

[0135] On the other hand, when the second receiving node in S3032 is a hard disk that serves as a storage node in a distributed storage system, for example, when the second receiving node is hard disk 503: The network interface card 5031 in the hard disk 503 is used to receive a second substream including a second priority identifier from the sending node (at this time, the sending node can specifically be the computing node 505, other storage nodes in the distributed storage system 50, or devices outside the distributed storage system 50).

[0136] Processor 5032 is used to process the sub-stream according to the second priority identifier corresponding to the second priority. For example, processor 5033 is used to determine the order of processing the second sub-stream according to the second priority, and write the data in the second sub-stream into storage medium 5033 in that order.

[0137] Storage medium 5033 is used to store data in the second substream.

[0138] Additionally, in one possible design, it is considered that when a storage node accesses the distributed storage system via a transmission node, for example... Figure 9 Storage nodes 501, 502, 503, and 504 can be connected to transmission nodes 5051, 5052, 5053, and 5054 respectively to access the distributed storage system 50. After receiving the sub-streams sent to each storage node, each transmission node can process the sub-streams according to the priority identifier in the sub-streams according to the content of S303 above, thereby improving the utilization efficiency of the scheduling resources of each transmission node and each storage node.

[0139] The following is based on Figure 9 Taking a distributed storage system 50 as an example, the specific process of a sending node (which can be a computing node 505, a storage node in the distributed storage system 50, or a device outside the distributed storage system 50) sending data to storage nodes 501, 502, 503, and 504 in the distributed storage system 50, and the data being stored by each storage node, is described below: First, the sending node can refer to the contents of S301 and S302 above to obtain each sub-stream in the co-stream and send the sub-stream including priority identifier to each storage node (i.e., storage node 501, storage node 502, storage node 503 and storage node 504).

[0140] Subsequently, when each substream is transmitted to its respective transmission node (i.e., transmission node 5051, transmission node 5052, transmission node 5053, and transmission node 5054), each transmission node can process the substream according to the priority corresponding to the priority identifier included in the substream, referring to the content of S303 above. Specifically, processing the substream may include: the transmission node determining the order in which the substreams are sent according to the priority corresponding to the priority identifier included in the substream, and sending the substreams to the corresponding storage nodes in that order.

[0141] Among them, transmission nodes 5051, 5052, 5053 and 5054 may include at least two transmission nodes whose processes for processing substreams conform to the contents of S3031 and S3032.

[0142] In practical applications, the transmission nodes connected to the storage nodes (e.g., transmission nodes 5051, 5052, 5053, and 5054) can specifically be: switches (e.g., top-of-rack (TOR) switches or spine switches), routers, or virtual switches, etc. The specific form of the transmission nodes is not limited in this embodiment.

[0143] The following examples illustrate the following principles. Figure 5 The process by which the transmitting node 110 determines the priority identifiers included in each substream during the operation of the communication system shown is described below: In this embodiment, it is considered that the FCT (hereinafter referred to as historical FCT) of each receiving node processing the data stream from sending node 110 before sending co-flow K can be used to predict the FCT of each receiving node when processing sub-streams in co-flow K. For example, if the FCT of receiving node 121 processing the data stream from sending node 110 according to a certain priority in the scheduling policy (taking priority 1 as an example) before receiving sub-streams in co-flow K is 10us, then if the load of receiving node 121 does not change drastically, the FCT of receiving node 121 processing sub-streams in co-flow K according to priority 1 will also be approximately 10us.

[0144] Furthermore, when determining the priority identifiers included in each sub-stream of the co-flow K, the priority identifiers included in each sub-stream can be determined based on the historical FCTs of the multiple receiving nodes (taking n receiving nodes as an example, where n is a positive integer greater than 1) processing the data stream from the sending node 110. This ensures that the completion time of each receiving node processing the sub-stream according to the priority corresponding to the priority identifier is close to the completion time of other receiving nodes among the n receiving nodes, thereby reducing the time difference in completing the processing of each sub-stream.

[0145] Specifically, such as Figure 10 As shown, the operation of the communication system 10 may include: S601, The sending node 110 obtains the FCT when each receiving node processes the data stream from the sending node 110.

[0146] In one possible design, the FCT of the receiving node in processing the data stream from the sending node 110 can be determined based on the time difference between the sending node 110 sending the data stream to the receiving node and the sending node 110 receiving a feedback message from the receiving node (which indicates that the data stream processing is complete).

[0147] like Figure 10 As shown, when sending node 110 sends data stream a to receiving node 121, it records a timestamp. Additionally, when sending node 110 receives a feedback message from receiving node 121, it records a timestamp. Then, receiving node 121 processes the FCT of data stream a from sending node 110, which can be done using... express.

[0148] Similarly, Figure 10 In the process, receiving node 122 processes the FCT of data stream b from sending node 110, which can be used... This indicates that the receiving node 122 processes the FCT of the data stream c from the sending node 110, which can be represented by... express.

[0149] In addition, the FCT obtained by the sending node 110 may specifically include: the FCT when the receiving node processes the data stream from the sending node 110 according to different priorities.

[0150] For example, when sending node 110 sends data stream a to receiving node 121, data stream a includes a priority identifier. Receiving node 121 can process data stream a according to the priority corresponding to this priority identifier (let's say priority 1) and send a feedback message after completing the processing of data stream a. In this way, This can represent the FCT when the receiving node 121 processes the data stream from the sending node 110 according to priority 1.

[0151] For example, the FCT obtained by the sending node 110 when each receiving node processes the data stream from the sending node 110 can be represented as shown in Table 1 below: Table 1

[0152] in, This indicates the FCT (Functional Time Transmission) when receiving node 121 processes the data stream from sending node 110 according to priority 1. This indicates the FCT when receiving node 121 processes the data stream from sending node 110 according to priority 2, and so on.

[0153] S602, The sending node 110 determines the priority identifier of each sub-stream based on the historical FCT of each receiving node when processing the data stream from the sending node 110.

[0154] The following describes the implementation process of S602 using the process by which the sending node 110 determines the priority identifier (hereinafter referred to as priority identifier p1) included in the substream (hereinafter referred to as substream f1) received by the receiving node 121 as an example. Specifically, S602 may include: S602a, Sending node 110 determines the priority identifier p1 included in the sub-stream f1 received by receiving node 121 based on the historical FCT of the n receiving nodes (where n is a positive integer greater than 1) that process the data stream from sending node 110.

[0155] The process of S602a will be described below through two implementation methods: In the first implementation, this application embodiment considers the following: On the one hand, the historical FCT of the data stream from the sending node 110 at each of the n receiving nodes corresponding to the co-flow can be used to predict the CCT of the co-flow, thereby determining a reference duration for indicating the CCT of the co-flow K. On the other hand, the FCT of each receiving node processing the sub-flow can be made to be less than the reference duration and as close as possible to the reference duration. This can reduce the time difference between the completion of processing of each sub-flow in the co-flow K. Therefore, S602a can be implemented through the following processes S602a1-S602a3: S602a1, The sending node 110 determines the reference duration T based on the historical FCT of the data stream from the sending node 110 when the n receiving nodes corresponding to the co-flow process the data stream.

[0156] The reference duration T is used to indicate the CCT of the co-current K.

[0157] In one possible design, the reference duration T can satisfy the following formula (1): Formula (1).

[0158] in, This represents the maximum historical FCT value when each receiving node processes the data stream from sending node 110 according to priority x. For example, referring to Table 1, The following formula (2) must be satisfied: Formula (2).

[0159] The following formula (3) must be satisfied: Formula (3).

[0160] Satisfy the following formula (4): Formula (4).

[0161] For example, assuming receiving nodes 121, 122, and 123, the historical FCTs of the three receiving nodes when processing the data stream from sending node 110 are shown in Table 2 below: Table 2

[0162] It can be seen that, among them, the maximum historical FCT value of the three receiving nodes when processing the data stream from sending node 110 according to priority 1 is 30us (i.e. The historical maximum FCT value when the three receiving nodes process the data stream from sending node 110 according to priority 2 is 36us (i.e. The historical maximum FCT value when the three receiving nodes process the data stream from sending node 110 according to priority 3 is 46us (i.e. Therefore, the reference duration T is 30us (i.e. ).

[0163] S602a2, The transmitting node 110 determines the first FCT in the FCT set (hereinafter referred to as the first FCT set for ease of description) that is less than the reference duration T and closest to the reference duration T.

[0164] The first FCT set includes: historical FCTs of the receiving node 121 when processing data streams from the sending node 110 according to different priorities.

[0165] For example, in Table 1, the historical FCT of receiving node 121 when processing the data stream from sending node 110 according to priority 1 is as follows: The historical FCT of receiving node 121 when processing the data stream from sending node 110 according to priority 2 is: And so on. Therefore, the first FCT set can include: those listed in Table 1. .

[0166] Furthermore, the first FCT can specifically be... The one that is smaller than the reference duration T and is closest to the reference duration T.

[0167] Continuing with the example in Table 2, the reference duration T is now 30µs, and the first FCT set includes: 10µs, 16µs, and 26µs. Therefore, the first FCT that is less than 30µs and closest to 30µs is 26µs.

[0168] S602a3, The sending node 110 determines the priority identifier p1 included in the sub-stream f1 received by the receiving node 121 in the co-stream K according to the priority corresponding to the first FCT.

[0169] For example, suppose the first FCT is in Table 1 If the priority of the first FCT is priority 1, then the priority identifier p1 can be determined to be the identifier corresponding to priority 1.

[0170] Continuing with the example shown in Table 2, the priority corresponding to the first FCT (i.e., 26us) is priority 3, so the priority identifier p1 is determined to be the identifier corresponding to priority 3.

[0171] After determining the priority identifiers included in each sub-flow by referring to the above S602a1-S602a3 process, the co-flow service can be executed by referring to the content of S301-S305.

[0172] In the second implementation, it is considered that in practical applications, the sending node 110 may not be able to fully obtain the historical FCT of the data streams from the sending node 110 processed by the n receiving nodes according to different priorities. For example, there may be some missing data in Table 1 or Table 2. In this case, the reference duration T calculated by the above formula (1) may not accurately indicate the CCT of the co-current. Furthermore, as Figure 11 As shown, S602a can be implemented through the following processes S602a4-S602a6: S602a4, The sending node 110 determines the reference duration based on the historical FCT of the data stream from the sending node 110 when the n receiving nodes corresponding to the co-stream process the data stream. .

[0173] Among them, reference duration CCT used to indicate co-current K.

[0174] Among them, reference duration The calculation process can refer to the process of calculating the reference duration T using formula (1) as described above. Missing data can be excluded from the calculation.

[0175] For example, taking n receiving nodes including receiving node 121, receiving node 122, and receiving node 123 as an example, the historical FCT of the three receiving nodes when processing the data stream from sending node 110 is shown in Table 3 below: Table 3

[0176] Specifically, the historical FCT of receiving node 121 when processing the data stream from sending node 110 according to priority 3, and the historical FCT of receiving node 123 when processing the data stream from sending node 110 according to priority 2 are missing. Therefore, the reference duration is calculated using other data. For example, the maximum historical FCT value when each receiving node processes the data stream from sending node 110 according to priority 1, as calculated by formula (2). At that time, among them If it is missing, then it can be used Sure Similarly, when calculating according to formula (3) Then you can use Sure Then, the reference duration is calculated using formula (1). At that time, the above-mentioned and Perform the calculation.

[0177] S602a5, Sending node 110 determines the FCT set (hereinafter referred to as the second FCT set for ease of description) and the reference duration. The closest second FCT.

[0178] The second FCT set includes historical FCTs used by the receiving node 121 to process data streams from the sending node 110 according to different priorities. For example, Table 3 shows that the second FCT set includes: Furthermore, the second FCT can specifically be... The one that is closest to the reference duration T.

[0179] Additionally, when the sending node 110 cannot obtain the historical FCT of the receiving node 121 when it processes the data stream from the sending node 110 according to different priorities, for example, the FCT obtained by the sending node 110 is shown in Table 4: Table 4

[0180] In this case, all entries corresponding to receiving node 121 are missing. The priority identifier of the preset priority can be used as priority identifier p1. For example, the middle priority among m priorities can be used. The corresponding preset priority identifier is used as priority identifier p1.

[0181] S602a6, The sending node 110 determines the priority identifier p1 included in the sub-stream f1 received by the receiving node 121 in the co-stream K according to one of the following three methods.

[0182] Method 1: When the second FCT is much smaller than the reference duration If the second FCT set does not include the FCT corresponding to the first target priority, then the priority identifier p1 is determined to be the priority identifier corresponding to the first target priority.

[0183] The first target priority is one level lower than the priority corresponding to the second FCT.

[0184] In other words, when the second FCT is much smaller than the reference duration Furthermore, if the FCT corresponding to the first target priority is missing in the second FCT set, then the priority of the receiving node 121 in processing sub-stream f1 needs to be reduced, and the priority identifier corresponding to the first target priority is used as the priority identifier p1.

[0185] In one implementation, the second FCT is much smaller than the reference duration. Specifically, it can include: the second FCT satisfies the following formula (5).

[0186] Formula (5).

[0187] in, Satisfy the following formula (6): Formula (6).

[0188] in, This represents the average historical FCT (Functional Time Transmission) of each receiving node when processing the data stream from sending node 110 according to priority x. For example, referring to Table 1, The following formula (7) must be satisfied: Formula (7).

[0189] The following formula (3) must be satisfied: Equation (8).

[0190] Satisfy the following formula (4): Formula (9).

[0191] This represents the weight of priority x in the scheduling policy. For example, when each receiving node processes each sub-stream in the co-stream according to the priorities in the SP scheduling policy, These represent the weights of priority 1, priority 2, ..., priority m in the SP scheduling policy.

[0192] Method 2: When the second FCT is greater than the reference duration If the second FCT set does not include the FCT corresponding to the second target priority, then the priority identifier p1 is determined to be the priority identifier corresponding to the second target priority.

[0193] The first target priority is one level higher than the priority corresponding to the second FCT.

[0194] In other words, when the second FCT is higher than the reference duration Furthermore, if the FCT corresponding to the second target priority is missing in the second FCT set, then the priority of the receiving node 121 in processing sub-stream f1 needs to be increased, and the priority identifier corresponding to the second target priority is used as the priority identifier p1.

[0195] Method 3: When the second FCT does not meet the conditions in Method 1 and Method 2, the priority identifier p1 included in the sub-stream f1 received by the receiving node 121 in the co-stream K is determined according to the priority corresponding to the second FCT.

[0196] In other words, when the second FCT does not meet the conditions in Method 1 and Method 2, the priority identifier corresponding to the priority of the second FCT can be used as the priority identifier p1.

[0197] After determining the priority identifiers included in each sub-flow by referring to the above S602a4-S602a6 process, the co-flow service can be executed by referring to the content of S301-S305.

[0198] In another possible design, after the sending node 110 determines the priority identifiers included in each sub-stream of co-flow K according to the above-described S601-S602, it can record the priority identifiers included in each sub-stream of co-flow K in a preset data set. This way, when sending other co-flows (which can be called co-flow K1) to the receiving nodes in co-flow K later, the priority identifiers included in each sub-stream of co-flow K in this preset data set can be used as the priority identifiers included in each sub-stream of co-flow K1. This avoids the need to determine the priority identifiers included in each sub-stream of co-flow K according to the above-described S601-S602 every time a co-flow is sent.

[0199] For example, the preset data set can be shown in Table 5 below: Table 5

[0200] Table 5 records the priority identifiers in the substreams corresponding to each receiving node when sending a costream to different receiving nodes. For example, when sending node 110 sends a costream to receiving nodes 121, 122, and 123, a lookup in the table shows that the substream corresponding to receiving node 121 includes priority identifier p1, the substream corresponding to receiving node 122 includes priority identifier p2, and the substream corresponding to receiving node 123 includes priority identifier p3. As another example, when sending node 110 sends a costream to receiving nodes 121 and 122, a lookup in the table shows that the substream corresponding to receiving node 121 includes priority identifier p4, and the substream corresponding to receiving node 122 includes priority identifier p5.

[0201] In addition, this application embodiment also provides a computer-readable storage medium storing instructions. When the instructions are executed at the sending node and receiving node of a communication system, the communication system can perform all or part of the methods provided in S301-S305 or S601-S602 above in this application embodiment.

[0202] This application also provides a computer program product containing instructions. When run on the transmitting and receiving nodes of a communication system, the communication system can perform all or part of the methods provided in S301-S305 or S601-S602 above in this application embodiment.

[0203] The functions, actions, operations, or steps in the above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented using software programs, they can be implemented, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or include one or more data storage devices such as servers and data centers that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0204] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data used can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. Additionally, in this embodiment, “at least one” means one or more, “more than one” means two or more, and other quantifiers are similarly used. “And / or” describes the relationship between related objects, indicating that three relationships can exist; for example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, for elements appearing in the singular forms "a", "an", and "the", unless the context explicitly specifies otherwise, it does not mean "one or only one", but rather "one or more one". For example, "a device" means one or more such devices. Moreover, "at least one of..." means one or any combination of the following associated objects, such as "at least one of A, B, and C" including A, B, C, AB, AC, BC, or ABC. In the textual description of this embodiment, the character " / " generally indicates an "or" relationship between the preceding and following associated objects; in the formula of this embodiment, the character " / " indicates a "division" relationship between the preceding and following associated objects.

Claims

1. A communication system, characterized in that, The communication system includes a transmitting node, a first receiving node, and a second receiving node, wherein: The sending node is used to send a first sub-stream to the first receiving node, the first sub-stream including a first priority identifier; The sending node is further configured to send a second sub-stream to the second receiving node, the second sub-stream including a second priority identifier, and the first sub-stream and the second sub-stream belonging to the same coflow. The first receiving node is configured to process the first sub-stream with priority based on the first priority corresponding to the first priority identifier; The second receiving node is configured to postpone processing the second sub-stream according to the second priority corresponding to the second priority identifier, such that the time difference between the first receiving node completing processing the first sub-stream and the second receiving node completing processing the second sub-stream satisfies at least one of the following: less than the time difference between the first receiving node and the second receiving node completing processing the first sub-stream and the second sub-stream according to the first priority, and less than the time difference between the first receiving node and the second receiving node completing processing the first sub-stream and the second sub-stream according to the second priority.

2. The communication system according to claim 1, characterized in that, The communication system is a distributed storage system, and the first receiving node and the second receiving node are storage nodes in the distributed storage system.

3. The communication system according to claim 2, characterized in that, The distributed storage system includes a hard disk; the hard disk includes a network interface card, a processor, and a storage medium; the first receiving node and / or the second receiving node are the hard disks in the distributed storage system; In the case where the first receiving node is the first hard disk in the distributed storage system, the network card in the first hard disk is used to receive the first sub-stream, the processor in the first hard disk is used to process the first sub-stream first according to the first priority identifier, and the storage medium in the first hard disk is used to store the data in the first sub-stream. In the case where the second receiving node is the second hard disk in the distributed storage system, the network card in the second hard disk is used to receive the second sub-stream, the processor in the second hard disk is used to postpone the processing of the second sub-stream according to the second priority identifier, and the storage medium in the second hard disk is used to store the data in the first sub-stream.

4. The communication system according to claim 2 or 3, characterized in that, The communication system is a distributed storage system, and the first receiving node and the second receiving node are respectively transmission nodes connected to the storage nodes in the distributed storage system.

5. The communication system according to any one of claims 1 to 4, characterized in that, The sending node is further configured to determine the first priority identifier based on the historical flow completion time (FCT) of the n receiving nodes corresponding to the co-flow when they process the data flow from the sending node, where n is a positive integer greater than 1.

6. The communication system according to claim 5, characterized in that, The sending node is further configured to determine the first priority identifier based on the historical flow completion time (FCT) of the n receiving nodes corresponding to the co-flow when processing the data flow from the sending node, including: The sending node is further configured to determine a reference duration based on the historical FCT of the data streams from the sending node when the n receiving nodes corresponding to the co-stream process the data streams from the sending node respectively; the reference duration is used to indicate the CCT of the co-stream; The sending node is further configured to determine the first FCT in the first FCT set that is less than the reference duration and closest to the reference duration; wherein, the first FCT set includes: the historical FCTs corresponding to the first receiving node when processing the data stream from the sending node according to different priorities; The transmitting node is further configured to determine the first priority identifier based on the priority corresponding to the first FCT.

7. A communication method, characterized in that, The communication method is applied to a communication system, the communication system including a transmitting node, a first receiving node, and a second receiving node, the method comprising: The sending node sends a first sub-stream to the first receiving node, the first sub-stream including a first priority identifier; The sending node sends a second sub-stream to the second receiving node. The second sub-stream includes a second priority identifier. The first sub-stream and the second sub-stream belong to the same Coflow. The first receiving node processes the first sub-stream first according to the first priority corresponding to the first priority identifier; The second receiving node processes the second sub-stream after the second priority is assigned to the second priority identifier, such that the time difference between the first receiving node completing the processing of the first sub-stream and the second receiving node completing the processing of the second sub-stream satisfies at least one of the following: less than the time difference between the first receiving node and the second receiving node completing the processing of the first sub-stream and the second sub-stream according to the first priority, and less than the time difference between the first receiving node and the second receiving node completing the processing of the first sub-stream and the second sub-stream according to the second priority.

8. The communication method according to claim 7, characterized in that, The method further includes: The sending node determines the first priority identifier based on the historical flow completion time (FCT) of the n receiving nodes corresponding to the co-flow when they process the data flow from the sending node, where n is a positive integer greater than 1.

9. The communication method according to claim 8, characterized in that, The sending node determines the first priority identifier based on the historical flow completion time (FCT) of the data streams from the sending node when the n receiving nodes corresponding to the co-flow process the data streams from the sending node, including: The sending node determines a reference duration based on the historical FCTs of the data streams from the sending node when the n receiving nodes corresponding to the co-stream process the data streams from the sending node; the reference duration is used to indicate the CCT of the co-stream. The sending node determines the first FCT in the first FCT set that is less than the reference duration and closest to the reference duration; wherein, the first FCT set includes: the historical FCTs corresponding to the first receiving node when processing the data stream from the sending node according to different priorities; The sending node determines the first priority identifier based on the priority corresponding to the first FCT.

10. The communication method according to any one of claims 7 to 9, characterized in that, The communication system is a distributed storage system, and the first receiving node and the second receiving node are storage nodes in the distributed storage system.

11. The communication method according to claim 10, characterized in that, The communication system is a distributed storage system, and the first receiving node and the second receiving node are respectively transmission nodes connected to the storage nodes in the distributed storage system.

12. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a transmitting node and a receiving node in the communication system, enables the communication system to perform the method as described in any one of claims 7 to 11.

13. A computer program product, characterized in that, The computer program product includes instructions that, when executed on a transmitting node and a receiving node in a communication system, enable the communication system to perform the method as described in any one of claims 7 to 11.

Images