Distributed communication method and device and electronic equipment

Abstract

The invention relates to the technical field of communication, and particularly provides a distributed communication method and device and electronic equipment. The method comprises the steps of determining a target task category to which a current task belongs; selecting a communication path output interface matched with the target task category from the communication path output interfaces; transmission rates of output interfaces of different communication paths are different; and transmitting the target data to the data receiving equipment through the communication path output interface matched with the target task category. Thus, according to the task category to which the current task belongs, the adaptive communication path output interface is allocated for the current task, and the resource utilization rate and the data transmission performance can be improved.

Description

Technical Field

[0001] This application relates to the field of communication technology, specifically to a method, apparatus, and electronic device for distributed communication. Background Technology

[0002] In distributed systems such as edge-distributed large-scale model inference systems, the collaborative efficiency among multiple computing nodes highly depends on the data communication performance between them. Under current technologies, a relatively single high-performance protocol is typically used for communication to complete various data exchange tasks. However, transmitting all types of data using the same high-performance protocol is costly. Summary of the Invention

[0003] In view of this, embodiments of this application provide a method, apparatus, and electronic device for distributed communication.

[0004] On one hand, this application provides a distributed communication method in its embodiments. This method is applied to computing nodes in an edge-distributed large-model inference system, where the computing nodes are deployed with multiple communication path output interfaces (such as data outflow ports of communication interfaces). The method includes: Determine the target task category to which the current task belongs; From the outgoing interfaces of each communication path, select the communication path outgoing interface that matches the target task category; the transmission rates of different communication path outgoing interfaces are different. The target data is transmitted to the data receiving device through the communication path interface that matches the target task category.

[0005] In one implementation, determining the target task category to which the current task belongs includes: For the current task, the target task category is determined based on one or more of the following: the amount of target data to be transmitted in the current task, the task priority of the current task, and the data transmission method of the current task. The data transmission method is either one-to-one transmission or one-to-many transmission.

[0006] In one implementation, the amount of target data to be transmitted differs for any first task and any second task of different categories; or, For any first task and any second task of different categories, the task priorities are different; or, The data transmission methods differ for any first task and any second task of different categories.

[0007] In one implementation, determining the target task category to which the current task belongs includes: Use any one of the following: If the amount of target data to be transmitted in the current task is lower than the first preset quantity threshold, and the priority of the target task is lower than the first priority threshold, then the target task category is determined to be the first task category. If the amount of target data is not less than the first preset quantity threshold and there are multiple data receiving devices, then the target task category is determined to be the second task category. If the amount of target data is not less than the first preset quantity threshold and the number of data receiving devices is one, then the target task category is determined to be the third task category. If the amount of target data is lower than the first preset quantity threshold and the priority of the target task is not lower than the first priority threshold, then the target task category is determined to be the fourth task category.

[0008] In one implementation, selecting the communication path outgoing interface that matches the target task category from each communication path outgoing interface includes: Obtain at least one communication path outgoing interface corresponding to the target task category; each task category has a corresponding relationship with at least one communication path outgoing interface; If there is only one communication path outgoing interface corresponding to the target task category, the communication path outgoing interface corresponding to the target task category will be used as the communication path outgoing interface matched with the target task category. When there are multiple communication path outgoing interfaces corresponding to the target task category, the communication path outgoing interface that matches the target task category is selected from the communication path outgoing interfaces corresponding to the target task category based on the interface priority of each communication path outgoing interface corresponding to the target task category.

[0009] In one implementation, based on the interface priority of each communication path outgoing interface corresponding to the target task category, communication path outgoing interfaces matching the target task category are selected from the communication path outgoing interfaces corresponding to the target task category, including: From the communication path outgoing interfaces corresponding to the target task category, select the communication path outgoing interface with the highest interface priority; Determine if the conditions for setting multiple interfaces are met; Under the condition of setting multiple interfaces, based on the interface priority of each communication path output interface corresponding to the target task category, multiple communication path output interfaces are selected from each communication path output interface corresponding to the target task category, and the selected multiple communication path output interfaces are used as the communication path output interfaces matching the target task category. If the conditions for setting multiple interfaces are not met, the communication path with the highest priority will be used as the communication path that matches the target task category.

[0010] In one implementation, determining whether the set multi-interface conditions are met includes: The condition for setting multiple interfaces is determined to be met if at least one of the following conditions is met: The amount of target data exceeds the second preset threshold. The target task has a priority higher than the second priority threshold; The current load of the communication method corresponding to the highest priority communication path of the interface is higher than the preset load threshold. The estimated latency of the communication method corresponding to the highest priority communication path of the interface is higher than the preset latency.

[0011] In one implementation, before selecting the communication path outgoing interface that matches the target task category from each communication path outgoing interface, the method further includes: For each task category, perform the following steps: Based on the task characteristics corresponding to the task category, at least one communication path output interface is selected from the first output interface, the second output interface, the third output interface, and the fourth output interface as the communication path output interface corresponding to the task category; wherein, each communication path output interface is divided according to the differences in transmission rate and transmission method; the task characteristics include one or more of the following: data volume range, priority range, and transmission method; If a task category corresponds to multiple communication path outgoing interfaces, then the interface priority of each communication path outgoing interface corresponding to the task category is set according to the task characteristics.

[0012] In one embodiment, the transmission rate of the first output interface is within a first rate range and supports communication with multiple data receiving devices. The transmission rate of the second output interface is within the second rate range and supports communication with multiple data receiving devices; The transmission rate of the third output interface is within the third rate range, and it only supports communication with one data receiving device. The transmission rate of the fourth output interface is within the fourth rate range and supports communication with multiple data receiving devices; Among them, the upper limits of the second and third rate ranges are both higher than the upper limit of the fourth rate range; the upper limit of the fourth rate range is higher than the upper limit of the first rate range.

[0013] In one implementation, before determining the target task category to which the current task belongs, the method further includes: If there are multiple communication tasks to be executed, the order of the communication tasks is adjusted according to their priority so that the current task can be selected according to the order of the communication tasks.

[0014] On one hand, this application provides a distributed communication device, which is applied to a computing node in an edge-distributed large-model inference system. The computing node is deployed with multiple communication path output interfaces, including: The determining unit is used to determine the target task category to which the current task belongs. The selection unit is used to select the communication path output interface that matches the target task category from each communication path output interface; the transmission rates of different communication path output interfaces are different. The transmission unit is used to transmit target data to the data receiving device through the communication path output interface that matches the target task category.

[0015] In one implementation, the determining unit is used to: For the current task, the target task category is determined based on one or more of the following: the amount of target data to be transmitted in the current task, the task priority of the current task, and the data transmission method of the current task. The data transmission method is either one-to-one transmission or one-to-many transmission.

[0016] In one implementation, the amount of target data to be transmitted differs for any first task and any second task of different categories; or, For any first task and any second task of different categories, the task priorities are different; or, The data transmission methods differ for any first task and any second task of different categories.

[0017] In one implementation, the determining unit is used to: Use any one of the following: If the amount of target data to be transmitted in the current task is lower than the first preset quantity threshold, and the priority of the target task is lower than the first priority threshold, then the target task category is determined to be the first task category. If the amount of target data is not less than the first preset quantity threshold and there are multiple data receiving devices, then the target task category is determined to be the second task category. If the amount of target data is not less than the first preset quantity threshold and the number of data receiving devices is one, then the target task category is determined to be the third task category. If the amount of target data is lower than the first preset quantity threshold and the priority of the target task is not lower than the first priority threshold, then the target task category is determined to be the fourth task category.

[0018] In one implementation, the selection unit is used for: Obtain at least one communication path outgoing interface corresponding to the target task category; each task category has a corresponding relationship with at least one communication path outgoing interface; If there is only one communication path outgoing interface corresponding to the target task category, the communication path outgoing interface corresponding to the target task category will be used as the communication path outgoing interface matched with the target task category. When there are multiple communication path outgoing interfaces corresponding to the target task category, the communication path outgoing interface that matches the target task category is selected from the communication path outgoing interfaces corresponding to the target task category based on the interface priority of each communication path outgoing interface corresponding to the target task category.

[0019] In one implementation, the selection unit is used for: From the communication path outgoing interfaces corresponding to the target task category, select the communication path outgoing interface with the highest interface priority; Determine if the conditions for setting multiple interfaces are met; Under the condition of setting multiple interfaces, based on the interface priority of each communication path output interface corresponding to the target task category, multiple communication path output interfaces are selected from each communication path output interface corresponding to the target task category, and the selected multiple communication path output interfaces are used as the communication path output interfaces matching the target task category. If the conditions for setting multiple interfaces are not met, the communication path with the highest priority will be used as the communication path that matches the target task category.

[0020] In one implementation, the selection unit is used for: The condition for setting multiple interfaces is determined to be met if at least one of the following conditions is met: The amount of target data exceeds the second preset threshold. The target task has a priority higher than the second priority threshold; The current load of the communication method corresponding to the highest priority communication path of the interface is higher than the preset load threshold. The estimated latency of the communication method corresponding to the highest priority communication path of the interface is higher than the preset latency.

[0021] In one embodiment, the selection unit is further configured to: For each task category, perform the following steps: Based on the task characteristics corresponding to the task category, at least one communication path output interface is selected from the first output interface, the second output interface, the third output interface, and the fourth output interface as the communication path output interface corresponding to the task category; wherein, each communication path output interface is divided according to the differences in transmission rate and transmission method; the task characteristics include one or more of the following: data volume range, priority range, and transmission method; If a task category corresponds to multiple communication path outgoing interfaces, then the interface priority of each communication path outgoing interface corresponding to the task category is set according to the task characteristics.

[0022] In one embodiment, the transmission rate of the first output interface is within a first rate range and supports communication with multiple data receiving devices. The transmission rate of the second output interface is within the second rate range and supports communication with multiple data receiving devices; The transmission rate of the third output interface is within the third rate range, and it only supports communication with one data receiving device. The transmission rate of the fourth output interface is within the fourth rate range and supports communication with multiple data receiving devices; Among them, the upper limits of the second and third rate ranges are both higher than the upper limit of the fourth rate range; the upper limit of the fourth rate range is higher than the upper limit of the first rate range.

[0023] In one embodiment, the determining unit is further configured to: If there are multiple communication tasks to be executed, the order of the communication tasks is adjusted according to their priority so that the current task can be selected according to the order of the communication tasks.

[0024] On one hand, this application provides a distributed communication system, including an edge-distributed large-model inference system and at least one client. The edge-distributed large-model inference system includes at least one computing node, and the computing node is deployed with multiple communication path output interfaces. The compute node is used to: determine the target task category to which the current task belongs; select the communication path output interface that matches the target task category from the various communication path output interfaces; different communication path output interfaces have different transmission rates; and transmit target data to the data receiving device through the communication path output interface that matches the target task category; the data receiving device can be any client or another compute node different from this compute node. Each client is used to: receive target data sent by the computing node with which it communicates.

[0025] On the one hand, this application provides a distributed communication system, including multiple communicable computing nodes, wherein any computing node is deployed with multiple communication path output interfaces; The compute node is used to: determine the target task category to which the current task belongs; select the communication path output interface that matches the target task category from the various communication path output interfaces; different communication path output interfaces have different transmission rates; and transmit data or commands to another compute node through the communication path output interface that matches the target task category.

[0026] On one hand, this application provides an electronic device, including: Processor; and The memory stores computer instructions that cause the processor to perform the steps of the methods provided in the various alternative implementations of any of the distributed communication methods described above.

[0027] On one hand, this application provides a communication device deployed within a computing node, the communication device including a processor coupled to a memory; Memory, used to store instructions; A processor is used to execute instructions in memory to cause the communication device to perform the steps of the methods provided in the various alternative implementations of any of the above-described distributed communication methods.

[0028] On one hand, this application provides a computer-readable storage medium storing computer instructions for causing a computer to perform the steps of the methods provided in various alternative implementations of any of the above-described distributed communication methods.

[0029] On one hand, this application provides a computer program product including computer-readable code or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device performs the steps of the method provided in any of the above-described alternative implementations of distributed communication.

[0030] The distributed communication method in this application includes determining the target task category to which the current task belongs; selecting a communication path output interface matching the target task category from various communication path output interfaces; different communication path output interfaces have different transmission rates; and transmitting target data to a data receiving device through the communication path output interface matching the target task category. In this way, allocating an appropriate communication path output interface to the current task based on its task category can improve resource utilization and data transmission performance. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of a distributed communication scenario in an embodiment of this application.

[0032] Figure 2 This is a flowchart of a distributed communication method according to an embodiment of this application.

[0033] Figure 3 This is a schematic diagram of a large model inference process in an embodiment of this application.

[0034] Figure 4 This is an example diagram of a node process in an embodiment of this application.

[0035] Figure 5 This is a schematic diagram of an interface communication embodiment of this application.

[0036] Figure 6 This is a structural block diagram of a distributed communication device according to an embodiment of this application.

[0037] Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0038] The technical solution of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Furthermore, the technical features involved in the different embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0039] The following explains some terms used in the embodiments of this application.

[0040] End-side: refers to the terminal devices that directly generate or process data, usually located at the very edge of the network (closest to the data source). Examples include network cameras (Internet Protocol Cameras, IPCs), smartphones, and IoT sensors.

[0041] Edge computing refers to the intermediate computing layer located near the endpoint, responsible for aggregating and processing data from the endpoint and acting as a relay connection to the cloud. Examples include network video recorders (NVRs), edge servers, and gateway devices.

[0042] End and side: End side and side.

[0043] Distributed: refers to a system architecture design paradigm that distributes different functional parts of a system or service across multiple physical or logical nodes, with each node collaborating through a communication network to jointly complete a complex task.

[0044] Large-scale model inference system: refers to a hardware and software integrated architecture specifically designed for deploying and running large-scale pre-trained models (e.g., Generative Pre-trained Transformer (GPT), Large Language Model Meta AI (LLaMA), and Multi-modal Model, etc.). Its design goal is to perform model inference tasks (i.e., inputting data into the model and generating prediction results) efficiently and stably.

[0045] In distributed computing systems, the efficiency of collaborative work among multiple computing nodes highly depends on the performance of data communication between nodes. Under relevant technologies, a relatively simple communication mechanism is typically used to accomplish all types of data exchange tasks.

[0046] However, under the relevant technologies, the single communication path leads to performance limitations and a lack of differentiated transmission strategies, resulting in low resource utilization.

[0047] For example, with the widespread application of large-scale pre-trained models, their enormous number of parameters and computational demands have far exceeded the capacity of a single computing device. Therefore, adopting a distributed computing architecture, which breaks down model computation tasks and deploys them across multiple computing nodes to complete collaboratively, has become a key technology supporting large-scale model inference and training.

[0048] In distributed large-scale model inference systems, the communication efficiency between computing nodes is one of the core bottlenecks determining the overall system performance. Related technologies are mostly based on general-purpose network communication frameworks.

[0049] However, it still has significant limitations: First, its communication path highly relies on a single network communication protocol, and the transmission rate is limited by network bandwidth and protocol overhead. When faced with the high-frequency, high-bandwidth internal data exchange requirements generated by tensor parallelism and pipeline parallelism, communication speed becomes the main performance bottleneck. Second, the general network protocol fails to differentiate and optimize for the different characteristics (such as data volume, priority, and transmission method) of various heterogeneous communication tasks between computing nodes (such as control commands, large batches of tensor data, and real-time pipeline data), and thus cannot achieve optimal allocation of communication resources.

[0050] Furthermore, in edge-distributed scenarios, more stringent requirements are placed on system cost, power consumption, and real-time performance. Under current technologies, it is difficult to simultaneously meet the diverse needs of edge-side inference tasks for high bandwidth, low latency, and high reliability while maintaining low cost.

[0051] Based on the deficiencies of the aforementioned related technologies, this application provides a method, apparatus, and electronic device for distributed communication, aiming to reduce transmission costs while maintaining communication speed.

[0052] This application provides a distributed communication method that can be applied to any computing node in a distributed system, enabling more flexible data or command communication between computing nodes, or data communication between computing nodes and clients in an edge-side distributed large model inference system. This application uses an edge-side distributed large model inference system as an example for illustration; in practical applications, the distributed system can also be other types of communication systems, and this is not a limitation. The edge-side distributed large model inference system can also be called a large model inference system, inference service system, distributed model inference cluster, or edge-side inference service engine, referring to a backend software system or hardware cluster specifically responsible for receiving inference requests, performing large model calculations, and returning results.

[0053] In this application embodiment, a communication method for a high-bandwidth, low-cost edge-side distributed large-model inference system is proposed to achieve the following objectives: In one possible application scenario, data communication between the client and the inference system involves prompts and inference results. This communication interaction is infrequent and involves few bytes, making it suitable for low-speed (e.g., >0.01GB / s) one-to-many communication. For example, a computing node can communicate with multiple clients via 100Mbps networks.

[0054] In another optional application scenario, there will be uncertain data communication between multiple computing nodes during tensor parallelism. This communication interaction frequency is high and the number of bytes is large. It is suitable to be mainly handled by high-speed (e.g., >1GB / s) one-to-many communication. For example, one computing node can communicate with multiple computing nodes via high-speed Peripheral Component Interconnect Express (PCIe) 3.0 (with switch).

[0055] In another optional application scenario, there will be fixed unidirectional data communication between computing nodes during pipelined parallelism. This communication interaction frequency is high and the number of bytes is large, which is suitable for high-speed (e.g., >2GB / s) unidirectional one-to-one communication to be mainly responsible. For example, one computing node can communicate with another computing node through unidirectional Mobile Industry Processor Interface (MIPI).

[0056] In another possible application scenario, there will be command communication between multiple computing nodes during the inference process. This communication interaction is frequent and involves few bytes, and is suitable to be mainly handled by medium-speed (e.g., >0.1GB / s) one-to-many communication. For example, one computing node can communicate with multiple computing nodes via gigabit network; that is, each computing node simultaneously supports a low-speed one-to-many communication protocol, a high-speed one-to-many communication protocol, a high-speed one-to-one communication protocol, and a medium-speed one-to-many communication protocol.

[0057] In this embodiment of the application, a communication method for a distributed large-model inference system on the edge side is proposed. This communication method for a distributed large-model inference system can effectively ensure the efficiency of the system. The main ideas include: First, in the embodiments of this application, based on the communication characteristics of the edge-to-edge large model inference system, and according to the advantages and disadvantages of different communication protocols, a communication scheme for a high-bandwidth, low-cost edge-to-edge distributed large model inference system is formed by combining low-speed one-to-many communication, medium-speed one-to-many communication, high-speed one-to-many communication, and high-speed one-to-one unidirectional communication.

[0058] Second, in the embodiments of this application, a high-speed one-to-one communication protocol such as MIPI is introduced as a communication scheme for transmitting data between computing nodes in the large model pipeline working mode.

[0059] Third, in the embodiments of this application, the communication interface of the computing node can be understood as the software concept of "communication interface". When a computing node needs to send data or commands to other computing nodes, it calls this communication interface. The communication interface selects the best communication method for each communication task according to the communication task queue and communication load, so as to decouple the computing logic and the communication logic.

[0060] The distributed system may include multiple computing nodes. These computing nodes can be any type of electronic device, such as servers, which will not be elaborated further in this application.

[0061] In this embodiment of the application, a software communication interface module is set in the computing node of the distributed system, and communication path output interfaces corresponding to different task categories are pre-deployed. During communication, the software communication interface module can reasonably allocate communication path output interfaces for communication tasks and send data through the allocated communication path output interfaces.

[0062] In one embodiment, a distributed communication system is provided, including an edge-distributed large model inference system and at least one client. The edge-distributed large model inference system includes at least one computing node, and the computing node is deployed with multiple communication path output interfaces.

[0063] The computing node is used to: determine the target task category to which the current task belongs; select the communication path output interface that matches the target task category from the various communication path output interfaces; different communication path output interfaces have different transmission rates; transmit target data to the data receiving device through the communication path output interface that matches the target task category; the data receiving device can be any client or another computing node different from the computing node; each client is used to: receive the target data sent by the computing node it communicates with.

[0064] In another embodiment, a distributed communication system is provided, including multiple communicable computing nodes, wherein any one computing node is deployed with multiple communication path output interfaces. The compute node is used to: determine the target task category to which the current task belongs; select the communication path output interface that matches the target task category from the various communication path output interfaces; different communication path output interfaces have different transmission rates; and transmit data or commands to another compute node through the communication path output interface that matches the target task category.

[0065] The following is combined with Figure 1 The application scenarios of the above distributed communication system are illustrated with examples. (See also...) Figure 1 The diagram shown is a schematic of a distributed communication scenario. Figure 1 The system includes client-side and edge-distributed large-scale model inference systems.

[0066] The client is responsible for sending the prompts entered by the user to the edge-distributed large model inference system. The edge-distributed large model inference system runs large models such as DeepSeek Large Model (DeepSeek) or Qwen Large Model (Qwen) to generate answers to the user's questions and sends the answers to the client.

[0067] Among them, edge-distributed large model inference systems typically involve multiple computing nodes working together to perform large model inference. These computing nodes can be graphics processing units (GPUs), neural processing units (NPUs), or field-programmable gate arrays (FPGAs), etc.

[0068] See Figure 2 The diagram shown is a flowchart of a distributed communication method according to an embodiment of this application. The following is a summary of the process. Figure 2 This method is described below. It is applied to computing nodes in an edge-distributed large-model inference system. The computing nodes are deployed with multiple communication path outgoing interfaces. The specific implementation process of this method is as follows: Step 201: Determine the target task category to which the current task belongs.

[0069] In one implementation, the target task category to which the current task belongs is determined based on task-related information of the current task. The task-related information may include at least one of the following: the target data to be transmitted by the current task, the data receiving device to which the target data is to be transmitted, the amount of target data, and the target task priority of the current task.

[0070] In other words, for the current task, the target task category is determined based on one or more of the following: the amount of target data to be transmitted by the current task, the task priority of the current task, and the data transmission method of the current task.

[0071] The task priority can be pre-set for the communication task, i.e., establishing a correspondence between the communication task and the task priority, or it can be determined in real time by the upstream module that generates the task based on at least one of the following: data type, delay time, user level, and event level.

[0072] For example, from a data type perspective, control signaling (such as system synchronization heartbeats) has a higher priority than data synchronization tasks. From a latency perspective, tasks with low latency requirements have a higher priority than those with high latency requirements. From a user level perspective, requests from administrators have a higher priority than those from ordinary users. From an event level perspective, tasks with high risk levels have a higher priority than those with low risk levels.

[0073] Optionally, if the communication task also has a deadline, the task priority can be determined based on the deadline.

[0074] It should be noted that the amount of target data to be transmitted may differ for any first task and any second task of different categories; or, the task priorities may differ for any first task and any second task of different categories; or, the data transmission methods may differ for any first task and any second task of different categories.

[0075] In other words, the amount of data to be transmitted varies under different task categories, and / or the task priority of the data to be transmitted varies under different task categories, and / or the data transmission method varies under different task categories; the data transmission method under any task category is either one-to-one transmission or one-to-many transmission.

[0076] The data transmission method is either one-to-one or one-to-many. The number of data receiving devices indicates the data transmission method for the current task.

[0077] In one implementation, when determining the target task category to which the current task belongs, any one of the following items can be adopted: If the data volume of the target data to be transmitted in the current task is lower than the first preset quantity threshold, and the target task priority is lower than the first priority threshold, then determine that the target task category is the first task category; If the data volume of the target data is not lower than the first preset quantity threshold, and the number of data receiving devices is multiple, then determine that the target task category is the second task category; If the data volume of the target data is not lower than the first preset quantity threshold, and the number of data receiving devices is one, then determine that the target task category is the third task category; If the data volume of the target data is lower than the first preset quantity threshold, and the target task priority is not lower than the first priority threshold, then determine that the target task category is the fourth task category.

[0078] In practical applications, both the first priority threshold and the first preset quantity threshold can be set according to the actual application scenario, and no limitation is made here.

[0079] In the embodiments of this application, only the communication task classification in the large model inference process of the edge-side distributed large model inference system is used as an example for illustration. Refer to Figure 3 As shown, it is a schematic diagram of a large model inference process. Figure 3 It is the large model inference process of Deepseek or Qwen waiting for the Decoder-Only Transformer Architecture. This process includes: Tokenizer, word embedding, position encoding, attention calculation, feedforward neural network...

[0080] Among them, the Tokenizer is the entrance of the model. It splits the original text input by the user (such as "How are you") into basic units (called "tokens") that the model can understand, and converts them into digital identifiers. Word embedding converts the digital identifier of each token into a high-dimensional and dense vector representation. This vector can capture the semantic information of the token. Position encoding is used to inject additional position information so that the model knows the order of each token in the sequence. Attention calculation is used to enable each token in the sequence to "attend" to all other relevant tokens in the sequence, so as to better understand the context. The feedforward neural network is a simple fully connected neural network that performs independent and non-linear transformations on the representation of each token.

[0081] Figure 3The text clearly demonstrates that large-scale model reasoning is a progressive and gradually abstracting process. This process can be divided into multiple layers: Layer 0, Layer 1, Layer 2, and so on. Tokenizers, word embeddings, and positional encoding are classified as Layer 0. Attention computation and feedforward neural networks are classified as Layer 1, and so on. Data flows from bottom to top. The input sequence first passes through Layer 0, and its output becomes the input for Layer 1, and so on, passing through all layers in sequence. At each layer, the vector representations of the tokens are updated and refined, incorporating more and more complex contextual information. The lower layers (such as Layer 0) focus on vocabulary and local grammatical structures. The middle layers begin to capture the semantic relationships between phrases and clauses. The higher layers ultimately form a deep, global understanding of the entire input sequence, preparing for answer generation.

[0082] One scenario is that during the inference process of a large model, it is usually necessary to send the inference results to the client. The amount of data to be sent in this communication task is relatively small (i.e., the number of bytes to be transmitted is relatively small) and the priority is low. Therefore, this communication task is classified as the first task category.

[0083] Another scenario is that large model inference usually involves tensor parallelism. In tensor parallelism, there will be uncertain data communication between multiple computing nodes. This communication involves a large number of bytes, that is, a communication task needs to send data to multiple data receiving devices, and the amount of data to be sent by the communication task is large. In this case, the communication task belongs to the second task category.

[0084] The third scenario is as follows: Large model inference usually involves pipelined parallelism. During pipelined parallelism, there will be fixed unidirectional data communication between computing nodes. This communication involves a large number of bytes, that is, a communication task only needs to send data to a fixed single data receiving device, and the amount of data to be sent by the communication task is large. Therefore, the communication task belongs to the third task category.

[0085] The fourth scenario is: During the inference process of a large model, there will be command communication between multiple computing nodes, such as synchronization commands and start commands. The amount of data to be sent in this communication task is small (i.e., the number of bytes to be transmitted is small) and the priority is high. Therefore, this communication task belongs to the fourth task category.

[0086] The following examples illustrate the pipelined parallelism and tensor parallelism transmission methods described above, using specific node flows as examples. (See also...) Figure 4 The diagram shown is an example of a node process. Figure 4 In this example, we will use processes a, b, and c, each including computational nodes A, B, C, and D, as illustrations. Figure 4 It can be seen that the computation nodes in process a run sequentially or in pipelined parallelism. Computation nodes B and C in process b execute in parallel. Computation nodes A and B in process c run in tensor parallelism, and computation nodes C and D in process c also run in tensor parallelism.

[0087] Combination Figure 4 It's understandable that tensor parallelism refers to multiple computing nodes jointly computing a single layer; for example, computing... Figure 4 In process c, nodes C and D jointly complete the computation of layer 1. Pipeline parallelism refers to dividing the model into multiple sequential stages by layer and deploying them on different computing nodes. By allowing different nodes to process different micro-batch data simultaneously and passing the output of the previous node as the input of the next node, a pipeline operation is formed to achieve overlap between computation and communication, thereby improving the overall utilization of the system.

[0088] The following is a combination of Table 1 and... Figure 4 The parallel flow is described in detail in Table 1. Figure 4 While compute node A in process a is still calculating layer 0, compute node B in process a has already started loading the parameters of layer 1. Then compute node A in process a transmits the result of layer 0 to compute node B in process a for processing. Similarly, other compute nodes process in sequence, which will not be elaborated here.

[0089] Table 1 In this embodiment, only the four task categories mentioned above are used as examples for illustration. In practical applications, task-related information may also include other task attributes for task classification. The number of task categories can also be set according to the actual application scenario. For example, task-related information may also include data type, which indicates whether the target data is control signaling or business data. Another example is that task-related information may also include the historical interaction frequency of the communication task, which is obtained statistically based on the historical communication data of the communication task.

[0090] Furthermore, step 201 may also include the following steps: If there are multiple communication tasks to be executed, the order of the communication tasks is adjusted according to their priority so that the current task can be selected according to the order of the communication tasks.

[0091] In one implementation, if there are multiple communication tasks to be executed, the order of the communication tasks can be adjusted according to their priority, and each communication task can be added to the sending queue in that order. Then, the communication task at the head of the sending queue, which is the current task, is retrieved.

[0092] In this way, different priorities can be assigned to each communication task according to different real-time system states or business scenarios, making the scheduling strategy extremely flexible.

[0093] Step 202: Select the communication path outgoing interface that matches the target task category from each communication path outgoing interface; the transmission rates of different communication path outgoing interfaces are different.

[0094] In one implementation, step 202 may be performed using the following steps: S2021: Obtain at least one communication path outgoing interface corresponding to the target task category; each task category has a corresponding relationship with at least one communication path outgoing interface.

[0095] S2022: If there is only one communication path outgoing interface corresponding to the target task category, the communication path outgoing interface corresponding to the target task category shall be used as the communication path outgoing interface that matches the target task category.

[0096] S2023: When there are multiple communication path outgoing interfaces corresponding to the target task category, based on the interface priority of each communication path outgoing interface corresponding to the target task category, the communication path outgoing interface matching the target task category is selected from each communication path outgoing interface corresponding to the target task category.

[0097] In this way, when there are multiple corresponding communication path output interfaces, the appropriate communication path output interface can be selected according to the interface priority, which improves the accuracy of communication path output interface allocation.

[0098] In one implementation, when executing S2023, the following steps may be taken: S2023-1: Select the communication path with the highest priority from the communication path output interfaces corresponding to the target task category.

[0099] S2023-2: Determine whether the conditions for setting multiple interfaces are met.

[0100] In one implementation, the condition for setting multiple interfaces is determined to be met if at least one of the following conditions is met: The amount of target data exceeds the second preset threshold. The target task has a priority higher than the second priority threshold; The current load of the communication method corresponding to the highest priority communication path of the interface is higher than the preset load threshold. The estimated latency of the communication method corresponding to the highest priority communication path of the interface is higher than the preset latency.

[0101] In practical applications, the second preset quantity threshold, the second priority threshold, the preset load threshold, and the preset delay can all be set according to the actual application scenario, and there are no restrictions here.

[0102] In this way, one or more communication path output interfaces can be allocated to the communication task according to the transmission status, thus adapting to a variety of application scenarios and improving the scope of application.

[0103] S2023-3: Under the condition of meeting the set multiple interface conditions, based on the interface priority of each communication path output interface corresponding to the target task category, multiple communication path output interfaces are selected from each communication path output interface corresponding to the target task category, and the selected multiple communication path output interfaces are used as the communication path output interfaces matching the target task category.

[0104] S2023-4: If the conditions for setting multiple interfaces are not met, the communication path with the highest priority will be used as the communication path that matches the target task category.

[0105] Furthermore, in the initial configuration phase, in this embodiment of the application, one or more communication path output interfaces are pre-configured for each task category. If a task category is configured with multiple communication path output interfaces, the interface priority of the multiple communication path output interfaces of the task category can also be configured to determine the interface selection order when the task category transmits data.

[0106] In one implementation, during initial configuration, the following steps can be performed for each task category: Based on the task characteristics corresponding to the task category, at least one communication path output interface is selected from the first output interface, the second output interface, the third output interface, and the fourth output interface as the communication path output interface corresponding to the task category; wherein, each communication path output interface is divided according to the differences in transmission rate and transmission method; the task characteristics include one or more of the following: data volume range, priority range, and transmission method; if the task category corresponds to multiple communication path output interfaces, then the interface priority of each communication path output interface corresponding to the task category is set according to the task characteristics.

[0107] Specifically, the transmission rate of the first output interface is within a first rate range and supports communication with multiple data receiving devices; the transmission rate of the second output interface is within a second rate range and supports communication with multiple data receiving devices; the transmission rate of the third output interface is within a third rate range and supports communication with only one data receiving device; the transmission rate of the fourth output interface is within a fourth rate range and supports communication with multiple data receiving devices; the upper limits of the second and third rate ranges are both higher than the upper limit of the fourth rate range; and the upper limit of the fourth rate range is higher than the upper limit of the first rate range.

[0108] Optionally, the first output interface can be a 100 Mbps Ethernet interface, with a transmission rate level typically at the 100 Mbps level. For example, the first rate range can be greater than 0 and less than 0.1 GB / s (gigabytes per second).

[0109] Optionally, the second output interface can be a Peripheral Component Interconnect Express (PCIe) 3.0 interface, with a transmission rate typically in the gigabyte range, i.e., GB / s. For example, the second rate range can be no less than 1 GB / s and no less than 2 GB / s.

[0110] Optionally, the third output interface can be a Mobile Industry Processor Interface (MIPI), whose transmission rate level is typically in the gigabits per second (Gbps) range. For example, the third rate range can be no less than 0.1 GB / s.

[0111] Optionally, the fourth output interface can be a gigabit Ethernet interface, with a transmission rate level typically at the gigabit level. For example, the fourth rate range can be no less than 0.1 GB / s and less than 1 GB / s.

[0112] In one embodiment, the communication path output interface corresponding to the first task category is a first interface, the communication path output interface corresponding to the second task category is a second interface, the communication path output interface corresponding to the third task category is a third interface, and the communication path output interface corresponding to the fourth task category is a fourth interface.

[0113] In this embodiment, only four communication path output interfaces are used as an example for illustration. In actual applications, the number of communication path output interfaces, the interface division method, the rate range of each interface, and the number of task categories can all be set according to the actual application scenario, and no restrictions are imposed here.

[0114] The following is combined with Figure 5 An example is provided to illustrate the allocation of the outgoing interfaces for the above communication paths. See [reference needed]. Figure 5The diagram shown is a schematic of an interface communication method. Figure 5 It includes an edge-distributed large model inference system and multiple clients. The edge-distributed large model inference system includes computing node A, computing node B, computing node C and computing node D.

[0115] The edge-distributed large model inference system is pre-deployed with four task categories and four communication path output interfaces. The four task categories are: Task Category 1, Task Category 2, Task Category 3, and Task Category 4. The four communication path output interfaces are: a 100 Mbps Ethernet interface (i.e., the first interface), a PCIe 3.0 interface (i.e., the second interface), a MIPI interface (i.e., the third interface), and a Gigabit Ethernet interface (i.e., the fourth interface), with one or more corresponding communication path output interfaces set up for each task category.

[0116] During communication, the edge-distributed large model inference system determines the task category of the communication task based on the data transmission volume, task priority, and / or transmission method of the communication task. Then, based on one or more communication path output interfaces corresponding to the task category, it allocates one or more superior communication path output interfaces to the communication task and sends the data of the communication task through the allocated communication path output interfaces.

[0117] Specifically, the computing node A sends the inference result to the client as the first communication task. Since the transmission rate of the first communication task is at the level of hundreds of megabits, the priority of the target task is lower than the first priority threshold, and the transmission mode can be one-to-one or one-to-many, the target task category is determined as the first task category, and communication is carried out through the hundred-megabit Ethernet interface corresponding to the first task category.

[0118] The second communication task involves compute node A sending data in parallel to compute nodes B, C, and D (i.e., a tensor parallel task). This second communication task has a gigabyte-level transmission rate and a one-to-many transmission mode, thus it falls under the second task category. The communication path for this second task category uses both Gigabit Ethernet and PCIe 3.0 interfaces. The PCIe 3.0 interface has higher priority than the Gigabit Ethernet interface. Therefore, if the transmission status (e.g., the amount of data transmitted) does not meet the multi-interface setting, communication is conducted through a PCIe 3.0 interface with a switch. Otherwise, communication occurs simultaneously through multiple interfaces, such as the Gigabit Ethernet interface and the PCIe 3.0 interface.

[0119] The third communication task is for computing node A to send a large amount of inference intermediate data to computing node B one-to-one. Since the transmission rate of this third communication task is at the level of several gigabits per second (Gbps) and the transmission method is one-to-one, it is determined to belong to the third task category and is carried out through the unidirectional MIPI communication corresponding to the third task category.

[0120] Computing node D sends control signaling to computing nodes A, B, and C as the fourth communication task. Since the transmission rate of this fourth communication task is at the gigabit level and the priority of the target task to which it belongs is not lower than the first priority threshold, it is determined to belong to the fourth task category and communicates through the gigabit network interface corresponding to the fourth task category.

[0121] The transmission rates of the first, second, third, and fourth communication tasks are, in order, low, high, high, and medium.

[0122] Step 203: Transmit the target data to the data receiving device through the communication path output interface that matches the target task category.

[0123] In related technologies, multiple computing nodes in a distributed large-scale model inference system typically communicate using a relatively single high-performance protocol to complete various data exchange tasks. However, transmitting all types of data using the same high-performance protocol is costly. Furthermore, data of different values ​​(such as high-priority core business data and low-priority redundant backup data) will compete for communication resources under the same protocol, which not only easily leads to increased latency in high-value data transmission but also results in the ineffective use of communication bandwidth, ultimately leading to low overall resource utilization efficiency.

[0124] In this embodiment, a software communication interface module is set in the computing node. Through this unified software communication interface module, communication tasks based on the data to be sent are classified, and data is sent according to one or more communication path outgoing interfaces corresponding to the obtained target task category. This cleverly combines communication path outgoing interfaces with different costs and performance levels to selectively transmit task data under different task categories, reducing costs while ensuring communication speed and maximizing the utilization of the overall system communication bandwidth. Furthermore, the optimal communication path can be dynamically selected for the communication task based on its task-related information and the current transmission status of the communication method corresponding to the communication path outgoing interface. Multi-path transmission can accelerate large-volume tasks, significantly reducing overall inference latency and improving system throughput. Additionally, the complex communication strategy management is separated from business computing logic, making the system easier to maintain and expand. Its intelligent load balancing capability avoids congestion on a single path, enhancing system reliability and effectively solving the key problem that traditional single communication methods cannot meet diverse communication needs.

[0125] Based on the same inventive concept, this application also provides a distributed communication device. Since the principle of the above-mentioned device and equipment in solving the problem is similar to that of a distributed communication method, the implementation of the above-mentioned device can refer to the implementation of the method, and repeated details will not be elaborated further. This device can be applied to electronic devices. This application does not limit the type of electronic device; it can be any suitable type of device, such as terminal devices and servers, etc., which will not be elaborated further in this application. The device embodiment can be implemented by software, or by hardware, or a combination of software and hardware. Taking software implementation as an example, as a logical device, it is formed by the processor of the electronic device reading the corresponding computer program instructions from the non-volatile memory into memory and running them.

[0126] See Figure 6 The diagram shown is a structural block diagram of a distributed communication apparatus according to an embodiment of this application. In some embodiments, the distributed communication apparatus exemplified in this application includes: Determining unit 601 is used to determine the target task category to which the current task belongs; The selection unit 602 is used to select the communication path output interface that matches the target task category from each communication path output interface; the transmission rates of different communication path output interfaces are different. The transmission unit 603 is used to transmit target data to the data receiving device through the communication path output interface that matches the target task category.

[0127] In one embodiment, the determining unit 601 is used to: For the current task, the target task category is determined based on one or more of the following: the amount of target data to be transmitted in the current task, the task priority of the current task, and the data transmission method of the current task. The data transmission method is either one-to-one transmission or one-to-many transmission.

[0128] In one implementation, the amount of target data to be transmitted differs for any first task and any second task of different categories; or, For any first task and any second task of different categories, the task priorities are different; or, The data transmission methods differ for any first task and any second task of different categories.

[0129] In one embodiment, the determining unit 601 is used to: Use any one of the following: If the amount of target data to be transmitted in the current task is lower than the first preset quantity threshold, and the priority of the target task is lower than the first priority threshold, then the target task category is determined to be the first task category. If the amount of target data is not less than the first preset quantity threshold and there are multiple data receiving devices, then the target task category is determined to be the second task category. If the amount of target data is not less than the first preset quantity threshold and the number of data receiving devices is one, then the target task category is determined to be the third task category. If the amount of target data is lower than the first preset quantity threshold and the priority of the target task is not lower than the first priority threshold, then the target task category is determined to be the fourth task category.

[0130] In one embodiment, the selection unit 602 is used for: Obtain at least one communication path outgoing interface corresponding to the target task category; each task category has a corresponding relationship with at least one communication path outgoing interface; If there is only one communication path outgoing interface corresponding to the target task category, the communication path outgoing interface corresponding to the target task category will be used as the communication path outgoing interface matched with the target task category. When there are multiple communication path outgoing interfaces corresponding to the target task category, the communication path outgoing interface that matches the target task category is selected from the communication path outgoing interfaces corresponding to the target task category based on the interface priority of each communication path outgoing interface corresponding to the target task category.

[0131] In one embodiment, the selection unit 602 is used for: From the communication path outgoing interfaces corresponding to the target task category, select the communication path outgoing interface with the highest interface priority; Determine if the conditions for setting multiple interfaces are met; Under the condition of setting multiple interfaces, based on the interface priority of each communication path output interface corresponding to the target task category, multiple communication path output interfaces are selected from each communication path output interface corresponding to the target task category, and the selected multiple communication path output interfaces are used as the communication path output interfaces matching the target task category. If the conditions for setting multiple interfaces are not met, the communication path with the highest priority will be used as the communication path that matches the target task category.

[0132] In one embodiment, the selection unit 602 is used for: The condition for setting multiple interfaces is determined to be met if at least one of the following conditions is met: The amount of target data exceeds the second preset threshold. The target task has a priority higher than the second priority threshold; The current load of the communication method corresponding to the highest priority communication path of the interface is higher than the preset load threshold. The estimated latency of the communication method corresponding to the highest priority communication path of the interface is higher than the preset latency.

[0133] In one embodiment, the selection unit 602 is further configured to: For each task category, perform the following steps: Based on the task characteristics corresponding to the task category, at least one communication path output interface is selected from the first output interface, the second output interface, the third output interface, and the fourth output interface as the communication path output interface corresponding to the task category; wherein, each communication path output interface is divided according to the differences in transmission rate and transmission method; the task characteristics include one or more of the following: data volume range, priority range, and transmission method; If a task category corresponds to multiple communication path outgoing interfaces, then the interface priority of each communication path outgoing interface corresponding to the task category is set according to the task characteristics.

[0134] In one embodiment, the transmission rate of the first output interface is within a first rate range and supports communication with multiple data receiving devices. The transmission rate of the second output interface is within the second rate range and supports communication with multiple data receiving devices; The transmission rate of the third output interface is within the third rate range, and it only supports communication with one data receiving device. The transmission rate of the fourth output interface is within the fourth rate range and supports communication with multiple data receiving devices; Among them, the upper limits of the second and third rate ranges are both higher than the upper limit of the fourth rate range; the upper limit of the fourth rate range is higher than the upper limit of the first rate range.

[0135] In one embodiment, the determining unit 601 is further configured to: If there are multiple communication tasks to be executed, the order of the communication tasks is adjusted according to their priority so that the current task can be selected according to the order of the communication tasks.

[0136] The distributed communication method in this application includes determining the target task category to which the current task belongs; selecting a communication path output interface matching the target task category from various communication path output interfaces; different communication path output interfaces have different transmission rates; and transmitting target data to a data receiving device through the communication path output interface matching the target task category. In this way, allocating an appropriate communication path output interface to the current task based on its task category can improve resource utilization and data transmission performance.

[0137] In this embodiment of the application, an electronic device is also provided, including: Processor; and The memory stores computer instructions that cause the processor to execute the methods of any of the above-described embodiments.

[0138] In this embodiment of the application, a communication device is also provided, which is deployed in a computing node. The communication device includes a processor coupled to a memory. Memory, used to store instructions; A processor is configured to execute instructions in memory to cause the communication device to perform the method of any of the above embodiments.

[0139] In this application embodiment, a computer-readable storage medium is provided, storing computer instructions for causing a computer to perform the methods of any of the above embodiments.

[0140] This application also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in the processor of an electronic device, the processor in the electronic device performs the method of any of the above-described embodiments.

[0141] Figure 7 A schematic diagram of the structure of an electronic device 7000 is shown. (See also...) Figure 7 As shown, the electronic device 7000 includes a processor 7010 and a memory 7020, and optionally may also include a power supply 7030, a display unit 7040, and an input unit 7050.

[0142] The processor 7010 is the control center of the electronic device 7000. It connects various components through various interfaces and lines, and performs various functions of the electronic device 7000 by running or executing software programs and / or data stored in the memory 7020, thereby performing overall monitoring of the electronic device 7000.

[0143] In this embodiment, when the processor 7010 calls the computer program stored in the memory 7020, it executes the steps in the above embodiments.

[0144] Optionally, the processor 7010 may include one or more processing units; preferably, the processor 7010 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 7010.

[0145] The memory 7020 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, various applications, etc.; the data storage area may store data created based on the use of the electronic device 7000, etc. In addition, the memory 7020 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device, etc.

[0146] Electronic device 7000 also includes a power supply 7030 (such as a battery) that supplies power to various components. The power supply can be logically connected to processor 7010 through a power management system, thereby enabling the management of charging, discharging, and power consumption.

[0147] The display unit 7040 can be used to display information input by the user or information provided to the user, as well as various menus of the electronic device 7000. In this embodiment, it is mainly used to display the display interface of various applications in the electronic device 7000, as well as text, images, and other objects displayed on the display interface. The display unit 7040 may include a display panel 7041. The display panel 7041 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0148] The input unit 7050 can be used to receive information such as numbers or characters input by the user. The input unit 7050 may include a touch panel 7051 and other input devices 7052. The touch panel 7051, also known as a touch screen, can collect touch operations on or near the touch panel 7051 (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 7051).

[0149] Specifically, the touch panel 7051 can detect user touch operations and the signals generated by these operations, convert them into touch point coordinates, send them to the processor 7010, and receive and execute commands from the processor 7010. Furthermore, the touch panel 7051 can be implemented using various types of sensors, including resistive, capacitive, infrared, and surface acoustic wave sensors. Other input devices 7052 can include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0150] Of course, the touch panel 7051 can cover the display panel 7041. When the touch panel 7051 detects a touch operation on or near it, it transmits the information to the processor 7010 to determine the type of touch event. Subsequently, the processor 7010 provides corresponding visual output on the display panel 7041 based on the type of touch event. Although in Figure 7 In this embodiment, the touch panel 7051 and the display panel 7041 are two separate components to realize the input and output functions of the electronic device 7000. However, in some embodiments, the touch panel 7051 and the display panel 7041 can be integrated to realize the input and output functions of the electronic device 7000.

[0151] The electronic device 7000 may also include one or more sensors, such as a pressure sensor, a gravity acceleration sensor, a proximity light sensor, etc. Of course, depending on the specific application, the electronic device 7000 may also include other components such as a camera. Since these components are not the focus of this application's embodiments, therefore... Figure 7 It is not shown in the text and will not be described in detail here.

[0152] Those skilled in the art will understand that Figure 7 This is merely an example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than shown, or a combination of certain components, or different components.

[0153] For ease of description, the above sections are divided into modules (or units) according to their functions and described separately. Of course, in implementing this application, the functions of each module (or unit) can be implemented in one or more software or hardware components.

Claims

1. A method for distributed communication, characterized in that, This method is applied to computing nodes in an edge-distributed large-model inference system, wherein the computing nodes are deployed with multiple communication path outgoing interfaces, and the method includes: Determine the target task category to which the current task belongs; From the various communication path output interfaces, select the communication path output interface that matches the target task category; the transmission rates of different communication path output interfaces are different; The target data is transmitted to the data receiving device through the communication path output interface that matches the target task category.

2. The method according to claim 1, characterized in that, Determining the target task category to which the current task belongs includes: For the current task, the target task category is determined based on one or more of the following: the amount of target data to be transmitted in the current task, the task priority of the current task, and the data transmission method of the current task. The data transmission method is either one-to-one transmission or one-to-many transmission.

3. The method according to claim 1 or 2, characterized in that, For any first task and any second task of different categories, the amount of target data to be transmitted is different; or, For any first task and any second task of different categories, the task priorities are different; or, The data transmission methods differ for any first task and any second task of different categories.

4. The method according to claim 1 or 2, characterized in that, Determining the target task category to which the current task belongs includes: Use any one of the following: If the amount of target data to be transmitted in the current task is lower than a first preset quantity threshold, and the priority of the target task is lower than a first priority threshold, then the target task category is determined to be the first task category; If the amount of the target data is not less than the first preset quantity threshold, and the number of data receiving devices is multiple, then the target task category is determined to be the second task category; If the amount of the target data is not less than the first preset quantity threshold, and the number of the data receiving devices is one, then the target task category is determined to be the third task category; If the amount of the target data is lower than the first preset quantity threshold, and the priority of the target task is not lower than the first priority threshold, then the target task category is determined to be the fourth task category.

5. The method according to claim 1 or 2, characterized in that, The step of selecting the communication path outgoing interface that matches the target task category from each communication path outgoing interface includes: Obtain at least one communication path outgoing interface corresponding to the target task category; each task category has a corresponding relationship with at least one communication path outgoing interface; If there is only one communication path outgoing interface corresponding to the target task category, then the communication path outgoing interface corresponding to the target task category shall be used as the communication path outgoing interface matched by the target task category. When there are multiple communication path outgoing interfaces corresponding to the target task category, the communication path outgoing interface matching the target task category is selected from the communication path outgoing interfaces corresponding to the target task category based on the interface priority of each communication path outgoing interface corresponding to the target task category.

6. The method according to claim 5, characterized in that, The step of filtering out communication path outgoing interfaces matching the target task category from among the communication path outgoing interfaces corresponding to the target task category based on the interface priority of each communication path outgoing interface corresponding to the target task category includes: From the communication path outgoing interfaces corresponding to the target task category, select the communication path outgoing interface with the highest interface priority; Determine if the conditions for setting multiple interfaces are met; Under the condition of setting multiple interfaces, based on the interface priority of each communication path output interface corresponding to the target task category, multiple communication path output interfaces are selected from each communication path output interface corresponding to the target task category, and the selected multiple communication path output interfaces are used as the communication path output interfaces matched by the target task category. If the conditions for setting multiple interfaces are not met, the communication path output interface with the highest priority will be used as the communication path output interface that matches the target task category.

7. The method according to claim 6, characterized in that, The determination of whether the set multi-interface conditions are met includes: The multi-interface setting condition is determined to be met if at least one of the following conditions is met: The amount of the target data is higher than the second preset quantity threshold; The priority of the target task is higher than the second priority threshold; The current load of the communication method corresponding to the highest priority communication path outgoing interface is higher than a preset load threshold. The estimated latency of the communication method corresponding to the highest priority communication path of the interface is higher than the preset latency.

8. The method according to claim 1 or 2, characterized in that, Before selecting the communication path outgoing interface that matches the target task category from among the various communication path outgoing interfaces, the method further includes: For each task category, perform the following steps: Based on the task characteristics corresponding to the task category, at least one communication path output interface is selected from the first output interface, the second output interface, the third output interface, and the fourth output interface as the communication path output interface corresponding to the task category; wherein, each communication path output interface is divided according to the differences in transmission rate and transmission method; the task characteristics include one or more of the following: data volume range, priority range, and transmission method; If the task category corresponds to multiple communication path outgoing interfaces, then the interface priority of each communication path outgoing interface corresponding to the task category is set according to the task characteristics.

9. The method according to claim 8, characterized in that, The transmission rate of the first output interface is within the first rate range and supports communication with multiple data receiving devices; The transmission rate of the second output interface is within the second rate range and supports communication with multiple data receiving devices; The transmission rate of the third output interface is within the third rate range, and it only supports communication with one data receiving device. The transmission rate of the fourth output interface is within the fourth rate range and supports communication with multiple data receiving devices. Wherein, the upper limit values ​​of the second rate range and the third rate range are both higher than the upper limit value of the fourth rate range; the upper limit value of the fourth rate range is higher than the upper limit value of the first rate range.

10. The method according to any one of claims 1-7, characterized in that, Before determining the target task category to which the current task belongs, the method further includes: If there are multiple communication tasks to be executed, the order of the communication tasks is adjusted according to their priority, so that the current task is selected according to the order of the communication tasks.

11. A distributed communication system, characterized in that, It includes an edge-distributed large model inference system and at least one client. The edge-distributed large model inference system contains at least one computing node, and the computing node is deployed with multiple communication path outgoing interfaces. The computing node is used to: determine the target task category to which the current task belongs; select a communication path output interface that matches the target task category from each communication path output interface; different communication path output interfaces have different transmission rates; and transmit target data to a data receiving device through the communication path output interface that matches the target task category; the data receiving device can be any client or another computing node different from this computing node. Each client is used to: receive target data sent by the computing node with which it communicates.

12. A distributed communication system, characterized in that, It includes multiple communicable computing nodes, wherein any one of the computing nodes is deployed with multiple communication path outgoing interfaces; The computing node is used to: determine the target task category to which the current task belongs; select a communication path output interface that matches the target task category from among the communication path output interfaces; different communication path output interfaces have different transmission rates; and transmit data or commands to another computing node through the communication path output interface that matches the target task category.

13. A distributed communication device, characterized in that, This device is applied to a computing node in an edge-distributed large-model inference system. The computing node is deployed with multiple communication path output interfaces. The device includes: The determining unit is used to determine the target task category to which the current task belongs. The selection unit is used to select the communication path output interface that matches the target task category from each communication path output interface; the transmission rates of different communication path output interfaces are different. The transmission unit is used to transmit target data to the data receiving device through the communication path output interface that matches the target task category.

14. An electronic device, characterized in that, include: processor; as well as A memory storing computer instructions for causing the processor to perform the method according to any one of claims 1 to 10.

15. A communication device, characterized in that, The communication device is deployed within a computing node and includes a processor coupled to memory. The memory is used to store instructions; The processor is configured to execute instructions in the memory to cause the communication device to perform the method as described in any one of claims 1 to 10.

Images