AI card message communication optimization method based on heterogeneous perception pipeline and dynamic routing

Abstract

The application discloses an AI card message communication optimization method based on a heterogeneous perception pipeline and dynamic routing, relates to the technical field of communication, and discloses the AI card message communication optimization method based on the heterogeneous perception pipeline and the dynamic routing, which effectively solves the problem of low communication efficiency in the prior art by collecting system state data, generating dynamic strategy data and performing adaptive processing, can dynamically adapt to network changes and heterogeneous computing environments, improves communication efficiency, reduces delay, and optimizes resource utilization.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to an AI card message communication optimization method based on heterogeneous sensing pipeline and dynamic routing. Background Technology

[0002] As the scale of artificial intelligence models continues to expand, AI card clusters (such as GPUs, NPUs, and other accelerated computing units) have become the core infrastructure supporting large-scale deep learning training. In distributed training scenarios, AI cards must achieve gradient synchronization and parameter exchange through efficient aggregate communication operations (such as AllReduce, AllGather, etc.) to ensure the collaborative completion of training tasks. However, existing AI card communication mechanisms suffer from multiple technical bottlenecks: communication paths generally employ preset static topologies (such as ring or tree topologies). These structures cannot perceive real-time fluctuations in network link quality (including bandwidth changes, latency fluctuations, and abnormal bit error rates), nor can they adapt to dynamic congestion conditions, leading to a significant increase in communication latency and low bandwidth resource utilization. Simultaneously, the computing capabilities of AI cards in a cluster often exhibit heterogeneity, with some AI cards processing faster (fast cards) and others slower (slow cards). Traditional communication solutions employ uniform data partitioning and task allocation strategies, forcing fast cards to wait for slow cards to complete their calculations, creating a "bottleneck effect" that severely restricts overall communication efficiency. Furthermore, the processing strategies of communication pipelines are typically fixed and lack the ability to adaptively adjust based on dynamic characteristics such as message size and task priority. For example, small-scale messages may incur increased overhead due to overprocessing, while large-scale messages may cause congestion due to underprocessing, further exacerbating communication performance bottlenecks. These problems make it difficult for existing technologies to meet the urgent needs of large-scale AI clusters for high-throughput, low-latency communication.

[0003] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide an AI card message communication optimization method based on heterogeneous sensing pipeline and dynamic routing, which aims to improve communication efficiency, reduce latency, and optimize resource utilization.

[0005] To achieve the above objectives, this application proposes an AI card message communication optimization method based on heterogeneous sensing pipeline and dynamic routing. The method includes:

[0006] Collect AI card cluster status information and current communication task information to generate system status data;

[0007] The system status data is subjected to heterogeneous perception and dynamic decision-making processing to generate dynamic routing strategy data, heterogeneous task allocation strategy data, and pipeline processing strategy data.

[0008] Based on the dynamic routing strategy data, the heterogeneous task allocation strategy data, and the pipeline processing strategy data, adaptive pipeline processing and dynamic routing processing are performed on the original communication message data that needs to be communicated to obtain optimized communication message data.

[0009] The optimized communication message data is transmitted in the AI ​​card cluster through a dynamically selected transmission path to complete message communication between AI cards.

[0010] In one embodiment, the step of collecting AI card cluster status information and current communication task information to generate system status data includes:

[0011] Periodically or event-triggered, collect the computing status information, interconnection link status information, and network status information of each AI card to obtain raw status data;

[0012] The original state data is integrated and updated in real time to generate AI card cluster state information, which includes computing power difference information, link bandwidth information and network congestion information.

[0013] Collect information on the type of communication operation to be executed and the message size to generate current communication task information;

[0014] The AI ​​card cluster status information and the current communication task information are combined and processed to generate the system status data.

[0015] In one embodiment, the step of performing heterogeneous sensing and dynamic decision processing on the system state data to generate dynamic routing strategy data, heterogeneous task allocation strategy data, and pipeline processing strategy data includes:

[0016] Based on the link bandwidth information and network congestion information in the system status data, dynamic topology reconstruction is performed to generate the dynamic routing policy data.

[0017] Based on the computing power difference information in the system status data, heterogeneous load balancing is performed to generate the heterogeneous task allocation strategy data.

[0018] Based on the message size information in the system status data, adaptive strategy selection is performed to generate the pipeline processing strategy data.

[0019] In one embodiment, the step of performing dynamic topology reconstruction based on link bandwidth information and network congestion information in the system status data to generate the dynamic routing policy data includes:

[0020] Based on the real-time bandwidth, latency, and bit error rate information of each link in the system status data, link quality assessment is performed to obtain link quality assessment data.

[0021] Based on the current communication task information in the link quality assessment data and the system status data, a weighted topology construction process is performed to generate a weighted communication directed acyclic graph.

[0022] Based on the network congestion information in the system status data, conflict detection and replanning are performed on the weighted communication directed acyclic graph to generate a real-time optimal weighted directed acyclic graph, which constitutes the dynamic routing strategy data.

[0023] In one embodiment, the step of performing weighted topology construction processing to generate a weighted communication directed acyclic graph based on the link quality assessment data and the current communication task information in the system status data includes:

[0024] Obtain task priority information from the current communication task information in the system status data to determine the priority level of the communication task;

[0025] Set a corresponding link weight coefficient for each priority level;

[0026] Based on the link quality assessment data and the link weight coefficients, a weighted calculation is performed to generate the weighted directed acyclic graph of communication.

[0027] In one embodiment, the step of performing heterogeneous load balancing processing based on the computing power difference information in the system status data to generate the heterogeneous task allocation strategy data includes:

[0028] Based on the calculation status information of each AI card in the system status data, fast and slow card identification processing is performed to obtain fast card identification information and slow card identification information;

[0029] Based on the fast card identification information and the slow card identification information, non-uniform task allocation processing is performed to generate a data block allocation scheme;

[0030] The data block allocation scheme is processed by task scheduling planning to generate the heterogeneous task allocation strategy data; the heterogeneous task allocation strategy data includes information on allocating data blocks of different sizes to AI cards with different computing capabilities.

[0031] In one embodiment, the step of performing adaptive strategy selection processing based on message size information in the system state data to generate the pipeline processing strategy data includes:

[0032] Obtain message size information from the system status data;

[0033] The message size information is compared with a preset threshold, and a strategy classification process is performed to obtain the strategy classification result.

[0034] Based on the strategy classification results, a corresponding processing strategy is selected from a variety of predefined processing strategies to generate the pipeline processing strategy data.

[0035] In one embodiment, based on the dynamic routing policy data, the heterogeneous task allocation policy data, and the pipelined processing policy data, adaptive pipelined processing and dynamic routing processing are performed on the original communication message data that needs to be communicated to obtain optimized communication message data, including:

[0036] Based on the pipeline processing strategy data, dynamic assembly of configurable processing units is performed to generate adaptive pipeline configuration data.

[0037] Based on the adaptive pipeline configuration data, the original communication message data that needs to be communicated is serialized, compressed, and scheduled to obtain pipelined message data.

[0038] Based on the dynamic routing strategy data, the optimal real-time transmission path is selected for the message data that has undergone pipeline processing, so as to generate path selection data.

[0039] Based on the heterogeneous task allocation strategy data, the message data that has undergone pipeline processing is processed into data blocks to obtain the optimized communication message data.

[0040] In one embodiment, the step of dynamically assembling configurable processing units based on the pipeline processing strategy data to generate adaptive pipeline configuration data includes:

[0041] Obtain processing strategy information from the pipeline processing strategy data;

[0042] Based on the processing strategy information, a corresponding processing unit is selected from multiple configurable processing units;

[0043] The selected processing units are dynamically assembled according to a preset processing order to generate the adaptive pipeline configuration data.

[0044] In one embodiment, the step of performing data block allocation processing on the pipelined message data based on the heterogeneous task allocation strategy data to obtain the optimized communication message data includes:

[0045] Information on allocating data blocks of different sizes to AI cards with different computing capabilities is obtained from the heterogeneous task allocation strategy data;

[0046] Based on the information that different sizes of data blocks are allocated to AI cards with different computing capabilities, the message data that has been pipelined is divided into non-uniform data blocks to generate a non-uniform data block set.

[0047] Data blocks of different sizes in the non-uniform data block set are assigned to the corresponding AI cards to obtain the optimized communication message data.

[0048] The AI ​​card message communication optimization method proposed in this application, based on heterogeneous sensing pipeline and dynamic routing, effectively solves the problem of low communication efficiency in the prior art by collecting system status data, generating dynamic policy data, and performing adaptive processing. It can dynamically adapt to network changes and heterogeneous computing environments, improve communication efficiency, reduce latency, and optimize resource utilization. Attached Figure Description

[0049] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a flowchart illustrating an embodiment of the AI ​​card message communication optimization method based on heterogeneous sensing pipeline and dynamic routing in this application.

[0052] Explanation of icon numbers:

[0053] 10. Memory; 20. Processor.

[0054] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0055] The technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of this application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0056] It should be understood that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0057] As the scale of AI models continues to expand, AI card clusters have become the mainstream computing platform for large-scale deep learning training. However, existing AI card communication methods have significant technical bottlenecks, such as fixed communication paths that are difficult to adapt to dynamic network changes, leading to increased communication latency and low bandwidth utilization; heterogeneous computing capabilities among different AI cards in the cluster, where traditional uniform data partitioning and task allocation can drag down overall efficiency due to slower cards; and fixed communication pipeline processing strategies that cannot be adaptively adjusted according to characteristics such as message size, making it difficult to optimize communication overhead.

[0058] Based on this, embodiments of this application provide an AI card message communication optimization method based on heterogeneous sensing pipelines and dynamic routing, referring to... Figure 1 The AI ​​card message communication optimization method based on heterogeneous sensing pipeline and dynamic routing includes steps S100 to S400, wherein:

[0059] Step S100: Collect AI card cluster status information and current communication task information to generate system status data;

[0060] Step S200: Perform heterogeneous perception and dynamic decision processing on the system status data to generate dynamic routing strategy data, heterogeneous task allocation strategy data, and pipeline processing strategy data.

[0061] Step S300: Based on the dynamic routing strategy data, the heterogeneous task allocation strategy data, and the pipeline processing strategy data, adaptive pipeline processing and dynamic routing processing are performed on the original communication message data that needs to be communicated, so as to obtain optimized communication message data.

[0062] Step S400: The optimized communication message data is transmitted in the AI ​​card cluster through a dynamically selected transmission path to complete message communication between AI cards.

[0063] In this embodiment, the AI ​​card cluster is a computing system composed of multiple AI cards (such as GPUs, NPUs, etc.) connected through a high-speed interconnect network. This cluster is used to collaboratively execute large-scale deep learning training or inference tasks, requiring frequent data exchange and message communication between the AI ​​cards. System status data refers to a comprehensive description of the AI ​​card cluster's operating status and current communication tasks. This data reflects the real-time performance, load, and characteristics of the communication tasks to be processed within the cluster, providing a basis for subsequent decision-making. Heterogeneous perception and dynamic decision processing refer to the system identifying and analyzing the heterogeneity (e.g., differences in computing power, differences in link quality) within the cluster based on the collected AI card cluster status data, and making real-time decisions based on these perception results to generate optimized strategies adapted to the current environment.

[0064] In this embodiment, dynamic routing strategy data is a set of rules for dynamically planning and adjusting message transmission paths within the AI ​​card cluster based on real-time network conditions and communication requirements. This strategy data is used to guide messages in selecting the optimal transmission path within the cluster to avoid congestion or faulty links. Heterogeneous task allocation strategy data is a set of rules for unevenly distributing communication tasks or data blocks within the AI ​​card cluster based on the differences in computing power among different AI cards. This strategy data is used to ensure that AI cards with higher computing power undertake more tasks, thereby improving overall processing efficiency. Pipeline processing strategy data is a set of rules for dynamically selecting and configuring message processing procedures (e.g., serialization, compression, chunking) based on the characteristics of communication messages (e.g., message size, task priority). This strategy data is used to optimize the preprocessing of messages before transmission to reduce communication overhead.

[0065] In this embodiment, adaptive pipelined processing refers to dynamically assembling and configuring message processing units according to pipelined processing strategy data, performing a series of preprocessing operations on the original communication message data. This process is used to transform the original message into a format more suitable for transmission and processing. Dynamic routing processing refers to selecting the optimal real-time transmission path for the pipelined message data according to dynamic routing strategy data. This process is used to ensure that the message can reach the target AI card efficiently and reliably. Optimized communication message data refers to the message data after adaptive pipelined processing and dynamic routing processing. This data is used for transmission in the AI ​​card cluster, and its format and transmission path have been optimized according to the current system state. The dynamically selected transmission path refers to the network path used to transmit the optimized communication message data in real time within the AI ​​card cluster according to dynamic routing strategy data. This path is used to avoid network congestion or failures, ensuring the efficiency and reliability of message transmission.

[0066] In this embodiment, the AI ​​card message communication optimization method based on heterogeneous sensing pipeline and dynamic routing first collects AI card cluster status information and current communication task information to generate system status data. Specifically, at preset fixed time intervals, the central controller sends query commands to each AI card in the AI ​​card cluster to obtain its current operating status, such as simple load rate or memory usage. Simultaneously, communication task information can be manually entered by the user or directly read from the task queue, such as specifying the source and target AI cards for communication and the initial message size. This information is simply aggregated to form preliminary system status data.

[0067] In this embodiment, the system status data undergoes heterogeneous sensing and dynamic decision-making processing to generate dynamic routing strategy data, heterogeneous task allocation strategy data, and pipeline processing strategy data. The received system status data can be input into a predefined decision module. This module can generate a basic routing suggestion based on simple rules, such as marking an AI card as "busy" if its load exceeds a threshold, for example, avoiding the busy AI card. For task allocation, a preliminary task allocation scheme can be generated based on the AI ​​card model or a preset performance level. For pipeline processing, whether to perform compression can be determined simply based on whether the message exceeds a certain fixed size threshold. Thus, preliminary dynamic routing strategy data, heterogeneous task allocation strategy data, and pipeline processing strategy data are generated.

[0068] In this embodiment, based on the dynamic routing policy data, the heterogeneous task allocation policy data, and the pipelined processing policy data, adaptive pipelined processing and dynamic routing processing are performed on the original communication message data that needs to be communicated to obtain optimized communication message data. Before sending, the original communication message data can be preprocessed according to the pipelined processing policy data. For example, if the policy indicates that compression is required, the message is processed using a general compression algorithm. Subsequently, according to the dynamic routing policy data, a path marked as "available" is selected from several preset fixed paths. Finally, according to the heterogeneous task allocation policy data, the message data is simply divided into fixed-size blocks and allocated to the target AI card. After these processes, preliminarily optimized communication message data is obtained.

[0069] In this embodiment, the optimized communication message data is transmitted within the AI ​​card cluster via a dynamically selected transmission path to complete message communication between AI cards. The optimized communication message data is encapsulated and sent through the network interface. The selection of the transmission path is determined based on the aforementioned dynamic routing policy data. For example, the system can maintain a simple routing table, which is updated according to the policy data, indicating which next-hop node the message should be forwarded through. The message is transmitted along the selected path in the physical interconnection network of the AI ​​card cluster until it reaches the target AI card, thus completing one message communication.

[0070] In this embodiment, by collecting AI card cluster status and communication task information, and performing heterogeneous perception and dynamic decision-making, dynamic routing, heterogeneous task allocation, and pipeline processing strategies are generated. As a result, raw communication message data is adaptively pipelined and dynamically routed, and transmitted within the AI ​​card cluster via dynamically selected transmission paths. This method effectively addresses issues such as fixed communication paths, heterogeneous AI card computing capabilities, and fixed communication pipeline processing strategies within AI card clusters, thereby improving the message communication efficiency and overall performance of the AI ​​card cluster.

[0071] In one feasible implementation, the step of collecting AI card cluster status information and current communication task information to generate system status data includes: periodically or event-triggeredly collecting computing status information, interconnection link status information, and network status information of each AI card to obtain raw status data; performing status integration and real-time update processing on the raw status data to generate AI card cluster status information, which includes computing capability difference information, link bandwidth information, and network congestion information; collecting information on the type of aggregated communication operation to be executed and message size information to generate current communication task information; and comprehensively processing the AI ​​card cluster status information and the current communication task information to generate the system status data.

[0072] In this embodiment, the computing status information, interconnection link status information, and network status information of each AI card are collected periodically or event-triggered to obtain raw status data, aiming to obtain the underlying operating status of the AI ​​card cluster. Computation status information can refer to indicators such as processor utilization, memory usage, temperature, and power consumption of the AI ​​card, reflecting the current computing load and processing capacity of the AI ​​card. This information can be reported periodically by the monitoring agent program inside the AI ​​card or triggered when specific indicators (such as utilization exceeding a threshold) are triggered. Interconnection link status information can refer to the physical layer and data link layer status of the interconnection links between AI cards, such as link activity, physical bandwidth, and error rate. This information can be obtained through statistical data from network interface controllers or switch ports. Network status information can refer to the overall operating status of the network in which the AI ​​card cluster is located, including network latency, packet loss rate, throughput, and routing table changes. This information can be collected through network monitoring tools or custom probes. Periodic collection ensures that the system continuously obtains the latest status, while event-triggered collection can respond promptly to emergencies and provide more timely status updates. Raw status data is unprocessed, basic monitoring data obtained directly from the hardware or operating system level.

[0073] In this embodiment, the raw state data undergoes state integration and real-time update processing to generate AI card cluster state information. This AI card cluster state information includes computing power difference information, link bandwidth information, and network congestion information. This step centrally processes the scattered raw state data, extracting high-level, meaningful cluster state information. State integration means aggregating, cleaning, deduplicating, and unifying the format of raw state data collected from different sources and at different times. Real-time update processing ensures that the AI ​​card cluster state information can reflect the latest status of the cluster in a timely manner, for example, through streaming data processing technology. The AI ​​card cluster state information is a macroscopic description of the overall operating status of the AI ​​card cluster. Computing power difference information refers to the differences in processing capabilities among different AI cards in the cluster, such as the difference between high-performance and low-performance models, or the difference in actual available computing power of the same model of AI card due to factors such as load and heat dissipation. This can be obtained by analyzing the computing state information. Link bandwidth information refers to the actual available bandwidth of the interconnection links between AI cards, which may be affected by physical limitations, traffic competition, and other factors. This can be calculated by analyzing the interconnection link state information and network state information. Network congestion information refers to whether there are traffic bottlenecks or overloads in the network, which may lead to increased data transmission delays or packet loss. It can be determined by analyzing network status information or feedback information from specific congestion control protocols.

[0074] In this embodiment, information about the type of the current collective communication operation to be executed and the message size information are collected to generate current communication task information. This step aims to obtain the specific characteristics of the communication task currently in progress or about to be performed, in order to provide task-level context for optimization. The collective communication operation type information refers to common communication patterns in AI training, such as All-Reduce, All-Gather, Reduce-Scatter, etc. Different operation types have different requirements for network topology and bandwidth. The message size information refers to the size of the data block transmitted in the collective communication operation, such as in bytes, KB, or MB, and is a key factor affecting communication latency and throughput. This information can usually be obtained from the communication library interface of the AI ​​training framework or explicitly provided by the application.

[0075] In this embodiment, the AI ​​card cluster status information and the current communication task information are comprehensively processed to generate the system status data. This step combines the operating status of the AI ​​card cluster with the requirements of the current communication task to form comprehensive system status data for decision-making. Comprehensive processing means associating, fusing, and structuring two different types of information. For example, combining the computing power difference information of a certain AI card with the message size information of a certain collective communication task it is about to participate in, the potential bottleneck of the AI ​​card in the task can be evaluated. The system status data is a structured data set that includes the heterogeneous characteristics of the AI ​​card cluster, dynamic load, network status, and the specific requirements of the current communication task, providing comprehensive input for subsequent heterogeneous perception and dynamic decision-making processing.

[0076] In this embodiment, through the above technical solution, the system can periodically or event-triggeredly collect the computing status information, interconnection link status information, and network status information of the AI ​​cards, thereby obtaining comprehensive and real-time raw status data. This raw status data, after status integration and real-time update processing, can accurately generate information reflecting the heterogeneous characteristics of the AI ​​card cluster, including computing power differences, link bandwidth information, and network congestion information. Simultaneously, by collecting information on the current cluster communication operation type and message size, the system can clearly understand the specific requirements of the communication task. Finally, by comprehensively processing this AI card cluster status information and current communication task information, comprehensive, accurate, and real-time system status data is generated. This detailed and dynamic status data collection and processing mechanism overcomes the problems of incomplete, untimely, or inaccurate status information that may exist in traditional methods. It ensures that subsequent heterogeneous perception and dynamic decision-making processing can be based on the most realistic and detailed cluster operating status and task requirements, thereby generating more accurate and adaptable dynamic routing strategy data, heterogeneous task allocation strategy data, and pipeline processing strategy data that are more adapted to the current heterogeneous environment and dynamic changes. For example, by accurately identifying differences in computing power, it is possible to avoid assigning a large number of computing tasks to slow AI cards; by sensing network congestion in real time, it is possible to avoid transmitting data to congested links, thereby improving the overall optimization effect and efficiency of message communication between AI cards.

[0077] In one feasible implementation, the step of performing heterogeneous perception and dynamic decision-making processing on the system status data to generate dynamic routing strategy data, heterogeneous task allocation strategy data, and pipeline processing strategy data includes: performing dynamic topology reconstruction processing based on link bandwidth information and network congestion information in the system status data to generate the dynamic routing strategy data; performing heterogeneous load balancing processing based on computing power difference information in the system status data to generate the heterogeneous task allocation strategy data; and performing adaptive strategy selection processing based on message size information in the system status data to generate the pipeline processing strategy data.

[0078] In this embodiment, dynamic topology reconstruction is performed based on link bandwidth and network congestion information in the system status data to generate dynamic routing policy data. Dynamic topology reconstruction refers to the process of dynamically adjusting or replanning communication paths and network structures based on real-time bandwidth utilization, packet loss rate, latency, and other network status indicators of each interconnecting link in the AI ​​card cluster. Its main purpose is to avoid network congestion and improve the efficiency and reliability of data transmission. In practical applications, the system can trigger this process periodically or when a significant change in network status is detected. For example, when the bandwidth utilization of a link is too high or a large number of packets are lost, dynamic topology reconstruction will recalculate and select a transmission path with lower current load and better performance, thereby guiding communication traffic from potentially congested areas to smoother paths. This can be achieved by constructing a network graph model and using various graph algorithms (such as shortest path algorithms and traffic engineering algorithms) to find and determine the optimal set of transmission paths, ultimately forming dynamic routing policy data.

[0079] In this embodiment, heterogeneous load balancing is further performed based on the computing power difference information in the system status data to generate heterogeneous task allocation strategy data. Heterogeneous load balancing aims to fully utilize the computing resources of different AI cards in the AI ​​card cluster, rationally allocating communication tasks or data blocks according to their respective computing power differences. Its core idea is to avoid evenly distributing all tasks, instead allocating more tasks or larger data blocks to AI cards with stronger computing power, and fewer tasks or smaller data blocks to AI cards with relatively weaker computing power, thereby avoiding the "weakest link" effect and improving overall task processing efficiency. During implementation, the system can continuously monitor the performance indicators of each AI card, such as CPU utilization, GPU utilization, memory usage, and processing speed, and evaluate its computing power accordingly. Based on these evaluation results, a weighted allocation strategy can be adopted, where the weight is proportional to the computing power of the AI ​​card, ensuring that task allocation matches the actual processing power of the AI ​​card, ultimately generating heterogeneous task allocation strategy data containing the task allocation scheme.

[0080] In this embodiment, furthermore, adaptive strategy selection processing is performed based on message size information in the system status data to generate pipeline processing strategy data. Adaptive strategy selection processing refers to dynamically selecting the most suitable pipeline processing strategy according to the specific size of the message to be communicated, in order to optimize message transmission and processing efficiency. Different message sizes may have different requirements for preprocessing methods. For example, for smaller messages, simple serialization may be sufficient; while for larger messages, complex processing such as efficient compression, chunking, and encryption may be required to reduce transmission time and network load. In practice, the system can predefine multiple pipeline processing strategies, each optimized for a specific range of message sizes. After obtaining message size information, the system compares it with a preset threshold to intelligently select the processing strategy that best matches the current message size. These strategies can cover multiple aspects such as whether to compress, which compression algorithm to use, whether to chunk, chunk size, and whether to encrypt, ultimately forming pipeline processing strategy data to guide pipeline processing.

[0081] In this embodiment, the above technical solution enables the generation of targeted optimization strategies when performing heterogeneous perception and dynamic decision-making processing on system status data. Specifically, by performing dynamic topology reconstruction based on link bandwidth and network congestion information, the system can perceive and respond to changes in network conditions in real time, effectively avoiding congested paths and selecting the current optimal transmission path, thereby significantly improving the efficiency and reliability of message transmission. Simultaneously, by performing heterogeneous load balancing based on differences in computing power, the system can fully utilize the AI ​​card resources with different computing capabilities within the AI ​​card cluster, avoiding overall efficiency degradation due to performance bottlenecks in some AI cards, and achieving more efficient and balanced task allocation. Furthermore, by performing adaptive strategy selection based on message size information, the system can dynamically select the most suitable pipeline processing method according to the actual characteristics of the message, avoiding unnecessary processing overhead and further optimizing message processing efficiency. These dynamically generated and adaptively adjusted strategies enable the entire AI card message communication system to better adapt to heterogeneous environments and dynamic changes, thereby improving the overall performance and resource utilization of message communication between AI cards.

[0082] In one feasible implementation, the step of performing dynamic topology reconstruction processing based on the link bandwidth information and network congestion information in the system status data to generate the dynamic routing policy data includes: performing link quality assessment processing based on the real-time bandwidth information, latency information, and bit error rate information of each link in the system status data to obtain link quality assessment data; performing weighted topology construction processing based on the link quality assessment data and the current communication task information in the system status data to generate a weighted communication directed acyclic graph; and performing conflict detection and replanning processing on the weighted communication directed acyclic graph based on the network congestion information in the system status data to generate a real-time optimal weighted directed acyclic graph to constitute the dynamic routing policy data.

[0083] In this embodiment, the link quality assessment process aims to comprehensively consider the real-time performance indicators of each interconnected link in the AI ​​card cluster to obtain a comprehensive evaluation of the link's communication capabilities. Real-time bandwidth information reflects the data transmission rate available to the link at the current moment; latency information represents the time required for a data packet to travel from the source to the destination, and is a key indicator for measuring the real-time performance of communication; bit error rate information reflects the reliability of data transmission, i.e., the probability of data errors occurring during transmission. By collecting and analyzing this multi-dimensional information, the health status and carrying capacity of each link can be assessed more accurately. For example, a weighted average method can be used, assigning different weights to bandwidth, latency, and bit error rate for a comprehensive score, or a machine learning model can be used to train historical data to predict the overall quality of the current link. The obtained link quality assessment data can be a comprehensive score, a multi-dimensional vector, or a status identifier, used for subsequent routing decisions.

[0084] In this embodiment, the weighted topology construction process aims to build a network model that reflects communication costs and priorities based on the actual quality of the links and the characteristics of the communication tasks. Link quality assessment data provides the "goodness" or "badness" of each link, while current communication task information (e.g., task type, message size, priority, etc.) defines the communication requirements. By converting link quality assessment data into link weights and adjusting these weights in conjunction with the priority or characteristics of the communication tasks, a weighted directed acyclic graph (DAG) for communication can be generated. In this graph, each node represents an AI card, each edge represents an interconnection link, and the edge weight comprehensively reflects the link's quality and task requirements. For example, for high-priority tasks, the weight of high-quality links on their paths can be reduced, making them easier to select; for tasks with large message volumes, high-bandwidth, low-latency links may be preferred. This graph provides the foundation for subsequent routing path selection.

[0085] In this embodiment, conflict detection and replanning aim to address congestion and conflict issues caused by multiple communication tasks simultaneously competing for limited network resources in a dynamically changing AI card cluster environment. Network congestion information identifies which links or nodes in the current network are experiencing resource bottlenecks. By analyzing network congestion information on the constructed weighted directed acyclic graph (DAG), potential routing conflict points can be identified, such as multiple paths simultaneously traversing the same congested link. Conflict detection can employ methods such as traffic prediction, resource reservation, or real-time monitoring. Once a conflict is detected, the system performs replanning, such as adjusting the weights of affected paths, finding alternative paths, or scheduling communication tasks to avoid or mitigate congestion. The goal of replanning is to generate a real-time optimal weighted DAG, ensuring that all communication tasks can be transmitted along the optimal path under current network conditions, thereby forming the final dynamic routing policy data.

[0086] In this embodiment, by introducing link quality assessment processing, this application can comprehensively and accurately assess the quality of interconnection links in the AI ​​card cluster by considering multiple dimensions such as real-time bandwidth information, latency information, and bit error rate information, overcoming the limitations of relying on only a single or a few indicators to evaluate links. Based on this, weighted topology construction processing is performed in conjunction with current communication task information, so that the generated weighted directed acyclic graph not only reflects the objective quality of the links but also incorporates the specific needs and priorities of the communication tasks, providing a more refined basis for subsequent routing decisions. Furthermore, by performing conflict detection and replanning processing on network congestion information, this application can effectively identify and resolve resource contention and congestion problems that may be caused by concurrent multi-task communication, dynamically adjusting routing paths to generate a real-time optimal weighted directed acyclic graph. This ensures that in the complex and ever-changing AI card cluster environment, communication messages can always be transmitted through the path most suitable for the current network conditions and task requirements, significantly improving the efficiency, reliability, and real-time performance of message communication, effectively avoiding communication bottlenecks and data loss caused by improper routing, thereby optimizing the message communication performance between AI cards.

[0087] In one feasible implementation, the step of generating a weighted directed acyclic graph (DAG) based on the link quality assessment data and the current communication task information in the system status data includes: obtaining task priority information from the current communication task information in the system status data to determine the priority level of the communication task; setting a corresponding link weight coefficient for each priority level; and performing weighted calculation based on the link quality assessment data and the link weight coefficient to generate the weighted DAG.

[0088] In this embodiment, task priority information refers to a quantitative representation of the importance or urgency of the current communication task. This information can be explicitly specified by the upper-layer application based on factors such as task type (e.g., model parameter synchronization, gradient aggregation, log transmission), task deadline, or user configuration, or it can be dynamically evaluated by the system based on the task's impact on the overall AI training or inference progress. For example, gradient aggregation tasks may be assigned a high priority, while log transmission tasks may be assigned a lower priority. The system parses the current communication task information, identifies and extracts these priority identifiers, and maps them to a predefined set of priority levels, such as "highest," "high," "medium," and "low," thereby providing a basis for subsequent differentiated processing.

[0089] In this embodiment, the link weight coefficient is a numerical factor used to adjust the "cost" of a communication link, and its magnitude directly affects the routing algorithm's preference for that link. For different task priority levels, a set of corresponding link weight coefficients can be preset or dynamically adjusted. For example, for the "highest" priority task, a smaller link weight coefficient can be set, making the link "better" for the high-priority task even with slightly lower link quality; while for the "low" priority task, a larger link weight coefficient can be set, making it more inclined to use links that are less critical to the high-priority task. These coefficients can be finely configured according to the actual network environment, AI application requirements, and performance goals to achieve differentiated routing for tasks of different priorities.

[0090] Furthermore, the weighted calculation process combines the objective physical quality of the link (reflected by link quality assessment data) with the subjective priority of the task (reflected by link weight coefficients). Specifically, multiplication, addition, or other composite functions can be used to calculate the final weight of each link. For example, link quality assessment data (such as latency, inverse bandwidth, etc.) can be multiplied by the link weight coefficient corresponding to the task priority level to obtain the "effective cost" of the link under a specific task priority. In this way, high-priority tasks will prioritize links with good physical quality and assigned lower effective costs when selecting paths, while low-priority tasks will avoid these "expensive" links. The resulting weighted directed acyclic graph of communication has each edge (link) with a weight value that combines physical quality and task priority, providing a more refined and intelligent decision-making basis for subsequent path selection.

[0091] In this embodiment, the above technical solution fully considers the priorities of different communication tasks when constructing a weighted directed acyclic graph (DAG). By obtaining task priority information from system status data and setting corresponding link weight coefficients for each priority level, the final weight of the link not only reflects its physical quality but also incorporates the importance of the task. Weighted calculation based on link quality assessment data and link weight coefficients generates a task priority-aware weighted DAG. This allows high-priority tasks to preferentially select better transmission paths during subsequent dynamic routing, ensuring effective protection even when network resources are scarce. This significantly reduces communication latency for critical tasks and improves the overall communication efficiency and timeliness of task completion in the AI ​​card cluster. This refined weighting strategy makes dynamic topology reconstruction and routing decisions more intelligent and efficient, better adapting to the high requirements of AI applications for real-time communication and reliability.

[0092] In one feasible implementation, the step of performing heterogeneous load balancing processing based on the computing power difference information in the system status data to generate the heterogeneous task allocation strategy data includes: performing fast and slow card identification processing based on the computing status information of each AI card in the system status data to obtain fast card identification information and slow card identification information; performing non-uniform task allocation processing based on the fast card identification information and the slow card identification information to generate a data block allocation scheme; and performing task scheduling planning processing on the data block allocation scheme to generate the heterogeneous task allocation strategy data; the heterogeneous task allocation strategy data includes information on allocating data blocks of different sizes to AI cards with different computing capabilities.

[0093] In this embodiment, when performing fast / slow card identification processing based on the computational status information of each AI card in the system status data to obtain fast and slow card identification information, real-time performance indicators of each AI card can be collected periodically, such as floating-point operation capability (FLOPS), memory bandwidth, GPU utilization, completion time of specific benchmark tasks, or processing speed of actual workloads. The system can preset performance thresholds, identifying AI cards with performance indicators higher than a certain threshold as fast cards and AI cards with performance indicators lower than another threshold as slow cards. Alternatively, a machine learning model can be used to dynamically classify AI cards into fast and slow categories by analyzing historical performance data and current system load. This identification processing can provide an accurate basis for subsequent non-uniform task allocation.

[0094] In this embodiment, when performing non-uniform task allocation processing based on fast card and slow card identification information to generate a data block allocation scheme, the system designs a non-uniform data block allocation strategy according to the identified fast and slow cards. For example, the original communication message data can be divided into data blocks of different sizes according to the computing power ratio of the AI ​​cards, and allocated proportionally to the corresponding AI cards. If the computing power of a fast card is twice that of a slow card, the fast card may be allocated twice the size or number of data blocks as the slow card. Alternatively, a weighted round-robin mechanism can be used to assign higher weights to fast cards, giving them more allocation opportunities during data block allocation. This non-uniform allocation aims to maximize the utilization of the computing potential of each AI card and avoid resource idleness or overload caused by capability mismatch.

[0095] In this embodiment, the data block allocation scheme is processed by task scheduling planning to generate the heterogeneous task allocation strategy data. This step transforms the abstract data block allocation scheme into executable scheduling instructions. This may involve determining the specific transmission time, processing order, and binding relationship of each data block with a specific AI card. The scheduling plan can consider data dependencies, communication latency, and the real-time load of the AI ​​cards to optimize the overall task execution process. For example, a priority scheduling algorithm can be used to ensure that critical data blocks are preferentially allocated to fast cards for processing, or predictive scheduling can be used to pre-transmit data blocks to fast cards that are about to become idle. The finally generated heterogeneous task allocation strategy data explicitly contains specific information on allocating data blocks of different sizes to AI cards with different computing capabilities. For example, it can be a list or data structure containing fields such as AI card ID, allocated data block size, and expected processing time.

[0096] In this embodiment, the above-described technical solution performs fast / slow card identification processing on the computing status information of each AI card in the AI ​​card cluster. This accurately obtains the differences in computing power among different AI cards, thereby generating fast card identification information and slow card identification information. Based on this refined identification information, non-uniform task allocation processing is performed, allowing AI cards with strong computing power to handle larger or more task data blocks, while AI cards with weak computing power are allocated smaller or fewer task data blocks. Subsequently, through task scheduling planning processing, this non-uniform data block allocation scheme is transformed into specific heterogeneous task allocation strategy data. This strategy data explicitly indicates the allocation of different sized data blocks to AI cards with different computing capabilities. This refined task allocation mechanism effectively avoids the performance bottlenecks and resource waste that may be caused by traditional uniform allocation or generalized load balancing, ensuring the full utilization of heterogeneous computing resources in the AI ​​card cluster and improving the overall efficiency and throughput of message communication between AI cards.

[0097] In one feasible implementation, the step of performing adaptive strategy selection processing based on message size information in the system state data to generate the pipeline processing strategy data includes: obtaining message size information from the system state data; comparing the message size information with a preset threshold and performing strategy classification processing to obtain a strategy classification result; and selecting a corresponding processing strategy from a variety of predefined processing strategies based on the strategy classification result to generate the pipeline processing strategy data.

[0098] In this embodiment, obtaining message size information from the system state data is fundamental to adaptive strategy selection. Message size is a key factor affecting communication efficiency and processing complexity. For example, when an AI card performs model parameter synchronization or gradient aggregation, the communication task information usually includes the size of the data block to be transmitted. The system state data collection module extracts message size information from these sources in real time or periodically to ensure the accuracy of subsequent strategy selection. Subsequently, the message size information is compared with a preset threshold to perform strategy classification processing and obtain strategy classification results. This step is the core of adaptive strategy selection. By discretizing continuous message size information into different categories, it provides a basis for selecting appropriate processing strategies. The preset threshold can be determined based on experience, simulation, or machine learning models. For example, multiple thresholds T1, T2, T3, etc., can be set to classify messages into categories such as "small message" (less than T1), "medium message" (between T1 and T2), "large message" (between T2 and T3), and "extra-large message" (greater than T3). The comparison can be a simple numerical comparison, and the strategy classification processing can be a lookup table, decision tree, or rule-based system.

[0099] Based on the strategy classification results, a corresponding processing strategy is selected from multiple predefined processing strategies to generate the pipeline processing strategy data. This step dynamically selects the most suitable pipeline processing strategy for the current message type based on the message classification results to optimize communication efficiency. Predefined processing strategies can include various combinations; for example, for small messages, the focus might be on low latency, choosing direct transmission or only lightweight serialization; for medium-sized messages, it might include serialization, light compression, and simple chunked transmission; for large messages, it might involve efficient serialization, high compression ratios, multi-level chunking, and complex scheduling strategies, such as overlapping communication and computation. The selection process can be a mapping function that maps the strategy classification results to specific pipeline processing strategy IDs or configuration parameter sets. The generated pipeline processing strategy data contains the specific configuration of the selected strategy, such as whether compression is enabled, compression algorithm type, chunk size, serialization method, etc.

[0100] In this embodiment, through the above technical solution, this application can dynamically select the most suitable pipeline processing strategy based on the actual message size. Specifically, by obtaining message size information and comparing it with a preset threshold, messages are divided into different categories, thereby avoiding a "one-size-fits-all" processing approach for all messages. For example, for small messages, a low-latency, lightweight processing strategy can be selected to reduce unnecessary overhead; for large messages, more complex compression, chunking, and scheduling strategies can be enabled to maximize bandwidth utilization and throughput. This adaptive strategy selection mechanism ensures that pipeline processing can be optimized for the characteristics of different communication tasks, significantly improving the overall efficiency and response speed of AI card cluster message communication, and effectively solving the problem of low pipeline processing efficiency caused by differences in message size.

[0101] In one feasible implementation, based on the dynamic routing policy data, the heterogeneous task allocation policy data, and the pipelined processing policy data, adaptive pipelined processing and dynamic routing processing are performed on the original communication message data that needs to communicate to obtain optimized communication message data. This includes: based on the pipelined processing policy data, performing dynamic assembly processing of configurable processing units to generate adaptive pipelined configuration data; based on the adaptive pipelined configuration data, performing serialization, compression, and scheduling processing on the original communication message data that needs to communicate to obtain pipelined message data; based on the dynamic routing policy data, selecting the real-time optimal transmission path for the pipelined message data to generate path selection data; and based on the heterogeneous task allocation policy data, performing data block allocation processing on the pipelined message data to obtain the optimized communication message data.

[0102] In this embodiment, dynamic assembly of configurable processing units is performed based on pipeline processing strategy data. This aims to flexibly construct a message processing pipeline according to a pre-generated pipeline processing strategy, adapting to the needs of different message types and system states. The pipeline processing strategy data may contain instructions for a series of processing steps, such as whether compression, encryption, or fragmentation is required. Configurable processing units can be independent software modules or hardware accelerators, such as data compression modules, data encryption modules, message fragmentation modules, and protocol encapsulation modules. Dynamic assembly refers to selecting and connecting these processing units at runtime according to the strategy data to form a customized message processing flow. For example, small messages may only require simple serialization; large messages may require compression, fragmentation, and encryption.

[0103] In this embodiment, based on adaptive pipeline configuration data, the original communication message data requiring communication is serialized, compressed, and scheduled to obtain pipelined message data. Its function is to preprocess the original message data according to the dynamically assembled pipeline configuration, making it more suitable for transmission and subsequent processing. Serialization converts data structures in memory into a transmittable byte stream format, which can use standard or custom protocols such as Protobuf, JSON, and XML. Compression aims to reduce the size of message data to save bandwidth and transmission time. Commonly used compression algorithms include Zlib, LZ4, and Snappy. The degree of compression and algorithm selection can be based on the instructions in the pipeline configuration data. Scheduling may include message priority marking, timestamp addition, and fragment management to prepare for subsequent routing and transmission. For example, large messages may be divided into multiple small data blocks, and each data block may be assigned a sequence number and a target AI card identifier.

[0104] In this embodiment, based on dynamic routing policy data, the system selects the optimal real-time transmission path for pipelined message data to generate path selection data. The purpose is to choose the best transmission path for the processed message data based on the real-time changing network conditions of the AI ​​card cluster, avoiding congested or inefficient links. The dynamic routing policy data includes real-time topology information of the AI ​​card cluster, link quality assessment data, and recommended routing paths. Path selection can be based on various routing algorithms, such as shortest path algorithms (Dijkstra, Bellman-Ford), traffic-aware routing algorithms, and multi-path routing algorithms. The system calculates the optimal real-time path from the source AI card to the target AI card based on the link weights and congestion information provided in the policy data, and generates path selection data containing this path information.

[0105] In this embodiment, based on heterogeneous task allocation strategy data, the pipelined message data undergoes data block allocation processing to obtain optimized communication message data. This process takes into account the differences in computing power among the AI ​​cards in the AI ​​card cluster, distributing the pipelined message data non-uniformly to achieve load balancing and prevent slower cards from becoming bottlenecks. The heterogeneous task allocation strategy data contains information on how to allocate data blocks to AI cards with different computing capabilities. For example, AI cards with strong computing power can be allocated larger or more data blocks, while AI cards with weaker computing power are allocated smaller data blocks. The data block allocation processing further refines or recombines the pipelined message data (which may have been fragmented) according to these strategies, forming data blocks suitable for the processing capabilities of different AI cards, thereby generating the final optimized communication message data.

[0106] In this embodiment, through the above technical solution, this application organically integrates pre-generated pipeline processing strategies, dynamic routing strategies, and heterogeneous task allocation strategies into the actual message processing flow. The dynamic assembly of configurable processing units enables message preprocessing to adaptively adjust according to message characteristics and system requirements, ensuring optimal message preparation before transmission. Real-time selection of the optimal transmission path effectively avoids network congestion and inefficient links, significantly reducing message transmission latency. Simultaneously, the non-uniform distribution of data blocks based on the heterogeneous task allocation strategy fully considers the differences in AI card computing capabilities, achieving intelligent load balancing and preventing slow cards from becoming system bottlenecks, thereby improving overall communication throughput and resource utilization. This comprehensive integration and adaptive processing mechanism ensures that raw communication message data can be efficiently and intelligently optimized and transmitted in the heterogeneous AI card cluster, significantly improving message communication efficiency and system performance between AI cards.

[0107] In one feasible implementation, the step of dynamically assembling configurable processing units based on the pipeline processing strategy data to generate adaptive pipeline configuration data includes: obtaining processing strategy information from the pipeline processing strategy data; selecting a corresponding processing unit from multiple configurable processing units based on the processing strategy information; and dynamically assembling the selected processing units according to a preset processing order to generate the adaptive pipeline configuration data.

[0108] In this embodiment, obtaining processing strategy information from the pipeline processing strategy data means that the pipeline processing strategy data is generated based on the system status data of the AI ​​card cluster (especially message size information) after adaptive strategy selection processing. It contains specific guidance for pipeline processing of the original communication message data. Obtaining the processing strategy information involves parsing detailed instructions from this strategy data to guide pipeline construction and operation. For example, these instructions may include whether data compression is required, which compression algorithm to use, whether data encryption is required, whether message fragmentation is required, and the fragment size, etc. Accurately obtaining this information provides a precise basis for subsequent pipeline configuration.

[0109] In this embodiment, selecting a corresponding processing unit from multiple configurable processing units based on the processing strategy information means that each configurable processing unit is a pre-designed and implemented series of modular functional components. Each component is responsible for performing a specific operation in the pipeline processing, such as a data serialization unit, a data compression unit, a data encryption unit, and a data fragmentation unit. Based on the acquired processing strategy information, the system intelligently selects one or more processing units that match the current strategy requirements from this processing unit library. For example, if the strategy information indicates that data compression is required, a data compression unit will be selected; if it indicates that message fragmentation is required, a message fragmentation unit will be selected. This selection mechanism ensures that the pipeline can load the necessary functional modules according to actual needs.

[0110] In this embodiment, the selected processing units are dynamically assembled according to a preset processing order to generate the adaptive pipeline configuration data. This dynamic assembly process connects and arranges the selected processing units from the previous step according to a pre-defined logical order, forming a complete and executable pipeline structure. The preset processing order typically follows general data processing logic; for example, data serialization is usually performed first, followed by compression, then encryption, and finally fragmentation. This assembly process is dynamic, meaning that the specific structure and function of the pipeline can be flexibly adjusted based on real-time processing strategy information, rather than using a fixed pipeline template. The generated adaptive pipeline configuration data details the current pipeline structure, the parameters of each processing unit, and their connection relationships, providing an executable configuration for subsequent pipeline processing of the original communication message data.

[0111] In this embodiment, through the above technical solution, this application can flexibly select and dynamically assemble the most suitable pipeline for the current communication task and system state from a preset processing unit library based on dynamically changing pipeline processing strategy data. Specifically, by obtaining processing strategy information from the pipeline processing strategy data, it is possible to accurately understand which processing operations are required for the current communication message. Subsequently, based on this processing strategy information, the corresponding functional module is selected from multiple configurable processing units, avoiding the loading of unnecessary functions and thus reducing resource consumption. Finally, the selected processing units are dynamically assembled according to a preset processing order to generate adaptive pipeline configuration data, enabling the pipeline to be adjusted in real time according to factors such as message size. For example, complex compression steps can be skipped for small messages, while efficient compression and fragmentation processing can be enabled for large messages. This dynamic and adaptive pipeline configuration capability significantly improves the flexibility and processing efficiency of AI card message communication, ensuring that the communication process can optimize resource utilization and performance according to the actual situation, thereby effectively solving the challenge that fixed pipeline configuration cannot adapt to dynamic communication needs.

[0112] In one feasible implementation, the step of performing data block allocation processing on the pipelined message data based on the heterogeneous task allocation strategy data to obtain the optimized communication message data includes: obtaining information from the heterogeneous task allocation strategy data on allocating data blocks of different sizes to AI cards with different computing capabilities; based on the information on allocating data blocks of different sizes to AI cards with different computing capabilities, performing non-uniform data block partitioning processing on the pipelined message data to generate a non-uniform data block set; and allocating data blocks of different sizes in the non-uniform data block set to the corresponding AI cards to obtain the optimized communication message data.

[0113] In this embodiment, obtaining information from the heterogeneous task allocation strategy data to allocate data blocks of different sizes to AI cards with different computing capabilities means that the system extracts specific guidance information on how to allocate data blocks according to the differences in computing capabilities of AI cards based on the pre-generated heterogeneous task allocation strategy. This information can be represented as a mapping relationship, for example, associating AI cards with different computing capability levels with corresponding data block size ratios or absolute values. For example, for AI cards with stronger computing capabilities, the strategy may instruct the allocation of larger data blocks; for AI cards with weaker computing capabilities, smaller data blocks are allocated. This step ensures that the subsequent data partitioning is dynamic and adaptable to the heterogeneity of AI cards.

[0114] In this embodiment, based on the information about allocating data blocks of different sizes to AI cards with different computing capabilities, the pipelined message data is divided into non-uniform data blocks to generate a non-uniform data block set. This means that after obtaining the allocation strategy information, the complete pipelined message data is actually divided. This division is not a simple equal division, but rather, according to the aforementioned strategy information, the message data is divided into multiple data blocks of different sizes according to a preset ratio or size. For example, if the strategy indicates that one AI card should process 30% of the total data volume and another AI card should process 20%, then the message data will be precisely divided into blocks of the corresponding ratio. This process can be implemented using byte offset, data structure boundary recognition, etc., ultimately forming a set containing multiple data blocks of different sizes.

[0115] In this embodiment, allocating data blocks of different sizes from the non-uniform data block set to corresponding AI cards to obtain the optimized communication message data refers to logically associating and assigning each data block of a specific size to a target AI card after the data block partitioning is completed. This typically involves adding metadata to each data block, such as the identifier of the target AI card, the sequence number of the data block (for reassembly on the target AI card), and the actual size of the data block. These non-uniformly partitioned data blocks with allocation information collectively constitute the optimized communication message data, which is ready to be transmitted to their respective target AI cards according to a dynamic routing strategy.

[0116] In this embodiment, through the above technical solution, this application can fully utilize the computing resources of a heterogeneous AI card cluster. By obtaining information from the heterogeneous task allocation strategy data on allocating data blocks of different sizes to AI cards with different computing capabilities, and based on this information, performing non-uniform data block partitioning on the pipelined message data, it can be ensured that AI cards with strong computing capabilities receive more data processing tasks, while AI cards with weak computing capabilities undertake a smaller workload. This non-uniform data block allocation method avoids performance bottlenecks caused by uniform allocation, allowing each AI card to operate under its optimal load, thereby maximizing the overall processing efficiency and communication throughput of the AI ​​card cluster. Finally, these non-uniform data blocks are allocated to the corresponding AI cards, enabling message data to be transmitted and processed between heterogeneous AI cards in a more balanced and efficient manner, improving the message communication optimization effect of the AI ​​card cluster.

[0117] In one specific implementation, a distributed deep learning training task is executed within a large-scale AI card cluster containing eight AI cards. This task involves frequent gradient synchronization, specifically manifested as AllReduce aggregate communication operations. The AI ​​cards in this cluster have varying hardware configurations, with some having strong computing power and others weaker. Furthermore, the quality of interconnect links and network congestion within the cluster dynamically change over time. In existing technologies, communication paths typically employ fixed topologies, uniform data partitioning, and pre-defined pipeline processing strategies, leading to low communication efficiency, especially in heterogeneous environments and dynamic network conditions.

[0118] This method first initiates a continuous monitoring and decision-making process. The system periodically, or when triggered by specific events, collects computational status information (such as GPU utilization and memory usage), interconnect link status information (such as PCIe bandwidth and link latency), and network status information (such as network interface card queue depth and packet loss rate) for each AI card to obtain raw status data. Subsequently, this raw status data is integrated and updated in real time to generate AI card cluster status information. For example, the system identifies AI cards 1 and 2 as having higher computational capabilities, while AI cards 3 and 4 have relatively lower computational capabilities, constituting information about computational capability differences. Simultaneously, the system monitors that the link bandwidth between AI cards 5 and 6 is 100Gbps, but the link between AI cards 7 and 8 currently experiences high network congestion, such as a packet loss rate of 5%. Furthermore, the system also collects information that the current required aggregate communication operation type is AllReduce, with a message size of 100MB, constituting current communication task information. Finally, this AI card cluster status information and current communication task information are comprehensively processed to generate comprehensive system status data.

[0119] In this embodiment, the system then performs heterogeneous perception and dynamic decision-making processing on these system state data to generate three types of key strategy data.

[0120] For dynamic routing strategy data, the system performs dynamic topology reconstruction based on link bandwidth and network congestion information from system status data. Specifically, the system assesses link quality based on real-time bandwidth, latency, and bit error rate information for each link, obtaining link quality assessment data. For example, the link between AI card 5 and AI card 6 is assessed as a high-quality link, while the link between AI card 7 and AI card 8 is assessed as a low-quality link due to a high packet loss rate. Simultaneously, the system obtains that the AllReduce task has high priority from the current communication task information. The system sets lower link weight coefficients for high-priority tasks and constructs a weighted directed acyclic graph (DAG) based on the link quality assessment data and link weight coefficients. Furthermore, the system combines network congestion information to perform conflict detection and replanning on this graph. For example, if multiple high-priority communication paths may simultaneously pass through a congested node, the system replans the paths to avoid conflicts, ultimately generating a real-time optimal weighted DAG, constituting the dynamic routing strategy data. Compared to existing fixed topologies, this dynamic routing strategy can adapt to network changes in real time, effectively avoiding congestion and reducing communication latency.

[0121] For heterogeneous task allocation strategy data, the system performs heterogeneous load balancing based on the computing power differences in the system status data. According to the computing status information of each AI card, the system identifies AI card 1 and AI card 2 as "fast cards," and AI card 3 and AI card 4 as "slow cards." Based on the fast and slow card identification information, the system performs non-uniform task allocation processing to generate a data block allocation scheme. For example, for a 100MB original message, the system might allocate 30MB to AI card 1, 30MB to AI card 2, and 20MB to AI card 3 and 20MB to AI card 4. This data block allocation scheme is processed by task scheduling planning to generate heterogeneous task allocation strategy data, which explicitly includes information on allocating different sized data blocks to AI cards with different computing capabilities. This non-uniform allocation method effectively solves the problem in existing technologies where uniform data partitioning causes slow cards to drag down overall efficiency.

[0122] For pipelined processing strategy data, the system performs adaptive strategy selection based on message size information in the system status data. The system receives a message size of 100MB and compares it to a preset threshold (e.g., 64MB). Since 100MB exceeds the threshold, the system performs strategy classification and selects a processing strategy suitable for large messages. This strategy may include enabling high-intensity compression and fine-grained chunking. Based on this strategy classification result, the system selects the corresponding processing strategy from a variety of predefined processing strategies to generate pipelined processing strategy data. This overcomes the limitation of existing fixed or preset pipelined strategies that cannot adapt to message characteristics.

[0123] In this embodiment, after acquiring dynamic routing strategy data, heterogeneous task allocation strategy data, and pipeline processing strategy data, the system begins to perform adaptive pipeline processing and dynamic routing processing on the original communication message data that needs to be communicated.

[0124] First, based on pipeline processing strategy data, the system dynamically assembles configurable processing units to generate adaptive pipeline configuration data. The system obtains processing strategy information from the pipeline processing strategy data, such as "high-intensity compression" and "fine-grained chunking." Based on this information, the system selects corresponding processing units from multiple configurable processing units, such as "serialization units," "lossless compression units," and "message chunking units." These selected processing units are dynamically assembled according to a preset processing order. Subsequently, the original 100MB communication message data undergoes serialization, compression (e.g., compressed to 60MB), and scheduling processing through this dynamically assembled pipeline to obtain pipelined message data.

[0125] Next, based on the dynamic routing strategy data, the system selects the optimal real-time transmission path for these pipelined message data, generating path selection data. For example, for data sent to AI Card 1, the system will select the least congested and lowest latency path.

[0126] Finally, based on the heterogeneous task allocation strategy data, the system performs data block allocation processing on the pipelined message data. The system obtains information from the heterogeneous task allocation strategy data regarding the allocation of data blocks of different sizes to AI cards with different computing capabilities (e.g., AI card 1 and AI card 2 each process 18MB of compressed data, and AI card 3 and AI card 4 each process 12MB of compressed data). The system divides the 60MB of pipelined message data into non-uniform data block sets, generates a non-uniform data block set, and allocates data blocks of different sizes in the set to the corresponding AI cards, thereby obtaining optimized communication message data.

[0127] Ultimately, these optimized communication message data are transmitted within the AI ​​card cluster via dynamically selected transmission paths to complete message communication between AI cards. In this way, the proposed method comprehensively considers the heterogeneity of the AI ​​card cluster, network dynamics, and message characteristics, achieving dynamic optimization of communication paths, intelligent load balancing, and adaptive adjustment of pipelined processing. This significantly improves the communication performance of large-scale AI clusters and solves problems such as high communication latency, low bandwidth utilization, and slow card effects in existing technologies.

[0128] In the embodiments of this application, the AI ​​card message communication optimization method based on heterogeneous sensing pipeline and dynamic routing effectively solves the problem of low communication efficiency in the prior art by collecting system status data, generating dynamic policy data and performing adaptive processing. It can dynamically adapt to network changes and heterogeneous computing environments, improve communication efficiency, reduce latency, and optimize resource utilization.

[0129] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0130] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. All equivalent structural transformations made under the technical concept of this application using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included within the scope of patent protection of this application.

Claims

1. A method for optimizing AI card message communication based on heterogeneous sensing pipeline and dynamic routing, characterized in that, The method includes: Collect AI card cluster status information and current communication task information to generate system status data; The system status data is subjected to heterogeneous perception and dynamic decision-making processing to generate dynamic routing strategy data, heterogeneous task allocation strategy data, and pipeline processing strategy data. Based on the dynamic routing strategy data, the heterogeneous task allocation strategy data, and the pipeline processing strategy data, adaptive pipeline processing and dynamic routing processing are performed on the original communication message data that needs to be communicated to obtain optimized communication message data. The optimized communication message data is transmitted in the AI ​​card cluster through a dynamically selected transmission path to complete message communication between AI cards; The steps of performing heterogeneous sensing and dynamic decision-making processing on the system state data to generate dynamic routing strategy data, heterogeneous task allocation strategy data, and pipeline processing strategy data include: Based on the link bandwidth information and network congestion information in the system status data, dynamic topology reconstruction is performed to generate the dynamic routing policy data. Based on the computing power difference information in the system status data, heterogeneous load balancing is performed to generate the heterogeneous task allocation strategy data. Based on the message size information in the system status data, adaptive strategy selection is performed to generate the pipeline processing strategy data.

2. The AI ​​card message communication optimization method based on heterogeneous sensing pipeline and dynamic routing as described in claim 1, characterized in that, The steps for collecting AI card cluster status information and current communication task information to generate system status data include: Periodically or event-triggered, collect the computing status information, interconnection link status information, and network status information of each AI card to obtain raw status data; The original state data is integrated and updated in real time to generate AI card cluster state information, which includes computing power difference information, link bandwidth information and network congestion information. Collect information on the type of communication operation to be executed and the message size to generate current communication task information; The AI ​​card cluster status information and the current communication task information are combined and processed to generate the system status data.

3. The AI ​​card message communication optimization method based on heterogeneous sensing pipeline and dynamic routing as described in claim 1, characterized in that, The steps for generating the dynamic routing policy data by performing dynamic topology reconstruction based on the link bandwidth information and network congestion information in the system status data include: Based on the real-time bandwidth, latency, and bit error rate information of each link in the system status data, link quality assessment is performed to obtain link quality assessment data. Based on the current communication task information in the link quality assessment data and the system status data, a weighted topology construction process is performed to generate a weighted communication directed acyclic graph. Based on the network congestion information in the system status data, conflict detection and replanning are performed on the weighted communication directed acyclic graph to generate a real-time optimal weighted directed acyclic graph, which constitutes the dynamic routing strategy data.

4. The AI ​​card message communication optimization method based on heterogeneous sensing pipeline and dynamic routing as described in claim 3, characterized in that, The steps for generating a weighted directed acyclic graph (DAG) based on the current communication task information in the link quality assessment data and the system status data include: Obtain task priority information from the current communication task information from the system status data to determine the priority level of the communication task; Set a corresponding link weight coefficient for each priority level; Based on the link quality assessment data and the link weight coefficients, a weighted calculation is performed to generate the weighted directed acyclic graph of communication.

5. The AI ​​card message communication optimization method based on heterogeneous sensing pipeline and dynamic routing as described in claim 1, characterized in that, The steps for generating heterogeneous task allocation strategy data by performing heterogeneous load balancing processing based on the computing power difference information in the system status data include: Based on the calculation status information of each AI card in the system status data, fast and slow card identification processing is performed to obtain fast card identification information and slow card identification information; Based on the fast card identification information and the slow card identification information, non-uniform task allocation processing is performed to generate a data block allocation scheme; The data block allocation scheme is processed by task scheduling planning to generate the heterogeneous task allocation strategy data; the heterogeneous task allocation strategy data includes information on allocating data blocks of different sizes to AI cards with different computing capabilities.

6. The AI ​​card message communication optimization method based on heterogeneous sensing pipeline and dynamic routing as described in claim 1, characterized in that, The steps for generating the pipeline processing strategy data by performing adaptive strategy selection based on the message size information in the system status data include: Obtain message size information from the system status data; The message size information is compared with a preset threshold to perform strategy classification processing and obtain the strategy classification result. Based on the strategy classification results, a corresponding processing strategy is selected from a variety of predefined processing strategies to generate the pipeline processing strategy data.

7. The AI ​​card message communication optimization method based on heterogeneous sensing pipeline and dynamic routing as described in claim 1, characterized in that, Based on the dynamic routing strategy data, the heterogeneous task allocation strategy data, and the pipelined processing strategy data, adaptive pipelined processing and dynamic routing processing are performed on the original communication message data that needs to be communicated, to obtain optimized communication message data, including: Based on the pipeline processing strategy data, dynamic assembly of configurable processing units is performed to generate adaptive pipeline configuration data. Based on the adaptive pipeline configuration data, the original communication message data that needs to be communicated is serialized, compressed, and scheduled to obtain pipelined message data. Based on the dynamic routing strategy data, the optimal real-time transmission path is selected for the message data that has undergone pipeline processing, so as to generate path selection data. Based on the heterogeneous task allocation strategy data, the message data that has undergone pipeline processing is processed into data blocks to obtain the optimized communication message data.

8. The AI ​​card message communication optimization method based on heterogeneous sensing pipeline and dynamic routing as described in claim 7, characterized in that, The step of dynamically assembling configurable processing units based on the pipeline processing strategy data to generate adaptive pipeline configuration data includes: Obtain processing strategy information from the pipeline processing strategy data; Based on the processing strategy information, a corresponding processing unit is selected from multiple configurable processing units; The selected processing units are dynamically assembled according to a preset processing order to generate the adaptive pipeline configuration data.

9. The AI ​​card message communication optimization method based on heterogeneous sensing pipeline and dynamic routing as described in claim 7, characterized in that, Based on the heterogeneous task allocation strategy data, the step of performing data block allocation processing on the pipelined message data to obtain the optimized communication message data includes: Information on allocating data blocks of different sizes to AI cards with different computing capabilities is obtained from the heterogeneous task allocation strategy data; Based on the information that different sizes of data blocks are allocated to AI cards with different computing capabilities, the message data that has been pipelined is divided into non-uniform data blocks to generate a non-uniform data block set. Data blocks of different sizes in the non-uniform data block set are assigned to the corresponding AI cards to obtain the optimized communication message data.

Images