A processing method, apparatus and electronic device

Abstract

This application discloses a processing method, apparatus, and electronic device, relating to the field of artificial intelligence computing infrastructure technology. The disclosed method includes: obtaining communication and operational status information corresponding to each processing component in a cluster; determining a first communication topology between the processing components based on the communication and operational status information corresponding to each processing component; extracting features from the first communication topology using a first model to obtain data features corresponding to the first communication topology; the data features are used to provide an embedding vector containing global context for the first communication topology, and the first model is a graph neural network model; determining an adjustment strategy for adjusting the first communication topology using a second model based on reinforcement learning and the data features; adjusting the structure of the first communication topology based on the adjustment strategy to obtain a second communication topology; the communication performance of the second communication topology is higher than that of the first communication topology.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence computing infrastructure technology, and in particular to a processing method, apparatus and electronic device. Background Technology

[0002] Currently, communication between processing components such as GPUs (Graphics Processing Units) in AI (Artificial Intelligence) server clusters mainly adopts fixed topologies, such as NVLink fully interconnected topologies, multi-node hierarchical topologies, and dedicated topologies. However, fixed topologies suffer from a series of problems, including inability to adapt to dynamically changing communication modes, high limitations in static optimization, low efficiency of manual tuning, and insufficient fault tolerance. Solving at least some of these technical problems has become a research hotspot in this field. Summary of the Invention

[0003] Therefore, this application discloses the following technical solution:

[0004] A processing method includes:

[0005] Obtain the communication and operational status information of each processing component in the cluster;

[0006] Based on the communication and operation status information corresponding to each processing component, a first communication topology between the processing components is determined; different nodes in the first communication topology represent different processing components in the cluster, and the first communication topology represents the node characteristics of each node, the connection relationship between each node, and the communication characteristics.

[0007] The first communication topology is subjected to feature extraction using a first model to obtain data features corresponding to the first communication topology; the data features are used to provide an embedding vector containing global context for the first communication topology; the first model is a graph neural network model.

[0008] The second model determines an adjustment strategy for adjusting the first communication topology based on reinforcement learning and the data features.

[0009] The structure of the first communication topology is adjusted based on the adjustment strategy to obtain the second communication topology; the communication performance of the second communication topology is higher than that of the first communication topology.

[0010] Optionally, the communication and operational status information corresponding to the processing component can be processed, including the computational status information, communication status information, and task characteristics of the processing component.

[0011] Optionally, based on the communication and operational status information corresponding to each processing component, a first communication topology between the processing components is determined, including:

[0012] An undirected weighted graph is constructed with processing components as nodes, the computational state information and task characteristics of processing components as node characteristics, and the communication links represented by the communication state information of processing components as edges.

[0013] The target undirected weighted graph is used to characterize the first communication topology.

[0014] Optionally, the graph structure information of the target undirected weighted graph includes at least some of the following:

[0015] A set of nodes; different nodes in the set of nodes represent different processing components;

[0016] An edge set; the edges in the edge set represent the communication links between different nodes connected by the edges;

[0017] Node feature matrix; each row of the node feature matrix represents a feature vector of a processing component, and the feature vector of the processing component includes the feature vectors corresponding to the computational state information and task features of the processing component, respectively.

[0018] The adjacency matrix; the elements in the adjacency matrix represent the communication weights between different processing components with communication links, and the communication weights are determined based on the communication state information of the different processing components.

[0019] Optionally, feature extraction is performed on the first communication topology using the first model to obtain data features corresponding to the first communication topology, including:

[0020] In response to the fact that the size of the cluster has not reached the size threshold, the local state of the nodes and the global association information of the topology in the first communication topology are extracted through the first graph convolutional network, and the first embedding vector corresponding to each processing component is generated based on the extracted local state of the nodes and the global association information of the topology.

[0021] In response to the cluster size reaching a size threshold, the local state of nodes in the first communication topology is extracted through a second graph convolutional network, the topology global association information in the first communication topology is extracted through a multi-head attention module, and a second embedding vector corresponding to each processing component is generated based on the extracted local state of nodes and topology global association information.

[0022] The data features corresponding to the first communication topology include the first embedding vector or the second embedding vector corresponding to each processing component; the first model includes the first graph convolutional network, or includes the second graph convolutional network and the multi-head attention module.

[0023] Optionally, the first graph convolutional network or the second graph convolutional network in the first model is trained based on the first loss function;

[0024] The first loss function characterizes the difference between the predicted communication performance value and the measured communication performance value of the processing component; the predicted communication performance value characterizes the communication performance of the processing component predicted based on the first or second embedding vector of the processing component generated by the first or second graph convolutional network.

[0025] Optionally, the second model determines an adjustment strategy for adjusting the first communication topology based on reinforcement learning and the data features, including:

[0026] The data features are at least input into the policy network to obtain the action probability distribution generated and output by the policy network based on the input data;

[0027] The second model includes the policy network constructed based on reinforcement learning, and the adjustment policy includes the action probability distribution; the action probability distribution includes each candidate action for adjusting the structure of the first communication topology and the probability corresponding to each candidate action.

[0028] Optionally, the policy network is constructed by adjusting the model parameters of the policy network during the training phase based on the value evaluation information generated by the value network.

[0029] The value assessment information includes the evaluation results of the value network on the actions represented by the action probability distribution output by the policy network during the training process.

[0030] A processing apparatus, comprising:

[0031] The acquisition module is used to obtain the communication and running status information of each processing component in the cluster.

[0032] The first determining module is used to determine the first communication topology between the processing components based on the communication and operation status information corresponding to each processing component; different nodes in the first communication topology represent different processing components in the cluster, and the first communication topology represents the node characteristics of each node, the connection relationship between each node, and the communication characteristics.

[0033] The feature extraction module is used to extract features from the first communication topology using a first model to obtain data features corresponding to the first communication topology; the data features are used to provide an embedding vector containing the global context of the first communication topology, and the first model is a graph neural network model;

[0034] The second determining module is used to determine an adjustment strategy for adjusting the first communication topology based on reinforcement learning and the data features through the second model;

[0035] An adjustment module is used to adjust the structure of the first communication topology based on the adjustment strategy to obtain a second communication topology; the communication performance of the second communication topology is higher than that of the first communication topology.

[0036] An electronic device, comprising:

[0037] Memory, used to store at least one set of computer instructions;

[0038] A processor is configured to perform the following processing by executing a set of instructions stored in the memory:

[0039] Obtain the communication and operational status information of each processing component in the cluster;

[0040] Based on the communication and operation status information corresponding to each processing component, a first communication topology between the processing components is determined; different nodes in the first communication topology represent different processing components in the cluster, and the first communication topology represents the node characteristics of each node, the connection relationship between each node, and the communication characteristics.

[0041] The first communication topology is subjected to feature extraction using a first model to obtain data features corresponding to the first communication topology; the data features are used to provide an embedding vector containing global context for the first communication topology; the first model is a graph neural network model.

[0042] The second model determines an adjustment strategy for adjusting the first communication topology based on reinforcement learning and the data features.

[0043] The structure of the first communication topology is adjusted based on the adjustment strategy to obtain the second communication topology; the communication performance of the second communication topology is higher than that of the first communication topology.

[0044] A storage medium carrying one or more computer instruction sets, which, when executed by an electronic device, enable the electronic device to perform any of the processing methods provided above. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0046] Figure 1 This is a schematic diagram of the processing method provided in this application;

[0047] Figure 2 This is a schematic diagram of the implementation scheme provided in this application;

[0048] Figure 3 This is a structural diagram of the processing apparatus provided in this application;

[0049] Figure 4 This is a structural diagram of the electronic device provided in this application. Detailed Implementation

[0050] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0051] This application provides a processing method, apparatus, and electronic device for solving problems such as rigidity of cluster communication topology and limited static optimization effect in the prior art by intelligent real-time optimization of cluster communication topology (e.g., real-time optimization of GPU point-to-point communication topology of large-scale AI server clusters).

[0052] See Figure 1 The flowchart shown illustrates the processing method provided in this application embodiment, which may include steps 101 to 105 as described below. These steps will be described in detail below.

[0053] Step 101: Obtain the communication and running status information of each processing component in the cluster.

[0054] The processing components in the cluster can be, but are not limited to, GPUs, CPUs (Central Processing Units), NPUs (Neural Network Processing Units), or other related processing units or processors capable of data processing. This application's embodiments primarily use GPUs as an example to illustrate the solution.

[0055] Optionally, the communication and operational status information corresponding to the processing components should include at least three core data categories: the computational status, communication status, and task characteristics of the processing components, to ensure the completeness of subsequent modeling as much as possible. The various data categories are as follows:

[0056] a. Computation status: This may include, but is not limited to, parameters such as computing utilization, video memory usage, SM (Streaming Multiprocessor) core load rate and / or temperature;

[0057] b. Communication status: This may include, but is not limited to, parameters such as link bandwidth utilization, end-to-end communication delay, communication traffic and / or packet loss rate;

[0058] c. Task characteristics: These may include, but are not limited to, parameters such as communication type (e.g., AllReduce / Point-to-Point / Broadcast), data block size, and / or communication frequency.

[0059] In this step, during cluster operation, communication and operational status information such as the computing status, communication status, and / or task characteristics of the processing components in the cluster can be collected in real time to provide data input for subsequent communication topology optimization processing.

[0060] Step 102: Based on the communication and operation status information corresponding to each processing component, determine the first communication topology between each processing component.

[0061] In the first communication topology, different nodes represent different processing components in the cluster. The first communication topology represents the node characteristics of each node, the connection relationships between each node, and the communication characteristics.

[0062] Optionally, the first communication topology can be implemented as an undirected weighted graph.

[0063] Optionally, in this step, a target undirected weighted graph can be constructed using processing components as nodes, the computational state information and task characteristics of the processing components as node features, and the communication links represented by the communication state information of the processing components as edges. This target undirected weighted graph can then be used to represent the first communication topology between the processing components in the cluster.

[0064] The graph structure information of the target undirected weighted graph may include, but is not limited to, at least some of the following:

[0065] A set of nodes; different nodes in the node set represent different processing components.

[0066] Edge set; the edges in the edge set represent the communication links between different nodes connected by the edges;

[0067] Node feature matrix; each row represents the feature vector of a processing component. The feature vector of a processing component includes the feature vectors corresponding to the computational state information and task features of the processing component.

[0068] Adjacency matrix; the elements in the adjacency matrix represent the communication weights between different processing components with communication links, and the communication weights are determined based on the communication state information of different processing components.

[0069] In practice, graph modeling can be used to quantify the relationships between nodes (processing components) and edges (communication links) in the cluster, abstract them into structured data, and obtain a target undirected weighted graph formed by a series of nodes and edges.

[0070] For example, the graph structure of the target undirected weighted graph can be defined as: G=(V,E,X,A).

[0071] The meanings of each part are as follows:

[0072] a and V represent the node set, where each processing component, such as a GPU, is a node, and the number of nodes is N. For example, assuming the cluster has 64 GPUs, then... .

[0073] b and E represent the set of edges. Edges exist between nodes of two processing components only when there is a physical communication link between them (such as NVLink, PCIe, or InfiniBand). (i / j represent the numbers of these two processing components or their node numbers, respectively). If there is no physical link, there is no corresponding edge (to avoid outputting an infeasible topology).

[0074] c and X represent the node feature matrices, with dimensions of . ( (This represents the number of node features), with each row corresponding to a GPU feature vector, such as the encoding feature vector for computational utilization / memory usage / task type, etc.

[0075] d and A represent the adjacency matrix with dimension 1. , element is This represents the communication weight between processing components i and j, for example, =Link bandwidth utilization (there is a link between processing components i and j), or =0 (There is no link between processing components i and j).

[0076] In response to the real-time changes in the load and communication requirements of processing components such as GPUs in the cluster, the implementation can update the graph structure of the target undirected weighted graph on a time-slice basis to reflect the dynamic and real-time changes in the communication topology between the processing components in the cluster.

[0077] The time slice can be adaptively selected according to the task type. For example, a 1-second time slice can be selected for a deep learning training task, while a 100-ms time slice can be selected for a real-time inference task.

[0078] Every time slice, the communication and running status information of each processing component in the cluster is re-collected to update the node feature matrix X (such as changes in GPU utilization) and the adjacency matrix A (such as when the bandwidth utilization of a certain link increases from 50% to 80%, then Aij is updated synchronously), and other graph structure information.

[0079] Step 103: Extract features from the first communication topology using the first model to obtain data features corresponding to the first communication topology; the data features are used to provide an embedding vector containing the global context of the first communication topology.

[0080] The first model is a graph neural network model, which can be, but is not limited to, a GCN (Graph Convolutional Network) model.

[0081] Traditional feature engineering loses node neighbor relationships (e.g., the communication latency of GPU A is affected by the load of its neighbor GPU B). To address this issue, this application's embodiments leverage the characteristic of graph neural networks, such as graph convolutional networks, that "can transform structured graph data into low-dimensional embedding vectors containing global context through neighborhood feature aggregation." A first model of a graph neural network class is pre-trained to extract features from the first communication topology to capture the global association information of the topology, thereby obtaining an embedding vector of the first communication topology containing global context, providing a high-quality state representation for subsequent topology optimization.

[0082] Optionally, feature extraction of the first communication topology using the first model to obtain the data features corresponding to the first communication topology can be implemented as one of the following steps "1-1"-"1-2":

[0083] 1-1: In response to the fact that the size of the cluster has not reached the size threshold, the local state of the nodes and the global association information of the topology in the first communication topology are extracted through the first graph convolutional network, and the first embedding vector corresponding to each processing component is generated based on the extracted local state of the nodes and the global association information of the topology.

[0084] 1-2: In response to the cluster size reaching a size threshold, the local state of nodes in the first communication topology is extracted through the second graph convolutional network, the topology global association information in the first communication topology is extracted through the multi-head attention module, and the second embedding vector corresponding to each processing component is generated based on the extracted local state of nodes and topology global association information.

[0085] The data features corresponding to the first communication topology include the first embedding vector or the second embedding vector corresponding to each processing component; the first model includes the first graph convolutional network mentioned above, or the second graph convolutional network mentioned above and the multi-head attention module.

[0086] Whether the cluster size has reached the size threshold can refer to, but is not limited to, whether the number of processing components contained in the cluster has reached a preset value, or whether the number of nodes in the constructed target undirected weighted graph has reached a preset value.

[0087] Specifically, the essence of graph neural networks such as GCN is feature propagation based on graph structure. The features of each node are fused with the features of its neighboring nodes, thereby capturing local topological relationships. Through multiple layers of convolution, graph neural networks can propagate local relationships to the global topology, and the final embedding vector contains both the node's own state and global topological relationship information (global feature dependency). However, if the server cluster is large, graph neural networks such as GCN can only capture local node features, while global feature dependencies can be captured using multi-head attention mechanisms.

[0088] The forward propagation formula for GCN can be expressed as:

[0089] .

[0090] in, Indicates the first Layer (GCN's first layer) The node feature matrix of a multilayer network. Input layer That is, the input of the first network layer of GCN is the original X, where X represents the node feature matrix of the target undirected weighted graph.

[0091] Represents the normalized adjacency matrix. , Represents the identity matrix. Let A represent the degree matrix and A represent the adjacency matrix. The purpose of normalization is to avoid gradient vanishing or exploding and to ensure that the feature weights of each node are balanced.

[0092] Indicates the first Layer (GCN's first layer) The trainable weight matrix of a network layer.

[0093] The activation function can be, but is not limited to, ReLU (Rectified Linear Unit). Optionally, the hidden layers of GCN use activation functions such as ReLU, while the output layer has no activation because the embedding vector needs to retain continuous values.

[0094] Step 104: Determine an adjustment strategy for adjusting the first communication topology using the second model based on reinforcement learning and the data features.

[0095] The second model includes a policy network built based on reinforcement learning.

[0096] Based on step 103, this step can at least input the data features of the first communication topology into the policy network to obtain the action probability distribution generated and output by the policy network based on the input data.

[0097] In addition to the data characteristics of the first communication topology, the input of the policy network may also include, but is not limited to, the global state characteristics of the first communication topology. These global state characteristics may include, but are not limited to, information such as the average communication latency of the cluster, the total bandwidth utilization, the task completion progress, and / or the number of topology adjustments.

[0098] The action probability distribution output by the policy network can be regarded as an adjustment policy for adjusting the first communication topology. This action probability distribution includes each candidate action for adjusting the structure of the first communication topology and the probability corresponding to each candidate action.

[0099] Each candidate action may include, but is not limited to, at least one of different types of actions such as bandwidth allocation and link switching.

[0100] Step 105: Adjust the structure of the first communication topology based on the adjustment strategy to obtain the second communication topology; the communication performance of the second communication topology is higher than that of the first communication topology.

[0101] This step can specifically determine the final target action for adjusting the structure of the first communication topology based on the action probability distribution output by the policy network. For example, the candidate action with the highest probability can be selected as the target action from the candidate actions included in the action probability distribution.

[0102] Based on this, the structure of the first communication topology can be further adjusted according to the target action to obtain the second communication topology. For example, the bandwidth allocation of the corresponding GPU nodes in the first communication topology can be adjusted, and / or the communication links between GPU node pairs can be adjusted (such as switching routes from physically feasible links).

[0103] The adjustment strategy underlying the communication topology adjustment is generated by the policy network of the second model through reinforcement learning based on big data. This ensures that the adjusted second communication topology is adapted to the current real-time communication and operational status of the cluster, resulting in higher communication performance for the adjusted second communication topology compared to the first communication topology. Communication performance includes, but is not limited to, at least one of communication latency and bandwidth utilization.

[0104] In summary, this application determines the current first communication topology of the cluster based on the communication and operational status information of each processing component in the cluster, extracts the data features of the first communication topology using a first model of graph neural network, and determines an adjustment strategy for adjusting the first communication topology based on reinforcement learning and the extracted data features using a second model. This adjusts the structure of the first communication topology and achieves intelligent real-time optimization of the cluster communication topology based on graph neural network processing and reinforcement learning. As a result, this application can adapt to dynamically changing communication modes and respond to changes in the network status of the cluster in real time. It overcomes the problems of rigid communication topology, limited static optimization effect, inefficient manual tuning (relying on expert experience to manually configure topology parameters, which is extremely costly and has limited effect in large-scale clusters such as those with tens of thousands of cards) and insufficient fault tolerance (fixed topologies lack adaptive adjustment ability when link failures or congestion occur, leading to a sharp decline in performance) caused by the use of fixed topology structures in known technologies.

[0105] In an alternative embodiment, the first graph convolutional network or the second graph convolutional network in the first model is trained based on the first loss function.

[0106] The first loss function characterizes the difference between the predicted communication performance (e.g., GPU communication latency) and the measured communication performance. The predicted communication performance characterizes the communication performance of the processing component predicted based on the first or second embedding vector generated by the first or second graph convolutional network. The measured communication performance characterizes the ground truth label corresponding to the predicted value.

[0107] This embodiment enables graph convolutional networks such as GCN to quickly learn the relationship between graph structure and communication performance through supervised pre-training.

[0108] The training objective is to enable graph convolutional networks such as GCN to output the embedding vector of the i-th processing component based on the input graph structure information. It can be used to effectively predict the communication performance of the i-th processing component (such as the communication latency of the GPU, with the corresponding truth label being the actual measured latency).

[0109] Where i is an integer, and i∈[1,N], and N represents the number of processing components.

[0110] The first loss function on which it is based can be, but is not limited to, expressed as:

[0111] .

[0112] in, This represents the feature vector / embedding vector of the i-th component, and the feature vectors of each component constitute the data features corresponding to the first communication topology. This represents the communication performance measurement value (communication performance label) of the i-th communication component, such as the actual latency measurement value of the i-th component. This indicates the prediction weight.

[0113] The dataset on which supervised pre-training is based may include pre-collected data for different tasks (such as All Reduce / Point-to-Point), large-scale graph samples under different loads (graph samples of undirected weighted graphs of cluster communication topology), and corresponding communication performance labels (such as communication delay labels).

[0114] This embodiment uses supervised pre-training of the graph convolutional network based on the first loss function, which enables the graph convolutional network to quickly learn the relationship between "graph structure and communication performance". Consequently, the processing component embedding vectors output by the graph convolutional network based on the input graph structure information can be used to effectively predict the communication performance of the processing components, thereby providing high-quality input for the subsequent formulation of cluster topology adjustment strategies (to improve the communication performance of the cluster communication topology).

[0115] In one optional embodiment, the policy network in the second model is a neural network trained based on reinforcement learning. During the training phase, reinforcement learning can be used to guide policy optimization based on the value assessment information generated by the value network, thereby adjusting the model parameters corresponding to the policy network during training to achieve the training of the policy network.

[0116] The value assessment information includes the assessment results of the value network on the action probability distribution represented by the action during the training process of the policy network.

[0117] The essence of this application is to solve the problem of optimizing the cluster communication topology. This embodiment transforms the cluster communication topology optimization into a dynamic sequence decision problem. Dynamic sequence decision refers to predicting / selecting adjustment actions (such as bandwidth allocation and link switching) based on the current communication topology state representation of the cluster (such as data features of the first communication topology, such as GCN embedding vectors) in each time slice, and optimizing the decision strategy through feedback rewards.

[0118] This embodiment uses the A2C (Advantage Actor-Critic) algorithm to balance exploration (trying new topologies) and utilization (preserving superior topologies), while reducing the variance of policy updates.

[0119] The A2C network consists of two networks:

[0120] 1. Actor (Policy Network): The input is the current state, and the output is the action probability distribution (which determines "which action to choose");

[0121] 2. Critic (Value Network): The input is a data pair of current state and action, i.e., "current state-action", and the output is a value assessment (judging whether "action is good or bad").

[0122] This embodiment connects the policy network and the value network through an advantage function, and uses the advantage function to weight the policy gradient, thereby reducing update variance and improving training stability.

[0123] The advantage function can be, but is not limited to, expressed as:

[0124] .

[0125] in, This represents the value of a "state-action" pair. Represents the value of a state.

[0126] The network input, action space definition, and reward function design involved in network training are as follows:

[0127] 1. Input Features

[0128] The inputs to the policy network (Actor) and the evaluation network (Critic) are data features of the cluster communication topology, which may include GCN embedding vectors and global state features.

[0129] The GCN embedding vector can include the embedding vectors of each processing component, and the embedding vectors of each processing component can be concatenated and then input into the network. For example, assuming each GPU corresponds to a 16-dimensional embedding vector. This allows you to concatenate the 16-dimensional embedding vectors of each GPU into a 16N-dimensional array before inputting them.

[0130] Global state characteristics may include, but are not limited to, average cluster communication latency, total bandwidth utilization, task completion progress, and / or the number of topology adjustments.

[0131] 2. Definition of Action Space

[0132] In the action space definition, actions must meet the "physical feasibility" requirement (e.g., a GPU without NVLink cannot switch to an NVLink link). Actions in the action space can be divided into two types, including bandwidth allocation and link switching, as shown in Table 1.

[0133] Table 1

[0134]

[0135] In practical applications, policy networks can filter out non-activities through a mask layer, such as setting the probability of a handover action to 0 for links without InfiniBand (unlimited bandwidth).

[0136] 3. Reward Function Design

[0137] Rewards are central to guiding Actors to output optimal actions and must align with optimization goals such as "low latency, high utilization, and low overhead." The reward function can be designed, but is not limited to, the following:

[0138] .

[0139] in, These are weights, which can be determined, but are not limited to, through cross-validation. For example, ; This represents the difference between the communication delay after the communication topology adjustment and the communication delay before the adjustment. The minus sign before the delay indicates that the reward is positive when the delay is reduced; This represents the difference between the bandwidth utilization rate after the communication topology adjustment and the bandwidth utilization rate before the adjustment. A positive difference indicates an improvement in the cluster's communication performance and contributes a positive reward. This represents the overhead of topology adjustments, such as the time / interruption cost of adjusting links, with a value ranging from 0 to 0.1. If no adjustment is made... It is 0.

[0140] For example, taking the first model as a GCN model, the policy network training process may include the following steps:

[0141] a. Initialization: Load the pre-trained GCN model and initialize the model parameters (such as weights) of Actor / Critic.

[0142] b. Sampling: For network training tasks running on the cluster, communication and operational status information of processing components such as GPUs in the cluster is collected at each time slice to construct an undirected weighted graph corresponding to the cluster communication topology. And obtain the GCN embedding vector (data features of the communication topology) through GCN.

[0143] c. Input: The GCN embedding vector is merged with the global state features and used as the current state input to the Actor, so that the Actor outputs an action. After the action is executed, the following information chain can be recorded:

[0144] “ ".

[0145] d. Value Calculation: Calculate the value of the current state using Critic. and the value of the next state The target value is calculated using Temporal Difference Learning (TD) error. .

[0146] in, Indicates a reward. As a discount factor, for example, .

[0147] e. Gradient update:

[0148] e-1: Critic loss.

[0149] For example, the Critic loss can be expressed as Furthermore, the model parameters of the policy network can be updated using the Adam optimizer based on the Critic loss value.

[0150] e-2: Actor gradient.

[0151] For example, the Actor gradient can be represented as Where 'a' represents the action and 's' represents the state. It can update the model parameters such as the weights of the policy network along the gradient ascent direction to maximize the expected reward.

[0152] f: Iteration: Repeat steps b~e until the reward converges. If the reward fluctuation is less than 5% for 1000 consecutive time slices, the training of the policy network can be completed.

[0153] In practical applications, see Figure 2 The proposed solution can be implemented by dividing it into the following five core modules to form a closed loop of "perception-embedding-decision-execution-feedback":

[0154] Data acquisition module: Acquires GPU computing / communication status in real time to provide input for subsequent modeling;

[0155] Graph modeling module: Abstracts the GPU cluster into a "node-edge" graph structure, quantifies node features and edge weights;

[0156] GCN Graph Embedding Module: Transforms high-dimensional, structured graph data into low-dimensional, globally related embedding vectors through graph convolutional networks;

[0157] The Actor-Critic optimization module: Based on reinforcement learning, it outputs the optimal topology adjustment actions (such as bandwidth allocation and link switching) and evaluates the value of the actions;

[0158] Topology execution and reward feedback: The hardware interface is called to perform topology adjustments and calculate rewards to update the neural network parameters.

[0159] In summary, this embodiment trains a policy network based on reinforcement learning, enabling the policy network to quickly and efficiently generate adjustment policies that adapt to diverse communication topology states and patterns. This dynamically optimizes the communication topology of server clusters such as GPUs, achieving intelligent real-time optimization of the communication topology.

[0160] Corresponding to the above method, this application embodiment also provides a processing apparatus, the composition of which is as follows: Figure 3 As shown, it includes:

[0161] The acquisition module 301 is used to obtain the communication and running status information of each processing component in the cluster.

[0162] The first determining module 302 is used to determine a first communication topology between processing components based on the communication and operation status information corresponding to each processing component; different nodes in the first communication topology represent different processing components in the cluster, and the first communication topology represents the node characteristics of each node, the connection relationship between each node, and the communication characteristics.

[0163] The feature extraction module 303 is used to extract features from the first communication topology using a first model to obtain data features corresponding to the first communication topology; the data features are used to provide an embedding vector containing the global context of the first communication topology, and the first model is a graph neural network model;

[0164] The second determining module 304 is used to determine an adjustment strategy for adjusting the first communication topology based on reinforcement learning and the data features through the second model.

[0165] The adjustment module 305 is used to adjust the structure of the first communication topology based on the adjustment strategy to obtain a second communication topology; the communication performance of the second communication topology is higher than that of the first communication topology.

[0166] In one optional implementation, the communication and operational status information corresponding to the processing component includes the computing status information, communication status information, and task characteristics of the processing component.

[0167] In an optional implementation, the first determining module 302 is specifically used for:

[0168] An undirected weighted graph is constructed with processing components as nodes, the computational state information and task characteristics of processing components as node characteristics, and the communication links represented by the communication state information of processing components as edges.

[0169] The target undirected weighted graph is used to characterize the first communication topology.

[0170] In one optional implementation, the graph structure information of the target undirected weighted graph includes at least some of the following:

[0171] A set of nodes; different nodes in the set of nodes represent different processing components;

[0172] An edge set; the edges in the edge set represent the communication links between different nodes connected by the edges;

[0173] Node feature matrix; each row of the node feature matrix represents a feature vector of a processing component, and the feature vector of the processing component includes the feature vectors corresponding to the computational state information and task features of the processing component, respectively.

[0174] The adjacency matrix; the elements in the adjacency matrix represent the communication weights between different processing components with communication links, and the communication weights are determined based on the communication state information of the different processing components.

[0175] In an optional implementation, the feature extraction module 303 is specifically used for:

[0176] In response to the fact that the size of the cluster has not reached the size threshold, the local state of the nodes and the global association information of the topology in the first communication topology are extracted through the first graph convolutional network, and the first embedding vector corresponding to each processing component is generated based on the extracted local state of the nodes and the global association information of the topology.

[0177] In response to the cluster size reaching a size threshold, the local state of nodes in the first communication topology is extracted through a second graph convolutional network, the topology global association information in the first communication topology is extracted through a multi-head attention module, and a second embedding vector corresponding to each processing component is generated based on the extracted local state of nodes and topology global association information.

[0178] The data features corresponding to the first communication topology include the first embedding vector or the second embedding vector corresponding to each processing component; the first model includes the first graph convolutional network, or includes the second graph convolutional network and the multi-head attention module.

[0179] In an optional implementation, the first graph convolutional network or the second graph convolutional network in the first model is trained based on a first loss function;

[0180] The first loss function characterizes the difference between the predicted communication performance value and the measured communication performance value of the processing component; the predicted communication performance value characterizes the communication performance of the processing component predicted based on the first or second embedding vector of the processing component generated by the first or second graph convolutional network.

[0181] In an optional implementation, the second determining module 304 is specifically used for:

[0182] The data features are at least input into the policy network to obtain the action probability distribution generated and output by the policy network based on the input data;

[0183] The second model includes the policy network constructed based on reinforcement learning, and the adjustment policy includes the action probability distribution; the action probability distribution includes each candidate action for adjusting the structure of the first communication topology and the probability corresponding to each candidate action.

[0184] In one optional implementation, the policy network is constructed by adjusting the model parameters of the policy network during the training phase based on the value evaluation information generated by the value network.

[0185] The value assessment information includes the evaluation results of the value network on the actions represented by the action probability distribution output by the policy network during the training process.

[0186] This application also discloses an electronic device, the composition and structure of which are as follows: Figure 4 As shown, it may include:

[0187] Memory 10 is used to store the computer instruction set;

[0188] Computer instruction sets can be implemented in the form of computer programs.

[0189] The processor 20 is configured to implement the processing method provided in any of the above method embodiments by executing a set of computer instructions in the memory.

[0190] The processor can be a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a neural network processor (NPU), a deep learning processor (DPU), or other programmable logic devices.

[0191] Optionally, the electronic device may also include a display device for displaying information.

[0192] Optionally, electronic devices may also include storage resources such as memory and cache.

[0193] Optionally, the electronic device may also include an image acquisition device.

[0194] In addition to these components, electronic devices may also include communication interfaces, communication buses, and other parts. Memory, processor, and communication interface communicate with each other through the communication bus.

[0195] Communication interfaces are used for communication between electronic devices and other devices. Communication buses can be Peripheral Component Interconnect (PCI) buses or Extended Industry Standard Architecture (EISA) buses, and can be categorized into address buses, data buses, control buses, etc.

[0196] This application also discloses a storage medium carrying one or more computer instruction sets, which, when executed by an electronic device, enable the electronic device to implement the processing method provided in any of the above method embodiments.

[0197] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0198] For ease of description, the above systems or devices are described separately as various modules or units based on their functions. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware components.

[0199] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence or the part that makes a creative contribution, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0200] Finally, it should be noted that in this document, relational terms such as first, second, third, and fourth are used to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0201] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A processing method, comprising: Obtain the communication and operational status information of each processing component in the cluster; Based on the communication and operation status information corresponding to each processing component, a first communication topology between the processing components is determined; different nodes in the first communication topology represent different processing components in the cluster, and the first communication topology represents the node characteristics of each node, the connection relationship between each node, and the communication characteristics. The first communication topology is subjected to feature extraction using a first model to obtain data features corresponding to the first communication topology; the data features are used to provide an embedding vector containing global context for the first communication topology; the first model is a graph neural network model. The second model determines an adjustment strategy for adjusting the first communication topology based on reinforcement learning and the data features. The structure of the first communication topology is adjusted based on the adjustment strategy to obtain the second communication topology; the communication performance of the second communication topology is higher than that of the first communication topology.

2. The processing method according to claim 1, wherein the communication and operation status information corresponding to the processing component includes the computing status information, communication status information and task characteristics of the processing component.

3. The processing method according to claim 2, based on the communication and operating status information corresponding to each processing component, determines the first communication topology between the processing components, including: An undirected weighted graph is constructed with processing components as nodes, the computational state information and task characteristics of processing components as node characteristics, and the communication links represented by the communication state information of processing components as edges. The target undirected weighted graph is used to characterize the first communication topology.

4. The processing method according to claim 3, wherein the graph structure information of the target undirected weighted graph includes at least some of the following: A set of nodes; different nodes in the set of nodes represent different processing components; An edge set; the edges in the edge set represent the communication links between different nodes connected by the edges; Node feature matrix; each row of the node feature matrix represents a feature vector of a processing component, and the feature vector of the processing component includes the feature vectors corresponding to the computational state information and task features of the processing component, respectively. The adjacency matrix; the elements in the adjacency matrix represent the communication weights between different processing components with communication links, and the communication weights are determined based on the communication state information of the different processing components.

5. The processing method according to claim 3, wherein feature extraction is performed on the first communication topology using a first model to obtain data features corresponding to the first communication topology, including: In response to the fact that the size of the cluster has not reached the size threshold, the local state of the nodes and the global association information of the topology in the first communication topology are extracted through the first graph convolutional network, and the first embedding vector corresponding to each processing component is generated based on the extracted local state of the nodes and the global association information of the topology. In response to the cluster size reaching a size threshold, the local state of nodes in the first communication topology is extracted through a second graph convolutional network, the topology global association information in the first communication topology is extracted through a multi-head attention module, and a second embedding vector corresponding to each processing component is generated based on the extracted local state of nodes and topology global association information. The data features corresponding to the first communication topology include the first embedding vector or the second embedding vector corresponding to each processing component; the first model includes the first graph convolutional network, or includes the second graph convolutional network and the multi-head attention module.

6. The processing method according to claim 5, wherein the first graph convolutional network or the second graph convolutional network in the first model is trained based on the first loss function; The first loss function characterizes the difference between the predicted communication performance value and the measured communication performance value of the processing component; the predicted communication performance value characterizes the communication performance of the processing component predicted based on the first or second embedding vector of the processing component generated by the first or second graph convolutional network.

7. The processing method according to claim 1, wherein an adjustment strategy for adjusting the first communication topology is determined by a second model based on reinforcement learning and the data features, comprising: The data features are at least input into the policy network to obtain the action probability distribution generated and output by the policy network based on the input data; The second model includes the policy network constructed based on reinforcement learning, and the adjustment policy includes the action probability distribution; the action probability distribution includes each candidate action for adjusting the structure of the first communication topology and the probability corresponding to each candidate action.

8. The processing method according to claim 7, wherein the policy network is constructed by adjusting the model parameters corresponding to the policy network during the training process based on the value evaluation information generated by the value network during the training phase; The value assessment information includes the evaluation results of the value network on the actions represented by the action probability distribution output by the policy network during the training process.

9. A processing apparatus, comprising: The acquisition module is used to obtain the communication and running status information of each processing component in the cluster. The first determining module is used to determine the first communication topology between the processing components based on the communication and operation status information corresponding to each processing component; different nodes in the first communication topology represent different processing components in the cluster, and the first communication topology represents the node characteristics of each node, the connection relationship between each node, and the communication characteristics. The feature extraction module is used to extract features from the first communication topology using a first model to obtain data features corresponding to the first communication topology; the data features are used to provide an embedding vector containing the global context of the first communication topology, and the first model is a graph neural network model; The second determining module is used to determine an adjustment strategy for adjusting the first communication topology based on reinforcement learning and the data features through the second model; An adjustment module is used to adjust the structure of the first communication topology based on the adjustment strategy to obtain a second communication topology; the communication performance of the second communication topology is higher than that of the first communication topology.

10. An electronic device, comprising: Memory, used to store at least one set of computer instructions; A processor is configured to perform the following processing by executing a set of instructions stored in the memory: Obtain the communication and operational status information of each processing component in the cluster; Based on the communication and operation status information corresponding to each processing component, a first communication topology between the processing components is determined; different nodes in the first communication topology represent different processing components in the cluster, and the first communication topology represents the node characteristics of each node, the connection relationship between each node, and the communication characteristics. The first communication topology is subjected to feature extraction using a first model to obtain data features corresponding to the first communication topology; the data features are used to provide an embedding vector containing global context for the first communication topology; the first model is a graph neural network model. The second model determines an adjustment strategy for adjusting the first communication topology based on reinforcement learning and the data features. The structure of the first communication topology is adjusted based on the adjustment strategy to obtain the second communication topology; the communication performance of the second communication topology is higher than that of the first communication topology.

Images