Gnn-based sas extender intelligent routing method and system

Abstract

The application discloses a SAS extender intelligent routing method and system based on GNN. The method comprises the following steps: S1, constructing a path link bipartite graph model; S2, performing load-aware graph neural network inference; S3, path allocation proportional decoding and normalization; S4, strategy compilation and hardware delivery; and S5, closed-loop monitoring and incremental learning optimization. The application realizes load balancing, low delay and globally optimal routing through graph neural network modeling; has the advantages of fast fault self-healing, strong generalization and closed-loop adaptive capacity; and is compatible with the existing SAS protocol, has strong real-time performance and low deployment cost.

Description

Technical Field

[0001] This invention relates to the field of high-performance storage technology, and in particular to a GNN-based intelligent routing method and system for SAS extenders. Background Technology

[0002] SAS, as a high-performance storage interface, is widely used in data center storage area networks. SAS expanders are responsible for forwarding data between hosts (HBAs) and hard drives, and their routing performance directly impacts system throughput and latency. Existing SAS expander routing methods (round-robin-based wide-port load balancing and request-direction-based priority arbitration) cannot detect link speed differences, link error rates, dynamic load changes, or achieve global load balancing and fault-adaptive rerouting. While graph neural network routing methods based on bipartite path graphs exist in the field of distributed routing and traffic scheduling, they are only suitable for wide area network distributed traffic engineering and are incompatible with the topology and I / O transmission characteristics of SAS storage networks. Furthermore, they suffer from problems such as complex models, difficulty in lightweight deployment on embedded hardware like SAS expanders, reliance on dedicated traffic matrices and offline optimal solution-supervised training, slow fault recovery convergence, and lack of optimization for hard drive I / O characteristics, failing to meet the requirements of storage systems for high reliability, low latency, dynamic load balancing, and rapid fault self-healing.

[0003] SAS (Serial Attached SCSI) is a high-performance point-to-point storage interface protocol widely used in data center storage area networks (SANs) and direct-attached storage (DAS) systems. SAS expanders are the core switching devices in a SAS domain, responsible for forwarding data frames between hosts (HBAs) and numerous hard disk drives (HDDs / SSDs). A typical SAS domain can contain multiple cascaded expanders, dozens of host interfaces, and hundreds of hard drives. Each expander maintains a routing table, determining the output physical port (Phy) based on the destination SAS address of the data frame. Because SAS supports multi-path connections (e.g., wide ports aggregate multiple Phies into a single logical link), multiple reachable paths often exist for the same destination address. Therefore, the quality of the routing algorithm directly affects the system's throughput, latency, and reliability.

[0004] Currently, routing improvement schemes based on the polling mechanism can achieve a formal traffic allocation by selecting idle Phys for forwarding traffic on wide ports in turn, but they have significant drawbacks: this mechanism treats all requests equally, ignoring both the priority differences of the requests themselves (such as not distinguishing between critical control commands and ordinary data requests) and the differences in the characteristics of the requests (such as mixing requests with high bandwidth requirements and requests with small data packets), which can easily lead to response delays for critical requests; at the same time, the polling mechanism cannot perceive the load status of the links in real time, and has obvious shortcomings in the rationality and efficiency of port scheduling.

[0005] In practical applications, SAS storage systems often employ a multi-level SAS expander cascaded topology. The SAS expander, as a key chip in the entire system, expands the number of interfaces through uplink ports (connecting to the host or the previous level expander), downlink ports (connecting to the next level expander), and hard drive ports (connecting SAS hard drives), thereby connecting a large number of hard drives to form a storage array. According to the LRU arbitration fairness mechanism of the SAS protocol, in a single-level expander scenario, each hard drive port has an equal opportunity to establish a communication link with the host. However, in a multi-level cascaded scenario, the link establishment opportunity between hard drive ports under adjacent expanders decreases progressively with the number of cascade levels, ultimately leading to an imbalance in the host's read / write performance across the hard drives. To address this issue, existing improvement methods primarily optimize the process through "request direction determination": First, the path request sent by the extended port is received, and the application type of the port (uplink, hard disk, or downlink) is determined. Then, the communication direction is determined based on the port application type. If it is a downlink direction (i.e., the request comes from the downlink port or the destination address is the downlink port), the request is set to high priority; otherwise, the original priority remains unchanged. Finally, the processed request is saved to the path request table, and the LRU mechanism is used for link-layer arbitration to establish a stable communication link.

[0006] The aforementioned prior art includes at least the following disadvantages. Polling mechanism defects: It treats all requests equally and cannot distinguish between priorities (such as critical control commands and ordinary data) and request characteristics (such as high bandwidth requirements and small data packets mixed together), resulting in delays in the response to critical requests; at the same time, it cannot perceive the link load status in real time.

[0007] The LRU arbitration fairness mechanism suffers from performance imbalance in multi-level cascading: In a single-level expander, each hard drive port can obtain an equal opportunity to establish a link, but in a multi-level cascading topology, the opportunity to establish a link decreases with the number of levels, resulting in uneven read and write performance of the host to the hard drive.

[0008] The topology adaptation limitations of the improved scheme based on request direction judgment: This scheme is only effective in hierarchical topologies with clear uplink and downlink relationships, such as tree and chain structures; however, in non-hierarchical topologies such as symmetrical interconnection of data centers and redundant deployment of dual-active hosts, the "uplink and downlink" judgment logic fails, the high-priority allocation mechanism is disordered, resulting in some extenders running overloaded and some node resources being idle, showing obvious shortcomings in topology adaptation. Summary of the Invention

[0009] In view of this, this application designs a graph neural network-based intelligent routing method for SAS storage networks, which addresses the above problems. By constructing a path-link bipartite graph, designing a load-aware GNN layer, and using the optimal solution of linear programming as a supervision label for multi-objective training, online real-time inference and closed-loop adaptive operation are achieved. This invention significantly improves link load balancing, reduces transmission latency, and accelerates fault self-healing, while remaining compatible with existing SAS hardware.

[0010] To overcome the shortcomings of the prior art, the technical solution of the present invention is as follows: A smart routing method for SAS extenders based on GNNs includes the following steps: S1, Construct a bipartite graph model of path links; S2, execute load-aware graph neural network inference; S3, path allocation ratio decoding and normalization; S4 is compiled by the strategy and distributed by the hardware. S5, closed-loop monitoring and incremental learning optimization.

[0011] Preferably, in step S1, communication is conducted with the host, extender, and hard disk in the SAS domain via the SMP protocol to collect the SAS address, physical port status, link negotiation rate, link latency, error rate, topology level, and zoning configuration of each device; based on the collected information, all acyclic candidate paths are enumerated for each end-to-end flow to construct a path link bipartite graph.

[0012] Preferably, step S1 specifically includes the following steps: S11, Enumerate end-to-end flows and candidate paths: For each host and disk pair, find all acyclic paths using a graph search algorithm. For each path, record its node sequence, link sequence, total latency, minimum capacity, and number of hops. S12, Create Nodes: Each candidate path is abstracted into a path node, and each physical link is abstracted into a link node; S13, Establish an edge: If a path passes through a link, then establish an undirected edge between the corresponding path node and the link node; S14, Extract node features, including: Path node characteristics: traffic requirements, I / O pattern, total latency, hop count, minimum effective available bandwidth, and path quality score; Link node characteristics: equivalent available bandwidth, latency, series coding, state, current load estimate, and random noise; S15, Auxiliary data processing, auxiliary data includes: The flow index is a one-dimensional tensor used to identify which end-to-end flow each path node belongs to. Its length is equal to the total number of path nodes in the graph, and the value of each element is a globally unique identifier of the flow to which the path belongs. The path link association table is an integer tensor of shape [M, 2], where each row (path_idx, link_idx) represents a path passing through a physical link. This table stores all path link associations in advance and is used to quickly accumulate link load, accumulating the temporary traffic of each path to the corresponding link according to the association table. Link capacity tensor: Stores the capacity of each link. ; Path demand tensor: Stores the flow demand for each path. .

[0013] Preferably, in step S2, the graph data output by the topology-aware module is received, and multi-layer graph attention and load-aware message passing is performed through the encoder. Each graph attention layer includes a multi-head attention mechanism and a load-aware submodule. The temporary allocation ratio is estimated based on the current path node characteristics, the temporary load of the link is calculated, and the link utilization is calculated using the equivalent available bandwidth. Then, the utilization is fed back to update the link node characteristics.

[0014] Preferably, the load-aware submodule performs the following data stream processing: Temporary allocation prediction: Based on the hidden features of the current path nodes, the temporary traffic allocation ratio is estimated. This temporary ratio is used to calculate the link utilization and provide load feedback to the link nodes. The first layer of the temporary prediction network reduces the input dimension and then performs a nonlinear transformation, and then performs regularization to prevent overfitting. The second layer maps the dimensionality-reduced features to a single scalar and then uses Sigmoid activation to compress it to the (0,1) interval as the temporary allocation ratio. Calculate temporary traffic: Multiply the temporary allocation ratio by the path demand tensor to obtain the temporary traffic carried on each path, in Gbps.

[0015] Accumulated link load: Using a path link association table, the temporary traffic of each path is accumulated across all the links it traverses to obtain the temporary load of each link. ; in This is a temporary allocation ratio. For effective path requirements; Calculate link utilization: Divide the temporary load on the link by the link capacity to obtain the link utilization. ; in For link The equivalent available bandwidth is defined as follows: ; Link error rate; Integrating and updating link node features: In the forward propagation of GNN, each node has a set of hidden feature vectors. These feature vectors gradually aggregate neighbor information as the layers deepen, and the above information is injected into the features of the link node, so that the link node knows its current state and thus influences the decisions of neighboring path nodes in the next layer of GNN.

[0016] Preferably, step S3 includes extracting path node features from the node features output by the encoder, mapping them to the original score through a fully connected layer, and then performing normalization processing by grouping by flow to ensure that the sum of the distribution ratios of all paths within each flow is 1, and finally outputting the flow distribution ratio of each path.

[0017] Preferably, step S4 includes the following steps: Convert the traffic allocation ratio of each path output by S3 into local routing table entries for each extender; Check each routing table entry for violations of Zoning constraints; if a violation is found, traffic is redistributed to other compliant paths. Encapsulate valid routing table entries into SMP management frames, send them to the hardware registers of the target extender via the SMP protocol, and wait for an acknowledgment response.

[0018] Preferably, step S5 includes the following steps: Regularly collect actual link utilization and end-to-end latency performance indicators, and calculate the deviation between the actual values ​​and the theoretical predictions of the model. When the deviation exceeds a preset threshold or a topology change event is detected, incremental learning is triggered to fine-tune the model parameters and redeploy the routing strategy.

[0019] Preferably, the method further includes an offline training phase: Randomly generate SAS topology samples containing different cascading levels, link rates, and fault scenarios; A hierarchical discount factor is introduced to simulate the attenuation of the effective bandwidth requirement of deep paths; With the objective function of minimizing the maximum link utilization, a linear programming model containing flow conservation constraints and link capacity constraints is established, and the globally optimal route allocation ratio is obtained as a supervision label. Supervised learning is performed using a multi-objective weighted loss function, which includes at least routing accuracy loss and link utilization loss.

[0020] To achieve the above objectives, the present invention also provides a SAS extender intelligent routing system based on GNN, comprising a topology-aware module, a graph neural network inference engine, a policy compilation module, and a closed-loop feedback unit connected in sequence, wherein... Topology awareness module: used to collect the status of SAS domain devices, enumerate candidate paths and construct a path link bipartite graph; Graph Neural Network Inference Engine: Includes encoder and decoder, used to receive graph data and infer the traffic allocation ratio of each path based on load-aware message passing mechanism; The policy compilation module is used to convert the allocation ratio into SMP routing table entries that conform to the SAS specification and satisfy the Zoning constraints, and then execute the distribution. Closed-loop feedback unit: used to monitor performance metrics in real time and trigger incremental model learning when performance deviations or topology changes are detected.

[0021] Compared with the prior art, the present invention has the following beneficial effects: 1. Significantly improves link load balancing performance By using bipartite graph modeling of path links and a load-aware message passing mechanism, the model can perceive the capacity differences (6 / 12 / 24 Gbps) and dynamic load of each link in real time, and rationally allocate traffic to idle links.

[0022] 2. Approaching the globally optimal route, close to the theoretical upper limit. By using bipartite graph modeling of path links and multi-objective joint optimization, the model can learn a routing strategy that is close to the global optimum.

[0023] 3. Reduce end-to-end transmission latency The multi-objective loss function explicitly includes a delay penalty term, and the path node features include the total path delay, so the model will actively select the low-latency path.

[0024] 4. Rapid fault self-healing and continuous optimization Link / extender failure scenarios are injected into the training data, enabling the model to learn to avoid failures and quickly reroute. The closed-loop feedback unit monitors the actual link status in real time, and automatically triggers incremental learning to fine-tune the model and update the routing strategy when performance degradation or topology changes are detected.

[0025] 5. High real-time performance, meeting engineering deployment requirements. Graph neural network inference is purely forward computation, and the time for a single inference iteration is nearly linearly related to the topology size. For a typical SAS domain containing 50 extenders, 200 links, and 30 flows, the time for a single inference iteration is less than 50ms, far below the acceptable latency for SAS route updates (typically in the order of seconds). Compared to centralized optimization schemes that require repeated solving of LP or integer programming problems, this invention eliminates the need for online solving of complex optimization problems, significantly improving real-time performance.

[0026] 6. Excellent generalization ability, adapting to unknown topologies. The training phase uses a large number of randomly generated topologies and traffic patterns, allowing the model to learn general routing patterns rather than overfitting to specific structures. A hierarchical discount factor (attenuating the demand for deep paths) is introduced into the training data, enabling the model to better adapt to the load characteristics of cascaded topologies.

[0027] 7. Easy to deploy and compatible with existing SAS hardware. The model output is directly compiled into standard SAS routing table entries (destination SAS address → output port) and distributed via the SMP protocol without requiring modification to the extender firmware or SAS protocol stack. The policy compilation module automatically handles SAS-specific constraints such as wide port aggregation and zoning, ensuring the generated routing table entries are compliant. Topology-level coding is added to the link node features, enabling the model to distinguish between ports of different priorities and upstream / downstream relationships, further optimizing service quality. Compared to SDN-based or deep reinforcement learning-based solutions, this invention eliminates the need to run complex agents on the extender, resulting in lower deployment costs.

[0028] 8. Closed-loop adaptive capability, continuous optimization The closed-loop feedback unit continuously monitors the actual network performance. When it detects a significant deviation between the model prediction and the actual load (e.g., due to traffic pattern drift or link aging), it automatically triggers incremental learning, fine-tuning the model with recent data and updating the routing strategy. This mechanism enables the system to adapt to dynamic environments online and maintain optimal performance over the long term, overcoming the limitations of static models or one-time training schemes.

[0029] 9. Multiple objectives are flexibly adjustable to meet different business needs. The loss function comprises five independently configurable weighted components (routing accuracy, latency, link utilization, fault recovery, and path switching). Users can flexibly adjust the weights according to their specific scenarios (e.g., prioritizing high throughput, low latency, or high reliability) without redesigning the model structure. The weights can be dynamically adjusted during training, for example, prioritizing accuracy in the early stages and load balancing in later stages, accelerating convergence and improving final performance.

[0030] 10. Improve training efficiency and sample utilization. By employing data-driven supervised learning instead of reinforcement learning, the massive amounts of online interactive samples and unstable training processes required by reinforcement learning are avoided. Offline sample generation can be processed in parallel, and training data can be reused once generated, significantly shortening the model development cycle. Attached Figure Description

[0031] Figure 1 is a flowchart illustrating the intelligent routing method of the SAS extender based on GNN of the present invention. Figure 2 is a schematic diagram of the transformation from physical network to path link bipartite graph in S1 of the SAS extender intelligent routing method based on GNN of the present invention; Figure 3 is a schematic diagram of the path link bipartite graph in S1 of the SAS extender intelligent routing method based on GNN of the present invention; Figure 4 is a schematic diagram of the graph neural network inference engine structure of the SAS extender intelligent routing system based on GNN of the present invention; Figure 5 is a schematic diagram of the data flow of the load-aware submodule in the graph neural network inference engine of the SAS extender intelligent routing system based on GNN of the present invention; Figure 6 is a schematic diagram of the process of fusing and updating link node features in S2 of the intelligent routing method of SAS extender based on GNN of the present invention. Figure 7 This is a flowchart illustrating the training process of the SAS extender intelligent routing method based on GNN of this invention. Detailed Implementation

[0032] The technical solution provided by the present invention will be further described below with reference to the accompanying drawings.

[0033] Figure 1 is a flowchart illustrating the intelligent routing method for the SAS extender based on GNN of this invention, which includes the following steps: S1, Construct a bipartite graph model of path links; S2, execute load-aware graph neural network inference; S3, path allocation ratio decoding and normalization; S4 is compiled by the strategy and distributed by the hardware. S5, closed-loop monitoring and incremental learning optimization.

[0034] S1 communicates with hosts (HBAs), expanders, and hard drives in the SAS domain via the SMP protocol, collecting information on each device's SAS address, physical port (Phy) status, link negotiation rate (6 / 12 / 24 Gbps), link latency (μs), error rate, topology level, and zoning configuration. Based on the collected information, this module enumerates all acyclic candidate paths for each end-to-end flow (host ⇆ hard drive) and constructs a path link bipartite graph.

[0035] See Figure 2 It is a simple SAS physical topology: the host HBA is connected to the expander EXP0 through wide ports (two Phy, which can be represented as Phy0 and Phy1), and EXP0 is mounted with two hard drives DISK0 and DISK1. Figure 3 The image shown is a bipartite path graph corresponding to this SAS physical topology.

[0036] The construction steps are as follows: (1) Enumerate end-to-end flows and candidate paths: For each “host-disk” pair (e.g., HBA→DISK0), find all acyclic simple paths using a graph search algorithm. Figure 3 Four candidate paths are shown: PATH0 (via Link0): HBA → EXP0 → DISK0 PATH1 (via Link1): HBA → EXP0 → DISK0 PATH2 (via Link0): HBA → EXP0 → DISK1 PATH3 (via Link1): HBA → EXP0 → DISK1 Each path records its node sequence, link sequence, total delay, minimum capacity, and number of hops.

[0037] (2) Create nodes: Each candidate path is abstracted into a path node. Figure 3 The left side is labeled PATH0~PATH3); each physical link is abstracted as a link node. Figure 3 The right side is labeled as Link0~Link3).

[0038] (3) Establishing edges: If a path passes through a link, then an undirected edge is established between the corresponding path node and the link node. For example, PATH0 passes through Link0 (HBA-EXP0) and Link2 (EXP0-DISK0), so PATH0 is connected to Link0 and Link2.

[0039] (4) Extract node features: Path node characteristics: traffic requirements, I / O mode, total latency, hop count, minimum effective available bandwidth, and path quality score.

[0040] Link node characteristics: equivalent available bandwidth, latency, series coding, status, current load estimate, random noise.

[0041] (5) Auxiliary data: Flow index: The flow index is a one-dimensional tensor used to identify which end-to-end flow each path node belongs to. Its length is equal to the total number of path nodes in the graph, and the value of each element is a globally unique identifier (integer, starting from 0) of the flow to which that path belongs.

[0042] The path link association table (path_link_assignment) is an integer tensor of shape [M, 2], where each row (path_idx, link_idx) represents a path (path node) passing through a physical link (link node). This table pre-stores all path link associations and is used in the load awareness submodule to quickly accumulate link load: through the index_add operation, the temporary traffic of each path is accumulated to the corresponding link according to the association table, thereby efficiently calculating link utilization.

[0043] Link capacity tensor (link_capacities): Stores the actual capacity (Gbps) of each link.

[0044] Path demand tensor (path_demands): Stores the flow demand (Gbps) for each path.

[0045] Through the above transformation, the SAS physical network is abstracted into a structured bipartite graph, on which GNNs can perform message passing.

[0046] S2 receives graph data output from the topology-aware module and performs multi-layer graph attention and load-aware message passing through the encoder. Each graph attention layer contains a multi-head attention mechanism and a load-aware submodule. It estimates the temporary allocation ratio based on the current path node characteristics, calculates the temporary load of the link, and calculates the link utilization using the equivalent available bandwidth. Then, it feeds back the utilization to update the link node characteristics.

[0047] See Figure 4 A graph neural network inference engine consists of an encoder and a decoder. (1) The encoder includes: Input projection layer: Maps the input features of path nodes and link nodes to the hidden dimension d through different linear transformations. Path nodes use path_linear, and link nodes use link_linear.

[0048] Multi-layer GNN: This embodiment uses 3 layers. Each layer contains the following sub-modules: Multi-head graph attention module: Based on the edge connections of the path link bipartite graph, it calculates the attention weight of each node and its neighboring nodes, and aggregates neighbor information. An adjacency matrix mask is used to limit the attention scope to adjacent nodes, and the number of attention heads is set to 8.

[0049] Load-aware submodule: The load-aware submodule is located inside each GNN layer, executing after the multi-head attention module and before the residual connections. See also Figure 5 This is a schematic diagram of the load awareness submodule, and its internal data flow is as follows: Temporary allocation prediction: Temporary allocation prediction is the first step of the load-aware submodule. Its purpose is to quickly estimate a temporary traffic allocation ratio based on the hidden characteristics of the current path nodes. This temporary ratio is not directly used as the final output, but is used to subsequently calculate link utilization, thereby providing load feedback to the link nodes.

[0050] The temporary prediction network consists of two fully connected layers: the first layer is Linear(hidden_dim,hidden_dim / / 2) + GELU + Dropout, which reduces the dimensionality of the input and then performs a nonlinear transformation, and finally regularization to prevent overfitting; the second layer is Linear(hidden_dim / / 2, 1) + Sigmoid, which maps the dimensionality-reduced features to a single scalar and then activates it with Sigmoid to compress it to the (0,1) interval as a temporary allocation ratio.

[0051] Calculate temporary traffic: Multiply the temporary allocation ratio by the path demand tensor (path_demands) to obtain the temporary traffic carried on each path, in Gbps.

[0052] Accumulate link load: Using the path link assignment table (path_link_assignment), the temporary traffic of each path can be accumulated to all the links it passes through using the index_add operation to obtain the temporary load of each link. ; in This is a temporary allocation ratio. For effective path requirements; Calculate link utilization: Divide the temporary load on the link by the link capacity to obtain the link utilization. ; in For link The equivalent available bandwidth is defined as follows: ; Link error rate; Integrating and updating link node features: In the forward propagation of GNN, each node has a set of hidden feature vectors. These feature vectors gradually aggregate neighbor information as the layers deepen, and the above information is injected into the features of the link node, so that the link node knows its current state and thus influences the decisions of neighboring path nodes in the next layer of GNN.

[0053] See Figure 6 The diagram shown is a logical diagram of the characteristics of the fusion link nodes, which will be used to define the current layer (assuming the first layer is the first layer). Hidden features of link nodes output by (layer) GNN Link utilization The input fusion network is concatenated along the feature dimension. The concatenation formula is as follows: ; The fusion network consists of two fully connected layers. The first layer, Linear(hidden_dim+1, hidden_dim) + GEL + Dropout, reduces the dimension from d+1 back to d and learns the non-linear combination between the original features and the utilization rate. The second layer, Linear(hidden_dim, hidden_dim), keeps the dimension unchanged and further transforms the features, outputting the updated link node features. ; Updated link node characteristics Shape Used to replace the original link node features It becomes part of the input to the next layer of GNN.

[0054] Residual connection and layer normalization: The output of the attention module is added to the input, and then layer normalization is performed.

[0055] Feedforward network: Two fully connected layers, with GELU activation and Dropout used in between, the output dimension keeps the hidden dimension unchanged, and then passes through residual connections and layer normalization.

[0056] Output projection layer: The hidden features output by the last GNN layer are transformed through a two-layer fully connected network.

[0057] S3 involves extracting path node features from the node features output by the encoder, mapping them to the original scores through a fully connected layer, and then performing Softmax normalization by grouping by flow to ensure that the sum of the distribution ratios of all paths within each flow is 1. Finally, it outputs the flow distribution ratio of each path.

[0058] S4 involves converting the path allocation ratio (traffic share per path) output by the decoder into routing table entries that the SAS extender can directly execute, and then distributing them to the hardware registers of each extender via the SMP protocol. Specifically, the following operations are performed: Path link mapping: Based on the node sequence and link sequence of each path, determine the ingress port and egress port of each extender that the path passes through, and collect the (target SAS address, output port, allocation ratio) triplet.

[0059] Routing table entry generation: For each extender, the path allocation ratio output by the decoder is first aggregated by (destination SAS address, output port) to obtain the total allocation ratio for each port. Then, for each target address, the allocation ratio of all its output ports is normalized to a sum of 1. Finally, based on the weight granularity supported by the hardware (e.g., 256 levels), the normalized ratios are converted into integer weights. max ,in This is the maximum weight value (e.g., 255). If rounding results in a non-maximum sum... If the weighted port is the largest, fine-tuning begins. The generated (destination address, output port, weight) entry is written to the extender's routing table, and the hardware performs weighted hash traffic allocation accordingly. If the hardware does not support weighting, proportional allocation is approximated by copying routing table entries.

[0060] Zoning constraint verification and correction: Read the Zoning partition configuration of the SAS domain and check whether each routing table entry allows communication. If the partition rule is violated, the traffic is redistributed to other compliant paths (if they exist); otherwise, the entry is deleted and the exception is logged.

[0061] SMP protocol delivery: The generated routing table entries are encapsulated into an SMP CONFIGURE ROUTE TABLE management frame and sent to the target extender via the SMP protocol, awaiting a response confirmation. If delivery fails, it is retried or rolled back to the previous version of the routing table.

[0062] S5 continuously monitors the actual operating status of the SAS domain and compares it with the predictions of the graph neural network model. When the deviation from the model's predictions is significant, or when the topology changes and then remains stable for a period of time, incremental learning is triggered to dynamically update the routing strategy, enabling the system to maintain optimal performance over the long term. The process is as follows: Performance data collection: The actual link utilization, error count, and topology status of each extender port are read at fixed intervals using the SMP protocol.

[0063] Deviation calculation: Based on the path allocation ratio output by the current model, calculate the theoretical utilization rate of each link and compare it with the actual utilization rate to obtain the absolute deviation.

[0064] Trigger judgment: If the deviation of a link exceeds a preset threshold (e.g., 20%) and continues for multiple cycles (e.g., 3 cycles), or if a topology change event is detected (e.g., link status change, hot-plugging, etc.), and the new state remains stable, then incremental learning is triggered.

[0065] Incremental learning: Extract recent samples from the cache (as shown in the figure, actual utilization rate), freeze the underlying parameters of the model, and fine-tune the last 1-2 layers of GNN and decoder only with a small learning rate (10-20 epochs).

[0066] Policy redeployment: Re-infer using the fine-tuned model, convert the newly allocated path proportions into SMP routing table entries through the policy compilation module, and distribute them to each extender.

[0067] This method also includes an offline training phase: (1) SAS topology generation Based on parameters in the configuration file (such as the number of extenders, cascading levels, number of extenders per level, number of hard drives mounted per extender, and link speed set), a large number of SAS topology samples are randomly generated. Two modes are supported: Completely random topology: The number of extenders, connection relationships, and link speeds (6 / 12 / 24 Gbps) are completely random, covering a variety of unknown scenarios.

[0068] User-specified topology: generated according to a fixed structure (e.g., three-level expanders, two expanders per level, and each expander connecting four hard drives), for optimization in specific deployment scenarios.

[0069] During the generation process, fault scenarios (single-link failure, dual-link failure, extender failure) can be selectively injected, enabling the model to learn fault response capabilities. Furthermore, link features include port arbitration priority and topology level information for subsequent node feature extraction.

[0070] (2) Generation of flow matrix For each topology, multiple end-to-end flows (host → hard drive) are generated. The bandwidth requirement of each flow is randomized within the range of 50–1500 Mbps, and the priority is randomly selected from {critical, high, medium, low, background}. The I / O mode is write or read. The total upper limit of the requirement is determined by multiplying the sum of the ingress bandwidth by the load factor (0.5 / 0.8 / 1.2 / 1.5). If the limit is exceeded, it is scaled proportionally. To simulate the downstream load attenuation in the cascaded topology, a tiered discount factor is introduced. (Using 0.8), the effective demand for each path is attenuated: ; in The raw demand for the flow (Mbps). The number of expander levels traversed by the path.

[0071] (3) Calculation of optimal route label During the offline training phase, the optimal route is assigned a label for each (topology, traffic matrix) sample by solving the following linear programming problem. The objective function and constraints are given below in a logical manner.

[0072] Objective function: ; in This represents the maximum link utilization rate in the entire SAS network.

[0073] Constraint expression: (1) Flow conservation constraint: , ; (2) Link capacity constraints: , ; (3) Variable range constraints: , , ; in, :flow In the path The allocation ratio (decision variable) on the [aspect]. Maximum link utilization; :flow The set of candidate paths; : Indicator function, if path via link The value is 1 if it is 1, otherwise it is 0. : Stream set; Link set.

[0074] The optimal allocation ratio of each flow on each candidate path is obtained as a supervision label. This problem is a linear programming problem and can be solved using a standard solver (such as HiGHS). If the solution fails, it falls back to a heuristic allocation based on path quality score (minimum equivalent bandwidth / total delay).

[0075] (4) Graph data construction Following the "path link bipartite graph" construction method described above, each sample is converted into a graph data structure, including: Node feature matrix (path node features and link node features) Edge index (path link connection relationship) Node type node_type(path, link) The flow index (marks the flow to which each path belongs) path_link_assignment table Equivalent available bandwidth

[0076] Effective demand

[0077] The final generated samples (graph data + optimal route label) are stored as training, validation, and test datasets.

[0078] (5) Offline training method Supervised learning of the graph neural network is performed using the generated dataset. The model structure, as described above, includes an encoder (containing a load-aware submodule) and a decoder.

[0079] 1) Loss Function A multi-objective weighted loss function is used: ; The loss in routing accuracy includes: ; Link utilization loss: ; in As an overload penalty, As a load balancing penalty, It is the traffic allocation ratio of the path predicted by the model. The optimal allocation ratio label is obtained through linear programming (or heuristics). It represents the total number of path nodes.

[0080] Each weight can be pre-configured according to the deployment scenario, or it can be dynamically adjusted during training (e.g., focusing on accuracy in the early stage and load balancing in the later stage).

[0081] 2) Optimizer and Training Strategy Optimizer: Adam, initial learning rate 0.00001, weight decay 0.0001, gradient clipping 0.1.

[0082] Learning rate scheduling: ReduceLROnPlateau, the learning rate is halved when the validation loss does not decrease for 4 consecutive epochs, with a minimum learning rate of 0.000001.

[0083] Early stop: If the validation loss does not improve for 20 consecutive epochs, then stop training.

[0084] Batch processing: Since the image sizes of different samples are different, a custom collate function is used to return batch data in the form of a list. The data is then processed one by one and the gradients are accumulated inside the model.

[0085] See Figure 7 The diagram shown is a training flowchart. The specific training process is as follows: (1) After the training process is officially started, the first step is to perform data preprocessing and construction. The first step is to generate path link bipartite graph data that meets the requirements of the model. Based on the network topology and business scenario, the core elements of the graph structure are extracted, and finally standardized graph data is output to complete all data preparation work before training.

[0086] (2) The prepared graph structure data is directly fed into the PathGNNEncoder encoder, which completes feature extraction and encoding through its internal PathGNNLayer layer. Inside the PathGNNLayer layer, multi-head graph attention operations, linear transformations, residual connections, and layer normalization operations are performed sequentially, and finally, the embedded features are output through a feedforward network (MLP). The encoded features are fed into the PathGNNDecoder decoder, which predicts link utilization and routing results, realizing the entire forward inference process of the model.

[0087] (3) After the decoder outputs the prediction results, it enters the dual-objective loss calculation stage. The link utilization loss (measuring the error between the predicted load and the actual load) and the routing accuracy loss (measuring the deviation between the predicted path and the actual routing strategy) are calculated separately. Then, the two losses are weighted and summed to obtain the total loss, which serves as the core basis for optimizing the model parameters.

[0088] (4) Based on the calculated total loss, the backpropagation process is initiated to calculate the gradients of the parameters of each layer of the model. The encoder and decoder parameters are updated through the optimizer. The updated parameters are synchronously fed back to the encoder and decoder to complete a single training iteration and prepare for the next round of training.

[0089] (5) After each epoch, evaluate the loss on the validation set. If it is better than the historical best, save the model. After the parameters are updated, end the training when the early stopping condition is met or the maximum number of epochs is reached, and output the best model parameters for online inference.

[0090] To achieve the above objectives, the present invention also provides a SAS extender intelligent routing system based on GNN, comprising a topology-aware module, a graph neural network inference engine, a policy compilation module, and a closed-loop feedback unit connected in sequence, wherein... Topology awareness module: used to collect the status of SAS domain devices, enumerate candidate paths and construct a path link bipartite graph; Graph Neural Network Inference Engine: Includes encoder and decoder, used to receive graph data and infer the traffic allocation ratio of each path based on load-aware message passing mechanism; The policy compilation module is used to convert the allocation ratio into SMP routing table entries that conform to the SAS specification and satisfy the Zoning constraints, and then execute the distribution. Closed-loop feedback unit: used to monitor performance metrics in real time and trigger incremental model learning when performance deviations or topology changes are detected.

[0091] Specific implementation details will not be elaborated further.

[0092] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Those skilled in the art can make various improvements and modifications to the invention without departing from its principles, and these improvements and modifications also fall within the scope of protection of the claims.

[0093] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A smart routing method for SAS extenders based on GNN, characterized in that, Includes the following steps: S1, Construct a bipartite graph model of path links; S2, execute load-aware graph neural network inference; S3, path allocation ratio decoding and normalization; S4 is compiled by the strategy and distributed by the hardware. S5, closed-loop monitoring and incremental learning optimization.

2. The intelligent routing method for SAS extenders based on GNN according to claim 1, characterized in that, In S1, communication with hosts, extenders, and hard drives in the SAS domain is achieved through the SMP protocol. The SAS address, physical port status, link negotiation rate, link latency, error rate, topology level, and zoning configuration of each device are collected. Based on the collected information, all acyclic candidate paths are enumerated for each end-to-end flow to construct a path link bipartite graph.

3. The intelligent routing method for SAS extenders based on GNN according to claim 1, characterized in that, S1 specifically includes the following steps: S11, Enumerate end-to-end flows and candidate paths: For each host and disk pair, find all acyclic paths using a graph search algorithm. For each path, record its node sequence, link sequence, total latency, minimum capacity, and number of hops. S12, Create Nodes: Each candidate path is abstracted into a path node, and each physical link is abstracted into a link node; S13, Establish an edge: If a path passes through a link, then establish an undirected edge between the corresponding path node and the link node; S14, Extract node features, including: Path node characteristics: traffic requirements, I / O pattern, total latency, hop count, minimum effective available bandwidth, and path quality score; Link node characteristics: equivalent available bandwidth, latency, series coding, state, current load estimate, and random noise; S15, Auxiliary data processing, auxiliary data includes: The flow index is a one-dimensional tensor used to identify which end-to-end flow each path node belongs to. Its length is equal to the total number of path nodes in the graph, and the value of each element is a globally unique identifier of the flow to which the path belongs. The path link association table is an integer tensor of shape [M, 2]. Each row (path_idx, link_idx) represents a path that passes through a physical link. This table stores all path link associations in advance and is used to quickly accumulate link load. Temporary traffic for each path is accumulated to the corresponding link according to the association table. Link capacity tensor: Stores the capacity of each link. ; Path demand tensor: Stores the flow demand for each path. .

4. The intelligent routing method for SAS extenders based on GNN according to claim 1, characterized in that, In S2, the graph data output by the topology sensing module is received, and multi-layer graph attention and load-aware message passing is performed through the encoder. Each graph attention layer contains a multi-head attention mechanism and a load-aware submodule. The temporary allocation ratio is estimated based on the current path node characteristics, the temporary load of the link is calculated, and the link utilization is calculated using the equivalent available bandwidth. Then, the utilization is fed back to update the link node characteristics.

5. The intelligent routing method for SAS extenders based on GNN according to claim 4, characterized in that, The load-aware submodule performs the following data stream processing: Temporary allocation prediction: Based on the hidden features of the current path nodes, the temporary traffic allocation ratio is estimated. This temporary ratio is used to calculate the link utilization and provide load feedback to the link nodes. The first layer of the temporary prediction network reduces the input dimension and then performs a nonlinear transformation, and then performs regularization to prevent overfitting. The second layer maps the dimensionality-reduced features to a single scalar and then uses Sigmoid activation to compress it to the (0,1) interval as the temporary allocation ratio. Calculate temporary traffic: Multiply the temporary allocation ratio by the path demand tensor to obtain the temporary traffic carried on each path, in Gbps; Accumulated Link Load: Based on the path link association table, the physical links traversed by each candidate path are determined, and the temporary traffic corresponding to each candidate path is accumulated onto the physical links it traverses, thereby obtaining the temporary load of each physical link. ； in This is a temporary allocation ratio. For the effective path demand, where is a tiered discount factor introduced to simulate downstream load attenuation in a cascaded topology. The effective demand for each path is attenuated: ； in, For the flow requirements corresponding to each path, The number of expander levels traversed by the path; Calculate link utilization: Divide the temporary load on the link by the link capacity to obtain the link utilization. ； in For link The equivalent available bandwidth is defined as follows: ； Link error rate; Integrating and updating link node features: In the forward propagation of GNN, each node has a set of hidden feature vectors. These feature vectors gradually aggregate neighbor information as the layers deepen, and the above information is injected into the features of the link node, so that the link node knows its current state and thus influences the decisions of neighboring path nodes in the next layer of GNN.

6. The intelligent routing method for SAS extenders based on GNN according to claim 5, characterized in that, S3 includes extracting path node features from the node features output by the encoder, mapping them to raw scores through a fully connected layer, grouping candidate paths according to the flow index, and normalizing the raw scores of each candidate path within the same end-to-end I / O flow so that the sum of their flow allocation ratios is 1.

7. The intelligent routing method for SAS extenders based on GNN according to claim 6, characterized in that, S4 includes the following steps: Convert the traffic allocation ratio of each path output by S3 into local routing table entries for each extender; Check each routing table entry for violations of Zoning constraints; if a violation is found, traffic is redistributed to other compliant paths. Encapsulate valid routing table entries into SMP management frames, send them to the hardware registers of the target extender via the SMP protocol, and wait for an acknowledgment response.

8. The intelligent routing method for SAS extenders based on GNN according to claim 7, characterized in that, S5 includes the following steps: Regularly collect actual link utilization and end-to-end latency performance indicators, and calculate the deviation between the actual values ​​and the theoretical predictions of the model. When the deviation exceeds a preset threshold or a topology change event is detected, incremental learning is triggered to fine-tune the model parameters and redeploy the routing strategy.

9. The intelligent routing method for SAS extenders based on GNN according to claim 1, characterized in that, This method also includes an offline training phase: Randomly generate SAS topology samples containing different cascading levels, link rates, and fault scenarios; A hierarchical discount factor is introduced to simulate the attenuation of the effective bandwidth requirement of deep paths; With the objective function of minimizing the maximum link utilization, a linear programming model containing flow conservation constraints and link capacity constraints is established, and the globally optimal route allocation ratio is obtained as a supervision label. Supervised learning is performed using a multi-objective weighted loss function, which includes at least routing accuracy loss and link utilization loss.

10. A SAS extender intelligent routing system based on GNN, characterized in that, The method described in any one of claims 1-9 comprises a topology perception module, a graph neural network inference engine, a policy compilation module, and a closed-loop feedback unit connected in sequence, wherein... Topology awareness module: used to collect the status of SAS domain devices, enumerate candidate paths and construct a path link bipartite graph; Graph Neural Network Inference Engine: Includes encoder and decoder, used to receive graph data and infer the traffic allocation ratio of each path based on load-aware message passing mechanism; The policy compilation module is used to convert the allocation ratio into SMP routing table entries that conform to the SAS specification and satisfy the Zoning constraints, and then execute the distribution. Closed-loop feedback unit: used to monitor performance metrics in real time and trigger incremental model learning when performance deviations or topology changes are detected.

Images