MIG fragment rearrangement method and system based on topology awareness and double-layer intelligent agent

Abstract

The invention provides an MIG fragment rearrangement method and system based on topology awareness and a double-layer agent, and belongs to the technical field of artificial intelligence. The method comprises the steps of collecting calculation of each partition and video memory occupation and remaining execution time, generating partition embedding through Kronecker multiplication, Hadamard product and PCIe neighborhood aggregation, constructing an address adjacency matrix, generating a global condition vector through Sigmoid gating and fusion pooling, and inputting a diffusion network to generate a fragment evolution trajectory and confidence. And calculating continuous splicable capacity based on confidence, constructing a graph structure, evaluating candidate migration actions by a lightweight graph strategy network, and selecting an optimal sequence through risk tensor and Bayesian calibration iterative search. And in the execution stage, the migration duration is controlled through an NVML lock table, RDMA straight pulling and a sliding window, and the device tree is updated after the migration duration is completed. Real evolution is continuously collected on line, deviation is monitored through divergence, the model is updated through distillation, and the optimal action and pause duration are recorded through an experience pool.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and more specifically relates to a MIG fragment rearrangement method and system based on topological perception and two-layer intelligent agents. Background Technology

[0002] In multi-tenant cloud computing pool scenarios, mainstream public cloud platforms often use static quotas or queue-based preemptive scheduling for GPU resource allocation. The scheduler sequentially traverses the node list based on the tenant's submitted memory and computing needs, selecting the GPU with the highest idle time and creating a MIG partition immediately. While this "first-come, first-served" strategy is simple and can complete resource delivery within seconds, it has significant drawbacks. Once a partition is created, its size and physical location are fixed. Subsequent high-frequency jobs arriving will create memory gaps of varying shapes and sizes within each GPU, leading to fragmentation. While the driver-layer memory partner allocator can defragment offline within a single card, the process is time-consuming, often requiring pausing some containers and performing memory-main memory-memory or inter-node copying, lasting in minutes. This is incompatible with the sub-second jitter scale of training and inference jobs, making it difficult to respond promptly to fragmentation issues.

[0003] To improve cluster resource utilization, some studies have introduced a dynamic transfer framework combining graph convolutional networks (GCNs) and reinforcement learning. This framework abstracts the GPU as graph nodes, extracts and embeds them through multiple GCNs, and then outputs actions via a policy network. While this can train fast-responding scheduling models in simulated environments, it faces numerous challenges in real-world deployment. State encoding, model inference, and action decision-making need to be completed within milliseconds. However, the temporal, topological, and resource dimensions are stored separately, requiring cross-tensor alignment before inference. Furthermore, the policy network often needs multiple samplings or backtracking to handle an exponential action space, resulting in significant online latency and computational overhead. In addition, the calculation of transfer costs during the decision-making phase is delayed, often leading to situations where selected transfer actions cannot be completed within the tolerable pause time.

[0004] Regarding GPU memory migration technology, existing solutions mostly employ CPU-based copy migration techniques, first performing page-based snapshots of the source GPU memory and then synchronizing the different pages to the target GPU. Others utilize NVLink P2P functionality to establish direct memory mapping between dual GPUs, using a DMA engine to pull data. While these systems combine bandwidth shaping and priority queue control to reduce migration disturbances, they fail to balance the risk of pauses with job priorities during the migration decision-making phase, easily triggering sudden jitter during peak load periods. CPU copy solutions are limited by the CPU and GPU data channels, resulting in low bandwidth utilization at both ends of the link. DMA migration based on NVLink P2P or direct connection within the same machine is constrained by hardware topology, requiring fallback to CPU or main memory in scenarios involving cross-switch or cross-data center connections.

[0005] In summary, existing technologies have shortcomings in GPU resource allocation, defragmentation, and memory data migration, making it difficult to meet the high requirements for resource utilization, scheduling efficiency, and job performance in multi-tenant cloud computing pool scenarios. Summary of the Invention

[0006] To address the above problems, the present invention aims to provide a MIG fragment rearrangement method and system based on topology awareness and a two-layer intelligent agent. This method can quickly predict the evolution trend of GPU MIG fragmentation under high-frequency load fluctuations. Under the premise of fully evaluating the migration costs of PCIe and RDMA, it selects a technical solution for cross-GPU partition splicing actions with minimal pause risk, thereby restoring continuous memory segments in a sub-second timescale and keeping job service jitter within the range allowed by the service level agreement.

[0007] To achieve the above objectives, the present invention employs the following technical solution: In a first aspect, embodiments of this application provide a MIG fragment rearrangement method based on topology awareness and a two-layer agent, comprising the following steps: S1: Collect the computing unit usage ratio and video memory usage ratio of the activated MIG partition, calculate the remaining execution time of the corresponding partition, and then superimpose the neighborhood aggregation result with the PCIe adjacency relationship; then use the linear mapping matrix to project the partition channels of different GPUs to a unified dimension, and sequentially concatenate them with the queued job template to obtain the three-dimensional embedding tensor. S2: Within each GPU, partitions are arranged in ascending order of physical memory address to construct a binary adjacency matrix. The partition size difference is combined with the Sigmoid gating function to obtain the concatenated gating weights. The gating weights are compressed into GPU condition vectors using bidirectional fusion pooling, and stacked according to GPU number to form a global condition matrix. S3: The 3D embedding tensor is copied by channel, aligned with the global condition matrix and spliced. The input is a UNet diffusion network containing high-resolution temporal convolution branches and low-resolution graph convolution branches. Multiple fragment evolution trajectories and corresponding confidence scores are generated by cross-scale gated coupling backsampling. A regional segmented integral calibration function is used to suppress spatiotemporal drift. S4: Based on the confidence level, the maximum continuous connectable memory ratio that each GPU can form within the future time window is weighted and averaged to obtain a continuous capacity tensor; a graph structure is constructed, and the original value of the action is output through a lightweight graph policy network, and the upper bound of the reward for each candidate action is calculated in combination with relevant factors; S5: Based on multiple fragment evolution trajectories, obtain the action time risk tensor, delete actions that do not meet the requirements, use the confidence level as a priori to perform Bayesian posterior calibration on the action reward, and use a depth-constrained iterative deepening backtracking algorithm to search for and output the action sequence with the best value under the constraint of the upper bound of the reward. S6: Call NVML to lock the page table of the partition to be migrated, estimate the upper bound of the theoretical migration time, trigger the RDMA hardware to perform data migration and adjust the sliding window in real time so that the actual migration time converges to the upper bound; dynamically rearrange the scheduler job priority within the migration window, and perform CRC32 verification, update the MIG device tree and release the page lock after the migration is completed. S7: After the migration action is completed, continuously collect the MIG fragment state of the GPU to obtain the real evolution sequence. Align the predicted sequence with the real sequence through dynamic time warping and calculate the relevant KL divergence. When the divergence exceeds the threshold, perform model distillation and replace the online instance. At the same time, write the optimal action and the real migration pause duration into the high-priority experience pool.

[0008] In an optional implementation, step S1 includes: S11: Obtain GPU index set For any GPU index Collect its activated MIG partition index set and the first The proportion of computing units and video memory usage in each partition is encapsulated into a vector. All vectors are stacked sequentially according to their partition numbers to obtain the base matrix. ; S12: Based on the benchmark matrix Collect the remaining execution time set for the corresponding partition. Construct a diagonal matrix ; Spatiotemporal alignment is achieved by combining Kronecker multiplication with Hadamard product, generating a spatiotemporal embedding tensor. :

[0009] in, Indicates Kronecker multiplication. Indicates element-wise multiplication. It is a matrix of all 1s; Spacetime Embedded Tensor While maintaining the two-dimensional resource description, the remaining execution time is pushed into the same quantity; S13: Introduce a global PCle adjacency matrix ,in If and only if the first Block GPUs are interconnected via a peer PCIe switch or a direct link; spatiotemporal embedding tensor Perform a neighborhood aggregation: first, for according to After filtering, summing on the resource dimension and superimposing it with the local tensor, we obtain the spatiotemporal embedding tensor. ; S14: Through a linear mapping matrix Embedding spacetime into tensors Projecting onto a fixed-dimensional representation generates a spatiotemporal embedding tensor. ;in, To unify the dimensions of the hidden space; S15: Obtain the vector matrix through isomorphic linear mapping using the queuing operation template. ; among them, each line and Maintain a consistent channel dimension; Embedding spacetime into tensors Stack them according to GPU number, then with Sequential cascading generates a 3D embedding tensor Z:

[0010] In this process, the stack is spliced ​​along the newly added sequence dimension using tensors. This is a sequential cascade operation.

[0011] In an optional implementation, step S2 includes: S21: In the On a block GPU, partition index set according to physical memory address Perform ascending sorting, and denote the sorted sequence as . Define the set of adjacent partition binary pairs. And convert it into a binary adjacency matrix. The matrix elements are 1 only at the corresponding positions of the two tuples; S22. To assign a splicing feasibility strength to each candidate edge, a size difference gating function is constructed for the binary adjacency matrix. Spatiotemporal embedding tensor of partitioned resource description To couple; The gating weight matrix of the size difference gating function Specifically as follows:

[0012] in, For the Sigmoid function; These are the defined video memory usage ratio and computing unit usage ratio, respectively. For empirical hyperparameters; S23: Constructing a feature splicing function ,in Indicates partition The implicit vector in step S1:

[0013] Based on the above formula, a GPU-level conditional vector with fixed dimensions is obtained. ; All GPU-level condition vectors Stack them sequentially according to GPU number to obtain the global condition matrix. .

[0014] In an optional implementation, step S3 includes: S31: For 3D embedding tensors Perform channel replication to align it with the feasibility weight matrix. Size compatible, aligned along partition dimensions and spliced ​​to obtain a joint state. ; Constructing high-resolution temporal convolution kernels Convolution kernel with low-resolution image Cross-scale coupled projection is employed:

[0015] in, This represents corresponding dimension convolution. It is a linear mapping matrix. and These are the time-domain and graph-domain normalization functions, respectively. Based on the above formula, the feasibility weights and dual-scale filtering are simultaneously fused at the input to generate a high-dimensional noise starting point. ; S32: Cross-scale coupled backsampling based on time-series-graph dual-resolution U-Net, assuming... For a fixed decay scheduling sequence, For independent Gaussian noise; for the first The trajectory uses cross-scale decoupling-coupling iteration:

[0016] In the formula, The time branch is composed of stacked one-dimensional separable convolutions, used to capture millisecond-level jitter in the job. For topological branches involving hierarchical graph convolution and pooling-upsampling, the adjacency matrix is ​​used. Dynamically adjust the receptive field to integrate PCIe bottlenecks into the hidden state; S33: Construct a calibration function based on partition-domain piecewise integration, and apply the following formula to the energy constraint of the time derivative for each trajectory:

[0017] in, Indicates the first Partitioning in a trajectory In GPU Time-domain occupancy vector; for and Instantaneous communication traffic between corresponding edges of the two GPUs in the topology graph; For PCle, it is the set of connected edges; S34: To generate confidence levels that can be directly used in migration value assessment, for the first... Calculate the invertible uniform entropy of the trajectories under the joint time-graphical distribution:

[0018] in, To slice the hidden tensor to the th The principal feature vectors obtained from two GPUs after singular value decomposition; It is a time-step increasing weight; Derived from the contiguous capacity expectation in S31, used to emphasize GPUs with severe fragmentation; Output confidence set and The evolutionary trajectories of these fragments collectively constitute the prior information for candidate migration generation.

[0019] In an optional implementation, step S4 includes: S41: Regarding the first GPU in the trajectory At time step The maximum continuously combinable video memory ratio that can be formed is denoted as . By introducing confidence weights and calculating the expectation within a time window, a continuous capacity tensor is constructed as follows. :

[0020] Among them, continuous capacity tensor Signals used to explicitly indicate the potential for capacity increases; S42: Construct a graph structure with partitions as nodes and connected edges as directed edges; The node features consist of the latent vector from step S1 and the continuous capacity tensor. The concatenation and edge features include the gated weight matrix from step S2. Compared with the predicted migration pause time ; In the graph structure, the original value of the action is output for each edge. First press Filter out the high-pause edges, then press Before selecting descending order Each action forms a preliminary candidate set.

[0021] S43: For each action The formula for calculating the upper bound of revenue, which includes the benefit-cost trade-off and is amplified by confidence level, is as follows:

[0022] in, Derived from the original value of action , The slope control constant is obtained by... This path compresses network scores to... The interval remains differentiable; Used to measure the extent of capacity improvement; Scaling of global confidence; This is the migration cost reduction factor. This is the predicted value for the duration of the action pause; After the calculation is completed, Encapsulated together with the action triple Generate a set of candidate migrations with an upper bound on the returns.

[0023] In an optional implementation, step S5 includes: S51: Reproject multiple fragment evolution trajectories onto the action dimension to obtain the action-time risk tensor. For each action Based on time step Calculate its confidence-based profit and loss equilibrium value across all trajectories:

[0024] in, For the first The GPU at time trace action target in the trajectory The maximum continuous idle ratio that can be formed Take the positive part. It is the numerical stability constant; A matrix is ​​generated after the calculation is completed. Used to numerically characterize if at time... implement The uncertainty balance between its benefits and costs provides a dense risk support surface for the necessary edge screening; S52: According to the matrix Define action-level confidence-risk indicators:

[0025] in, As a risk threshold, Increase modulation intensity For pause smoothing constant, For indicator functions; Targeting actions Only when The edge is considered a necessary edge and added to the simplified set. Otherwise, discard it directly. S53: Backtracking search based on state Indicates depth Action sequence; current extended action At that time, based on the trajectory confidence vector as prior, Bayesian posterior inference is performed on the value function:

[0026] in, For the first The trajectory is being executed Actual capacity increase forecast at that time For the trajectory in the window The variance of the internal capacity prediction. This is the linear cost reduction factor; S54: Employs depth restrictions The iterative deepening strategy optimizes the backtracking search, including: Set depth threshold ,exist Within the range, perform a depth-first search, increasing the search level exponentially in each round. Until ; After each round of search, the optimal value obtained from sampling will be... As a pre-pruning threshold for the next round, and carrying validated high-value sequences as a hot start path each time it is deepened, the cost of branch re-exploration caused by the dynamic fluctuation of MIG fragments is reduced. S55: When the search depth reaches If the upper bound of the value of all feasible branches is lower than the current global optimum, record the optimal action sequence. and corresponding comprehensive value Extract the first action of the sequence. Output and generate a risk receipt. Feedback is sent to the experience pool.

[0027] In an optional implementation, step S6 includes: S61: Query via NVML Partitions in the source GPU's page mapping table And perform locking on each physical page in user space; Based on RDMA port rate Link hop count and partition page compression factor An upper bound is given for the theoretical transfer time:

[0028] in, For the first Skip the switching chip forwarding delay, This represents the effective payload utilization rate of the jump. For the first Page weight compression; This serves as the theoretical upper limit, constraining the time window used for subsequent priority adjustments. S62: RDMA zero-copy data plane migration, after the locked page table is established, sends a work request to trigger the RDMA hardware engine to directly pull data from the source GPU memory; Real-time monitoring of link throughput curves ,like ( (As a load threshold coefficient), the IB sliding window is dynamically adjusted to ensure that the RDMA pipeline depth is synchronized with the link peak, so as to converge the actual migration time to the theoretical upper limit. ; S63: Time windows can be hot-swapped via the scheduler interface. Original priority sequence of all jobs Mapping to new differentiable priority trajectories The optimization objective is to maximize the time integral of the available contiguous partition length for each GPU after the migration is complete.

[0029] in, Indicates partition In GPU At the moment Occupied status, For indicator functions; S64: After the RDMA transfer is completed, verify that the CRC32 check value of the target GPU memory is consistent with that of the source. After successful verification, call NVML to update the MIG device tree and release the corresponding partition mapping. Differentiable priority trajectory The priority table is embedded into the scheduler.

[0030] In an optional implementation, step S7 includes: S71: Migration actions via scheduler Upon completion, a high-resolution monitoring task is initiated, continuously collecting global GPU-MIG fragment status at the same time granularity as in step S3, until future coverage is achieved. Step 1: After uniformly encoding all the original records, a true evolutionary sequence is formed. And make its structure match the diffusion model predicted sequence. Maintain correspondence; S72: The scheduler first uses a fixed-window dynamic time warping algorithm to... and Alignment is performed to ensure that the same event index reflects the same physical time. The difference between the prediction and reality distributions is calculated step-by-step across three dimensions: partition occupancy distribution, continuous idle segment distribution, and cross-GPU communication overhead. This difference is then summarized into an overall KL divergence index. ;like If the threshold is exceeded, the diffusion model is determined to require rapid distillation. S73: In the distillation stage, the original model weights are used as the teacher network, and its pruned lightweight copy is used as the student network. The following experience transfer learning scheme is adopted: Will As input, the intermediate multi-scale feature tensor output by the teacher serves as the soft label; Fine-tuning the student network using a joint loss of temperature scaling and cross-information entropy; After distillation is complete, the online inference instances are replaced with student network weights to achieve hot updates; S74: The following tuples Write the risk-backtracking agent's high-priority experience pool; among which... The actual migration pause duration is used; the high-priority experience pool adopts a time decay-confidence hybrid priority sampling mechanism to perform periodic incremental training on the policy network.

[0031] Secondly, embodiments of this application also provide a MIG fragment rearrangement system based on topology awareness and a two-layer intelligent agent, comprising: The data acquisition and embedding tensor generation module is used to acquire the computing unit occupancy ratio and video memory occupancy ratio of the activated MIG partition, and after processing with the remaining execution time of the corresponding partition, it is superimposed with the neighborhood aggregation result of the PCIe adjacency relationship; then, the partition channels of different GPUs are projected to a unified dimension using a linear mapping matrix, and sequentially concatenated with the queued job template to obtain the three-dimensional embedding tensor. The adjacency matrix and condition matrix construction module is used to construct a binary adjacency matrix within each GPU by partitioning in ascending order of physical memory address. The sigmoid gating function is used to obtain concatenated gating weights by combining the partition size differences. The gating weights are compressed into GPU condition vectors using bidirectional fusion pooling, and stacked according to GPU number to form a global condition matrix. The fragment evolution trajectory generation module is used to copy the 3D embedded tensor by channel, align it with the global condition matrix and stitch it together. The input is a UNet diffusion network containing high-resolution temporal convolution branches and low-resolution graph convolution branches. Multiple fragment evolution trajectories and corresponding confidence scores are generated through cross-scale gated coupling backsampling. A regional segmented integral calibration function is used to suppress spatiotemporal drift. The continuous capacity and action reward evaluation module is used to calculate the weighted average of the maximum continuous connectable memory ratio that each GPU can form within a future time window based on confidence level, and obtain the continuous capacity tensor; construct a graph structure, output the original value of the action through a lightweight graph policy network, and calculate the upper bound of the reward for each candidate action in combination with relevant factors; The optimal action sequence search module is used to obtain the action time risk tensor based on multiple fragment evolution trajectories, delete actions that do not meet the requirements, use the confidence level as a priori to perform Bayesian posterior calibration on the action reward, and use a depth-constrained iterative deepening backtracking algorithm to search for and output the action sequence with the best value under the upper bound constraint of the reward. The video memory migration and scheduling adjustment module is used to call NVML to lock the page table of the partition to be migrated, estimate the upper bound of the theoretical migration time, trigger the RDMA hardware to perform data migration and adjust the sliding window in real time so that the actual migration time converges to the upper bound; dynamically rearrange the scheduler job priority within the migration window, and perform CRC32 verification, update the MIG device tree and release the page lock after the migration is completed. The model optimization and experience update module is used to continuously collect the MIG fragment state of the GPU after the migration action is completed to obtain the real evolution sequence. It aligns the predicted sequence with the real sequence through dynamic time warping and calculates the relevant KL divergence. When the divergence exceeds the threshold, it performs model distillation and replaces the online instance. At the same time, it writes the optimal action and the real migration pause duration into the high-priority experience pool.

[0032] Thirdly, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the MIG fragment rearrangement method based on topology sensing and two-layer agents as described in any of the above.

[0033] Fourthly, embodiments of this application also provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the MIG fragment rearrangement method based on topology sensing and two-layer agents as described in any of the above embodiments.

[0034] As can be seen from the above technical solutions, the present invention has the following advantages: The MIG fragment rearrangement method based on topology awareness and two-layer agents provided in this application first collects the computation, memory usage, and remaining execution time of each partition. Partition embeddings are generated through Kronecker multiplication, Hadamard product, and PCIe neighborhood aggregation. Simultaneously, an address adjacency matrix is ​​constructed, and a global condition vector is generated through Sigmoid gating and fusion pooling. These two are input into a diffusion network to produce fragment evolution trajectories and confidence scores. Based on the confidence scores, the continuous splicable capacity is calculated, and a graph structure is constructed. A lightweight graph policy network evaluates candidate migration actions, and the optimal sequence is selected through iterative search using risk tensors and Bayesian calibration. During the execution phase, migration duration is controlled through NVML table locking, RDMA direct pull, and sliding window. After completion, CRC verification and device tree updates are performed. Real-world evolution is continuously collected online, and deviations are monitored using DTW and KL divergence. The model is updated by distillation, and the optimal action and pause duration are recorded in an experience pool. This method can quickly predict the evolution trend of GPUMIG fragmentation under high-frequency load fluctuations. Under the premise of fully evaluating the migration costs of PCIe and RDMA, it selects cross-GPU partition splicing actions with minimal pause risk, thereby restoring continuous memory segments in a sub-second timescale and keeping job service jitter within the range allowed by the service level agreement.

[0035] This application employs a unified embedding method combining Kronecker multiplication, Hadamard product, and neighborhood aggregation, integrating three types of key information into a single tensor at the input stage. Since there is no need to broadcast and align the scattered tensors before inference, the state encoding latency is significantly compressed. Furthermore, topological semantics participate in feature propagation at the shallowest level, allowing subsequent prediction and selection processes to naturally consider communication bottlenecks. This avoids, in principle, the feasibility shift caused by adding link costs only during the action generation stage in previous schemes.

[0036] In the prediction stage, this application introduces millisecond-level temporal convolution and low-resolution graph convolution simultaneously through a dual-resolution diffusion network, followed by cross-projection using learnable gating. This allows for the simultaneous capture of job jitter and topological relationships without significantly increasing the number of parameters. Compared to single-resolution temporal convolution or pure GCN prediction modes, this structure can provide more accurate and confident fragment evolution trajectories for high-frequency local variations and cross-GPU resource rearrangements, leaving reliable boundaries for subsequent decision-making.

[0037] This application employs a confidence-driven risk backtracking search that weights continuous capacity expectation, action value, and migration pauses within a unified upper bound of revenue. The search tree width scales linearly with risk, significantly reducing the combinatorial complexity of policy reasoning. Selected actions directly trigger GPUDirectRDMA zero-copy migration, and, in conjunction with priority hot-adjustment, maximize the integral of continuous idle time within the migration window, achieving a second-level closed loop from decision to execution. Compared to schemes requiring CPU relay or post-processing bandwidth shaping, this mechanism structurally shortens the data path, reduces interference with high-priority jobs during migration, and demonstrates better real-time scheduling adaptability. Attached Figure Description

[0038] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 A flowchart illustrating the MIG fragment rearrangement method based on topology sensing and two-layer agents provided in this application.

[0040] Figure 2 A flowchart illustrating another MIG fragment rearrangement method based on topology awareness and two-layer agents provided in this application.

[0041] Figure 3 A flowchart illustrating the method for constructing the feasibility weights for partition splicing provided in this application.

[0042] Figure 4 A flowchart illustrating the dual-resolution diffusion trajectory prediction method provided in this application. Figure 5 A schematic diagram of the structure of the MIG fragment rearrangement system based on topology perception and two-layer intelligent agents provided in this application.

[0043] Figure 6 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0044] The various embodiments of this disclosure will be described more fully in the detailed steps of the MIG fragment rearrangement method based on topology awareness and two-layer agents described below. This disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of this disclosure to the specific embodiments disclosed herein, but rather this disclosure should be understood to cover all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of this disclosure.

[0045] To facilitate a clear description of the technical solutions in the embodiments of this application, some terms and technologies involved in the embodiments of this application will be briefly introduced below: MIG: Multi-Instance GPU is a hardware-level GPU virtualization technology designed for data centers and AI computing scenarios. Its core function is to divide a single physical GPU into multiple independent small GPU instances, each of which can be allocated dedicated computing, memory, and caching resources, achieving resource isolation and efficient utilization.

[0046] In the following, the terms “comprising” or “may include”, which may be used in various embodiments of this disclosure, indicate the presence of the disclosed functions, operations, or elements, and do not limit the addition of one or more functions, operations, or elements. Furthermore, as used in various embodiments of this disclosure, the terms “comprising,” “having,” and their cognates are intended only to indicate a particular feature, number, step, operation, element, component, or combination of the foregoing, and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing, or the possibility of adding one or more combinations of the foregoing.

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] Please see Figure 1 The diagram shows a flowchart of a MIG fragment rearrangement method based on topology awareness and a two-layer agent in a specific embodiment. The method includes the following steps: S1: Collect the computing unit usage ratio and video memory usage ratio of the activated MIG partition, and after specific calculations, combine them with the remaining execution time of the corresponding partition and overlay them with the neighborhood aggregation results of the PCIe adjacency relationship; then use the linear mapping matrix to project the partition channels of different GPUs to a unified dimension, and concatenate them sequentially with the queued job template to obtain the three-dimensional embedding tensor.

[0049] In a specific implementation, the computational unit occupancy rate and video memory occupancy rate of each GPU in the cluster that has activated the MIG partition are first collected, and Kronecker multiplication and Hadamard product operations are performed with the remaining execution time of the corresponding partition. The results are then superimposed with the neighborhood aggregation results representing the PCle adjacency relationship.

[0050] Then, the partition channels of different GPUs are projected to a unified dimension using a linear mapping matrix and sequentially concatenated with the queued job template to obtain a single three-dimensional embedding tensor.

[0051] S2: Within each GPU, partitions are arranged in ascending order of physical memory address to construct a binary adjacency matrix. The partition size difference is combined with the Sigmoid gating function to obtain the concatenated gating weights. The gating weights are compressed into GPU condition vectors using bidirectional fusion pooling and stacked according to GPU number to form a global condition matrix.

[0052] S3: The 3D embedding tensor is copied by channel, aligned with the global condition matrix and concatenated. The input is a UNet diffusion network containing high-resolution temporal convolution branches and low-resolution graph convolution branches. Multiple fragment evolution trajectories and corresponding confidence scores are generated through cross-scale gated coupling backsampling. A regional segmented integral calibration function is used to suppress spatiotemporal drift.

[0053] In a specific implementation, the embedding tensor obtained in step S1 is copied by channel and aligned with the global condition matrix obtained in step S2 and then concatenated. This is then used as input to a UNet diffusion network that includes a high-resolution temporal convolution branch and a low-resolution graph convolution branch.

[0054] Then, multiple fragment evolution trajectories and corresponding confidence levels are generated through cross-scale gated coupling backsampling, and spatiotemporal drift is suppressed by a regional segmented integral calibration function.

[0055] S4: Based on the confidence level, the maximum continuous splicable memory ratio that each GPU can form within the future time window is weighted and averaged to obtain the continuous capacity tensor; a graph structure is constructed, and the original value of the action is output through a lightweight graph policy network, and the upper bound of the revenue of each candidate action is calculated in combination with relevant factors.

[0056] In a specific implementation, the maximum proportion of continuous connectable video memory that each GPU can form within a future time window is first weighted and averaged based on the confidence level to obtain a continuous capacity tensor.

[0057] Then, a graph structure is constructed using partitions as nodes and feasible edges as directed edges. Node features are composed of embedded vectors and continuous capacity, while edge features include gating weights and migration pause prediction values. The original value of the action is output through a lightweight graph policy network.

[0058] Finally, candidate actions are filtered by pause threshold and sorting, and the upper bound of the revenue for each candidate action is calculated by combining capacity increase, original value, global confidence and migration cost.

[0059] S5: Based on multiple fragment evolution trajectories, obtain the action time risk tensor, delete actions that do not meet the requirements, use the confidence level as a priori to perform Bayesian posterior calibration on the action reward, and use a depth-constrained iterative deepening backtracking algorithm to search for and output the action sequence with the optimal value under the constraint of the upper bound of the reward.

[0060] In a specific implementation, the action time risk tensor is first obtained by reprojecting multiple fragment evolution trajectories, and then actions with low confidence and high risk are deleted according to a preset risk threshold.

[0061] Then, using confidence as a priori, Bayesian posterior calibration is performed on the action reward. A depth-constrained iterative backtracking algorithm is used to search for and output the action sequence with the optimal value under the upper bound constraint of the reward.

[0062] S6: Call NVML to lock the page table of the partition to be migrated, estimate the upper bound of the theoretical migration time, trigger the RDMA hardware to perform data migration and adjust the sliding window in real time so that the actual migration time converges to the upper bound; dynamically rearrange the scheduler job priority within the migration window, and perform CRC32 verification, update the MIG device tree and release the page lock after the migration is completed.

[0063] In a specific implementation, NVML is first invoked to lock the page table of the partition to be migrated, and the upper bound of the theoretical migration time is estimated based on the RDMA link rate, hop count and page compression factor.

[0064] Then, the RDMA hardware is triggered to directly pull data from the GPU memory and adjust the sliding window in real time, so that the actual migration time converges to the upper bound.

[0065] Finally, within the migration window, the scheduler job priorities are dynamically rearranged to maximize the time integral of the length of consecutive free partitions for each GPU after the migration is completed. After the migration is completed, CRC32 verification is performed, the MIG device tree is updated, and page locks are released.

[0066] S7: After the migration action is completed, continuously collect the MIG fragment state of the GPU to obtain the real evolution sequence. Align the predicted sequence with the real sequence through dynamic time warping and calculate the relevant KL divergence. When the divergence exceeds the threshold, perform model distillation and replace the online instance. At the same time, write the optimal action and the real migration pause duration into the high-priority experience pool.

[0067] In a specific implementation, after the migration action is completed, the GPUMIG fragment state is continuously collected at the same granularity as in step S3 to obtain the real evolution sequence. The predicted sequence and the real sequence are aligned by dynamic time warping, and the KL divergence, which includes partition occupancy distribution, continuous free segment distribution and cross-GPU communication overhead, is calculated.

[0068] When the divergence exceeds the threshold, temperature scaling and cross-entropy joint distillation are performed using the original model as the teacher and the lightweight copy as the student, and the online instance is replaced.

[0069] Meanwhile, the optimal action and the actual transfer pause duration are written into the high-priority experience pool for subsequent incremental training of the policy network.

[0070] In this embodiment, the efficiency and accuracy of MIG fragment rearrangement are effectively improved through multi-dimensional information fusion and intelligent decision-making mechanisms. Specifically, a unified three-dimensional embedding tensor is constructed using operations such as Kronecker multiplication, Hadamard product, and neighborhood aggregation. A global condition matrix is ​​generated by combining the differences in memory physical addresses and partition sizes, providing rich input features for subsequent network processing. The fragment evolution trajectory is generated by using a U-Net diffusion network to suppress spatiotemporal drift. High-value, low-risk actions are selected by combining a graph policy network and Bayesian posterior calibration, achieving intelligent optimization of migration actions. The migration process is dynamically adjusted based on RDMA technology, which shortens the migration time while ensuring data integrity and maximizes the time integral of continuous idle partitions, thereby improving memory resource utilization. Through dynamic time warping alignment prediction and real sequences, combined with joint distillation and experience pooling mechanisms, the model is continuously optimized, enabling the system to adapt to changes in cluster state. Ultimately, this demonstrates significant advantages in reducing cross-GPU communication overhead, reducing fragment aggregation, improving resource allocation flexibility, and enhancing long-term system robustness.

[0071] Furthermore, as a refinement and extension of the specific implementation methods described above, and to fully illustrate the specific implementation process in this embodiment, another MIG fragment rearrangement method based on topology awareness and two-layer intelligent agents is provided, such as... Figure 2 As shown, the specific steps are as follows: Step 1: Collect the computation unit occupancy and memory occupancy ratio of the activated MIG partition for each GPU in the cluster, and perform Kronecker multiplication and Hadamard product operations with the remaining execution time of the corresponding partition. Superimpose the results with the neighborhood aggregation results representing PCle adjacency relationships. Then, use a linear mapping matrix to project the partition channels of different GPUs to a unified dimension and concatenate them sequentially with the queued job template to obtain a single three-dimensional embedding tensor.

[0072] Step 2: Within each GPU, partitions are arranged in ascending order of physical memory address. A binary adjacency matrix is ​​constructed based on the adjacency relationship. Then, the partition size difference is combined with the Sigmoid gating function to obtain the concatenation gating weights. The gating weights are compressed into GPU condition vectors using bidirectional fusion pooling and stacked according to GPU number to form a global condition matrix.

[0073] Step 3: Copy the embedded tensor obtained in Step 1 by channel, align it with the global condition matrix obtained in Step 2, and concatenate them. Input a U-Net diffusion network containing high-resolution temporal convolution branches and low-resolution graph convolution branches. Generate multiple fragment evolution trajectories and corresponding confidence scores through cross-scale gated coupling backsampling, and use a regional segmented integral calibration function to suppress spatiotemporal drift.

[0074] Step 4: Based on the confidence level, perform a weighted average of the maximum continuous splicable memory ratio that each GPU can form within the future time window to obtain a continuous capacity tensor; construct a graph structure with partitions as nodes and splicable feasible edges as directed edges. The node features are composed of embedding vectors and continuous capacity splicing, and the edge features include gating weights and migration pause prediction values. The original value of the action is output through a lightweight graph policy network. Candidate actions are selected according to the pause threshold and sorting, and the upper bound of the revenue for each candidate action is calculated by combining the capacity increase, original value, global confidence level, and migration cost.

[0075] Step 5: Obtain the action-time risk tensor based on the reprojection of multiple fragment evolution trajectories, delete actions with low confidence and high risk according to the preset risk threshold; use the confidence level as the prior to perform Bayesian posterior calibration on the action reward, and use the depth-constrained iterative deepening backtracking algorithm to search for and output the action sequence with the optimal value under the upper bound constraint of the reward.

[0076] Step 6: Invoke NVML to lock the page table of the partition to be migrated, and estimate the upper bound of the theoretical migration time based on the RDMA link rate, hop count, and page compression factor; trigger the RDMA hardware to directly pull data from the GPU memory and adjust the sliding window in real time so that the actual migration time converges to the upper bound; dynamically rearrange the scheduler job priorities within the migration window to maximize the time integral of the length of consecutive free partitions of each GPU after the migration is completed; after the migration is completed, perform CRC32 verification, update the MIG device tree, and release the page lock.

[0077] Step 7: After the migration action is completed, GPU-MIG fragment states are continuously collected at the same granularity as in Step 3 to obtain the real evolution sequence. The predicted sequence and the real sequence are aligned by dynamic time warping, and the KL divergence, which includes the partition occupancy distribution, the continuous free segment distribution, and the cross-GPU communication overhead, is calculated. When the divergence exceeds the threshold, temperature scaling and cross-information entropy joint distillation are performed with the original model as the teacher and the lightweight copy as the student, and the online instance is replaced. At the same time, the optimal action and the real migration pause duration are written into the high-priority experience pool for subsequent incremental training of the policy network.

[0078] In one embodiment of the present invention, based on step S1, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0079] Step S1 includes the following sub-steps: S11: First, provide the GPU index set at the cluster level. For any Collect its activated MIG partition index set and the first The proportion of computing units and video memory usage in each partition is encapsulated into a vector. All vectors are stacked sequentially according to their partition numbers to obtain the base matrix. The matrix only depicts the shape of spatial fragments.

[0080] S12: For the matrix Internally, time-dimensional semantics are written, and the remaining execution time set of the corresponding partition is collected. Construct a diagonal matrix Spatiotemporal alignment is achieved using Kronecker multiplication combined with Hadamard product, generating a spatiotemporal embedding tensor:

[0081] in Indicates Kronecker multiplication. Indicates element-wise multiplication. It is an all-1 matrix, and its function is to replicate the time weight across the resource channel dimension.

[0082] result While maintaining the two-dimensional resource description, the remaining execution time is pushed into the same tensor, avoiding additional cross-tensor alignment operations in the later network.

[0083] S13: To explicitly perceive cross-GPU migration costs within the same GPU embedding, this step introduces a global PCIe adjacency matrix. ,in If and only if the first Block GPUs communicate with each other via a peer PCIe switch or a direct link. Perform a neighborhood aggregation: first, for according to After filtering, summing the results over the resource dimension and superimposing them on the local tensor, we obtain... .

[0084] Unlike existing technologies that only use topological information in subsequent graph convolutional layers, this fusion method injects the communication neighborhood distribution at the input end, generating migration feasibility attention in advance at the shallow layer of the network.

[0085] S14: Due to the different number of partitions on different GPUs Inconsistency requires a linear mapping matrix. ( To unify the dimensions of the latent space) Projected onto a fixed dimension: This completes cross-GPU channel alignment and also provides equal-length hidden feature channels for subsequent serialization.

[0086] S15: The queuing operation template is obtained as a vector matrix through isomorphic linear mapping. .

[0087] Each row and Maintain a consistent channel dimension. Ultimately, each Stack them according to GPU number, then with Sequential concatenation forms a unified topology-temporal embedding sequence:

[0088] In this process, the stack is spliced ​​along the newly added sequence dimension using tensors. This is a sequential cascade operation.

[0089] Based on this The complete encoding of spatial fragmentation within a single form, along with the remaining execution time and PCIe topology information, provides sufficient information and minimal redundancy for subsequent fragment attention gating and dual-resolution diffusion networks, resulting in a unified input.

[0090] In one embodiment of the present invention, based on step S2, the following will provide a possible embodiment and its specific implementation in a non-limiting manner. See also Figure 3 As shown, step S2 discloses a method for constructing a partition splicing feasibility weight, which specifically includes the following sub-steps: S21: In the On a block GPU, partition index set according to physical memory address Perform ascending sorting, and denote the sorted sequence as . Define the set of adjacent partition pairs. And convert it into a binary adjacency matrix. The matrix elements are set to 1 only at the corresponding positions of the binary pairs. This matrix can be used to identify potential concatenable edges at the top level of the computation graph.

[0091] S22: To assign a splicing feasibility strength to each candidate edge, a size difference gating function is constructed. With partition resource description Coupled in place. Gated weight matrix. It is given by the following formula:

[0092] in, For the Sigmoid function; The steps are as follows The video memory usage ratio and computing unit usage ratio are defined in the document; This is an empirical hyperparameter.

[0093] Partitions that are close in size and physically adjacent are mapped to high weights, which can directly quantify the feasibility of fragment splicing. For partitions with large resource differences or that are not adjacent, the Sigmoid output approaches zero, achieving hard pruning.

[0094] S23: To compress partition-level gating information into GPU-level conditional vectors, weighted bidirectional fusion pooling is employed. First, a feature concatenation function is constructed. ,in Indicates partition The implicit vector in step S1:

[0095] Based on the above formula, a GPU-level conditional vector with fixed dimensions is obtained. All By stacking the GPUs sequentially by their numbers, the global condition matrix can be obtained. This matrix serves as the feasibility weight condition input for the subsequent dual-resolution U-Net diffusion model.

[0096] This step eliminates gradient paths for non-transferable fragments at the model front end by using joint sparsification of the physical adjacency matrix and size difference gating, thereby reducing the ineffective exploration space during the diffusion stage.

[0097] In one embodiment of the present invention, based on step S3, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0098] See Figure 4 As shown, step S3 discloses a dual-resolution diffusion trajectory prediction method, which specifically includes the following sub-steps: S31: The structured tensor output from step S1 Perform channel replication to align it with the feasibility weight matrix. Size compatible; then aligned and stitched along the partition dimensions to obtain the joint state. To accommodate the scale requirements of subsequent dual-resolution streams, high-resolution temporal convolutional kernels are constructed. Convolution kernel with low-resolution image Cross-scale coupled projection is employed:

[0099] in This represents corresponding dimension convolution. It is a linear mapping matrix. and These are the time-domain and graph-domain normalization functions, respectively.

[0100] Based on the above formula, the feasibility weights and dual-scale filtering are simultaneously fused at the input to generate a high-dimensional noise starting point. This ensures that each subsequent step of the anti-proliferation process explicitly detects which fragment is more likely to be pieced together.

[0101] S32: Cross-scale coupled backsampling based on time-series-graph dual-resolution U-Net, assuming... For a fixed decay scheduling sequence, It is independent Gaussian noise. For the ... The trajectory uses cross-scale decoupling-coupling iteration:

[0102] In the formula, The time branch is composed of stacked one-dimensional separable convolutions, used to capture millisecond-level jitter in the job. For topological branches involving hierarchical graph convolution and pooling-upsampling, the adjacency matrix is ​​used. Dynamically adjust the receptive field to integrate PCIe bottlenecks into the hidden state.

[0103] After the two branches are coupled at each layer through a learnable gated tensor, they participate in the above recursion to achieve cross-resolution collaborative backsampling.

[0104] S33: To suppress potential spatiotemporal drift during backsampling, a calibration function based on partition-domain piecewise integration is constructed and applied to each trajectory:

[0105] in, Indicates the first Partitioning in a trajectory In GPU Time-domain occupancy vector; for and Instantaneous communication traffic between corresponding edges of the two GPUs in the topology graph; This is the set of PCle connection edges.

[0106] That is, applying energy constraints to the time derivative based on the double integral in the above formula is equivalent to implementing second-order smoothing in the latent space, so that the trajectory remains physically feasible in both the time domain and the topological domain. This calibration loss is only used within the sampling process and is not visible in the external process, effectively reducing the probability of trajectory oscillations in sudden high-density operation scenarios.

[0107] S34: To generate confidence levels that can be directly used in migration value assessment, for the first... Calculate the invertible uniform entropy of the trajectories under the joint time-graphical distribution:

[0108] in, To slice the hidden tensor to the th The principal feature vectors obtained from two GPUs after singular value decomposition; It is a time-step increasing weight; Derived from the contiguous capacity expectation in S31, it is used to emphasize GPUs with severe fragmentation.

[0109] This sub-step measures the representation drift in the topological domain from a distributional perspective and normalizes it after exponential mapping. A higher value indicates that the trajectory is more stable in terms of global spatiotemporal consistency, and therefore more reliable.

[0110] Finally, this sub-step outputs a set of confidence scores. and The evolutionary trajectories of these fragments collectively constitute the prior information for generating subsequent candidate migrations.

[0111] In one embodiment of the present invention, based on step S4, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0112] Step S4 includes the following sub-steps: S41: Calculation of confidence-weighted continuous capacity tensors. First, for the first... GPU in the trajectory At time step The maximum continuously combinable video memory ratio that can be formed is denoted as . To extract stable capacity enhancement potential from multiple trajectories, confidence weights are introduced and the expectation is calculated within a time window to construct a continuous capacity tensor:

[0113] In this calculation, the numerator is a confidence-weighted sum of all trajectories within the same time step, the denominator is normalized, and the time average reflects future trajectories. The expected probability that the GPU will generate a long, continuous free segment within a step. It will be used as a global feature of nodes in the subsequent graph-policy network to explicitly indicate scenario signals with capacity improvement potential.

[0114] S42: Construct a graph structure with partitions as nodes and connectable edges as directed edges: The node features are derived from the latent vector in step S1 and the latent vector in sub-step S41. The concatenation and edge features include the gating weights from step S2 and the predicted migration pause time. .

[0115] The Lightweight Graph-Policy Network employs two layers of graph convolutions and one layer of linear policy heads, outputting the original value of the action for each edge. To ensure that the cost is limited, firstly... Filter out the high-pause edges, then press Before selecting descending order Each action forms a preliminary candidate set. .

[0116] S43: Assigning an upper bound to the confidence-weighted return, to ensure that the subsequent risk-backtracking agent can focus on high-value branches, for each action... Calculate the upper bound of revenue that includes the benefit-cost trade-off and is amplified by confidence level.

[0117] in, The original value of the action from S42 The slope control constant is obtained by... This path compresses network scores to... The interval remains differentiable; Measure the extent of capacity improvement; It plays a role in global confidence scaling in the formula; This is the migration cost reduction factor. This is the predicted value for the duration of the action pause. After calculation, it will be... Encapsulated together with the action triple This approach generates a candidate migration set with an upper bound on the benefit. The upper bound simultaneously considers capacity improvement potential, action value score, global confidence, and pause cost, ensuring that the next backtracking search can converge quickly along the optimal direction within the minimum width.

[0118] In one embodiment of the present invention, based on step S5, a possible embodiment will be given below, and its specific implementation will be described in a non-limiting manner.

[0119] Step S5 includes the following sub-steps: S51: Trajectory-Action Joint Risk Surface Construction. This sub-step reprojects multiple fragmented evolution trajectories onto the action dimension to obtain the action-time risk tensor. For each action Time step Calculate its confidence-based profit and loss equilibrium value across all trajectories:

[0120] in, For the first The GPU at time trace action target in the trajectory The maximum continuous idle ratio that can be formed Take the positive part. This is the numerical stability constant. (Matrix) Visualization, that is, numerical representation of the value at time... implement The balance between the uncertainty of its benefits and costs provides a dense risk support surface for the necessary edge screening.

[0121] S52: Must-pass edge confidence clipping, based on Define action-level confidence-risk indicators:

[0122] in As a risk threshold, Increase modulation intensity For pause smoothing constant, This is an indicator function.

[0123] action Only when The edge is considered a necessary edge and added to the simplified set. Otherwise, discard it directly to avoid the subsequent backtracking tree diverging in the low confidence region.

[0124] The strategy filters out high-risk events at multiple times using the above formula, while applying a power-law penalty to the pause cost, making it specialized for hardware scenarios where the RDMA zero-copy migration window is limited.

[0125] S53: Backtracking search based on state Indicates depth Action sequence. Extended actions At that time, based on the trajectory confidence vector as prior, Bayesian posterior inference is performed on the value function:

[0126] in, For the first The trajectory is being executed Actual capacity increase forecast at that time For the trajectory in the window The variance of the internal capacity prediction. This is the linear cost reduction factor.

[0127] The above formula uses Gaussian likelihood to measure the difference between action gain and trajectory prediction, and uses confidence level. Assign weights and output the calibrated value; if the posterior update is... If the search width falls below the current optimal upper bound, backtrack immediately to ensure that the search width is contracted in tandem with the risk.

[0128] S54: Depth-bounded iterative backtracking, to avoid combinatorial explosion in high-dimensional action space, the agent employs depth constraints. Iterative deepening strategy: First set a smaller depth threshold ,exist Within the range, perform a depth-first search, then increase the search level exponentially in successive rounds. Until .

[0129] After each round of search, the optimal value obtained from sub-step S53 is sampled. This serves as a pre-pruning threshold for the next round, thereby incrementally refining the value profile and carrying validated high-value sequences as a hot-start path each time it is deepened, reducing the cost of branch re-exploration caused by the dynamic fluctuations of MIG fragments.

[0130] S55: When the search depth reaches If the upper bound of the value of all feasible branches is lower than the current global optimum, record the optimal action sequence. and corresponding comprehensive value The agent only extracts the first action of the sequence. Output and generate a risk receipt. Feedback is sent to the experience pool.

[0131] Since substeps S51-S54 explicitly couple trajectory confidence and benefit-cost terms at each layer It ensures that migration pauses minimize the disturbance to high-priority jobs, while maximizing the lower bound of revenue in the face of uncertain fragmentation evolution, thus meeting the actual needs of cloud GPU pools for high utilization and low jitter.

[0132] In one embodiment of the present invention, based on step S6, a possible embodiment will be given below, and its specific implementation will be described in a non-limiting manner.

[0133] Step S6 includes the following sub-steps: S61: Query via NVML Partitions in the source GPU's page mapping table And in user space, lock each physical page.

[0134] Subsequently based on the RDMA port rate Link hop count and partition page compression factor An upper bound is given for the theoretical transfer time:

[0135] in, For the first Skip the switching chip forwarding delay, This represents the effective payload utilization rate of the jump. For the first Page weight compression; Time window boundary constraints used for subsequent priority adjustments.

[0136] S62: RDMA zero-copy data plane migration. After the page table is locked and established, a work request is sent, triggering the RDMA hardware engine to directly pull data from the source GPU memory without CPU copying.

[0137] During the monitoring of the link throughput curve ,like ( (If the load threshold coefficient is used, the IB sliding window is dynamically adjusted to ensure that the RDMA pipeline depth is synchronized with the link peak, thereby converging the actual migration time to the theoretical upper limit of sub-step S61.)

[0138] S63: To avoid migration pauses squeezing high-priority training jobs, the time window is hot-swapped via the scheduler interface. Original priority sequence of all jobs Mapping to new differentiable priority trajectories The optimization objective is to maximize the time integral of the available contiguous partition length for each GPU after the migration is complete.

[0139] in, Indicates partition In GPU At the moment Occupied status, S63 is an indicator function. By integrating the durations of consecutive idle segments, it directly drives the scheduler's priority adjustment to converge toward maximizing the span of consecutive idle time segments.

[0140] S64: After the RDMA transfer is completed, verify that the CRC32 check value of the target GPU memory is consistent with the source end, and then call NVML to update the MIG device tree and release the corresponding partition mapping; Simultaneously, the optimal rearrangement obtained in sub-step S63 is... The priority table is embedded into the scheduler.

[0141] Based on this, a closed loop of page locking, zero-copy migration, and dynamic priority adjustment is completed, maximizing the length of the target GPU's continuously available partition. The entire process is transparent to user jobs, meeting the dual requirements of cloud service level agreements for high availability and low jitter.

[0142] In one embodiment of the present invention, based on step S7, a possible embodiment will be given below, and its specific implementation will be described in a non-limiting manner.

[0143] Step S7 includes the following sub-steps: S71: Scheduler during migration actions Upon completion, immediately initiate a high-resolution monitoring task, continuously collecting global GPU-MIG fragment status at the same time granularity as step S3, until future coverage is achieved. Step. All original records are uniformly encoded to form a true evolutionary sequence. Its structure and diffusion model predicted sequence Maintain correspondence.

[0144] S72: To eliminate slight timing drift caused by load jitter, the scheduler first uses a fixed-window dynamic time warping algorithm to... and Alignment is performed to ensure that the same event index reflects the same physical time. Then, the difference between the prediction and reality distributions is calculated time-step by time across three dimensions: partition occupancy distribution, continuous idle segment distribution, and cross-GPU communication overhead. This difference is then summarized into an overall KL divergence index. .like If the threshold is exceeded, the diffusion model is determined to require rapid distillation.

[0145] S73: In the distillation stage, the original model weights are used as the teacher network, and its pruned lightweight copy is used as the student network. An experience transfer learning scheme is adopted. Will As input, the intermediate multi-scale feature tensor output by the teacher serves as the soft label; Rapidly fine-tune the student network using a joint loss of temperature scaling and cross-information entropy; After distillation, online inference instances are replaced with student network weights to achieve hot updates. This strategy can update within seconds. Compress to below the threshold, no full retraining is required.

[0146] S74: Finally, the following tuples Write risk – the high-priority experience pool of the backtracking agent, where The actual migration pause duration is determined. The experience pool employs a time decay-confidence hybrid priority sampling mechanism to perform periodic incremental training on the policy network. After the update, the agent will use the latest policy function and convergence prior in the subsequent S5 search, thereby further shortening the backtracking depth and reducing the number of redundant migration triggers.

[0147] like Figure 5 As shown, the following are embodiments of the MIG fragment rearrangement system based on topology awareness and two-layer agents provided in this disclosure. This system and the MIG fragment rearrangement method based on topology awareness and two-layer agents in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the MIG fragment rearrangement system based on topology awareness and two-layer agents, please refer to the embodiments of the MIG fragment rearrangement method based on topology awareness and two-layer agents described above.

[0148] A MIG fragment rearrangement system based on topology awareness and a two-layer intelligent agent includes: The data acquisition and embedding tensor generation module is used to acquire the computing unit occupancy ratio and video memory occupancy ratio of the activated MIG partition, and after specific calculations with the remaining execution time of the corresponding partition, it is superimposed with the neighborhood aggregation result of the PCIe adjacency relationship; then, the partition channels of different GPUs are projected to a unified dimension using a linear mapping matrix, and sequentially concatenated with the queued job template to obtain the three-dimensional embedding tensor.

[0149] The adjacency matrix and condition matrix construction module is used to construct a binary adjacency matrix in each GPU by partitioning the GPUs in ascending order of their physical memory addresses. The module combines the partition size differences with the sigmoid gating function to obtain the concatenated gating weights. The gating weights are compressed into GPU condition vectors using bidirectional fusion pooling and stacked according to GPU number to form a global condition matrix.

[0150] The fragment evolution trajectory generation module is used to copy the 3D embedded tensor by channel, align it with the global condition matrix, and then stitch it together. The input is a UNet diffusion network containing high-resolution temporal convolution branches and low-resolution graph convolution branches. Multiple fragment evolution trajectories and corresponding confidence scores are generated through cross-scale gated coupling backsampling, and spatiotemporal drift is suppressed by using a regional segmented integral calibration function.

[0151] The continuous capacity and action reward evaluation module is used to perform a weighted average of the maximum continuous splicable memory ratio that each GPU can form within a future time window based on confidence level, and obtain a continuous capacity tensor; construct a graph structure, output the original value of the action through a lightweight graph policy network, and calculate the upper bound of the reward for each candidate action in combination with relevant factors.

[0152] The optimal action sequence search module is used to obtain the action time risk tensor based on multiple fragment evolution trajectories, delete actions that do not meet the requirements, use the confidence level as a priori to perform Bayesian posterior calibration on the action reward, and use a depth-constrained iterative deepening backtracking algorithm to search for and output the action sequence with the optimal value under the constraint of the upper bound of the reward.

[0153] The video memory migration and scheduling adjustment module is used to call NVML to lock the page table of the partition to be migrated, estimate the upper bound of the theoretical migration time, trigger the RDMA hardware to perform data migration and adjust the sliding window in real time so that the actual migration time converges to the upper bound; dynamically rearrange the scheduler job priority within the migration window, and perform CRC32 verification, update the MIG device tree and release the page lock after the migration is completed.

[0154] The model optimization and experience update module is used to continuously collect the MIG fragment state of the GPU after the migration action is completed to obtain the real evolution sequence. It aligns the predicted sequence with the real sequence through dynamic time warping and calculates the relevant KL divergence. When the divergence exceeds the threshold, it performs model distillation and replaces the online instance. At the same time, it writes the optimal action and the real migration pause duration into the high-priority experience pool.

[0155] The MIG fragment rearrangement system based on topology awareness and two-layer intelligent agents provided in this embodiment generates high-value, low-risk migration action sequences through multi-dimensional information fusion and intelligent decision-making. Combined with dynamic migration technology, it shortens the time and improves the utilization rate of continuous free partitions of video memory. At the same time, through prediction and real sequence calibration and model self-optimization mechanism, it achieves a comprehensive improvement in fragment rearrangement efficiency, resource allocation flexibility and system robustness.

[0156] Figure 6 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0157] The MIG fragment rearrangement method based on topology awareness and two-layer intelligent agents provided in this application embodiment can be applied to electronic devices. Those skilled in the art will understand that the electronic device structure involved in the embodiments of this invention does not constitute a limitation on the electronic device. An electronic device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. In the embodiments of this invention, electronic devices include, but are not limited to, laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0158] Electronic devices may include processors, external memory interfaces, internal memory, universal serial bus (USB) interfaces, charging management modules, power management modules, batteries, wireless communication modules, audio modules, speakers, microphones, sensor modules, buttons, cameras, displays, and SIM card interfaces, etc.

[0159] A processor may include one or more processing units, such as: a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0160] The processor can serve as the nerve center and command center of an electronic device. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0161] The processor may also include memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can store instructions or data that the processor has just used or that are used repeatedly. If the processor needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces processor latency, and thus improves system efficiency.

[0162] An external storage interface (ESI) can be used to connect external memory cards, such as microSD cards, to expand the storage capacity of electronic devices. The external memory card communicates with the processor through the ESI to perform data storage functions, such as saving music and video files on the external memory card.

[0163] Internal memory can be used to store computer executable program code, which includes instructions. The processor executes various functional applications and data processing of electronic devices by running the instructions stored in internal memory. Internal memory can include a program storage area and a data storage area. Internal memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0164] Wireless communication functionality in electronic devices can be achieved through antennas, wireless communication modules, modem processors, and baseband processors.

[0165] Wireless communication modules can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies.

[0166] Electronic devices can implement audio functions through audio modules, speakers, receivers, microphones, headphone jacks, and application processors.

[0167] Electronic devices can achieve shooting functions through ISPs, cameras, video codecs, GPUs, displays, and application processors.

[0168] Electronic devices can achieve display functions through GPUs, displays, and application processors.

[0169] A GPU is a microprocessor for image processing, connected to the display screen and application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. A processor may include one or more GPUs, which execute program instructions to generate or modify display information.

[0170] A display screen is used to display images, videos, etc. A display screen includes a display panel.

[0171] The aforementioned electronic device realizes the MIG fragment rearrangement method based on topology perception and two-layer intelligent agents proposed in this application. It constructs embedded tensors and global condition matrices through multi-dimensional information fusion, generates fragment evolution trajectories through UNet diffusion networks, optimizes action sequences through graph policy networks, and adjusts the data migration process in real time through dynamic migration technology. This achieves the beneficial effects of improving memory resource utilization, shortening migration time, reducing spatiotemporal drift risk, and enhancing the system's self-optimization capability.

[0172] The storage medium provided in this application stores a program product capable of implementing the MIG fragment rearrangement method based on topology sensing and two-layer agents.

[0173] The MIG fragment rearrangement method based on topology awareness and two-layer agents includes the following steps: S1: Collect the computing unit usage ratio and video memory usage ratio of the activated MIG partition, and after specific calculations, combine them with the remaining execution time of the corresponding partition and overlay them with the neighborhood aggregation results of the PCIe adjacency relationship; then use the linear mapping matrix to project the partition channels of different GPUs to a unified dimension, and sequentially concatenate them with the queued job template to obtain the three-dimensional embedding tensor. S2: Within each GPU, partitions are arranged in ascending order of physical memory address to construct a binary adjacency matrix. The partition size difference is combined with the Sigmoid gating function to obtain the concatenated gating weights. The gating weights are compressed into GPU condition vectors using bidirectional fusion pooling, and stacked according to GPU number to form a global condition matrix. S3: The 3D embedding tensor is copied by channel, aligned with the global condition matrix and spliced. The input is a UNet diffusion network containing high-resolution temporal convolution branches and low-resolution graph convolution branches. Multiple fragment evolution trajectories and corresponding confidence scores are generated by cross-scale gated coupling backsampling. A regional segmented integral calibration function is used to suppress spatiotemporal drift. S4: Based on the confidence level, the maximum continuous connectable memory ratio that each GPU can form within the future time window is weighted and averaged to obtain a continuous capacity tensor; a graph structure is constructed, and the original value of the action is output through a lightweight graph policy network, and the upper bound of the reward for each candidate action is calculated in combination with relevant factors; S5: Based on multiple fragment evolution trajectories, obtain the action time risk tensor, delete actions that do not meet the requirements, use the confidence level as a priori to perform Bayesian posterior calibration on the action reward, and use a depth-constrained iterative deepening backtracking algorithm to search for and output the action sequence with the best value under the constraint of the upper bound of the reward. S6: Call NVML to lock the page table of the partition to be migrated, estimate the upper bound of the theoretical migration time, trigger the RDMA hardware to perform data migration and adjust the sliding window in real time so that the actual migration time converges to the upper bound; dynamically rearrange the scheduler job priority within the migration window, and perform CRC32 verification, update the MIG device tree and release the page lock after the migration is completed. S7: After the migration action is completed, continuously collect the MIG fragment state of the GPU to obtain the real evolution sequence. Align the predicted sequence with the real sequence through dynamic time warping and calculate the relevant KL divergence. When the divergence exceeds the threshold, perform model distillation and replace the online instance. At the same time, write the optimal action and the real migration pause duration into the high-priority experience pool. In some possible implementations, the MIG fragment rearrangement method based on topology awareness and two-layer agents disclosed herein can be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0174] The storage medium disclosed herein may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0175] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A MIG fragment rearrangement method based on topology sensing and two-layer agents, characterized in that, Includes the following steps: S1: Collect the computing unit usage ratio and video memory usage ratio of the activated MIG partition, calculate the remaining execution time of the corresponding partition, and then superimpose the neighborhood aggregation result with the PCIe adjacency relationship; then use the linear mapping matrix to project the partition channels of different GPUs to a unified dimension, and sequentially concatenate them with the queued job template to obtain the three-dimensional embedding tensor. S2: Within each GPU, partitions are arranged in ascending order of physical memory address to construct a binary adjacency matrix. The partition size difference is combined with the Sigmoid gating function to obtain the concatenated gating weights. The gating weights are compressed into GPU condition vectors using bidirectional fusion pooling, and stacked according to GPU number to form a global condition matrix. S3: The 3D embedding tensor is copied by channel, aligned with the global condition matrix and spliced. The input is a UNet diffusion network containing high-resolution temporal convolution branches and low-resolution graph convolution branches. Multiple fragment evolution trajectories and corresponding confidence scores are generated by cross-scale gated coupling backsampling. A regional segmented integral calibration function is used to suppress spatiotemporal drift. S4: Based on the confidence level, the maximum continuous connectable memory ratio that each GPU can form within the future time window is weighted and averaged to obtain a continuous capacity tensor; a graph structure is constructed, and the original value of the action is output through a lightweight graph policy network, and the upper bound of the reward for each candidate action is calculated in combination with relevant factors; S5: Based on multiple fragment evolution trajectories, obtain the action time risk tensor, delete actions that do not meet the requirements, use the confidence level as a priori to perform Bayesian posterior calibration on the action reward, and use a depth-constrained iterative deepening backtracking algorithm to search for and output the action sequence with the best value under the constraint of the upper bound of the reward. S6: Call NVML to lock the page table of the partition to be migrated, estimate the upper bound of the theoretical migration time, trigger the RDMA hardware to perform data migration and adjust the sliding window in real time so that the actual migration time converges to the upper bound. Within the migration window, scheduler job priorities are dynamically rearranged. After the migration is complete, CRC32 verification is performed, the MIG device tree is updated, and page locks are released. S7: After the migration action is completed, continuously collect the MIG fragment state of the GPU to obtain the real evolution sequence. Align the predicted sequence with the real sequence through dynamic time warping and calculate the relevant KL divergence. When the divergence exceeds the threshold, perform model distillation and replace the online instance. At the same time, write the optimal action and the real migration pause duration into the high-priority experience pool.

2. The MIG fragment rearrangement method based on topology sensing and two-layer agents according to claim 1, characterized in that, Step S1 includes: S11: Obtain GPU index set For any GPU index Collect its activated MIG partition index set and will the The proportion of computing units and video memory usage in each partition is encapsulated into a vector. All vectors are stacked sequentially according to their partition numbers to obtain the base matrix. ; S12: Based on the benchmark matrix Collect the remaining execution time set for the corresponding partition. Construct a diagonal matrix ; Spatiotemporal alignment is achieved by combining Kronecker multiplication with Hadamard product, generating a spatiotemporal embedding tensor. : in, Indicates Kronecker multiplication. Indicates element-wise multiplication. It is a matrix of all 1s; Spacetime Embedded Tensor While maintaining the two-dimensional resource description, the remaining execution time is pushed into the same quantity; S13: Introduce a global PCle adjacency matrix ,in If and only if the first Block GPUs are interconnected via a peer PCIe switch or a direct link; spatiotemporal embedding tensor Perform a neighborhood aggregation: first, for according to After filtering, summing on the resource dimension and superimposing it with the local tensor, we obtain the spatiotemporal embedding tensor. ; S14: Through a linear mapping matrix Embedding spacetime into tensors Projecting onto a fixed-dimensional representation generates a spatiotemporal embedding tensor. ;in, To unify the dimensions of the hidden space; S15: Obtain the vector matrix through isomorphic linear mapping using the queuing operation template. ; among them, each line and Maintain a consistent channel dimension; Embedding spacetime into tensors Stack them according to GPU number, then with Sequential cascading generates a 3D embedding tensor Z: In this process, the stack is spliced ​​along the newly added sequence dimension using tensors. This is a sequential cascade operation.

3. The MIG fragment rearrangement method based on topology sensing and two-layer agents according to claim 2, characterized in that, Step S2 includes: S21: In the On a block GPU, partition index set according to physical memory address Perform ascending sorting, and denote the sorted sequence as . Define the set of adjacent partition binary pairs. And convert it into a binary adjacency matrix. The matrix elements are 1 only at the corresponding positions of the two tuples; S22. To assign a splicing feasibility strength to each candidate edge, a size difference gating function is constructed for the binary adjacency matrix. Spatiotemporal embedding tensor of partitioned resource description To couple; The gating weight matrix of the size difference gating function Specifically as follows: in, For the Sigmoid function; These are the defined video memory usage ratio and computing unit usage ratio, respectively. For empirical hyperparameters; S23: Constructing a feature splicing function ,in Indicates partition The implicit vector in step S1: Based on the above formula, a GPU-level conditional vector with fixed dimensions is obtained. ; All GPU-level condition vectors Stack them sequentially according to GPU number to obtain the global condition matrix. .

4. The MIG fragment rearrangement method based on topology sensing and two-layer agents according to claim 3, characterized in that, Step S3 includes: S31: For 3D embedding tensors Perform channel replication to align it with the feasibility weight matrix. Size compatible, aligned along partition dimensions and spliced ​​to obtain a joint state. ; Constructing high-resolution temporal convolution kernels Convolution kernel with low-resolution image Cross-scale coupled projection is employed: in, This represents corresponding dimension convolution. It is a linear mapping matrix. and These are the time-domain and graph-domain normalization functions, respectively. Based on the above formula, the feasibility weights and dual-scale filtering are simultaneously fused at the input to generate a high-dimensional noise starting point. ; S32: Cross-scale coupled backsampling based on time-series-graph dual-resolution U-Net, assuming... For a fixed decay scheduling sequence, For independent Gaussian noise; for the first The trajectory uses cross-scale decoupling-coupling iteration: In the formula, The time branch is composed of stacked one-dimensional separable convolutions, used to capture millisecond-level jitter in the job. For topological branches involving hierarchical graph convolution and pooling-upsampling, the adjacency matrix is ​​used. Dynamically adjust the receptive field to integrate PCIe bottlenecks into the hidden state; S33: Construct a calibration function based on partition-domain piecewise integration, and apply the following formula to the energy constraint of the time derivative for each trajectory: in, Indicates the first Partitioning in a trajectory In GPU Time-domain occupancy vector; for and Instantaneous communication traffic between corresponding edges of the two GPUs in the topology graph; For PCle, it is the set of connected edges; S34: To generate confidence levels that can be directly used in migration value assessment, for the first... Calculate the invertible uniform entropy of the trajectories under the joint time-graph domain distribution: in, To slice the hidden tensor to the th The principal feature vectors obtained from two GPUs after singular value decomposition; It is a time-step increasing weight; The contiguous capacity expectation derived from S31 is used to emphasize GPUs with severe fragmentation. Output confidence set and The evolutionary trajectories of these fragments collectively constitute the prior information for candidate migration generation.

5. The MIG fragment rearrangement method based on topology sensing and two-layer agents according to claim 4, characterized in that, Step S4 includes: S41: Regarding the first GPU in the trajectory At time step The maximum continuously combinable video memory ratio that can be formed is denoted as . By introducing confidence weights and calculating the expectation within a time window, a continuous capacity tensor is constructed as follows. : Among them, continuous capacity tensor Signals used to explicitly indicate the potential for capacity increases; S42: Construct a graph structure with partitions as nodes and connected edges as directed edges; The node features consist of the latent vector from step S1 and the continuous capacity tensor. The concatenation and edge features include the gated weight matrix from step S2. Compared with the predicted migration pause time ; In the graph structure, the original value of the action is output for each edge. First press Filter out the high-pause edges, then press Before selecting descending order Each action forms a preliminary candidate set. S43: For each action The formula for calculating the upper bound of revenue, which includes the benefit-cost trade-off and is amplified by confidence level, is as follows: in, Derived from the original value of action , The slope control constant is obtained by... This path compresses network scores to... The interval remains differentiable; Used to measure the extent of capacity improvement; Scaling of global confidence; This is the migration cost reduction factor. This is the predicted value for the duration of the action pause; After the calculation is completed, Encapsulated together with the action triple Generate a set of candidate migrations with an upper bound on the returns.

6. The MIG fragment rearrangement method based on topology sensing and two-layer agents according to claim 5, characterized in that, Step S5 includes: S51: Reproject multiple fragment evolution trajectories onto the action dimension to obtain the action-time risk tensor. For each action Based on time step Calculate its confidence-based profit and loss equilibrium value across all trajectories: in, For the first The GPU at time trace action target in the trajectory The maximum continuous idle ratio that can be formed Take the positive part. It is the numerical stability constant; A matrix is ​​generated after the calculation is completed. Used to numerically characterize if at time... implement The uncertainty balance between its benefits and costs provides a dense risk support surface for the necessary edge screening; S52: According to the matrix Define action-level confidence-risk indicators: in, As a risk threshold, Increase modulation intensity For pause smoothing constant, For indicator functions; Targeting actions Only when The edge is considered a necessary edge and added to the simplified set. Otherwise, discard it directly. S53: Backtracking search based on state Indicates depth Action sequence; current extended action At that time, based on the trajectory confidence vector as prior, Bayesian posterior inference is performed on the value function: in, For the first The trajectory is being executed Actual capacity increase forecast at that time For the trajectory in the window The variance of the internal capacity prediction. This is the linear cost reduction factor; S54: Employs depth restrictions The iterative deepening strategy optimizes the backtracking search, including: Set depth threshold ,exist Within the range, perform a depth-first search, increasing the search level exponentially in each round. Until ; After each round of search, the optimal value obtained from sampling will be... As a pre-pruning threshold for the next round, and carrying validated high-value sequences as a hot start path each time it is deepened, the cost of branch re-exploration caused by the dynamic fluctuation of MIG fragments is reduced. S55: When the search depth reaches If the upper bound of the value of all feasible branches is lower than the current global optimum, record the optimal action sequence. and corresponding comprehensive value Extract the first action of the sequence. Output and generate a risk receipt. Feedback is sent to the experience pool.

7. The MIG fragment rearrangement method based on topology sensing and two-layer agents according to claim 6, characterized in that, Step S6 includes: S61: Query via NVML Partitions in the source GPU's page mapping table And perform locking on each physical page in user space; Based on RDMA port rate Link hop count and partition page compression factor An upper bound is given for the theoretical transfer time: in, For the first Skip the switching chip forwarding delay, This represents the effective payload utilization rate of the jump. For the first Page weight compression; This serves as the theoretical upper limit, constraining the time window used for subsequent priority adjustments. S62: RDMA zero-copy data plane migration, after the locked page table is established, sends a work request to trigger the RDMA hardware engine to directly pull data from the source GPU memory; Real-time monitoring of link throughput curves ,like ( (As a load threshold coefficient), the IB sliding window is dynamically adjusted to ensure that the RDMA pipeline depth is synchronized with the link peak, so as to converge the actual migration time to the theoretical upper limit. ; S63: Time windows can be hot-swapped via the scheduler interface. Original priority sequence of all jobs Mapping to new differentiable priority trajectories The optimization objective is to maximize the time integral of the available contiguous partition length for each GPU after the migration is complete. in, Indicates partition In GPU At the moment Occupied status, For indicator functions; S64: After the RDMA transfer is completed, verify that the CRC32 check value of the target GPU memory is consistent with that of the source. After successful verification, call NVML to update the MIG device tree and release the corresponding partition mapping. Differentiable priority trajectory The priority table is embedded into the scheduler.

8. The MIG fragment rearrangement method based on topology sensing and two-layer agents according to claim 7, characterized in that, Step S7 includes: S71: Migration actions via scheduler Upon completion, a high-resolution monitoring task is initiated, continuously collecting global GPU-MIG fragment status at the same time granularity as in step S3, until future coverage is achieved. Step 1: After uniformly encoding all the original records, a true evolutionary sequence is formed. And make its structure match the diffusion model predicted sequence. Maintain correspondence; S72: The scheduler first uses a fixed-window dynamic time warping algorithm to... and Alignment is performed to ensure that the same event index reflects the same physical time. The difference between the prediction and reality distributions is calculated step-by-step across three dimensions: partition occupancy distribution, continuous idle segment distribution, and cross-GPU communication overhead. This difference is then summarized into an overall KL divergence index. ;like If the threshold is exceeded, the diffusion model is determined to require rapid distillation. S73: In the distillation stage, the original model weights are used as the teacher network, and its pruned lightweight copy is used as the student network. The following experience transfer learning scheme is adopted: Will As input, the intermediate multi-scale feature tensor output by the teacher serves as the soft label; Fine-tuning the student network using a joint loss of temperature scaling and cross-information entropy; After distillation is complete, the online inference instances are replaced with student network weights to achieve hot updates; S74: The following tuples Write the risk-backtracking agent's high-priority experience pool; among which... The actual migration pause duration is used; the high-priority experience pool adopts a time decay-confidence hybrid priority sampling mechanism to perform periodic incremental training on the policy network.

9. A MIG fragment rearrangement system based on topology sensing and a two-layer intelligent agent, characterized in that, The system employs the MIG fragment rearrangement method based on topology awareness and two-layer agents as described in any one of claims 1 to 8; The system includes: The data acquisition and embedding tensor generation module is used to acquire the computing unit occupancy ratio and video memory occupancy ratio of the activated MIG partition, and after processing with the remaining execution time of the corresponding partition, it is superimposed with the neighborhood aggregation result of the PCIe adjacency relationship; then, the partition channels of different GPUs are projected to a unified dimension using a linear mapping matrix, and sequentially concatenated with the queued job template to obtain the three-dimensional embedding tensor. The adjacency matrix and condition matrix construction module is used to construct a binary adjacency matrix within each GPU by partitioning in ascending order of physical memory address. The sigmoid gating function is used to obtain concatenated gating weights by combining the partition size differences. The gating weights are compressed into GPU condition vectors using bidirectional fusion pooling, and stacked according to GPU number to form a global condition matrix. The fragment evolution trajectory generation module is used to copy the 3D embedded tensor by channel, align it with the global condition matrix and stitch it together. The input is a UNet diffusion network containing high-resolution temporal convolution branches and low-resolution graph convolution branches. Multiple fragment evolution trajectories and corresponding confidence scores are generated through cross-scale gated coupling backsampling. A regional segmented integral calibration function is used to suppress spatiotemporal drift. The continuous capacity and action reward evaluation module is used to calculate the weighted average of the maximum continuous connectable memory ratio that each GPU can form within a future time window based on confidence level, and obtain the continuous capacity tensor; construct a graph structure, output the original value of the action through a lightweight graph policy network, and calculate the upper bound of the reward for each candidate action in combination with relevant factors; The optimal action sequence search module is used to obtain the action time risk tensor based on multiple fragment evolution trajectories, delete actions that do not meet the requirements, use the confidence level as a priori to perform Bayesian posterior calibration on the action reward, and use a depth-constrained iterative deepening backtracking algorithm to search for and output the action sequence with the best value under the upper bound constraint of the reward. The video memory migration and scheduling adjustment module is used to call NVML to lock the page table of the partition to be migrated, estimate the upper bound of the theoretical migration time, trigger the RDMA hardware to perform data migration and adjust the sliding window in real time so that the actual migration time converges to the upper bound; dynamically rearrange the scheduler job priority within the migration window, and perform CRC32 verification, update the MIG device tree and release the page lock after the migration is completed. The model optimization and experience update module is used to continuously collect the MIG fragment state of the GPU after the migration action is completed to obtain the real evolution sequence. It aligns the predicted sequence with the real sequence through dynamic time warping and calculates the relevant KL divergence. When the divergence exceeds the threshold, it performs model distillation and replaces the online instance. At the same time, it writes the optimal action and the real migration pause duration into the high-priority experience pool.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the MIG fragment rearrangement method based on topology awareness and two-layer agents as described in any one of claims 1 to 9.

Images