22 results about "GPU cluster" patented technology

The application provides a GPU resource intelligent scheduling method and system based on reinforcement learning, relates to the field of cloud computing resource scheduling, and comprises the following steps: collecting resource state data of a GPU cluster and attribute data of a to-be-scheduled task, pre-processing, constructing a reinforcement learning model, and training the reinforcement learning model by using historical scheduling records; inputting the resource state data of the current GPU cluster and the task attribute data of the to-be-scheduled task into the trained reinforcement learning model to obtain a preliminary scheduling scheme; generating a candidate scheduling scheme set based on the preliminary scheduling scheme, selecting a final scheduling scheme from the candidate scheduling scheme set by using a multi-objective optimization algorithm; performing resource slice configuration on a target GPU node in the GPU cluster according to the final scheduling scheme, and starting the to-be-scheduled task in a task execution container; and updating the parameters of the reinforcement learning model by using experience data. The application realizes adaptive evolution and multi-objective balanced optimization of a scheduling strategy.

This invention relates to the field of gene data analysis technology, and discloses a gene data analysis method and system based on GPU acceleration. The method involves: sorting the resources of each GPU card in a GPU cluster to obtain a first resource sorting queue and calculating the target GPU memory usage of sub-regions in each genome; simultaneously constructing a subtask priority queue; matching the GPU cards for each subtask in the subtask priority queue and constructing a subtask binding list for each GPU card; constructing a Pileup tensor for each bound subtask on each GPU card, performing convolutional classification on the Pileup tensor, and outputting the probabilities of different genotypes. This invention overcomes the technical problem of existing static allocation strategies being completely unaware of the real-time load status of GPU cards, ensuring that subtask allocation decisions are always based on the latest resource status of the cluster, systematically eliminating load skew problems from the scheduling layer.

1. Name of the product in this design: Power supply equipment. 2. Purpose of this design: This design is intended to provide power support for high-performance servers, AI chips (such as GPU clusters), and storage devices. 3. The key design feature of this product is its shape. 4. The image or photograph that best illustrates the design's key points: 3D rendering 1.

The application provides a GPU cluster-oriented computing power resource dynamic scheduling method, comprising the following steps: collecting current power quotas and memory occupation scales of each GPU computing card of a GPU cluster, and extracting running features of a non-sensitive task to obtain first resource situation data; inputting the first resource situation data into a random forest regression model previously established on a GPU cluster management node to output an estimated remaining duration of the non-sensitive task; adding the estimated remaining duration to a current system time, and combining historical running records of the non-sensitive task in the GPU cluster to determine a memory release time of the non-sensitive task; and calculating a time interval between the memory release time of the non-sensitive task and an expected arrival time of a to-be-scheduled sensitive task to obtain a first time delay result.

The application provides a GPU resource dynamic allocation method and device, electronic equipment and medium. The method collects real-time running data of multiple cloud games of a target tenant in a cloud game platform and inputs the data into a pre-trained load prediction model to determine a total GPU resource demand prediction amount of the multiple cloud games in a future target time window and a GPU resource demand prediction value of each cloud game. The total resource amount of a GPU resource pool is adjusted based on the total GPU resource demand prediction amount. Based on the GPU resource demand prediction value of each cloud game, a preset frame rate and a preset delay target value, a target GPU resource adjustment amount of the cloud game is determined, and a game type-resource demand portrait is determined to determine a target GPU instance of the cloud game. Thus, the GPU cluster resource utilization is improved while ensuring user experience.

The application discloses a data parallel distributed fine-tuning method and system based on a trusted data space. In the method, a large model is divided into a first model part deployed in a trusted execution environment and a second model part deployed in a GPU cluster, and low-rank adaptation is used for fine-tuning; each data provider transmits encrypted data to the trusted execution environment, and after decryption and integration, the data is divided into multiple batches by a master node in one training round and distributed to each slave node for parallel forward calculation to obtain a feature map; the feature map is transmitted to the GPU cluster after being processed in a secret state to complete forward and backward propagation, and the gradient is returned in a secret state; the low-rank parameters of the first model part are updated after the trusted execution environment node decrypts the gradient; and after each training round ends, cross-node synchronization of the low-rank parameters is performed between nodes. The application realizes efficient distributed training while guaranteeing data privacy, and has high engineering practicability.

The application discloses a data processing method, device, equipment and storage medium, and belongs to the technical field of computers. The method specifically comprises the following steps: obtaining a hardware resource usage and a draft history acceptance rate; wherein the hardware resource usage is determined based on the running condition of a GPU cluster system performing inference of a large language model, and the draft history acceptance rate is determined based on an inference result of the large language model; based on the hardware resource usage and the draft history acceptance rate, an inference acceleration strategy prediction model is used to obtain a speculation parameter adjustment strategy of a speculation decoding algorithm; and based on the speculation parameter adjustment strategy, the inference of the large language model is optimized.

A Zetta-scale computing architecture is disclosed, comprising a massive array of ultra-high- frequency, mixed-precision processing elements integrated within a continuous All-Silicon Domain. To achieve ZettaFLOPS performance in a single rack, the processing elements are radically simplified and pipelined to operate at frequencies exceeding 10 GHz (e.g., 15 GHz). The architecture utilizes a mixed-precision W4A8 dataflow, where 4-bit weights are stored locally and 8-bit activations are broadcast through the all-silicon fabric. By eliminating the bandwidth bottlenecks and power penalties of conventional packaging, and by maximizing the transistor utility for arithmetic rather than control, the system enables an unprecedented density of billions of processing elements, offering a 1,000x improvement in performance and efficiency over state-of-the-art packaged GPU clusters.

This invention discloses a model training and resource scheduling method and system based on LlamaFactory, relating to the field of resource scheduling and distributed model training optimization technology. Specifically, this invention combines the LlamaFactory model training process with a dynamic priority scheduling mechanism to construct an intelligent training and resource allocation system for multi-task, multi-GPU clusters. This achieves adaptive adjustment of task sorting and resource allocation. By combining a comprehensive dynamic priority score function with a cat swarm optimization algorithm, it realizes efficient scheduling solutions under multi-objective constraints, obtaining high-quality scheduling solutions within a limited time, improving the overall energy efficiency and task throughput of the cluster. Furthermore, by combining the LlamaFactory training loop structure with a safe preemption point design, it achieves refined interruption control during the training process, improving the success rate of task migration and recovery.

This application provides a GPU computing resource scheduling method, system, device, and medium. The method acquires task information for at least one task to be scheduled and constructs a corresponding task profile based on the task information. It also acquires resource status information for each computing node in the GPU cluster and constructs a corresponding node status based on the resource status information. Then, based on the task profile and node status, it determines the scheduling priority of the task to be scheduled. Finally, based on the task profile, node status, and scheduling priority, it determines a target GPU resource combination for executing the task to be scheduled. Thus, under conditions of large-scale GPU cluster operation, diverse task types, and dynamically changing resource requirements, it selects a target GPU resource combination that can improve overall resource utilization, reduce inter-task interference, and shorten task completion time while ensuring timely execution of high-priority tasks.

The present application relates to the technical field of micro-service architecture, and particularly to a GPU cluster load balancing system based on micro-service architecture, which comprises a micro-service topology construction module for real-time capturing source service identifiers and target service identifiers in requests according to remote procedure call request streams generated by components in the micro-service architecture. In the present application, the original cross-node remote network communication is converted into high-speed memory access within the node, the network bandwidth pressure and communication delay within the cluster are reduced, the idle state of the physical node is locked and the health degree is evaluated and screened, the tasks are ensured to be placed on the most stable computing resources, the independent computing units and address spaces are divided according to the memory requirements of the tasks by using the GPU multi-instance segmentation interface, the hardware-level isolation of resources is realized at the physical level, and the concurrent processing capacity and overall resource utilization of the GPU cluster are improved.

The application relates to the technical field of GPU cluster design, in particular to a multi-GPU interconnection system, which comprises M PCIe switches and GPU chip sets corresponding to the M PCIe switches respectively, a single GPU chip set comprises N GPU chips, the GPU chips contained in the GPU chip set are connected with the corresponding PCIe switch, the nth GPU chip contained in the M GPU chip sets is connected through a preset connection relationship, the data transmission between the GPU chip sets does not need to pass through the PCIe interface between the CPU or the PCIe switch, so that the data transmission efficiency of the GPU chip interconnection architecture is guaranteed, at most, a single GPU chip needs to be connected with two other GPU chips, the wiring cost of the GPU chip interconnection is reduced, and a low-cost GPU chip can also form a GPU cluster.

This invention discloses a distributed training scheduling method and apparatus for heterogeneous GPU clusters based on multi-agent collaboration. The invention employs a first agent to extract unit GPU memory based on the greatest common divisor, transforming heterogeneous devices into a logically homogeneous virtual resource pool to eliminate GPU memory fragmentation. A second agent uses dynamic programming to elastically over-partition the model computation graph, generating a decoupled micro-stage task pool. A third agent acts as the core coordinator, using a large language model to generate exploration boundary constraints, guiding the reinforcement learning algorithm to map micro-stages to the virtual resource pool and solve for the optimal parallelism configuration. Finally, a fourth agent constructs the model for cost simulation, returning reinforcement learning numerical penalties and large model semantic diagnostics through internal closed-loop processing, and using real physical feedback for online dynamic calibration. This invention effectively improves the training throughput and dynamic robustness of heterogeneous clusters.

The application provides a large model inference intelligent scheduling and memory management method and device. The method provided by the application comprises: uniformly abstracting each computing device in a heterogeneous GPU cluster, obtaining real-time state data of each computing device, and constructing a global resource state matrix based on the real-time state data; splitting a received inference request into a pre-filling task and a decoding task; executing a hybrid scheduling strategy according to the stage characteristics of the pre-filling task and the decoding task and the global resource state matrix, and determining a target device for executing each task; assigning the pre-filling task to a first type of device cluster for execution and assigning the decoding task to a second type of device cluster for execution; performing memory management on the target device according to memory state information of the target device; and optimizing and updating the hybrid scheduling strategy according to a scheduling result of the hybrid scheduling strategy, a task execution condition of the first type of device cluster and the second type of device cluster, and a memory management effect of the target device.

Embodiments of the present application provide a GPU cluster system and a server, the system comprises: a computing node and a switching node; a first board card on which the computing node is located and a second board card on which the switching node is located are arranged in an orthogonal manner in a cabinet, and each GPU of the computing node is interconnected with each switching chip of the switching node through an orthogonal OD connector. Therefore, through the embodiments of the present application, the problem that the number of GPU interconnections is limited in the related art, resulting in low GPU cluster computing scale and performance, can be solved, and the effect of improving the number of GPU interconnections and computing performance is achieved.

The application provides a data grouping method, system and computing device for GPU architecture, and relates to the technical field of parallel computing.The method gradually distributes sub-grids to different GPU groups in an iterative manner, and dynamically evaluates load balancing and communication overhead during the distribution process to realize a final grouping scheme.The method gives up the calculation cost of finding an absolute global optimal solution, limits the inter-group communication overhead to a specific threshold by relaxing the constraint condition, and realizes load balancing under the premise.The method does not depend on a specific CFD grid type, adopts multi-element scoring and threshold constraint, and when the number of GPU cluster nodes increases, the grouping algorithm can still maintain high execution speed and excellent parallel speedup ratio.In addition to the CFD field, it can also be migrated to other scientific and engineering computing fields such as electromagnetic field simulation, finite element analysis and other grid communication dependent fields.

This invention relates to a hybrid RPC communication system and method for heterogeneous GPU communication protocols. The system includes an application layer, a physical hardware layer, a heterogeneous hardware abstraction layer, a dynamic transport router, and a hybrid RPC engine layer. The application layer initiates communication requests between heterogeneous GPUs. The physical hardware layer includes the hardware and network interface cards of the heterogeneous GPUs. The heterogeneous hardware abstraction layer encapsulates the low-level memory registration and memory addressing interfaces of the heterogeneous GPUs, generating a unified virtual memory descriptor that is universal across platforms. The dynamic transport router dynamically determines the optimal data plane transmission path based on the hardware topology between the heterogeneous GPUs. The hybrid RPC engine layer includes a control plane RPC proxy and a data plane pass-through engine. The control plane RPC proxy performs communication negotiation, and the data plane pass-through engine directly performs zero-copy data transfer between the heterogeneous GPUs based on the optimal data plane transmission path. Compared with existing technologies, this invention improves communication performance and cross-platform compatibility in heterogeneous GPU clusters.

The application discloses a deep learning training system based on a GPU cluster and a related method. The system comprises a host and a GPU cluster composed of multiple GPU nodes; the multiple GPU nodes form a topological ring. The host is used for distributing multiple training batches of a same pre-trained deep learning model to multiple corresponding GPU nodes respectively, broadcasting initial values of parameters of the model to each GPU node, saving parameter values of the model, receiving the single training batch and the initial values of the parameters of the model distributed by the host, executing a training process of the model, realizing gradient data sharing on the multiple GPU nodes through multiple loop transmissions, determining updated parameters of the model, and sending the parameter values of the model to the host. The training system disclosed by the application improves the training performance and the training efficiency of the model.

This invention discloses a GPU cluster computing power optimization architecture for large model training, relating to the field of GPU large model training technology. The architecture includes a computing resource management module, a task scheduling module, a computing power optimization module, and a result feedback module. The computing resource management module monitors the GPU cluster's computing load, memory usage, temperature, power consumption, and other statuses in real time, employing a multi-factor comprehensive evaluation strategy to achieve intelligent allocation and load balancing of GPU resources. The task scheduling module performs task scheduling based on a priority algorithm, supporting multi-task parallelism, dynamic priority adjustment, and task preemption. The computing power optimization module predicts computing power requirements through machine learning and dynamically fine-tunes them during task execution. The result feedback module collects computing power usage data and iteratively optimizes the scheduling strategy. This invention also adds a fault warning module and a graphical user interface to achieve GPU anomaly prediction and cluster visualization management. This invention can significantly improve the computing power utilization efficiency of GPU clusters, reduce the cost of large model training, and enhance system operational stability, making it suitable for large-scale deep learning model training scenarios.

The application discloses a GPU resource optimization method and system based on dynamic prediction and topology heat map. The method comprises the following steps: constructing a server heat map, the heat value of which is calculated based on historical task characteristics and node topology connection edge quantity weighting; predicting future integrated GPU resource demand based on a to-be-scheduled task queue and historical task data to generate a resource demand heat map; quantitatively identifying GPU resource fragmentation risks based on the resource demand heat map and a dynamic reference value calculated based on the server heat map; selecting an optimal optimization strategy in response to the fragmentation risks; and executing the selected strategy, including dynamically migrating tasks supporting checkpoints and / or scaling resources of tasks supporting elastic scaling, to realize lossless reconstruction of GPU resources. Through the intelligent closed loop of prediction-decision-execution, the application actively resolves resource fragmentation, and significantly improves the resource utilization and task scheduling efficiency of a large-scale GPU cluster.

This invention discloses an adaptive training scheduling method for multimodal large language models based on selective co-location. It achieves this by constructing a closed-loop framework consisting of an analyzer, a planner, and an executor, encompassing detection / perception, decision-making / planning, and dynamic execution. The analyzer acquires the resolution of the input image and the computation time, memory usage, and communication traffic parameters of each model layer. The planner inputs these parameters into a cost model based on mixed-integer linear programming. While ensuring that memory usage does not exceed hardware capacity and strictly maintaining the original hierarchical order of the model, a selective co-location strategy is used to reduce communication overhead across modal boundaries, solving for the relationship matrix between each model layer and the device. The executor reallocates model layers and switches loads across the GPU cluster based on this relationship matrix. This invention solves the problem of load imbalance caused by structural differences between different modalities during multimodal large language model training, improving training throughput while ensuring memory safety.