176 results about "GPU cluster" patented technology

The invention relates to the technical field of cluster scheduling, and provides a GPU heterogeneous cluster scheduling method and system oriented to large model training and reasoning, which constructs a set of complete cluster scheduling system by integrating multi-source information such as hardware features, running states and historical task data and applying technologies such as a clustering algorithm, a fuzzy comprehensive evaluation method and reinforcement learning. Comprehensive, intelligent and dynamic management and scheduling of GPU cluster resources are realized, the cluster scheduling system can significantly improve the execution efficiency of GPU heterogeneous clusters in large model training and reasoning tasks, the resource utilization rate is improved, the energy consumption is reduced, and the stability and adaptability of the system are enhanced. And an efficient and reliable solution is provided for large-scale deep learning application.

The invention provides a cloud collaborative three-dimensional building modeling rendering method and system based on WebGL (Web Graphics Library). The method comprises the following steps: acquiring design parameters of a target building, analyzing the design parameters into geometric topological data of a plurality of independent components on the basis of a functional attribute division rule, and generating a multi-stage detail geometric model; adding material physical attributes to the surface vertex data, calculating a diffuse reflection coefficient and specular reflection intensity through a ray tracing algorithm, and generating lightweight component data carrying physical rendering parameters; in response to a view angle switching instruction of a terminal user, according to a space block corresponding to a current view angle, a main GPU node is distributed from a distributed GPU cluster, the main GPU node is divided according to the space block, adjacent blocks of the space block are distributed to edge GPU nodes, parallel ray tracing calculation is executed based on a WebGL protocol, and the parallel ray tracing calculation is executed based on the WebGL protocol. And returning to the main GPU node synthesis terminal display image and displaying. According to the technical scheme provided by the invention, the response speed is remarkably improved while the physical-level rendering effect is kept.

The invention relates to the technical field of data analysis, and discloses a GPU computing power resource scheduling method and system, and the method comprises the steps: collecting node hardware parameters and dynamic load indexes of a GPU cluster to construct a multi-dimensional resource feature vector of the GPU cluster, and constructing a resource portrait of the GPU cluster; establishing a node health degree scoring model of the GPU cluster, and generating a health degree score of a cluster node corresponding to the GPU cluster; analyzing a video memory demand of the GPU task request, and calculating an intensive identifier and a communication dependency relationship; determining the SLA weight of the GPU task request, calculating the resource shortage sensitivity of the GPU task request based on the video memory demand, and calculating the target task priority of the GPU task request in combination with the SLA weight; and determining a resource scheduling node group requested by the GPU task in the resource portrait, generating resource scheduling parameters of the resource scheduling node group, and executing scheduling of computing power resources of the GPU cluster based on the resource scheduling parameters. According to the method, the scheduling efficiency of the GPU computing power resources can be improved.

A network environment comprises a plurality of host machines that are communicatively coupled to each other via a network fabric comprising a plurality of switches that in turn include a plurality of ports. Each host machine comprises one or more GPUs that execute customer workloads. Described herein are different approaches that provide for addressing the problem of handling network overlay encapsulation without causing adverse impact to the performance of workloads executed on the GPU clusters.

The invention provides a heterogeneous GPU resource management scheduling method, and relates to the technical field of GPU resource allocation, heterogeneous equipment management and unified abstract modeling are carried out, GPU resources of different architectures are registered to a container arrangement platform, and a unified abstract layer is constructed to shield bottom layer hardware differences; gPU cluster optimization management based on a multi-dimensional real-time monitoring and intelligent scheduling strategy is carried out, GPU operation indexes are collected, priorities are dynamically calibrated for tasks, and task performance portraits are constructed; scheduling decision making is carried out through multi-strategy cooperation, and optimal GPU resources are distributed for tasks; carrying out fine-grained resource allocation, carrying out space or time segmentation on the GPU, and dynamically adjusting resource allocation according to a load state; aPI conversion of cross-architecture tasks is realized through a unified runtime library, and task execution data is collected to feed back an optimization scheduling model; automatic detection, isolation and task migration of GPU faults are carried out, and unified monitoring and alarm are provided.

A GPU cluster computing power optimization architecture oriented to large model training is characterized in that a computing power resource management module is used for monitoring and managing the computing power resource state of each GPU in a GPU cluster in real time, a task scheduling module is used for allocating GPU resources according to the requirements of training tasks, and a computing power optimization module is used for dynamically optimizing the computing power of the GPU cluster in the task execution process. And the result feedback module is used for collecting and analyzing the optimized computing power use condition so as to adjust a subsequent task scheduling strategy. According to the method, the computing power resource state of the GPU cluster is monitored and managed in real time, the GPU resources are dynamically allocated according to the demand of the training task, and the computing power of the GPU cluster is dynamically optimized in the task execution process, so that the computing power utilization efficiency of the GPU cluster is effectively improved, and the training cost is reduced.

The invention relates to a heterogeneous GPU cluster scheduling optimization method and system based on resource affinity, and belongs to the field of computer resource scheduling. Comprising the steps of 1, according to an affinity historical library of historical jobs, matching the historical jobs with relatively high similarity through job characteristics, and estimating resource requirements; 2, for a new job, receiving the job, performing probe testing on each type of GPU, and obtaining and recording job performance data; 3, establishing an affinity evaluation model for each job by using historical information or test data, evaluating affinity values of the jobs under different resource configurations, and providing data support for resource allocation decisions; and 4, entering a scheduling optimization period cycle, constructing an allocation matrix according to a target function of an optimization problem, and allocating corresponding resources to each job. According to the method, accurate resource matching and data support are achieved, the dynamic adaptive capacity is guaranteed, and the system expandability is enhanced.

The invention discloses a GPU cluster scheduling strategy optimization system based on deep reinforcement learning, which comprises a simulation environment layer, a reinforcement learning layer and a strategy evaluation layer, and is characterized in that the simulation environment layer is used for providing a real and credible GPU cluster scheduling environment, reproducing a core mechanism of actual cluster scheduling and simultaneously providing controllable experiment conditions; comparative evaluation and iterative optimization of strategies are facilitated; the reinforcement learning layer converts a GPU cluster scheduling problem into a Markov decision process based on a deep neural network, and obtains an optimal scheduling strategy by applying a reinforcement learning strategy; and the strategy evaluation layer counts each key index, compares and analyzes a plurality of scheduling strategies, and visually displays an analysis result to realize optimization of the GPU cluster scheduling strategy. The system solves the time sequence short view problem of a traditional scheduler, so that the scheduling decision can consider the influence on the future, global optimization instead of local optimization is realized, and the overall resource utilization efficiency is improved.

The invention relates to a computing power resource dynamic scheduling and monitoring method based on a cloud platform. The method is suitable for an intelligent scheduling scene of a high-performance GPU cluster. The method comprises seven steps of task portrait modeling, GPU node state acquisition, resource trend prediction, SLA tracking, scheduling scoring and deployment, operation monitoring and task migration, and SLA feedback optimization. According to the system, task semantics are represented by constructing task vectors, node health states, topology affinity, SLA historical performance conditions and resource prediction risks are fused, a multi-factor adjustable scheduling scoring mechanism is constructed, and second-level perception and task thermal migration of high-temperature nodes are achieved. Compared with a traditional Kubernetes static scheduling scheme, the method has the advantages that the GPU utilization rate, the task SLA achievement rate and the system stability are remarkably improved, the learning ability, the self-adaptive ability and the high availability are achieved, and the method is an intelligent scheduling closed-loop system oriented to AI reasoning and training scenes.

The invention relates to the technical field of distributed computing and computing power scheduling, in particular to a computing power resource fusion method based on a distributed flow pipeline, which comprises the following steps of: disassembling a user computing power request into a flow pipeline unit for packaging computing logic, an input / output interface and a resource demand label; meanwhile, computing power types, real-time load rates, memory occupancy rates, network round-trip delays, geographic positioning and energy consumption data of cloud edge end nodes are collected, a hierarchical topology network framework is constructed based on the collected data, node computing power available values are calculated, an inter-node transmission cost matrix is generated, a fault probability prediction model is constructed, and a fault probability prediction model is constructed. And finally, analyzing a data dependency relationship of the pipeline unit through a four-dimensional joint decision engine, executing dynamic mapping, and preferentially mapping the high-computing-power demand unit to a GPU cluster node, so as to realize non-interruption reconstruction during pipeline topology operation. And the global computing power resource utilization rate, the task operation efficiency and the service stability are improved.

The invention discloses a large model training resource configuration optimization method for a cloud heterogeneous GPU cluster. The method comprises heterogeneous GPU resource configuration and resource perception stage deployment. A cost tree pruning mechanism is provided for resource configuration, a plan with low cost is searched preferentially, a configuration scheme is expressed as a tree path, meanwhile, a multi-dimensional pruning method is adopted to reduce unnecessary branches, and efficient GPU configuration is achieved. According to stage deployment, through defining a Compute-Memory Ratio (Compute-Memory Ratio, CMR), the requirements of each stage on the GPU computing power and memory capacity are considered in a combined manner, the communication and computing efficiency in the training process is optimized, and the large model training time is remarkably shortened. And finally, a large model training resource configuration optimization method Espresso oriented to the cloud heterogeneous GPU cluster is provided in combination with resource configuration and stage deployment strategies, and the user cost is reduced to the maximum extent while the training performance is ensured.

The invention discloses a special cooling device for a computer GPU (Graphic Processing Unit) cluster server, which comprises a box body, a support frame, a circulating pipe, a cooling box, a first piston cylinder, a liquid outlet pipe, an injection pipe, a liquid inlet pipe, a heat dissipation mechanism, a cooling mechanism, a detection mechanism and an auxiliary mechanism, meanwhile, a filter plate can prevent dust in air from entering, a detection mechanism monitors the temperature in the box body in real time by utilizing the thermal expansion characteristic of alcohol, and a signal is transmitted to an external controller through a pressure sensor so that a worker can master the operation temperature state of the server in real time; the cooling mechanism automatically adjusts the flow speed and flow of cooling liquid according to temperature changes so as to meet the heat dissipation requirements of the server under different loads, the auxiliary mechanism conducts rolling sweeping on a filter plate through a cleaning roller, smooth air circulation is ensured, the efficient and intelligent cooling effect is achieved, and the service life of the server is prolonged. And the operation stability and the service life of the GPU cluster server are improved.

The invention discloses a deep neural network model parallel reasoning method for a heterogeneous GPU cluster, and relates to the field of distributed machine learning, and the method comprises the steps: obtaining current information, remaining selectable DNN models, deployed DNN models on each GPU server, and GPU servers which do not meet the number constraint of the DNN models; the scheduler selects a DNN model and deploys the DNN model on the selected GPU server, and calculates throughput for executing parallel reasoning at the moment; the combination of the DNN model and the GPU server with the maximum throughput is found, and related information is updated; judging whether the DNN models deployed on the GPU meet the number constraint or not, and updating GPU cluster information until all GPUs meet the specific DNN model number constraint; and repeating the steps until the algorithm converges. According to the method, limited heterogeneous GPU resources are fully utilized, and the DNN model with high compatibility is selected to deploy and execute parallel reasoning, so that the throughput is maximized.

The invention belongs to the technical field of semiconductors, and discloses a multi-core GPU interconnection architecture and a self-adaptive cache allocation method thereof.The multi-core GPU interconnection architecture comprises a plurality of GPU clusters and a global cluster, the global cluster comprises a global router, a global scheduler, a directory, a memory and a plurality of GPU clusters, each GPU cluster is composed of a plurality of GPU cores, and the GPU cores are arranged in the global router. Each GPU core grain comprises a plurality of independent primary data caches, secondary data caches, a local router and a network interface, and the network interface judges whether a current request is processed by the GPU core grains in a GPU cluster or not according to whether a target address in a target node belongs to the GPU cluster or not, so that a high-level multi-core grain interconnection architecture supporting cross-cluster communication is formed. According to the method, performance is optimized by adapting to different access modes, private and shared cache modes are switched in real time, single data access delay is reduced, the overall system efficiency is improved, and the method is used for a multi-core-particle heterogeneous system with a plurality of GPUs.

The invention provides a server GPU computing power allocation method and system and a server, belongs to the field of server resource management, and solves the problems of low utilization rate and inflexible decision of traditional allocation resources. The method comprises the steps that a central agent constructs a directed weighted game diagram of tasks and a GPU cluster, and a global allocation strategy is solved based on Nash equilibrium; a local agent distributes tasks to a specific GPU through reinforcement learning, intelligent migration is achieved in combination with a dynamic threshold value and anomaly detection, and a strategy is iteratively optimized through federal learning. The system comprises a central agent, a local agent, a GPU cluster and an experience pool module, and a server carries the system execution method. According to the scheme, through double-layer agent cooperation and multi-algorithm fusion, the task cluster adaptation relation is quantified, the allocation strategy is dynamically adjusted, the resource utilization rate and the task processing efficiency are remarkably improved, and the adaptivity and reliability in a complex scene are enhanced.

The invention discloses a GPU Box hardware decoupling system based on a Retimer card and a PCIe Switch chip, and belongs to the technical field of computer hardware architecture and high-speed interconnection. According to the system, a Retimer card and a PCIe Switch chip are integrated in an independent GPU Box, and a decoupling link of a CPU server and a GPU acceleration card is constructed; the Retimer card realizes 30-meter long-distance PCIe signal transmission and breaks through physical distance limitation; the PCIe Switch chip pools GPU resources through a dynamic routing and MRIOV technology, supports flexible allocation of computing power by multiple servers, and realizes Peer-to-Peer direct connection communication between GPUs. Aiming at a large model reasoning scene, the system optimizes KV cache bandwidth allocation and video memory and memory cooperative scheduling, so that the 100B parameter model reasoning throughput is greatly improved; and meanwhile, the usability of the system is greatly improved through fault isolation and hot plug design. According to the method, the problems of physical binding of the CPU and the GPU, limited transmission distance, rigid resource allocation and the like in a traditional architecture are solved, and the method is suitable for large-scale AI calculation and distributed GPU cluster deployment.

The invention relates to the technical field of network security awareness, in particular to a security awareness method and system for a GPU cluster, and the method comprises the steps: collecting multi-source security situation data, and providing basic information for subsequent security analysis; by constructing and updating a cluster security situation map in real time, information timeliness is updated and ensured in real time, and powerful support is provided for risk assessment; a dynamic risk assessment model is utilized to calculate vertex and edge scores, high-risk vertexes and key risk conduction paths are identified, and a basis is provided for resource isolation and scheduling; the targeted instruction is generated according to the security risk score and the preset security policy library, different security risks are effectively handled, and safe and stable operation of the cluster is guaranteed; by converting the instruction into a specific control command and distributing and executing the specific control command, effective implementation of security measures is ensured, and the cluster security protection capability is improved; by setting a finite-state machine monitoring instruction execution state, a recovery mechanism is automatically triggered, and continuous and safe operation of the cluster is guaranteed.

The invention discloses a simulation method, device and equipment, a medium and a program product. Comprises: obtaining a first set; wherein the first set comprises a set of at least part of physical nodes in the GPU cluster; the physical nodes comprise GPU nodes in the GPU cluster and node equipment on which communication between any GPU nodes in the GPU cluster is based; creating a second set corresponding to the first set in a discrete event simulation system; wherein the second set comprises a set of virtual nodes corresponding to the at least part of physical nodes; constructing a virtual communication network corresponding to the second set in the discrete event simulation system; and simulating data processing states of the at least part of the physical nodes based on at least one virtual running state of the virtual nodes in the virtual communication network. According to the scheme, the simulation requirements of the GPU nodes in the GPU cluster can be met.

The invention provides an LLM model resource allocation method and device under a GPU cluster based on mixed integer programming, and relates to the technical field of computers.The method comprises the steps that a computing resource allocation request of a user is received, and resource demand information and a budget upper limit of at least one LLM model are determined based on the computing resource allocation request; obtaining available resource information of the GPU cluster; wherein the GPU cluster comprises a plurality of heterogeneous GPUs; using the available resource information, the resource demand information and the budget upper limit as input of a pre-constructed resource allocation model, solving decision variables of the resource allocation model through a prepared optimization mode, and obtaining a target allocation scheme; a resource allocation model is constructed based on a mixed integer programming algorithm framework; and carrying out computing resource allocation on the at least one LLM model based on the target allocation scheme. According to the scheme, the globally optimal target allocation scheme can be obtained by utilizing the resource allocation model, and the resource utilization rate is improved by utilizing the target allocation scheme to perform computing resource allocation.

The invention discloses a high-performance CPU-GPU (Central Processing Unit-Graphics Processing Unit) cooperative processing architecture of an audio frequency integrated signal processor, belonging to the technical field of audio frequency integrated signal processing, the architecture is based on a dynamic priority scheduling model and realizes efficient collaboration of a CPU and a GPU through hierarchical resource management and protocol level optimization, a hardware layer adopts a multi-GPU cluster and a distributed storage node, and the CPU-GPU cooperative processing architecture has the advantages that the multi-GPU cluster and the distributed storage node are integrated; high-concurrency task processing is supported; the transmission layer fuses RapidIO and an Ethernet protocol, and adapts to a short frame control signal and a long packet data stream respectively; and the application layer calculates task resource demands through dynamic priority weights, and ensures low time delay of key tasks in combination with a normalized allocation algorithm. According to the architecture, in an audio signal processing scene, the task scheduling efficiency is improved by 40%, the short frame transmission delay is as low as 0.5, the throughput of a long data stream reaches 100 Gbps, and the requirements for real-time performance and calculation precision in a complex acoustic environment can be met.

The invention discloses an execution time aware computing power network heterogeneous GPU cluster deep learning task node allocation method and system. The method comprises the following steps: firstly, based on a deep learning task, extracting and preprocessing task features and available node features; secondly, a sampler equally divides new tasks without historical data to available nodes, and each node performs mixed sampling on the tasks until all the tasks estimate execution time data; taking execution time data as a training set, taking the task features and the node features as a test set, and using a regression decision tree model to predict the execution time of the task on each node; performing task allocation on each node by using a cost search algorithm and a short job total JCT priority strategy; and finally, periodically monitoring node resources released in the cluster to obtain an optimal node allocation result. According to the method, task delay and total task JCT are remarkably reduced, cluster node resource changes are monitored in real time, and the resource utilization rate is increased.

The invention relates to a GPU cluster, a redundancy optimization method of the GPU cluster, electronic equipment, a storage medium and a computer program product, the GPU cluster comprises a plurality of GPU cabinets, and each GPU cabinet comprises a plurality of GPU nodes, a GPU extension frame and an interconnection module; for any GPU cabinet, each GPU node in the GPU cabinet comprises a plurality of physical GPUs, and a GPU expansion frame in the GPU cabinet comprises a plurality of standby virtual GPUs; the interconnection module in the GPU cabinet is used for carrying out GPU interconnection on a plurality of GPU nodes and a plurality of GPU expansion frames in the GPU cabinet; and the plurality of standby virtual GPUs in the GPU extension frame in the GPU cabinet are used for providing redundant computing resources for the plurality of physical GPUs in each GPU node in the GPU cabinet. According to the embodiment of the invention, the GPU card-level redundancy capability can be effectively realized.

The invention relates to a model training method and device based on a heterogeneous GPU cluster and a storage medium. The method comprises the steps of obtaining hardware index data of each GPU in the heterogeneous GPU cluster; obtaining a plurality of operation types corresponding to the deep learning algorithm in the to-be-trained model, and measuring operation performance data of operation corresponding to each operation type executed by each GPU; acquiring communication bandwidth multidimensional features and a load state sensing strategy of each GPU; according to the hardware index data, the operation performance data, the communication bandwidth multi-dimensional features and the load state sensing strategy corresponding to each GPU, constructing a multi-dimensional operation performance matrix of each GPU; according to the multi-dimensional operation performance matrix corresponding to each GPU, the GPU is allocated to each structure layer in the to-be-trained model for model training to obtain a target model, the training efficiency of the model is improved, and the training time of the model is shortened.

The embodiment of the invention provides an image processing unit (GPU) cluster communication method and an electronic device, and the method comprises the steps: splitting a GPU cluster, and obtaining at least one algorithm isomorphic group and at least one algorithm heterogeneous group; the protocol distribution communication and the full collection communication are performed through the isomorphic GPUs in each algorithm isomorphic group, and the full protocol communication is performed through the heterogeneous GPUs in each algorithm heterogeneous group. Therefore, through the embodiment of the invention, the problems that the communication traffic between heterogeneous GPUs is increased and extra data copy overhead is introduced due to the difference of communication libraries between different GPUs in a heterogeneous GPU cluster environment can be solved.

The invention provides an asymmetric segmentation scheduling system and method in a heterogeneous GPU cluster, and the method comprises the steps: S1, checking the features of an incoming request, and grouping the incoming request into different request buckets according to the token length; s2, in a heterogeneous GPU cluster environment, optimizing a large language model reasoning instance by adopting a double-layer strategy; and S3, calculating the matching degree of the request buckets and the big language model reasoning instances, and scheduling different request buckets to the big language model reasoning instance with the highest matching degree according to a calculation result. According to the method provided by the invention, the model layer can be asymmetrically segmented according to the computing power and the video memory capacity of each GPU on the premise of satisfying the model parallelism degree constraint, the dynamic balance of the execution duration between stages is realized, assembly line cavitation bubbles are fundamentally reduced, and the overall throughput rate is improved.

The invention provides a GPU program optimization method in a parallel environment, a plurality of target models are configured in a GPU cluster for parallel training, and the method comprises the following steps: performing strategy configuration and memory bottleneck pre-judgment on parallel training configuration of the target models; carrying out throughput optimization through parallel strategy combination and instruction-level performance monitoring by utilizing a pre-judgment result; kernel re-optimization is carried out on an instruction level bottleneck appearing in the optimization process, so that the model training efficiency and the GPU instruction execution efficiency are synchronously improved.

The invention provides an intelligent GPU resource scheduling method and system based on reinforcement learning, and relates to the field of cloud computing resource scheduling, and the method comprises the steps: collecting resource state data of a GPU cluster and attribute data of a to-be-scheduled task, carrying out the preprocessing, building a reinforcement learning model, and training the reinforcement learning model through historical scheduling records; inputting the resource state data of the current GPU cluster and the task attribute data of the to-be-scheduled task into the trained reinforcement learning model to obtain a preliminary scheduling scheme; generating a candidate scheduling scheme set based on the preliminary scheduling scheme, and selecting a final scheduling scheme from the candidate scheduling scheme set by adopting a multi-objective optimization algorithm; performing resource slice configuration on the target GPU node in the GPU cluster according to the final scheduling scheme, and starting the to-be-scheduled task in the task execution container; and updating reinforcement learning model parameters by using empirical data. According to the invention, adaptive evolution and multi-target balance optimization of the scheduling strategy are realized.

The invention relates to a fault network card isolation method and device applied to a GPU cluster, electronic equipment, a storage medium and a computer program product, and the method comprises the steps: determining whether a fault network card exists on any GPU node in the GPU cluster or not according to the state information of each network card in the GPU node; when it is determined that fault network cards exist on the GPU node and the number of the fault network cards is smaller than or equal to a preset threshold value, an NCCL topological file of the GPU node is updated, and the NCCL topological file of the GPU node is used for indicating the topological relation between available GPU devices and available network cards included in the GPU node; and according to the NCCL topology file of the GPU node, isolating a fault network card in the GPU node when carrying out load scheduling on the GPU node. According to the embodiment of the invention, the fault isolation granularity can be effectively refined to a single network card in the GPU cluster.

Disclosed herein is an apparatus and method for 3D parallelization for a heterogeneous GPU cluster. The method may include generating initialization information based on GPU memory capacity in order to parallelize a model across multiple nodes constituting a heterogeneous GPU cluster, pipeline-parallelizing the model based on the multiple nodes using the generated initialization information, and data / tensor-parallelizing layers of the model, which are allocated to each of the multiple nodes according to pipeline parallelization, based on GPUs mounted in the corresponding node.

The invention provides a GPU (Graphics Processing Unit) hardware health prediction and active maintenance method and system, and belongs to the technical field of computing hardware maintenance and fault prediction. Based on the GPU health state data, a health degree comprehensive score of the GPU is calculated by using a preset health degree evaluation model, and a key health index in a future set time period is predicted by using a time sequence model. Calculating the fault probability of a preset type of fault risk by using a preset fault risk model based on the health degree comprehensive score of the GPU and / or the predicted key health index; and comparing the calculated fault risk probability with a preset threshold value, and executing a main preset maintenance action when the fault risk probability exceeds the preset threshold value. Through multi-source data acquisition and time sequence prediction, the potential fault risk of the GPU is found in advance, the maintenance action is actively executed, task interruption and data loss are reduced, and the reliability and availability of a GPU cluster are improved.