Industrial personal computer and multi-graphics card collaborative parallel operation acceleration system

Abstract

The invention discloses an industrial personal computer and multi-graphics card collaborative parallel computation acceleration system, which relates to the technical field of industrial resource allocation and parallel computation, and comprises a resource monitoring and predicting module, a resource management module and a prediction type resource preparation module, the task splitting and collaborative execution module comprises a task splitting module, a cross-node collaborative module and a collaborative operation engine; the intelligent scheduling and dynamic resource allocation module comprises an intelligent scheduler, a dynamic resource allocation module and a conflict avoidance module. According to the method, the GPU video memory utilization rate, the core utilization rate, the temperature, the video memory fragment rate, the available video memory total amount, the CPU core total utilization rate, the load condition and the idle core number index are collected in real time through a resource monitoring module, and the video memory capacity, the GPU core occupancy rate and the CPU load requirement are predicted in advance before a task is submitted in combination with a gradient boosting decision tree and a neural network prediction model; and resources are reserved, so that the scheduling delay is remarkably reduced, and the scheduling hit rate is improved.

Description

Technical Field

[0001] This invention relates to the field of industrial resource allocation and parallel computing technology, and in particular to an industrial control computer and multi-GPU collaborative parallel computing acceleration system. Background Technology

[0002] Industrial resource allocation and parallel computing technology is a core technology supporting the intelligent transformation of modern industry. It mainly solves the problems of efficient processing of large-scale computing tasks and optimized management of hardware resources in industrial scenarios. By integrating distributed computing, real-time scheduling and intelligent optimization algorithms, it promotes the improvement of computing efficiency and maximizes resource utilization in industrial scenarios, and provides underlying technical support for applications such as intelligent manufacturing and digital twins.

[0003] In existing technologies, the video memory and computing core resources of multi-GPUs are usually allocated statically with fixed quotas. This cannot be dynamically adjusted according to the actual resource usage during task execution, resulting in insufficient video memory or computing cores for some tasks while other tasks have idle resources, leading to a decrease in resource utilization. There is a lack of adaptive resource allocation strategies based on task type and running status, making it difficult to balance high utilization with the stability of task execution. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing an industrial control computer and multi-graphics card collaborative parallel computing acceleration system.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: an industrial control computer and multi-GPU collaborative parallel computing acceleration system, comprising the following modules: an industrial control computer cluster hardware platform consisting of multiple industrial control computer nodes; a resource monitoring and prediction module including a resource monitoring module and a predictive resource preparation module; a task splitting and collaborative execution module including a task splitting module, a cross-node collaboration module, and a collaborative computing engine; a priority and quality of service (QoS) management module adjusting scheduling weights according to task priority and QoS level; an intelligent scheduling and dynamic resource allocation module including an intelligent scheduler, a dynamic resource allocation module, and a conflict avoidance module; and an active fragmentation management module for detecting and integrating GPU fragment resources.

[0006] As a further description of the above technical solution:

[0007] Each node of the industrial control computer cluster hardware platform is equipped with at least two independent GPUs and a multi-core central processing unit, as well as high-speed memory. The nodes are connected via Ethernet for cross-node data exchange and collaborative computing.

[0008] As a further description of the above technical solution:

[0009] The resource monitoring module periodically collects cluster resource status via interfaces. It calls the NVIDIA SMI interface to collect GPU memory usage, core utilization, and temperature in real time; it calls the Prometheus interface to collect GPU memory fragmentation rate; and it calls the Kubelet Stats interface to collect CPU load information and GPU running status in a containerized environment. The collection behavior is executed in a polling manner at fixed time intervals. The monitoring data is used for resource suitability calculation and scheduling decisions. The predictive resource preparation module includes a data collection unit, a model training unit, and a predictive application unit. Based on historical task types, data volume, and resource requirements, it uses gradient boosting trees or neural networks to train models and uses them for predicting resource requirements and reserving resources for new tasks.

[0010] As a further description of the above technical solution:

[0011] The task splitting module divides tasks into CPU subtasks and GPU subtasks, with CPU subtasks used for data preprocessing and coordination control, and GPU subtasks used for parallel computing; the cross-node collaboration module uses NCCL or PyTorch Distributed framework to allocate tasks and synchronize parameters based on node latency and resource availability; the collaborative computing engine coordinates data interaction and computation synchronization between the CPU and GPU through PCIe or Ethernet channels.

[0012] As a further description of the above technical solution:

[0013] The priority and QoS management module adjusts scheduling priorities by setting task weight factors. The weight factors include priority weight factors and QoS weight factors. The priority weight factors are dynamically set based on the user-preset priority levels, including first priority, second priority, and third priority. The QoS weight factors are dynamically set based on the user-preset QoS categories, including real-time QoS tasks, batch tasks, and background tasks. The intelligent scheduler uses the product of the priority weight factors and the QoS weight factors as the node scoring parameters.

[0014] As a further description of the above technical solution:

[0015] The intelligent scheduler employs Binpack, Spread, performance-first, and fragmentation optimization strategies, using the product of resource suitability and task weight as the objective function for scheduling. The dynamic resource allocation module dynamically adjusts the allocation of video memory and computing core resources based on the current resource status and task requirements. The adjustment is based on the predicted video memory requirements provided by the predictive resource preparation module, the deviation between the actual load and the predicted values ​​during task execution, and the video memory fragmentation status reported by the resource monitoring module. The conflict avoidance module sets initial values ​​for video memory utilization and core utilization thresholds based on the performance parameters and recommended operating loads published by the GPU manufacturer. It dynamically adjusts the thresholds based on the statistical analysis results of historical task resource usage data, setting differentiated thresholds according to task type. For deep learning tasks, the video memory fragmentation rate threshold is increased, while the core utilization threshold is relaxed for batch tasks. When predicted resource limits are exceeded, task rejection or migration operations are performed.

[0016] As a further description of the above technical solution:

[0017] The active fragmentation management module includes a fragmentation scanning unit, a fragmentation merging unit, and a fragmentation migration unit. The fragmentation scanning unit is used to periodically scan GPU resources, identify discontinuous free memory blocks with memory capacity below a threshold, and computing cores with consistently low utilization. The fragmentation merging unit is used to physically merge multiple discontinuous free memory blocks on the same GPU to form contiguous memory blocks. The fragmentation migration unit is used to migrate a task that occupies a fragmentation area when it is identified as having a third-priority task, in order to release fragmentation resources for the fragmentation merging unit to perform the merging operation.

[0018] As a further description of the above technical solution:

[0019] The output of the resource monitoring module is connected to the input of the intelligent scheduler; the task splitting module is controlled by the intelligent scheduler to split tasks; the output of the priority and QoS management module is connected to the intelligent scheduler, providing task weight factors; the intelligent scheduler generates scheduling instructions based on the output data of the resource monitoring module and the priority and QoS management module, and outputs them to the task splitting module and the dynamic resource allocation module respectively; the input of the dynamic resource allocation module is connected to the output of the predictive resource preparation module, and performs the allocation of video memory and computing core resources; the constraint signal output of the conflict avoidance module is connected to the dynamic resource allocation module; the input of the active fragmentation management module is connected to the resource monitoring module, and the output is connected to the dynamic resource allocation module.

[0020] The present invention has the following beneficial effects:

[0021] 1. In this invention, a resource monitoring module first collects real-time data on GPU memory usage, core utilization, temperature, memory fragmentation rate, total available memory, CPU core utilization, load, and idle core count. Combined with a predictive resource preparation module using a gradient boosting decision tree and neural network prediction model built from historical task data, the system predicts memory capacity, GPU core utilization, and CPU load requirements before task submission and reserves resources accordingly. This avoids the resource scanning process before scheduling, significantly reducing scheduling latency and improving scheduling hit rate. A conflict avoidance module dynamically adjusts GPU memory usage, GPU core utilization, and CPU load safety thresholds based on historical running data and real-time resource status, supporting composite condition judgments. It rejects requests before scheduling that might lead to memory overflow, GPU throttling, or out-of-processing. The memory management module prevents system overload and task interruption. The cross-node collaboration module and collaborative computing engine support distributed fault tolerance and exception recovery mechanisms. When a node fails, tasks can be suspended, intermediate calculation results and parameter snapshots can be migrated, and execution can be resumed to ensure continuous system operation. The active fragmentation management module periodically performs memory fragmentation scanning, merging, and migration operations. It organizes the memory blocks occupied by the third priority tasks. It uses the underlying operations of `cudaMemGetInfo()`, `cudaMemcpyPeerAsync()`, `cudaFree()`, and `cudaMalloc()` to complete the memory continuity reconstruction in the background and includes the organized resources into the resource pool for the scheduler to call, avoiding scheduling failures caused by memory fragmentation and improving memory continuity and utilization.

[0022] 2. In this invention, the task splitting and collaborative execution module supports three granularities: sub-model level splitting, data block level splitting, and network layer level splitting. It combines data parallel mode, model parallel mode, and cross-node parallel mode, and utilizes the NVIDIA NCCL and PyTorch Distributed framework to achieve efficient cross-node communication and parameter synchronization. The collaborative computing engine uniformly coordinates the execution order, data flow, and gradient exchange of multi-core CPUs and GPUs, and cross-node tasks, ensuring the consistency and efficiency of parallel task execution. The priority and QoS management module encodes task priorities and QoS levels into priority weight factors and QoS weight factors, and combines them with resource adaptability, GPU performance level, and memory fragmentation rate factors to form a comprehensive scheduling weight, driving the intelligent scheduler to select the optimal node to execute tasks. In multi-task, high-load scenarios, it ensures that the first priority and real-time QoS tasks obtain resources first, improving the overall service quality compliance rate of the system. The industrial control computer cluster hardware platform has the ability to operate stably in industrial environments with high temperature, humidity, and electromagnetic interference. Combined with a multi-node, multi-GPU, high-bandwidth, low-latency network architecture, it provides continuous high-performance computing power support for industrial AI training and real-time inference. Attached Figure Description

[0023] Figure 1 This is a system architecture diagram of the present invention;

[0024] Figure 2 This is a system flowchart of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] Reference Figure 1-2 This invention provides an embodiment of an industrial control computer and multi-GPU collaborative parallel computing acceleration system, comprising the following modules: an industrial control computer cluster hardware platform consisting of multiple industrial control computer nodes; a resource monitoring and prediction module including a resource monitoring module and a predictive resource preparation module; a task splitting and collaborative execution module including a task splitting module, a cross-node collaboration module, and a collaborative computing engine; a priority and quality of service (QoS) management module adjusting scheduling weights according to task priority and QoS level; an intelligent scheduling and dynamic resource allocation module including an intelligent scheduler, a dynamic resource allocation module, and a conflict avoidance module; and an active fragmentation management module for detecting and integrating GPU fragment resources.

[0027] The industrial PC cluster hardware platform consists of multiple industrial PC nodes, each equipped with at least two independent GPUs and a multi-core CPU. It features high-speed memory and industrial-grade stability, and utilizes a 10Gbps Ethernet low-latency connection to achieve high-speed data exchange and parallel computing between nodes. Multiple industrial-grade industrial PC nodes are networked to form a cluster structure, creating a distributed computing architecture that provides fundamental support for collaborative processing and parallel scheduling of upper-layer tasks. Each industrial PC node is configured with multiple high-performance GPUs, serving as the execution platform for high-intensity computing tasks, supporting three execution modes: data parallelism, model parallelism, and cross-node parallelism. The multi-core CPUs in the industrial PCs handle task scheduling. The platform coordinates, preprocesses, and aggregates data, working in conjunction with GPUs to achieve efficient task allocation and parallel execution. Low-latency communication between nodes is achieved via 10Gbps Ethernet, supporting data exchange and parameter synchronization across nodes. This is a key foundation for achieving collaborative computing and efficient execution after task decomposition. The platform is capable of stable operation in industrial environments with high temperatures, humidity, and electromagnetic interference, ensuring continuous and efficient system operation in complex environments. Through the combination of multiple nodes, multiple GPUs, and high-bandwidth networks, the industrial control computer cluster hardware platform can handle high-concurrency, large-scale computing tasks, providing ample computing power support for AI training and real-time image processing in industrial applications.

[0028] The resource monitoring and prediction module consists of a resource monitoring module and a predictive resource preparation module. It is primarily responsible for real-time collection and analysis of CPU and GPU resource status within the cluster, and predicting future task resource requirements based on historical task data. This provides accurate data support for subsequent scheduling, acting as a perception and prediction function within the system. The resource monitoring module comprehensively and in real-time monitors the resource usage status of GPUs and CPUs in the industrial control computer cluster, providing accurate and timely data support for subsequent predictive resource preparation, intelligent scheduling, and dynamic resource allocation. The resource monitoring module periodically collects data on each GPU's memory usage, core utilization, temperature, memory fragmentation rate, and currently available memory. The system collects five categories of metrics, reflecting the GPU's workload, resource consumption, and potential availability. These metrics form the basis for evaluating GPU scheduling capabilities and operational stability. The resource monitoring module simultaneously collects the total CPU core utilization, current load, and number of idle cores for each node. This CPU data is used to assess the overall system load and the availability of CPU resources during task scheduling. The module obtains GPU memory usage, core utilization, and temperature via the NVIDIA SMI interface, GPU memory fragmentation rate and CPU load via the Prometheus interface, and data from the KubeletStats interface for containerized environments. The system monitors the GPU and CPU operating status to obtain various resource status data. These interfaces ensure that the resource monitoring module can acquire multi-source data efficiently and stably across platforms, improving the system's monitoring coverage and accuracy. The resource monitoring module performs polling collection at set time periods to ensure continuous updates of resource data in the system and maintain the timeliness of monitoring data, supporting dynamic responses to resource changes. All collected data will enter the predictive resource preparation module to participate in the prediction process of new task resource requirements. However, the resource monitoring module itself does not execute prediction logic. Before the user submits a task, the predictive resource preparation module builds a prediction model by analyzing historical task data to predict the resource requirements of the upcoming task. The system predicts and reserves resources for task memory and computational core requirements in advance, significantly reducing scheduling latency and improving system response speed. A data collection unit gathers key features and resource usage data from completed task execution records. This data includes task type, input data volume, actual GPU memory usage, actual GPU core utilization, and actual CPU load. This historical data serves as the foundation for subsequent model training and prediction, supporting prediction accuracy. The model training unit uses gradient boosting decision trees for feature fitting and predictive model training. The gradient boosting decision tree formula is:

[0029] R GPUThis is the GPU memory required for the prediction task, in MB, where M is the total number of decision trees, and γ is... m h is the learning rate coefficient of the m-th tree. m (·) is the decision function of the m-th tree, used for regression prediction; T is the task type, such as image recognition or video analysis; D is the amount of input data, such as image size or number of frames; H mem This refers to the actual GPU memory capacity used in historical missions, H. core This refers to the actual GPU core utilization rate used by historical tasks and the actual CPU load level generated by historical tasks. A nonlinear mapping model is established using a neural network to capture the complex relationship between task type and resource requirements. The neural network prediction model formula is as follows: f is the GPU core utilization required for the prediction task, a real number ranging from 0 to 1. θ (·) is the neural network function, θ represents the set of model parameters, and x is the input vector, which consists of task features, i.e., [T, D, H]. mem H core H cpu ], W i b is the weight matrix of the i-th layer. i σ is the bias vector of the i-th layer, and σ is the activation function, such as ReLU or Sigmoid. These two machine learning methods are used to model the relationship between tasks and resource usage. The model training process is based on collected historical data, independent of the current task's scheduling flow, and is only used to generate model parameters that can be used to predict new tasks. The prediction application unit in the module is triggered immediately after the user submits a new task. Based on the new task's feature information and the trained model, it predicts the GPU memory capacity required for the task, the GPU core utilization required for the task, and the CPU resource load demand of the task during execution. The formula for the multi-output neural network model that predicts CPU load is: This predicts the CPU load level of each task, expressed as a percentage. These predictions directly guide the subsequent dynamic resource allocation module in reserving resources. The predicted resource requirements for each task are output as parameters for the intelligent scheduling and dynamic resource allocation module, which then reserves appropriate GPU memory and computing core resources for the task in advance based on the predictions. This approach avoids the resource scanning process before scheduling and reduces scheduling latency. To improve the prediction accuracy of the predictive resource preparation module, this system introduces parameter tuning and cross-validation mechanisms during the model training phase. The key parameters of the gradient boosting tree model include the tree... The maximum depth, learning rate, and minimum number of leaf node samples were determined using a grid search combined with five-fold cross-validation to select the parameter combination that resulted in the lowest overall prediction error on historical task datasets. The neural network model was optimized by adjusting the number of hidden layer nodes, activation function type, learning rate, and regularization coefficient. Early stopping was used during training to prevent overfitting. The model evaluation metrics included mean absolute error, root mean square error, and prediction accuracy. The prediction accuracy was calculated as the proportion of actual task resource requirements within ±10% of the predicted value. The optimized model achieved a prediction accuracy of over 92% on typical industrial AI inference tasks.

[0030] The task decomposition and collaborative execution module consists of a task decomposition and collaboration module, a cross-node collaboration module, and a collaborative computing engine. It is responsible for rationally decomposing complex computational tasks into parallelizable subtasks and coordinating their concurrent execution among nodes within the industrial control computer cluster. Based on task type and computational scale, the task decomposition and collaboration module divides tasks into CPU subtasks and GPU subtasks. CPU subtasks perform data preprocessing, parameter aggregation, and scheduling coordination operations; these tasks are suitable for multi-core CPUs and handle control and organization computational functions. GPU subtasks perform high-intensity parallel computing tasks, including matrix multiplication and deep learning inference computationally intensive work; these tasks are allocated to GPUs for accelerated processing. The main computational consumption of the task, after task splitting, supports the following three standardized parallel execution modes to adapt to the needs of different computing models and task structures: Data parallel mode applies the same model structure to different data subsets, with each subtask running on an independent GPU, suitable for batch data processing scenarios; Model parallel mode decomposes a large-scale model according to network layers or structural units and distributes it across multiple GPUs for execution, suitable for scenarios with large model structures that cannot be accommodated by a single GPU; Cross-node parallel mode splits the entire task into multiple subtasks and schedules their execution on multiple industrial control computer nodes, suitable for large-scale task scenarios requiring multi-machine resource collaboration. The task splitting and collaboration module adopts a multi-level task granularity control mechanism. Depending on the scale and type of the specific task, it supports three splitting granularities: sub-model level splitting, data block level splitting, and network layer level splitting. Sub-model level splitting divides the complete model into multiple sub-model modules and allocates them to different processing units. Data block level splitting divides the input data into multiple data blocks and distributes them to various computing nodes for parallel processing. Network layer level splitting divides the deep neural network by layer or path and executes them on different GPUs according to their structural order. The task splitting and coordination module not only completes the logical task decomposition but also is responsible for the initial data division configuration and the formulation of computing coordination strategies. Specifically, this includes allocating data preprocessing tasks to CPU subtasks, allocating high-intensity computing tasks to GPU subtasks, and clarifying the dependencies between subtasks. The relationship and synchronization mechanism requires that the cross-node collaboration module rely on the low-latency communication capabilities provided by 10Gbps Ethernet, combined with a multi-node, multi-GPU distributed computing architecture, to achieve efficient parallel execution of large tasks. The cross-node collaboration module allocates the subtasks, already decomposed by the task splitting module, to the most suitable nodes for execution based on the availability of GPU resources and node communication latency on each industrial control computer node. Supported task allocation strategies include optimization based on inter-node communication latency and resource adaptation selection based on the idle video memory capacity and core utilization of the GPUs on each node. Under the premise of meeting resource conditions, it achieves a balanced distribution of subtasks, improving the overall system execution efficiency. Cross-node collaboration is achieved by calling a standardized high-performance communication framework.This includes using NVIDIA Collective Communications Library for efficient inter-GPU communication and the PyTorch Distributed framework for cross-node parameter synchronization and data exchange during data parallelism and model parallelism. This communication mechanism ensures that subtasks across nodes maintain data consistency, progress synchronization, and correctness of computation results when executing large-scale computing tasks. It supports distributed processing of subtasks at different granularities, including sub-model-level subtasks, data block-level subtasks, and network layer-level subtasks. Sub-model-level subtasks divide large models into modules and execute them independently on multiple nodes. Data block-level subtasks divide large models into modules and execute them independently on multiple nodes. The input is divided into multiple parts and distributed to multiple nodes for parallel processing. Network-level subtasks divide the neural network model hierarchically, distributing inference or training across different nodes. Flexible task granularity management enables finer load distribution and inter-node collaborative control. The module not only completes cross-node deployment of subtasks but also ensures accurate parameter transmission and efficient data synchronization between nodes, real-time synchronization of model parameter updates to prevent computational deviations, synchronization of intermediate layer outputs and inputs to ensure consistency between front-end and back-end connections, and control of gradient exchange and backpropagation between multi-node GPUs. The collaborative computing engine coordinates the parallel task execution process between CPUs, GPUs, and multiple industrial control computer nodes in an industrial control computer cluster, ensuring orderly data flow. The collaborative computing engine ensures the synchronization and consistency between tasks, as well as the correctness and efficiency of overall computation. After task decomposition, it coordinates task execution among multi-level heterogeneous resources. Its main coordination targets include data scheduling and preprocessing task synchronization between multi-core CPUs, computational task allocation and execution order control between multiple GPUs, and time synchronization and execution coordination of subtasks between different industrial control computer nodes. The collaborative computing engine continuously intervenes throughout the task execution cycle, coordinating the task execution progress, data transmission status, and timing of synchronization nodes to ensure that all components of the task can proceed in parallel and in an orderly manner. The collaborative computing engine is responsible for the scheduling and transmission control of critical data, specifically including model parameter update data, data transmission between network layers, and intermediate computations. The resulting data, including model parameter update data, is required when multiple GPUs collaborate on model training, necessitating synchronous updates of model parameters during forward and backward propagation. Inter-layer data transmission occurs when the model is distributed across different GPUs or nodes, requiring the output of each layer to be used as the input of the next layer for cross-device transmission. Intermediate computation results are transmitted in real-time to related subtasks when dependencies exist between them. Based on the data dependencies in the task decomposition structure, data channels are configured to ensure data integrity, order, and real-time performance. To ensure computational efficiency and data synchronization rate, the following communication channels are scheduled for data interaction, including high-speed data transmission between CPUs and GPUs within the same node via the PCIe bus.Based on the NVIDIA NCCL framework, parameter and gradient synchronization between GPUs is implemented. Intermediate data and parameter exchange between nodes is achieved via 10Gbps Ethernet. The communication mechanisms are scheduled based on task granularity, data volume, and communication timing to ensure low-latency, high-throughput data interaction efficiency in a multi-node, multi-GPU environment. The collaborative computing engine is responsible for maintaining the execution consistency and computational correctness of the entire parallel computing process. This is mainly achieved by uniformly controlling the start, execution, and termination timing of tasks in three execution modes: data parallelism, model parallelism, and cross-node parallelism. During distributed training or inference, the order of model parameter and gradient updates is uniformly managed to prevent update conflicts or computational drift. Event-driven and synchronization barrier mechanisms are used to achieve process synchronization of each subtask, ensuring the logical correctness and physical consistency of task outputs. The cross-node collaborative module and collaborative computing... The engine supports distributed fault tolerance and anomaly recovery mechanisms. When any node experiences hardware failure, GPU driver malfunction, or network connection interruption, the system triggers a node failure signal from the health detection module. Failure detection employs a dual approach of heartbeat packet detection and task execution latency assessment. The default heartbeat detection cycle is 500ms, and the latency threshold is three times the average task execution time. Once a node failure is determined, the system automatically suspends the subtasks currently running on that node and migrates its intermediate calculation results and necessary parameter snapshots, including model weights and data buffer states, to a standby or idle node for continued execution. The migration process uses PCIe bus, NCCL, or TCP / IP socket transmission to ensure data consistency. During recovery, the system verifies data integrity through checksums. If corruption is detected, execution is rolled back to the most recent task checkpoint to ensure the correctness of the final task result.

[0031] The core function of the Priority and QoS Management module is to adjust the scheduling priority and resource allocation weight of tasks based on user-defined task priorities and service quality levels. This ensures the orderly execution of tasks in a multi-tasking, high-load computing environment, guaranteeing the real-time performance and resource accessibility of critical tasks. The module allows users to specify task priorities and service quality levels upon task submission via annotations or custom resource definitions. Priorities are categorized into first, second, and third priorities, while QoS levels are categorized into real-time, batch, and background types. These two attributes of the submitted task are parsed and encoded to provide a basis for subsequent scheduling weight calculations. A weighted scoring mechanism is used to assign fixed numerical weights to both priority and QoS levels, forming a combined weight factor. This weight factor includes a priority weight factor and a QoS weight factor. The priority weight factor is dynamically set based on user-preset priority levels (first, second, and third priorities), while the QoS weight factor is dynamically set based on user-preset QoS categories (real-time QoS tasks, batch tasks, etc.). The scheduling weight for both primary and background tasks is calculated as follows: Task Overall Scheduling Weight = Priority Weight × QoS Weight. This scheduling weight influences the node score ranking in the scheduling strategy, determining the task scheduling order. The Priority and QoS Management module does not perform task scheduling, but its output scheduling weight is passed to the intelligent scheduler, becoming one of the important factors in calculating the node score. The node score is calculated as: Node Score = Resource Adaptability × Priority Weight × QoS Weight. Resource adaptability is provided by other modules such as the resource monitoring module, while the scheduling logic is controlled by the intelligent scheduler. The Priority and QoS Management module only provides numerical indicators that affect the priority of task scheduling. When multiple tasks are waiting to be scheduled simultaneously, the scheduler will use the scheduling weight provided by this module to rank the tasks, thereby ensuring that the first priority and real-time QoS tasks are scheduled first, while the scheduling of second and third priority or background tasks is delayed. When resources are scarce, third priority tasks may be delayed or migrated to release resources. This strategy ensures that resources prioritize critical tasks in a multi-task concurrent environment, improving the overall service quality compliance rate of the system.

[0032] The intelligent scheduling and dynamic resource allocation module consists of an intelligent scheduler, a dynamic resource allocation module, and a conflict avoidance module. It is a key module in the system responsible for formulating scheduling strategies, dynamically allocating resources, and preventing resource overflow. It primarily implements task scheduling decisions, resource allocation optimization, and scheduling security. The intelligent scheduler undertakes scheduling decision-making in the scheduling path. Before scheduling begins, it summarizes five key scheduling factors: resource suitability, priority weight, QoS level weight, GPU performance level, and memory fragmentation rate. These factors are used to comprehensively evaluate the suitability of each candidate node. Resource suitability is mainly derived from the current GPU memory usage, GPU core utilization, and remaining node resources provided by the resource monitoring module. Priority weight is derived from priority and QoS level. The S management module reflects the importance of the task. QoS level weights, derived from priority and the QoS management module, reflect service quality requirements. GPU performance levels include differences in GPU hardware model and computing power. Memory fragmentation rate reflects the continuity and distribution integrity of available memory, used to assess memory availability. The intelligent scheduler constructs a node scoring model based on five collected scheduling factors, calculating a comprehensive scheduling score for each candidate node. Scheduling is performed using the product of resource adaptability and task weight as the objective function. The score calculation formula is: Node Score = Resource Adaptability × Priority Weight × QoS Weight, where resource adaptability is determined based on memory free capacity and core usage, and priority weight and QoS weight are set by annotations during task submission. The decision is to incorporate GPU performance level and memory fragmentation rate as correction factors into resource suitability calculation to improve score accuracy. Resource suitability = α × memory idle ratio + β × core idle ratio + γ × GPU performance coefficient - δ × memory fragmentation rate, where α, β, γ, and δ are preset weighting coefficients within the scheduler. Memory idle ratio = 1 - memory utilization rate, core idle ratio = 1 - core utilization rate, GPU performance coefficient is assigned based on GPU model and computing power, and memory fragmentation rate comes from the resource monitoring module. The intelligent scheduler, based on the set scheduling strategy and node score results, determines the final scheduling destination of tasks, supporting four scheduling strategies: Binpack strategy, Spread strategy, performance-first strategy, and fragmentation optimization strategy. The Binpack strategy prioritizes the use of high-load node resources to improve resource utilization. If the current task has low resource requirements and it is desirable to concentrate node resources, the Binpack strategy is enabled. The Spread strategy prioritizes distributing tasks to avoid overloading some node resources and enhance system stability. If the current node load varies significantly and it is desirable to improve system stability, the Spread strategy is enabled. The Performance-First strategy prioritizes nodes with high-performance GPUs to provide computing power guarantees for high-performance tasks. If the task is deep learning inference or high-performance computing, nodes with high GPU performance levels are selected. The Fragmentation Optimization strategy prioritizes nodes with low memory fragmentation rates to improve memory continuity and reduce the probability of scheduling failures.If a task requires a large amount of contiguous GPU memory and the current GPU memory fragmentation rate is too high, nodes with lower fragmentation rates will be prioritized. The intelligent scheduler automatically switches scheduling strategies based on real-time resource status and task type during runtime. The triggering conditions for strategy switching include: when a node's GPU utilization is higher than 80% and the overall cluster average utilization difference exceeds 20%, the Binpack strategy automatically switches to the Spread strategy to balance the load; when the task to be scheduled is marked as a high-performance computing task and at least one node's GPU performance level is significantly higher than average, a performance-first strategy is prioritized; when the task requires more than 512MB of GPU memory and the target node's GPU memory fragmentation rate exceeds 15%, the strategy automatically switches to a fragmentation optimization strategy. Controlled by the scheduler's internal state machine, a strategy evaluation and update are performed before each scheduling decision to ensure the real-time performance and rationality of the decision. The scheduler is configured for automatic application. After completing score calculation and strategy judgment, the scheduling decision is executed. All nodes meeting the resource conditions are scored, and the node with the highest score is selected as the scheduling target. The task is then submitted to that node for execution. The scheduling decision result is submitted to the dynamic resource allocation module, which then handles the actual allocation of GPU and computing resources. Based on the real-time usage of GPU and CPU resources in the system and the actual needs of the tasks, the dynamic resource allocation module dynamically allocates appropriate GPU memory capacity and computing core resources to each task, working in conjunction with the intelligent scheduler's scheduling decisions to improve resource utilization. To improve task execution efficiency and avoid resource waste and scheduling conflicts, the dynamic resource allocation module receives predictive data from the predictive resource preparation module before a task is scheduled for execution. This data includes the GPU memory capacity required by the task, the GPU core usage ratio, and the CPU load requirements. The predictive resource preparation module provides the predicted memory requirements, the deviation between the actual load and the predicted values ​​during task execution, and the memory fragmentation status reported by the resource monitoring module. This predictive data serves as the basis for dynamic resource allocation, ensuring that resource allocation is forward-looking and adaptable. Based on the actual resource status of each GPU node in the cluster and the task requirements, dynamic allocation of memory and computing cores is performed. Specific allocation includes setting memory allocation for the task. The resource allocation module sets GPU core utilization quotas to ensure tasks run within resource limits without overloading other tasks. It supports dynamic scaling of resource quotas. During task execution, the system continuously monitors the task's load. If a discrepancy is found between the actual and predicted resource usage, the dynamic resource allocation module can adjust the resource quota without interrupting the task. For example, it can increase memory or compute core allocation when the load increases and reclaim some resources for other tasks when the load decreases. The dynamic resource allocation module extends Kubernetes' resource request and limit mechanisms to achieve cluster-level resource management and control. This module can be integrated with Kubernetes' scheduling logic.Resource allocation results are declaratively passed to the cluster management system to achieve consistency with other modules. The resource allocation behavior of the dynamic resource allocation module is constrained by the conflict avoidance module. When there is a risk of resource overflow during task execution, such as memory usage exceeding 90% or GPU core usage exceeding 85%, the system will reject the resource allocation request or migrate the task to other nodes with sufficient resources, ensuring that the entire cluster does not become overloaded due to resource requests from individual tasks. The conflict avoidance module plays a protective role in the intelligent scheduling and dynamic resource allocation process, specifically responsible for judging whether resource usage may exceed the safe threshold and intervening during the scheduling phase to prevent system overload or memory overflow errors. The conflict avoidance module sets resource allocation parameters... The resource usage threshold is determined by comprehensively considering the hardware characteristics of the GPU and CPU, system operating experience, and actual scheduling requirements. During initial system deployment, based on the GPU manufacturer's publicly available performance parameters and recommended operating loads, initial safety thresholds are set for GPU memory and core utilization. CPU load can be referenced from the average load capacity of a multi-core CPU, initially set to no more than 80%. During long-term system operation, the system continuously collects the results of various tasks under different loads, including task completion time, failure rate, GPU throttling events, and Out-of-Memory (OOM) error rate. The conflict avoidance module utilizes this historical operating data to calculate the system stability corresponding to each resource usage indicator through statistical analysis. The system determines the load range that leads to resource overflow or performance degradation during actual operation. Based on statistical analysis of historical task resource usage data, the threshold is dynamically adjusted, and differentiated thresholds are set according to task type. For example, when GPU memory utilization exceeds 90% or GPU core utilization exceeds 85%, the system considers the current node's resources to be close to saturation, posing a potential risk. The conflict avoidance module performs resource usage prediction and judgment. During the scheduling phase, it receives the task resource requirement prediction results output by the predictive resource preparation module, including the required memory capacity, GPU core utilization, and CPU load. It compares these prediction results with the current node's resource status to determine whether the task will cause resource overload risk under the current resource environment. When it is determined that the execution of a task will cause resource usage to exceed a preset threshold, a protection mechanism is immediately triggered to reject the resource allocation request for that task on the current node, preventing GPU overflow, system lag, or task failure. If conditions permit, the module will also work with the intelligent scheduler to migrate the task to other nodes with idle resources that meet the demand, ensuring that the task can continue to execute without affecting the overall system operation. The conflict avoidance module, through proactive detection of resource usage status and threshold judgment, provides early warnings of potential OutOfMemory errors during task scheduling and GPU throttling behavior caused by overload, preventing performance degradation, training interruption, or inference failure due to resource bottlenecks during task execution. For example, for deep learning inference tasks...The system has higher requirements for memory continuity, so memory fragmentation rate can be set as an additional threshold control indicator. If the fragmentation rate is higher than 15%, even if the total memory utilization does not exceed 90%, it can be considered as insufficient resources. Deep learning tasks will have an increased memory fragmentation rate threshold. For background batch tasks, execution can continue under high load conditions, and the threshold can be appropriately relaxed. Before scheduling, the conflict avoidance module compares the task resource requirement data provided by the predictive resource preparation module with the current node resource status. If the predicted value plus the current usage is close to or exceeds the set threshold, a resource overload risk flag is triggered. At this point, even if the system's current resources have not yet reached the absolute threshold, the module can reject task allocation or issue a migration suggestion in advance. The conflict avoidance module does not use a single threshold. Instead of using numerical values ​​as the basis for decision-making, a composite condition is set for judgment. Typical combined judgment logic includes: if GPU memory usage > 90% and memory fragmentation > 15%, the memory is considered unavailable; if GPU core usage > 85% and the task is a first-priority task, an attempt is made to migrate a third-priority task to release core resources; if the number of idle CPU cores < 2 and the current task is a data preprocessing intensive task, task scheduling avoidance is triggered. The threshold determination method of the conflict avoidance module comprehensively considers hardware specifications, historical system operation data, task characteristics, and predictive information, employing a combination of static presets and dynamic learning to achieve fine-grained control over the safety boundaries of resource usage, thereby effectively ensuring the system's operational stability and scheduling success rate under multi-task concurrency conditions.

[0033] The proactive fragmentation management module identifies, consolidates, and migrates fragmented GPU memory and computing core resources in the industrial control computer cluster. This improves the continuity, utilization, and scheduling success rate of GPU resources, thereby ensuring resource availability and overall system efficiency during task execution. The module comprises a fragmentation scanning unit, a fragmentation merging unit, and a fragmentation migration unit. These units work collaboratively to manage fragmented resources. The fragmentation scanning unit periodically scans the GPU resource status of each node in the industrial control computer cluster, identifying small free blocks in the memory and cores whose load is consistently below a threshold. The identification of memory fragments is based on whether the capacity of a single free memory block is lower than the schedulable capacity. The minimum requirements for the task are assessed, including whether the memory fragmentation rate exceeds a set threshold, whether there is a high degree of discontinuity in the distribution of free memory blocks, whether the GPU core utilization rate is consistently below a set percentage, and whether the marked resources have been in a fragmented state for several consecutive monitoring cycles. The core objective of this stage is to build a complete view of the system's fragmented resources, identify resource portions that cannot be directly scheduled but can be consolidated, and after identifying fragmented resources, the fragmentation merging unit initiates a merging strategy to consolidate multiple small memory blocks on the same GPU card, merging multiple discontinuous free memory segments into a contiguous large memory block that can be scheduled. The merging method involves statistically analyzing the current task resource usage on the GPU card and marking the third-priority memory area occupied by migrated tasks. The process involves releasing resources occupied by third-priority tasks within the fragmented area, reallocating and aligning free memory, and incorporating the merged large blocks of video memory into the resource pool for use by the intelligent scheduler and dynamic resource allocation module. This process does not affect the operation of first-priority tasks; it only operates on third-priority tasks to avoid interrupting or interfering with the main business process. The fragment migration unit performs migration operations for tasks with long running times and low priority. The migration criteria are: the current task is at third priority; the current task's running time exceeds a set time threshold; the GPU where the current task resides has a high fragmentation rate; there is a target node that can be migrated, whose GPU resources can meet the resource requirements after the task is migrated; and the system scheduling state is at low load, allowing migration. The migration process involves first recording the original task state and input / output information, suspending the task from the original GPU, transferring the relevant state data to the target GPU, reallocating resources on the target node, resuming task execution, releasing fragmented resources in the original GPU, and merging them into the resource pool. This not only releases fragmented resources but also enables task migration to nodes with better resources, improving overall running efficiency and task completion rate. During memory merging, the active fragmentation management module calls the CUDA driver API to complete underlying memory operations. The merging process includes: calling `cudaMemGetInfo()` to obtain the current GPU's free memory block information, and using the internal resource mapping table to locate the physical addresses of non-contiguous memory blocks.For the third-priority task, `cudaMemcpyPeerAsync()` is called to migrate its data to another node or a contiguous region of video memory. `cudaFree()` is then called to release the original video memory block, and `cudaMal loc()` is used to allocate a new contiguous video memory block and register it in the system resource pool. This process executes in the background and will not block the execution of high-priority tasks. Furthermore, during the migration, task data is protected from interrupted computation through double buffering.

[0034] The output of the resource monitoring module is connected to the input of the intelligent scheduler; the task splitting module is controlled by the intelligent scheduler to split tasks; the output of the priority and QoS management module is connected to the intelligent scheduler, providing task weight factors; the intelligent scheduler generates scheduling instructions based on the output data of the resource monitoring module and the priority and QoS management module, and outputs them to the task splitting module and the dynamic resource allocation module respectively; the input of the dynamic resource allocation module is connected to the output of the predictive resource preparation module, and performs the allocation of video memory and computing core resources; the constraint signal output of the conflict avoidance module is connected to the dynamic resource allocation module; the input of the active fragmentation management module is connected to the resource monitoring module, and the output is connected to the dynamic resource allocation module.

[0035] S1, User Task Input

[0036] Users submit tasks through the interface, setting task priority and QoS level; the system records basic task information and puts the task into a waiting scheduling queue.

[0037] S2, Priority and QoS Analysis

[0038] The priority and QoS management module parses task attributes and generates priority weight factors and QoS weight factors; it then calculates the preliminary overall task scheduling weights.

[0039] S3, Real-time Resource Acquisition

[0040] Data is collected via NVIDIASMI / Prometheus / KubeletStats interfaces, including: GPU memory usage, core utilization, temperature, memory fragmentation rate, and total available memory; CPU core utilization, load, and number of idle cores; data is updated at a fixed polling interval.

[0041] S4. Resource Demand Forecasting

[0042] The trained GBDT and neural network model are invoked to predict, based on historical task characteristics, the required GPU memory capacity, GPU core utilization, and CPU load requirements for the task. The prediction results are then passed to the intelligent scheduler and dynamic resource allocation module for advance resource reservation.

[0043] S5, Task Splitting

[0044] The tasks are divided into: CPU subtasks: data preprocessing, coordination and control, parameter aggregation; GPU subtasks: parallel computing, inference, matrix operations; and data parallelism / model parallelism / cross-node parallelism modes are selected according to the task scale.

[0045] S6, Cross-node Collaborative Planning

[0046] The cross-node collaboration module selects the execution node based on node latency and resource availability; it uses NCCL / PyTorchDistributed for parameter synchronization and data exchange.

[0047] S7, Cooperative Computing Engine Configuration

[0048] Configure data channels between CPUs and GPUs, and between nodes; ensure task dependencies, execution timing, and data consistency.

[0049] S8, Comprehensive Calculation of Scheduling Factor

[0050] The node score is calculated by considering the following factors: resource suitability (memory idle ratio, core idle ratio, GPU performance coefficient, fragmentation rate); priority weight; QoS weight; and selection of an appropriate scheduling strategy: Binpack / Spread / performance-first / fragmentation optimization strategy.

[0051] S9, Dynamic Resource Allocation

[0052] Allocate video memory and computing cores to tasks based on predicted values ​​and real-time status; support dynamic adjustment of quotas during operation;

[0053] S10, Conflict Avoidance Check

[0054] Determine whether the allocation will trigger a risk of excessive memory / core usage or high fragmentation; if it exceeds the limit, perform task migration or refuse scheduling.

[0055] S11. Task Startup and Parallel Execution

[0056] CPU subtasks and GPU subtasks execute in parallel; the collaborative computing engine ensures synchronization of tasks across different nodes.

[0057] S12, Active Fragment Management

[0058] Regularly scan for memory fragmentation and low core utilization; migrate third-priority tasks to release memory and perform physical merging; use CUDA API to implement low-level merging.

[0059] S13, Fault Tolerance and Recovery

[0060] When a node failure is detected by heartbeat and latency detection, the task is migrated to a backup node; a task checkpoint rollback mechanism is used to ensure data consistency.

[0061] S14. Summary of Results

[0062] The calculation results of each subtask are aggregated through the collaborative computing engine; the execution results are merged, post-processed, and output.

[0063] S15, Resource Release and Recording

[0064] Release the GPU memory and cores occupied by the task; write the running data to the historical task library for future prediction model training.

[0065] Example: Execute a batch image AI inference task from an industrial camera. The user submits the task via an interface, setting the priority to first priority and QoS to real-time. The task splitting module divides the task into CPU subtasks and GPU subtasks. The predictive resource preparation module predicts the GPU memory requirement as 3GB, GPU core utilization as 65%, and CPU load as 2 cores based on historical similar tasks. The intelligent scheduler selects a performance-priority strategy based on resource monitoring data and prediction results, allocating the task to a node with an A100 GPU. The dynamic resource allocation module allocates 3GB of GPU memory and 70% core utilization to the task, while reserving 2 cores for the CPU subtask. The collaborative computing engine controls the synchronization of parameters between the CPU and GPU, and across nodes, to ensure consistency of inference results. During execution, the proactive fragmentation management module detects high fragmentation rates on another node and merges the GPU memory to release resources. After inference is completed, the system returns the results to the user, and the task ends.

[0066] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An industrial control computer and multi-graphics card collaborative parallel computing acceleration system, characterized in that: The system includes the following modules: an industrial control computer cluster hardware platform consisting of multiple industrial control computer nodes; a resource monitoring and prediction module including a resource monitoring module and a predictive resource preparation module; a task splitting and collaborative execution module including a task splitting module, a cross-node collaboration module, and a collaborative computing engine; a priority and quality of service (QoS) management module that adjusts scheduling weights based on task priority and QoS level; and an intelligent scheduling and dynamic resource allocation module including an intelligent scheduler, a dynamic resource allocation module, and a conflict avoidance module. The Active Fragmentation Management module is used to detect and consolidate GPU fragment resources.

2. The industrial control computer and multi-graphics card collaborative parallel computing acceleration system according to claim 1, characterized in that: Each node of the industrial control computer cluster hardware platform is equipped with at least two independent GPUs and a multi-core central processing unit, as well as high-speed memory. The nodes are connected via Ethernet for cross-node data exchange and collaborative computing.

3. The industrial control computer and multi-graphics card collaborative parallel computing acceleration system according to claim 1, characterized in that: The resource monitoring module collects cluster resource status periodically through interfaces, calls the NVIDIA SMI interface to collect GPU memory usage, core utilization and temperature in real time, calls the Prometheus interface to collect GPU memory fragmentation rate, and calls the Kubelet Stats interface to collect CPU load information and GPU running status in a containerized environment. The collection behavior is executed in a polling manner at fixed time intervals, and the monitoring data is used for resource adaptation calculation and scheduling decisions. The predictive resource preparation module includes a data collection unit, a model training unit, and a predictive application unit. Based on historical task types, data volume, and resource requirements, it uses gradient boosting trees or neural networks to train models and then uses them to predict resource requirements and reserve resources for new tasks.

4. The industrial control computer and multi-graphics card collaborative parallel computing acceleration system according to claim 1, characterized in that: The task splitting module divides tasks into CPU subtasks and GPU subtasks, with CPU subtasks used for data preprocessing and coordination control, and GPU subtasks used for parallel computing; the cross-node collaboration module uses NCCL or PyTorch Distributed framework to allocate tasks and synchronize parameters based on node latency and resource availability; the collaborative computing engine coordinates data interaction and computation synchronization between the CPU and GPU through PCIe or Ethernet channels.

5. The industrial control computer and multi-graphics card collaborative parallel computing acceleration system according to claim 1, characterized in that: The priority and QoS management module adjusts scheduling priorities by setting task weight factors. The weight factors include priority weight factors and QoS weight factors. The priority weight factors are dynamically set based on the user-preset priority levels, including first priority, second priority, and third priority. The QoS weight factors are dynamically set based on the user-preset QoS categories, including real-time QoS tasks, batch tasks, and background tasks. The intelligent scheduler uses the product of the priority weight factors and the QoS weight factors as the node scoring parameters.

6. The industrial control computer and multi-graphics card collaborative parallel computing acceleration system according to claim 1, characterized in that: The intelligent scheduler employs Binpack, Spread, performance-first, and fragmentation optimization strategies, using the product of resource suitability and task weight as the objective function for scheduling. The dynamic resource allocation module dynamically adjusts the allocation of video memory and computing core resources based on the current resource status and task requirements. The adjustment is based on the predicted video memory requirements provided by the predictive resource preparation module, the deviation between the actual load and the predicted values ​​during task execution, and the video memory fragmentation status reported by the resource monitoring module. The conflict avoidance module sets initial values ​​for video memory utilization and core utilization thresholds based on the performance parameters and recommended operating loads published by the GPU manufacturer. It dynamically adjusts the thresholds based on the statistical analysis results of historical task resource usage data, setting differentiated thresholds according to task type. For deep learning tasks, the video memory fragmentation rate threshold is increased, while the core utilization threshold is relaxed for batch tasks. When predicted resource limits are exceeded, task rejection or migration operations are performed.

7. The industrial control computer and multi-graphics card collaborative parallel computing acceleration system according to claim 1, characterized in that: The active fragmentation management module includes a fragmentation scanning unit, a fragmentation merging unit, and a fragmentation migration unit. The fragmentation scanning unit is used to periodically scan GPU resources, identify discontinuous free memory blocks with memory capacity below a threshold, and computing cores with consistently low utilization. The fragmentation merging unit is used to physically merge multiple discontinuous free memory blocks on the same GPU to form contiguous memory blocks. The fragmentation migration unit is used to migrate a task that occupies a fragmentation area when it is identified as having a third-priority task, in order to release fragmentation resources for the fragmentation merging unit to perform the merging operation.

8. The industrial control computer and multi-graphics card collaborative parallel computing acceleration system according to claim 1, characterized in that: The output of the resource monitoring module is connected to the input of the intelligent scheduler; the task splitting module is controlled by the intelligent scheduler to split tasks. The output of the priority and QoS management module is connected to the intelligent scheduler to provide task weighting factors; The intelligent scheduler generates scheduling instructions based on the output data of the resource monitoring module and the priority and QoS management module, and outputs them to the task splitting module and the dynamic resource allocation module respectively. The input of the dynamic resource allocation module is connected to the output of the predictive resource preparation module to perform the allocation of video memory and computing core resources. The constraint signal output of the conflict avoidance module is connected to the dynamic resource allocation module. The input of the active fragmentation management module is connected to the resource monitoring module, and the output is connected to the dynamic resource allocation module.

Images