Computing power dynamic segmentation method and system, computer equipment and storage medium

Abstract

The invention belongs to the technical field of computers, and discloses a computing power dynamic segmentation method and system, computer equipment and a storage medium, and the method comprises the steps: collecting system historical load data and a task execution mode, and predicting a future resource use condition by adopting a machine learning algorithm model; monitoring the overall load data of the system; searching available computing resources in the system; if the overall load data of the system exceeds an overall load threshold value, the resource demand of a single task exceeds a resource tension threshold value and / or the future resource use condition exceeds the computing power of the system, triggering a computing power segmentation decision; matching the compatibility between the available computing resources and resource parameters of a to-be-executed task, and obtaining a compatibility result; according to the compatibility result, the available computing resources and resource parameters of the to-be-executed task, a segmentation strategy is formulated; and redistributing the available computing resources according to the segmentation strategy. The technical problems of low resource utilization rate and long waiting time existing in computing power segmentation can be solved.

Description

Technical Field

[0001] The present application relates to the technical field of computers, and in particular to a method, system, computer device, and storage medium for dynamically dividing computing power. Background Art

[0002] Currently, computing power sharding refers to the process of dividing the computing power resources of a physical GPU into multiple virtual GPUs (commonly called vGPUs). The purpose of this is to improve GPU utilization efficiency and allow multiple users or applications to share the computing power resources of the same physical GPU at the same time.

[0003] Computing power can be divided primarily by physical device and by time slice. When divided by physical device, such as in a data center, servers or computing clusters are assigned to specific users or tasks. However, this approach results in low resource utilization, can leave some devices idle, and complicates cluster management. When divided by time slice, similar to the multi-user time-sharing system of early mainframe computers, while it can leverage device computing power, excessive tasks or excessively long individual tasks can lead to increased wait times and delayed responses. Summary of the Invention

[0004] To this end, embodiments of the present application provide a method, system, computer device, and storage medium for dynamic computing power sharding, which can address the technical issues of low resource utilization and long waiting times associated with computing power sharding. The specific technical solutions are as follows: In a first aspect, an embodiment of the present application provides a method for dynamically dividing computing power, the method comprising: Collect historical system load data and task execution patterns, and use machine learning algorithm models to predict future resource usage; Monitor the overall load data of the system; Search for available computing resources in the system; If the overall system load exceeds the overall load threshold, the resource requirements of a single task exceed the resource shortage threshold, and / or the future resource usage exceeds the system computing capacity, the computing power split decision is triggered; Match the compatibility of available computing resources with the resource parameters of the task to be executed, and obtain a compatibility result; Develop a sharding strategy based on compatibility results, available computing resources, and resource parameters of the tasks to be executed; Redistribute available computing resources based on the sharding strategy.

[0005] Preferably, the method further comprises: Set task sorting rules, which include task priority strategy and / or time slice rotation strategy; Maintain a task queue based on the task sorting rules. The task queue stores tasks to be executed and resource parameters for the tasks to be executed. The splitting strategy is formulated based on the compatibility results, available computing resources, and resource parameters of the tasks to be executed: Develop a sharding strategy based on compatibility results, task queues, and available computing resources.

[0006] Preferably, the method further comprises: Set task sorting rules, including priority strategies; Obtain the priority of the task being executed by the priority policy; The computing resources of the currently executing tasks whose priorities are lower than the preset low priority value are released as available computing resources and / or the computing resources of the currently executing tasks whose priorities are lower than the tasks to be executed are released as available computing resources.

[0007] Preferably, the method further comprises: When the first task in the executing task is interrupted; Save the execution status of the first task; Releasing the computing resources of the first task as available computing resources; Until the interruption disappears, re-execute the first task.

[0008] Preferably, before reallocating available computing resources according to the sharding strategy, the method further includes: Use modeling tools or a sandbox environment to predict the operating status of system computing resources after sharding based on the sharding strategy.

[0009] Preferably, the method further comprises: Conduct validation testing, including benchmark testing, stress testing, and / or stability testing; If the verification test result is negative, roll back to the previous system computing resource allocation configuration; If the verification test results do not meet expectations, adjust the segmentation strategy or optimize the task execution method.

[0010] Preferably, the computing resources available in the search system include: Check system computing resources; Detect the execution progress and performance indicators of tasks being executed in the system; Obtain available computing resources based on data on system computing resources occupied by tasks being executed.

[0011] In a second aspect, an embodiment of the present application provides a computing power dynamic slicing system, the system comprising: Resource and task monitoring module, which collects historical system load data and task execution patterns and uses machine learning algorithm models to predict future resource usage; The load assessment and demand analysis module is also used to monitor the overall load data of the system; A resource query module that searches for available computing resources in the system. A judgment module is used to trigger a computing power split decision if the overall system load data exceeds a preset load threshold, the resource demand of a single task exceeds a preset task threshold, and / or the future resource usage exceeds the system computing capacity; A resource compatibility check module is used to match the compatibility of available computing resources with the resource parameters of the task to be executed and obtain the compatibility result; The strategy formulation module is used to formulate the segmentation strategy based on the compatibility results, available computing resources, and resource parameters of the tasks to be executed; The execution module is used to reallocate available computing resources according to the sharding strategy.

[0012] In a third aspect, an embodiment of the present application provides a computer device comprising a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of any one of the above-mentioned methods for dynamically dividing computing power are implemented.

[0013] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of any one of the above-mentioned methods for dynamically dividing computing power.

[0014] In summary, compared with the prior art, the technical solutions provided in the embodiments of the present application have at least the following beneficial effects: 1. Continuously monitor resource usage and load trends for each computing task, predict future resources through machine learning algorithm models, promptly identify computing power adjustment needs, and formulate segmentation strategies by setting thresholds and combining the matching characteristics of tasks and resources to ensure compatibility and accessibility, fully utilize resources, shorten waiting times, and achieve reasonable computing power allocation; 2. Task sorting rules can be either priority strategy or time slice rotation strategy, or a combination of both. This facilitates sorting and / or scheduling tasks based on their urgency, real-time requirements, and other characteristics, ensuring efficient resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a flowchart of a method for dynamically dividing computing power provided in one embodiment of the present application.

[0016] Figure 2 This is one of the flow charts of a method for dynamically dividing computing power provided in another embodiment of the present application.

[0017] Figure 3 This is the second flow chart of a method for dynamically dividing computing power provided in another embodiment of the present application.

[0018] Figure 4 This is the third flow chart of a method for dynamically dividing computing power provided in another embodiment of the present application.

[0019] Figure 5 This is the fourth flow chart of a method for dynamically dividing computing power provided in another embodiment of the present application.

[0020] Figure 6 This is the fifth flow chart of a method for dynamically dividing computing power provided in another embodiment of the present application.

[0021] Figure 7 This is the sixth flow chart of a method for dynamically dividing computing power provided in another embodiment of the present application. DETAILED DESCRIPTION

[0022] This specific embodiment is merely an explanation of the present application and is not a limitation of the present application. After reading this specification, those skilled in the art may make non-creative modifications to the present embodiment as needed, but as long as they are within the scope of the claims of the present application, they are protected by the patent law.

[0023] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0024] In addition, the term "and / or" in this application is simply a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this application, unless otherwise specified, generally indicates that the related objects are in an "or" relationship.

[0025] In this application, the terms "first", "second", etc. are used to distinguish identical or similar items with substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there any limitation on the quantity and execution order.

[0026] In the present application, the term "at least one" means one or more, and the term "plurality" means three or more. For example, a plurality of first positions means three or more first positions.

[0027] The embodiments of the present application are described in further detail below with reference to the accompanying drawings.

[0028] Reference Figure 1 In one embodiment of the present application, a method for dynamically dividing computing power is provided. The main steps of the method are described as follows: S1: Collects historical system load data and task execution patterns, and uses machine learning algorithm models to predict future resource usage; S2: monitors the overall load data of the system; S3: Search for available computing resources in the system; S4: If the overall system load exceeds the preset load threshold, the resource requirements of a single task exceed the preset task threshold, and / or the future resource usage exceeds the system computing capacity, a computing power splitting decision is triggered; S5: Match the compatibility of available computing resources with the resource parameters of the task to be executed, and obtain a compatibility result; S6: Develop a partitioning strategy based on the compatibility results, available computing resources, and resource parameters of the tasks to be executed; S8: Reallocate available computing resources according to the partitioning strategy.

[0029] To fully illustrate this embodiment, an example of software and hardware coordination that can implement the method of this embodiment is provided below. Existing software and hardware coordination examples that achieve the same functionality can also be used in other embodiments of this application and are not detailed here. Through deep integration with Kubernetes (K8s), efficient scheduling of GPU resources is achieved. Leveraging device plugins from NVIDIA and other vendors, K8s can accurately identify and allocate GPU resources. Users simply specify the GPU device and video memory requirements in the pod definition to achieve efficient resource utilization. Kubernetes, with its exceptional scalability, provides solid GPU support for containerized applications. Using MIG technology to implement multi-instance GPUs, the GPU can be finely divided into multiple independent computing units. Applications can enjoy both fine-grained resource isolation and resource sharing, ensuring efficient and secure operation in a virtualized environment. And with the help of Kubernetes' Horizontal Pod Autoscaler or self-developed autoscaler plug-in, resources can be intelligently adjusted according to the GPU load. This automated mechanism can automatically increase or decrease GPU resources.

[0030] In this embodiment, first, the computing system including the GPU card is turned on and the relevant driver and management software are loaded. In this embodiment, the driver includes: Resource Management Module 1) Hardware resource awareness Responsible for detecting and identifying various computing resources in the system, including CPU cores, GPU cores, memory modules, storage devices, and so on. For example, in a system with multiple CPU cores, the driver needs to accurately know each core's model, frequency, cache size, and other parameters. When identifying a GPU, it determines the GPU model, memory capacity, memory bandwidth, and the number of CUDA cores (for NVIDIA GPUs) or stream processors (for AMD GPUs).

[0031] This information is obtained by communicating with the underlying interfaces of the computer hardware. Taking the Linux system as an example, the driver may use relevant entries in the / proc and / sys file systems, or call the BIOS / UEFI interface provided by the hardware manufacturer to collect detailed data on hardware resources.

[0032] This operation allows you to search for information about available computing resources in the system, facilitating subsequent task matching and compatibility matching.

[0033] 2) Resource Allocation Strategy Based on application requirements and the overall system status, the resource management module determines a computing power allocation strategy. For example, when multiple applications simultaneously request computing power, the driver can adopt a priority strategy. Applications with high real-time requirements, such as real-time video streaming software, are assigned a higher priority to ensure they receive sufficient computing power first.

[0034] A time-slice rotation strategy can also be used to allocate computing resources between different applications at regular intervals. For example, every certain period (e.g., 10 milliseconds), the use of a CPU core is switched from one application to another to ensure that each application receives a certain amount of computing time.

[0035] 3) Resource monitoring and adjustment Continuously monitoring computing resource usage is crucial. The driver tracks each application's utilization of CPU, GPU, and other resources in real time, for example by reading performance counters provided by the operating system or hardware-specific monitoring registers.

[0036] When an application is found to be over-utilizing resources or under-utilizing resources, the driver can automatically make adjustments. For example, if a data processing application is consuming a large amount of GPU resources for an extended period, causing other graphics rendering applications to stall, the driver can appropriately reduce the GPU resource allocation for the data processing application to ensure the normal operation of the graphics rendering application.

[0037] Task scheduling module 1) Task queue management The driver maintains a task queue to store pending computational tasks. These tasks can come from different applications, and each task contains information such as its type (e.g., CPU-intensive, GPU-intensive), priority, and required resources. When a new task enters the queue, the driver inserts it into the appropriate slot based on its priority and resource requirements. For example, if a high-priority deep learning training task is needed, the driver will place it at the front of the task queue to allocate computing resources as quickly as possible.

[0038] 2) Task allocation and execution Based on resource availability and task requirements, the task scheduling module assigns tasks to appropriate computing units for execution. For heterogeneous CPU-GPU computing systems, it determines whether a task is best executed on the CPU or the GPU. For example, for highly parallel computational tasks like matrix multiplication, the driver program assigns it to the GPU because the GPU has a large number of parallel computing units, enabling faster computation.

[0039] During task execution, the driver also handles task interruption and resumption. If an interruption occurs (such as a hardware failure or the insertion of a high-priority task), the driver must pause the currently executing task, save the task's execution status, and then reallocate computing resources to a new task or handle the interruption. Once the interruption is handled, the driver can resume the suspended task.

[0040] Interface module 1) Application Programming Interface (API) APIs include function calls, system calls, or object-oriented programming interfaces. For example, an application can request a certain number of CPU cores or GPU memory by calling API functions provided by the driver.

[0041] These APIs can also be used by applications to pass task-related information to the driver, such as task priority, estimated execution time, etc. At the same time, the driver can use the API to provide feedback to the application on the results of resource allocation, such as whether the required resources were successfully allocated, and the specific information of the allocated resources (such as the allocated CPU core number, GPU device ID, etc.).

[0042] 2) Operating system interface These interfaces are used to receive scheduling instructions from the operating system, report resource usage to the operating system, etc. For example, when the operating system needs to suspend an application to free up resources, it sends instructions to the driver through the driver's operating system interface.

[0043] The driver also obtains global system information through the operating system interface, such as system load, status of other devices, etc. This helps the driver better allocate resources and schedule tasks to adapt to the overall operating environment of the system.

[0044] In this embodiment, NVIDIA vGPU software is used to partition a virtual GPU. NVIDIA allows the computing power of a physical GPU to be divided into multiple virtual GPUs, each of which can be allocated to different virtual machines or containers. For example, in a data center environment, multiple users can use vGPU technology to share a single physical GPU and simultaneously run graphics-intensive applications or deep learning model training. It supports multiple GPU partitioning modes, including fixed allocation (each virtual GPU is allocated a fixed number of CUDA cores, video memory, and other resources) and dynamic allocation (dynamically adjusting virtual GPU resources based on application needs). Using HAMi open-source partitioning software, multiple computing power partitioning modes, such as MIG and MPS, are supported, allowing for granular partitioning down to 1% of computing power and memory partitioning in megabytes. It is fully compatible with both domestic and non-domestic computing power, enabling mixed deployment, unified scheduling, and management of computing power clusters from various vendors, including NVIDIA, Huawei Ascend, Hygon, Cambricon, Tianshu Zhixin, Muxi, and Moore Thread. Through virtualization technology, "one card for multiple uses" is achieved, improving hardware utilization.

[0045] Through the nvidia-smi and rocmon commands in the Linux system, the NVIDIA GPU can be monitored in real time. By using the interface to feedback parameter information, the software can collect various parameters of the GPU card in real time, including but not limited to video memory usage, core frequency, temperature, utilization of each computing unit, and other information.

[0046] In this embodiment, the overall load data is a weighted average of the utilization of the video memory and the computing core. In other embodiments of the present application, other parameters may be used, such as any one, two or more combinations of the following: GPU Utilization: This refers to how busy the graphics processing unit is during a specific period of time, usually expressed as a percentage. 100% means the GPU is running at full capacity.

[0047] Memory Usage: This refers to the proportion of video memory used, usually expressed as a percentage, or directly displayed as the ratio of used video memory to total video memory.

[0048] Temperature: The graphics card's operating temperature is also an important indicator of its load. Excessively high temperatures may cause performance degradation or automatic frequency reduction to protect the hardware.

[0049] Fan Speed: The speed of the fan reflects the graphics card's efforts to maintain proper temperature, usually expressed in RPM or as a percentage of maximum speed.

[0050] Power Consumption: The amount of power your graphics card is currently consuming is another key metric that helps you understand how hard it's working. Power consumption is usually measured in watts (W).

[0051] Clock Speed: This includes the core frequency and memory frequency, and reflects the current operating speed of the graphics card. Dynamic frequency adjustment is a common way for modern graphics cards to manage performance and power consumption.

[0052] API Call Rate: For some applications, especially games, indicators such as Draw Calls per Second or frame generation time can also reflect the load of the graphics card.

[0053] Render Latency: The time from submitting a rendering command to the actual completion of the rendering can also be used as an indicator of load.

[0054] Other custom metrics: Depending on different application scenarios, there may be other load-related metrics, such as throughput and inference time in machine learning tasks.

[0055] In this embodiment, historical data collection includes, but is not limited to, collecting performance monitoring data from the past period, including key metrics such as CPU, GPU, memory, and network bandwidth. Workload logs are collected to record workload conditions over different time periods, such as task submission frequency, task type, and execution time.

[0056] Task execution patterns: Use clustering algorithms to group similar workloads to better understand the resource requirements of different types of tasks.

[0057] Then, existing machine learning algorithms and models are used for predictive analysis. This combines machine learning algorithms with historical load data and task execution patterns to predict future load changes. If a resource bottleneck is predicted—that is, future resource usage exceeds the system's computing capacity—then a decision can be made to split computing power, even if the overall load threshold is currently not exceeded.

[0058] 3) Evaluate existing resources Resource Utilization Analysis: Evaluate the utilization of existing resources and identify idle or overused resources, which represent available computing resources. Identify bottlenecks in the system, such as CPU / GPU bottlenecks, memory bottlenecks, and network bottlenecks, and analyze their impact on overall performance.

[0059] Threshold comparison: Multiple thresholds can be set, including overall GPU load thresholds (e.g., memory usage exceeding 80% and compute core usage exceeding 70%) and resource stress thresholds for individual tasks (e.g., a task's memory requirements increase by a certain percentage within a short period of time and approach the currently allocated upper limit). Therefore, resource caps are set for each task or user group to prevent a single task from consuming excessive resources. When monitoring data exceeds these thresholds, a computing power splitting decision is triggered.

[0060] In this embodiment, when any one of the following occurs: the overall system load data exceeds the overall load threshold, the resource demand of a single task exceeds the resource shortage threshold, and the future resource usage exceeds the system computing capacity, the computing power splitting decision is triggered. In other embodiments of this application, it can also be triggered by any two or three of the above.

[0061] Next, the system searches for available GPU resources for allocation. This refers to available computing resources, including other idle GPU cards, underutilized portions of the current GPU card (e.g., idle compute units), or computing resources that can be temporarily reclaimed from low-priority tasks. A resource compatibility check is then performed to ensure that the resources to be allocated are compatible with the currently running task and system environment. For example, this check includes checking for compatibility with the GPU architecture, driver version, and memory type. This results in a compatibility match between the task and available computing resources, known as a compatibility result. In this embodiment, a clustering algorithm is used to group similar workloads to better understand the resource requirements of different types of tasks.

[0062] Finally, based on the characteristics of the task: formulate a splitting strategy based on the compatibility results, available computing resources, and resource parameters of the task to be executed. The resource parameters of the task to be executed are the GPU architecture, driver version, video memory type, etc. required for the execution of the task. For high-priority critical tasks (such as online deep learning inference services), their computing power requirements are prioritized; for tasks that do not require high real-time performance (such as background model pre-training tasks), their computing power allocation can be adjusted appropriately. When splitting computing power, try to maintain load balance between GPU cards to avoid situations where some GPU cards are overused while other GPU cards are idle. Tasks and computing power can be reasonably allocated based on the performance differences of GPU cards (such as different GPU models have different computing powers). In other embodiments of the present application, splitting strategies can also be formulated based on other parameters that need to be considered.

[0063] Dynamically adjust the GPU card's computing power through the interface provided by the GPU management software or driver. This may include reallocating video memory size, adjusting the number of allocated computing cores, changing the execution queue of tasks, and other operations.

[0064] This application continuously monitors the resource usage and load trends of each computing task, predicts future resources through machine learning algorithm models, promptly identifies computing power adjustment needs, and formulates segmentation strategies by setting thresholds and combining the matching characteristics of tasks and resources to ensure compatibility and accessibility, fully utilize resources, shorten waiting time, and realize computing power allocation.

[0065] Reference Figure 2 In another embodiment of the present application, the method further includes: S61: Setting task sorting rules, where the task sorting rules include a task priority strategy and / or a time slice rotation strategy; S62: Maintaining a task queue according to the task sorting rule, where the task queue stores tasks to be executed and resource parameters of the tasks to be executed; Step S6: Formulate a partitioning strategy based on the compatibility results, available computing resources, and resource parameters of the task to be executed: S63: Formulate a partitioning strategy based on the compatibility results, task queues, and available computing resources.

[0066] In this embodiment, the task sorting rule can be either a priority strategy or a time slice rotation strategy, or a combination of the two. If the task sorting rule is set to a priority strategy, it is convenient to sort tasks according to their urgency, real-time requirements, and other characteristics, and to make rational use of resources.

[0067] Reference Figure 3 In another embodiment of the present application, the method further includes: S301: Setting task sorting rules, which include priority strategies; S302: Obtaining the priority of the task being executed according to the priority policy; S303: releasing computing resources of currently executing tasks whose priorities are lower than a preset low priority value as available computing resources and / or releasing computing resources of currently executing tasks whose priorities are lower than tasks to be executed as available computing resources.

[0068] In this embodiment, priorities are set for tasks. When the overall load data of the system exceeds the overall load threshold, the resources of low-priority tasks are occupied, so that high-priority tasks can be executed first, reducing the delay or operational jerks that occur when the user uses the system. When the resources of low-priority tasks are occupied, the low-priority tasks are re-sorted. In other examples of this embodiment, the low-priority tasks whose computing resources are occupied can also wait for the tasks that occupy their computing resources to complete the calculation before continuing to use the original computing resources. In this embodiment, the preset low priority value is a priority level that is preset before the resources occupied by low-priority tasks are released. For example, when fine computing power division is required for tasks, the preset low priority value is level 3, and the resources occupied by all tasks with a priority level of level 3 are released. In other examples of this embodiment, the priority level of the task that currently exceeds the resource shortage threshold can also be set as the preset priority, thereby realizing dynamic scheduling of resources, releasing more tasks with low real-time requirements, and providing computing resources for tasks with high priority.

[0069] Reference Figure 4 In another embodiment of the present application, the method further includes: S311: When the first task among the tasks being executed is interrupted; S312: Save the execution status of the first task; S313: releasing the computing resources of the first task as available computing resources; S314: until the interruption disappears, re-execute the first task.

[0070] In this embodiment, the first task is any type of task being executed, and is not limited here.

[0071] When the first task is interrupted, the resources occupied by the first task are released, so that the resources occupied by the first task can be fully utilized and the flexibility of resource application is improved.

[0072] Reference Figure 5 In another embodiment of the present application, before step S8, the method further includes: S7: Use modeling tools or a sandbox environment to predict the operating status of system computing resources after segmentation based on the segmentation strategy.

[0073] In this embodiment, a modeling tool or a sandbox environment is used to predict the impact after the adjustment, evaluate possible risks and benefits, and reduce the adverse effects caused by the execution of the segmentation strategy.

[0074] Reference Figure 6 In another embodiment of the present application, the method further includes: S9: Perform verification testing, which includes benchmark testing, stress testing, and / or stability testing; S10: If the verification test result is negative, roll back to the previous system computing resource allocation configuration; S11: If the verification test results do not meet the expected values, adjust the segmentation strategy or optimize the task execution method.

[0075] In this embodiment, the verification test includes any one or more combinations of a benchmark test, a stress test, and a stability test.

[0076] Benchmarking: Run a series of benchmark tests covering both everyday operations and corner cases to evaluate the effectiveness of the new configuration.

[0077] Stress testing: Apply high load to the system to check its performance under extreme conditions and ensure the robustness of the system.

[0078] Stability test: Run the system for a long time to observe whether there is performance degradation or other abnormal conditions.

[0079] Compare the performance data under the new configuration with the previous baseline data to confirm whether the expected optimization goals are achieved. Further adjust the configuration based on the test results, and continue to optimize until all goals are met. If the new configuration has a negative effect, immediately roll back to the previous configuration and re-evaluate the adjustment plan. If the performance of a task is not meeting expectations, adjust its computing power distribution or optimize the execution of the task on the GPU card.

[0080] The specific fine-tuning steps can be divided into: 1) Preparation and Planning Determine the optimization goal: Identify the specific goals you want to achieve through fine-tuning, such as improving the performance of a specific task, reducing overall costs, and improving resource utilization.

[0081] Choose tools: Select appropriate monitoring and management tools, such as NVIDIA System Management Interface (nvidia-smi), Prometheus, and Grafana, to collect and analyze data.

[0082] 2) Current situation assessment Collect baseline data: Use monitoring tools to record current resource usage (CPU, GPU, memory, network, etc.).

[0083] Record key performance indicators (KPIs) such as response time, throughput, error rate, etc.

[0084] Identify bottlenecks: Analyze existing data to identify resource misallocations or performance bottlenecks.

[0085] Determine which tasks or services experience resource contention during peak hours.

[0086] 3) Develop an adjustment plan Define the tuning parameters: Task Prioritization: Set priorities for different tasks based on their business importance.

[0087] Resource quota: Set resource limits for each task or user group to prevent a single task from consuming too many resources.

[0088] Dynamic scheduling: Enable or optimize existing dynamic scheduling algorithms to automatically adjust resource allocation based on real-time load.

[0089] Isolation mechanism: Consider using containerization (Docker, Kubernetes) or virtual machine technology to isolate tasks and ensure more flexible and controllable resource allocation.

[0090] Simulation impact: Use modeling tools or sandbox environments to predict the impact of adjustments and assess possible risks and benefits.

[0091] 4) Implement adjustments Small-scale pilot: Test new configurations in a non-production environment to ensure their stability and effectiveness.

[0092] Collect data during the pilot period and compare performance changes before and after adjustments.

[0093] Gradually promote: Based on the pilot results, gradually roll out the new configuration in the production environment, starting with low-risk areas.

[0094] After each rollout, closely monitor system performance to ensure no new issues are introduced.

[0095] 5) Verify the effect Benchmarks: Run a series of benchmark tests covering both everyday operations and corner cases to evaluate the effectiveness of the new configuration.

[0096] Stress testing: Apply high load to the system and check its performance under extreme conditions to ensure the robustness of the system.

[0097] Stability test: Run the system for a long time and observe whether performance degradation or other abnormal conditions occur.

[0098] 6) Analysis and Iteration Comparative analysis: Compare the performance data under the new configuration with the previous baseline data to confirm whether the expected optimization goals are achieved.

[0099] Continuous Improvement: Further adjust the configuration based on the test results and continue to optimize until all goals are met.

[0100] If you find that the new configuration has brought negative effects, immediately roll back to the previous configuration and re-evaluate the adjustment plan.

[0101] If you find that the performance of a task does not meet your expectations, you can appropriately adjust its computing power allocation or optimize the execution method of the task on the GPU card.

[0102] Through the configuration of this embodiment, the practicality, stability and other characteristics of the updated system computing power splitting strategy are evaluated, the occurrence of system failures is reduced, and the reliability of the system is improved.

[0103] Reference Figure 7 In another embodiment of the present application, step S3 includes: S31: Detection system computing resources; S32: Detect the execution progress and performance indicators of the tasks being executed in the system; S33: Obtain available computing resources based on the data of system computing resources occupied by the task being executed.

[0104] This implementation accurately detects the system's available computing resources. System computing resources include CPU cores, GPU cores, memory modules, storage devices, and more. For systems with multiple CPU cores, the driver needs to accurately know each core's model, frequency, cache size, and other parameters.

[0105] In another embodiment of the present application, it further includes: Notify administrators and record operation logs; Continuously monitor the operation of system computing resources.

[0106] In this embodiment, administrator notification is performed via email, system messages, and other means to inform administrators of the execution status of power sharding, including the reason for the sharding, the details of the adjustment, and the impact on task performance. Operation logging is performed, recording relevant data from the entire power sharding process in a log file, including monitoring data, the basis for sharding decisions, and executed operations, for subsequent auditing and analysis. Continuous monitoring is performed by returning to the resource and task monitoring step to continue real-time monitoring of GPU power usage and task load, forming a closed-loop dynamic management process that allows for timely adjustment of the next power sharding process based on system changes.

[0107] In another embodiment of the present application, the method further includes: Data synchronization and migration.

[0108] In this embodiment, if the partitioning involves moving data between different GPU memory areas, ensure accurate data synchronization and migration. For example, in distributed deep learning training, model parameters and data may need to be redistributed across different GPU cards.

[0109] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0110] In one embodiment of the present application, a computing power dynamic slicing system is provided, which corresponds one-to-one to a computing power dynamic slicing method in the above embodiment. The computing power dynamic slicing system includes: Resource and task monitoring module, which collects historical system load data and task execution patterns and uses machine learning algorithm models to predict future resource usage; The load assessment and demand analysis module is also used to monitor the overall load data of the system; A resource query module that searches for available computing resources in the system. A judgment module is used to trigger a computing power split decision if the overall system load data exceeds a preset load threshold, the resource demand of a single task exceeds a preset task threshold, and / or the future resource usage exceeds the system computing capacity; A resource compatibility check module is used to match the compatibility of available computing resources with the resource parameters of the task to be executed and obtain the compatibility result; The strategy formulation module is used to formulate the segmentation strategy based on the compatibility results, available computing resources, and resource parameters of the tasks to be executed; The execution module is used to reallocate available computing resources according to the sharding strategy.

[0111] Each module of the aforementioned dynamic computing power sharding system can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor within a computer device in hardware form, or stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0112] In one embodiment of the present application, a computer device is provided, which may be a server. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device can be implemented by any type of volatile or non-volatile storage device, or a combination thereof. Volatile or non-volatile storage devices include, but are not limited to, magnetic disks, optical disks, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM (Erasable Programmable Read Only Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), magnetic storage, flash memory, and PROM (Programmable Read-Only Memory). The memory of the computer device provides an environment for the operation of the operating system and computer programs stored therein. The network interface of the computer device is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements the steps of the method for dynamic computing power slicing described in the above embodiment.

[0113] In one embodiment of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program. When executed by a processor, the computer program implements the steps of a method for dynamically dividing computing power in the above embodiment. Computer-readable storage media include ROM (Read-Only Memory), RAM (Random-Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic disks, floppy disks, and the like.

[0114] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device of the present application can be divided into different functional units or modules to complete all or part of the functions described above.

Claims

1. A method for dynamic computing power division, characterized in that: The method comprises: Collect historical system load data and task execution patterns, and use machine learning algorithm models to predict future resource usage; Monitor the overall load data of the system; Search for available computing resources in the system; If the overall system load exceeds the overall load threshold, the resource demand of a single task exceeds the resource shortage threshold, and / or the future resource usage exceeds the system computing capacity, a computing power splitting decision is triggered; Matching the compatibility of the available computing resources with the resource parameters of the task to be executed, and obtaining a compatibility result; Formulate a segmentation strategy based on the compatibility result, the available computing resources, and the resource parameters of the task to be executed; The available computing resources are reallocated according to the partitioning strategy.

2. A method for dynamic computing power division according to claim 1, characterized in that: The method further comprises: Setting task sorting rules, wherein the task sorting rules include task priority strategy and / or time slice rotation strategy; Maintaining a task queue according to the task sorting rule, wherein the task queue stores the tasks to be executed and resource parameters of the tasks to be executed; The segmentation strategy formulated according to the compatibility result, the available computing resources and the resource parameters of the task to be executed is: A partitioning strategy is formulated based on the compatibility result, the task queue, and the available computing resources.

3. A method for dynamic computing power division according to claim 1, characterized in that: The method further comprises: Setting task sorting rules, wherein the task sorting rules include a priority strategy; Obtaining the priority of the task being executed according to the priority strategy; The computing resources of the task being executed whose priority is lower than the preset low priority value are released as available computing resources and / or the computing resources of the task being executed whose priority is lower than the task to be executed are released as available computing resources.

4. A method for dynamic computing power division according to claim 1, characterized in that: The method further comprises: When the first task in the executing task is interrupted; Saving the execution status of the first task; Releasing the computing resources of the first task as available computing resources; Until the interruption disappears, the first task is executed again.

5. A method for dynamic computing power division according to claim 1, characterized in that: Before reallocating the available computing resources according to the slicing strategy, the method further includes: Use a modeling tool or sandbox environment to predict the operation status of the system computing resources after the segmentation according to the segmentation strategy.

6. A method for dynamic computing power division according to claim 1, characterized in that: The method further comprises: Performing validation testing, including benchmark testing, stress testing, and / or stability testing; If the verification test result is negative, roll back to the previous system computing resource allocation configuration; If the verification test result does not meet the expected value, adjust the segmentation strategy or optimize the execution method of the task.

7. A method for dynamic computing power division according to claim 1, characterized in that: The available computing resources in the search system include: Detection system computing resources; Detect the execution progress and performance indicators of tasks being executed in the system; Available computing resources are obtained according to data of the system computing resources occupied by the task being executed.

8. A computing power dynamic division system, characterized in that: The system comprises: Resource and task monitoring module, which collects historical system load data and task execution patterns and uses machine learning algorithm models to predict future resource usage; The load assessment and demand analysis module is also used to monitor the overall load data of the system; A resource query module that searches for available computing resources in the system. A judgment module is configured to trigger a computing power splitting decision if the overall system load exceeds a preset load threshold, the resource requirement of a single task exceeds a preset task threshold, and / or the future resource usage exceeds the system computing capacity; A resource compatibility checking module is used to match the compatibility of the available computing resources with the resource parameters of the task to be executed and obtain a compatibility result; A strategy formulation module, configured to formulate a segmentation strategy based on the compatibility result, the available computing resources, and the resource parameters of the task to be executed; An execution module is used to reallocate the available computing resources according to the segmentation strategy.

9. A computer device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the computer program, the steps of a method for dynamically dividing computing power according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a method for dynamically dividing computing power according to any one of claims 1 to 7 are implemented.

Images