A heterogeneous computing resource aggregation and intelligent task matching and distribution system and method

Abstract

This application discloses a heterogeneous computing power resource aggregation and intelligent task matching and distribution system and method, belonging to the field of computing power resource distribution technology. The heterogeneous computing power resource aggregation and intelligent task matching and distribution system includes a computing power cluster registration module, a task publishing module, and a computing power market coordination module. The computing power cluster registration module is used for clusters to declare their heterogeneous hardware and software environment capabilities; the task publishing module allows users to define task requirements; the computing power market coordination module has a built-in intelligent scheduling engine, which achieves optimal matching between tasks and computing power clusters based on multiple dimensions such as cluster capabilities, real-time load, pricing strategy, user budget, task priority, and cluster reputation score, and issues task instructions to the proxy service of the matched cluster. This invention aims to solve the problem of difficult unified management and efficient utilization of heterogeneous computing power resources, and improves resource allocation efficiency and user task execution satisfaction through unified resource description and multi-dimensional intelligent matching.

Description

Technical Field

[0001] This application belongs to the field of computing power resource distribution technology, and in particular relates to a heterogeneous computing power resource aggregation and intelligent task matching and distribution system and method. Background Technology

[0002] With the explosive growth in demand for high-performance computing such as AI training and scientific computing, heterogeneous computing architectures (CPU / GPU / FPGA / NPU, etc.) have become a core means of improving computing efficiency because they can provide differentiated computing power support for different tasks. However, the heterogeneity of their hardware types, software environments, and network topologies leads to three major challenges in resource management: 1. Fragmented resource descriptions: The hardware configurations (such as GPU models and memory capacity) and software dependencies (such as CUDA versions and container environments) of different clusters lack standardized descriptions, making it difficult to make horizontal comparisons; 2. Single scheduling dimension: Traditional schedulers (such as Kubernetes) only support basic resource (CPU / memory) matching, ignoring key factors such as network latency, software compatibility, and user cost constraints; 3. Insufficient dynamic coordination: Real-time load fluctuations and cross-domain resource collaboration (such as multi-cloud scheduling) require dynamic adjustment of strategies, but existing systems rely on static rules or manual intervention, resulting in delayed response. The industry is attempting to break through the current predicament through two approaches: Technology-driven solutions: Optimize hardware layer resource allocation (such as 8Vision's dynamic migration patent); Market-oriented mechanism: Introducing economic models to coordinate supply and demand (such as computing power trading platforms), but not yet deeply coupled with the scheduling engine.

[0003] However, none of the existing solutions have solved the problem of standardizing resource descriptions and coordinating multi-dimensional market mechanisms.

[0004] Therefore, it is necessary to provide a new system and method for heterogeneous computing resource aggregation and intelligent task matching and distribution to solve the above-mentioned technical problems. Summary of the Invention

[0005] The purpose of this disclosure is to provide a heterogeneous computing resource aggregation and intelligent task matching and distribution system and method to solve the above problems.

[0006] This disclosure achieves the above objectives through the following technical solutions: A heterogeneous computing power resource aggregation and intelligent task matching and distribution system includes a computing power cluster registration module, a task publishing module, a computing power market coordination module, and a cluster agent service; The cluster proxy service is deployed on each registered computing power cluster, and the computing power cluster registers its computing resources and capabilities information with the computing power cluster registration module through its respective cluster proxy service; The computing power cluster registration module is used to register the computing resources and capabilities information of each computing power cluster; The task publishing module is used to provide users with the ability to define and submit their computing tasks. The computing power market coordination module receives computing tasks, matches resources, and distributes computing tasks.

[0007] As a further optimization of this disclosure, the computing power market coordination module includes an intelligent scheduling engine, a resource description and information database, a task queue, a pricing strategy module, and a cluster reputation score module; The resource description and information database is used to store the hardware and software capability information of the computing power cluster collected by the computing power cluster registration module; The task queue is used to cache tasks to be scheduled submitted by the task publishing module; The pricing strategy module is used to manage and provide billing information for each computing power cluster; The cluster reputation score module is used to maintain the reputation score of each computing power cluster and is dynamically updated based on task completion and stability. The intelligent scheduling engine obtains tasks from the task queue, retrieves static information of the computing power cluster from the resource description and information database, and obtains the real-time status of the computing power cluster through the cluster proxy service. The intelligent scheduling engine combines the pricing strategy provided by the pricing strategy module and the computing power cluster reputation score provided by the cluster reputation score module to perform multi-dimensional matching decisions and issues task instructions to the selected computing power cluster's proxy service for execution.

[0008] As a further optimization of this disclosure, the specific process by which the computing power market coordination module receives computing tasks, matches resources, and distributes computing tasks includes: The intelligent scheduling engine receives task requests from the task queue, static information of computing clusters from the resource description and information database, real-time status of computing clusters from each of the cluster agent services, current billing strategies of each computing cluster from the pricing strategy module, historical performance reputation scores of each computing cluster from the cluster reputation score module, and user-set budget and task priorities. Candidate computing power clusters are selected from the resource description and information database based on the task's hard hardware requirements and software dependencies. The candidate computing power clusters are comprehensively scored; Based on the comprehensive score, a preset weighted scoring algorithm is used to calculate the comprehensive score of each candidate computing power cluster; Select the computing power cluster with the highest overall score as the target execution cluster; Send task instructions to the agent service of the selected computing cluster. As a further optimization of this disclosure, the real-time status of the computing cluster includes CPU / GPU utilization, memory usage, and network congestion.

[0009] As a further optimization of this disclosure, the candidate computing power clusters are comprehensively scored based on the following dimensions: Capability matching: The degree to which the hardware and software environment of the computing cluster matches the task requirements; Real-time load: Prioritize computing clusters with lower loads to ensure timely task startup and efficient operation; Pricing strategy: Within the user's budget, consider the pricing of computing power clusters and prioritize those with high cost-performance ratios; User Budget: Ensure that the cost of the selected computing cluster does not exceed the user's set budget; Task priority: High-priority tasks are allowed to be matched to clusters that are slightly more expensive but have better performance or are less busy; Cluster reputation score: Prioritize clusters with a history of high task completion quality and good stability.

[0010] As a further optimization of this disclosure, the reputation score of the computing power cluster can be calculated based on historical task success rate, on-time completion rate, and user feedback.

[0011] As a further optimization of this disclosure, the task instructions include a task script, data location, and required environment configuration.

[0012] A method for heterogeneous computing resource aggregation and intelligent task matching and distribution, applicable to the aforementioned heterogeneous computing resource aggregation and intelligent task matching and distribution system, includes the following steps: The computing power cluster registers its hardware and software resource information with the computing power cluster registration module through its cluster agent service; Users define and submit task requirements through the task publishing module, including hardware preferences, software dependencies, budget, and priority. After the task information is formatted, it enters the task queue of the computing power market coordination module; The intelligent scheduling engine retrieves tasks to be processed from the task queue; The intelligent scheduling engine queries resource descriptions and information databases to obtain static capability information for all registered computing power clusters; The intelligent scheduling engine queries and collects the real-time load status of the cluster agent services of each computing power cluster. The intelligent scheduling engine integrates task requirements, cluster static capabilities, real-time load, price information provided by the pricing strategy module, and reputation points provided by the cluster reputation points module. It then executes a multi-dimensional matching algorithm to calculate a comprehensive score for each computing power cluster that meets the basic conditions. Determine if a cluster that meets the criteria and has the best score has been found; if so, proceed to the next step; if no suitable cluster is found at the moment, the task can choose to wait or notify the user according to the strategy. The intelligent scheduling engine sends the task execution instructions to the cluster agent service of the selected optimal computing power cluster; The cluster agent service of the computing power cluster prepares and executes tasks in the local computing power cluster environment based on the received instructions; The computing power cluster reports the execution status and results of the tasks to the computing power market coordination module; The computing power market coordination module updates the reputation score of the relevant cluster in the cluster reputation score module based on the completion status of the task, and updates the cluster usage statistics in the resource description and information database, and the process ends.

[0013] An electronic device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to execute the program stored in the memory to implement the heterogeneous computing resource aggregation and intelligent task matching and distribution method.

[0014] A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the heterogeneous computing resource aggregation and intelligent task matching and distribution method.

[0015] The beneficial effects of this disclosure are as follows: A method is proposed for the standardized, structured declaration and registration of heterogeneous hardware (GPU model / quantity / video memory, CPU, memory, storage, network) and software environment capabilities (operating system, library, framework version) of computing power clusters; The intelligent scheduling engine can make optimal matching decisions between tasks and clusters by comprehensively considering more than six dimensions, including the cluster's static capabilities, real-time dynamic load, the pricing strategy provided by the cluster, the user-defined budget, the task's own priority, and the cluster's historical reputation score. Not only does it send task instructions to the agent service matching the cluster, the agent service also reports the real-time status of the cluster and the results of task execution, forming an information feedback loop, which is used to dynamically update the cluster reputation and assist in future scheduling decisions; By taking user budget constraints and cluster pricing strategies as important dimensions for matching decisions, cost-sensitive task scheduling has been achieved. A cluster reputation score mechanism is introduced, and historical service quality is taken into account in scheduling considerations to incentivize the cluster to provide high-quality services and help users avoid low-quality resources. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a system architecture diagram of an embodiment of this disclosure; Figure 2 This is a flowchart of a method in an embodiment of this disclosure; Figure 3 This is a block diagram of the device structure in an embodiment of this disclosure. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] like Figure 1 As shown, a heterogeneous computing power resource aggregation and intelligent task matching and distribution system includes a computing power cluster registration module, a task publishing module, a computing power market coordination module, and a cluster agent service. Users interact with the Task Publishing Module (TPM) through the user interface / API to submit task requests.

[0020] The task publishing module transmits task information to the computing power market coordination module (TMCM).

[0021] Computing clusters (such as computing clusters 1 to N) register their resource and capability information with the Computing Cluster Registration Module (SCRM) through their respective cluster agent services (CAS1 to CASN). The Computing Cluster Registration Module stores this information and provides it to the computing market coordination module.

[0022] The computing power market coordination module is the core of the system, comprising the Intelligent Scheduling Engine (ISE), Resource Description and Information Base (RDI), Task Queue (TQ), Pricing Strategy Module (PSM), and Cluster Reputation Score Module (RSM). The Resource Description and Information Base (RDI) stores cluster hardware and software capability information collected by the SCRM; the Task Queue (TQ) caches tasks submitted by the TPM for scheduling; and the Pricing Strategy Module (PSM) manages and provides billing information for each cluster (such as hourly billing and billing based on resource usage).

[0023] The Cluster Reputation Score (RSM) module maintains the reputation score of each cluster, dynamically updating it based on task completion status, stability, and other factors. The intelligent scheduling engine retrieves tasks from the task queue, obtains static cluster information from the resource description and information database, and acquires the real-time status of the clusters (such as load) through the cluster proxy service. The intelligent scheduling engine combines pricing strategies and cluster reputation scores to perform multi-dimensional matching decisions and issues task instructions to the selected cluster's proxy service for execution.

[0024] In this embodiment, the components of the heterogeneous computing power resource aggregation and intelligent task matching and distribution system are described in detail below: 1. Compute Cluster Registration Module (SCRM) Function: Allows each computing cluster to declare the computing resources it can provide.

[0025] Implementation: Cluster administrators submit detailed information about their clusters through interfaces provided by this module (such as a web interface or API).

[0026] Heterogeneous hardware information: GPU: Model (e.g., NVIDIA A100, RTX 3090), quantity, single card memory size (e.g., 40GB, 24GB), total memory.

[0027] CPU: Model, number of cores, clock speed.

[0028] Memory: Total capacity, type (e.g., DDR4).

[0029] Storage: type (e.g., SSD, HDD), total capacity, available capacity, IOPS.

[0030] Network: Bandwidth (e.g., 10Gbps, 100Gbps), latency characteristics.

[0031] Software environment capabilities: Operating system: such as Linux (Ubuntu 20.04, CentOS 7).

[0032] Core libraries: such as CUDA version, cuDNN version.

[0033] Frameworks / software: such as TensorFlow versions, PyTorch versions, specific MPI implementations, Docker support, etc.

[0034] Custom environment: Supports declaring pre-configured Docker images or Conda environments.

[0035] Data storage: Registration information is stored in a structured manner (e.g., in JSON or XML format) in the Resource Description and Information Base (RDI) for querying by the intelligent scheduling engine.

[0036] 2. Task Publishing Module (TPM) Function: Users can define and submit their computing task requirements through this module.

[0037] Implementation: Users submit tasks through a graphical user interface (GUI) or a programming interface (API).

[0038] Hardware preferences: GPU requirements: minimum GPU model, minimum quantity, minimum single-card video memory.

[0039] CPU requirements: minimum number of cores, minimum clock speed.

[0040] Memory requirements: Minimum memory capacity.

[0041] Storage requirements: Minimum storage space.

[0042] Software dependencies: Required operating system, library version (such as CUDA 11.x), and framework version (such as PyTorch 1.10+).

[0043] You can specify the Docker image name or the Conda environment configuration file.

[0044] Budget: The maximum amount a user is willing to pay for the task (e.g., an hourly fee cap or a total budget).

[0045] Task priority: Users can set the urgency level of a task (e.g., high, medium, low).

[0046] Other: Task execution scripts, data input / output paths, etc.

[0047] Data flow: Task requirements are formatted and stored in the task queue (TQ) to await scheduling.

[0048] 3. Compute Market Coordination Module (TMCM) Function: The core scheduling hub, responsible for receiving tasks, matching resources, and distributing tasks.

[0049] Core Component: Intelligent Scheduling Engine (ISE) enter: (1) Task requirements from the task queue (TQ).

[0050] (2) Cluster static capability information from the Resource Description and Information Base (RDI).

[0051] (3) Real-time cluster load information from each cluster agent service (CAS) (such as CPU / GPU utilization, memory usage, network congestion).

[0052] (4) Current billing policies for each cluster from the pricing strategy module (PSM).

[0053] (5) Historical performance reputation scores of each cluster from the Cluster Reputation Score Module (RSM).

[0054] (6) User-defined budget and task priorities.

[0055] Matching logic (considering multiple factors): a. Preliminary screening: Select candidate clusters from RDI based on the task's hardware requirements (such as required GPU model and minimum video memory) and software dependencies (such as specific CUDA version and operating system).

[0056] b. Scoring and Ranking: Candidate clusters are scored comprehensively based on the following dimensions: CapabilityMatch: The degree to which cluster hardware (such as GPU performance exceeds requirements), software environment, and task requirements are well-matched.

[0057] Real-time load: Prioritize clusters with lower loads to ensure timely task startup and efficient operation.

[0058] Price Normalized: Within the user's budget, considering the cluster's pricing, clusters with high cost-performance ratios may be prioritized.

[0059] User Budget (BudgetFit): Ensures that the cost of the selected cluster does not exceed the user-defined budget.

[0060] Task Priority Factor: High-priority tasks may be allowed to be matched to clusters that are slightly more expensive but have better performance or are less busy.

[0061] Cluster Reputation Score: Prioritizes clusters with a history of high task completion quality and stability. The reputation score can be calculated based on historical task success rate, on-time completion rate, user feedback, etc.

[0062] c. Decision Algorithm: A pre-defined weighted scoring algorithm or a more complex optimization algorithm (such as a machine learning-based prediction model or a multi-objective optimization algorithm) is used to calculate the comprehensive score for each candidate cluster. For example: Score=w1\*CapabilityMatch+w2\*(1-Load)+w3\*(1-PriceNormalized)+w4\*BudgetFit+w5\*PriorityFactor + w6\*ReputationScore; w1 to w6 represent the weights of each factor, which can be dynamically adjusted according to the system strategy.

[0063] d. Optimal selection: Select the cluster with the highest overall score as the target execution cluster.

[0064] Task distribution: Send task instructions (including task scripts, data locations, required environment configurations, etc.) to the agent service (CAS) of the selected computing power cluster via the network.

[0065] 4. Cluster Agent Service (CAS) Function: Deployed on each registered computing cluster, serving as the interface between that cluster and the system core.

[0066] accomplish: Register the cluster's resource information with SCRM.

[0067] Regularly report the real-time load status of this cluster (CPU / GPU utilization, memory, network, etc.) to ISE.

[0068] Receive task instructions from ISE.

[0069] Prepare the task execution environment in the local cluster environment (such as pulling Docker images and configuring the Conda environment).

[0070] Start and monitor task execution.

[0071] The task execution status (running, completed, failed) and results are fed back to TMCM.

[0072] The following is an example: Suppose user A submits a deep learning training task through the task publishing module: Hardware preference: At least one NVIDIA A100 GPU with at least 40GB of video memory.

[0073] Software dependencies: PyTorch 1.12, CUDA 11.3, Ubuntu 20.04.

[0074] Budget: $2 / hour.

[0075] Task priority: High.

[0076] The system has the following registered clusters: Cluster X: 2 x NVIDIA A100 (80GB VRAM), Ubuntu 20.04, PyTorch 1.12, CUDA 11.3. Real-time load: GPU 80%. Price: $2.5 / hour. Reputation score: 95 / 100.

[0077] Cluster Y: 4x NVIDIA V100 (32GB VRAM), Ubuntu 18.04, PyTorch 1.10, CUDA 10.2. Real-time load: GPU 20%. Price: $1.5 / hour. Reputation score: 90 / 100.

[0078] 3. Cluster Z: 1 x NVIDIA A100 (40GB VRAM), Ubuntu 20.04, PyTorch 1.12, CUDA 11.3. Real-time load: GPU 30%. Price: $1.8 / hour. Reputation score: 88 / 100.

[0079] The matching process of the Intelligent Scheduling Engine (ISE): 1. Initial screening: Cluster X: Meets hardware (A100, 80GB > 40GB) and software (PyTorch 1.12, CUDA 11.3, Ubuntu 20.04) requirements.

[0080] Cluster Y: Does not meet the hardware (V100, 32GB < 40GB) and software (PyTorch, CUDA, OS version all do not meet) requirements and is therefore excluded.

[0081] Cluster Z: Meets hardware (A100, 40GB = 40GB) and software requirements.

[0082] The candidate clusters are: Cluster X and Cluster Z.

[0083] 2. Scoring and Ranking (assuming equal weights, actual weights are configurable): Cluster X: Capability match: High (80GB VRAM far exceeds requirements).

[0084] Real-time load: High (80%) -> Low score.

[0085] Pricing strategy: $2.5 / hour -> Over budget.

[0086] User budget fit: Not fit.

[0087] Task priority: High -> Budget can be slightly relaxed or load can be tolerated.

[0088] Cluster reputation score: High (95).

[0089] Cluster Z: Capability matching: Medium (40GB VRAM meets the requirements).

[0090] Real-time load: Low (30%) -> High score.

[0091] Pricing strategy: $1.8 / hour -> within budget.

[0092] User budget compliance: Compliant.

[0093] Task priority: High.

[0094] Cluster reputation score: Medium (88).

[0095] 3. Decision Algorithm: Cluster X is typically downgraded or excluded if it exceeds the budget, unless a high-priority factor can cover the budget overrun and there are no other options.

[0096] Cluster Z is within budget, has low load, and meets basic requirements.

[0097] The intelligent scheduling engine's comprehensive evaluation shows that even though cluster X has higher reputation and absolute capacity, cluster Z has significant advantages in terms of budget and real-time load, and meets the task requirements. If high priority allows for a slight overrun of the budget or system configuration allows it, and cluster X's performance advantage can significantly shorten the execution time of high-priority tasks, then cluster X might also be chosen (e.g., if the value of shortened task execution time outweighs the budget overrun). However, in this example, it is assumed that the budget is a hard constraint or low load takes precedence.

[0098] 4. Optimal choice: The intelligent scheduling engine selects cluster Z.

[0099] 5. Task Issuance: The system sends the task instructions and related data to the agent service of cluster Z, and the agent service starts the task on cluster Z.

[0100] like Figure 2 As shown, this embodiment also discloses a method for heterogeneous computing resource aggregation and intelligent task matching and distribution, applicable to the aforementioned heterogeneous computing resource aggregation and intelligent task matching and distribution system, including the following steps: Step S1: The process begins.

[0101] Step S2: The computing cluster registers its hardware and software resource information with the Computing Cluster Registration Module (SCRM) through its agent service.

[0102] Step S3: Users define and submit task requirements through the Task Publishing Module (TPM), including hardware preferences, software dependencies, budget, and priority.

[0103] Step S4: After the task information is formatted, it enters the task queue (TQ) of the computing power market coordination module (TMCM).

[0104] Step S5: The Intelligent Scheduling Engine (ISE) retrieves a task to be processed from the Task Queue (TQ).

[0105] Step S6: ISE queries the Resource Description and Information Base (RDI) to obtain static capability information (hardware and software environment) for all registered clusters.

[0106] Step S7: ISE queries and collects the real-time load status of the proxy service (CAS) of each cluster.

[0107] Step S8: ISE integrates task requirements, cluster static capabilities, real-time load, pricing information provided by the pricing strategy module (PSM), and reputation score provided by the cluster reputation score module (RSM), and executes a multi-dimensional matching algorithm to calculate a comprehensive score for each cluster that meets the basic conditions.

[0108] Step S9: Determine if a cluster that meets the criteria and has the best score has been found. If found, proceed to step S10; if no suitable cluster is available temporarily (e.g., all clusters that meet the criteria are over budget or under excessive load), the task can choose to wait or notify the user according to the strategy.

[0109] Step S10: ISE sends the task execution instructions (such as code, data links, and environment configuration requirements) to the agent service (CAS) of the selected optimal cluster.

[0110] Step S11: The cluster's proxy service (CAS) prepares and executes tasks in the local cluster environment based on the received instructions.

[0111] Step S12: CAS reports the execution status of the task (such as start, running, success, failure) and the execution result (if applicable) to TMCM.

[0112] Step S13: TMCM updates the reputation score of the relevant cluster in the Cluster Reputation Score Module (RSM) based on the completion status of the task (such as whether it was successful or on time), and may also update the cluster usage statistics in the Resource Description and Information Repository.

[0113] Step S14: Process ends (one scheduling for this task). ISE will continue to fetch the next task from TQ (return to S5).

[0114] See Figure 3 The electronic device provided in the embodiments of this disclosure includes a processor 1110, a communication interface 1120, a memory 1130 and a communication bus 1140, wherein the processor 1110, the communication interface 1120 and the memory 1130 communicate with each other through the communication bus 1140. Memory 1130 is used to store computer programs; The processor 1110, when executing the program stored in the memory 1130, implements the above-mentioned heterogeneous computing resource aggregation and intelligent task matching and distribution system. The aforementioned communication bus 1140 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus 1140 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, it is represented by only one thick line in the figure, but this does not indicate that there is only one bus or one type of bus.

[0115] The communication interface 1120 is used for communication between the above-mentioned electronic device and other devices.

[0116] The memory 1130 may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory 1130 may also be at least one storage device located remotely from the aforementioned processor 1110.

[0117] Embodiments of this disclosure also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the heterogeneous computing resource aggregation and intelligent task matching and distribution system described above.

[0118] The embodiments described above are merely examples of several implementations of this disclosure, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent disclosure. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this disclosure, and these modifications and improvements all fall within the protection scope of this disclosure.

Claims

1. A heterogeneous computing resource aggregation and intelligent task matching and distribution system, characterized in that, This includes a computing cluster registration module, a task publishing module, a computing market coordination module, and cluster agent services; The cluster proxy service is deployed on each registered computing power cluster, and the computing power cluster registers its computing resources and capabilities information with the computing power cluster registration module through its respective cluster proxy service; The computing power cluster registration module is used to register the computing resources and capabilities information of each computing power cluster; The task publishing module is used to provide users with the ability to define and submit their computing tasks. The computing power market coordination module receives computing tasks, matches resources, and distributes computing tasks.

2. The heterogeneous computing power resource aggregation and intelligent task matching and distribution system according to claim 1, characterized in that, The computing power market coordination module includes an intelligent scheduling engine, a resource description and information database, a task queue, a pricing strategy module, and a cluster reputation score module. The resource description and information database is used to store the hardware and software capability information of the computing power cluster collected by the computing power cluster registration module; The task queue is used to cache tasks to be scheduled submitted by the task publishing module; The pricing strategy module is used to manage and provide billing information for each computing power cluster; The cluster reputation score module is used to maintain the reputation score of each computing power cluster and is dynamically updated based on task completion and stability. The intelligent scheduling engine obtains tasks from the task queue, retrieves static information of the computing power cluster from the resource description and information database, and obtains the real-time status of the computing power cluster through the cluster proxy service. The intelligent scheduling engine combines the pricing strategy provided by the pricing strategy module and the computing power cluster reputation score provided by the cluster reputation score module to perform multi-dimensional matching decisions and issues task instructions to the selected computing power cluster's proxy service for execution.

3. The heterogeneous computing power resource aggregation and intelligent task matching and distribution system according to claim 2, characterized in that, The specific process by which the computing power market coordination module receives computing tasks, matches resources, and distributes computing tasks includes: The intelligent scheduling engine receives task requests from the task queue, static information of computing clusters from the resource description and information database, real-time status of computing clusters from each of the cluster agent services, current billing strategies of each computing cluster from the pricing strategy module, historical performance reputation scores of each computing cluster from the cluster reputation score module, and user-set budget and task priorities. Candidate computing power clusters are selected from the resource description and information database based on the task's hard hardware requirements and software dependencies. The candidate computing power clusters are comprehensively scored; Based on the comprehensive score, a preset weighted scoring algorithm is used to calculate the comprehensive score of each candidate computing power cluster; Select the computing power cluster with the highest overall score as the target execution cluster; Send task instructions to the agent service of the selected computing cluster.

4. The heterogeneous computing power resource aggregation and intelligent task matching and distribution system according to claim 3, characterized in that, The real-time status of the computing cluster includes CPU / GPU utilization, memory usage, and network congestion.

5. The heterogeneous computing power resource aggregation and intelligent task matching and distribution system according to claim 3, characterized in that, The candidate computing power clusters are comprehensively scored based on the following dimensions: Capability matching: The degree to which the hardware and software environment of the computing cluster matches the task requirements; Real-time load: Prioritize computing clusters with low load to ensure timely task startup and efficient operation; Pricing strategy: Within the user's budget, consider the pricing of computing power clusters and prioritize those with high cost-performance ratios; User Budget: Ensure that the cost of the selected computing cluster does not exceed the user's set budget; Task priority: High-priority tasks are allowed to be matched to clusters that are slightly more expensive but have better performance or are less busy; Cluster reputation score: Prioritize clusters with a history of high task completion quality and good stability.

6. The heterogeneous computing power resource aggregation and intelligent task matching and distribution system according to claim 1, characterized in that, The reputation score of the computing power cluster can be calculated based on historical task success rate, on-time completion rate, and user feedback.

7. The heterogeneous computing power resource aggregation and intelligent task matching and distribution system according to claim 3, characterized in that, The task instructions include the task script, data location, and required environment configuration.

8. A method for heterogeneous computing resource aggregation and intelligent task matching and distribution, applicable to the heterogeneous computing resource aggregation and intelligent task matching and distribution system as described in any one of claims 1 to 7, characterized in that, Includes the following steps: The computing power cluster registers its hardware and software resource information with the computing power cluster registration module through its cluster agent service; Users define and submit task requirements through the task publishing module, including hardware preferences, software dependencies, budget, and priority. After the task information is formatted, it enters the task queue of the computing power market coordination module; The intelligent scheduling engine retrieves tasks to be processed from the task queue; The intelligent scheduling engine queries resource descriptions and information databases to obtain static capability information for all registered computing power clusters; The intelligent scheduling engine queries and collects the real-time load status of the cluster agent services of each computing power cluster. The intelligent scheduling engine integrates task requirements, cluster static capabilities, real-time load, price information provided by the pricing strategy module, and reputation points provided by the cluster reputation points module. It then executes a multi-dimensional matching algorithm to calculate a comprehensive score for each computing power cluster that meets the basic conditions. Determine if a cluster that meets the criteria and has the best score has been found; if so, proceed to the next step. If a suitable cluster is not available at the moment, the task can choose to wait or notify the user based on the strategy. The intelligent scheduling engine sends the task execution instructions to the cluster agent service of the selected optimal computing power cluster; The cluster agent service of the computing power cluster prepares and executes tasks in the local computing power cluster environment based on the received instructions; The computing power cluster reports the execution status and results of the tasks to the computing power market coordination module; The computing power market coordination module updates the reputation score of the relevant cluster in the cluster reputation score module based on the completion status of the task, and updates the cluster usage statistics in the resource description and information database, and the process ends.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor is used to execute the program stored in the memory to implement the heterogeneous computing resource aggregation and intelligent task matching and distribution method as described in claim 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the heterogeneous computing resource aggregation and intelligent task matching and distribution method as described in claim 8.

Images