Task scheduling method, heterogeneous scheduling system and computer device

Abstract

The application relates to a task scheduling method, a heterogeneous scheduling system and computer equipment. A plurality of subtasks obtained after a composite computing task to be processed is subjected to task splitting processing are acquired; from the plurality of subtasks, a to-be-scheduled subtask satisfying a preset scheduling readiness condition is determined; for each to-be-scheduled subtask, based on a preset operator-acceleration card mapping table, a candidate acceleration card set matched with the to-be-scheduled subtask is determined from a heterogeneous near-storage computing card pool; scheduling decision analysis is performed on the candidate acceleration card set, so as to screen a first acceleration card satisfying a preset optimal scheduling condition from the candidate acceleration card set; the to-be-scheduled subtask is distributed to the first acceleration card, so that the first acceleration card executes the to-be-scheduled subtask to generate a corresponding task result; the problem of low task processing efficiency caused by cross-system data migration is effectively avoided, and the overall energy efficiency of the system is effectively improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a task scheduling method, a heterogeneous scheduling system, and a computer device. Background Technology

[0002] With the deep integration of bioinformatics, artificial intelligence, and data science, the industry's demand for processing massive amounts of data is exhibiting new characteristics: computationally intensive, complex in type, and multi-layered in process. Taking the field of intelligent pharmaceuticals as an example, processing high-throughput protein sequence or gene data typically requires first encrypting and protecting sensitive biometric data, followed by performing intelligent analyses such as machine learning, deep learning, or large-scale sequence alignment on the encrypted or decrypted data. This hybrid computational load of "encryption preprocessing + intelligent analysis" poses comprehensive challenges to the performance, energy efficiency, and versatility of computing systems. However, related technologies exhibit a highly specialized "computing silo" state; for example, a specific near-in-memory computing device (such as UPMEM) may only achieve tens of times the energy efficiency improvement for cryptographic algorithms (such as AES, SHA-256), while another near-in-memory computing device is specifically optimized for neural network matrix operations. Faced with end-to-end composite tasks of "encryption before intelligent analysis," users have to manually break down the task process and repeatedly transfer data between heterogeneous hardware systems, resulting in severe scheduling inefficiency, resource redundancy, and energy waste.

[0003] There is currently no effective solution to the problem of low scheduling efficiency of composite tasks in related technologies. Summary of the Invention

[0004] Therefore, it is necessary to provide a task scheduling method, a heterogeneous scheduling system, and a computer device to address the aforementioned technical problems.

[0005] Firstly, this application provides a task scheduling method applied to a server host in a heterogeneous scheduling system; the heterogeneous scheduling system further includes a heterogeneous near-memory computing card pool; the method includes:

[0006] Obtain multiple subtasks obtained after the composite computing task to be processed has been split into tasks;

[0007] From the multiple subtasks, determine the subtasks to be scheduled that meet the preset scheduling ready conditions;

[0008] For each of the scheduled subtasks, a set of candidate accelerator cards matching the scheduled subtasks is determined from the heterogeneous near-memory computing card pool based on a preset operator-accelerator card mapping table.

[0009] The candidate accelerator card set is subjected to scheduling decision analysis in order to select the first accelerator card that meets the preset optimal scheduling conditions from the candidate accelerator card set;

[0010] The scheduled subtask is assigned to the first acceleration card so that the first acceleration card executes the scheduled subtask and generates the corresponding task result.

[0011] In one embodiment, the step of determining a set of candidate accelerator cards matching the subtask to be scheduled from the heterogeneous near-memory computing card pool based on a preset operator-accelerator card mapping table includes:

[0012] Obtain the preset operator-accelerator card mapping table;

[0013] Identify the operator type of the subtask to be scheduled;

[0014] Based on the operator type of the subtask to be scheduled, at least one candidate accelerator card that matches the operator type of the subtask to be scheduled is retrieved from the preset operator-accelerator card mapping table.

[0015] At least one of the candidate accelerator cards is determined as a set of candidate accelerator cards that match the scheduled subtask.

[0016] In one embodiment, the candidate accelerator card set includes at least one candidate accelerator card; the step of performing scheduling decision analysis on the candidate accelerator card set to select a first accelerator card that meets preset optimal scheduling conditions includes:

[0017] Calculate the overall scheduling score of the subtask to be scheduled on each of the candidate accelerator cards;

[0018] The candidate accelerator card with the highest comprehensive scheduling score is determined as the first accelerator card that meets the preset optimal scheduling conditions.

[0019] In one embodiment, calculating the comprehensive scheduling score of the subtask to be scheduled on each of the candidate accelerator cards includes:

[0020] For each candidate accelerator card, a matching degree analysis is performed on the operator type of the subtask to be scheduled and the computing power of the candidate accelerator card to obtain a first factor score;

[0021] Based on the input data required by the subtask to be scheduled, data locality analysis is performed on the candidate accelerator cards to obtain a second factor score.

[0022] The current load status of the candidate accelerator cards is quantitatively analyzed according to the preset load scoring function to obtain the third factor score;

[0023] Based on preset weighting coefficients, the first factor score, the second factor score, and the third factor score are weighted and fused to obtain the comprehensive scheduling score of the subtask to be scheduled on the candidate accelerator card.

[0024] In one embodiment, the step of performing data locality analysis on the candidate accelerator card based on the input data required by the subtask to be scheduled, to obtain a second factor score, includes:

[0025] If the input data is stored in the candidate accelerator card, a first score is generated, and the first score is determined as the second factor score;

[0026] If the input data is stored on an adjacent accelerator card that has a point-to-point interconnection path with the candidate accelerator card, a second score is generated, and the second score is determined as the second factor score.

[0027] If the input data is stored in the server host, a third score is generated, and the third score is determined as the second factor score;

[0028] Wherein, the first score is greater than the second score, and the second score is greater than the third score.

[0029] In one embodiment, the method further includes:

[0030] Identify at least one successor subtask that has a data dependency relationship with the subtask to be scheduled;

[0031] For each of the successor subtasks, if the successor subtask meets the preset scheduling ready conditions, a second accelerator card matching the successor subtask is determined from the heterogeneous near-memory computing card pool.

[0032] If the second accelerator card and the first accelerator card are different accelerator cards, but there is a point-to-point interconnection path between the first accelerator card and the second accelerator card, then a direct memory access channel is established between the first accelerator card and the second accelerator card, so that the first accelerator card can transmit the task result corresponding to the scheduled subtask to the second accelerator card through the direct memory access channel.

[0033] In one embodiment, determining the scheduled subtask that satisfies the preset scheduling readiness condition from the plurality of subtasks includes:

[0034] Determine the data dependencies between the subtasks;

[0035] A task dependency graph is obtained by constructing an image based on the multiple sub-tasks and the data dependencies between them.

[0036] The task dependency graph identifies the subtasks that meet the preset scheduling readiness conditions.

[0037] In one embodiment, the heterogeneous near-memory computing card pool includes at least two types of accelerator cards; the method for constructing the preset operator-accelerator card mapping table includes:

[0038] Obtain the capability profile of each accelerator card in the heterogeneous near-memory computing card pool;

[0039] Based on the capability profiles of each accelerator card, determine the types of native operators supported by each accelerator card;

[0040] For each type of native operator, multiple operator-accelerator card key-value pairs are constructed, using the native operator type as the key and the identifiers of all accelerator cards that support the native operator type as the value.

[0041] A preset operator-accelerator card mapping table is constructed based on multiple operator-accelerator card key-value pairs.

[0042] Secondly, this application provides a heterogeneous scheduling system, which includes a server host and a heterogeneous near-memory computing card pool; the heterogeneous near-memory computing card pool includes at least two types of accelerator cards; the server host is communicatively connected to the heterogeneous near-memory computing card pool;

[0043] The server host is used to execute the task scheduling method described above.

[0044] Thirdly, this application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0045] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0046] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described above.

[0047] The aforementioned task scheduling method, heterogeneous scheduling system, and computer equipment divide the complex computing task to be processed into multiple subtasks through task splitting. For each subtask to be scheduled that meets the preset scheduling readiness conditions, an efficient query with O(1) time complexity can be achieved based on the preset operator-accelerator card mapping table to quickly obtain a set of candidate accelerator cards compatible with the subtask to be scheduled. Then, a scheduling decision analysis is performed on the set of candidate accelerator cards to select the first accelerator card that meets the preset optimal scheduling conditions. The subtask to be scheduled is assigned to the first accelerator card for execution, ensuring that the subtask to be scheduled runs on the accelerator card that is most suitable for it. This realizes the efficient collaboration of heterogeneous near-memory computing resources and the unified processing of hybrid computing tasks, effectively avoiding the problem of low task processing efficiency caused by cross-system data migration, and effectively improving the overall energy efficiency of the heterogeneous scheduling system. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a schematic diagram of the structure of a heterogeneous scheduling system in one embodiment;

[0050] Figure 2 This is a flowchart illustrating a task scheduling method in one embodiment;

[0051] Figure 3 This is a flowchart illustrating the steps for determining the candidate accelerator card set in one embodiment;

[0052] Figure 4 This is a flowchart illustrating the first accelerator card determination step in one embodiment;

[0053] Figure 5 This is a flowchart illustrating the comprehensive scheduling score calculation steps in one embodiment;

[0054] Figure 6 This is a flowchart illustrating the task scheduling method in another embodiment;

[0055] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0057] The task scheduling method provided in this application is applied to a server host in a heterogeneous scheduling system. The server host serves as the control and coordination center of the heterogeneous scheduling system. The server host is equipped with a high-performance central processing unit (CPU) and system memory. The heterogeneous scheduling system also includes a heterogeneous near-memory computing card pool. The heterogeneous near-memory computing card pool is connected to the server host via a PCIe 4.0 / 5.0 high-speed bus or the CXL (Compute Express Link) interconnect protocol.

[0058] The heterogeneous near-memory computing card pool includes at least two types of accelerator cards. In an exemplary embodiment, see... Figure 1 The heterogeneous near-memory computing card pool includes two complementary types of accelerator cards. The first type is for regularized computation, such as the UPMEM-based in-memory computing data processing unit (DPU) card. Its characteristic is that a large number of computing cores are distributed across memory granularities, making it extremely suitable for high-density regularized computation tasks such as cryptographic operations (e.g., symmetric encryption algorithms AES and asymmetric encryption algorithms RSA), data compression, regular expression matching, and large-scale sequence alignment. The second type is for non-regularized intelligent computation, such as the Lingxi-based in-memory computing AI accelerator card. It features circuit-level optimizations for matrix multiplication and convolution operations in neural networks, making it suitable for inference and training tasks in machine learning and deep learning models. These accelerator cards all have large amounts of local memory (such as HBM or GDDR), and the computing and storage units are tightly coupled, conforming to the "in-memory computing" characteristic of near-memory computing.

[0059] In other embodiments, in application scenarios with extremely high security requirements, the heterogeneous scheduling system can also connect to a TEE (Trusted Execution Environment) server cluster via a network or internal bus. The TEE server cluster is used to provide a hardware-level isolated trusted execution environment for specific computational subtasks involving sensitive privacy data.

[0060] In one embodiment, such as Figure 2 As shown, Figure 2 This is a flowchart illustrating a task scheduling method in one embodiment; the task scheduling method of this embodiment is applied to a server host in a heterogeneous scheduling system; the task scheduling method includes the following steps:

[0061] Step S201: Obtain multiple subtasks obtained after the composite computing task to be processed is split into tasks.

[0062] Among them, the composite computing task to be processed refers to the task submitted by the upper-layer application that contains both regular computing logic and irregular intelligent computing logic.

[0063] It should be noted that due to their complexity and diversity, complex computational tasks are often difficult for a single type of hardware to handle efficiently simultaneously. For example, in a smart pharmaceutical scenario, a client might submit an analysis request for a massive amount of protein sequences, which includes decrypting the original encrypted data (cryptographic regularization), searching for similarity between protein sequences (alignment regularization), and predicting protein function based on the search results (AI non-regularization). In a digital finance scenario, the task might involve decrypting homomorphically encrypted data and subsequently performing financial risk control model inference. When submitting a task, the client typically includes the path to the input data (or the data itself) and the expected output format. This application aims to achieve efficient scheduling and execution of complex computational tasks through a heterogeneous scheduling system, avoiding the inefficiency caused by cross-system data migration.

[0064] In an exemplary embodiment, obtaining multiple subtasks from a composite computing task to be processed after task splitting includes: receiving a composite computing task to be processed submitted by an upper-layer application; and splitting the composite computing task to be processed into multiple subtasks. The upper-layer application may, but is not limited to, submit requests for composite computing tasks to be processed through an SDK, RESTful API, or domain-specific language (DSL) provided by a heterogeneous scheduling system. The task splitting process may, but is not limited to, semantic analysis, structured parsing, and task segmentation.

[0065] In an exemplary embodiment, the server host includes a task parser, which is used to perform task decomposition processing on the complex computing task to be processed. For example, the task parser performs semantic recognition on the complex computing task to be processed, identifying the computational operators contained within the task. Subsequently, based on a pre-set rule base, the identified computational operators are categorized to obtain the operator type; for example, "AES_Decrypt" is categorized as a cryptographic operator, and "ResNet_Inference" is categorized as an AI operator. The pre-set rule base includes mapping rules for operator type identification; the pre-set rule base needs to be established based on the hardware architecture characteristics of the accelerator card, the computational semantic features of the operators, etc., and is not specifically limited here.

[0066] Based on the operator types obtained after classification, the task parser divides the complex computation task to be processed into multiple fine-grained sub-tasks. Furthermore, the task parser can analyze the data dependencies (i.e., input / output dependencies) between the sub-tasks and determine the execution order of each sub-task accordingly. For example, if the input of sub-task B depends on the output of sub-task A, a directed dependency edge from A to B is established between them. Finally, a task dependency graph in the form of a directed acyclic graph (DAG) is constructed in the server host memory.

[0067] The task dependency graph depicts the topological dependency structure between subtasks; subtasks are nodes in the graph; data dependencies between subtasks are directed edges, indicating that the output data of the predecessor subtask is consumed by the successor subtask. Each node is appended with metadata, which may include, but is not limited to, task type (normalized / irregularized computation), estimated computational cost (e.g., FLOPs or OPS), and required memory capacity, providing crucial information for task scheduling decisions.

[0068] Step S202: Determine the subtasks to be scheduled from multiple subtasks that meet the preset scheduling ready conditions.

[0069] Among them, the preset scheduling readiness conditions can be a set of logical rules to determine whether a subtask is qualified to be executed. This is used to ensure that a subtask is included in the scheduling candidate only if the data is complete and the dependencies are satisfied, thereby avoiding invalid scheduling and resource contention.

[0070] The preset scheduling readiness conditions may include, but are not limited to, one or more of the following: input data is in place, and preceding subtasks have been completed. For example, if all the preceding dependent subtasks of a certain subtask have been successfully completed and its own status has been set to "ready", then the subtask is determined as a scheduled subtask that meets the preset scheduling readiness conditions.

[0071] In one exemplary embodiment, the server host further includes a scheduler. The scheduler is used to efficiently schedule multiple subtasks. A method for determining the subtasks to be scheduled from the multiple subtasks that meet preset scheduling readiness conditions may be: obtaining a task dependency graph including multiple subtasks and the data dependencies between them; periodically traversing the subtask nodes in the task dependency graph through the scheduler, filtering out the subtasks to be scheduled that meet the preset scheduling readiness conditions, and adding them to the scheduling queue.

[0072] It should be noted that for each subtask to be scheduled, the scheduler does not simply allocate accelerator cards to it in a round-robin manner. Instead, it ensures that each subtask to be scheduled is scheduled to the most suitable dedicated accelerator card by executing steps S203 to S205, thereby achieving efficient collaboration of heterogeneous near-memory computing resources and optimization of global performance.

[0073] Step S203: For each subtask to be scheduled, based on the preset operator-accelerator card mapping table, determine the set of candidate accelerator cards that match the subtask to be scheduled from the heterogeneous near-memory computing card pool.

[0074] The preset operator-accelerator card mapping table is an inverted index table. It is built based on the operator support capabilities of various accelerator cards in the heterogeneous near-memory computing card pool. It is used to record the correspondence between different native operator types and all accelerator cards that support each type of native operator.

[0075] The pre-defined operator-accelerator card mapping table includes multiple operator-accelerator card key-value pairs. The key of each key-value pair is the native operator type, and the value is an identifier for all accelerator cards supporting each native operator type. The native operator type refers to the category of operators supported by the accelerator card; the accelerator card identifier is used to uniquely identify each accelerator card in the heterogeneous near-memory computing pool, ensuring that each accelerator card can be accurately addressed and managed during task scheduling.

[0076] Understandably, based on the preset operator-accelerator card mapping table, efficient lookup with O(1) time complexity can be achieved: directly based on the operator type of the subtask to be scheduled, the set of candidate accelerator cards that support the operator type can be quickly located without traversing all hardware resources in the heterogeneous near-memory computing pool, thereby effectively improving task scheduling efficiency and system response speed.

[0077] The candidate accelerator card set includes at least one candidate accelerator card; a candidate accelerator card refers to an accelerator card that supports the subtask to be scheduled; it can be understood that the candidate accelerator card set is used to provide an effective range of candidate resources for subsequent optimal scheduling decisions; that is, the candidate accelerator cards in the candidate accelerator card set only represent a preliminary range of candidates, and whether they are ultimately used to execute the subtask to be scheduled still needs to be further evaluated and screened through scheduling decision analysis to ensure that the subtask to be scheduled is ultimately scheduled to the accelerator card that is most suitable for it and has the best overall performance.

[0078] Step S204: Perform scheduling decision analysis on the candidate accelerator card set to select the first accelerator card that meets the preset optimal scheduling conditions from the candidate accelerator card set.

[0079] The scheduling decision analysis refers to the comprehensive quantitative evaluation of each candidate accelerator card in the candidate accelerator card set from multiple evaluation dimensions, in order to select the first accelerator card that meets the preset optimal scheduling conditions from the candidate accelerator card set; the evaluation dimensions may include, but are not limited to, hardware compatibility, data locality, real-time load status, etc.

[0080] Among them, the preset optimal scheduling conditions refer to the pre-set evaluation criteria used to determine whether a candidate accelerator card is the optimal execution carrier for the current subtask to be scheduled.

[0081] Understandably, by performing scheduling decision analysis on the candidate accelerator card set, it is possible to effectively avoid assigning scheduled subtasks to accelerator cards that are already overloaded or require expensive data transfer, thereby achieving the optimization of the overall system performance.

[0082] Step S205: Assign the subtask to be scheduled to the first acceleration card so that the first acceleration card can execute the subtask to be scheduled and generate the corresponding task result.

[0083] For example, the server host in the heterogeneous scheduling system receives the composite computing tasks submitted by the upper-layer application and splits them into multiple subtasks. From these subtasks, the scheduled subtasks that meet preset scheduling readiness conditions are identified. For each scheduled subtask, a set of candidate accelerator cards matching the scheduled subtask is determined from the heterogeneous near-memory computing card pool based on a preset operator-accelerator card mapping table. Scheduling decision analysis is performed on the candidate accelerator card set to select the first accelerator card that meets preset optimal scheduling conditions. The scheduled subtask is assigned to the first accelerator card, enabling it to execute the scheduled subtask and generate the corresponding task results. Simultaneously, during task execution, the execution progress of the composite computing tasks is monitored in real time until all subtasks are completed. Then, the final task results, dispersed in the local memory of each accelerator card, are retrieved back to the server host's system memory, and necessary data format conversions or concatenations are performed. Finally, the complete processing results are returned to the client application via API callbacks or file writing, completing a full-process hybrid computing service.

[0084] In this embodiment, the composite computing task to be processed is divided into multiple subtasks through task splitting. For each subtask to be scheduled that meets the preset scheduling readiness conditions, an efficient query with O(1) time complexity can be achieved based on the preset operator-accelerator card mapping table to quickly obtain a set of candidate accelerator cards compatible with the subtask to be scheduled. Then, a scheduling decision analysis is performed on the set of candidate accelerator cards to select the first accelerator card that meets the preset optimal scheduling conditions. The subtask to be scheduled is assigned to the first accelerator card for execution to ensure that the subtask to be scheduled runs on the accelerator card that is most suitable for it. This realizes the efficient collaboration of heterogeneous near-memory computing resources and the unified processing of hybrid computing tasks, effectively avoiding the problem of low task processing efficiency caused by cross-system data migration, and effectively improving the overall energy efficiency of the heterogeneous scheduling system.

[0085] In one embodiment, such as Figure 3 As shown, Figure 3 This is a flowchart illustrating the steps for determining the candidate accelerator card set in one embodiment. Based on a preset operator-accelerator card mapping table, a candidate accelerator card set matching the subtask to be scheduled is determined from a heterogeneous near-memory computing card pool, including the following steps:

[0086] Step S301: Obtain the preset operator-accelerator card mapping table.

[0087] The pre-defined operator-accelerator card mapping table is represented by an inverted index. This table includes multiple operator-accelerator card key-value pairs; the key of each pair is the native operator type, and the value is an identifier for all accelerator cards supporting each native operator type. The native operator type refers to the category of operators supported by the accelerator card; the accelerator card identifier uniquely identifies each accelerator card in the heterogeneous near-memory computing pool, ensuring that each accelerator card can be accurately addressed and managed during task scheduling.

[0088] In an exemplary embodiment, the method for constructing a preset operator-accelerator card mapping table includes the following steps:

[0089] Step 1: Obtain the capability profiles of each accelerator card in the heterogeneous near-memory computing card pool.

[0090] Among them, the capability profile is a structured description file (such as JSON format) used to define the hardware attributes and software stack capabilities of the corresponding accelerator card.

[0091] For example, the capability profile of the first type of accelerator card (such as the UPMEM card) would declare its proficiency in "logic operations," "bit operations," and "integer arithmetic," and list the supported algorithm libraries (AES-GCM, SHA-256, Blast-P, etc.); the capability profile of the second type of accelerator card (such as the Lingxi card) would declare its proficiency in "tensor operations" and "floating-point calculations," and list the supported model frameworks (TensorFlow, PyTorch, etc.). In addition, the capability profile may also include physical parameters, such as local memory capacity (e.g., 16GB), peak memory bandwidth (e.g., 1TB / s), core frequency, and power consumption.

[0092] It should be noted that the heterogeneous scheduling system of this application has fine-grained awareness of the underlying heterogeneous hardware resources. During the system initialization phase or when new hardware is hot-swapped, the drivers of various accelerator cards (i.e., near-memory computing accelerator cards) register their capability profiles with the server host in the heterogeneous scheduling system, laying the foundation for task scheduling.

[0093] In an exemplary embodiment, after the heterogeneous scheduling system starts, the device management daemon is loaded. During loading, the drivers for each heterogeneous accelerator card call the `register_device_capability()` interface provided by the server host. The driver uploads a description file containing device metadata. This file includes not only static hardware parameters (such as vendor ID and memory size) but also dynamic runtime information (such as driver version and supported operator library versions). The server host performs validity checks and parsing on the received description file, converting the unstructured description into an internal unified data model. This data model contains several key fields, which may include, but are not limited to: accelerator card type (`device_type`): used to identify the device category, such as a cryptographic card or an AI card; supported operation primitives (`supported_ops`): listing the types of operators natively supported by the accelerator card; and performance metrics (`performance_metrics`): recording measured performance data under typical workloads. For example, for a UPMEM card, its profile records its throughput under the AES-256 algorithm as XGbps; for a Lingxi card, its FPS (frames per second) for ResNet-50 inference is recorded as Y.

[0094] Step 2: Determine the native operator types supported by each accelerator card based on its corresponding capability profile.

[0095] Among them, native operator type refers to the type of operator supported by the accelerator card.

[0096] Step 3: For each type of native operator, construct multiple operator-accelerator card key-value pairs, using the native operator type as the key and the identifiers of all accelerator cards that support the native operator type as the values.

[0097] The identifier of the accelerator card is used to uniquely identify each accelerator card in the heterogeneous near-memory computing pool, ensuring that each accelerator card can be accurately addressed and managed during task scheduling.

[0098] Step 4: Construct a preset operator-accelerator card mapping table based on multiple operator-accelerator card key-value pairs.

[0099] Step S302: Identify the operator type of the subtask to be scheduled.

[0100] The operator type of the subtask to be scheduled can be, but is not limited to, determined by category matching based on a pre-defined rule base. The pre-defined rule base includes mapping rules for operator type identification; it needs to be established based on the hardware architecture characteristics of the accelerator card, the computational semantic features of the operators, etc., but is not specifically limited here.

[0101] In an exemplary embodiment, the method for identifying the operator type of the subtask to be scheduled may be: identifying the computational operators contained in the subtask to be scheduled; and classifying the identified computational operators based on a preset rule base to obtain the operator type of the subtask to be scheduled. For example, "AES_Decrypt" may be classified as a cryptographic operator, and "ResNet_Inference" may be classified as an AI operator.

[0102] Step S303: Based on the operator type of the subtask to be scheduled, at least one candidate accelerator card that matches the operator type of the subtask to be scheduled is retrieved from the preset operator-accelerator card mapping table.

[0103] Among them, candidate accelerator cards refer to accelerator cards that support the subtasks to be scheduled.

[0104] Step S304: Determine at least one candidate accelerator card as a set of candidate accelerator cards that match the subtask to be scheduled.

[0105] In this embodiment, based on the preset operator-accelerator card mapping table, the system can achieve efficient query with O(1) time complexity: directly locate the set of candidate accelerator cards that support the operator type according to the operator type of the subtask to be scheduled, without having to traverse all hardware resources in the heterogeneous near-memory computing pool, thereby effectively improving task scheduling efficiency and system response speed.

[0106] In one embodiment, such as Figure 4 As shown, Figure 4This is a flowchart illustrating the first accelerator card determination step in one embodiment; the scheduling decision analysis of the candidate accelerator card set is performed to select the first accelerator card that meets the preset optimal scheduling conditions, including the following steps:

[0107] Step S401: Calculate the comprehensive scheduling score of the subtask to be scheduled on each candidate accelerator card.

[0108] The comprehensive scheduling score is used to quantify the overall suitability of candidate accelerator cards for executing scheduled subtasks. The value of the comprehensive scheduling score is calculated based on a weighted average of multiple evaluation dimensions. These evaluation dimensions may include, but are not limited to, hardware suitability, data locality, and real-time load status.

[0109] In an exemplary embodiment, calculating the comprehensive scheduling score of the subtask to be scheduled on each candidate accelerator card includes: if there are unavailable candidate accelerator cards in the candidate accelerator card set, filtering out the unavailable candidate accelerator cards; for each available candidate accelerator card in the filtered candidate accelerator card set, calculating the comprehensive scheduling score of the subtask to be scheduled on each available candidate accelerator card; if there are no unavailable candidate accelerator cards in the candidate accelerator card set, calculating the comprehensive scheduling score of the subtask to be scheduled on each candidate accelerator card.

[0110] Among them, unavailable candidate accelerator cards refer to candidate accelerator cards in the candidate accelerator card set that are offline, faulty, or under maintenance.

[0111] Step S402: The candidate accelerator card with the highest comprehensive scheduling score is determined as the first accelerator card that meets the preset optimal scheduling conditions.

[0112] In an exemplary embodiment, if there are unavailable candidate accelerator cards in the candidate accelerator card set, these unavailable candidate accelerator cards are filtered out. The comprehensive scheduling score of the subtask to be scheduled on each available candidate accelerator card is calculated, and the comprehensive scheduling scores of the subtask to be scheduled on each available candidate accelerator card are ranked. The available candidate accelerator card with the highest comprehensive scheduling score is determined as the first accelerator card that meets the preset optimal scheduling conditions. If the candidate accelerator card with the highest comprehensive scheduling score is suddenly unable to receive the subtask to be scheduled, the second-best candidate accelerator card is selected as the first accelerator card.

[0113] It should be noted that once the mapping relationship between the scheduled subtask and the first accelerator card is determined, the server host immediately generates a task distribution instruction and sends it to the command queue of the first accelerator card to trigger the first accelerator card to execute the scheduled subtask. This task distribution instruction may include, but is not limited to, code pointers, data addresses, and startup parameters.

[0114] In this embodiment, by calculating the comprehensive scheduling score of the subtask to be scheduled on each candidate accelerator card, the candidate accelerator card with the highest comprehensive scheduling score is determined as the first accelerator card that meets the preset optimal scheduling conditions. This ensures that the subtask to be scheduled is scheduled to the accelerator card that is most suitable for it and has the best comprehensive performance, thereby improving the overall throughput of the system, reducing task execution latency, and effectively avoiding resource contention and inefficient allocation.

[0115] In one embodiment, such as Figure 5 As shown, Figure 5 This is a flowchart illustrating the comprehensive scheduling score calculation steps in one embodiment; calculating the comprehensive scheduling score of the subtask to be scheduled on each candidate accelerator card includes the following steps:

[0116] Step S501: For each candidate accelerator card, perform a matching degree analysis between the operator type of the subtask to be scheduled and the computing power of the candidate accelerator card to obtain the first factor score.

[0117] The first factor score is a normalized value; the value of the first factor score ranges from 0 to 1.

[0118] In an exemplary embodiment, if the candidate accelerator card natively supports the operator type of the scheduled subtask at the hardware level (e.g., the UPMEM near-memory computing card performs AES encryption and decryption cryptographic tasks), the first factor score is set to close to 1, indicating a high degree of fit; if the candidate accelerator card only performs the operator type of the scheduled subtask through a general-purpose computing unit (e.g., the CPU performs AI inference tasks in software), the first factor score is set to a lower value to indicate a weaker match.

[0119] Step S502: Based on the input data required by the subtask to be scheduled, perform data locality analysis on the candidate accelerator cards to obtain the second factor score.

[0120] The second factor score is a binary or continuous variable.

[0121] In an exemplary embodiment, step S502, which involves performing data locality analysis on candidate accelerator cards based on the input data required by the subtask to be scheduled, to obtain a second factor score, includes the following steps:

[0122] Step 1: If the input data is stored in the candidate accelerator card, a first score is generated, and the first score is determined as the second factor score.

[0123] Step 2: If the input data is stored on an adjacent accelerator card that has a point-to-point interconnection path with the candidate accelerator card, a second score is generated and determined as the second factor score.

[0124] Among them, the adjacent accelerator card refers to the accelerator card that is directly connected to the candidate accelerator card in physical topology; the point-to-point interconnection path is used to realize high-speed, low-latency direct data transmission between two accelerator cards, which can bypass the server host memory and directly complete cross-accelerator card data access, thereby significantly reducing unnecessary data transfer overhead.

[0125] Step 3: If the input data is stored on the server host, a third score is generated and determined as the second factor score.

[0126] The first score is higher than the second score, and the second score is higher than the third score. This means that by setting differentiated first, second, and third scores, the local quality levels at different data locations can be accurately reflected.

[0127] Understandably, by performing data locality analysis on candidate accelerator cards, the system can quantify the data transfer cost of each candidate accelerator card and assign a higher second factor score to the candidate accelerator card with the lowest data migration overhead, thereby effectively reducing communication latency, improving bandwidth utilization, and achieving high-efficiency scheduling.

[0128] Step S503: Quantitatively analyze the current load status of the candidate accelerator cards according to the preset load scoring function to obtain the third factor score.

[0129] The preset load scoring function is used to map the current load status of the candidate accelerator card to a numerical value. The input to the preset load scoring function may include, but is not limited to, at least one of the following: the current task queue length of the candidate accelerator card and the video memory usage rate.

[0130] The third factor score characterizes the current load status (i.e., current busyness) of the candidate accelerator card. A higher third factor score indicates a higher current load (i.e., a busier candidate accelerator card). Conversely, a lower third factor score indicates a lower current load (i.e., a less busy candidate accelerator card).

[0131] It should be noted that the first and second factor scores are used as positive factors in the weighted fusion calculation, while the third factor score is used as a negative factor. That is, the higher the third factor score, the more points are deducted during the weighted fusion calculation to prevent congestion of hotspot devices.

[0132] Step S504: Based on the preset weight coefficients, the first factor score, the second factor score, and the third factor score are weighted and fused to obtain the comprehensive scheduling score of the subtask to be scheduled on the candidate acceleration card.

[0133] The preset weight coefficients include a first weight coefficient, a second weight coefficient, and a third weight coefficient; wherein the first weight coefficient corresponds to the first factor score, the second weight coefficient corresponds to the second factor score, and the third weight coefficient corresponds to the third factor score.

[0134] It should be noted that the preset weight coefficient is a configurable hyperparameter and needs to be dynamically adjusted according to the actual scheduling strategy (such as "performance priority" or "energy saving priority"). No specific restrictions are made here.

[0135] In an exemplary embodiment, the first factor score is denoted as Match(Ti, Cj), the second factor score as Locality(Ti, Cj), and the third factor score as Load(Cj); where Cj is the candidate accelerator card and Ti is the task to be scheduled. Assuming the first weight coefficient is w1, the second weight coefficient is w2, and the third weight coefficient is w3, a weighted fusion calculation is performed on the first factor score Match(Ti, Cj), the second factor score Locality(Ti, Cj), and the third factor score Load(Cj) to obtain the comprehensive scheduling score Score(Ti, Cj) of the subtask to be scheduled on the candidate accelerator card:

[0136] .

[0137] In this embodiment, the multi-dimensional collaborative scheduling and scoring mechanism can comprehensively consider three key factors: hardware adaptability, data locality, and real-time load status. This enables refined quantification of the matching quality between the subtasks to be scheduled and the candidate accelerator cards, improves the global optimality of scheduling decisions, and achieves efficient, intelligent, and adaptive task scheduling in a heterogeneous near-memory computing environment.

[0138] In one embodiment, such as Figure 6 As shown, Figure 6 This is a flowchart illustrating a task scheduling method in another embodiment. The task scheduling method further includes the following steps:

[0139] Step S601: Determine at least one successor subtask that has a data dependency relationship with the subtask to be scheduled.

[0140] Among them, the successor subtask refers to a subtask in the execution process of the composite computing task to be processed, in which the input data is partly or entirely derived from the task results corresponding to the subtask to be scheduled.

[0141] In an exemplary embodiment, the method for determining at least one successor subtask that has a data dependency relationship with the subtask to be scheduled may be: obtaining a task dependency graph constructed based on the composite computing task to be processed, and determining at least one successor subtask that has a data dependency relationship with the subtask to be scheduled from the task dependency graph; it is understood that the task dependency graph includes multiple subtasks and the data dependencies between each subtask.

[0142] Step S602: For each successor subtask, if the successor subtask meets the preset scheduling ready conditions, then determine the second accelerator card that matches the successor subtask from the heterogeneous near-memory computing card pool.

[0143] It is understandable that during task execution, there is a definite execution order among the subtasks; only when a subtask meets the preset scheduling ready conditions can it be formally scheduled for execution.

[0144] Among them, the preset scheduling readiness conditions can be a set of logical rules to determine whether a subtask is qualified to be executed. This is used to ensure that a subtask is included in the scheduling candidate only if the data is complete and the dependencies are satisfied, thereby avoiding invalid scheduling and resource contention.

[0145] The preset scheduling readiness conditions may include, but are not limited to, one or more of the following: input data is in place, preceding subtasks are completed. For example, if all preceding dependent subtasks of a certain successor subtask have been successfully completed and its own status has been set to "ready", then the successor subtask is determined to meet the preset scheduling readiness conditions.

[0146] It should be noted that the method for determining the second accelerator card that matches the subsequent subtask is the same as the method for determining the first accelerator card that matches the subtask to be scheduled.

[0147] In an exemplary embodiment, if the successor subtask meets the preset scheduling readiness conditions, a set of target candidate accelerator cards matching the successor subtask is determined from the heterogeneous near-memory computing card pool based on a preset operator-accelerator card mapping table. Scheduling decision analysis is then performed on the target candidate accelerator card set to select a second accelerator card that meets the preset optimal scheduling conditions. It should be noted that the specific limitations of the method for determining the target candidate accelerator card set and the scheduling decision analysis method in this embodiment can be found in the limitations of the method for determining the candidate accelerator card set and the scheduling decision analysis method in the above embodiments, and will not be repeated here.

[0148] Step S603: If the second accelerator card and the first accelerator card are different accelerator cards, but there is a point-to-point interconnection path between the first accelerator card and the second accelerator card, then a direct memory access channel is established between the first accelerator card and the second accelerator card, so that the first accelerator card can transmit the task result corresponding to the subtask to be scheduled to the second accelerator card through the direct memory access channel.

[0149] In this context, the second accelerator card and the first accelerator card are different accelerator cards, meaning they are two physically independent acceleration devices. In this situation, since the preceding subtask (i.e., the subtask to be scheduled) and the succeeding subtask are executed by different accelerator cards, cross-device data transfer is inevitable. To reduce the latency and bandwidth overhead caused by relaying data through the server host memory and to improve data transmission efficiency, this application proposes to utilize existing point-to-point interconnection paths to construct a direct memory access channel, achieving efficient and low-overhead transmission of task results.

[0150] In an exemplary embodiment, for subtasks with a successor-successor relationship (i.e., a data dependency relationship) (e.g., subtask A generates data, subtask B consumes data), if subtask A and subtask B are scheduled to different physical accelerator cards (e.g., subtask A is scheduled to accelerator card A, subtask B is scheduled to accelerator card B), the server host will first check whether there is a hardware-level peer-to-peer (P2P) interconnection path between the two accelerator cards (e.g., direct routing via PCIeSwitch). If it exists, a Direct Memory Access (DMA) channel will be established to directly move data from the video memory of accelerator card A to the video memory of accelerator card B, completely bypassing the server host CPU and system memory, significantly reducing latency and bus bandwidth usage.

[0151] If P2P transmission is not supported between accelerator card A and accelerator card B (e.g., two accelerator cards across NUMA nodes, or hardware not supporting it), data is written back from the video memory of accelerator card A to a temporary buffer in the server host system memory, and then the data is sent from the server host system memory to the video memory of accelerator card B. This embodiment effectively reduces bus contention and CPU interrupt overhead by prioritizing the P2P path.

[0152] For example, a task dependency graph constructed based on the composite computing task to be processed is obtained. During the execution of the task dependency graph, when the scheduled subtask Pre completes and generates output data D, and its successor subtask Next is about to start, the system performs the following operations:

[0153] First, it is identified that data D is currently stored in the first accelerator card (Addr_Src); at the same time, it is identified that the successor subtask Next is scheduled to the second accelerator card (Dev_Dst), and the first accelerator card (Addr_Src) and the second accelerator card (Dev_Dst) are two physically independent accelerator cards.

[0154] In this scenario, the system further checks whether a point-to-point interconnection path exists between the first accelerator card (Addr_Src) and the second accelerator card (Dev_Dst). Specifically, it determines whether the first accelerator card (Addr_Src) and the second accelerator card (Dev_Dst) are connected to the same PCIe Root Complex, or whether ACS (Access Control Services) is enabled to support P2P transmission.

[0155] If P2P transmission is supported, a Direct Memory Access Channel (DMA) is configured on the first accelerator card (Addr_Src) and the second accelerator card (Dev_Dst) to establish a direct data transmission link. Data D will be transmitted directly from the first accelerator card (Addr_Src) to the second accelerator card (Dev_Dst) via a PCIe switch, without passing through the server host memory, thus achieving low-latency, high-bandwidth data transmission.

[0156] If P2P transmission is not supported (e.g., the two accelerator cards are located on different NUMA nodes, or the P2P function is not enabled at the hardware / driver level), a "write-back-retransmit" strategy is adopted: first, the data D is written back from the first accelerator card (Addr_Src) to a temporary buffer in the server host system memory; then, the data D is sent from the buffer to the second accelerator card (Dev_Dst).

[0157] In this embodiment, after the scheduled subtask is completed, the successor subtask with a data dependency relationship with the scheduled subtask is determined from the task dependency graph. When the successor subtask meets the scheduling ready condition, the optimal second accelerator card is matched for it from the heterogeneous near-memory computing card pool. When the second accelerator card and the first accelerator card executing the scheduled task are different accelerator cards but have a point-to-point interconnection path, the system automatically establishes a direct memory access channel, so that the task result of the scheduled subtask can be efficiently transmitted without passing through the server host memory, reducing unnecessary data transfer, improving bandwidth utilization and reducing server host resource overhead.

[0158] In one embodiment, determining the scheduleable subtasks that meet preset scheduling readiness conditions from a plurality of subtasks includes the following steps:

[0159] Step 1: Determine the data dependencies between the subtasks.

[0160] Step 2: Construct a graph based on multiple subtasks and the data dependencies between them to obtain a task dependency graph.

[0161] The task dependency graph depicts the topological dependency structure between subtasks; subtasks are nodes in the graph; data dependencies between subtasks are directed edges, indicating that the output data of the predecessor subtask is consumed by the successor subtask. Each node is appended with metadata, which may include, but is not limited to, task type (normalized / irregularized computation), estimated computational cost (e.g., FLOPs or OPS), and required memory capacity, providing crucial information for task scheduling decisions.

[0162] Step 3: Identify the scheduled subtasks that meet the preset scheduling readiness conditions from the task dependency graph.

[0163] For example, the task parser in the server host performs semantic analysis on each subtask, identifies the input / output data dependencies between them, and determines the execution order of the subtasks accordingly: if the input data of subtask B depends on the output of subtask A, a directed dependency edge from A to B is established between the two. Based on all such dependencies, a task dependency graph in the form of a directed acyclic graph is constructed in the memory of the server host. Subsequently, the scheduler in the server host traverses the task dependency graph and selects subtasks that meet preset scheduling readiness conditions (such as all predecessor subtasks have been completed and the required resources are available) as subtasks to be scheduled.

[0164] In this embodiment, by parsing the data dependencies between multiple subtasks after the composite computing task to be processed is split into tasks, a task dependency graph is constructed accordingly. Based on the task dependency graph, the scheduled subtasks that meet the preset scheduling readiness conditions are determined, which avoids task blocking or failure caused by dependency conflicts, improves the overall task execution efficiency, and ensures that each subtask can be executed efficiently in the optimal order.

[0165] In one embodiment, a heterogeneous scheduling system is provided; the heterogeneous scheduling system includes a server host and a heterogeneous near-memory computing card pool; the heterogeneous near-memory computing card pool includes at least two types of acceleration cards; the server host and the heterogeneous near-memory computing card pool are communicatively connected.

[0166] The server host is used to execute the task scheduling method described in any of the above embodiments.

[0167] The heterogeneous near-memory computing card pool includes at least two types of accelerator cards. For example, the heterogeneous near-memory computing card pool includes a first type of accelerator card and a second type of accelerator card with complementary capabilities; the first type of accelerator card is used for regularized computing, and the second type of accelerator card is used for irregularized intelligent computing.

[0168] The heterogeneous near-memory computing card pool is connected to the server host via a PCIe 4.0 / 5.0 high-speed bus or the CXL (Compute ExpressLink) interconnect protocol.

[0169] The server host may include, but is not limited to, a task parser, scheduler, capability profile registration module, resource monitoring module, data collaboration module, and task executor.

[0170] The task parser is used to receive and parse the complex computation tasks submitted by the client. It incorporates a parsing engine for common languages ​​such as Python and C++, enabling it to identify the computational kernel in the code and construct a task dependency graph based on data flow dependencies.

[0171] The scheduler periodically scans the subtasks to be scheduled that meet the preset scheduling readiness conditions, and determines a set of candidate accelerator cards that match the subtasks to be scheduled from the heterogeneous near-memory computing card pool based on the preset operator-accelerator card mapping table; performs scheduling decision analysis on the candidate accelerator card set to select the first accelerator card that meets the preset optimal scheduling conditions; and assigns the subtasks to be scheduled to the first accelerator card so that the first accelerator card executes the subtasks to be scheduled and generates the corresponding task results.

[0172] The capability profile registration module is used to maintain the system's hardware resource view. It provides a standard registration interface for drivers of heterogeneous accelerator cards such as UPMEM and Lingxi to report their computing power characteristics, memory specifications, and interconnection topology information, and maintains a real-time capability database in memory.

[0173] The resource monitoring module is used to monitor the system status in real time. Through out-of-band management (BMC) or in-band driver interfaces, the resource monitoring module collects data on the utilization rate, memory usage, temperature, and power consumption of each accelerator card at millisecond granularity, providing data support for the scheduler's load balancing decisions.

[0174] The data collaboration module manages data flow. It maintains a global data address mapping table, intelligently plans data transmission paths, prioritizes P2P direct connections between accelerator cards, and manages data caching and eviction policies.

[0175] The task executor is used to shield the differences in underlying hardware. It encapsulates a unified execution interface for different types of accelerator cards, and is responsible for loading the compiled binary kernel or model file onto the accelerator card, as well as controlling the start, suspension, and termination of tasks.

[0176] This heterogeneous scheduling system achieves efficient collaboration of heterogeneous near-memory computing resources and unified processing of hybrid computing tasks, effectively avoiding the problem of low task processing efficiency caused by cross-system data migration and significantly improving the overall energy efficiency of the heterogeneous scheduling system. Simultaneously, by utilizing point-to-point (P2P) direct connections between accelerator cards to achieve cross-device data transmission, it bypasses server host memory relay, significantly reducing unnecessary data movement. This further optimizes task execution efficiency and energy efficiency while reducing bandwidth consumption and energy usage. Furthermore, this heterogeneous scheduling system provides a unified development view and excellent versatility. Application developers can efficiently develop hybrid computing applications without delving into the underlying hardware details. The system can flexibly adapt to the complex computing needs of various scenarios such as intelligent pharmaceuticals and digital finance.

[0177] In other embodiments, the heterogeneous scheduling system further includes a TEE server cluster connected to a server host; the task scheduling method further includes the following steps: when parsing the composite computing task to be processed, identifying secure computing subtasks involving sensitive privacy data in the composite computing task to be processed; scheduling the secure computing subtasks to be executed in a trusted execution environment provided by the TEE server cluster; and after the TEE server cluster completes execution, returning the encrypted result data to the heterogeneous near-memory computing card pool for subsequent processing.

[0178] In a specific embodiment, taking a smart pharmaceutical application scenario as an example, assuming the complex computation task to be processed is a structure prediction task based on encrypted protein data, the task scheduling method includes the following steps:

[0179] The user uploads a 100GB AES-encrypted protein sequence database to the client, requesting decryption, similarity alignment (BLAST), and AlphaFold structure prediction for highly similar sequences. The task parser in the server host receives this complex computation task and decomposes it into T1 (AES decryption), T2 (BLAST alignment), and T3 (structure prediction). The scheduler in the server host, through task scheduling methods, identifies T1 and T2 as regular computations, and notes that the UPMEM card has extremely high energy efficiency in such tasks (high first factor score Match score), therefore scheduling T1 and T2 to the UPMEM DPU card cluster in the system; it also identifies T3 as an AI heavy computation task, and notes that the Lingxi AI card has the strongest computing power in such tasks, therefore scheduling T3 to the Lingxi AI card.

[0180] Encrypted data is loaded onto the UPMEM card. The plaintext data decrypted by T1 remains directly in the UPMEM card's memory for use by T2 (high Locality score in the second factor evaluation). A small number of candidate sequences selected by T2 are transferred directly from the UPMEM card to the Lingxi card's HBM memory via PCIe P2P technology, without CPU involvement in data transfer. The Lingxi card reads the data and executes T3 to complete structure prediction. This task scheduling process fully utilizes the high-concurrency integer processing capabilities of UPMEM and the AI ​​floating-point capabilities of Lingxi, and eliminates over 90% of redundant data transfer through P2P technology, achieving several times the energy efficiency improvement compared to traditional GPU solutions.

[0181] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0182] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores task scheduling-related data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a task scheduling method.

[0183] Those skilled in the art will understand that Figure 7The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0184] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0185] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0186] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0187] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0188] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0189] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0190] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A task scheduling method, characterized in that, Server hosts used in heterogeneous scheduling systems; The heterogeneous scheduling system also includes a heterogeneous near-memory computing card pool; the method includes: Obtain multiple subtasks obtained after the composite computing task to be processed has been split into tasks; From the multiple subtasks, determine the subtasks to be scheduled that meet the preset scheduling ready conditions; For each of the scheduled subtasks, a set of candidate accelerator cards matching the scheduled subtasks is determined from the heterogeneous near-memory computing card pool based on a preset operator-accelerator card mapping table. The candidate accelerator card set is subjected to scheduling decision analysis in order to select the first accelerator card that meets the preset optimal scheduling conditions from the candidate accelerator card set; The scheduled subtask is assigned to the first acceleration card so that the first acceleration card executes the scheduled subtask and generates the corresponding task result.

2. The method according to claim 1, characterized in that, The set of candidate accelerator cards matching the subtask to be scheduled is determined from the heterogeneous near-memory computing card pool based on the preset operator-accelerator card mapping table, including: Obtain the preset operator-accelerator card mapping table; Identify the operator type of the subtask to be scheduled; Based on the operator type of the subtask to be scheduled, at least one candidate accelerator card that matches the operator type of the subtask to be scheduled is retrieved from the preset operator-accelerator card mapping table. At least one of the candidate accelerator cards is determined as a set of candidate accelerator cards that match the scheduled subtask.

3. The method according to claim 1, characterized in that, The candidate accelerator card set includes at least one candidate accelerator card; the step of performing scheduling decision analysis on the candidate accelerator card set to select a first accelerator card that meets preset optimal scheduling conditions includes: Calculate the overall scheduling score of the subtask to be scheduled on each of the candidate accelerator cards; The candidate accelerator card with the highest comprehensive scheduling score is determined as the first accelerator card that meets the preset optimal scheduling conditions.

4. The method according to claim 3, characterized in that, The calculation of the comprehensive scheduling score of the subtask to be scheduled on each of the candidate accelerator cards includes: For each candidate accelerator card, a matching degree analysis is performed on the operator type of the subtask to be scheduled and the computing power of the candidate accelerator card to obtain a first factor score; Based on the input data required by the subtask to be scheduled, data locality analysis is performed on the candidate accelerator cards to obtain a second factor score. The current load status of the candidate accelerator cards is quantitatively analyzed according to the preset load scoring function to obtain the third factor score; Based on preset weighting coefficients, the first factor score, the second factor score, and the third factor score are weighted and fused to obtain the comprehensive scheduling score of the subtask to be scheduled on the candidate accelerator card.

5. The method according to claim 4, characterized in that, The step of performing data locality analysis on the candidate accelerator cards based on the input data required by the subtask to be scheduled, and obtaining a second factor score, includes: If the input data is stored in the candidate accelerator card, a first score is generated, and the first score is determined as the second factor score; If the input data is stored on an adjacent accelerator card that has a point-to-point interconnection path with the candidate accelerator card, a second score is generated, and the second score is determined as the second factor score. If the input data is stored in the server host, a third score is generated, and the third score is determined as the second factor score; Wherein, the first score is greater than the second score, and the second score is greater than the third score.

6. The method according to claim 1, characterized in that, The method further includes: Identify at least one successor subtask that has a data dependency relationship with the subtask to be scheduled; For each of the successor subtasks, if the successor subtask meets the preset scheduling ready conditions, a second accelerator card matching the successor subtask is determined from the heterogeneous near-memory computing card pool. If the second accelerator card and the first accelerator card are different accelerator cards, but there is a point-to-point interconnection path between the first accelerator card and the second accelerator card, then a direct memory access channel is established between the first accelerator card and the second accelerator card, so that the first accelerator card can transmit the task result corresponding to the scheduled subtask to the second accelerator card through the direct memory access channel.

7. The method according to any one of claims 1 to 6, characterized in that, The step of determining the subtask to be scheduled from the plurality of subtasks that meets the preset scheduling readiness conditions includes: Determine the data dependencies between the subtasks; A task dependency graph is obtained by constructing an image based on the multiple sub-tasks and the data dependencies between them. The task dependency graph identifies the subtasks that meet the preset scheduling readiness conditions.

8. The method according to any one of claims 1 to 6, characterized in that, The heterogeneous near-memory computing card pool includes at least two types of accelerator cards; the method for constructing the preset operator-accelerator card mapping table includes: Obtain the capability profile of each accelerator card in the heterogeneous near-memory computing card pool; Based on the capability profiles of each accelerator card, determine the types of native operators supported by each accelerator card; For each type of native operator, multiple operator-accelerator card key-value pairs are constructed, using the native operator type as the key and the identifiers of all accelerator cards that support the native operator type as the value. A preset operator-accelerator card mapping table is constructed based on multiple operator-accelerator card key-value pairs.

9. A heterogeneous scheduling system, characterized in that, The heterogeneous scheduling system includes a server host and a heterogeneous near-memory computing card pool; the heterogeneous near-memory computing card pool includes at least two types of acceleration cards; the server host is communicatively connected to the heterogeneous near-memory computing card pool; The server host is used to execute the task scheduling method according to any one of claims 1 to 8.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

Images