A task processing method and device applied to a heterogeneous computing power system

Abstract

A task processing method and device applied to a heterogeneous computing power system are used to obtain the running condition of computing power resources used by a certain task. The method comprises the following steps: a scheduling center determines the computing power demand of a to-be-processed task; the scheduling center determines a first abstract resource conforming to the computing power demand from an abstract resource pool; each abstract resource in the abstract resource pool is obtained by the scheduling center based on the abstract classification of each computing power resource reported by each provider in a unified mode according to the computing power; the scheduling center assigns the to-be-processed task to the first abstract resource, and collects the running condition of the first abstract resource; the running condition comprises a first task identifier of the to-be-processed task.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a task processing method and apparatus for heterogeneous computing power systems. Background Technology

[0002] With the rapid development of computer technology, distributed architecture has been widely used in task processing. Distributed architecture communicates through a network, with each computer node coordinating to complete a common task. In different application scenarios, different heterogeneous computing power collaborates based on distributed architecture to maximize computational efficiency.

[0003] In existing technologies, the operation of tasks and the different computing resources used are monitored separately. When an anomaly occurs during the operation of a task, it is impossible to know which specific computing resource the task used when the anomaly occurred.

[0004] Therefore, there is an urgent need for a solution to obtain the operational status of computing resources used by a certain task. Summary of the Invention

[0005] This application provides a task processing method and apparatus for heterogeneous computing power systems, used to obtain the operating status of computing resources used by a certain task.

[0006] In a first aspect, embodiments of this application provide a task processing method applied to a heterogeneous computing power system, the method comprising:

[0007] The scheduling center determines the computing power requirements of the tasks to be processed;

[0008] The scheduling center determines the first abstract resource that meets the computing power requirement from the abstract resource pool; each abstract resource in the abstract resource pool is obtained by the scheduling center by abstracting and classifying the computing power resources reported by each provider in a unified mode based on their computing capabilities.

[0009] The scheduling center distributes the task to be processed to the first abstract resource and collects the operation status of the first abstract resource; the operation status includes the first task identifier of the task to be processed.

[0010] The above technical solution involves the scheduling center abstracting and classifying the computing resources reported by various providers according to a unified model based on their computing power, resulting in abstract resources. On the one hand, this unifies the different computing resources from different providers; on the other hand, abstracting and classifying based on computing power masks the differences in computing power among different resources, improving the scheduling center's efficiency in assigning tasks to selected abstract resources based on their computing power requirements. Furthermore, by collecting the operational status of the abstract resources and the task identifiers of the tasks included in the operational status, it is possible to trace which abstract resources the task used and the operational status of those abstract resources.

[0011] In one possible design, the abstract resources in the abstract resource pool are obtained by the scheduling center through abstracting and classifying the computing power resources reported by each provider according to a unified mode based on their computing capabilities, including:

[0012] The scheduling center receives computing resource information reported by the provider, which includes attribute values ​​of various set attributes of the computing resources; these set attributes are pre-set uniform patterns.

[0013] The scheduling center classifies each computing resource according to its computing power based on the information of each computing resource, and abstracts each computing resource into an abstract resource with an abstract resource identifier.

[0014] The above technical solution allows the scheduling center to receive the various set attributes of the unified computing resources reported by the providers, thus unifying the different computing resources from different providers. The scheduling center classifies each computing resource according to its computing power and abstracts it into abstract resources with abstract resource identifiers, which can mask the differences in computing power between different computing resources and improve the task distribution efficiency of the scheduling center. The abstract resource identifiers also facilitate finding the abstract resource information used by a certain task.

[0015] In one possible design, the computing resource information also includes computing resource certification certificates; before classifying the computing resources according to their computing power, the following steps are also included:

[0016] The scheduling center verifies the computing resource certification certificate; the computing resource certification certificate is used to indicate that the reported computing resources have been certified by the certification center.

[0017] In one possible design, the computing resource certification certificate is obtained through the following methods:

[0018] The certification center receives a computing resource certification request sent by the provider; the computing resource certification request includes authentication information; the authentication information is used to indicate the provider of the computing resource and the type of computing resource;

[0019] After the authentication center verifies the authentication information, it sends an authentication approval instruction to the provider; the authentication approval instruction is used to obtain a computing power resource authentication certificate.

[0020] The above technical solution allows the certification center to uniformly certify the different computing resources provided by various providers. The certification certificate indicates that the computing resources provided by the provider are safe and reliable, thus ensuring the security of user data.

[0021] In one possible design, the authentication center receives a computing resource authentication request sent by the provider, including:

[0022] The certification center receives the computing resource certification request sent by the provider through a network request interface, or the certification center receives the computing resource certification request sent by the provider through a software development kit (SDK) interface.

[0023] In one possible design, the authentication information includes a fixed key for indicating the provider of computing resources and descriptive information for describing the computing resources.

[0024] The authentication information is encrypted according to the Advanced Encryption Standard (AES)-Cryptographic Block Connection (CBC) mode.

[0025] In one possible design, the set attributes include at least one of the following: resource name, resource description, resource type, timestamp, specified static resource or specified dynamic resource, the holding object of exclusive resource or the sharing scope of shared resource.

[0026] In one possible design, the abstract resource pool includes custom abstract resources;

[0027] The custom abstract resource is an abstract resource combination with an abstract resource identifier, which is an abstraction of each computing power resource that meets the screening criteria by the scheduling center; the screening criteria include at least one of the following: the manufacturer of the computing power resource, the scope of use, the usage time, the resource type, and the computing type.

[0028] In one possible design, the collection of the operational status of the first abstract resource includes:

[0029] Determine the task type of the task to be processed;

[0030] Determine the collection cycle based on the task type;

[0031] The operational status of the first abstract resource is collected according to the stated collection cycle.

[0032] The above technical solution determines the collection cycle for collecting the running status of abstract resources based on the task type of the task to be processed, which can save the computing power and storage space consumed in collecting the running status of abstract resources.

[0033] In one possible design, the method further includes:

[0034] The scheduling center receives an operation status query request sent by the operation and maintenance center; the operation status query request includes a second task identifier;

[0035] The scheduling center queries the operation status of the second abstract resource that is running the task corresponding to the second task identifier from the operation status of each abstract resource based on the second task identifier.

[0036] With the above technical solution, users can query the running status of a task from the operation and maintenance center based on the task identifier. The scheduling center can query the running status of the abstract resources corresponding to the task identifier based on the query request sent by the operation and maintenance center, so that users can clearly understand the running status of the task and the running status of the abstract resources used by the task.

[0037] Secondly, embodiments of this application provide a task processing device for a heterogeneous computing system, the device comprising:

[0038] The determination module is used to determine the computing power requirements of the task to be processed; and to determine the first abstract resource that meets the computing power requirements from the abstract resource pool; each abstract resource in the abstract resource pool is obtained by the scheduling module by abstracting and classifying the computing power resources reported by each provider in a unified mode based on their computing capabilities.

[0039] The scheduling module is used to distribute the task to be processed to the first abstract resource and collect the running status of the first abstract resource; the running status includes the first task identifier of the task to be processed.

[0040] In one possible design, the determining module is further configured to: receive computing resource information reported by the provider, wherein the computing resource information includes attribute values ​​of various set attributes of the computing resources; wherein each set attribute is a pre-set uniform pattern;

[0041] Based on the information of each computing resource, each computing resource is classified according to its computing power, and each computing resource is abstracted into an abstract resource with an abstract resource identifier.

[0042] In one possible design, before the determining module classifies each computing resource according to its computing power, it is further configured to: verify the computing resource certification certificate; the computing resource certification certificate is used to indicate that the reported computing resource has been certified by the certification center.

[0043] In one possible design, the device further includes an authentication module, configured to: receive a computing resource authentication request sent by a provider; the computing resource authentication request includes authentication information; the authentication information is used to indicate the provider of the computing resource and the type of computing resource;

[0044] After successful authentication based on the authentication information, an authentication success instruction is sent to the provider; the authentication success instruction is used to obtain a computing power resource authentication certificate.

[0045] In one possible design, when the authentication module is used to receive a computing resource authentication request sent by the provider, it is specifically used to: receive the computing resource authentication request sent by the provider through a network request interface, or receive the computing resource authentication request sent by the provider through a software development kit (SDK) interface.

[0046] In one possible design, the authentication information includes a fixed key for indicating the provider of computing resources and descriptive information for describing the computing resources.

[0047] The authentication information is encrypted according to the Advanced Encryption Standard (AES)-Cryptographic Block Connection (CBC) mode.

[0048] In one possible design, the set attributes include at least one of the following: resource name, resource description, resource type, timestamp, specified static resource or specified dynamic resource, the holding object of exclusive resource or the sharing scope of shared resource.

[0049] In one possible design, the abstract resource pool includes custom abstract resources; the scheduling module is specifically used to: abstract each computing power resource that meets the filtering conditions into an abstract resource combination with an abstract resource identifier; the filtering conditions include at least one of the following: the manufacturer of the computing power resource, the scope of use, the usage time, the resource type, and the computing type.

[0050] In one possible design, when the scheduling module is used to collect the running status of the first abstract resource, it is specifically used to: determine the task type of the task to be processed;

[0051] Determine the collection cycle based on the task type;

[0052] The operational status of the first abstract resource is collected according to the stated collection cycle.

[0053] In one possible design, the scheduling module is further configured to: receive a runtime status query request sent by the operations and maintenance center; the runtime status query request includes a second task identifier;

[0054] Based on the second task identifier, query the running status of the second abstract resource that is running the task corresponding to the second task identifier from the running status of each abstract resource.

[0055] Thirdly, embodiments of this application also provide a computing device, including:

[0056] Memory, used to store program instructions;

[0057] A processor is configured to invoke program instructions stored in the memory and execute the method described in various possible designs of the first aspect, according to the obtained program instructions.

[0058] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer-readable instructions that, when read and executed by a computer, cause the method described in the first aspect or any possible design of the first aspect to be implemented. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 A schematic diagram of a heterogeneous computing system provided in an embodiment of this application;

[0061] Figure 2 A flowchart illustrating a task processing method applied to a heterogeneous computing system, provided as an embodiment of this application;

[0062] Figure 3 A schematic diagram illustrating the specific process of constructing an abstract resource pool as provided in an embodiment of this application;

[0063] Figure 4 A schematic diagram illustrating the specific process of obtaining the operational status of abstract resources provided in this application embodiment;

[0064] Figure 5 A schematic diagram of a task processing device applied to a heterogeneous computing system provided in this application embodiment;

[0065] Figure 6 This is a schematic diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0067] In the embodiments of this application, "multiple" refers to two or more. Terms such as "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or order.

[0068] Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclusively include, for example, a product or device that includes a series of components is not necessarily limited to those that are explicitly listed, but may include other components that are not explicitly listed or that are inherent to such product or device.

[0069] To better understand the embodiments of this application, the terms and functions involved in this application are explained below:

[0070] 1. Heterogeneous Computing Power: Heterogeneous computing power (CPU, GPU, FPGA, ASIC, etc.) is a collective term for differentiated high-performance computing resources. The demand for computing power in the entire artificial intelligence industry has promoted the diversity of computing architectures and the continuous improvement of computing performance. In different application scenarios, heterogeneous computing power works together to maximize computing efficiency. Traditional CPUs use the von Neumann architecture, suitable for applications with complex control but low computational density. GPUs use the SIMD / SIMT architecture, suitable for high-performance computing, image processing, and AI. Field-Programmable Gate Arrays (FPGAs) use a dataflow-driven computing architecture, featuring high spatial concurrency and low latency, suitable for low-cost, high-performance computing scenarios. ASICs are dedicated chips whose computing power and efficiency can be customized according to the scenario.

[0071] 2. Distributed Scheduling: To adapt to different application scenarios, different load attributes, and different load combinations, the unified scheduling layer needs to schedule and control timed tasks in a distributed environment. It consists of a unified scheduling framework and built-in scheduling strategies.

[0072] 3. Computing power tracing: This usually refers to tracing upstream to the source, metaphorically seeking historical roots. Computing power tracing aims to protect data security. Every computing link is monitored in real time. Once a problem occurs, operations and maintenance personnel can use tracing methods to find out which specific computing resource or step is abnormal.

[0073] 4. Authentication: There are usually two authentication methods for calling software interfaces. One is token authentication, which is added to the request message header when calling the API to authenticate the user and obtain the right to operate the API. The other is AK (Access Key ID) / SK (Secret Access Key) authentication, which adds signature information to the message header when making a request to obtain the user's identity.

[0074] Figure 1 An exemplary heterogeneous computing power system applicable to embodiments of this application is illustrated. This heterogeneous computing power system includes an authentication center, a scheduling center, and an operation and maintenance center. The authentication center is primarily used to authenticate the computing power resources provided by the provider; the scheduling center is primarily used to form an abstract resource pool and distribute tasks; and the operation and maintenance center is primarily used to query the operational status of tasks and computing power resources.

[0075] The following is combined Figure 1 and Figure 2 This paper details the implementation process of the task processing method for heterogeneous computing systems provided in this application. Figure 2 An exemplary embodiment of this application illustrates a task processing method applied to a heterogeneous computing system.

[0076] like Figure 2 As shown, the method includes the following steps:

[0077] Step 201: The scheduling center determines the computing power requirements of the tasks to be processed.

[0078] When users need to complete certain applications using the heterogeneous computing power system provided in this application, such as AI applications, big data applications, and high-performance applications, the scheduling center provides an interface to abstract each application into different tasks and assigns a task ID to each task. The scheduling center determines the computing power requirements of each generated task to be processed, that is, determines the number of computations required for each generated task to be processed.

[0079] Step 202: The scheduling center determines the first abstract resource that meets the computing power requirements from the abstract resource pool. The abstract resources in the abstract resource pool are obtained by the scheduling center through abstracting and classifying the computing power resources reported by each provider according to a unified model based on their computing capabilities.

[0080] For example, Table 1 shows a unified resource model pre-set by the scheduling center. This unified resource model unifies the computing power resource information of heterogeneous computing power provided by various providers. Providers can report the computing power resource information of the provided computing power according to the unified resource model and in a unified mode. The computing power resource information includes the attribute values ​​of various set attributes of the computing power resource, as shown in Table 1. The set attributes include: resource name, resource description, resource type, timestamp, specified static resource or specified dynamic resource, the holding object of exclusive resource or the sharing scope of shared resource. The resource description may include the problem that the computing power resource can solve or the number of cores of the computing power provided by the computing power resource; the resource type includes whether it is a dynamic resource, an available resource, or a shared resource. If the computing power resource is an exclusive resource, the holding object of the exclusive resource needs to be specified; if the computing power resource is a shared resource, the sharing scope of the shared resource needs to be specified; the timestamp can be the time when the computing power resource information is reported. It should be noted that the various settings attributes of the computing resources reported by the provider may include at least one of the above, and this application does not limit this. Furthermore, the computing resource information in the unified resource model can be expanded according to the computing resources reported by different providers.

[0081] Table 1

[0082]

[0083] In one possible implementation, the computing resource information also includes a computing resource certification certificate. Before classifying each computing resource according to its computing power, the scheduling center needs to verify the computing resource certification certificate, which indicates that the reported computing resource has been certified by the certification center.

[0084] Computing resource authentication certificates can be obtained through the following method: the authentication center receives a computing resource authentication request sent by the provider. Here, the authentication center can receive the computing resource authentication request sent by the provider through a network request interface or through a software development kit (SDK) interface. The computing resource authentication request includes authentication information, which indicates the provider of the computing resource and the type of computing resource. Optionally, the authentication information may include a fixed key indicating the provider of the computing resource and descriptive information describing the computing resource. This authentication information is encrypted according to the Advanced Encryption Standard (AES) in Cipher Block Connection (CBC) mode. For example, the fixed key in the authentication information can be a fixed string of characters. When the authentication center obtains this fixed key by parsing the authentication information and matches it with a fixed key stored in the system, it can confirm the legitimacy of the provider and then continue to parse other reported information, such as the model of the computing resource and other inherent attribute information indicating the type of computing resource. A fixed key can be shared by all providers and used solely to indicate the provider's legitimacy. In this case, other reported information can also include the identifier of the computing resource provider. Alternatively, a fixed key can be set up separately between each provider and the certification authority, with each provider having a different fixed key, thus allowing the provider's legitimacy to be determined through the fixed key.

[0085] After the authentication center successfully authenticates the provider based on the authentication information, it sends an authentication approval instruction to the provider. This instruction is used to obtain the computing resource authentication certificate. For example, after successful authentication, the center sends the provider an address to obtain the computing resource authentication certificate. The provider then downloads the certificate from this address and stores it in the trusted zone of the operating system.

[0086] In one possible implementation, after receiving the computing power resource information reported by the provider, the scheduling center classifies the computing power resources according to their computing capabilities based on the information, and abstracts each computing power resource into an abstract resource with an abstract resource identifier. Different types of abstract resources constitute an abstract resource pool.

[0087] For example, a scheduling center can categorize computing resources into logical computing capabilities, parallel computing capabilities, and neural network capabilities based on their computational power. Logical computing capability is a general-purpose, basic computing capability, represented by the CPU. This type of computing resource requires a large amount of space to house storage and control units. Due to the limited number of computing units, it is not well-suited for large-scale parallel computing, but excels at logic control, i.e., controlling the execution order of instructions in a program, such as using logic gates for addition and subtraction to determine the result. Its computing power is generally measured in TOPS (tera operations per second), where 1 TOPS represents one trillion operations per second. Typical CPU models include Intel dual-core processors and AMD's Athlon series. Parallel computing capability refers to efficient computing power specifically for processing data types such as graphics and images, represented by the GPU. This type of computing resource has numerous computing units and extremely long pipelines, and its computing power is generally measured in TFLOPS (tera floating-point operations per second), where 1 TFLOPS represents one trillion floating-point operations per second. Typical GPU models include the NVIDIA GeForce series, NVIDIA Tesla series, and AMD RX Vega series. Neural network computing power is primarily aimed at computationally intensive tasks such as AI neural networks and machine learning. Computing power is represented by NPUs and TPUs. These processors often employ a "data-driven parallel computing" architecture and are specifically designed for artificial intelligence. Different units of measurement are used depending on the type of AI chip; generally, floating-point operations per second (PFLOPS) can be used to measure its computing power. 1 PFLOPS represents 1 quadrillion floating-point operations per second. Typical AI chip models include the HUAWEI HiKey series and the YITU Questcore chip.

[0088] When a provider reports computing power resource information, it reports to the scheduling center the number of cores the resource can provide. The scheduling center converts this core count into the number of operations per second, thus abstracting the computing power resource into an abstract resource. When the scheduling center issues a task, it selects an abstract resource that meets the computing power requirements based on the number of calculations needed for the task determined in step 201, and issues the task to one or more abstract resources. This masks the differences in computing power provided by different providers, improving the scheduling center's task issuance efficiency. A unique abstract resource identifier identifies these different types of abstract resources, facilitating quick identification of the abstract resource used by the task when querying its execution status. It should be noted that each type of abstract resource may contain computing power resources provided by multiple providers; for example, a logic operation abstract resource may contain CPUs provided by provider A, provider B, and provider C. Similarly, a unique computing power resource identifier identifies different computing power resources provided by different providers. The system stores a mapping between abstract resource identifiers and computing resource identifiers, so that when querying the running status of a task, it can quickly find which computing resource among the abstract resources used by the task.

[0089] In one possible implementation, the abstract resource pool includes custom abstract resources. Here, custom abstract resources refer to the scheduling center abstracting various computing resources that meet the filtering criteria into combinations of abstract resources with unique abstract resource identifiers. Users can customize different types of abstract resource combinations according to their needs to meet personalized computing scenarios. The filtering criteria include at least one of the following: the manufacturer of the computing resources, the scope of use, the usage time, the resource type, and the computing type. For example, if a user selects dedicated neural network + logic computing resources from manufacturer A on nodes 1 and 2, the scheduling center will select a matching NPU + CPU type abstract resource combination and identify it with a unique abstract resource identifier.

[0090] Step 203: The scheduling center distributes the tasks to be processed to the first abstract resource and collects the operation status of the first abstract resource, which includes the first task identifier of the tasks to be processed.

[0091] In one possible implementation, when the scheduling center collects the operational status of the first abstract resource, it first determines the task type of the task to be processed, determines the collection period based on the task type, and then collects the operational status of the first abstract resource according to the collection period. For example, when the task type is a batch processing task, the task may continuously occupy a certain type of abstract resource, so it is necessary to collect the operational status of the abstract resource at short intervals, such as collecting the operational status of the abstract resource periodically at a collection period of 3 seconds; when the task type is a long-running service task, the task may only occasionally use a certain type of abstract resource, so it is not necessary to collect the operational status of the abstract resource frequently, such as collecting the operational status of the abstract resource periodically at a collection period of half an hour.

[0092] In one possible implementation, the scheduling center receives a runtime status query request from the operations and maintenance center, the request including a second task identifier. Based on the second task identifier, the scheduling center queries the runtime status of the second abstract resource running the task corresponding to that second task identifier from the runtime status of each abstract resource.

[0093] After issuing tasks, the scheduling center collects the operational status of the first abstract resource. On one hand, it records the operational status of the abstract resources used by each task. When a task encounters an anomaly, the system retrieves the operational status of the abstract resource that caused the anomaly by using the task's identifier and the identifier of the abstract resource it uses. Furthermore, it can also retrieve the operational status of the computing resources under the abstract resource that caused the anomaly by using the task's identifier, the identifier of the abstract resource it uses, and the identifier of the computing resources. On the other hand, it also clearly shows the usage ratio of each computing resource under each type of abstract resource, allowing the scheduling center to adjust the scheduling strategy of abstract resources in a timely manner based on task resource requirements and remaining resources.

[0094] To better explain the embodiments of this application, the following is in conjunction with... Figure 3 and Figure 4 Describe the specific process of task processing applied to heterogeneous computing power systems in a specific implementation scenario.

[0095] Figure 3 The following is an exemplary illustration of the specific process for constructing an abstract resource pool provided in an embodiment of this application, including the following steps:

[0096] Step 301: The certification center receives the computing resource certification request sent by the provider.

[0097] Step 302: The authentication center performs authentication based on the authentication information. If the authentication is successful, proceed to step 303; otherwise, the process ends.

[0098] Step 303: The certification center sends the address for obtaining the certification certificate to the provider.

[0099] Step 304: The scheduling center receives computing resource information reported by each provider.

[0100] The scheduling center receives computing resource information reported by each provider in a unified manner, which also includes computing resource certification certificates.

[0101] Step 305: The scheduling center abstracts and classifies each computing resource according to its computing power.

[0102] Step 306: The scheduling center identifies each type of abstract resource and the computing power resources provided by different providers.

[0103] Figure 4 The following is an exemplary illustration of the specific process for obtaining the runtime status of an abstract resource provided in an embodiment of this application, including the following steps:

[0104] Step 401: The scheduling center abstracts the user's application into a task.

[0105] Step 402: The scheduling center determines the computing power requirements of the tasks to be processed.

[0106] Step 403: The scheduling center determines the abstract resources that meet the computing power requirements from the abstract resource pool.

[0107] Step 404: The scheduling center distributes the tasks to be processed to the first abstract resource. If the task distribution is successful, proceed to step 405; if the task distribution fails, repeat step 404.

[0108] Step 405: The scheduling center collects the operational status of the abstract resources.

[0109] Step 406: The Operation and Maintenance Center sends a request to query the operation status.

[0110] The operations and maintenance center sends an operation status query request to the dispatch center. The operation status query request includes the task identifier of the task to be queried.

[0111] Step 407: The scheduling center queries the operation status of the abstract resource.

[0112] Based on the task identifier, the scheduling center queries the operation status of the abstract resource that is running the task corresponding to that task identifier from the operation status of each abstract resource.

[0113] This application provides a task processing method for heterogeneous computing power systems. The scheduling center abstracts and categorizes the computing resources reported by various providers according to a unified model based on their computing power capabilities, resulting in abstract resources. On the one hand, this unifies the different computing resources from different providers; on the other hand, abstracting and categorizing based on computing power capabilities masks the differences in computing power among different resources, improving the efficiency of the scheduling center in assigning tasks to selected abstract resources based on their computing power requirements. Furthermore, by collecting the operational status of the abstract resources and the task identifiers of the tasks included in the operational status, it is possible to trace which abstract resources the task used and the operational status of these abstract resources.

[0114] Based on the same technological concept Figure 5 An exemplary embodiment of this application provides a task processing apparatus for a heterogeneous computing system, which is used to implement the task processing method for a heterogeneous computing system in the above embodiment.

[0115] like Figure 5 As shown, the device 500 includes:

[0116] The determination module 501 is used to determine the computing power requirements of the task to be processed; and to determine the first abstract resource that meets the computing power requirements from the abstract resource pool; each abstract resource in the abstract resource pool is obtained by the scheduling module by abstracting and classifying the computing power resources reported by each provider in a unified mode based on their computing capabilities.

[0117] The scheduling module 502 is used to distribute the task to be processed to the first abstract resource and collect the running status of the first abstract resource; the running status includes the first task identifier of the task to be processed.

[0118] In one possible design, the determining module 501 is further configured to: receive computing resource information reported by the provider, wherein the computing resource information includes attribute values ​​of various set attributes of the computing resources; wherein each set attribute is a pre-set uniform pattern;

[0119] Based on the information of each computing resource, each computing resource is classified according to its computing power, and each computing resource is abstracted into an abstract resource with an abstract resource identifier.

[0120] In one possible design, before the determining module classifies each computing resource according to its computing power, the determining module 501 is further configured to: verify the computing resource certification certificate; the computing resource certification certificate is used to indicate that the reported computing resource has been certified by the certification center.

[0121] In one possible design, the device further includes an authentication module 503, configured to: receive a computing resource authentication request sent by a provider; the computing resource authentication request includes authentication information; the authentication information is used to indicate the provider of the computing resource and the type of computing resource;

[0122] After successful authentication based on the authentication information, an authentication success instruction is sent to the provider; the authentication success instruction is used to obtain a computing power resource authentication certificate.

[0123] In one possible design, when the authentication module 503 is used to receive a computing resource authentication request sent by the provider, it is specifically used to: receive the computing resource authentication request sent by the provider through a network request interface, or receive the computing resource authentication request sent by the provider through a software development kit (SDK) interface.

[0124] In one possible design, the authentication information includes a fixed key for indicating the provider of computing resources and descriptive information for describing the computing resources.

[0125] The authentication information is encrypted according to the Advanced Encryption Standard (AES)-Cryptographic Block Connection (CBC) mode.

[0126] In one possible design, the set attributes include at least one of the following: resource name, resource description, resource type, timestamp, specified static resource or specified dynamic resource, the holding object of exclusive resource or the sharing scope of shared resource.

[0127] In one possible design, the abstract resource pool includes custom abstract resources; the scheduling module 502 is specifically used to: abstract each computing power resource that meets the filtering conditions into an abstract resource combination with an abstract resource identifier; the filtering conditions include at least one of the following: the manufacturer of the computing power resource, the scope of use, the usage time, the resource type, and the computing type.

[0128] In one possible design, when the scheduling module is used to collect the running status of the first abstract resource, the scheduling module 502 is also used to: determine the task type of the task to be processed;

[0129] Determine the collection cycle based on the task type;

[0130] The operational status of the first abstract resource is collected according to the stated collection cycle.

[0131] In one possible design, the scheduling module 502 is further configured to: receive a running status query request sent by the operation and maintenance center; the running status query request includes a second task identifier;

[0132] Based on the second task identifier, query the running status of the second abstract resource that is running the task corresponding to the second task identifier from the running status of each abstract resource.

[0133] Based on the same technical concept, embodiments of this application provide a computer device, such as... Figure 6 As shown, it includes at least one processor 601 and a memory 602 connected to at least one processor. In this embodiment, the specific connection medium between the processor 601 and the memory 602 is not limited. Figure 6 Taking the connection between the processor 601 and the memory 602 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.

[0134] In this embodiment of the application, the memory 602 stores instructions that can be executed by at least one processor 601. By executing the instructions stored in the memory 602, at least one processor 601 can perform the steps of the task processing method applied to the heterogeneous computing system described above.

[0135] The processor 601 is the control center of the computer device, capable of connecting various parts of the computer device via various interfaces and lines. It performs resource configuration by running or executing instructions stored in the memory 602 and accessing data stored in the memory 602. Optionally, the processor 601 may include one or more processing units. The processor 601 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may not be integrated into the processor 601. In some embodiments, the processor 601 and the memory 602 may be implemented on the same chip; in other embodiments, they may be implemented on separate chips.

[0136] Processor 601 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0137] Memory 602, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 602 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 602 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In the embodiments of this application, memory 602 may also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0138] Based on the same technical concept, embodiments of this application also provide a computer-readable storage medium storing a computer-executable program, which is used to cause a computer to perform the task processing methods for heterogeneous computing systems listed in any of the above methods.

[0139] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0140] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0141] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0142] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0143] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0144] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A task processing method applied to a heterogeneous computing power system, characterized in that, The heterogeneous computing power system includes an authentication center, a scheduling center, and an operation and maintenance center. The method includes: The certification center receives the computing resource certification request sent by the provider, performs authentication based on the computing resource certification request, and sends an authentication certificate to the provider if the authentication is successful. The scheduling center receives computing resource information reported by the provider in a unified mode. The computing resource information includes the authentication certificate and the attribute values ​​of each set attribute of the computing resource. The set attribute includes at least one of the following: resource name, resource description, resource type, timestamp, specified static resource or specified dynamic resource, holder of exclusive resource or sharing scope of shared resource. The scheduling center abstracts and classifies the computing resources according to their computing capabilities to obtain an abstract resource pool; the computing capabilities include logical computing capabilities, parallel computing capabilities, and neural network capabilities. The scheduling center determines the computing power requirements of the tasks to be processed; The scheduling center determines a first abstract resource that meets the computing power requirements from the abstract resource pool; the scheduling center then distributes the task to be processed to the first abstract resource. The scheduling center determines the task type of the task to be processed, determines the collection period based on the task type, and collects the operation status of the first abstract resource according to the collection period; the operation status includes the first task identifier of the task to be processed. The operation and maintenance center sends an operation status query request to the scheduling center, and the operation status query request includes a second task identifier; The scheduling center receives the operation status query request sent by the operation and maintenance center, and queries the operation status of the second abstract resource running the task corresponding to the second task identifier from the operation status of each abstract resource according to the second task identifier.

2. The method according to claim 1, characterized in that, The computing resource information also includes computing resource certification certificates; before classifying each computing resource according to its computing power, the following is also included: The scheduling center verifies the computing resource certification certificate; the computing resource certification certificate is used to indicate that the reported computing resources have been certified by the certification center.

3. The method according to claim 2, characterized in that, The computing resource certification certificate is obtained through the following methods: The certification center receives a computing resource certification request sent by the provider; the computing resource certification request includes authentication information; the authentication information is used to indicate the provider of the computing resource and the type of computing resource; After the authentication center verifies the authentication information, it sends an authentication approval instruction to the provider; the authentication approval instruction is used to obtain a computing power resource authentication certificate.

4. The method according to claim 3, characterized in that, The certification center receives computing resource certification requests sent by the provider, including: The certification center receives the computing resource certification request sent by the provider through a network request interface, or the certification center receives the computing resource certification request sent by the provider through a software development kit (SDK) interface.

5. The method according to claim 3, characterized in that, The authentication information includes a fixed key used to indicate the provider of the computing power resources and descriptive information used to describe the computing power resources; The authentication information is encrypted according to the Advanced Encryption Standard (AES)-Cryptographic Block Connection (CBC) mode.

6. The method according to claim 1, characterized in that, The abstract resource pool includes custom abstract resources; The custom abstract resource is an abstract resource combination with an abstract resource identifier, which is an abstraction of each computing power resource that meets the screening criteria by the scheduling center; the screening criteria include at least one of the following: the manufacturer of the computing power resource, the scope of use, the usage time, the resource type, and the computing type.

7. A task processing device for heterogeneous computing power systems, characterized in that, The heterogeneous computing power system includes an authentication center, a scheduling center, and an operation and maintenance center. The device includes: The determination module is used to receive a computing resource authentication request sent by the provider, perform authentication based on the request, and send an authentication certificate to the provider if the authentication is successful; receive computing resource information reported by the provider according to a unified mode, the computing resource information including the authentication certificate and attribute values ​​of various set attributes of the computing resource; the set attributes include at least one of the following: resource name, resource description, resource type, timestamp, specified static resource or specified dynamic resource, holder of exclusive resource or sharing scope of shared resource; the scheduling center abstracts and classifies the computing resources according to computing power to obtain an abstract resource pool; the computing power includes logical operation capability, parallel computing capability and neural network capability; determine the computing power requirements of the task to be processed; and determine the first abstract resource that meets the computing power requirements from the abstract resource pool. The scheduling module is used to distribute the tasks to be processed to the first abstract resource; determine the task type of the tasks to be processed, determine the collection period according to the task type, and collect the operation status of the first abstract resource according to the collection period; the operation status includes a first task identifier of the tasks to be processed; send an operation status query request, the operation status query request including a second task identifier; receive an operation status query request sent by the operation and maintenance center, and query the operation status of the second abstract resource running the task corresponding to the second task identifier from the operation status of each abstract resource according to the second task identifier.

8. A computing device, characterized in that, include: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the method as described in any one of claims 1 to 6 according to the obtained program instructions.

9. A computer-readable storage medium, characterized in that, It includes computer-readable instructions that, when read and executed by a computer, cause the computer to perform the method as described in any one of claims 1 to 6.

Images