Scheduling method and scheduling device for chip system

Abstract

The embodiment of the invention provides a scheduling method and a scheduling device for a chip system, relates to the technical field of chips, and aims to enable a user to program in a single-chip mode for a multi-node chip system, so that the usability and compatibility of programming are improved while the multi-node execution performance is guaranteed. The method comprises the steps that when a processor sends a calculation task to a chip system, the calculation task is segmented into a plurality of subtasks, and logic resources needed by the subtasks are determined; task descriptions of a plurality of nodes in the chip system for executing a plurality of subtasks are determined, the nodes comprise a master node and at least one slave node, and the task description corresponding to each subtask is used for indicating the subtask and physical resources for executing the subtask; and sending the task descriptions corresponding to the plurality of sub-tasks to the master node, wherein the master node sends the task description of each sub-task in the sub-tasks of the at least one slave node to the corresponding slave node. The resource management method and device are used for resource management of the multi-NUMA architecture of the AI chip.

Description

Technical Field

[0001] The present application relates to the field of chip technology, and in particular to a scheduling method and a scheduling device for a chip system. Background Art

[0002] The non-uniform memory access (NUMA) architecture is a common architecture that combines multiple central processing units (CPUs) to improve performance. With the demand for improved computing power in artificial intelligence (AI), the use of NUMA architecture in AI chips has become a trend.

[0003] For AI chips under the NUMA architecture, how to maintain users' programming habits so that users can use multiple NUMA nodes like a single chip in addition to using AI chips as multiple NUMA nodes to enhance computing power has become an urgent problem for chip manufacturers to solve. Summary of the invention

[0004] The embodiments of the present application provide a scheduling method and a scheduling device for a chip system. For a multi-node chip system, users can use a single-chip method to program, thereby improving the ease of use and compatibility of programming while ensuring the execution performance of multiple nodes.

[0005] In order to achieve the above-mentioned purpose, the embodiment of the present application adopts the following technical solution.

[0006] In a first aspect, a scheduling method for a chip system is provided, wherein the chip system includes multiple bare chips, each bare chip is a node, and the method includes: when a processor sends a computing task to the chip system, the processor divides the computing task into multiple subtasks and determines the logical resources required for the multiple subtasks; the processor determines task descriptions for multiple nodes in the chip system to execute the multiple subtasks, the multiple nodes include a master node and at least one slave node, and the task description corresponding to each subtask is used to indicate the subtask and the physical resources required to execute the subtask; the processor sends a task description corresponding to each of the multiple subtasks to the master node, wherein the multiple subtasks include subtasks of the master node and subtasks of at least one slave node, and each subtask of the at least one slave node corresponds to one of the at least one slave nodes; the master node sends the task description of each subtask of the at least one slave node to one of the corresponding at least one slave nodes.

[0007] The present application can be used in a chip system with multiple dies. For example, the architecture of the chip system is a multi-NUMA architecture of an AI chip. In the multi-NUMA architecture of an AI chip, each AI chip can be understood as a NUMA node, and each AI chip can be a die integrated in a multi-NUMA architecture chip, or a chiplet. Each AI chip can perform a computing task, or an AI task, or an AI computing task.

[0008] That is to say, when executing a computing task, the present application can divide the computing task into multiple subtasks and determine the logical resources required for multiple subtasks. In this way, when the logical resources are mapped to the physical resources and the physical resources to be executed for multiple subtasks are determined, the task descriptions of multiple subtasks can be assembled according to the multiple subtasks and the physical resources of the multiple subtasks. Then, a master node can be selected from multiple nodes, and the processor sends the task descriptions of multiple subtasks to the master node, and the master node schedules and distributes multiple subtasks. In this way, the master node and at least one slave node only need to execute the distributed subtasks on the physical resources, and there is no need to perform cross-node data access when executing a single operator (single computing task). In this application, the program programmed by the user for the CPU can use the chip system in a single chip like a single chip, for example, the program programmed for the CPU can use the NUMA architecture of multiple AI chips like a single chip. In this way, users can program in a single-chip manner, improve the ease of use and compatibility of CPU programming while ensuring the multi-node execution performance in the chip system, and complete the high-performance execution of a single operator in a multi-node scenario. Among them, the operator can be understood as a computing task to be executed.

[0009] In a second aspect, a scheduling method for a chip system is provided, the method comprising: when sending a computing task to the chip system, dividing the computing task into multiple subtasks, and determining the logical resources required for the multiple subtasks; determining task descriptions for multiple nodes in the chip to respectively execute the multiple subtasks, the multiple nodes including a master node and at least one slave node, and the task description corresponding to each subtask is used to indicate the subtask and the physical resources required to execute the subtask; sending task descriptions corresponding to each of the multiple subtasks to the master node, wherein the multiple subtasks include subtasks of the master node and subtasks of at least one slave node, each subtask of the at least one slave node corresponds to one of the at least one slave node, and the task description of each subtask of the at least one slave node is respectively sent to one of the corresponding at least one slave node.

[0010] The second aspect can be executed by a processor in the device, and the processor can send the divided multiple subtasks to the chip system. The beneficial effects of the second aspect can refer to the beneficial effects of the first aspect.

[0011] In the first aspect and / or the second aspect:

[0012] In one possible design, dividing a computing task into multiple subtasks includes: dividing the computing task into multiple subtasks according to the execution time of the first operator corresponding to each of the multiple task splitting methods of the computing task; wherein the execution time of the first operator corresponding to the first task splitting method of the computing task is used to indicate the time taken for a single node to execute a subtask under the first task splitting method, and the first task splitting method is any task splitting method.

[0013] The operator can be understood as a computing task to be performed, such as an AI task. That is to say, when determining the number of AI tasks to be split, the splitting principle is to compare the execution time of the first operator under each splitting method, so as to determine the first task splitting method with the shorter execution time.

[0014] In one possible design, a computing task is divided into multiple subtasks according to the execution time of the first operator corresponding to each of the multiple task splitting methods of the computing task, including: determining the execution time of the first operator corresponding to each of the multiple task splitting methods of the computing task; determining a target task splitting method in which the execution time of the first operator task meets a condition among the multiple task splitting methods; and dividing the computing task into multiple subtasks using the target task splitting method.

[0015] That is, this application can determine the target task segmentation method that meets the conditions by comparing the execution time and conditions of the first operator under each task segmentation method. In this way, when multiple subtasks are divided according to the target task segmentation method, when multiple nodes execute the multiple subtasks of the segmentation, multiple nodes do not need to perform cross-node data access when executing a single operator (a single computing task, such as a single AI task), which also saves the time of task execution. Therefore, for the NUMA architecture of multiple AI chips, when this application can use the NUMA architecture of multiple AI chips like a single chip, it can enable users to program in a single-chip manner, while ensuring the execution performance of multiple NUMA nodes, while improving the ease of use and compatibility of programming, and completing the high-performance execution of a single operator in a multi-NUMA node scenario.

[0016] In a possible design, the number of subtasks is N; N is an integer greater than or equal to 2; the execution time of the first operator includes: (1 / N)×the total cross-node data copy time of the computing task, (1 / N)×the total operator calculation time of the computing task and the scheduling software and hardware time; the condition includes: (1 / N)×the total cross-node data copy time of the computing task + the scheduling software and hardware time < (1 / N)×the total operator calculation time of the computing task.

[0017] Taking the computing task as an AI task and the chip system as a NUMA architecture with multiple AI chips as an example, the total cross-NUMA data copy time of the AI ​​task can be understood as the total input, output, read and write time between NUMA nodes. Scheduling software and hardware time can include data copy software time and scheduling synchronization time. The total operator calculation time of the AI ​​task can be understood as the calculation time when a single NUMA node completes the AI ​​task. If the AI ​​task is divided into N parts and N subtasks are obtained, the current operator division method can be considered profitable when 1 / N·total cross-NUMA data copy time of the AI ​​task + scheduling software and hardware time <(1 / N)×total operator calculation time of the AI ​​task. When N subtasks are executed synchronously on N NUMAs, the operator performance cost is better.

[0018] In one possible design, dividing a computing task into multiple subtasks includes: determining the execution time of a second operator when the computing task is executed by a single node, the second operator execution time being the maximum value of the data reading and writing time and the operator calculation time when the computing task is executed by a single node; if the second operator execution time is greater than or equal to the time threshold, then when the processor sends the computing task to the chip system, the computing task is divided into multiple subtasks.

[0019] Taking the NUMA architecture where the computing task is an AI task and the chip system is a multi-AI chip as an example, when a single NUMA node executes an AI task, data reading and writing and operator calculation are performed simultaneously, and the maximum value of the two is taken as the execution time of the second operator. If the execution time of the second operator when the AI ​​task is executed on a single NUMA node is relatively long, the AI ​​task can be split, and multiple NUMA nodes can synchronously execute the multiple subtasks after the split, so as to avoid cross-NUMA data access when a single operator is executed on multiple NUMA nodes, and complete the high-performance execution of a single operator in a multi-NUMA node scenario.

[0020] In one possible design, the processor determines that the task description of multiple nodes in the chip system to execute multiple subtasks includes: determining multiple nodes that execute multiple subtasks, and determining the physical resources required for the subtasks executed by each node based on the logical resources required for the multiple subtasks; determining the task description of the subtask executed by each node based on the multiple nodes and the physical resources required for the subtasks executed by each node. Taking the computing task as an AI task and the chip system as a NUMA architecture with multiple AI chips as an example, determining multiple nodes that execute multiple subtasks is equivalent to determining which AI chips in the NUMA architecture execute multiple subtasks. The physical resources required for each subtask can be understood as the storage resources, computing power resources, and task execution resources when each AI chip executes the subtask. The multiple AI chips and the physical resources of each subtask that are finally determined are repackaged into the task description of each subtask and sent to multiple AI chips. There is no need to access multiple AI chips across NUMA nodes, and the execution of AI tasks takes less time. For programmers, multiple subtasks only need to be sent to the main NUMA node, which enables users to program the CPU using a single chip. While ensuring the multi-node execution performance in the chip system, it improves the ease of use and compatibility of CPU programming and completes high-performance execution of a single operator in a multi-node scenario.

[0021] In a possible design, the task description of the subtask corresponding to the master node is used to instruct the master node to send the task description of each subtask in the subtask of at least one slave node to one of the corresponding at least one slave node. In this way, for the processor, when sending multiple subtasks to multiple nodes, it is not necessary to additionally instruct the master node to distribute the subtasks in addition to sending the task descriptions of the multiple subtasks, and the efficiency of task sending is higher.

[0022] Alternatively, the method further includes: the processor instructs the master node to send the task description of each subtask in the subtask of at least one slave node to one of the corresponding at least one slave node. This situation is equivalent to sending an instruction to instruct the master node to send the task description of the subtask corresponding to the slave node to at least one slave node after the processor sends the task description of each subtask to the master node. In this way, the master node and at least one slave node can execute the subtasks received locally, avoiding cross-node access.

[0023] In one possible design, before sending the task descriptions corresponding to the multiple subtasks to the master node, the method further includes: selecting a node with the smallest load among the multiple nodes as the master node according to the load information of each node among the multiple nodes, wherein the load information is used to indicate the load degree of the node.

[0024] Taking the NUMA architecture in which the computing task is an AI task and the chip system is a multi-AI chip as an example, the node load here can be understood as the affinity of the NUMA node. For example, multiple NUMA nodes can be sorted according to the load from small to large, and the NUMA node with the smallest load among multiple NUMA nodes can be selected as the main NUMA node. The NUMA task load can be understood as the load on the current task resources. In this way, if the NUMA node with the smallest load can be used as the main NUMA node for task scheduling, that is, distributing subtasks to the slave NUMA nodes, the load of each NUMA node in the chip with a multi-NUMA architecture can be made more balanced.

[0025] In a possible design, for each node, the load information includes at least one of: the task load on the current task execution resource of the node, the length of the task queue waiting to be executed, and the historical load statistics. That is, these types of load information will affect the load of the node. For example, for a chip system with a NUMA architecture, when determining the main NUMA node based on the load information of multiple NUMA nodes, a NUMA node with a smaller load is selected as the main NUMA node, which can make the task scheduling more efficient and the load of each NUMA node more balanced.

[0026] In a possible design, the method further includes: the slave node synchronizes the execution status of the subtask to the master node, and the execution status includes starting execution, ending execution, or abnormal termination. In this way, for example, for a chip system with a NUMA architecture, when the master NUMA node learns the execution status of each slave NUMA node, it can report possible exceptions to the processor at any time, as well as whether the task has started execution, whether the task has ended execution, etc., so that the processor can grasp the execution status of the currently issued AI task.

[0027] According to a third aspect, a scheduling method for a chip system is provided, wherein the chip system comprises a plurality of bare chips, each bare chip being a node, the method comprising: receiving task descriptions corresponding to a plurality of subtasks respectively sent by a processor, the task description corresponding to each subtask being used to indicate the subtask and the physical resources required to execute the subtask, and the plurality of subtasks being obtained by dividing the computing task by the processor; wherein the plurality of subtasks comprise subtasks of a master node and subtasks of at least one slave node, each subtask of the at least one slave node corresponds to one of the at least one slave nodes, and the task description of each subtask of the at least one slave node is respectively sent to one of the corresponding at least one slave nodes.

[0028] The beneficial effects of the third aspect can refer to the description of the beneficial effects of the first aspect.

[0029] In one possible design, the task description of the subtask corresponding to the master node instructs the master node to send the task description of each subtask of at least one slave node to one of the corresponding at least one slave nodes respectively; the method also includes: receiving an instruction from a processor, the instruction from the processor being used to instruct the master node to send the task description of each subtask of at least one slave node to one of the corresponding at least one slave nodes respectively.

[0030] In a possible design, the execution status of a subtask of at least one slave node is synchronized to the master node, and the execution status includes starting execution, ending execution, or abnormal termination.

[0031] In a fourth aspect, a scheduling device is provided, which can be, for example, a processor. The scheduling device includes: a task splitting unit, which is used to split the computing task into multiple subtasks when sending a computing task to a chip system, and determine the logical resources required for the multiple subtasks; a resource mapping unit, which is used to determine the task descriptions of multiple nodes in the chip system to execute multiple subtasks, the multiple nodes include a master node and at least one slave node, and the task description corresponding to each subtask is used to indicate the subtask and the physical resources required to execute the subtask; a task scheduling unit, which is used to send the task description corresponding to each of the multiple subtasks to the master node, wherein the multiple subtasks include subtasks of the master node and subtasks of at least one slave node, each subtask of the at least one slave node corresponds to one of the at least one slave node, and the task description of each subtask of the at least one slave node is sent to one of the corresponding at least one slave node.

[0032] The beneficial effects of the fourth aspect can be found in the description of the beneficial effects of the first aspect, which will not be repeated here.

[0033] In one possible design, the task splitting unit is used to: split the computing task into multiple sub-tasks according to the execution time of the first operator corresponding to each of the multiple task splitting methods of the computing task; wherein the execution time of the first operator corresponding to the first task splitting method of the computing task is used to indicate the time taken for a single node to execute a subtask under the first task splitting method, and the first task splitting method is any task splitting method.

[0034] In one possible design, the task splitting unit is used to: determine the execution time of the first operator corresponding to each of the multiple task splitting methods for the computing task; determine a target task splitting method in which the execution time of the first operator task meets the conditions among the multiple task splitting methods; and divide the computing task into multiple subtasks using the target task splitting method.

[0035] In a possible design, the number of subtasks is N; N is an integer greater than or equal to 2; the execution time of the first operator includes: (1 / N)×the total cross-node data copy time of the computing task, (1 / N)×the total operator calculation time of the computing task and the scheduling software and hardware time; the conditions include: (1 / N)×the total cross-data copy time of the computing task + the scheduling software and hardware time <(1 / N)×the total operator calculation time of the computing task.

[0036] In a possible design, the task scheduling unit is also used to: determine the execution time of a second operator when a computing task is executed by a single node, the second operator execution time being the maximum value of the data reading and writing time and the operator calculation time when the computing task is executed by a single node; if the second operator execution time is greater than or equal to the time threshold, then when the processor sends the computing task to the chip of multiple non-uniform memory access nodes, the computing task is divided into multiple subtasks.

[0037] In one possible design, the resource mapping unit is used to: determine multiple nodes that execute multiple subtasks, and determine the physical resources required for the subtasks executed by each node based on the logical resources required for the multiple subtasks; determine the task description of the subtask executed by each node based on the multiple nodes and the physical resources required for the subtasks executed by each node.

[0038] In one possible design, the task description of the subtask corresponding to the master node is used to instruct the master node to send the task description of each subtask in the subtask of at least one slave node to one of the corresponding at least one slave nodes; or, the task scheduling unit is also used to instruct the master node to send the task description of each subtask in the subtask of at least one slave node to one of the corresponding at least one slave nodes.

[0039] In a possible design, the task scheduling unit is further used to: select a node with the smallest load among the multiple nodes as a master node based on load information of each node among the multiple nodes; wherein the load information is used to indicate the load level of the node.

[0040] In a possible design, for each node, the load information includes: at least one of: the task load on the current task execution resources of the node, the length of the task queue waiting to be executed, and historical load statistics.

[0041] In a fifth aspect, a chip system is provided, which includes multiple bare chips, each bare chip is a node, and the method includes: a receiving unit, used to receive task descriptions corresponding to multiple subtasks sent by a processor, the task description corresponding to each subtask is used to indicate the subtask and the physical resources required to execute the subtask, the multiple subtasks are obtained by the processor dividing the computing task, and the multiple subtasks are executed by a master node and at least one slave node in the chip system; wherein the multiple subtasks include subtasks of the master node and subtasks of at least one slave node, each subtask of the at least one slave node corresponds to one of the at least one slave node, and the task description of each subtask of the at least one slave node is sent to a corresponding one of the at least one slave node.

[0042] The beneficial effects of the fifth aspect can be found in the description of the beneficial effects of the first aspect, which will not be repeated here.

[0043] In one possible design, the task description of the subtask corresponding to the master node instructs the master node to send the task description of each subtask of at least one slave node to one of the corresponding at least one slave nodes respectively; or, the receiving unit is also used to: receive an instruction from the processor, and the instruction from the processor is used to instruct the master node to send the task description of each subtask of at least one slave node to one of the corresponding at least one slave node respectively.

[0044] In a possible design, the execution status of a subtask of at least one slave node is synchronized to the master node, and the execution status includes starting execution, ending execution, or abnormal termination.

[0045] In a sixth aspect, a system provides a scheduling device, the scheduling device comprising a processor and a chip system, the chip system comprising multiple bare chips, each bare chip being a node, wherein: the processor is used to divide the computing task into multiple subtasks when sending a computing task to the chip system, and determine the logical resources required for the multiple subtasks; determine the task descriptions of multiple nodes in the chip system to execute the multiple subtasks, the multiple nodes include a master node and at least one slave node, and the task description corresponding to each subtask is used to indicate the subtask and the physical resources required to execute the subtask; send the task descriptions corresponding to each of the multiple subtasks to the master node, wherein the multiple subtasks include subtasks of the master node and subtasks of at least one slave node, and each subtask of the at least one slave node corresponds to one of the at least one slave nodes; the master node is used to send the task descriptions of each subtask of the at least one slave node to one of the corresponding at least one slave node.

[0046] In a seventh aspect, an electronic device is provided, the electronic device comprising: a memory and a processor. The memory and the processor are coupled. The memory is used to store a computer program code, the computer program code comprising a computer instruction. The transceiver is used to receive data and send data. When the processor executes the computer instruction, the electronic device executes any one of the scheduling methods of the chip system provided in the first aspect or its corresponding possible design.

[0047] In an eighth aspect, the present application provides a chip system, which is applied to an electronic device. The chip system includes one or more interface circuits and one or more processors. The interface circuit and the processor are interconnected by a line; the interface circuit is used to receive a signal from the processor, and the signal includes a computer instruction stored in the memory. When the processor executes the computer instruction, the electronic device executes the method in any possible design of the first aspect and / or the second aspect and the first aspect and / or the second aspect.

[0048] In a ninth aspect, a communication device is provided, comprising at least one processor, wherein the at least one processor is connected to a memory, and the at least one processor is used to read and execute a program stored in the memory so that the device executes a method as described in the first aspect and / or the second aspect above and any possible design of the first aspect and / or the second aspect.

[0049] In the tenth aspect, a chip is provided, which is coupled to a memory and is used to read and execute program instructions stored in the memory to implement a method as described in the first aspect and / or the second aspect and any possible design of the first aspect and / or the second aspect.

[0050] In the eleventh aspect, an embodiment of the present application provides a scheduling device, which is included in an electronic device, and has the function of implementing the above-mentioned first aspect and / or second aspect and any possible design of the first aspect and / or second aspect. The function can be implemented by hardware, or it can be implemented by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above-mentioned functions. For example, a task segmentation module or unit, a resource mapping module or unit, and a task scheduling module or unit.

[0051] In the twelfth aspect, an embodiment of the present application provides a computer-readable storage medium, including computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method in the above-mentioned first aspect and / or second aspect and any possible design of the first aspect and / or second aspect.

[0052] In the thirteenth aspect, an embodiment of the present application provides a computer program product, which, when running on a computer or a processor, enables the computer or the processor to execute the method in the above-mentioned first aspect and / or second aspect and any possible design of the first aspect and / or the second aspect.

[0053] It can be understood that any of the electronic devices, scheduling devices, chip systems, computer-readable storage media or computer program products provided above can be applied to the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods and will not be repeated here.

[0054] These and other aspects of the present application will become more apparent from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A schematic diagram of a deployment scenario of an AI chip provided in an embodiment of the present application;

[0056] Figure 2 A schematic diagram of a NUMA architecture of a CPU provided in an embodiment of the present application;

[0057] Figure 3 A schematic diagram of the NUMA architecture of an AI chip provided in an embodiment of the present application;

[0058] Figure 4 A schematic diagram of the scheduling of a NUMA architecture of an AI chip provided in an embodiment of the present application;

[0059] Figure 5 A schematic diagram of a software architecture provided for an embodiment of the present application;

[0060] Figure 6 A schematic diagram of a flow chart of a scheduling method for a chip system provided in an embodiment of the present application;

[0061] Figure 7 A schematic diagram of determining whether to split an AI task provided in an embodiment of the present application;

[0062] Figure 8 A schematic diagram of a multi-NUMA node jointly executing multiple subtasks of an AI task provided in an embodiment of the present application;

[0063] Fig. 9 A schematic diagram of a process flow of a resource management method based on multiple NUMA nodes provided in an embodiment of the present application;

[0064] Fig.10 A schematic diagram of determining whether to split an AI task provided in an embodiment of the present application;

[0065] Fig.11 A possible schematic diagram of the composition of an electronic device provided in an embodiment of the present application;

[0066] Fig.12 A possible schematic diagram of the composition of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0067] For ease of understanding, some examples of concepts related to the embodiments of the present application are provided for reference as follows.

[0068] Non-uniform memory access (NUMA) architecture: A computer memory design for multiple processors where memory access time depends on the memory location of the processor. Under NUMA, processors can access local memory faster than non-local memory.

[0069] Artificial intelligence (AI) chips: can be called AI accelerators or computing cards, that is, modules specifically used to accelerate a large number of computing tasks in AI applications (other non-computing tasks are still handled by the CPU). In a broad sense, chips for AI computing applications can be called AI chips.

[0070] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0071] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0072] Chiplet technology can be understood as integrating and packaging some pre-produced bare chips (dies) that realize specific functions together through integration technology to form a system chip. Among them, these basic bare chips can be understood as chiplets.

[0073] If chiplet technology is applied to the field of AI, each die can be understood as an AI chip.

[0074] AI chips include not only AI acceleration chips represented by graphics processing units (GPUs), field-programmable gate arrays (FPGAs), and application-specific integrated circuits (ASICs) (based on traditional chip architectures to accelerate AI computing for a certain type of algorithm or scenario), but also neural processor units (NPUs), as well as cutting-edge research, such as brain-like chips and reconfigurable general-purpose AI chips.

[0075] For example, GPU uses single instruction multiple data (SIMD), that is, one instruction operates multiple data, and has a large number of computing units and an extremely long graphics and image processing pipeline. Because most of the transistors inside the GPU can form various dedicated circuits and multiple pipelines, the GPU's computing speed is much higher than the CPU, and it has more powerful floating-point computing capabilities, which can alleviate the training difficulties of deep learning algorithms and release the potential of AI. Therefore, it is widely used in the field of deep learning algorithms. However, the GPU lacks complex computing logic units and must be scheduled by the CPU.

[0076] FPGA can lock instructions on the hardware architecture, and then use the hardware instruction stream to run data. In simple terms, it implements the AI ​​computing architecture with hardware circuits, and then continuously inputs the data stream into the system to complete the calculation. Unlike GPUs, FPGAs can have both hardware pipeline parallel and data parallel processing capabilities, and are suitable for processing data streams in a hardware pipeline manner. Therefore, they are very suitable for the AI ​​reasoning stage, and have obvious performance or energy consumption advantages over CPUs and GPUs.

[0077] ASIC can implement AI-specific algorithms and requires customized chips. The so-called customization refers to the architecture designed specifically for AI algorithms, which can help improve chip performance and power consumption.

[0078] NPU can adopt a data-driven parallel computing architecture and is good at processing massive multimedia data such as videos and images. In some scenarios, NPU can be used for artificial intelligence in the Internet of Things (IoT) to accelerate the operation of neural networks and solve the problem of low efficiency in traditional neural network operations.

[0079] In the embodiments of the present application, Figure 1 The figure shows a deployment scenario diagram of an AI chip. The AI ​​chip can usually be deployed in an electronic device, such as a cloud or terminal. Depending on the deployment location, AI chips can be divided into: cloud AI chips (or also called cloud AI chips) and terminal AI chips (or also called terminal AI chips). Of course, they can also be deployed in other devices, and this application does not limit them.

[0080] The cloud, that is, the data center, requires a huge amount of data and a large amount of computing power in the training phase of deep learning, which cannot be completed independently by a single processor. Therefore, the training phase can be implemented in the cloud.

[0081] Terminals are smart devices that perform edge computing, such as mobile phones, security cameras, cars, smart home devices, and various IoT devices. The number of terminals is huge, and their requirements vary greatly.

[0082] Cloud AI chips are characterized by powerful performance, the ability to support a large number of calculations at the same time, and the ability to flexibly support different AI applications such as images, voice, and video. Technology based on cloud AI chips can enable various smart devices and cloud servers to connect quickly, and the connection can maintain maximum stability. The characteristics of end AI chips are small size, low power consumption, and the performance does not need to be particularly powerful. Usually, only one or two AI capabilities need to be supported. Compared with cloud AI chips, end AI chips need to be embedded in the device. When the end AI chip is embedded in the device, the AI ​​capabilities of the device can be further enhanced, and the device can use the corresponding AI capabilities even when it is not connected to the Internet, so that AI coverage becomes more comprehensive.

[0083] From the perspective of the tasks that AI undertakes, the implementation of AI includes two links: training and reasoning. Therefore, according to the different tasks undertaken, AI chips can be divided into: training chips and reasoning chips. Generally speaking, training chips are mainly used to build neural network models, and reasoning chips mainly use neural network models for reasoning and prediction.

[0084] With the demand for increased computing power of single AI chips, the use of NUMA architecture in AI chips has become a trend. Since the resource management and scheduling of the CPU's NUMA architecture are already very mature, in order to facilitate the understanding of the use of NUMA architecture in AI chips, here we first explain the resource management and scheduling of the CPU's NUMA. Figure 2The figure shows a schematic diagram of the NUMA architecture of a CPU, which shows four CPUs: CPU0 to CPU3. Each CPU includes local memory, and the four CPUs correspond to memory A to memory D respectively. The program run by the CPU can be a program programmed by the user for the CPU, or a user driver program (driver package / acceleration package). The user who programs the CPU can be called the user of the CPU (CPU process). The user driver program does not require programming by the user of the CPU. The user driver program can be, for example, a driver package of an accelerator card. The user driver program can be loaded onto the CPU and run.

[0085] Among them, the program programmed by the user for the CPU can determine the maximum utilization efficiency of the computing power and memory of the CPU NUMA architecture. For example, the program programmed by the user for the CPU can select one of the CPUs, generate a task execution request, and at the same time apply for memory on the CPU to receive the task and store the intermediate data and result data of the task execution, so as to avoid the performance degradation caused by remote access as much as possible. Once the program programmed for the CPU determines that one CPU cannot meet the memory size required for executing the task, the program programmed for the CPU can select other CPUs, or select two CPUs with the best topology (topo) structure, and make task execution requests and apply for memory respectively. That is, the program programmed for the CPU can reduce the computing power and memory reduction caused by the multi-NUMA architecture by avoiding remote access. Among them, each CPU can be understood as a CPU NUMA node, and the CPU NUMA architecture includes multiple CPU NUMA nodes.

[0086] For the CPU NUMA scheduling mechanism, the program that programs the CPU knows that there are multiple CPU NUMA nodes in the CPU NUMA architecture. The order in which the program that programs the CPU schedules the CPU NUMA nodes and applies for memory is: first, there is scheduling, that is, the program that programs the CPU selects a CPU in the CPU NUMA architecture to run the task to be executed, and then applies for memory from multiple CPUs. At this time, according to the CPU selected by the scheduler, the CPU with the shortest path to the CPU selected by the scheduler can be selected to apply for memory. The tasks of each process of the CPU are published, scheduled, and executed on the same CPU NUMA node selected by the scheduler, thereby avoiding the scenario of scheduling across CPU NUMA nodes. Among them, the CPU with the shortest path can be understood as the CPU with the shortest CPU affinity path selected according to the CPU NUMA affinity.

[0087] Based on this, the initiative to fully utilize the computing power and memory under the CPU NUMA architecture is given to the program that programs the CPU, or the CPU user, which increases the complexity of CPU programming. Moreover, a task is executed on a CPU NUMA node, and it is impossible to achieve the requirement of multiple NUMA nodes to jointly complete a task.

[0088] In the NUMA scenario of AI chips, Figure 3 The figure shows a schematic diagram of the NUMA architecture of an AI chip, which includes four NUMA nodes: AI chip 1 to AI chip 4, and each AI chip is a NUMA node. Local memory is deployed in each NUMA node, and the local memory of each NUMA node can be accessed by other NUMA nodes, so the memory of each NUMA node may include remote cache. In order to achieve the purpose of CPU accessing multiple AI chips in the same way as the CPU calls a single chip (onechip), the industry increases the cache space size of the remote cache between NUMA nodes and improves the memory access bandwidth between NUMA nodes, so that the CPU calls multiple AI chips in a single-chip manner. However, the essence of this method is to increase the bandwidth and remote cache between multiple NUMA nodes to the level of uniform memory access (UMA), which has high requirements on the hardware process of AI chips and cannot be followed up when the hardware process is limited.

[0089] Specifically, in the scenario where the CPU calls multiple AI chips as a single chip, due to the scheduling of the NUMA architecture of the AI ​​chip, memory application and task release are completed by the user of the AI ​​chip. The user here is, for example, the user driver mentioned above. Moreover, when the user driver uses the NUMA architecture of the AI ​​chip to perform tasks, it first applies for memory to the NUMA architecture of the AI ​​chip, and then releases tasks to the AI ​​chip. That is, when the user driver of the AI ​​chip calls a logical device composed of multiple AI chips (equivalent to a single chip including multiple groups of logical devices that perform multiple tasks, and multiple AI chips that perform the same task are a group of logical devices), and cannot directly call the NUMA node of a single AI chip, the user driver will first apply for memory on the AI ​​chip, and then release tasks to the AI ​​chip. It is necessary to find a task scheduler based on the AI ​​chip where the memory is located. For example, Figure 4The figure shows a scheduling diagram of the NUMA architecture of an AI chip. When CPU0 loads and runs the user driver of the NUMA architecture that schedules the AI ​​chip, each AI chip can be understood as a NUMA node, and each AI chip is deployed with local memory, and the local memory can be accessed by the remote AI chip after remote access. At this time, multiple AI chips, such as AI chip 1 and AI chip 2, form a logical device. Assuming that when CPU0 first applies for memory for the task to be executed, the AI ​​chips involved in the memory applied for include AI chip 1 and AI chip 2. If CPU0 publishes the task to AI chip 1, CPU0 must not only find the task schedulers corresponding to AI chip 1 and AI chip 2 respectively, so that AI chip NUMA1 can not only access local memory, but also access the local memory of AI chip NUMA2 through remote access, and there is cross-NUMA node scheduling. However, this method has high requirements on the hardware process of AI chips, and there are situations where the hardware process is limited.

[0090] Therefore, an embodiment of the present application provides a scheduling method for a chip system, which can be applied to a chip system, wherein the chip system includes multiple chips (e.g., dies). For example, the architecture of the chip system is a multi-NUMA node architecture of an AI chip. Through the resource management of operator slices and the master-slave scheduling of the AI ​​chip, cross-NUMA data access of a single operator is avoided during multi-NUMA execution, and high-performance execution of a single operator in a multi-NUMA node scenario is completed. Among them, the operator can be understood as a computing task. In the multi-NUMA architecture of the chip, the operator can be understood as the AI ​​task to be executed.

[0091] It should be noted that the NUMA architecture used in this application is only illustrative, and the solution provided in this application can be applied to a variety of architectures, as long as the architecture is a multi-AI chip architecture, it can be applied to the solution of this application. In other words, the architecture of multi-AI chips is called NUMA architecture. In the subsequent evolution process, the architecture of multi-AI chips may have other architecture names, and this application does not limit the architecture name of multi-AI chips.

[0092] In the method embodiment of the present application, from the perspective of software architecture, Figure 5 The figure shows a schematic diagram of a software module provided by an embodiment of the present application, which may include a multi-node memory management and task segmentation module 501, a runtime task distribution module 502 and a chip scheduling module 503.

[0093] Among them, the multi-node memory management and task splitting module 501 can be used to determine the number of subtasks that a computing task can be split into and split the computing task to obtain multiple subtasks and the logical resources required for the multiple subtasks after the splitting. For example, the logical resources include computing power resources, storage resources and task execution resources for logically executing multiple subtasks. Each chip in the chip system can be regarded as a node, that is, a single computing task can be divided into multiple subtasks that multiple nodes work together.

[0094] The runtime task distribution module 502 can be used to complete the mapping of logical resources to physical resources. In addition, the computing task can also determine the task description of each node according to the affinity principle, and send the task description of each node to multiple nodes.

[0095] The chip scheduling module 503 can be used to control the master node among multiple nodes to distribute the task description (used to indicate subtasks and physical resources) of the slave node to at least one slave node, so that at least one slave node and the master node can coordinate to complete the computing task. For example, in the multi-NUMA architecture of the chip, the chip system includes Figure 5 NUMA nodes 1 to NUMA nodes 4 shown in the figure include a master NUMA node and multiple slave NUMA nodes among the determined multiple nodes.

[0096] From the perspective of hardware architecture, the software programs of the multi-node memory management and task splitting module 501 and the runtime task distribution module 502 can run on the CPU that issues computing tasks, and the software program of the chip scheduling module 503 can run on the chip that serves as the master node.

[0097] As mentioned above, the program running on the CPU can be a program programmed by the user for the CPU, or it can be a user driver (driver package / acceleration package). The chip system corresponding to the NUAM architecture of the above-mentioned AI chip can be understood as an accelerator card, for example. When an accelerator card is installed on the device, the software program corresponding to the accelerator card can be understood as a user driver program for the accelerator card. When the CPU schedules the chip system, the CPU loads and runs the user driver program. The CPU will perform the functions of the above-mentioned multi-node memory management and task splitting module 501 and the runtime task distribution module 502 in the process of running the user driver program, that is, perform task splitting, and elect the master node and slave node. The user driver program then controls the master node to execute the function of the chip scheduling module 503 on the chip system based on the elected master node and at least one slave node, that is, distributes the subtask corresponding to at least one slave node to at least one slave node.

[0098] It should be understood that although the CPU will run the above-mentioned user driver, the program programmed by the user for the CPU does not know the functions of the node memory management and task segmentation module 501 and the runtime task distribution module 502, or in other words, the program programmed by the user for the CPU is not used to implement the functions of the memory management and task segmentation module 501 and the runtime task distribution module 502. The user driver does not need to feedback the task segmentation method and the election results of the master and slave nodes to the program programmed by the user for the CPU. The program programmed by the user for the CPU still schedules the chip system like scheduling a single chip (one chip), enabling the user to program the CPU using a single chip, while ensuring the multi-node execution performance in the chip system, while improving the ease of use and compatibility of CPU programming, and completing the high-performance execution of a single operator in a multi-node scenario.

[0099] Applying the above-mentioned software and hardware architecture, the embodiments of the present application are described below.

[0100] like Figure 6 The flowchart of a scheduling method based on a chip system provided in an embodiment of the present application is shown. The chip system in the embodiment of the present application is a chipset or chipset composed of multiple chips, and for the CPU, the chip system is "one chip", so the CPU accesses or schedules the chip system and the CPU accesses or schedules a certain chip in the same way. The following example is explained by taking the chip system architecture as a multi-NUMA architecture of an AI chip and the computing task as an AI task as an example. The AI ​​task can also be understood as an AI computing task or an AI work task.

[0101] The method includes the following process.

[0102] 601. When the processor sends a computing task to the chip system, the processor divides the computing task into multiple subtasks and determines the logical resources required for the multiple subtasks.

[0103] In some embodiments, the above-mentioned processor and chip system may be included in a scheduling device, and the scheduling device may be included in an electronic device.

[0104] In some embodiments, before any CPU in an electronic device sends an AI task to a onechip (chip system) composed of multiple AI chips, the CPU may first select the task splitting method with the best performance based on the performance evaluation algorithm / strategy when loading and running the user driver of the chip system, that is, determine the number of parts that the AI ​​task can be split into. In other words, determine the number of NUMA nodes that execute multiple subtasks of the AI ​​task, or in other words, determine the number of AI chips (die) that execute the AI ​​task.

[0105] In some embodiments, the basis of the performance evaluation algorithm / strategy may be the execution time of the operator. Therefore, dividing the computing task into multiple subtasks may include: dividing the computing task into multiple subtasks according to the execution time of the first operator corresponding to the multiple task division methods of the computing task. Among them, the execution time of the first operator corresponding to the first task division method of the computing task is used to indicate the time taken for a single node to execute a subtask under the first task division method, and the first task division method is any task division method.

[0106] That is, the processor may select a task splitting method with a better execution time of the first operator according to the execution time of the first operator when a single node executes a subtask under each task splitting method, and split the computing task into multiple subtasks.

[0107] In some embodiments, the processor may divide the computing task into multiple subtasks according to the execution time of the first operator corresponding to each of the multiple task division modes of the computing task, and may include:

[0108] Determine the execution time of the first operator corresponding to each task splitting method of the computing task; determine a target task splitting method in which the execution time of the first operator task meets the conditions among multiple task splitting methods; and divide the computing task into multiple subtasks using the target task splitting method.

[0109] That is, the present application can set conditions to select a target task segmentation method that meets the conditions from a plurality of task segmentation methods, and divide the computing task into a plurality of subtasks according to the selected target task segmentation method.

[0110] For example, the processor may start by dividing the computing task into two subtasks and try to calculate, obtain the execution time of the first operator corresponding to each subtask, and determine whether the execution time of the first operator meets the conditions. If the conditions are met, it is determined that the computing task is divided into two subtasks. If the conditions are not met, the processor tries to divide the computing task into three subtasks and then tries to calculate, obtain the execution time of the first operator corresponding to each subtask, and determine whether the execution time of the first operator meets the conditions. By analogy, determine the target task division method that meets the conditions.

[0111] In some embodiments, assuming that the number of subtasks is N, where N is an integer greater than or equal to 2:

[0112] The time consumption of executing the first operator may include:

[0113] (1 / N)×the total cross-node data copy time of the computing task, (1 / N)×the total operator computing time of the computing task and the scheduling software and hardware time;

[0114] The above conditions include: (1 / N)×total cross-node data copy time of the computing task + time consumed by scheduling software and hardware < (1 / N)×total operator calculation time of the computing task.

[0115] Specifically, in the multi-NUMA architecture of the AI ​​chip, if the AI ​​task is divided into N subtasks, the above cross-NUMA data copy time can be understood as the total time taken by one NUMA node to perform input, output, read and write to the remaining N-1 NUMA nodes. If N NUMA nodes execute N subtasks at the same time, the data copy time of one NUMA node can be 1 / N·the total cross-NUMA data copy time of the AI ​​task.

[0116] The above scheduling software and hardware time consumption can be understood as the sum of the software time consumption of data copying on one NUMA node and the hardware scheduling synchronization time consumption.

[0117] The above operator calculation time can be understood as the time it takes for one NUMA node to perform the entire AI task calculation. If N NUMA nodes execute N subtasks simultaneously, the operator calculation time of one NUMA node can be (1 / N) × operator calculation time.

[0118] If (1 / N)×total cross-NUMA data copy time of AI tasks + scheduling software and hardware time <(1 / N)×total operator calculation time of AI tasks, it can be considered that when the AI ​​task is divided into N subtasks, the required data reading and writing time is smaller, the operator splitting theory is beneficial, and the performance of AI tasks (single operator) completed by N NUMA nodes working together is better.

[0119] When N NUMA nodes execute corresponding subtasks at the same time, the total operator execution time (including cross-NUMA data copy time and operator theoretical calculation time) when the N NUMA nodes execute their respective subtasks may include: (N-1) / N×total cross-NUMA data copy time of AI tasks + scheduling software and hardware time + (1 / N)×total operator calculation time of AI tasks. That is, when performing cross-NUMA data copy, the cross-NUMA data copy time of a NUMA node may include the sum of the cross-NUMA data copy time of 1 NUMA node to N-1 NUMA nodes.

[0120] Of course, the above conditions are only an exemplary description, and this application does not limit the conditions for the method of dividing the target tasks.

[0121] In some embodiments, the logical resources required for the multiple subtasks obtained by the processor can be regarded as the logical resources required for executing computing tasks on onechip, that is, a single chip. The logical resources may include logical computing resources, storage resources, and task execution resources.

[0122] Among them, computing power can be understood as the computing power to achieve the target result output by processing information data. In the NUMA architecture of the AI ​​chip of this application, the AI ​​chip is the main carrier of computing power. Logical computing power resources can be understood as the number of NUMA nodes required to execute multiple subtasks after the AI ​​task is divided, that is, the number of AI chips. Assuming that the number of multiple subtasks is N, accordingly, the logical computing power resources correspond to N NUMA nodes.

[0123] For example, if the AI ​​task is to perform graphics calculations, each AI chip can be a die on the GPU, and the logical operator resources are equivalent to determining the number of dies on the GPU.

[0124] Logically, storage resources can be understood as the amount of storage space required for each subtask when it is executed on the corresponding AI chip.

[0125] Logically, task execution resources can be understood as the number of internal concurrent task streams, where concurrent streams can be understood as streams that support concurrent operations.

[0126] Among them, the task execution resource can be understood as a task scheduler, which includes a predefined task manager for executing general tasks and a custom task manager for defining one's own tasks, or can be used to organize and define one or more time-consuming tasks from different types of programs.

[0127] In some embodiments, when the processor divides the computing task, it can divide the task according to the execution capability of the nodes in the chip system, and try to ensure that the execution time of multiple subtasks on the selected multiple nodes is similar. In this way, when selecting the task division method, the time taken by a single node to execute a subtask after division can be compared to select the target task division method.

[0128] Among them, step 601 can be executed by the multi-node memory management and task segmentation module 501 in the above software architecture, and executed by the CPU that issues the computing task in the hardware architecture.

[0129] 602. The processor determines task descriptions for executing multiple subtasks by multiple nodes in the chip system, where the multiple nodes include a master node and at least one slave node, and the task description corresponding to each subtask is used to indicate the subtask and the physical resources required to execute the subtask.

[0130] That is, before scheduling subtasks, the processor needs to map the logical resources required by multiple subtasks to the physical resources of multiple nodes, and obtain the task descriptions corresponding to the multiple subtasks. The multiple subtasks include subtasks of the master node and subtasks of at least one slave node, and each subtask of the at least one slave node corresponds to one of the at least one slave nodes.

[0131] Step 601 is equivalent to simply dividing the computing task into multiple subtasks and determining the logical resources required to execute the multiple subtasks after division on a single chip. It is necessary to further complete the mapping and conversion of the logical resources of the single chip to the physical resources. That is, it is necessary to determine the physical computing resources, storage resources and task execution resources.

[0132] In some embodiments, the processor may map and convert logical resources to physical resources according to the affinity principle, and assemble task descriptions of multiple subtasks on corresponding nodes. The task description of each node is used to indicate the subtasks distributed to the node and the physical resources for executing the subtasks.

[0133] In some embodiments, the processor may first determine multiple nodes that execute multiple subtasks, and determine the physical resources required for the subtasks executed by each node based on the logical resources required for the multiple subtasks. The task description of the subtask executed by each node is determined based on the multiple nodes and the physical resources required for the subtasks executed by each node.

[0134] like Figure 7 The figure shows a schematic diagram of mapping and converting logical resources to physical resources. Assume that in the multi-NUMA architecture of the AI ​​chip, the AI ​​task is divided into N parts, and N subtasks are obtained. Step 602 is equivalent to mapping the logical resources of a single chip (the logical resources required for a single chip to execute an AI task before the AI ​​task is divided) to physical resources on N NUMA nodes (NUMA node 1 to NUMA node N).

[0135] Mapping logical computing resources to physical computing resources is equivalent to determining the identifiers of the N NUMA nodes corresponding to the N subtasks and the accelerated computing resources on each NUMA node. In other words, it is necessary to determine which NUMA nodes the N subtasks are to be executed on and which accelerated computing resources of each NUMA node are to be occupied.

[0136] In some embodiments, there may be multiple ways to map computing power resources on logical resources to physical NUMA nodes. For example, assuming that the number of subtasks determined in step 601 is 4, step 602 needs to determine which 4 NUMA nodes these 4 subtasks will be executed on, and which accelerated computing resources of these 4 NUMA nodes will be occupied. Among them, the 4 AI chips with the best overall system load in the multi-NUMA nodes of the NUMA architecture, or the 4 AI chips that meet the load requirements and have the smallest system load, can be used as the selected 4 NUMA nodes, and the node identifiers of these 4 NUMA nodes can be obtained. Exemplarily, the possible future load conditions can be estimated based on the historical task load of each NUMA node on a single chip, and the current load conditions of each NUMA node can be used for weighted balancing calculation to obtain the load weight of each NUMA node, so as to select 4 NUMA nodes whose load meets the requirements according to the load weight.

[0137] The physical accelerated computing resources to be determined on each NUMA node vary with different AI tasks.

[0138] For example, assuming that the AI ​​task is a task of performing neural network calculations, each NUMA node is a die on the NPU, and the accelerated computing resources on each NUMA node may include: AICore, AICPU, and digital video pre-processing (DVPP), etc.

[0139] Among them, AICore can be understood as the core tensor computing unit, which is responsible for the convolution calculation of matrix tensors. For example, each AICore can have 16×16×16 half-floating point precision / cycle, or 256 half-floating point precision / cycle of vectorUnit.

[0140] AICPU can be understood as a physical computing unit.

[0141] DVPP integrates functions such as virtual private cloud (VPC), joint photographic experts group encoder (JPEGE), JPEG decoder (JPEGD), portable network graphics decoder (PNGD), video decoding (VDEC) and video encoding (VENC).

[0142] For another example, assuming that the AI ​​task is a task for performing graphics calculations, each NUMA node is a die on the GPU, and the accelerated computing resources on each NUMA node may include a streaming multiprocessor (SM) or a streaming processor (SP). In some examples, several SPs may be attached with some other units to form an SM. An SM can be regarded as the basic unit of GPU computing scheduling. That is, for a NUMA node integrated on a GPU, a NUMA node may include multiple SMs, and the SM may support the concurrent execution of up to hundreds or thousands of threads. In some examples, an SM may have 64 cores for computing.

[0143] Mapping logical storage resources to physical storage resources can be understood as determining the memory address range occupied by the memory space required by each subtask on the corresponding NUMA node. For example, in some embodiments, from the bottom layer implementation, determining the memory address range occupied on each NUMA node is equivalent to determining the address range of high bandwidth memory (HBM) memory.

[0144] Mapping logical task execution resources to physical task execution resources can be understood as determining the hardware task queue corresponding to the number of concurrent task streams (streams) of each subtask on a single NUMA node.

[0145] In some embodiments, after determining the physical resources of multiple subtasks on corresponding NUMA nodes, task descriptions of the NUMA nodes corresponding to the multiple subtasks can be determined based on the multiple subtasks and the physical resources corresponding to each subtask, and the task descriptions of the multiple NUMA nodes can be sent to the chip scheduling module 503.

[0146] Step 602 may be executed by the runtime task distribution module 502 in the software architecture, and executed by the CPU that issues the computing task in the hardware architecture.

[0147] 603. The processor sends a task description corresponding to each of the multiple subtasks to the master node.

[0148] The multiple subtasks include subtasks of the master node and subtasks of at least one slave node, and each subtask of the at least one slave node corresponds to one of the at least one slave node.

[0149] Exemplarily, after obtaining the task descriptions of multiple nodes corresponding to multiple subtasks, from the perspective of hardware architecture, when the CPU runs the user driver program of the chip system, it can send the task descriptions of the multiple nodes to the main node among the multiple nodes.

[0150] Therefore, before the CPU sends the task descriptions of multiple nodes to the master node, the user driver of the chip system running the CPU must first select the master node from multiple nodes, and the CPU can then interact with the master node. Among the multiple nodes, the node other than the master node is at least one slave node. At least one slave node can interact with the master node, and the CPU does not perceive the slave node when scheduling multiple subtasks.

[0151] In some embodiments, before the processor sends the task descriptions corresponding to the multiple subtasks to the master node, the method may further include: the processor selects the node with the smallest load among the multiple nodes as the master node according to the load information of each node among the multiple nodes. The load information is used to indicate the load degree of the node.

[0152] The process of determining the master node may be executed by the runtime task distribution module 502 in the software architecture, or by the CPU that issues the computing task in the hardware architecture.

[0153] In some embodiments, the load information includes at least one of: the task load on the current task execution resources of the node, the length of the task queue waiting to be executed, and historical load statistics.

[0154] Exemplarily, the CPU can determine the load of each NUMA node based on the task load on the current task execution resources of each NUMA node in multiple NUMA nodes, the length of the task queue waiting to be executed, and the historical load statistics value, as well as the corresponding weights, thereby selecting the NUMA node with the smallest weighted value as the NUMA node with the smallest load, that is, the primary NUMA node.

[0155] This application does not limit the implementation method of selecting a master NUMA node from multiple NUMA nodes.

[0156] After the processor determines the master NUMA node among the multiple NUMA nodes, the CPU can send the task descriptions of the multiple NUMA nodes to the master NUMA node. The task description corresponding to each NUMA node is used to indicate the subtasks to be executed and the physical resources occupied by executing the subtasks.

[0157] 604. The master node sends the task description of each subtask of the at least one slave node to one of the corresponding at least one slave node.

[0158] Here, “corresponding” can be understood as at least one slave node corresponding to the task description of at least one slave node's subtask, and one slave node corresponding to the task description of one slave node's subtask.

[0159] In some embodiments, the task description of the subtask corresponding to the master node is used to instruct the master node to send the task description of each subtask in the subtask of at least one slave node to one of the corresponding at least one slave node. In this way, signaling interaction resources between the processor and the chip system can be saved.

[0160] Alternatively, the method further includes: the processor instructs the master node to send the task description of each subtask in the subtask of at least one slave node to one of the corresponding at least one slave node. That is, in addition to sending the task descriptions of multiple subtasks to the master node, the processor can also send instructions to the master node to instruct the master node to send the subtasks to at least one slave node. In this way, when the task description of each node is obtained, there is no need to further modify the task description of the master node, and the method of generating the task description is relatively simple.

[0161] In some embodiments, in the multi-NUMA architecture of the AI ​​chip, when the master NUMA node receives the task description corresponding to each NUMA node, the master NUMA node distributes a task description of the subtask to each slave NUMA node, and at the same time, the master NUMA node also saves a task description of the subtask. The master NUMA node and the slave NUMA node execute their respective subtasks at the same time. There is no need for cross-NUMA data access between the master NUMA node and the slave NUMA node, and between the slave NUMA nodes. Each NUMA node can execute its own subtask.

[0162] For example, Figure 8 A schematic diagram of a plurality of subtasks of an AI task jointly executed by multiple NUMA nodes provided in an embodiment of the present application. Assuming that the AI ​​task is divided into four subtasks, after the runtime task distribution module 502 completes the task assembly and obtains the task descriptions of the four subtasks on the corresponding NUMA nodes, it first selects the master NUMA node, and then sends the task descriptions corresponding to the four NUMA nodes (master NUMA node, slave NUMA node 1, slave NUMA node 2, and slave NUMA node 3) to the master NUMA node, that is, sends the task to the master NUMA node.

[0163] For the chip scheduling unit 503 in the master NUMA node, after receiving the task description from the runtime task distribution module 502, the subtask corresponding to each slave NUMA node can be distributed to slave NUMA node 1, slave NUMA node 2 and slave NUMA node 3 through the scheduler queue element (SQE), and a subtask is retained in the master NUMA node. In some embodiments, when the master NUMA node distributes the subtask, it can be distributed to each slave NUMA node randomly, or it can be distributed to each slave NUMA node according to a predetermined rule, which is not limited in this application.

[0164] For example, when there are multiple slave NUMA nodes, the master NUMA node can send the task descriptions corresponding to the multiple slave NUMA nodes to the multiple slave NUMA nodes at the same time, and the subtask distribution efficiency is high. Alternatively, the master NUMA node can send the task descriptions corresponding to the slave NUMA nodes to the multiple slave NUMA nodes in sequence.

[0165] Wherein, for each NUMA node among the multiple NUMA nodes, a prefetch buffer module and an accelerator task execution module may be included. For the master NUMA node, the prefetch buffer module may be used to receive the task descriptions of multiple subtasks and distribute a subtask to the prefetch buffer module of each slave node. For each slave NUMA node, the prefetch buffer module may be used to receive the subtask sent from the prefetch buffer module of the master NUMA node. For the master NUMA node, the subtask to be executed may be cached in the hardware task queue of the master NUMA node, that is, the subtask obtained from the prefetch buffer module of the master NUMA node is queued, and the subtask is sent to the accelerator task execution module of the master NUMA node. The accelerator task execution module of the master NUMA node executes the subtask received from the prefetch buffer module. For the slave NUMA node, similar to the master NUMA node, its subtask to be executed may be cached in the hardware task queue of the slave NUMA node, that is, the subtask obtained from the prefetch buffer module of the NUMA node is queued, and the subtask is sent to the accelerator task execution module of the slave NUMA node. The accelerator task execution module of the slave NUMA node executes the subtask received from the pre-read cache module.

[0166] In this way, the master NUMA node and each slave NUMA node can execute the assigned subtasks at the same time. There is no need for cross-NUMA data access between the master NUMA node and the slave NUMA nodes, and between slave NUMA nodes, so high-performance execution of a single operator can be achieved in a multi-NUMA node scenario.

[0167] In some embodiments, the chip scheduling module 503 may also be used to control the synchronization of the execution status of the subtask from the NUMA node to the master NUMA node, where the execution status includes the start of execution, the end of execution, and abnormal termination.

[0168] Exemplary, reference Figure 8 , taking slave NUMA node 1 as an example, the task module to be executed in the slave NUMA node can also be used to send the execution status of the subtask to the task module to be executed of the master NUMA node after the slave accelerator task execution module learns the execution status of the subtask. The remaining slave NUMA nodes can send the execution status of the subtask to the master NUMA node similar to slave NUMA node 1. When the master NUMA node learns the execution status sent by each slave NUMA node, it can report the execution status of the AI ​​task to the CPU in a timely manner, so that the CPU can promptly understand the execution status of the currently issued AI task. Among them, the interaction between the CPU and the master NUMA node can also be understood as the interaction between the user-mode software stack and the master NUMA node.

[0169] In some embodiments, after multiple nodes complete the execution of multiple subtasks, the execution results of the subtasks may be stored in a local cache of the node, and the CPU may subsequently actively access the chip system to obtain the task execution results from the cache.

[0170] In some embodiments, when multiple CPUs have computing tasks to send to the chip system at the same time, if a node in the chip system receives multiple subtasks sent by multiple CPUs at the same time, the node can execute the subtasks sent by multiple CPUs at the same time. For example, the nodes in the chip system can execute a concurrency guarantee mechanism to ensure that when a node receives subtasks sent by multiple CPUs, the concurrent execution of multiple subtasks of multiple CPUs can be achieved. For example, a node can concurrently execute and process multiple subtasks sent by multiple CPUs through multiple streams.

[0171] In some embodiments, when multiple subtasks generated by multiple CPUs access the same address memory, the solution of the present application may be preferentially followed to select the node closest to the memory address in the chip system to execute multiple subtasks of multiple CPUs, and multiple subtasks of multiple CPUs are executed serially on the node. Alternatively, in some embodiments, when the processor executes a performance evaluation algorithm, the algorithm can be used to evaluate the execution time of operators that execute multiple subtasks of multiple CPUs in series on a single node, and compare it with the execution time of operators that execute the multiple subtasks concurrently on multiple nodes and access memory across nodes, and select the method with the best performance to execute the multiple subtasks. For example, a method with shorter operator execution time can be selected to execute multiple subtasks generated by multiple CPUs.

[0172] In some embodiments, before step 601, the present application may also first determine whether to split the computing task, that is, whether the computing task needs to be executed on multiple nodes, through the multi-node memory management and task splitting module 501 (CPU).

[0173] Therefore, if Fig. 9 The flowchart of a scheduling method of a chip system provided by an embodiment of the present application is shown. Before step 601, the following process may also be included.

[0174] 901. The processor determines a second operator execution time when a computing task is executed by a single node, where the second operator execution time is a maximum value of a data read and write time and an operator calculation time when the computing task is executed by a single node.

[0175] 902. If the execution time of the second operator is greater than or equal to the preset time threshold, when the processor sends the computing task to the chip system, the computing task is divided into multiple subtasks.

[0176] That is to say, the principle for determining whether to perform cross-node computing task splitting in this application may be based on an evaluation of the operator performance cost, where the operator performance cost may be the execution time of the operator.

[0177] In some embodiments, for example, the CPU may first evaluate the execution time of the second operator when a single NUMA node executes the AI ​​task, and the execution time of the second operator is: max (data read and write time, operator calculation time), that is, the maximum value of the data read and write time and the operator calculation time when the AI ​​task is executed by a single NUMA node. This is because the data read and write time and the operator calculation time are performed at the same time, so the maximum value of the two can be used as the time for the single NUMA node to complete the execution of the AI ​​task.

[0178] If the execution time of the second operator is greater than or equal to the preset time threshold, it can be considered that the execution time of the AI ​​task by a single NUMA node is too long, and it is determined that the AI ​​task needs to be split and multiple NUMA nodes need to execute multiple subtasks simultaneously.

[0179] like Fig.10The figure shows a schematic diagram for determining whether to split an AI task. If the execution time of the second operator when executing the AI ​​task by a single NUMA node is less than the preset time threshold, the CPU can determine that the number of NUMA nodes is 1, which is equivalent to not splitting the AI ​​task. The single NUMA node selection strategy can be executed to execute the AI ​​task on a single NUMA node. If the execution time of the second operator is greater than or equal to the preset time threshold, it can be determined that the number of NUMA nodes is greater than or equal to 2, which is equivalent to splitting the AI ​​task, and a multi-NUMA node selection strategy can be executed. For the specific multi-NUMA node selection strategy, please refer to the description in step 601.

[0180] In some embodiments, in step 602, for the implementation of mapping logical storage resources to physical storage resources, in order to ensure high memory utilization and locality, when determining the NUMA nodes of the N subtasks to be executed, it is also possible to determine on which N NUMA nodes the subtasks are to be executed and memory is to be requested based on the memory allocation mode. For example, the three modes provided under the CPU NUMA architecture can be referred to: the default / system setting / predetermined (default) mode, the preferred-new (new priority) mode, and the preferred-disuse (enable priority) mode.

[0181] Among them, the default mode can be understood as applying for memory on the AI ​​chip indicated by the specified NUMA node list when applying for memory and obtaining the physical memory address range. Or it can be understood as applying for memory on the specified die list. If the available memory size of the NUMA node on the specified die list meets the logical storage resources required for each subtask, determine the NUMA node that executes multiple subtasks on the specified die list and apply for the corresponding memory address range. If the remaining memory space of the NUMA node on the specified die list is insufficient, that is, the remaining memory space is less than the memory space size required for the subtask determined in the logical resources, the memory application fails, and there is no need to find a new die to apply for memory. At this point, it can be considered that the AI ​​task execution has failed.

[0182] The preferred-new mode can be understood as old data on old dies and new data on new dies. When determining N NUMA nodes, this application can select a new NUMA node that does not store data to perform N subtasks. For example, for a 4-die diagonal die, a neighbor die with a smaller cost can be selected as a candidate NUMA node for the N NUMA nodes, and memory can be applied for on the selected NUMA node.

[0183] The dedault mode can be understood as selecting the NUMA node corresponding to the initial die ID as a candidate NUMA node for N NUMA nodes and applying for memory based on the memory load balancing principle. If the memory of the NUMA node corresponding to the initial die ID is insufficient, the NUMA node corresponding to the die ID with a smaller distance can be selected as a candidate NUMA node for N NUMA nodes and memory can be applied for. The distance can reflect the speed of accessing the NUMA node. The smaller the distance, the faster the access speed. Here, the NUMA node with a smaller distance to the CPU can be selected as a candidate NUMA node for N NUMA nodes and memory can be applied for.

[0184] Of course, the above three modes are possible implementation methods for determining N NUMA nodes, and the present application does not limit the method for determining physical resources.

[0185] Therefore, in the chip system of the present application, when executing a computing task, the computing task can be divided into multiple subtasks, and the logical resources required for the multiple subtasks can be determined. In this way, when the logical resources are mapped to the physical resources and the physical resources of the multiple nodes to execute multiple subtasks are determined, the master node and at least one slave node can be selected in the chip system, and the master node performs the scheduling and distribution of multiple subtasks. In this way, the master node and each slave node only need to execute the distributed subtasks, and there is no need to perform cross-node data access when executing a single operator (single computing task). For example, for the NUMA architecture of multiple AI chips, the program programmed by the user for the CPU in this application can use the NUMA architecture of multiple AI chips like a single chip, that is, it can enable users to program in a single-chip manner, while ensuring the execution performance of multiple NUMA nodes, while improving the ease of use and compatibility of CPU programming, that is, while ensuring the execution performance of multiple NUMA nodes, programming is imperceptible, and high-performance execution of a single operator in a multi-NUMA node scenario is completed.

[0186] Moreover, compared with the prior art in which data is accessed across NUMA nodes, the NUMA nodes of the present application only have accesses across NUMA nodes when state synchronization is performed during the execution of subtasks, and the number of cross-NUMA node accesses is relatively small. Moreover, the data size for data access across NUMA nodes in the prior art is larger than the data size for state synchronization in the present application. When state synchronization is performed across NUMA nodes in the present application, for example, only 1 bit of data may be consumed to indicate a subtask, while the data size for data access across NUMA nodes in the prior art is often as large as a dozen bits, dozens of bits, or even hundreds of bits.

[0187] Furthermore, in the prior art, there may be a situation where the data address of the intermediate result of AI task execution on one NUMA node or multiple NUMA nodes is discontinuous, which makes the CPU frequently access data across NUMA nodes when accessing data, and the CPU takes a long time to access data in a single scattered manner. In the present application, after each NUMA node executes the received subtask and stores the data, the execution result of the subtask is only stored on this node, and the data address is continuous. When the CPU performs a single scattered access, it can access the data on one node, and the single scattered access takes a short time. In addition, the master NUMA node and the slave NUMA node of the present application only perform state synchronization, which can avoid the problem of frequent data access across NUMA nodes.

[0188] It is understandable that in order to implement the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of the present application.

[0189] In this embodiment, the electronic device can be divided into functional modules according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0190] In the case of dividing each functional module into corresponding functional modules, Fig.11 A possible schematic diagram of the composition of the electronic device 110 involved in the above embodiment is shown. Fig.11 As shown, the electronic device 110 may include: a task splitting unit 1101, a resource mapping unit 1102, and a task scheduling unit 1103. The task splitting unit 1101 may have the same function as the multi-NUMA memory management and task splitting module 501, the resource mapping unit 1102 may have the same function as the runtime task distribution module 502, and the task scheduling unit 1103 may have the same function as the chip scheduling module 503. The functions of the task splitting unit 1101 and the resource mapping unit 1102 may be performed by the CPU, and the function of the chip scheduling module 503 may be performed by the master NUMA node.

[0191] The task segmentation unit 1101 may be used to support the electronic device 110 in executing the above-mentioned steps 601, 901, 902, etc., and / or other processes of the technology described herein.

[0192] The resource mapping unit 1102 may be used to support the electronic device 110 in executing the above step 602 and / or other processes of the technology described herein.

[0193] The task scheduling unit 1103 may be used to support the electronic device 110 in executing the above step 603 and / or other processes of the technology described herein.

[0194] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0195] The electronic device 110 provided in this embodiment is used to execute the scheduling method of the above-mentioned chip system, and thus can achieve the same effect as the above-mentioned implementation method.

[0196] In the case of using integrated units, the electronic device 110 may include a processing module, a storage module and a communication module. Among them, the processing module can be used to control and manage the actions of the electronic device 110, for example, it can be used to support the electronic device 110 to perform the steps performed by the above-mentioned task segmentation unit 1101, resource mapping unit 1102 and task scheduling unit 1103. The storage module can be used to support the electronic device 1100 to store program codes and data, etc. The communication module can be used to support the communication between the electronic device 110 and other devices, such as communication with wireless access devices.

[0197] Among them, the processing module can be a processor or a controller. It can implement or execute various exemplary logic boxes, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a device that interacts with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, etc.

[0198] In one embodiment, when the processing module is a processor, the storage module is a memory, and the communication module is a transceiver, the electronic device involved in this embodiment can be a Fig.12 An electronic device 120 of the structure shown.

[0199] The embodiment of the present application also provides an electronic device, including one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, and the computer program codes include computer instructions. When the one or more processors execute the computer instructions, the electronic device executes the above-mentioned related method steps to implement the scheduling method of the chip system in the above-mentioned embodiment.

[0200] An embodiment of the present application also provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the scheduling method of the chip system in the above-mentioned embodiment.

[0201] An embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer executes the above-mentioned related steps to implement the scheduling method of the chip system executed by the electronic device in the above-mentioned embodiment.

[0202] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer execution instructions, and when the device is running, the processor can execute the computer execution instructions stored in the memory so that the chip executes the scheduling method of the chip system executed by the electronic device in the above-mentioned method embodiments.

[0203] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above and will not be repeated here.

[0204] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0205] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0206] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0207] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0208] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0209] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A scheduling method for a chip system, characterized in that: The chip system includes a plurality of bare chips, each bare chip is a node, and the method includes: When the processor sends a computing task to the chip system, the processor divides the computing task into a plurality of subtasks and determines the logic resources required by the plurality of subtasks; The processor determines a task description for executing the multiple subtasks by multiple nodes in the chip system, the multiple nodes including a master node and at least one slave node, and the task description corresponding to each subtask is used to indicate the subtask and the physical resources required to execute the subtask; The processor sends a task description corresponding to each of the multiple subtasks to the master node, wherein the multiple subtasks include subtasks of the master node and subtasks of the at least one slave node, and each subtask of the at least one slave node corresponds to one of the at least one slave node; The master node sends the task description of each subtask of the at least one slave node to a corresponding slave node of the at least one slave node.

2. A scheduling method for a chip system, characterized in that: The method comprises: When sending a computing task to the chip system, the computing task is divided into a plurality of subtasks, and logical resources required by the plurality of subtasks are determined; Determine task descriptions for executing the multiple subtasks respectively by multiple nodes in the chip system, the multiple nodes include a master node and at least one slave node, and the task description corresponding to each subtask is used to indicate the subtask and the physical resources required to execute the subtask; Sending a task description corresponding to each of the multiple subtasks to the master node, wherein the multiple subtasks include subtasks of the master node and subtasks of the at least one slave node, each subtask of the at least one slave node corresponds to one of the at least one slave nodes, and the task description of each subtask of the at least one slave node is sent to a corresponding one of the at least one slave node.

3. The method according to claim 1 or 2, characterized in that: The dividing the computing task into a plurality of subtasks comprises: The computing task is divided into the multiple sub-tasks according to the first operator execution time corresponding to the multiple task splitting methods of the computing task; wherein the first operator execution time corresponding to the first task splitting method of the computing task is used to indicate the time taken for a single node to execute a subtask under the first task splitting method, and the first task splitting method is any task splitting method.

4. The method according to claim 3, characterized in that The step of dividing the computing task into the plurality of subtasks according to the execution time consumption of the first operator respectively corresponding to the plurality of task division modes of the computing task comprises: Determine the execution time of the first operator corresponding to each of the plurality of task segmentation modes of the computing task; Determine a target task segmentation method in which the execution time of the first operator task satisfies a condition among the multiple task segmentation methods; The computing task is divided into the plurality of subtasks using the target task division method.

5. The method according to claim 4, characterized in that The number of the subtasks is N; N is an integer greater than or equal to 2; The execution time of the first operator includes: (1 / N)×the total cross-node data copy time of the computing task, (1 / N)×the total operator computing time of the computing task and the scheduling software and hardware time; The conditions include: (1 / N)×total cross-node data copy time of the computing task+scheduling software and hardware time<(1 / N)×total operator calculation time of the computing task.

6. The method according to any one of claims 1 to 5, characterized in that: The dividing the computing task into a plurality of subtasks comprises: Determine a second operator execution time when a single node executes the computing task, where the second operator execution time is a maximum value of a data reading and writing time and an operator calculation time when the single node executes the computing task; If the execution time of the second operator is greater than or equal to the time threshold, the step of dividing the computing task into a plurality of subtasks is performed when the processor sends the computing task to the chip system.

7. The method according to any one of claims 1 to 6, characterized in that: The processor determines that a plurality of nodes in the chip system execute a task description of the plurality of subtasks, including: Determine a plurality of nodes for executing the plurality of subtasks, and determine the physical resources required for the subtasks executed by each node according to the logical resources required for the plurality of subtasks; A task description of the subtask executed by each node is determined according to the multiple nodes and the physical resources required for the subtask executed by each node.

8. The method according to any one of claims 1 to 7, characterized in that: The task description of the subtask corresponding to the master node is used to instruct the master node to send the task description of each subtask in the subtask of the at least one slave node to one of the corresponding at least one slave node; Alternatively, the method further includes: the processor instructing the master node to send the task description of each subtask of the at least one slave node to a corresponding slave node among the at least one slave node.

9. The method according to any one of claims 1 to 8, characterized in that: Before sending the task descriptions respectively corresponding to the multiple subtasks to the master node, the method further includes: According to the load information of each node among the multiple nodes, select a node with the smallest load among the multiple nodes as the master node, and select nodes among the multiple nodes other than the master node as the at least one slave node; The load information is used to indicate the load level of the node.

10. The method according to claim 9, characterized in that For each node, the load information includes at least one of: the task load on the current task execution resources of the node, the length of the task queue waiting to be executed, and historical load statistics.

11. The method according to any one of claims 1 or 3 to 10, characterized in that: The method further comprises: The at least one slave node synchronizes the execution status of the subtask to the master node, where the execution status includes starting execution, ending execution, or abnormal termination.

12. A scheduling method for a chip system, characterized in that: The chip system includes a plurality of bare chips, each bare chip is a node, and the method includes: Receive task descriptions corresponding to multiple subtasks sent by a processor, where the task description corresponding to each subtask is used to indicate the subtask and physical resources required to execute the subtask, and the multiple subtasks are obtained by dividing the computing task by the processor; Among them, the multiple subtasks include subtasks of the master node and subtasks of the at least one slave node, each subtask of the at least one slave node corresponds to one of the at least one slave nodes, and the task description of each subtask of the at least one slave node is sent to a corresponding one of the at least one slave node respectively.

13. The method according to claim 12, characterized in that The task description of the subtask corresponding to the master node indicates that the master node sends the task description of each subtask in the subtask of the at least one slave node to one of the corresponding at least one slave node; The method further includes: receiving an instruction from the processor, wherein the instruction from the processor is used to instruct the master node to send a task description of each subtask of the at least one slave node to a corresponding one of the at least one slave node.

14. The method according to claim 12 or 13, characterized in that The execution status of the subtask of the at least one slave node is synchronized to the master node, and the execution status includes starting execution, ending execution, or abnormal termination.

15. A scheduling device, characterized in that: The scheduling device comprises: A task division unit, used for dividing a computing task into a plurality of subtasks when sending the computing task to the chip system, and determining the logic resources required by the plurality of subtasks; a resource mapping unit, configured to determine task descriptions for executing the multiple subtasks by multiple nodes in the chip system, wherein the multiple nodes include a master node and at least one slave node, and the task description corresponding to each subtask is used to indicate the subtask and the physical resources required to execute the subtask; A task scheduling unit is used to send a task description corresponding to each of the multiple subtasks to the master node, wherein the multiple subtasks include subtasks of the master node and subtasks of the at least one slave node, each subtask of the at least one slave node corresponds to one of the at least one slave nodes, and the task description of each subtask of the at least one slave node is sent to a corresponding one of the at least one slave node.

16. The dispatching device according to claim 15, characterized in that: The task segmentation unit is used for: The computing task is divided into the multiple sub-tasks according to the first operator execution time corresponding to the multiple task splitting methods of the computing task; wherein the first operator execution time corresponding to the first task splitting method of the computing task is used to indicate the time taken for a single node to execute a subtask under the first task splitting method, and the first task splitting method is any task splitting method.

17. The dispatching device according to claim 16, characterized in that: The task segmentation unit is used for: Determine the execution time of the first operator corresponding to each of the plurality of task segmentation modes of the computing task; Determine, among the multiple task splitting methods, a target task splitting method in which the execution time of the first operator task satisfies a condition; and divide the computing task into the multiple subtasks using the target task splitting method.

18. The dispatching device according to claim 17, characterized in that: The number of the subtasks is N; N is an integer greater than or equal to 2; The execution time of the first operator includes: (1 / N)×the total cross-NUMA data copy time of the computing task, (1 / N)×the total operator computing time and scheduling software and hardware time of the computing task; The conditions include: (1 / N)×total cross-node data copy time of the computing task+scheduling software and hardware time<(1 / N)×total operator calculation time of the computing task.

19. The dispatching device according to any one of claims 15 to 18, characterized in that: The task scheduling unit is further used for: Determine a second operator execution time when a single node executes the computing task, where the second operator execution time is a maximum value of a data reading and writing time and an operator calculation time when the single node executes the computing task; If the execution time of the second operator is greater than or equal to the time threshold, the step of dividing the computing task into multiple subtasks is performed when the processor sends the computing task to chips of multiple non-uniform memory access nodes.

20. The dispatching device according to any one of claims 15 to 19, characterized in that: The resource mapping unit is used for: Determine a plurality of nodes for executing the plurality of subtasks, and determine the physical resources required for the subtasks executed by each node according to the logical resources required for the plurality of subtasks; A task description of the subtask executed by each node is determined according to the multiple nodes and the physical resources required for the subtask executed by each node.

21. The dispatching device according to any one of claims 15 to 20, characterized in that: The task description of the subtask corresponding to the master node is used to instruct the master node to send the task description of each subtask in the subtask of the at least one slave node to one of the corresponding at least one slave node; Alternatively, the task scheduling unit is further used to: instruct the master node to send the task description of each subtask of the at least one slave node to a corresponding slave node among the at least one slave node.

22. The dispatching device according to any one of claims 15 to 21, characterized in that: The task scheduling unit is also used for: According to the load information of each node among the multiple nodes, select a node with the smallest load among the multiple nodes as the master node, and select nodes among the multiple nodes other than the master node as the at least one slave node; The load information is used to indicate the load level of the node.

23. The dispatching device according to claim 22, characterized in that: For each node, the load information includes at least one of: the task load on the current task execution resources of the node, the length of the task queue waiting to be executed, and historical load statistics.

24. A chip system, characterized in that: The chip system includes a plurality of bare chips, each bare chip is a node, and the method includes: A receiving unit, used to receive task descriptions corresponding to a plurality of subtasks sent by a processor, wherein the task description corresponding to each subtask is used to indicate the subtask and the physical resources required to execute the subtask, wherein the plurality of subtasks are obtained by dividing the computing task by the processor, and the plurality of subtasks are executed by a master node and at least one slave node in the chip system; Among them, the multiple subtasks include subtasks of the master node and subtasks of the at least one slave node, each subtask of the at least one slave node corresponds to one of the at least one slave nodes, and the task description of each subtask of the at least one slave node is sent to a corresponding one of the at least one slave node respectively.

25. The chip system according to claim 24, characterized in that: The task description of the subtask corresponding to the master node indicates that the master node sends the task description of each subtask in the subtask of the at least one slave node to one of the corresponding at least one slave node; Alternatively, the receiving unit is further used to: receive an instruction from the processor, wherein the instruction from the processor is used to instruct the master node to send the task description of each subtask of the at least one slave node to a corresponding one of the at least one slave node.

26. The chip system according to claim 24 or 25, characterized in that: The execution status of the subtask of the at least one slave node is synchronized to the master node, and the execution status includes starting execution, ending execution, or abnormal termination.

27. A scheduling device, characterized in that: The scheduling device includes a processor and a chip system, wherein the chip system includes a plurality of bare chips, each bare chip being a node, wherein: The processor is used to divide the computing task into multiple subtasks when sending the computing task to the chip system, and determine the logical resources required by the multiple subtasks; Determine task descriptions for executing the multiple subtasks by multiple nodes in the chip system, the multiple nodes including a master node and at least one slave node, the task description corresponding to each subtask being used to indicate the subtask and the physical resources required to execute the subtask; Sending a task description corresponding to each of the multiple subtasks to the master node, wherein the multiple subtasks include subtasks of the master node and subtasks of the at least one slave node, and each subtask of the at least one slave node corresponds to one of the at least one slave node; The master node is used to send the task description of each subtask of the at least one slave node to a corresponding slave node among the at least one slave node.

28. A chip, characterized in that: The chip stores computer-executable instructions, and when the computer-executable instructions are executed, the method described in any one of claims 1 to 14 is executed.

29. A computer-readable storage medium, characterized in that: The method comprises computer instructions, which, when executed on an electronic device, cause the electronic device to execute the method according to any one of claims 1 to 14.

Images