Scheduling methods, devices and electronic equipment for artificial intelligence chips

Abstract

This disclosure provides a scheduling method, apparatus, and electronic device for artificial intelligence chips, relating to the field of artificial intelligence technology, particularly deep learning and chip technology. The method involves: in response to at least one operator data being written into the first memory space of a first processor, notifying the artificial intelligence chip to execute an operator calculation process based on the operator data written into the first memory space; and in response to the completion of the operator data execution in the first memory space, and if no operator data has been written into the first memory space within a preset time period, initiating an interrupt to the second processor. This avoids the artificial intelligence chip being idle while the second processor is writing operator data into the second memory space, reducing waiting time and latency, thereby improving the computation speed of the neural network.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more particularly to the fields of deep learning and chip technology, and more particularly to scheduling methods, apparatus and electronic devices for artificial intelligence chips. Background Technology

[0002] Currently, in the field of artificial intelligence, neural networks consist of a large number of computationally intensive operations, which are executed by new AI chips. The smallest granularity of a neural network is an operator, which interacts with the AI ​​chip at this granularity.

[0003] In related technologies, the scheduling method of artificial intelligence chips mainly involves using system calls to batch write operator commands and operator parameters (i.e., operator data) into the configuration register to initiate a request. After receiving the calculation request, the artificial intelligence chip starts the calculation. After the calculation is completed, it sends an interrupt to the CPU (processor). After receiving the interrupt, the CPU indicates that the operation has been completed and then executes the subsequent operators, repeating the calling process.

[0004] In the above scheme, the AI ​​chip is idle during the process of issuing operator commands and operator parameters in batches, which increases the latency. Summary of the Invention

[0005] This disclosure provides a scheduling method, apparatus, and electronic device for artificial intelligence chips.

[0006] According to one aspect of this disclosure, a scheduling method for an artificial intelligence chip is provided, applied to a first processor. The method includes: in response to at least one operator data being written into a first memory space of the first processor, notifying the artificial intelligence chip to perform an operator calculation process based on the operator data written into the first memory space; wherein, there is a mapping relationship between the first memory space and a second memory space of a second processor; the second processor writes operator data into the second memory space by writing a single batch of operator data at a time; in response to the completion of the execution of the operator data in the first memory space, and no operator data being written into the first memory space within a preset time period, initiating an interrupt to the second processor to notify the second processor to write the operator data to be processed into the second memory space.

[0007] According to another aspect of this disclosure, a scheduling method for an artificial intelligence chip is provided, applied to a second processor. The method includes: writing a single batch of operator data into a second memory space of the second processor, wherein there is a mapping relationship between the second memory space and a first memory space of a first processor; the operator data in the first memory space is used by the first processor to notify the artificial intelligence chip to perform an operator calculation process; in response to receiving an interrupt initiated by the first processor, writing the next batch of operator data into the second memory space, wherein the interrupt is initiated by the first processor completing the execution of the operator data in the first memory space, and no operator data has been written into the first memory space within a preset time period.

[0008] According to another aspect of this disclosure, a scheduling method for an artificial intelligence chip is provided, applied to a scheduling system for an artificial intelligence chip. The method includes: a second processor in the scheduling system writing a single batch of operator data into a second memory space of the second processor; wherein, there is a mapping relationship between the second memory space and a first memory space of a first processor in the scheduling system; the first processor, in response to at least one operator data being written into the first memory space, notifying the artificial intelligence chip in the scheduling system to perform an operator calculation process based on the operator data written into the first memory space; the artificial intelligence chip performing the operator calculation process based on the operator data written into the first memory space; the first processor, in response to the completion of the operator data execution in the first memory space and no operator data being written into the first memory space within a preset time period, initiating an interrupt to the second processor; the second processor, in response to receiving the interrupt initiated by the first processor, writing the next batch of operator data into the second memory space.

[0009] According to another aspect of this disclosure, a scheduling device for an artificial intelligence chip is provided, applied to a first processor. The device includes: a notification module, configured to notify the artificial intelligence chip to perform an operator calculation process based on the operator data written in the first memory space in response to at least one operator data being written in the first memory space of the first processor; wherein, there is a mapping relationship between the first memory space and the second memory space of a second processor; the second processor writes operator data in the second memory space by writing a single batch of operator data at a time; and an interrupt module, configured to initiate an interrupt to the second processor in response to the completion of the operator data execution in the first memory space and the absence of operator data written in the first memory space within a preset time period, to notify the second processor to write the operator data to be processed in the second memory space.

[0010] According to another aspect of this disclosure, a scheduling device for an artificial intelligence chip is provided, applied to a second processor. The device includes: a writing module, configured to write a single batch of operator data into a second memory space of the second processor, wherein there is a mapping relationship between the second memory space and a first memory space of a first processor; the operator data in the first memory space is used by the first processor to notify the artificial intelligence chip to perform an operator calculation process; the writing module is further configured to write the next batch of operator data into the second memory space upon receiving an interrupt initiated by the first processor, wherein the operator data in the first memory space is used by the first processor to notify the artificial intelligence chip to perform an operator calculation process.

[0011] According to another aspect of this disclosure, a scheduling system for an artificial intelligence chip is provided, the system comprising: a first processor, a second processor, and an artificial intelligence chip; wherein the second processor is connected to the first processor, and there is a mapping relationship between a second memory space of the second processor and a first memory space of the first processor; the first processor is connected to the artificial intelligence chip; the system applies the scheduling method for the artificial intelligence chip as proposed in yet another aspect.

[0012] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0013] At least one processor; and

[0014] A memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor, which are executed by the at least one processor to enable the at least one processor to execute the scheduling method for an artificial intelligence chip proposed in one aspect of this disclosure, or the scheduling method for an artificial intelligence chip proposed in another aspect, or the scheduling method for an artificial intelligence chip proposed in yet another aspect.

[0016] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided that stores computer instructions for causing the computer to execute the scheduling method for an artificial intelligence chip proposed in one aspect of this disclosure, or the scheduling method for an artificial intelligence chip proposed in another aspect, or the scheduling method for an artificial intelligence chip proposed in yet another aspect.

[0017] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the scheduling method for an artificial intelligence chip proposed in one aspect of this disclosure, or the steps of the scheduling method for an artificial intelligence chip proposed in another aspect, or the steps of the scheduling method for an artificial intelligence chip proposed in yet another aspect.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0019] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0020] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure;

[0021] Figure 2 This is a schematic diagram according to the second embodiment of the present disclosure;

[0022] Figure 3 This is a schematic diagram according to the third embodiment of the present disclosure;

[0023] Figure 4 This is a schematic diagram of an artificial intelligence chip architecture;

[0024] Figure 5 This is a schematic diagram according to the fourth embodiment of the present disclosure;

[0025] Figure 6 This is a schematic diagram according to the fifth embodiment of the present disclosure;

[0026] Figure 7 This is a schematic diagram according to the sixth embodiment of the present disclosure;

[0027] Figure 8 This is a block diagram of an electronic device used to implement embodiments of the present disclosure. Detailed Implementation

[0028] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0029] Currently, in the field of artificial intelligence, neural networks consist of a large number of computationally intensive operations, such as matrix multiplication, convolution, pooling, and activation. These operations are performed by new artificial intelligence chips. Examples of artificial intelligence chips include GPUs (Graphics Processing Units), FPGAs (Field-Programmable Gate Arrays), and ASICs (Application Specific Integrated Circuits). Communication between artificial intelligence chips and CPUs (processors) is generally achieved via PCIe (Peripheral Component Interconnect Express), with the artificial intelligence chip acting as a coprocessor for the CPU. The smallest granularity of a neural network is the operator, which interacts with the artificial intelligence chip at this level.

[0030] In related technologies, the scheduling method for artificial intelligence chips mainly involves using system calls to batch write operator commands and operator parameters (i.e., operator data) into the configuration register to initiate a request. After receiving the computation request, the artificial intelligence chip begins computation. Upon completion of the computation, it sends an interrupt to the CPU (processor). Upon receiving the interrupt, the CPU indicates that the operation has been completed and then executes subsequent operators, repeating the call process.

[0031] In the above scheme, the AI ​​chip is idle during the process of issuing operator commands and operator parameters in batches, which increases the latency.

[0032] To address the aforementioned issues, this disclosure proposes a scheduling method, apparatus, and electronic device for artificial intelligence chips.

[0033] It should be noted that the scheduling method for artificial intelligence chips in this disclosure can be applied to a scheduling device for artificial intelligence chips. The scheduling device for artificial intelligence chips in this disclosure can be, for example, a processor to perform the scheduling function of artificial intelligence chips.

[0034] It should be noted that, for a specific processor, the processor can be a first processor that writes operator data to the first memory space; or, it can be a second processor that writes operator data to the second memory space.

[0035] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure, as shown below. Figure 1 As shown, the scheduling method for the artificial intelligence chip is applied to the first processor, and the scheduling method for the artificial intelligence chip may include the following steps:

[0036] Step 101: In response to the fact that at least one operator data has been written into the first memory space of the first processor, the artificial intelligence chip is notified to perform the operator calculation process according to the operator data written into the first memory space; wherein, there is a mapping relationship between the first memory space and the second memory space of the second processor; the second processor writes operator data into the second memory space by writing a single batch of operator data at a time.

[0037] In this embodiment of the disclosure, the first processor may execute step 101 for example, by notifying the artificial intelligence chip to perform an operator calculation process based on the operator data written in the first memory space when a single operator data is written in the first memory space, or when a first number of operator data is written in the first memory space; wherein the first number is less than the number of operator data in a single batch of operator data.

[0038] The operator data may include operator commands and operator parameters. Specifically, when a single piece of operator data is written into the first memory space, the first processor can instruct the AI ​​chip to perform operator calculations based on the operator data written into the first memory space.

[0039] Specifically, when a first number of operator data less than a single batch is written into the first memory space, the artificial intelligence chip is notified to execute the operator calculation process.

[0040] When at least one operator data is written into the first memory space of the first processor, the artificial intelligence chip can be notified to execute the operator calculation process without waiting for the batch of operator data to be written. This avoids the artificial intelligence chip being idle while the second processor is writing operator data into the configuration register, reducing waiting time and latency, thereby improving the calculation speed of the neural network.

[0041] The memory space can include user-mode memory space and kernel-mode memory space. The second memory space is located within the user-mode memory space of the second processor, and the second processor has the permission to read and write to the second memory space using non-system instructions.

[0042] If the second memory space is located in the kernel-mode memory space, the second processor does not have permission to read or write to it; only the operating system has this permission. Therefore, system calls, i.e., system instructions, are required to read and write to the second memory space.

[0043] The second memory space, which uses the user-mode memory space, has the permission to read and write to the second memory space through non-system instructions. This allows the processor to read and write to the second memory space without needing to use system instructions, thereby reducing scheduling overhead.

[0044] There is a mapping relationship between the second memory space and the first memory space of the first processor. In other words, the second memory space of the second processor and the first memory space of the first processor can be mapped to the same physical storage space.

[0045] The first memory space also contains an execution graph, which is used to represent the execution order among multiple operator data in a single batch of operator data.

[0046] In the process of calculating and executing multiple operator data, the execution of later operators will use the calculation results of earlier operators, and the operator data written first is the operator data to be calculated first.

[0047] The first memory space contains an execution graph, which represents the execution order of multiple operator data in a single batch of operator data. This allows the operator data in a single batch to be written in the order of execution. Once the entire batch of operator data has been executed, an interrupt is sent, reducing interrupt overhead and thus reducing scheduling overhead.

[0048] The first processor can be a device processor, such as ARM (Advanced RISC Microprocessors) or RISC-V (Instruction Set Architecture), which is low-cost and has powerful functions.

[0049] The artificial intelligence chip and the first processor share the first memory space.

[0050] The AI ​​chip and the first processor share the first memory space. This means that the operator data synchronized in the first memory space does not need to be transmitted to the AI ​​chip again, making the operator's calculation process more flexible and reducing time consumption.

[0051] Step 102: In response to the completion of operator data execution in the first memory space and the absence of operator data written to the first memory space within a preset time period, an interrupt is initiated to the second processor to notify the second processor to write the operator data to be processed in the second memory space.

[0052] The scheduling method for an artificial intelligence chip according to this embodiment of the present disclosure, in response to the writing of at least one operator data into the first memory space of a first processor, notifies the artificial intelligence chip to perform an operator calculation process based on the operator data written into the first memory space; wherein, there is a mapping relationship between the first memory space and the second memory space of a second processor; the second processor writes operator data into the second memory space in a single batch at a time; in response to the completion of the execution of the operator data in the first memory space, and if no operator data is written into the first memory space within a preset time period, an interrupt is initiated to the second processor to notify the second processor to write the operator data to be processed into the second memory space. Therefore, if operator data is written into the first memory space of the first processor, the artificial intelligence chip is notified to perform an operator calculation process based on the operator data written into the first memory space, thereby avoiding the artificial intelligence chip being idle during the process of writing operator data into the first memory space, reducing waiting time and latency, and thus improving the calculation speed of the neural network.

[0053] Figure 2 This is a schematic diagram based on the second embodiment of the present disclosure, as shown below. Figure 2 As shown, the scheduling method for the artificial intelligence chip is applied to the second processor, and the scheduling method for the artificial intelligence chip may include the following steps:

[0054] Step 201: Write a single batch of operator data into the second memory space of the second processor. There is a mapping relationship between the second memory space and the first memory space of the first processor. The operator data in the first memory space is used by the first processor to notify the artificial intelligence chip to perform the operator calculation process.

[0055] The second processor writes a single batch of operator data into the second memory space, that is, writes operator commands and operator parameters for multiple operators.

[0056] The second memory space is located in the user-mode memory space of the second processor, and the second processor has the right to read and write to the second memory space through non-system instructions.

[0057] The second memory space, which uses the user-mode memory space, has the permission to read and write to the second memory space through non-system instructions. This allows the processor to read and write to the second memory space without needing to use system instructions, thereby reducing scheduling overhead.

[0058] There is a mapping relationship between the second memory space and the first memory space of the first processor. That is, the second memory space of the second processor and the first memory space of the first processor can be mapped to the same physical storage space. When operator data is written in the second memory space, the operator data is actually written to the physical storage space. When data is read from the first memory space, the data read is the data in the physical storage space.

[0059] The second processor writes an execution graph into the second memory space, which is used to represent the execution order among multiple computational data.

[0060] In the process of calculating and executing multiple operator data, the execution of later operators will use the calculation results of earlier operators, and the operator data written first is the operator data to be calculated first.

[0061] The second processor writes the execution graph into the second memory space, which is the execution order of multiple operator data in a single batch of operator data. This allows the second processor to write the single batch of operator data one by one in the execution order. Once the entire batch of operator data has been executed, the second processor only needs to send one interrupt, reducing interrupt overhead and thus reducing scheduling overhead.

[0062] The second processor can be the Host processor.

[0063] Step 202: In response to receiving an interrupt initiated by the first processor, write the next batch of operator data into the second memory space. The interrupt is initiated when the first processor completes the execution of the operator data in the first memory space and no operator data has been written into the first memory space within a preset time period.

[0064] When an interrupt is received from the first processor, it indicates that the operator data in the configuration register has been executed and no operator data has been written to the first memory space within a preset time period. The next batch of operator data can then be written to the second memory space. The AI ​​chip then executes the operator data calculation process in a loop.

[0065] The scheduling method for an artificial intelligence chip according to this embodiment of the present disclosure involves writing a single batch of operator data into the second memory space of a second processor. A mapping relationship exists between the second memory space and the first memory space of a first processor. The operator data in the first memory space is used by the first processor to notify the artificial intelligence chip to perform operator calculations. In response to an interrupt initiated by the first processor, the next batch of operator data is written into the second memory space. The interrupt is initiated when the first processor completes the execution of the operator data in the first memory space and no operator data has been written into the first memory space within a preset time period. Therefore, writing operator data into the second memory space of the second processor and writing the next batch of operator data in response to an interrupt initiated by the first processor notifies the artificial intelligence chip to perform operator calculations based on the written operator data. This avoids the artificial intelligence chip being idle while the second processor is writing operator data into the second memory space, reducing waiting time and latency, thereby improving the computation speed of the neural network.

[0066] Figure 3 This is a schematic diagram based on the third embodiment of the present disclosure, as shown below. Figure 3 As shown, the scheduling method for artificial intelligence chips is applied to the scheduling system of artificial intelligence chips. The scheduling method for artificial intelligence chips may include the following steps:

[0067] Step 301: The second processor in the scheduling system writes a single batch of operator data into the second memory space of the second processor; wherein, there is a mapping relationship between the second memory space and the first memory space of the first processor in the scheduling system.

[0068] The second memory space is located in the user-mode memory space of the second processor, and the second processor has the right to read and write to the second memory space through non-system instructions.

[0069] The second memory space, which uses the user-mode memory space, has the permission to read and write to the second memory space through non-system instructions. This allows the processor to read and write to the second memory space without needing to use system instructions, thereby reducing scheduling overhead.

[0070] Step 302: The first processor, in response to the fact that at least one operator data has been written into the first memory space, notifies the artificial intelligence chip in the scheduling system to perform the operator calculation process based on the operator data written into the first memory space.

[0071] Step 303: The artificial intelligence chip performs operator calculations based on the operator data written in the first memory space.

[0072] The artificial intelligence chip and the first processor share the first memory space.

[0073] The AI ​​chip and the first processor share the first memory space. This means that the operator data synchronized in the first memory space does not need to be transmitted to the AI ​​chip again, making the operator's calculation process more flexible and reducing time consumption.

[0074] Step 304: The first processor, in response to the completion of the execution of operator data in the first memory space and the fact that no operator data has been written to the first memory space within a preset time period, initiates an interrupt to the second processor.

[0075] Step 305: The second processor, in response to receiving an interrupt initiated by the first processor, writes the next batch of operator data into the second memory space.

[0076] In some embodiments, the second processor writes an execution graph into the second memory space. The execution graph is used to characterize the execution order among multiple operator data in a single batch of operator data. The artificial intelligence chip performs operator calculations on the operator data written in the first memory space according to the execution graph.

[0077] In the process of calculating and executing multiple operator data, the execution of later operators will use the calculation results of earlier operators, and the operator data written first is the operator data to be calculated first.

[0078] The second memory space contains an execution graph, which represents the execution order of multiple operator data in a single batch of operator data. This allows the operator data in a single batch to be written in the order of execution. When the entire batch of operator data has been executed, the first processor sends an interrupt to reduce interrupt overhead and thus reduce scheduling overhead.

[0079] The scheduling method for an artificial intelligence chip according to this embodiment involves a second processor in the scheduling system writing a single batch of operator data into the second memory space of the second processor. The second memory space is mapped to the first memory space of the first processor in the scheduling system. The first processor, in response to the presence of at least one operator data written into the first memory space, notifies the artificial intelligence chip in the scheduling system to perform an operator calculation process based on the operator data written into the first memory space. The artificial intelligence chip performs the operator calculation process based on the operator data written into the first memory space. The first processor, in response to the completion of the operator data execution in the first memory space and the absence of operator data written into the first memory space within a preset time period, initiates an interrupt to the second processor. The second processor, in response to receiving the interrupt from the first processor, writes the next batch of operator data into the second memory space to notify the artificial intelligence chip to perform the operator calculation process based on the written operator data. This avoids the artificial intelligence chip being idle while the first processor is writing operator data into the first memory space, reducing waiting time and latency, thereby improving the computational speed of the neural network.

[0080] For example, Figure 4 This is a schematic diagram of an artificial intelligence chip architecture. Figure 4 In this approach, a dedicated CPU can be configured within the AI ​​chip, utilizing low-cost ARM (Advanced RISC Machines, RISC microprocessors) or RISC-V (instruction set architecture). The scheduling method for the AI ​​chip primarily comprises three stages: initialization, graph construction, and computation. During initialization, the PCIe configuration space base register (BAR) is used. Specifically, through the PCIe BAR, a segment of memory (second memory space) from the Host CPU (second processor) is mapped to a segment of memory (first memory space) of the Device CPU within the AI ​​chip. Subsequent operation commands and parameters (operator data) are issued through this memory segment, recorded in the configuration register. The Device CPU then monitors this memory segment; if operator data is written, it notifies the AI ​​chip to begin the operator computation process.

[0081] During the graph construction and computation phase, 1) the Host CPU writes operator data into the configuration register of the second memory space to construct an execution graph; 2) during the graph construction process, if the Device CPU detects operator data being written, it will synchronously begin computation; 3) after the Host CPU has written all operator commands and parameters, it returns, can perform other operations, and waits for an interrupt from the AI ​​chip; 4) the Device CPU listens to whether the operator data has been executed. If it has been executed, it sends an interrupt to the Host CPU; 5) after receiving the interrupt from the Device CPU, the Host CPU indicates that the operation is complete and jumps to 1) to continue executing the next batch of operator data writing operations.

[0082] To implement the above embodiments, this disclosure also proposes a scheduling device for an artificial intelligence chip, applied to a first processor.

[0083] Figure 5 This is a schematic diagram based on the fourth embodiment of the present disclosure, as shown below. Figure 5 As shown, the scheduling device 500 of the artificial intelligence chip includes: a notification module 510 and an interrupt module 520;

[0084] The notification module 510 is used to notify the artificial intelligence chip to perform an operator calculation process based on the operator data written in the first memory space in response to the fact that at least one operator data has been written in the first memory space of the first processor; wherein, there is a mapping relationship between the first memory space and the second memory space of the second processor; the second processor writes operator data in the second memory space by writing a single batch of operator data at a time;

[0085] The interrupt module 520 is used to initiate an interrupt to the second processor in response to the completion of the execution of operator data in the first memory space and the fact that no operator data has been written to the first memory space within a preset time period, so as to notify the second processor to write the operator data to be processed in the second memory space.

[0086] As one possible implementation of this disclosure, the second memory space is located in the user-mode memory space of the second processor, and the second processor has the permission to read and write the second memory space through non-system instructions.

[0087] As one possible implementation of this disclosure, the first memory space is further written with an execution graph, which is used to characterize the execution order among multiple operator data in a single batch of operator data.

[0088] As one possible implementation of this disclosure, the notification module 510 is specifically used to notify the artificial intelligence chip to perform an operator calculation process based on the operator data written in the first memory space when a single operator data is written in the first memory space, or when a first number of operator data is written in the first memory space; wherein, the first number is less than the number of operator data in a single batch of operator data.

[0089] As one possible implementation of this disclosure, the artificial intelligence chip and the first processor share the first memory space.

[0090] The scheduling device for the artificial intelligence chip in this embodiment of the present disclosure, in response to the writing of at least one operator data into the first memory space of the first processor, notifies the artificial intelligence chip to perform an operator calculation process based on the operator data written into the first memory space. A mapping relationship exists between the first memory space and the second memory space of the second processor. The second processor writes operator data into the second memory space in a single batch. In response to the completion of operator data processing in the first memory space, and if no operator data is written into the first memory space within a preset time period, an interrupt is initiated to the second processor to notify it to write unprocessed operator data into the second memory space. Therefore, when operator data is written into the first memory space of the first processor, the artificial intelligence chip is notified to perform an operator calculation process based on the written operator data, thereby avoiding the artificial intelligence chip being idle during the process of writing operator data into the first memory space, reducing waiting time and latency, and thus improving the computation speed of the neural network.

[0091] To implement the above embodiments, this disclosure also proposes a scheduling device for an artificial intelligence chip, applied to a second processor.

[0092] Figure 6 This is a schematic diagram based on the fifth embodiment of the present disclosure, as shown below. Figure 6 As shown, the scheduling device 600 of the artificial intelligence chip includes: a writing module 610;

[0093] The writing module 610 is used to write a single batch of operator data into the second memory space of the second processor, wherein there is a mapping relationship between the second memory space and the first memory space of the first processor; the operator data in the first memory space is used by the first processor to notify the artificial intelligence chip to perform the operator calculation process.

[0094] The writing module 610 is further configured to write the next batch of operator data into the second memory space when receiving an interrupt initiated by the first processor, wherein the interrupt is initiated by the first processor executing the operator data in the first memory space and when no operator data has been written into the first memory space within a preset time period.

[0095] As one possible implementation of this disclosure, the second memory space is located in the user-mode memory space of the second processor, and the second processor has the permission to read and write the second memory space through non-system instructions.

[0096] As one possible implementation of this disclosure, the writing module 610 is further configured to write an execution graph into the second memory space, the execution graph being used to characterize the execution order among the plurality of computational data.

[0097] The scheduling device for the artificial intelligence chip in this embodiment of the present disclosure writes a single batch of operator data into the second memory space of the second processor. The second memory space has a mapping relationship with the first memory space of the first processor. The operator data in the first memory space is used by the first processor to notify the artificial intelligence chip to perform operator calculations. In response to receiving an interrupt initiated by the first processor, the next batch of operator data is written into the second memory space. The interrupt is initiated when the first processor completes the execution of the operator data in the first memory space and no operator data has been written into the first memory space within a preset time period. Therefore, writing operator data into the second memory space of the second processor and writing the next batch of operator data in response to an interrupt initiated by the first processor notifies the artificial intelligence chip to perform operator calculations based on the written operator data. This avoids the artificial intelligence chip being idle while the second processor is writing operator data into the second memory space, reducing waiting time and latency, thereby improving the computation speed of the neural network.

[0098] Figure 7 This is a schematic diagram based on the sixth embodiment of the present disclosure, as shown below. Figure 7 As shown, this disclosure provides a scheduling system for an artificial intelligence chip. The scheduling system 700 includes: a first processor 710, a second processor 720, and an artificial intelligence chip 730; wherein, the second processor 720 is connected to the first processor 710, and there is a mapping relationship between the second memory space of the second processor 720 and the first memory space of the first processor 710; the first processor 710 is connected to the artificial intelligence chip 730; the system 700 is used to execute the method described in the third embodiment of this disclosure.

[0099] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision, and disclosure of users' personal information are all carried out with the consent of the users, and all comply with the provisions of relevant laws and regulations, and do not violate public order and good morals.

[0100] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0101] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0102] like Figure 8 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0103] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0104] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the scheduling method for an AI chip. For example, in some embodiments, the scheduling method for an AI chip can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the scheduling method for an AI chip described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the scheduling method for an AI chip by any other suitable means (e.g., by means of firmware).

[0105] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0106] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0107] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0108] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0109] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0110] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0111] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0112] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A scheduling method of an artificial intelligence chip, applied to a first processor, the method comprising: in response to at least one operator data being written in a first memory space of the first processor, notifying the artificial intelligence chip to perform an operator computing process according to the operator data written in the first memory space; wherein the first memory space has a mapping relationship with a second memory space of a second processor; the second processor writes a single batch of operator data at a time in the second memory space; in response to the execution of the operator data in the first memory space being completed and no operator data being written in the first memory space within a preset time period, initiating an interrupt to the second processor to notify the second processor to write the operator data to be processed in the second memory space.

2. The method of claim 1, wherein, The second memory space is located in the memory space of the user state of the second processor, and the second processor has the permission to read and write the second memory space through non-system instructions.

3. The method of claim 1, wherein, The first memory space also has an execution graph written therein, and the execution graph is used to represent the execution order between a plurality of operator data in a single batch of operator data.

4. The method of claim 1, wherein, The response to at least one operator data being written in the first memory space, notifying the artificial intelligence chip to perform an operator computing process according to the operator data written in the first memory space, comprises: in response to a single operator data being written in the first memory space, or a first number of operator data being written in the first memory space, notifying the artificial intelligence chip to perform an operator computing process according to the operator data written in the first memory space; wherein the first number is less than the number of operator data in a single batch of operator data.

5. The method of claim 1, wherein, The artificial intelligence chip and the first processor share the first memory space.

6. A scheduling method of an artificial intelligence chip, applied to a second processor, the method comprising: writing a single batch of operator data in a second memory space of the second processor; wherein the second memory space has a mapping relationship with a first memory space of a first processor; the operator data in the first memory space is notified by the first processor to the artificial intelligence chip to perform an operator computing process; in response to receiving an interrupt initiated by the first processor, writing a next batch of operator data in the second memory space, wherein the interrupt is initiated by the first processor when the execution of the operator data in the first memory space is completed and no operator data is written in the first memory space within a preset time period.

7. The method of claim 6, wherein, The second memory space is located in the memory space of the user state of the second processor, and the second processor has the permission to read and write the second memory space through non-system instructions.

8. The method of claim 6, wherein, The method further comprises: writing an execution graph in the second memory space, the execution graph being used to represent the execution order between a plurality of operator data.

9. A scheduling method of an artificial intelligence chip, applied to a scheduling system of an artificial intelligence chip, the method comprising: The second processor in the scheduling system writes a single batch of operator data in a second memory space of the second processor; wherein the second memory space and a first memory space of a first processor in the scheduling system have a mapping relationship; The first processor, in response to at least one operator data being written in the first memory space, notifies an artificial intelligence chip in the scheduling system to perform an operator calculation process according to the operator data written in the first memory space; The artificial intelligence chip performs an operator calculation process according to the operator data written in the first memory space; The first processor, in response to the completion of the execution of the operator data in the first memory space and the absence of operator data being written in the first memory space within a preset time period, initiates an interruption to the second processor; The second processor, in response to receiving the interruption initiated by the first processor, writes a next batch of operator data in the second memory space.

10. The method of claim 9, wherein, The second memory space is located in a user state memory space of the second processor, and the second processor has the permission to read and write the second memory space through non-system instructions.

11. The method of claim 9, wherein, The method further comprises: The second processor writes an execution graph in the second memory space, and the execution graph is used to represent the execution order between a plurality of operator data in a single batch of operator data; The artificial intelligence chip performs an operator calculation process on the operator data written in the first memory space according to the execution graph.

12. The method of claim 9, wherein, The artificial intelligence chip and the first processor share the first memory space.

13. A scheduling device of an artificial intelligence chip, applied to a first processor, the device comprising: A notification module, configured to, in response to at least one operator data being written in a first memory space of the first processor, notify an artificial intelligence chip to perform an operator calculation process according to the operator data written in the first memory space; wherein the first memory space and a second memory space of a second processor have a mapping relationship; and the second processor writes a single batch of operator data at a time. An interruption module, configured to, in response to the completion of the execution of the operator data in the first memory space and the absence of operator data being written in the first memory space within a preset time period, initiate an interruption to the second processor to notify the second processor to write operator data to be processed in the second memory space.

14. The apparatus of claim 13, wherein, The second memory space is located in a user state memory space of the second processor, and the second processor has the permission to read and write the second memory space through non-system instructions.

15. A scheduling device of an artificial intelligence chip, applied to a second processor, the device comprising: A writing module, configured to write a single batch of operator data in a second memory space of the second processor; wherein the second memory space and a first memory space of a first processor have a mapping relationship; and the operator data in the first memory space is notified by the first processor to an artificial intelligence chip to perform an operator calculation process. The write module is further configured to write the next batch of operator data in the second memory space when receiving an interrupt initiated by the first processor, wherein the interrupt is initiated by the first processor when the execution of the operator data in the first memory space is completed and no operator data is written in the first memory space within a preset time period.

16. The apparatus of claim 15, wherein, The second memory space is located in the memory space of the user mode of the second processor, and the second processor has the permission to read and write the second memory space through non-system instructions.

17. A scheduling system of an artificial intelligence chip, the system comprising: a first processor, a second processor, and an artificial intelligence chip; wherein the second processor is connected with the first processor, and a mapping relationship exists between the second memory space of the second processor and the first memory space of the first processor; the first processor is connected with the artificial intelligence chip. The system applies the method according to any one of claims 9 to 12.

18. An electronic device comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 1 to 5, or execute the method according to any one of claims 6 to 8, or execute the method according to any one of claims 9 to 12.

19. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to execute the method according to any one of claims 1 to 5; or enable the computer to execute the method according to any one of claims 6 to 8; or enable the computer to execute the method according to any one of claims 9 to 12.

20. A computer program product comprising a computer program which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 5; or implements the steps of the method according to any one of claims 6 to 8; or implements the steps of the method according to any one of claims 9 to 12.

Images