Artificial intelligence chip integrated with on-chip memory array, computing task processing method and electronic equipment

Abstract

The invention relates to the field of artificial intelligence chips, in particular to an artificial intelligence chip integrated with an on-chip memory array, a computing task processing method and electronic device.The chip comprises a controller and a plurality of processor-memory modules, obtaining a plurality of sub-calculation tasks; the processor-memory module comprises a processing unit group, a memory and a scheduler, the processing unit group comprises a plurality of processing units, and the processing units process the sub-calculation tasks; the memory stores data related to the plurality of processing units in the processing unit group when executing the corresponding sub-calculation tasks; the scheduler is connected to the plurality of processing units and the memory and is used for scheduling access operation of the plurality of processing units to the memory, the processing unit group is arranged in a processing layer of the artificial intelligence chip, the memory is arranged in a storage layer of the artificial intelligence chip, and the processing unit group is used for processing the processing units. The processing layer and the storage layer are arranged in a stacked mode.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence chips, and in particular to an artificial intelligence chip with an integrated on-chip memory array, a computing task processing method, and an electronic device. Background Art

[0002] Artificial intelligence chips usually include multiple processing units (PEs) for performing various calculations involved in artificial intelligence tasks. The data to be calculated by the processing unit is usually stored in an off-chip memory (e.g., DDR) or a memory coplanar with the processing unit in the chip (e.g., SRAM). When the processing unit performs calculations, it needs to first take out data from the off-chip memory or the on-chip memory and put it in the chip's cache, and then read the data from the cache for calculation. The data calculated by the processing unit is first put into the cache, and then written from the cache to the off-chip memory or the on-chip memory. The data access speed of the processing unit to the memory is limited by the physical limit of the data transmission line bandwidth process. When the data access demand of the processing unit exceeds the data transmission line bandwidth, some processing units need to queue up and wait for the data access of other processing units to complete before accessing the data. At present, artificial intelligence chips contain at least a dozen processing units, and at most contain hundreds or even more processing units. When the number of processing units increases, the bandwidth of the data transmission line will seriously restrict the data access speed of the processing unit, thereby affecting the processing efficiency of the artificial intelligence chip and limiting the further improvement of the processing power of the artificial intelligence chip.

[0003] At the same time, the frequent access of the processing unit to the memory will also lead to increased memory power consumption and heat generation, which will affect the stability and life of the chip. As the number of processing units increases, the power consumption and heat generation problems will be further aggravated, thus becoming another bottleneck limiting the improvement of the processing power of artificial intelligence chips. Summary of the invention

[0004] In order to solve the problems in the related art, the embodiments of the present disclosure provide an artificial intelligence chip, a computing task processing method and an electronic device with an integrated on-chip memory array.

[0005] A first aspect of the present disclosure provides an artificial intelligence chip with an integrated on-chip memory array, comprising a plurality of processor-memory modules, wherein the processor-memory modules include a processing unit group, a memory, and a scheduler, wherein:

[0006] The processing unit group is arranged in the processing layer of the artificial intelligence chip, and includes a plurality of processing units, and the processing units are used to process sub-computing tasks obtained based on the computing tasks;

[0007] The memory is arranged in a storage layer of the artificial intelligence chip, and is used to store data involved in the execution of corresponding sub-computing tasks by the plurality of processing units in the processing unit group, and the processing layer and the storage layer are arranged in a stacked manner;

[0008] The scheduler is connected to the multiple processing units and the memory, and is used to schedule access operations of the multiple processing units to the memory, wherein the scheduler includes one or more arbitration units, and is used to arbitrate memory access requests of the multiple processing units, and the arbitration unit includes multiple input terminals, and the arbitration mode of the arbitration unit includes a priority mode and a weight mode, wherein:

[0009] In the priority mode, the arbitration unit sequentially forwards the memory access request from the corresponding input terminal to the memory according to the respective priorities of the plurality of input terminals;

[0010] In the weight mode, the arbitration unit allocates time slots to memory access requests from corresponding input terminals in proportion according to the weights of the multiple input terminals respectively, and forwards the corresponding memory access requests to the memory in the allocated time slots.

[0011] According to an embodiment of the present disclosure, the processing layer and the storage layer are stacked along the vertical direction of the surface of the artificial intelligence chip; the scheduler is arranged in the storage layer; the processing unit is connected to the scheduler through a data transmission line along the vertical direction, and the scheduler is connected to the memory through a data transmission line arranged in the storage layer.

[0012] According to an embodiment of the present disclosure, the plurality of processing units are respectively connected to the scheduler via corresponding data transmission lines along the vertical direction.

[0013] According to an embodiment of the present disclosure, the processing unit includes a cache, which is used to store data involved in the sub-computing task; the processing unit is connected to the scheduler via a data transmission line along the vertical direction, including: the cache of the processing unit is connected to the scheduler via a data transmission line along the vertical direction.

[0014] According to an embodiment of the present disclosure, in the processor-memory module, the positions of the memory, the scheduler, and the processing unit group are set to maximize the overlapping area of ​​the memory and the scheduler projected with the processing unit group on the surface of the artificial intelligence chip.

[0015] According to an embodiment of the present disclosure, the processing layer and the storage layer are stacked in a vertical direction of the surface of the artificial intelligence chip; the scheduler is arranged in the processing layer; the processing unit is connected to the scheduler through a data transmission line arranged in the processing layer, and the scheduler is connected to the memory through a data transmission line along the vertical direction.

[0016] According to an embodiment of the present disclosure, the plurality of processing units are respectively connected to the scheduler via corresponding data transmission lines arranged in the processing layer.

[0017] According to an embodiment of the present disclosure, the processing unit includes a cache, which is used to store data involved in the sub-computing task; the processing unit is connected to the scheduler via a data transmission line set in the processing layer, including: the cache of the processing unit is connected to the scheduler via a data transmission line set in the processing layer; the scheduler is connected to the memory via a data transmission line along the vertical direction.

[0018] According to an embodiment of the present disclosure, in the processor-memory module, the positions of the memory, the scheduler, and the processing unit group are set to maximize the overlapping area of ​​the scheduler, the processing unit group, and the memory projected on the surface of the artificial intelligence chip.

[0019] According to an embodiment of the present disclosure, the processing unit groups in the multiple processor-memory modules are arranged in the same processing layer; and the memories in the multiple processor-memory modules are arranged in the same processing layer.

[0020] According to an embodiment of the present disclosure, the artificial intelligence chip also includes a controller, which is used to parse the computing task to obtain multiple sub-computing tasks.

[0021] According to an embodiment of the present disclosure, the arbitration unit includes a configuration register, and the controller configures the arbitration mode of the arbitration unit by setting a value of the configuration register.

[0022] According to an embodiment of the present disclosure, the controller sets the priority of the input end of the arbitration unit according to the execution order of the corresponding sub-computing tasks processed by the multiple processing units; and / or the controller sets the weight of the input end of the arbitration unit according to the memory access frequency of the corresponding sub-computing tasks processed by the multiple processing units.

[0023] According to an embodiment of the present disclosure, when the scheduler includes one arbitration unit, the arbitration unit is connected to multiple processing units in the processing unit group; when the scheduler includes multiple arbitration units, the multiple arbitration units are connected into an arbitration tree, the root node of the arbitration tree is connected to the memory, and the leaf nodes of the arbitration tree are connected to the corresponding processing units.

[0024] According to an embodiment of the present disclosure, the arbitration tree includes a binary tree structure.

[0025] According to an embodiment of the present disclosure, when the scheduler includes a plurality of arbitration units, the plurality of arbitration units adopt the same arbitration mode, or at least one of the plurality of arbitration units adopts an arbitration mode different from that of the other arbitration units.

[0026] According to an embodiment of the present disclosure, processing units in different processing unit groups are interconnected through data transmission lines and perform data transmission to collaboratively complete the computing task.

[0027] A second aspect of the present disclosure provides a computing task processing method for an artificial intelligence chip with an integrated on-chip memory array, wherein the artificial intelligence chip comprises a plurality of processor-memory modules, the processor-memory module comprises a processing unit group, a memory, and a scheduler, the processing unit group comprises a plurality of processing units, the scheduler is connected to the plurality of processing units and the memory, the scheduler comprises one or more arbitration units for arbitrating memory access requests of the plurality of processing units, the arbitration unit comprises a plurality of input terminals, the arbitration mode of the arbitration unit comprises a priority mode and a weight mode, in the priority mode, the arbitration unit forwards memory access requests from the corresponding input terminals to the memory in sequence according to the respective priorities of the plurality of input terminals, in the weight mode, the arbitration unit allocates time slots to the memory access requests from the corresponding input terminals in proportion according to the respective weights of the plurality of input terminals, and forwards the corresponding memory access requests to the memory in the allocated time slots, the processing unit group is arranged in the processing layer of the artificial intelligence chip, the memory is arranged in the storage layer of the artificial intelligence chip, the processing layer and the storage layer are stacked, and the method comprises:

[0028] Acquire multiple sub-computing tasks obtained based on the computing task, and assign the multiple sub-computing tasks to corresponding processing units;

[0029] storing the data involved in the processing of the sub-computing task by the processing unit in a memory of a processor-memory module where the processing unit is located;

[0030] The sub-computing task is processed by the processing unit, wherein the access operation of the processing unit to the memory is scheduled by a scheduler in a processor-memory module where the processing unit is located.

[0031] According to an embodiment of the present disclosure, the processing layer and the storage layer are stacked along the vertical direction of the surface of the artificial intelligence chip; the scheduler is arranged in the storage layer; the processing unit is connected to the scheduler through a data transmission line along the vertical direction, and the scheduler is connected to the memory through a data transmission line arranged in the storage layer.

[0032] According to an embodiment of the present disclosure, the plurality of processing units are respectively connected to the scheduler via corresponding data transmission lines along the vertical direction.

[0033] According to an embodiment of the present disclosure, the processing unit includes a cache, which is used to store data involved in the sub-computing task; the processing unit is connected to the scheduler via a data transmission line along the vertical direction, including: the cache of the processing unit is connected to the scheduler via a data transmission line along the vertical direction.

[0034] According to an embodiment of the present disclosure, in the processor-memory module, the positions of the memory, the scheduler, and the processing unit group are set to maximize the overlapping area of ​​the memory and the scheduler projected with the processing unit group on the surface of the artificial intelligence chip.

[0035] According to an embodiment of the present disclosure, the processing layer and the storage layer are stacked in a vertical direction of the surface of the artificial intelligence chip; the scheduler is arranged in the processing layer; the processing unit is connected to the scheduler through a data transmission line arranged in the processing layer, and the scheduler is connected to the memory through a data transmission line along the vertical direction.

[0036] According to an embodiment of the present disclosure, the plurality of processing units are respectively connected to the scheduler via corresponding data transmission lines arranged in the processing layer.

[0037] According to an embodiment of the present disclosure, the processing unit includes a cache, which is used to store data involved in the sub-computing task; the processing unit is connected to the scheduler via a data transmission line set in the processing layer, including: the cache of the processing unit is connected to the scheduler via a data transmission line set in the processing layer; the scheduler is connected to the memory via a data transmission line along the vertical direction.

[0038] According to an embodiment of the present disclosure, in the processor-memory module, the positions of the memory, the scheduler, and the processing unit group are set to maximize the overlapping area of ​​the scheduler, the processing unit group, and the memory projected on the surface of the artificial intelligence chip.

[0039] According to an embodiment of the present disclosure, the processing unit groups in the multiple processor-memory modules are arranged in the same processing layer; and the memories in the multiple processor-memory modules are arranged in the same processing layer.

[0040] According to an embodiment of the present disclosure, the artificial intelligence chip also includes a controller, and the acquiring of multiple sub-computing tasks based on the computing task and assigning the multiple sub-computing tasks to corresponding processing units includes: acquiring the computing task through the controller, parsing the computing task to obtain multiple sub-computing tasks, and assigning the multiple sub-computing tasks to corresponding processing units.

[0041] According to an embodiment of the present disclosure, the arbitration unit includes a configuration register, and the method further includes: configuring the arbitration mode of the arbitration unit by setting a value of the configuration register.

[0042] According to an embodiment of the present disclosure, the method also includes: setting, by the controller, the priority of the input end of the arbitration unit according to the execution order of the corresponding sub-computing tasks processed by the multiple processing units; and / or setting, by the controller, the weight of the input end of the arbitration unit according to the memory access frequency of the corresponding sub-computing tasks processed by the multiple processing units.

[0043] According to an embodiment of the present disclosure, when the scheduler includes one arbitration unit, the arbitration unit is connected to multiple processing units in the processing unit group; when the scheduler includes multiple arbitration units, the multiple arbitration units are connected into an arbitration tree, the root node of the arbitration tree is connected to the memory, and the leaf nodes of the arbitration tree are connected to the corresponding processing units.

[0044] According to an embodiment of the present disclosure, the arbitration tree includes a binary tree structure.

[0045] According to an embodiment of the present disclosure, when the scheduler includes multiple arbitration units, the method also includes: configuring the multiple arbitration units to adopt the same arbitration mode by setting the value of the configuration register, or configuring at least one of the multiple arbitration units to adopt an arbitration mode different from that of other arbitration units.

[0046] According to an embodiment of the present disclosure, the method further includes: the processing units in different processing unit groups are interconnected through data transmission lines and perform data transmission to collaboratively complete the computing task.

[0047] A third aspect of the present disclosure provides an electronic device comprising an artificial intelligence chip with an integrated on-chip memory array as described above.

[0048] According to the embodiments of the present disclosure, by dividing the processing unit and memory on the artificial intelligence chip into multiple processor-memory modules, the multiple processing units in each processor-memory module have their own scheduler and memory, and the processing unit accesses the memory through the scheduler, effectively reducing the restriction of data transmission bandwidth on data access speed, reducing the occurrence of processing unit queuing, and improving the processing efficiency of the artificial intelligence chip. At the same time, since the data used by the processing unit on the artificial intelligence chip when performing computing tasks is scattered and stored in different memories, the number of accesses and access frequencies to a single memory are significantly reduced, effectively reducing the power consumption and heat generation of the chip, reducing the heat dissipation requirements, and also helping to improve the chip stability and extend the service life of the chip. The processing unit and the memory are respectively arranged in different layers of the chip, which can significantly reduce the chip area, reduce the wiring complexity, shorten the length of the data transmission line, facilitate production and manufacturing, and help improve the chip stability, reduce power consumption, and improve the yield. By designing the arbitration unit in the scheduler to have two working modes, namely, priority mode and weight mode, the scheduling operation of the scheduler can be flexibly implemented according to the memory access requirements of different sub-computing tasks. When the scheduler includes multiple arbitration units, the two working modes can be combined to match different memory access requirements, further improving the memory access efficiency of the processing unit.

[0049] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Other features, objectives and advantages of the present disclosure will become more apparent through the following detailed description of non-limiting embodiments in conjunction with the accompanying drawings. In the accompanying drawings:

[0051] Figure 1 A logical block diagram of an artificial intelligence chip according to an embodiment of the present disclosure is shown.

[0052] Figure 2 A schematic diagram of the three-dimensional structure of an artificial intelligence chip according to an embodiment of the present disclosure is shown.

[0053] Figure 3 Another three-dimensional structural schematic diagram of an artificial intelligence chip according to an embodiment of the present disclosure is shown.

[0054] Figure 4 A scheduler including one arbitration unit according to an embodiment of the present disclosure is shown.

[0055] Figure 5 A scheduler including a plurality of arbitration units according to an embodiment of the present disclosure is shown.

[0056] Figure 6 A flowchart of a method for processing computing tasks for an artificial intelligence chip with an integrated on-chip memory array according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0057] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for the sake of clarity, parts not related to the description of the exemplary embodiments are omitted in the accompanying drawings.

[0058] In the present disclosure, it should be understood that terms such as "include" or "have" are intended to indicate the presence of features, numbers, steps, behaviors, components, parts, or a combination thereof disclosed in the present specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, behaviors, components, parts, or a combination thereof exist or are added.

[0059] It should also be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0060] As mentioned above, the data access speed of the processing unit to the memory is limited by the physical limit of the data transmission line bandwidth process. When the data access demand of the processing unit exceeds the data transmission line bandwidth, some processing units need to wait in line for the data access of other processing units to complete before accessing the data. At present, artificial intelligence chips contain at least a dozen processing units, and at most hundreds or even more processing units. When the number of processing units increases, the data transmission line bandwidth will seriously restrict the data access speed of the processing unit, thereby affecting the processing efficiency of the artificial intelligence chip and limiting the further improvement of the processing power of the artificial intelligence chip.

[0061] At the same time, the frequent access of the processing unit to the memory will also lead to increased memory power consumption and heat generation, which will affect the stability and life of the chip. As the number of processing units increases, the power consumption and heat generation problems will be further aggravated, thus becoming another bottleneck limiting the improvement of the processing power of artificial intelligence chips.

[0062] The embodiment of the present disclosure provides an artificial intelligence chip with an integrated on-chip memory array, including a plurality of processor-memory modules, wherein the processor-memory module includes a processing unit group, a memory, and a scheduler, and in the processor-memory module:

[0063] The processing unit group is arranged in the processing layer of the artificial intelligence chip, and includes a plurality of processing units, and the processing units are used to process sub-computing tasks obtained based on the computing tasks;

[0064] The memory is arranged in a storage layer of the artificial intelligence chip, and is used to store data involved in the execution of corresponding sub-computing tasks by the plurality of processing units in the processing unit group, and the processing layer and the storage layer are arranged in a stacked manner;

[0065] The scheduler is connected to the multiple processing units and the memory, and is used to schedule access operations of the multiple processing units to the memory, wherein the scheduler includes one or more arbitration units, and is used to arbitrate memory access requests of the multiple processing units, and the arbitration unit includes multiple input terminals, and the arbitration mode of the arbitration unit includes a priority mode and a weight mode, wherein:

[0066] In the priority mode, the arbitration unit sequentially forwards the memory access request from the corresponding input terminal to the memory according to the respective priorities of the plurality of input terminals;

[0067] In the weight mode, the arbitration unit allocates time slots to memory access requests from corresponding input terminals in proportion according to the weights of the multiple input terminals respectively, and forwards the corresponding memory access requests to the memory in the allocated time slots.

[0068] According to the embodiments of the present disclosure, by dividing the processing unit and memory on the artificial intelligence chip into multiple processor-memory modules, the multiple processing units in each processor-memory module have their own scheduler and memory, and the processing unit accesses the memory through the scheduler, effectively reducing the restriction of data transmission bandwidth on data access speed, reducing the occurrence of processing unit queuing, and improving the processing efficiency of the artificial intelligence chip. At the same time, since the data used by the processing unit on the artificial intelligence chip when performing computing tasks is scattered and stored in different memories, the number of accesses and access frequencies to a single memory are significantly reduced, effectively reducing the power consumption and heat generation of the chip, reducing the heat dissipation requirements, and also helping to improve the chip stability and extend the service life of the chip. The processing unit and the memory are respectively arranged in different layers of the chip, which can significantly reduce the chip area, reduce the wiring complexity, shorten the length of the data transmission line, facilitate production and manufacturing, and help improve the chip stability, reduce power consumption, and improve the yield. By designing the arbitration unit in the scheduler to have two working modes, namely, priority mode and weight mode, the scheduling operation of the scheduler can be flexibly implemented according to the memory access requirements of different sub-computing tasks. When the scheduler includes multiple arbitration units, the two working modes can be combined to match different memory access requirements, further improving the memory access efficiency of the processing unit.

[0069] Figure 1 A logical block diagram of an artificial intelligence chip according to an embodiment of the present disclosure is shown.

[0070] like Figure 1 As shown, the artificial intelligence chip according to the embodiment of the present disclosure includes a controller and a plurality of processor-memory modules, and the plurality of processor-memory modules can constitute an m*n processor-memory module array, wherein each row includes n processor-memory modules and each column includes m processor-memory modules.

[0071] According to an embodiment of the present disclosure, the processor-storage module includes a plurality of processing units (PEs), and the processing units include a cache. The processor-storage module also includes a scheduler and a memory (e.g., SRAM). In the processor-storage module, the plurality of processing units are respectively connected to the scheduler through data transmission lines, and the scheduler forwards the memory access request of the processing unit to the memory according to a preset scheduling rule, thereby realizing the access of the processing unit to the memory.

[0072] Different processing units in the same processor-storage module can be interconnected and transmit data through data transmission lines. For example, the calculation result of one processing unit can be sent to another processing unit for use. Different processing units in different processor-storage modules can also be interconnected and transmit data through data transmission lines. For example, the calculation result of a processing unit in one processor-storage module can be sent to a processing unit in another processor-storage module for use. In this way, multiple processor-storage modules of an artificial intelligence chip can collaborate to complete artificial intelligence computing tasks.

[0073] Specifically, when executing an artificial intelligence computing task, the controller of the artificial intelligence chip according to the embodiment of the present disclosure obtains the computing task, parses the computing task, obtains multiple sub-computing tasks, and assigns the multiple sub-computing tasks to multiple processing units located in one or more processor-memory modules. The processing unit processes the sub-computing tasks, and the memory in the same processor-memory module as the processing unit is used to store the data involved in the processing unit when executing the corresponding sub-computing tasks. The scheduler is used to schedule the access operation of the processing unit to the memory. According to the processing flow of the computing task, data can be transmitted between the processing units located in one or more processor-memory modules, so as to collaboratively complete the artificial intelligence computing task.

[0074] According to an embodiment of the present disclosure, the artificial intelligence chip includes a processing layer and a storage layer that are stacked, the processing unit group is arranged in the processing layer, and the memory is arranged in the storage layer.

[0075] Figure 2 A schematic diagram of the three-dimensional structure of an artificial intelligence chip according to an embodiment of the present disclosure is shown.

[0076] like Figure 2 As shown, the artificial intelligence chip according to an embodiment of the present disclosure includes a processing layer and a storage layer, and the processing layer and the storage layer are stacked in a vertical direction along the surface of the artificial intelligence chip.

[0077] A plurality of processing unit groups are provided in the processing layer, and each processing unit group includes a plurality of processing units. The figure shows a case where four processing unit groups, each of which includes two processing units. However, it is to be understood that other numbers of processing unit groups and / or processing units may be provided according to actual needs. Each processing unit includes a cache, and the cache is used to store data related to the sub-computing task. A plurality of schedulers and a plurality of memories corresponding to the plurality of processing unit groups are provided in the storage layer, and for the sake of clarity, only one scheduler is shown in the figure.

[0078] like Figure 2 As shown, in the processor-memory module, the processing unit is connected to the scheduler via a data transmission line along the vertical direction. According to an embodiment of the present disclosure, a plurality of processing units are respectively connected to the scheduler via corresponding data transmission lines along the vertical direction. The scheduler is connected to the memory via a data transmission line arranged in the storage layer.

[0079] According to an embodiment of the present disclosure, the processing unit is connected to the scheduler via a data transmission line along the vertical direction, including: a cache of the processing unit is connected to the scheduler via a data transmission line along the vertical direction.

[0080] According to an embodiment of the present disclosure, in the processor-memory module, the positions of the memory, the scheduler, and the processing unit group are set to maximize the overlapping area of ​​the projection of the memory and the scheduler with the processing unit group on the surface of the artificial intelligence chip. Figure 2 As shown, the scheduler and the memory are both arranged in the storage layer, and the processing unit group is arranged in the processing layer. By setting the positional relationship of the three so that the projections of the memory and the scheduler on the surface of the artificial intelligence chip overlap as much as possible with the projection of the processing unit group on the surface of the artificial intelligence chip, the chip size can be reduced and the chip miniaturization can be achieved.

[0081] Figure 3 Another three-dimensional structural schematic diagram of an artificial intelligence chip according to an embodiment of the present disclosure is shown.

[0082] like Figure 3As shown, the artificial intelligence chip according to an embodiment of the present disclosure includes a processing layer and a storage layer, and the processing layer and the storage layer are stacked in a vertical direction along the surface of the artificial intelligence chip.

[0083] A plurality of processing unit groups are provided in the processing layer, and each processing unit group includes a plurality of processing units. The figure shows a case where four processing unit groups, each of which includes two processing units. However, it is to be understood that other numbers of processing unit groups and / or processing units may be provided according to actual needs. Each processing unit includes a cache, and the cache is used to store data related to the sub-computing task. A plurality of schedulers and a plurality of memories corresponding to the plurality of processing unit groups are provided in the storage layer, and for the sake of clarity, only one scheduler is shown in the figure.

[0084] like Figure 3 As shown, in the processor-memory module, the processing unit is connected to the scheduler via a data transmission line arranged in the processing layer. According to an embodiment of the present disclosure, a plurality of processing units are respectively connected to the scheduler via corresponding data transmission lines arranged in the processing layer. The scheduler is connected to the memory via a data transmission line along the vertical direction.

[0085] According to an embodiment of the present disclosure, the processing unit is connected to the scheduler along a data transmission line arranged in the processing layer, including: the cache of the processing unit is connected to the scheduler along a data transmission line arranged in the processing layer.

[0086] According to an embodiment of the present disclosure, in the processor-memory module, the positions of the memory, the scheduler, and the processing unit group are set to maximize the overlapping area of ​​the processing unit group and the scheduler projected on the surface of the artificial intelligence chip with the memory. Figure 3 As shown, the memories are all arranged in the storage layer, the scheduler and the processing unit group are arranged in the processing layer, and by setting the positional relationship of the three so that the projections of the processing unit group and the scheduler on the surface of the artificial intelligence chip overlap as much as possible with the projection of the memory on the surface of the artificial intelligence chip, the chip size can be reduced and the chip miniaturization can be achieved.

[0087] Reference below Figure 4 and Figure 5 A scheduler according to an embodiment of the present disclosure is described.

[0088] Figure 4 A scheduler including one arbitration unit according to an embodiment of the present disclosure is shown.

[0089] like Figure 4As shown, the scheduler according to an embodiment of the present disclosure includes an arbitration unit, which includes multiple input terminals, which are respectively connected to multiple processing units in a processing unit group, and are used to arbitrate memory access requests of multiple processing units in the processing unit group, thereby realizing scheduling of memory access requests.

[0090] As an example, the arbitration unit in the figure includes four input terminals IN0, IN1, IN2, and IN3, which are respectively connected to caches C0, C1, C2, and C3 of four processing units in the processing unit group, and are used to arbitrate memory access requests of the four processing units in the processing unit group.

[0091] According to an embodiment of the present disclosure, the arbitration mode of the arbitration unit includes a priority mode and a weight mode, wherein: in the priority mode, the arbitration unit forwards the memory access requests from the corresponding input terminals to the memory in sequence according to the respective priorities of the multiple input terminals; in the weight mode, the arbitration unit allocates time slots to the memory access requests from the corresponding input terminals in proportion according to the respective weights of the multiple input terminals, and forwards the corresponding memory access requests to the memory in the allocated time slots.

[0092] For example, in Figure 4 In the priority mode, the priorities of the four input terminals IN0, IN1, IN2, and IN3 can be set from high to low as IN0, IN1, IN2, and IN3. In this mode, the arbitration unit gives priority to forwarding memory access requests from high-priority input terminals. In one implementation, each input terminal can have its own buffer queue for receiving memory access requests from the processing unit connected to the input terminal. When an unforwarded memory access request appears in the buffer queue of the high-priority input terminal, the arbitration unit interrupts the forwarding of the memory access request in the buffer queue of the low-priority input terminal, and forwards the memory access request in the buffer queue of the high-priority input terminal. After the memory access request in the buffer queue of the high-priority input terminal is forwarded, the memory access request in the buffer queue of the low-priority input terminal continues to be forwarded.

[0093] On the other hand, in the weight mode, the weights of the four input terminals IN0, IN1, IN2, and IN3 can be set to W0, W1, W2, and W3, respectively. In this mode, time slots are allocated to memory access requests from corresponding input terminals in proportion according to the weights of the multiple input terminals, and the corresponding memory access requests are forwarded in the allocated time slots. For example, assuming that a forwarding cycle has N time slots, N*W0 / (W0+W1+W2+W3) time slots are used to forward memory access requests from input terminal IN0, N*W1 / (W0+W1+W2+W3) time slots are used to forward memory access requests from input terminal IN1, N*W2 / (W0+W1+W2+W3) time slots are used to forward memory access requests from input terminal IN2, and N*W3 / (W0+W1+W2+W3) time slots are used to forward memory access requests from input terminal IN3. By setting the input terminal weights, more sophisticated forwarding scheduling can be achieved.

[0094] According to an embodiment of the present disclosure, the arbitration unit includes a configuration register, and the controller configures the arbitration mode of the arbitration unit by setting the value of the configuration register. For example, when the value of the configuration register is set to 0, the arbitration unit adopts a priority mode, and when the value of the configuration register is set to 1, the arbitration unit adopts a weight mode.

[0095] Figure 5 A scheduler including a plurality of arbitration units according to an embodiment of the present disclosure is shown.

[0096] like Figure 5 As shown, the scheduler according to an embodiment of the present disclosure includes a plurality of arbitration units, the plurality of arbitration units are connected to form an arbitration tree, the root node of the arbitration tree is connected to the memory, and the leaf nodes of the arbitration tree are connected to the corresponding processing units. According to an embodiment of the present disclosure, the arbitration tree includes a binary tree structure.

[0097] Figure 5 A binary tree structure scheduler including three nodes is shown. Figure 5 In the example, the two input terminals IN0 and IN1 of arbitration unit 1 are respectively connected to the caches C0 and C1 of the two processing units in the processing unit group, the two input terminals IN2 and IN3 of arbitration unit 2 are respectively connected to the caches C2 and C3 of the two processing units in the processing unit group, and the two input terminals IN4 and IN5 of arbitration unit 3 are respectively connected to the output terminals of arbitration unit 1 and arbitration unit 2. The logic of the binary tree structure is clear and easy to implement. Each arbitration unit can have the same or similar structure, with low process complexity, high circuit stability and reliability, and can be easily expanded according to the number of connected processing units.

[0098] According to an embodiment of the present disclosure, the arbitration mode of the arbitration unit includes a priority mode and a weight mode, wherein: in the priority mode, the arbitration unit forwards the memory access request from the corresponding input terminal in sequence according to the respective priorities of the multiple input terminals. In the weight mode, the arbitration unit allocates time slots to the memory access request from the corresponding input terminal in proportion according to the respective weights of the multiple input terminals, and forwards the corresponding memory access request in the allocated time slot. By setting the input terminal weight, a more refined forwarding scheduling can be achieved.

[0099] For example, for arbitration unit 1, in priority mode, the priorities of the two input terminals IN0 and IN1 can be set from high to low as IN0 and IN1, respectively. In this mode, the arbitration unit gives priority to forwarding memory access requests from the input terminal with high priority. Specifically, each input terminal can have its own buffer queue for receiving memory access requests from the processing unit connected to the input terminal. When an unforwarded memory access request appears in the buffer queue of the high-priority input terminal, the arbitration unit will interrupt the forwarding of the memory access request in the buffer queue of the low-priority input terminal, and forward the memory access request in the buffer queue of the high-priority input terminal. After the memory access request in the buffer queue of the high-priority input terminal is forwarded, the memory access request in the buffer queue of the low-priority input terminal will continue to be forwarded. On the other hand, in weight mode, the weights of the two input terminals IN0 and IN1 can be set as W0 and W1 respectively. In this mode, according to the weights of the multiple input terminals, time slots are allocated to the memory access requests from the corresponding input terminals in proportion, and the corresponding memory access requests are forwarded in the allocated time slots. For example, assuming that one forwarding cycle has N time slots, N*W0 / (W0+W1) time slots are used to forward memory access requests from input terminal IN0, and N*W1 / (W0+W1) time slots are used to forward memory access requests from input terminal IN1.

[0100] For arbitration unit 2, in priority mode, the priorities of the two input terminals IN2 and IN3 can be set from high to low as IN2 and IN3, then in this mode, the arbitration unit gives priority to forwarding memory access requests from the input terminal with high priority. Specifically, each input terminal can have its own buffer queue for receiving memory access requests from the processing unit connected to the input terminal. When an unforwarded memory access request appears in the buffer queue of the high-priority input terminal, the arbitration unit will interrupt the forwarding of the memory access request in the buffer queue of the low-priority input terminal, and forward the memory access request in the buffer queue of the high-priority input terminal. After the memory access request in the buffer queue of the high-priority input terminal is forwarded, the memory access request in the buffer queue of the low-priority input terminal will continue to be forwarded. On the other hand, in weight mode, the weights of the two input terminals IN2 and IN3 can be set as W2 and W3 respectively. In this mode, according to the weights of the multiple input terminals, time slots are allocated to the memory access requests from the corresponding input terminals in proportion, and the corresponding memory access requests are forwarded in the allocated time slots. For example, assuming that one forwarding cycle has N time slots, N*W2 / (W2+W3) time slots are used to forward memory access requests from input terminal IN2, and N*W3 / (W2+W3) time slots are used to forward memory access requests from input terminal IN3.

[0101] For arbitration unit 3, in priority mode, the priorities of the two input terminals IN4 and IN5 can be set from high to low as IN4 and IN5, respectively. In this mode, the arbitration unit gives priority to forwarding memory access requests from the input terminal with high priority. Specifically, each input terminal can have its own buffer queue for receiving memory access requests from the processing unit connected to the input terminal. When an unforwarded memory access request appears in the buffer queue of the high-priority input terminal, the arbitration unit will interrupt the forwarding of the memory access request in the buffer queue of the low-priority input terminal, and forward the memory access request in the buffer queue of the high-priority input terminal. After the memory access request in the buffer queue of the high-priority input terminal is forwarded, the memory access request in the buffer queue of the low-priority input terminal will continue to be forwarded. On the other hand, in weight mode, the weights of the two input terminals IN4 and IN5 can be set to W4 and W5 respectively. In this mode, according to the weights of the multiple input terminals, time slots are allocated to the memory access requests from the corresponding input terminals in proportion, and the corresponding memory access requests are forwarded in the allocated time slots. For example, assuming that one forwarding cycle has N time slots, N*W4 / (W4+W5) time slots are used to forward memory access requests from input terminal IN4, and N*W5 / (W4+W5) time slots are used to forward memory access requests from input terminal IN5.

[0102] According to an embodiment of the present disclosure, in a weighted mode, forwarding periods of different arbitration units may be the same or different.

[0103] According to an embodiment of the present disclosure, the arbitration unit includes a configuration register, and the controller configures the arbitration mode of the arbitration unit by setting the value of the configuration register. For example, when the value of the configuration register is set to 0, the arbitration unit adopts a priority mode, and when the value of the configuration register is set to 1, the arbitration unit adopts a weight mode.

[0104] According to an embodiment of the present disclosure, when the scheduler includes multiple arbitration units, the multiple arbitration units may adopt the same arbitration mode, or at least one of the multiple arbitration units may adopt an arbitration mode different from that of the other arbitration units. For example, arbitration unit 1, arbitration unit 2, and arbitration unit 3 may all adopt a priority mode, or may all adopt a weight mode. Alternatively, any one or more of arbitration unit 1, arbitration unit 2, and arbitration unit 3 may adopt a priority mode, and the others may adopt a weight mode. For example, arbitration units 1 and 2 may adopt a weight mode, and arbitration unit 3 may adopt a priority mode; or arbitration units 1 and 2 may adopt a priority mode, and arbitration unit 3 may adopt a weight mode. By setting the arbitration modes of each of the multiple arbitration units, different memory access requirements of the processing unit can be met in a variety of different combinations.

[0105] According to an embodiment of the present disclosure, the controller sets the priority of the input end of the arbitration unit according to the execution order of the corresponding sub-computing tasks processed by the multiple processing units; and / or the controller sets the weight of the input end of the arbitration unit according to the memory access frequency of the corresponding sub-computing tasks processed by the multiple processing units.

[0106] For example, the input end for receiving the memory access request of the processing unit for processing the sub-computing task executed later can have a higher priority than the input end for receiving the memory access request of the processing unit for processing the sub-computing task executed earlier, so as to ensure the smoothness of the entire task processing pipeline. The input end for receiving the memory access request of the processing unit for processing the sub-computing task with high memory access frequency can have a higher weight than the input end for receiving the memory access request of the processing unit for processing the sub-computing task with low memory access frequency, so as to ensure the balance of the overall computing efficiency.

[0107] An embodiment of the present disclosure also provides a computing task processing method for an artificial intelligence chip with an integrated on-chip memory array, wherein the artificial intelligence chip includes a controller and multiple processor-memory modules, the processor-memory module includes a processing unit group, a memory, and a scheduler, the processing unit group includes multiple processing units, the scheduler is connected to the multiple processing units and the memory, the processing unit group is arranged in the processing layer of the artificial intelligence chip, the memory is arranged in the storage layer of the artificial intelligence chip, and the processing layer and the storage layer are stacked. The method includes: obtaining a computing task through the controller, parsing the computing task to obtain multiple sub-computing tasks, and allocating the multiple sub-computing tasks to multiple processing units; storing data involved in the processing of the sub-computing tasks by the processing unit in the memory of the processor-memory module where the processing unit is located; processing the sub-computing tasks by the processing unit, wherein the access operation of the processing unit to the memory is scheduled by the scheduler in the processor-memory module where the processing unit is located.

[0108] Figure 6 A flowchart of a method for processing computing tasks for an artificial intelligence chip with an integrated on-chip memory array according to an embodiment of the present disclosure is shown.

[0109] According to an embodiment of the present disclosure, the artificial intelligence chip with integrated on-chip memory array may be Figure 1-Figure 5 The described artificial intelligence chip includes multiple processor-memory modules, and the processor-memory module includes a processing unit group, a memory, and a scheduler. The processing unit group includes multiple processing units, and the scheduler is connected to the multiple processing units and the memory. The scheduler includes one or more arbitration units for arbitrating memory access requests of the multiple processing units. The arbitration unit includes multiple input terminals, and the arbitration mode of the arbitration unit includes a priority mode and a weight mode. In the priority mode, the arbitration unit forwards the memory access requests from the corresponding input terminals to the memory in sequence according to the respective priorities of the multiple input terminals. In the weight mode, the arbitration unit allocates time slots to the memory access requests from the corresponding input terminals in proportion according to the respective weights of the multiple input terminals, and forwards the corresponding memory access requests to the memory in the allocated time slots. The processing unit group is arranged in the processing layer of the artificial intelligence chip, and the memory is arranged in the storage layer of the artificial intelligence chip. The processing layer and the storage layer are stacked.

[0110] like Figure 6 As shown, according to an embodiment of the present disclosure, a computing task processing method for an artificial intelligence chip with an integrated on-chip memory array includes steps S1 to S3.

[0111] In step S1, a plurality of sub-computing tasks based on the computing task are obtained, and the plurality of sub-computing tasks are allocated to corresponding processing units.

[0112] In step S2, the data involved in the processing of the sub-computing task by the processing unit is stored in the memory of the processor-memory module where the processing unit is located;

[0113] In step S3, the sub-computing task is processed by the processing unit, wherein the access operation of the processing unit to the memory is scheduled by a scheduler in a processor-memory module where the processing unit is located.

[0114] According to an embodiment of the present disclosure, the processing layer and the storage layer are stacked in a vertical direction along the surface of the artificial intelligence chip, and the scheduler can be as follows: Figure 2 As shown in the storage layer, it can also be set as Figure 3 The arrangement shown is in the processing layer.

[0115] According to an embodiment of the present disclosure, when the scheduler is arranged in the storage layer, the processing unit is connected to the scheduler through a data transmission line along the vertical direction. According to an embodiment of the present disclosure, the plurality of processing units are respectively connected to the scheduler through corresponding data transmission lines along the vertical direction. The scheduler is connected to the memory through a data transmission line arranged in the storage layer.

[0116] According to an embodiment of the present disclosure, the processing unit includes a cache, which is used to store data involved in the sub-computing task; the processing unit is connected to the scheduler via a data transmission line along the vertical direction, including: the cache of the processing unit is connected to the scheduler via a data transmission line along the vertical direction.

[0117] According to an embodiment of the present disclosure, in the processor-memory module, the positions of the memory, the scheduler, and the processing unit group are set to maximize the overlapping area of ​​the projection of the memory and the scheduler with the processing unit group on the surface of the artificial intelligence chip. Figure 2 As shown, the scheduler and the memory are both arranged in the storage layer, and the processing unit group is arranged in the processing layer. By setting the positional relationship of the three so that the projections of the memory and the scheduler on the surface of the artificial intelligence chip overlap as much as possible with the projection of the processing unit group on the surface of the artificial intelligence chip, the chip size can be reduced and the chip miniaturization can be achieved.

[0118] According to an embodiment of the present disclosure, when the scheduler is arranged in the processing layer, the processing unit is connected to the scheduler through a data transmission line arranged in the processing layer. According to an embodiment of the present disclosure, the plurality of processing units are respectively connected to the scheduler through corresponding data transmission lines arranged in the processing layer. The scheduler is connected to the memory through a data transmission line along the vertical direction.

[0119] According to an embodiment of the present disclosure, the processing unit includes a cache, which is used to store data involved in the sub-computing task; the processing unit is connected to the scheduler via a data transmission line set in the processing layer, including: the cache of the processing unit is connected to the scheduler via a data transmission line set in the processing layer; the scheduler is connected to the memory via a data transmission line along the vertical direction.

[0120] According to an embodiment of the present disclosure, in the processor-memory module, the positions of the memory, the scheduler, and the processing unit group are set to maximize the overlapping area of ​​the scheduler, the processing unit group, and the memory projected on the surface of the artificial intelligence chip. Figure 3 As shown, the memories are all arranged in the storage layer, the scheduler and the processing unit group are arranged in the processing layer, and by setting the positional relationship of the three so that the projections of the processing unit group and the scheduler on the surface of the artificial intelligence chip overlap as much as possible with the projection of the memory on the surface of the artificial intelligence chip, the chip size can be reduced and the chip miniaturization can be achieved.

[0121] According to an embodiment of the present disclosure, the processing unit groups in the multiple processor-memory modules are arranged in the same processing layer; and the memories in the multiple processor-memory modules are arranged in the same processing layer.

[0122] According to an embodiment of the present disclosure, the artificial intelligence chip further includes a controller, and the acquiring of multiple sub-computing tasks based on the computing task and assigning the multiple sub-computing tasks to corresponding processing units includes: acquiring the computing task through the controller, parsing the computing task to obtain multiple sub-computing tasks, and assigning the multiple sub-computing tasks to corresponding processing units. Specifically, the controller can be set on the artificial intelligence chip or outside the artificial intelligence chip.

[0123] According to an embodiment of the present disclosure, when the scheduler includes one arbitration unit, the arbitration unit is connected to multiple processing units in the processing unit group; when the scheduler includes multiple arbitration units, the multiple arbitration units are connected into an arbitration tree, the root node of the arbitration tree is connected to the memory, and the leaf nodes of the arbitration tree are connected to the corresponding processing units.

[0124] According to an embodiment of the present disclosure, the arbitration tree includes a binary tree structure. The binary tree structure has clear logic and is easy to implement. Each arbitration unit can have the same or similar structure, has low process complexity, high circuit stability and reliability, and is easily expandable according to the number of connected processing units.

[0125] According to an embodiment of the present disclosure, the arbitration unit includes a configuration register, and the method further includes: configuring the arbitration mode of the arbitration unit by setting a value of the configuration register.

[0126] According to an embodiment of the present disclosure, when the scheduler includes multiple arbitration units, the method further includes: configuring the multiple arbitration units to adopt the same arbitration mode, or configuring at least one of the multiple arbitration units to adopt an arbitration mode different from that of other arbitration units by setting the value of the configuration register. By setting the arbitration modes of the multiple arbitration units, different memory access requirements of the processing unit can be met in a variety of different combinations.

[0127] According to an embodiment of the present disclosure, the method also includes: setting, by the controller, the priority of the input end of the arbitration unit according to the execution order of the corresponding sub-computing tasks processed by the multiple processing units; and / or setting, by the controller, the weight of the input end of the arbitration unit according to the memory access frequency of the corresponding sub-computing tasks processed by the multiple processing units.

[0128] For example, the input end for receiving the memory access request of the processing unit for processing the sub-computing task executed later can have a higher priority than the input end for receiving the memory access request of the processing unit for processing the sub-computing task executed earlier, so as to ensure the smoothness of the entire task processing pipeline. The input end for receiving the memory access request of the processing unit for processing the sub-computing task with high memory access frequency can have a higher weight than the input end for receiving the memory access request of the processing unit for processing the sub-computing task with low memory access frequency, so as to ensure the balance of the overall computing efficiency.

[0129] According to an embodiment of the present disclosure, the method further includes: the processing units in different processing unit groups are interconnected through data transmission lines and perform data transmission to collaboratively complete the computing task.

[0130] An embodiment of the present disclosure also provides an electronic device, comprising an artificial intelligence chip with an integrated on-chip memory array as described above.

[0131] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present disclosure is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other.

Claims

1. An artificial intelligence chip with an integrated on-chip memory array, characterized in that: The invention comprises a plurality of processor-memory modules, wherein the processor-memory modules include a processing unit group, a memory, and a scheduler, and in the processor-memory modules: The processing unit group is arranged in the processing layer of the artificial intelligence chip, and includes a plurality of processing units, and the processing units are used to process sub-computing tasks obtained based on the computing tasks; The memory is arranged in a storage layer of the artificial intelligence chip, and is used to store data involved in the execution of corresponding sub-computing tasks by the plurality of processing units in the processing unit group, and the processing layer and the storage layer are arranged in a stacked manner; The scheduler is connected to the multiple processing units and the memory, and is used to schedule access operations of the multiple processing units to the memory, wherein the scheduler includes one or more arbitration units, and is used to arbitrate memory access requests of the multiple processing units, and the arbitration unit includes multiple input terminals, and the arbitration mode of the arbitration unit includes a priority mode and a weight mode, wherein: In the priority mode, the arbitration unit sequentially forwards the memory access request from the corresponding input terminal to the memory according to the respective priorities of the plurality of input terminals; In the weight mode, the arbitration unit allocates time slots to memory access requests from corresponding input terminals in proportion according to the weights of the multiple input terminals respectively, and forwards the corresponding memory access requests to the memory in the allocated time slots.

2. The artificial intelligence chip according to claim 1, characterized in that: The processing layer and the storage layer are stacked in a vertical direction along the surface of the artificial intelligence chip; The scheduler is arranged in the storage layer; The processing unit is connected to the scheduler through a data transmission line along the vertical direction, and the scheduler is connected to the memory through a data transmission line arranged in the memory layer.

3. The artificial intelligence chip according to claim 2, characterized in that: The plurality of processing units are respectively connected to the scheduler via corresponding data transmission lines along the vertical direction.

4. The artificial intelligence chip according to claim 2, characterized in that: The processing unit includes a cache, and the cache is used to store data involved in the sub-computing task; The processing unit is connected to the scheduler via a data transmission line along the vertical direction, including: a cache of the processing unit is connected to the scheduler via a data transmission line along the vertical direction.

5. The artificial intelligence chip according to claim 2, characterized in that: In the processor-memory module, the positions of the memory, the scheduler, and the processing unit group are set to maximize the overlapping area of ​​the memory and the scheduler projected with the processing unit group on the surface of the artificial intelligence chip.

6. The artificial intelligence chip according to claim 1, characterized in that: The processing layer and the storage layer are stacked in a vertical direction along the surface of the artificial intelligence chip; The scheduler is arranged in the processing layer; The processing unit is connected to the scheduler through a data transmission line arranged in the processing layer, and the scheduler is connected to the memory through a data transmission line along the vertical direction.

7. The artificial intelligence chip according to claim 6, characterized in that: The plurality of processing units are respectively connected to the scheduler via corresponding data transmission lines arranged in the processing layer.

8. The artificial intelligence chip according to claim 6, characterized in that: The processing unit includes a cache, and the cache is used to store data involved in the sub-computing task; The processing unit is connected to the scheduler via a data transmission line arranged in the processing layer, including: a cache of the processing unit is connected to the scheduler via a data transmission line arranged in the processing layer; The scheduler is connected to the memory through a data transmission line along the vertical direction.

9. The artificial intelligence chip according to claim 6, characterized in that: In the processor-memory module, the positions of the memory, the scheduler, and the processing unit group are set to maximize the overlapping area of ​​the scheduler, the processing unit group, and the memory projected on the surface of the artificial intelligence chip.

10. The artificial intelligence chip according to claim 1, characterized in that: The processing unit groups in the plurality of processor-memory modules are arranged in the same processing layer; The memories in the plurality of processor-memory modules are arranged in the same processing layer.

11. The artificial intelligence chip according to claim 1, characterized in that: It also includes a controller, which is used to parse the computing task to obtain multiple sub-computing tasks.

12. The artificial intelligence chip according to claim 11, characterized in that: The arbitration unit includes a configuration register, and the controller configures the arbitration mode of the arbitration unit by setting the value of the configuration register.

13. The artificial intelligence chip according to claim 11, characterized in that: The controller sets the priority of the input terminal of the arbitration unit according to the execution order of the corresponding sub-computing tasks processed by the plurality of processing units; and / or The controller sets the weight of the input terminal of the arbitration unit according to the memory access frequency of the corresponding sub-computing tasks processed by the plurality of processing units.

14. The artificial intelligence chip according to claim 1, characterized in that: When the scheduler includes an arbitration unit, the arbitration unit is connected to a plurality of processing units in the processing unit group; When the scheduler includes a plurality of arbitration units, the plurality of arbitration units are connected to form an arbitration tree, a root node of the arbitration tree is connected to the memory, and leaf nodes of the arbitration tree are connected to corresponding processing units.

15. The artificial intelligence chip according to claim 14, characterized in that: The arbitration tree includes a binary tree structure.

16. The artificial intelligence chip according to claim 14, characterized in that: When the scheduler includes a plurality of arbitration units, the plurality of arbitration units adopt the same arbitration mode, or at least one of the plurality of arbitration units adopts an arbitration mode different from that of the other arbitration units.

17. The artificial intelligence chip according to claim 1, characterized in that: The processing units in different processing unit groups are interconnected through data transmission lines and perform data transmission to collaboratively complete the computing task.

18. A method for processing computing tasks for an artificial intelligence chip with an integrated on-chip memory array, characterized in that: The artificial intelligence chip includes multiple processor-memory modules, the processor-memory module includes a processing unit group, a memory, and a scheduler. The processing unit group includes multiple processing units. The scheduler is connected to the multiple processing units and the memory. The scheduler includes one or more arbitration units for arbitrating memory access requests of the multiple processing units. The arbitration unit includes multiple input terminals. The arbitration mode of the arbitration unit includes a priority mode and a weight mode. In the priority mode, the arbitration unit forwards the memory access requests from the corresponding input terminals to the memory in sequence according to the respective priorities of the multiple input terminals. In the weight mode, the arbitration unit allocates time slots to the memory access requests from the corresponding input terminals in proportion according to the respective weights of the multiple input terminals, and forwards the corresponding memory access requests to the memory in the allocated time slots. The processing unit group is set in the processing layer of the artificial intelligence chip, and the memory is set in the storage layer of the artificial intelligence chip. The processing layer and the storage layer are stacked. The method includes: Acquire multiple sub-computing tasks obtained based on the computing task, and assign the multiple sub-computing tasks to corresponding processing units; storing the data involved in the processing of the sub-computing task by the processing unit in a memory of a processor-memory module where the processing unit is located; The sub-computing task is processed by the processing unit, wherein the access operation of the processing unit to the memory is scheduled by a scheduler in a processor-memory module where the processing unit is located.

19. The method according to claim 18, characterized in that: The processing layer and the storage layer are stacked in a vertical direction along the surface of the artificial intelligence chip; The scheduler is arranged in the storage layer; The processing unit is connected to the scheduler through a data transmission line along the vertical direction, and the scheduler is connected to the memory through a data transmission line arranged in the memory layer.

20. The method according to claim 19, characterized in that The plurality of processing units are respectively connected to the scheduler via corresponding data transmission lines along the vertical direction.

21. The method according to claim 19, characterized in that: The processing unit includes a cache, and the cache is used to store data involved in the sub-computing task; The processing unit is connected to the scheduler via a data transmission line along the vertical direction, including: a cache of the processing unit is connected to the scheduler via a data transmission line along the vertical direction.

22. The method according to claim 19, characterized in that In the processor-memory module, the positions of the memory, the scheduler, and the processing unit group are set to maximize the overlapping area of ​​the memory and the scheduler projected with the processing unit group on the surface of the artificial intelligence chip.

23. The method according to claim 18, characterized in that: The processing layer and the storage layer are stacked in a vertical direction along the surface of the artificial intelligence chip; The scheduler is arranged in the processing layer; The processing unit is connected to the scheduler through a data transmission line arranged in the processing layer, and the scheduler is connected to the memory through a data transmission line along the vertical direction.

24. The method according to claim 23, characterized in that The plurality of processing units are respectively connected to the scheduler via corresponding data transmission lines arranged in the processing layer.

25. The method according to claim 23, characterized in that: The processing unit includes a cache, and the cache is used to store data involved in the sub-computing task; The processing unit is connected to the scheduler via a data transmission line arranged in the processing layer, including: a cache of the processing unit is connected to the scheduler via a data transmission line arranged in the processing layer; The scheduler is connected to the memory through a data transmission line along the vertical direction.

26. The artificial intelligence chip according to claim 23, characterized in that: In the processor-memory module, the positions of the memory, the scheduler, and the processing unit group are set to maximize the overlapping area of ​​the scheduler, the processing unit group, and the memory projected on the surface of the artificial intelligence chip.

27. The method according to claim 18, characterized in that: The processing unit groups in the plurality of processor-memory modules are arranged in the same processing layer; The memories in the plurality of processor-memory modules are arranged in the same processing layer.

28. The method according to claim 18, characterized in that The artificial intelligence chip further includes a controller, wherein the acquiring of a plurality of sub-computing tasks based on the computing task and assigning the plurality of sub-computing tasks to corresponding processing units includes: The computing task is acquired through the controller, the computing task is parsed to obtain a plurality of sub-computing tasks, and the plurality of sub-computing tasks are allocated to corresponding processing units.

29. The method according to claim 28, characterized in that The arbitration unit includes a configuration register, and the method further includes: The arbitration mode of the arbitration unit is configured by setting the value of the configuration register.

30. The method according to claim 28, characterized in that Also includes: By means of the controller, according to the execution order of the corresponding sub-computing tasks processed by the plurality of processing units, setting the priority of the input terminal of the arbitration unit; and / or The controller sets the weight of the input terminal of the arbitration unit according to the memory access frequency of the corresponding sub-computing tasks processed by the multiple processing units.

31. The method according to claim 18, characterized in that: When the scheduler includes an arbitration unit, the arbitration unit is connected to a plurality of processing units in the processing unit group; When the scheduler includes a plurality of arbitration units, the plurality of arbitration units are connected to form an arbitration tree, a root node of the arbitration tree is connected to the memory, and leaf nodes of the arbitration tree are connected to corresponding processing units.

32. The method according to claim 31, characterized in that The arbitration tree includes a binary tree structure.

33. The method according to claim 31, characterized in that When the scheduler includes a plurality of arbitration units, the method further includes: By setting the value of the configuration register, the plurality of arbitration units are configured to adopt the same arbitration mode, or at least one of the plurality of arbitration units is configured to adopt an arbitration mode different from that of the other arbitration units.

34. The method according to claim 18, characterized in that Also includes: The processing units in different processing unit groups are interconnected through data transmission lines and perform data transmission to collaboratively complete the computing task.

35. An electronic device, characterized in that: An artificial intelligence chip comprising an integrated on-chip memory array according to any one of claims 1-17.

Images