A processing method, an NPU and an electronic device

Abstract

The embodiment of the application discloses a processing method, which is applied to an NPU, and the NPU comprises a command scheduler, a data management unit, an operation unit and a local memory, wherein the data management unit is electrically connected with an external memory, and the command scheduler decodes a layer command of a current to-be-processed object to obtain a clock accuracy command of the current to-be-processed object, and sends the clock accuracy command of the current to-be-processed object to the operation unit; the operation unit performs operation on data of the current to-be-processed object obtained from the local memory according to the clock accuracy command of the current to-be-processed object, obtains an operation result of the current to-be-processed object, stores the operation result of the current to-be-processed object to the local memory, and the local memory outputs the operation result of the current to-be-processed object to the data management unit. The embodiment of the application also simultaneously provides an NPU and an electronic device.

Description

TECHNICAL FIELD

[0001] The present application relates to a data processing technology of a neural-network processing unit (NPU), in particular to a processing method, an NPU and an electronic device. BACKGROUND

[0002] At present, in the related art, the execution flow of the NPU is as follows: instruction analysis, instruction decoding, data fetching, instruction distribution, and operation. After the analysis and decoding of the executed instruction, the starting address, the destination address, and the data size of the data to be moved are obtained, so as to realize the movement of the data required by the current instruction. In the related art, the decoding, data fetching, and other operations of the next instruction can be performed in advance when the current operation unit is operating by increasing the operation queue, so as to improve the system performance. However, when processing massive multimedia data, the processing capacity of the NPU cannot be improved by using this method. It can be seen that the existing NPU has the technical problem of low processing efficiency. SUMMARY

[0003] The embodiments of the present application provide a processing method, an NPU and an electronic device, which can improve the processing efficiency of the NPU.

[0004] The technical solution of the present application is implemented as follows:

[0005] In a first aspect, the embodiments of the present application provide a processing method, which is applied to an NPU, and the NPU includes a command scheduler, a data management unit, an operation unit, and a local memory. The data management unit is electrically connected with an external memory and includes:

[0006] The command scheduler decodes a layer command of a current to-be-processed object to obtain a clock accuracy command of the current to-be-processed object, and sends the clock accuracy command of the current to-be-processed object to the operation unit.

[0007] The operation unit performs operation on data of the current to-be-processed object obtained from the local memory according to the clock accuracy command of the current to-be-processed object, obtains an operation result of the current to-be-processed object, and stores the operation result of the current to-be-processed object to the local memory. The data of the current to-be-processed object is obtained from the external memory by the data processing unit and sent to the local memory.

[0008] The local memory outputs the operation result of the current to-be-processed object to the data management unit.

[0009] In a second aspect, an embodiment of the present application provides an NPU, comprising: a command scheduler, a data management unit, an operation unit and a local memory; wherein the data management unit is electrically connected with an external memory; wherein the NPU executes the processing method according to any one or more of the above embodiments.

[0010] In a third aspect, an embodiment of the present application provides an electronic device, comprising: an NPU according to any one or more of the above embodiments.

[0011] An embodiment of the present application provides a processing method, an NPU and an electronic device. The NPU comprises: a command scheduler, a data management unit, an operation unit and a local memory, wherein the data management unit is electrically connected with an external memory, and the NPU executes the processing method according to any one or more of the above embodiments. In the processing method, the command scheduler decodes a layer command of a current to-be-processed object to obtain a clock precision command of the current to-be-processed object, and sends the clock precision command of the current to-be-processed object to the operation unit. The operation unit performs operation on data of the current to-be-processed object obtained from the local memory according to the clock precision command of the current to-be-processed object to obtain an operation result of the current to-be-processed object, and stores the operation result of the current to-be-processed object in the local memory. The data of the current to-be-processed object is obtained from the external memory by the data management unit and sent to the local memory. The local memory outputs the operation result of the current to-be-processed object to the data management unit. That is, in the embodiment of the present application, when the NPU processes the current to-be-processed object, the command scheduler decodes the layer command of the current to-be-processed object and sends the obtained clock precision command of the current to-be-processed object to the operation unit. When the operation unit receives the clock precision command of the current to-be-processed object, the data of the current to-be-processed object has been obtained from the external memory by the data management unit and stored in the local memory. Therefore, the operation unit can directly process the data of the current to-be-processed object according to the clock precision command of the current to-be-processed object to obtain the operation result of the current to-be-processed object. In this way, the NPU can send data to the local memory before sending instructions, so as to avoid the situation that instructions arrive at the operation unit while data does not arrive at the local memory, thereby improving the processing efficiency of the NPU. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 An optional flowchart of a processing method according to an embodiment of the present application is shown in the figure.

[0013] Figure 2 A structural diagram of an NPU in the related art is shown in the figure.

[0014] Figure 3 A flowchart of a processing method executed by an NPU in the related art is shown in the figure.

[0015] Figure 4 A schematic diagram of the structure of an optional NPU example provided in this application embodiment;

[0016] Figure 5 A flowchart illustrating an example of an optional processing method provided in an embodiment of this application;

[0017] Figure 6 A schematic diagram of an optional NPU provided for an embodiment of this application;

[0018] Figure 7 This is a schematic diagram of an optional electronic device provided in an embodiment of this application. Detailed Implementation

[0019] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0020] This application provides a processing method applied in an NPU, which includes a command scheduler, a data management unit, a processing unit, and local memory, wherein the data management unit is electrically connected to external memory. Figure 1 A flowchart illustrating an optional processing method provided in an embodiment of this application is shown below. Figure 1 As shown, the processing method may include:

[0021] S101: The command scheduler decodes the layer command of the current object to be processed, obtains the command of the clock precision of the current object to be processed, and sends the command of the clock precision of the current object to the arithmetic unit.

[0022] Figure 2 This is a schematic diagram of the structure of an NPU in related technologies, such as... Figure 2 As shown, the NPU includes an instruction cache module 21, a scalar instruction processing queue 22, an instruction dispatch module 23, a matrix operation queue 24, a vector operation queue 25, a storage conversion queue 26, and an event synchronization module 27. Figure 3 This is a flowchart illustrating the NPU execution processing method in related technologies. The method includes: instruction fetch 1, decoding 1, data fetch 1, instruction dispatch 1, arithmetic 1, instruction fetch 2, ..., where, based on... Figure 2 and Figure 3The execution flow of the NPU can include: the instruction cache module 21 obtains instructions from a compiler, i.e., fetch 1 and decode 1, and the scalar instruction processing queue 22 obtains data required by the instructions from an external memory and stores the data to a local memory, the instruction dispatch module 23 dispatches the instructions to the matrix operation queue 24, the vector operation queue 25 and the storage conversion queue 26, so that the matrix operation queue 24, the vector operation queue 25 and the storage conversion queue 26 perform operations on the required data according to the instructions to obtain operation results, the time synchronization module 27 is configured to synchronize the operation results, different operation units and different modules, and the processing of the instruction 2 is similar to that of the instruction 1, which will not be described herein.

[0023] The above processing manner can also improve the performance of the NPU by increasing the operation queue, but there is still a problem of low processing efficiency. In order to improve the processing efficiency of the NPU, an embodiment of the present application provides a processing method, which is applied to an NPU and includes a command scheduler (Cmd Scheduler), a data management unit, an operation unit and a local memory. The local memory can be a synchronous dynamic random access memory (SRAM), and the data management unit is electrically connected with an external memory. The external memory can be a double data rate synchronous dynamic random access memory (DDR SRAM).

[0024] Based on the above NPU, when processing each to-be-processed object, the layer command of the current to-be-processed object can be read from itself or an external object, the layer command of the current to-be-processed object is decoded to obtain the clock precision command of the current to-be-processed object, and the clock precision command of the current to-be-processed object is sent to the operation unit. In this way, the process of fetching, decoding and dispatching is completed.

[0025] The current to-be-processed object can be an image or a video, which is not limited in the embodiment of the present application.

[0026] S102: The operation unit performs operations on the data of the current to-be-processed object obtained from the local memory according to the clock precision command of the current to-be-processed object, to obtain operation results of the current to-be-processed object, and stores the operation results of the current to-be-processed object to the local memory.

[0027] S103: The local memory outputs the operation result of the current to-be-processed object to the data management unit.

[0028] After the operation unit receives the command of the clock precision of the current to-be-processed object sent by the command scheduler, the data of the current to-be-processed object has been stored in the local memory, wherein the data of the current to-be-processed object is obtained from the external memory by the data processing unit and then sent to the local memory. That is, when the operation unit receives the command of the clock precision of the current to-be-processed object, the data management unit has already moved the data of the current to-be-processed object to the local memory. Thus, after the operation unit receives the command of the clock precision of the current to-be-processed object, the operation unit can directly operate the data of the current to-be-processed object according to the command of the clock precision of the current to-be-processed object, obtain the operation result of the current to-be-processed object, and store the operation result of the current to-be-processed object in the local memory. The local memory outputs the operation result of the current to-be-processed object to the data management unit, and finally outputs the operation result of the current to-be-processed object to the NPU through the data management unit.

[0029] In this way, the data of the current to-be-processed object reaches the local memory before the time when the command of the clock precision of the current to-be-processed object reaches the operation unit, so that the processing efficiency of the NPU can be improved.

[0030] In order to make the data of the current to-be-processed object reach the local memory before the time when the command of the clock precision of the current to-be-processed object reaches the operation unit, in an optional embodiment, the above method further comprises:

[0031] When the command scheduler decodes the layer command of the current to-be-processed object, the data management unit obtains the data of the current to-be-processed object from the external memory and sends the data of the current to-be-processed object to the local memory.

[0032] It can be understood that when the command scheduler receives the processing instruction of the to-be-processed object, the command scheduler obtains the layer command of the current to-be-processed object and decodes the layer command of the current to-be-processed object, and at the same time, when the data management unit receives the processing instruction of the to-be-processed object, the data management unit obtains the data of the current to-be-processed object. In this way, the instruction fetching and data fetching are performed at the same time, so that when the operation unit receives the command of the clock precision of the current to-be-processed object, the data of the current to-be-processed object has been stored in the local memory, and thus the operation unit can directly operate the data of the current to-be-processed object, thereby improving the processing efficiency of the NPU.

[0033] In addition, in order to make the data of the current to-be-processed object reach the local memory before the time when the command of the clock precision of the current to-be-processed object reaches the operation unit, in an optional embodiment, the above method further comprises:

[0034] When the operation unit operates the data of the previous to-be-processed object according to the clock precision command of the previous to-be-processed object, the data management unit obtains the data of the current to-be-processed object from the external memory and sends the data of the current to-be-processed object to the local memory.

[0035] It can be understood that when the data of the previous to-be-processed object is operated, the data management unit obtains the data of the current to-be-processed object from the external memory and sends the data of the current to-be-processed object to the local memory, so that when the previous to-be-processed object is operated, the data of the current to-be-processed object is moved to the local memory, so that when the current to-be-processed object is operated, the data of the current to-be-processed object can be directly obtained from the local memory, thereby improving the processing efficiency of the NPU.

[0036] It should be noted that the data movement of the current to-be-processed object can occur when the instruction fetching and decoding of the current to-be-processed object occurs, and can also occur when the data of the previous to-be-processed object is operated, and of course can also occur at other times before the clock precision command of the current to-be-processed object reaches the operation unit. Here, the embodiments of the present application do not make specific limitations.

[0037] In addition, in order to implement instruction fetching and decoding, in an optional embodiment, the command scheduler decodes the layer command of the current to-be-processed object to obtain the clock precision command of the current to-be-processed object, including:

[0038] The command scheduler reads the layer command of the current to-be-processed object from the memory of the command scheduler;

[0039] The command scheduler decodes the layer command of the current to-be-processed object to obtain the clock precision command of the current to-be-processed object.

[0040] It can be understood that the memory is provided in the command scheduler, and the memory is used to store the layer command of the to-be-processed object, so that the command processor reads the layer command of the current to-be-processed object from the memory of the command scheduler, and then decodes the layer command of the current to-be-processed object to obtain the clock precision command of the current to-be-processed object.

[0041] In order to move the data of the current to-be-processed object to the local memory, in an optional embodiment, before the operation unit operates the data of the current to-be-processed object obtained from the local memory according to the clock precision command of the current to-be-processed object, the above method further includes:

[0042] The data management unit obtains the data of the current to-be-processed object from the external memory according to the data movement instruction of the current to-be-processed object;

[0043] The data management unit sends the data of the current to-be-processed object to the local memory.

[0044] It can be understood that, before the operation unit operates the data of the current to-be-processed object obtained from the local memory according to the clock precision command of the current to-be-processed object, the data management unit obtains the data of the current to-be-processed object from the external memory according to the data migration instruction of the current to-be-processed object, and sends the data of the current to-be-processed object to the local memory, so as to realize the data migration of the current to-be-processed object, and make the data of the current to-be-processed object reach the local memory before the time when the command of the clock precision of the current to-be-processed object reaches the operation unit.

[0045] In order to realize the migration of the data of the current to-be-processed object, in an optional embodiment, the data management unit obtains the data of the current to-be-processed object from the external memory according to the data migration instruction of the current to-be-processed object, comprising:

[0046] The data management unit reads the data migration instruction of the current to-be-processed object from the memory of the data management unit;

[0047] The data management unit obtains the data of the current to-be-processed object from the external memory according to the data migration instruction of the current to-be-processed object.

[0048] It can be understood that, the memory is arranged in the data management unit, and is used to store the data migration instruction of the to-be-processed object, so that the data management unit reads the data migration instruction of the current to-be-processed object from the memory of the data management unit, and obtains the data of the current to-be-processed object from the external memory according to the data migration instruction of the current to-be-processed object, so as to realize the migration of the data of the current to-be-processed object to the local memory before the time when the command of the clock precision of the current to-be-processed object reaches the operation unit, thereby improving the processing efficiency of the NPU.

[0049] In order to store the layer command and the data transfer command of the to-be-processed object into the memory of the object, in an optional embodiment, the method further comprises:

[0050] The command scheduler receives the layer command of at least two to-be-processed objects, and stores the layer command of the at least two to-be-processed objects into the memory of the command scheduler;

[0051] The data management unit receives the data transfer instruction of at least two to-be-processed objects, and stores the data transfer instruction of the at least two to-be-processed objects into the memory of the data management unit.

[0052] It can be understood that in order to implement the processing of the to-be-processed object, the command scheduler of the NPU stores the layer command of the at least two to-be-processed objects into the memory of the command scheduler after receiving the layer command of the at least two to-be-processed objects, and the data management unit stores the data transfer instruction of the at least two to-be-processed objects into the memory of the data management unit after receiving the data transfer instruction of the at least two to-be-processed objects. In this way, the command scheduler performs instruction decoding and distribution according to the layer command of each to-be-processed object, and the data management unit implements data transfer of the data of each to-be-processed object according to the data transfer instruction of each to-be-processed object, so as to implement the processing of the to-be-processed object.

[0053] In order to implement the processing of each to-be-processed object to improve the processing efficiency of the NPU, in an optional embodiment, after the command scheduler decodes the layer command of the current to-be-processed object to obtain the clock accuracy command of the current to-be-processed object, before sending the clock accuracy command of the current to-be-processed object to the operation unit, the above method further comprises:

[0054] The command scheduler sends an inquiry request to the data management unit.

[0055] After receiving the inquiry request, the data management unit determines that the data length of the data stored in the data management unit is greater than or equal to the preset threshold, and returns a success response message to the command scheduler.

[0056] The command scheduler receives the success response message and sends the clock accuracy command of the current to-be-processed object to the operation unit.

[0057] It can be understood that after obtaining the clock accuracy command of the current to-be-processed object, before sending the clock accuracy command of the current to-be-processed object to the operation unit, the command scheduler sends an inquiry request (Req) to the data management unit to inquire whether the data amount in the data processing unit is sufficient. After receiving the inquiry request, the data management unit determines whether the data length of the data stored in the data management unit is greater than or equal to the preset threshold. If it is greater than or equal to, it means that the data amount in the data management unit is sufficient, so the data management unit sends a success response message (Ack) to the command scheduler, so that the command scheduler sends the clock accuracy command of the current to-be-processed object to the operation unit after receiving the success response message. If it is less than, it means that the data amount in the data management unit is not sufficient, so the data management unit continues to transfer data from the external memory until the data amount in the data management unit is sufficient.

[0058] Thus, the command scheduler sends the command of the clock precision of the current to-be-processed object to the operation unit only when the amount of data in the data management unit is sufficient, so that the situation that the operation unit receives the command of the clock precision of the current to-be-processed object in advance and the local memory does not have the data of the current to-be-processed object is avoided.

[0059] To implement the processing of the to-be-processed object, in an optional embodiment, the operation unit comprises one or more of the following: a matrix multiplication operation unit, a vector operation unit, and a partial sum operation unit.

[0060] Exemplarily, when the operation unit comprises the matrix multiplication operation unit and the vector operation unit, the command scheduler distributes the command of the clock precision of the current to-be-processed object to the matrix multiplication operation unit and the vector operation unit according to the operation type, the matrix multiplication operation unit performs operation on the data of the current to-be-processed object obtained from the local memory according to the distributed command to obtain a first operation result, the vector operation unit performs operation on the data of the current to-be-processed object obtained from the local memory according to the distributed command to obtain a second operation result, the first operation result and the second operation result constitute the operation result of the current to-be-processed object, and are stored in the local memory.

[0061] Finally, to output the operation result of the current to-be-processed object to the external memory, in an optional embodiment, the method further comprises:

[0062] The command scheduler sends an update instruction to the data management unit when it is determined that the operation on the current to-be-processed object is completed.

[0063] The data management unit sends the operation result of the current to-be-processed object to the external memory after receiving the update instruction.

[0064] It can be understood that, since the command of the clock precision of the current to-be-processed object is decoded in the command scheduler, the command scheduler knows when the data in the local memory is consumed, and when the command scheduler determines that the operation on the current to-be-processed object is completed, at this time, the data in the local memory is consumed, the command scheduler sends an update instruction to the data management unit, so that the data management unit sends the operation result of the current to-be-processed object to the external memory after receiving the update instruction, thereby enabling the data management unit to release the address space in time.

[0065] The processing method described in one or more embodiments will be described below by way of example.

[0066] Figure 4 An optional structure schematic diagram of an example of an NPU provided in the embodiments of the present application is shown in FIG. 1. Figure 4As shown, the NPU core in this embodiment is composed of a Cmd scheduler 41, a Data management 42, a Matrix matrix product (MMP) 43, a Vector store processing (VSP) 44, a Partial sum (PSUM) 45, a Local Synchronous Dynamic Random Access Memory (Local SRAM) 46, and the like. The Cmd scheduler 41 is used for Layer command to Cycle command conversion, command decoding, and scheduling commands of different operation types to the MMP 43 or the VSP 44. The Cmd scheduler 41 includes a memory (cmd memory) 411 for storing Layer commands. The Data Management 42 is connected to an external DDR SRAM 47 and is used for data transfer management. The Data Management 42 includes a data memory 421 for storing data transfer commands. The MMP 43 is an operation unit for matrix product operations. The VSP 44 is an operation unit for vector operations. The PSUM 45 is an operation unit for partial sum operations. The Local SRAM 46 is a storage area for temporarily storing data required by the MMP 43 / VSP 44 / PSUM 45 operation units, for temporarily storing data transferred from the DDR SRAM 47, and for temporarily storing data after operations in the MMP 43 / VSP 44 / PSUM 45.

[0067] In order to implement data processing, after the NPU core is started, the Data Management 42 and the Cmd scheduler 41 modules are simultaneously started. The Data Management 42 analyzes data transfer instructions in the data memory 421 row by row, transfers data types / images and the like required by NPU operations to the Local SRAM 46, and supplies the MMP 43 / VSP 44 / PSUM 45 with the data for corresponding operations.

[0068] The Cmd scheduler 41 takes out commands in the cmd Memory 411 row by row, decodes the commands, and distributes the decoded commands to the MMP 43 or the VSP 44 module according to different operation types.

[0069] When the Cmd scheduler 41 sends the cmd to the MMP 43 / VSP 44, it sends a Req request to the Data management 42. After the Data management 42 receives the Req request, it judges whether the data amount in the data memory 421 is sufficient. If it is sufficient, it returns an Ack to the Cmd scheduler 41, so as to perform the MMP 43 / VSP 44 operation. When the data in the Local SRAM 46 is consumed, the Cmd scheduler 41 sends an update to the Data management 42, and the Data management 42 releases the corresponding address space.

[0070] Figure 5 An example of a flowchart of an optional processing method provided for the embodiment of the present application is shown in FIG. 1, which is as follows. Figure 5 As shown in FIG. 1, at the same time of fetching instruction 1, data 1 is fetched, then the instruction 1 is decoded, the instruction 1 is dispatched, and the operation 1 is performed. At the same time of the operation 1, data 2 is fetched, then the instruction 2 is fetched, the instruction 2 is decoded, and so on. The operation result of each processing object is obtained through the above-mentioned cycle.

[0071] That is, since the NPU is a data-dependent special-purpose operation unit, the data demand is large, so the present example adopts the mode of separating the cmd memory and the data memory. During the current instruction operation process, the data required by the next instruction can be carried at the same time. If the Local SRAM space is sufficient, even the data required by the next N instructions can be carried, so as to greatly improve the system bandwidth. The Data Management is used for the analysis and execution of the data carrying instruction. The Cmd scheduler is used for the instruction fetching, decoding, and instruction dispatching of the network control command. Since the cmd memory and the data memory are separated, the data required by the NPU operation can be sent to the Local SRAM in advance, instead of obtaining the source address and the destination address, the data size, and other parameters required by the data carrying in the related art after the cmd decoding. In the present example, the data required by the NPU operation is sent in advance of the current control command, so when the command reaches the MMP operation unit, the data required by the operation has been carried to the local SRAM, so as to greatly reduce the dependence on the data, improve the system bandwidth, and reduce the operation time delay of the system.

[0072] In the example, the cmd memory and the data memory are separated, data migration and cmd analysis are performed synchronously, and in the execution process, the NPU operation is accelerated by means of pipeline. The handshake synchronization is performed between the cmd scheduler and the data management through Req and Ack. After the cmd scheduler executes the cmd, the update is sent to the data management, so as to release the corresponding data space. In addition, the hardware realizes the conversion from the layer command to the cycle command, and reduces the development work of the compiler.

[0073] The embodiment of the application provides a processing method, and an NPU includes a command scheduler, a data management unit, an operation unit and a local memory, wherein the data management unit is electrically connected with an external memory, and the command scheduler decodes a layer command of a current to-be-processed object to obtain a clock precision command of the current to-be-processed object, and sends the clock precision command of the current to-be-processed object to the operation unit, the operation unit performs operation on data of the current to-be-processed object obtained from the local memory according to the clock precision command of the current to-be-processed object to obtain an operation result of the current to-be-processed object, and stores the operation result of the current to-be-processed object to the local memory; wherein the data of the current to-be-processed object is obtained from the external memory by the data processing unit and sent to the local memory, and the local memory outputs the operation result of the current to-be-processed object to the data management unit; that is, in the embodiment of the application, when the NPU processes the current to-be-processed object, the command scheduler decodes the layer command of the current to-be-processed object and sends the obtained clock precision command of the current to-be-processed object to the operation unit, and when the operation unit receives the clock precision command of the current to-be-processed object, the data of the current to-be-processed object has been obtained from the external memory by the data management unit in the local memory, so that the data of the current to-be-processed object is stored in the local memory when the operation unit operates, and then the operation unit can directly process the data of the current to-be-processed object according to the clock precision command of the current to-be-processed object, and further obtain the operation result of the current to-be-processed object, so that the NPU can make the data reach the local memory before the instruction, and avoid the situation that the instruction reaches the operation unit and the data does not reach the local memory, thereby improving the processing efficiency of the NPU.

[0074] Based on the same inventive concept, the embodiment of the application provides an NPU, Figure 6 An optional structure schematic diagram of the NPU is provided for the embodiment of the application, as shown in Figure 6As shown, the NPU includes: a command scheduler 61, a data management unit 62, an operation unit 63, and a local memory 64; the data management unit 62 is electrically connected with an external memory 65; the NPU performs the processing method as described in one or more of the above embodiments.

[0075] Figure 7 An optional structure schematic diagram of an electronic device provided by the embodiment of the present application is shown in FIG. 7. Figure 7 As shown, the embodiment of the present application provides an electronic device 700, which includes the NPU 71 as described in one or more of the above embodiments.

[0076] The computer readable storage medium can be a ferromagnetic random access memory (FRAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM), etc.

[0077] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage, etc.) containing computer-usable program code.

[0078] The present application is described with reference to the flowcharts and / or block diagrams of the method, device (system), and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as a combination of the flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1apparatus for performing the functions specified in the flow or flows and / or blocks. Figure 1

[0079] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the flow Figure 1 apparatus for performing the functions specified in the flow or flows and / or blocks. Figure 1

[0080] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow Figure 1 apparatus for performing the functions specified in the flow or flows and / or blocks. Figure 1

[0081] The above description is only preferred embodiments of the present application, and not intended to limit the protection scope of the present application.​​​

Claims

1. A processing method characterized by, The method is applied to an NPU, and the NPU comprises a command scheduler, a data management unit, an operation unit and a local memory; wherein the data management unit is electrically connected with an external memory and comprises: The command scheduler decodes a layer command of a current to-be-processed object to obtain a clock accuracy command of the current to-be-processed object, and sends the clock accuracy command of the current to-be-processed object to the operation unit; wherein the current to-be-processed object comprises at least one of an image and a video; The operation unit performs operation on data of the current to-be-processed object obtained from the local memory according to the clock accuracy command of the current to-be-processed object to obtain an operation result of the current to-be-processed object, and stores the operation result of the current to-be-processed object to the local memory; wherein the data of the current to-be-processed object is obtained from the external memory by the data management unit and then sent to the local memory; wherein the data of the current to-be-processed object reaches the local memory before the clock accuracy command of the current to-be-processed object reaches the operation unit; The local memory outputs the operation result of the current to-be-processed object to the data management unit.

2. The method of claim 1, wherein, The method further comprises: When the command scheduler decodes the layer command of the current to-be-processed object, the data management unit obtains the data of the current to-be-processed object from the external memory and sends the data of the current to-be-processed object to the local memory.

3. The method of claim 1, wherein, The method further comprises: When the operation unit performs operation on the data of the previous to-be-processed object obtained from the local memory according to the clock accuracy command of the previous to-be-processed object, the data management unit obtains the data of the current to-be-processed object from the external memory and sends the data of the current to-be-processed object to the local memory.

4. The method of claim 1, wherein, The command scheduler decodes a layer command of a current to-be-processed object to obtain a clock accuracy command of the current to-be-processed object, and sends the clock accuracy command of the current to-be-processed object to the operation unit; wherein the current to-be-processed object comprises at least one of an image and a video; The command scheduler reads the layer command of the current to-be-processed object from a memory of the command scheduler; The command scheduler decodes the layer command of the current to-be-processed object to obtain the clock accuracy command of the current to-be-processed object.

5. The method of claim 1, wherein, Before the operation unit performs operation on the data of the current to-be-processed object obtained from the local memory according to the clock accuracy command of the current to-be-processed object, the method further comprises: The data management unit obtains the data of the current to-be-processed object from the external memory according to a data moving instruction of the current to-be-processed object; The data management unit sends the data of the current to-be-processed object to the local memory.

6. The method of claim 5, wherein, The data management unit obtains the data of the current to-be-processed object from the external memory according to a data moving instruction of the current to-be-processed object, comprising: The data management unit reads the data moving instruction of the current to-be-processed object from a memory of the data management unit; The data management unit obtains data of the current to-be-processed object from the external memory according to a data moving instruction of the current to-be-processed object.

7. The method according to claim 4 or 6, characterized in that, The method further comprises: The command scheduler receives layer commands of at least two to-be-processed objects, and stores the layer commands of the at least two to-be-processed objects into a memory of the command scheduler; The data management unit receives data transfer instructions of the at least two to-be-processed objects, and stores the data transfer instructions of the at least two to-be-processed objects into a memory of the data management unit.

8. The method of claim 1, wherein, After the command scheduler decodes the layer command of the current to-be-processed object to obtain a clock-precision command of the current to-be-processed object, before sending the clock-precision command of the current to-be-processed object to the operation unit, the method further comprises: The command scheduler sends an inquiry request to the data management unit; After receiving the inquiry request, the data management unit returns a success response message to the command scheduler when a data length of the data stored in the data management unit is greater than or equal to a preset threshold value; The command scheduler receives the success response message, and sends the clock-precision command of the current to-be-processed object to the operation unit.

9. The method of claim 1, wherein, The operation unit comprises one or more of the following: a matrix product operation unit, a vector operation unit, and a partial sum operation unit.

10. The method of claim 1, wherein, The method further comprises: The command scheduler sends an update instruction to the data management unit when it is determined that the operation on the current to-be-processed object has been completed; After receiving the update instruction, the data management unit sends an operation result of the current to-be-processed object to the external memory.

11. An NPU, comprising: Comprise: a command scheduler, a data management unit, an operation unit, and a local memory; wherein the data management unit is electrically connected with an external memory; wherein the NPU executes the processing method as claimed in any one of claims 1 to 10.

12. An electronic device, comprising: comprise the NPU as claimed in claim 11.

Images