Operation unit lock control method and device and electronic equipment

Abstract

The invention relates to the technical field of neural network processors, and provides an arithmetic unit lock control method and device and electronic equipment. The method comprises the following steps: in response to a to-be-authorized instruction, obtaining the computing power of a target virtual neural network processor in operation of a distributor; if the computing power of the target virtual neural network processor meets a preset condition, enabling the target virtual neural network processor to obtain a control right for at least one computing unit; obtaining the working state of at least one arithmetic unit in response to the to-be-finished-of-work instruction; and if the working state of all the at least one arithmetic unit is that the work is finished, releasing the control right of the target virtual neural network processor for the at least one arithmetic unit. According to the method, the operation unit is locked by executing the instruction to be authorized; the operation unit is unlocked by executing the to-be-finished instruction, so that the utilization rate of the operation unit is improved, and the working efficiency of the neural network processor is further improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of neural network processors, and in particular to a method, device and electronic device for controlling a computing unit lock. Background Art

[0002] In the current application scenarios of Neural Processing Unit (NPU) chips, there is a need for multiple users to run simulations simultaneously. Multiple users can run simultaneously through hard partitioning, but the hard partitioning implementation method requires multiple computing units and memories, which increases the chip area and production costs. Therefore, the reuse of computing units can be achieved through time division multiplexing.

[0003] The current time-division multiplexing method is that multiple virtual neural network processors (vNPUs) take turns to use the same computing unit. Taking the case of two vNPUs (vNPU0 and vNPU1) as an example, the priority of vNPU0 is fixed directly, and a timer is used to control the computing unit to be used by vNPU0 in a certain time period and by vNPU1 in another specific time period, so as to achieve time-sharing sharing of the same computing unit by multiple vNPUs.

[0004] However, the above time-division multiplexing method will cause a waste of computing power: first, because the time allocated to a vNPU will be longer than the actual time for the vNPU to run the simulation, the computing unit will remain idle after the vNPU actually completes the simulation task, and this idle time cannot be used by other vNPUs, reducing the overall computing efficiency of the chip. Secondly, this design has low flexibility, and the priority and rotation method directly fixed by the hardware cannot be dynamically adjusted according to the actual operation situation, which limits the performance and resource utilization efficiency of the chip under different circumstances. Summary of the invention

[0005] To solve the above problems, the present application provides a method, device and electronic device for controlling a computing unit lock, which can solve the technical problem that the computing unit utilization rate in the NPU is low, resulting in low NPU computing efficiency.

[0006] To achieve the above-mentioned purpose, in a first aspect, the present application provides a computing unit lock control method, which is applied to an NPU, wherein the NPU includes at least one computing unit and a distributor, and the distributor runs at least one vNPU. The computing unit lock control method includes: in response to an instruction to be authorized, obtaining the computing power of the target vNPU running by the distributor, and the instruction to be authorized is used to instruct the target vNPU to obtain authorization; if the computing power of the target vNPU meets the preset conditions, the instruction to be authorized is executed successfully, so that the target vNPU obtains control over at least one computing unit; in response to an instruction to be completed, obtaining the working status of at least one computing unit, and the instruction to be completed is used to instruct at least one computing unit to feedback the working status, and the working status includes the end of work; if the working status of all at least one computing unit is the end of work, the instruction to be completed is executed successfully, and the control of the target vNPU over the at least one computing unit is released.

[0007] The computing unit lock control method provided in the present application can determine whether the target vNPU can obtain the control right of the computing unit by executing the instruction to be authorized. When the target vNPU obtains the control right of the computing unit, other vNPUs can no longer control the computing unit, thereby locking the computing unit; then, by executing the instruction to be completed, it is determined whether the target vNPU releases the control right of the computing unit. After the target vNPU releases the control right, other vNPUs can obtain the control right of the computing unit according to the instruction to be authorized, thereby unlocking the computing unit. Avoid idle time of the computing unit, improve the utilization rate of the computing unit, and thus improve the work efficiency of the NPU.

[0008] In an implementation method of the first aspect, the allocator includes a current limiting module; in response to an instruction to be authorized, the computing power of the target vNPU on which the allocator is running is obtained, including: determining the target vNPU according to the instruction to be authorized; reading computing power information related to the target vNPU through the current limiting module, the computing power information including at least one of the computing power currently being used by the target vNPU, the remaining computing power, and the computing power usage history data.

[0009] In the above method, the computing power information of the target vNPU is obtained through the current limiting module, and whether the target NPU can obtain the control right of the computing unit can be determined according to the computing power information to achieve locking of the computing unit.

[0010] In an implementation method of the first aspect, the preset conditions include: within a preset time, the remaining computing power of the target vNPU is greater than or equal to the computing power required to perform the current work; wherein the computing power required to perform the current work is estimated based on the complexity of the current work and the required computing resources.

[0011] In the above method, the preset time can be the time before the computing unit starts to execute the work of the target vNPU, or the time after the computing unit completes the work of the target vNPU. Judging whether the target vNPU can obtain the control right of the computing unit according to the preset conditions can improve the flexibility of locking the computing unit.

[0012] In an implementation manner of the first aspect, the successful execution of the instruction to be authorized includes: the instruction type of the instruction to be authorized is correct; the instruction to be authorized passes verification; and the target vNPU successfully obtains usage information of all at least one computing unit.

[0013] In the above method, when the instruction to be authorized is correctly executed, the usage information of all the operation units can be obtained, the operation units can be locked, and the utilization rate of the operation units can be improved.

[0014] In an implementation method of the first aspect, if the computing power of the target vNPU meets the preset conditions, after the authorization instruction is successfully executed and the target vNPU obtains the control right over at least one computing unit, the computing unit lock control method also includes: obtaining a computing instruction, the computing instruction is used to instruct at least one computing unit to start a computing function; and controlling the computing unit to execute the work of the target vNPU according to the computing instruction.

[0015] In the above method, after the target vNPU obtains the control right of the computing unit, it starts to execute the computing instruction, so that the computing unit starts to execute the work in the target vNPU, thereby realizing the control of the computing unit by the target vNPU.

[0016] In an implementation method of the first aspect, in response to an instruction to complete work, the working status of at least one computing unit is obtained, including: sending a working status query request to at least one computing unit, the working status query request is used to request at least one computing unit to continuously send the working status; receiving the working status sent by at least one computing unit.

[0017] In an implementation manner of the first aspect, the successful execution of the instruction to be completed includes: the instruction type of the instruction to be completed is correct; the instruction to be completed is verified; and the working status of at least one computing unit is successfully acquired.

[0018] In the above method, by executing the waiting work completion instruction, it can be determined whether the computing unit has completed the task of the target vNPU. If the task is completed, the target vNPU can release the control right of the computing unit to achieve unlocking, avoid idle time of the computing unit, and improve the utilization rate of the computing unit.

[0019] In an implementation method of the first aspect, the instruction to be authorized is a binary instruction with a length of 64 bits, including: bits 0 to 7 are instruction type bits, used to indicate that the type of the binary instruction is an instruction to be authorized; bits 8 to 12 are check bits, used to check whether the reading of the instruction to be authorized is correct; bits 13 to 17 are reserved spaces for function expansion; bit 18 is a new job judgment bit, used to judge whether the current job is a new job; bit 19 is a dot judgment bit, used to judge whether it is necessary to print the time point when the instruction to be authorized ends; bits 20 to 35 are operation unit enable mark bits, used to mark whether all at least one operation unit is enabled; bits 36 to 63 are reserved spaces for function expansion.

[0020] In the above method, the instruction to be authorized can obtain the enabled state of the computing unit, and can also implement extended functions by reserving empty spaces, thereby improving the working efficiency and scalability of the NPU.

[0021] In an implementation method of the first aspect, the instruction to be completed is a binary instruction with a length of 64 bits, including: bits 0 to 7 are instruction type bits, used to indicate that the type of the binary instruction is the instruction to be completed; bits 8 to 12 are check bits, used to check whether the reading of the instruction to be completed is correct; bits 13 to 17 are reserved spaces for function expansion; bit 18 is a new work judgment bit, used to judge whether the current work is a new job; bit 19 is a dot judgment bit, used to judge whether it is necessary to print the time point when the instruction to be completed needs to be printed; bits 20 to 27 are operation unit working status check bits, used to mark whether all at least one operation unit needs to check the working status; bits 28 to 63 are reserved spaces for function expansion.

[0022] In the above method, the work completion instruction can obtain the working status of the operation unit, and can also realize the expansion function by reserving empty spaces, thereby improving the working efficiency and scalability of the NPU.

[0023] In the second aspect, the present application also provides an operation unit lock control device, which is applied to NPU, the NPU includes at least one operation unit and a distributor, the distributor runs at least one vNPU, and the operation unit lock control device includes: a computing power acquisition module, configured to: in response to an instruction to be authorized, obtain the computing power of the target vNPU running by the distributor, and the instruction to be authorized is used to instruct the target vNPU to obtain authorization; a locking module, configured to: if the computing power of the target vNPU meets the preset conditions, the instruction to be authorized is executed successfully, so that the target vNPU obtains control over at least one operation unit; a working status acquisition module, configured to: in response to an instruction to be completed, obtain the working status of at least one operation unit, and the instruction to be completed is used to instruct at least one operation unit to feedback the working status, and the working status includes the end of work; an unlocking module, configured to: if the working status of all at least one operation unit is the end of work, the instruction to be completed is executed successfully, and the control of the target vNPU over at least one operation unit is released.

[0024] In a third aspect, the present application also provides an electronic device, comprising: one or more processors; a memory configured to store one or more programs; wherein, when the one or more programs are executed by one or more processors, the one or more processors implement the operation unit lock control method in the first aspect and any optional implementation method thereof.

[0025] It can be understood that the beneficial effects that can be achieved by the technical solutions of the second and third aspects provided above can refer to the beneficial effects of the first aspect and any optional implementation method thereof, and will not be repeated here.

[0026] It can be seen from the above technical scheme that the present application provides a method, device and electronic device for controlling a computing unit lock, which are applied to an NPU. The NPU includes at least one computing unit and a distributor, and the distributor runs at least one vNPU. The computing unit lock control method includes: in response to an instruction to be authorized, obtaining the computing power of the target vNPU being run by the distributor, and the instruction to be authorized is used to instruct the target vNPU to obtain authorization; if the computing power of the target vNPU meets the preset conditions, the instruction to be authorized is executed successfully, so that the target vNPU obtains control over at least one computing unit; in response to an instruction to be completed, obtaining the working status of at least one computing unit, and the instruction to be completed is used to instruct at least one computing unit to feedback the working status, and the working status includes the end of work; if the working status of all at least one computing unit is the end of work, the instruction to be completed is executed successfully, and the control of the target vNPU over at least one computing unit is released.

[0027] The computing unit lock control method provided in the present application can determine whether the target vNPU can obtain the control right of the computing unit by executing the instruction to be authorized. When the target vNPU obtains the control right of the computing unit, other vNPUs can no longer control the computing unit, thereby locking the computing unit; then, by executing the instruction to be completed, it is determined whether the target vNPU releases the control right of the computing unit. After the target vNPU releases the control right, other vNPUs can obtain the control right of the computing unit according to the instruction to be authorized, thereby unlocking the computing unit. Avoid idle time of the computing unit, improve the utilization rate of the computing unit, and thus improve the work efficiency of the NPU. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0029] Figure 1 A schematic diagram of an NPU structure provided in an embodiment of the present application; Figure 2 A schematic diagram of a computing unit lock control method provided in an embodiment of the present application; Figure 3 A schematic diagram of a method for determining whether a target vNPU obtains control of a computing unit provided in an embodiment of the present application; Figure 4 A schematic diagram of a computing unit lock control device provided in an embodiment of the present application; Figure 5 A schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following embodiments do not represent all embodiments consistent with the present application.

[0031] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0032] The terms "first", "second", "third", etc. in the specification and the above drawings of this application are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise specified. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0033] To facilitate understanding of the solution, the following explains the relevant terms: Neural network processor: A neural network processor is a processor specifically designed to process neural network algorithms. It can efficiently perform mathematical operations involved in deep learning and machine learning tasks, such as matrix multiplication and convolution operations.

[0034] Distributor: The distributor is the key hardware in the neural network processor. Its main function is to allocate and schedule data and computing resources to ensure that data and computing tasks flow between the various modules of the neural network processor.

[0035] Arithmetic unit: The arithmetic unit is the core hardware responsible for performing various mathematical operations in the neural network algorithm. It is the basic execution unit of the NPU to complete computing tasks and directly participates in the calculation process of the neural network, such as performing matrix multiplication, addition, convolution operation, pooling operation, activation function calculation and other operations.

[0036] Instructions: Instructions are used to instruct the neural network processor how to perform various basic operations, such as data reading, storage, arithmetic operations (addition, subtraction, multiplication, and division), logical operations (AND, OR, and NOT), control flow operations (jumps, loops, branches), etc.

[0037] NPU can process neural network algorithms through computing units to realize the functions of neural network algorithms, such as image processing, speech recognition, etc. When using NPU, there is a need to process multiple different functions and data at the same time, that is, the same NPU needs to perform both image processing and speech recognition. This requirement can be achieved through hard segmentation, but the hard segmentation implementation method requires multiple computing units and memories, which increases the chip area of ​​the NPU and increases the production cost. Therefore, the reuse of computing units can be achieved through time division multiplexing.

[0038] The basic logic of the time-division multiplexing method is that multiple vNPUs can run in the same NPU, each vNPU represents a neural network algorithm that implements different functions, and multiple vNPUs take turns using the same computing unit. Taking the case of two vNPUs (vNPU0 and vNPU1) as an example, the priority of vNPU0 is directly fixed, and a timer is used to control the computing unit to be used by vNPU0 in a certain time period and by vNPU1 in another specific time period, so as to realize the time-sharing sharing of the same computing unit by multiple vNPUs.

[0039] However, since the time allocated to a vNPU is longer than the time the vNPU actually runs the simulation, the computing unit is still idle after the vNPU actually completes the simulation task, and this idle time cannot be used by other vNPUs, reducing the overall computing efficiency and utilization of the NPU.

[0040] To solve the above problems, the embodiments of the present application provide a method, device and electronic device for controlling the lock of an operation unit. By adding instructions to be authorized, the target vNPU that executes the instructions to be authorized obtains the control right over the operation unit; and by using instructions to be completed, the vNPU that currently has the control right over the operation unit releases its control right, thereby realizing time division multiplexing of the operation unit based on lock control, avoiding idle time of the operation unit, and improving the operation efficiency and utilization rate of the NPU.

[0041] In some embodiments, instructions for instructing the NPU to work may be generated by a programming language (such as C, C++, Java, etc.) and packaged and sent to the NPU.

[0042] Figure 1 A schematic diagram of an NPU structure provided in an embodiment of the present application.

[0043] like Figure 1 As shown, the NPU includes an on-chip memory (OCM), a distributor, and at least one operation unit, for example, operation unit 1, operation unit 2, operation unit 3, and operation unit 4. The distributor includes a current limiting module, a decoding module, and an instruction loading module, and the operation unit includes a control module and a logic processing module (the operation unit 1 is taken as an example in the figure).

[0044] In some embodiments, the instructions are first stored in a double data rate synchronous dynamic random access memory (DDR). The OCM in the NPU needs to fetch instructions from the DDR to instruct the NPU to run the neural network algorithm. Figure 1 The NPU in the system is in a two-level control mode. First, the instruction loading module in the distributor reads the first-level instruction, and then the decoding module decodes the first-level instruction. The decoded first-level instruction is used to instruct the operation unit to perform the corresponding work. At this time, the control module in the operation unit obtains and decodes the second-level instruction according to the instruction of the first-level instruction. The decoded second-level instruction is used to instruct the logic processing module to perform specific operation work.

[0045] Below, based on Figure 1The NPU shown introduces the specific implementation of the operation unit lock control method. It should be understood that the operation unit lock control method provided in the embodiment of the present application is used to realize that one NPU can process multiple different functions and data at the same time, so multiple vNPUs can be run in the NPU distributor, and different vNPUs are neural network models for implementing different functions.

[0046] Figure 2 Schematic diagram of a computing unit lock control method provided in an embodiment of the present application. Figure 2 As shown, the operation unit lock control method includes steps S100-S400.

[0047] S100: In response to the instruction to be authorized, obtain the computing power of the target vNPU that the allocator is running.

[0048] In some embodiments, in order to realize time-division multiplexing of computing units in the NPU by multiple vNPUs and improve the flexibility of multiple vNPUs running in the NPU allocator to obtain control over the computing units, when instructions for instructing the NPU to work are generated by a programming language, authorization instructions can be added to instruct the target vNPU to obtain authorization.

[0049] Specifically, the instruction to be authorized can be a binary instruction with a length of 64 bits, including: bits 0 to 7 are instruction type bits, used to indicate that the type of the binary instruction is an instruction to be authorized; bits 8 to 12 are check bits, used to check whether the reading of the instruction to be authorized is correct; bits 13 to 17 are reserved spaces for function expansion; bit 18 is a new job judgment bit, used to judge whether the current job is a new job; bit 19 is a dot judgment bit, used to judge whether it is necessary to print the time point when the instruction to be authorized ends; bits 20 to 35 are operation unit enable mark bits, used to mark whether at least one operation unit is enabled; bits 36 to 63 are reserved spaces for function expansion.

[0050] Exemplarily, Table 1 is an instruction format of an instruction to be authorized.

[0051] Table 1 .

[0052] In the pending authorization instructions shown in Table 1, bits 0 to 7 indicate that the instruction type is a pending authorization instruction, which is represented by an 8-bit hexadecimal number in the table and converted to binary as 10101010; bits 8 to 12 are the CRC check value of the instruction, which is used to check whether there is an error in the instruction reading process; bits 13 to 17 indicate reserved spaces, which can be used to add other functions to the instruction later; bit 18 indicates whether the current job is a new job, for example, 0 indicates that it is not a new job, and 1 indicates that it is a new job; bit 19 indicates whether the time point at which the instruction ends needs to be printed, for example, 0 indicates that printing is not required, and 1 indicates that printing is required; bits 20 to 35 indicate the enabling status of each operation unit during the control process of the operation unit by the vNPU this time, for example, the NPU includes a total of 16 operation units, and each bit from bits 20 to 35 indicates the enabling status of an operation unit, 0 indicates disabled, and 1 indicates enabled. In addition, if the number of computing units in the NPU is less than 16, only the values ​​of the bits equal to the number of computing units are set, and the other bit positions are empty. For example, if the NPU includes 8 computing units, only the values ​​of bits 20 to 27 are set, and bits 28 to 35 are empty. The values ​​of bits 20 to 35 can be 0000000010011101, indicating that in the control process of the computing units by the vNPU this time, the 0th, 2nd, 3rd, 4th, and 7th computing units are enabled; bits 36 to 63 are reserved for setting the enabling status of the excess computing units when the number of computing units in the NPU exceeds 16. For example, if the NPU includes 20 computing units, bits 36 to 39 are used to indicate the enabling status of the other 4 computing units exceeding 16, 0 means disabled, and 1 means enabled.

[0053] It should be noted that the instruction to be authorized may also be in other instruction formats, as long as it can indicate the enabling status of the operation unit, and no specific limitation is made in the embodiments of the present application.

[0054] In some embodiments, assuming that two vNPUs, vNPU0 and vNPU1, can run in the distributor, and the vNPU currently running in the distributor is vNPU0, then vNPU0 is recorded as the target vNPU. At this time, the target vNPU needs to obtain the control right of the computing unit. Therefore, when the distributor runs to the target vNPU's pending authorization instruction, it is necessary to determine whether the pending authorization instruction can be successfully executed based on the computing power of the target vNPU, that is, whether the target vNPU can obtain the control right of the computing unit. Therefore, it is necessary to obtain the computing power of the target vNPU.

[0055] In some embodiments, to obtain the computing power of the target vNPU, it is first necessary to determine the target vNPU according to the instruction to be authorized. For example, the target vNPU is determined to be vNPU0.

[0056] Secondly, the computing power information related to the target vNPU is read from the current limiting module, and the computing power information includes at least one of the computing power currently being used by the target vNPU, the remaining computing power, and the computing power usage history data. In some embodiments, the computing power information of the vNPU can be determined based on the token bucket principle. Specifically, a token bucket is allocated to each vNPU, and a preset number of tokens are added to the token bucket at every preset time interval; during the operation of the vNPU, a preset number of tokens are deducted from the token bucket of the vNPU in each operation cycle. Based on this, the current limiting module can obtain the number of tokens in the token bucket of the target vNPU at any time, that is, the computing power of the target vNPU.

[0057] It should be noted that in addition to the token bucket principle, other methods can also be used to obtain computing power information related to the target vNPU, which is not specifically limited in the embodiments of the present application.

[0058] S200: If the computing power of the target vNPU meets the preset conditions, the authorization instruction is executed successfully, so that the target vNPU obtains control over at least one computing unit.

[0059] In some embodiments, in order to improve the accuracy of locking the computing unit, it is necessary to determine whether the target vNPU can obtain the control right of the computing unit according to the computing power of the target vNPU.

[0060] Figure 3 A schematic diagram of a method for determining whether a target vNPU obtains control of a computing unit provided in an embodiment of the present application.

[0061] like Figure 3 As shown, first, step S301 is performed: obtaining the computing power of the target vNPU. For example, there are 10 tokens in the token bucket of the target vNPU.

[0062] Then execute step S302: determine whether the computing power of the target vNPU meets the preset condition. In some embodiments, the preset condition may be: within a preset time, the remaining computing power of the target vNPU is greater than or equal to the computing power required to perform the current work; wherein the computing power required to perform the current work is estimated based on the complexity of the current work and the required computing resources.

[0063] It should be noted that the computing power required for the target vNPU to perform the current work can be set according to actual needs, for example, setting the token in the token bucket of the target vNPU to be no less than 1, or setting the token in the token bucket of the target vNPU to be greater than or equal to the sum of the number of tokens required for the target vNPU to complete the current work. No specific limitation is made in the embodiments of the present application.

[0064] Exemplarily, if the preset time is the time point when the instruction to be authorized is successfully executed, that is, the time point when the target vNPU obtains control of the computing unit and controls the computing unit to start executing the current work of the target vNPU, then the number of tokens in the token bucket of the target vNPU is not less than 1, which means that the preset conditions are met.

[0065] As another example, if the preset time is the time point when the computing unit completes the current work of the target vNPU, the number of tokens in the token bucket of the target vNPU needs to be greater than or equal to the sum of the number of tokens required for the target vNPU to complete the current work in order to meet the preset conditions.

[0066] In some embodiments, if the computing power of the target vNPU meets the preset conditions, step S303 is executed: the authorization instruction is executed successfully, so that the target vNPU obtains control over at least one computing unit.

[0067] Specifically, the successful execution of the instruction to be authorized requires the following: the instruction type of the instruction to be authorized is correct; the instruction to be authorized is verified; the target vNPU successfully obtains the usage information of at least one computing unit. For example, if the instruction type of the instruction to be authorized is 0XAA and it is verified, the bit information representing the enable status of all computing units in the NPU in the instruction to be authorized is correct, and the allocator determines that the computing power of the target vNPU meets the preset conditions, then the execution of the instruction to be authorized is successful. At this point, the target vNPU that runs the instruction to be authorized can obtain control of the computing unit, that is, the target vNPU begins to control the execution of the computing unit, and other vNPUs except the target vNPU cannot use the computing unit, thereby achieving the locking of the computing unit by the target vNPU.

[0068] It should be noted that the conditions for successful execution of the authorization instruction are determined by its instruction format and are not specifically limited in the embodiments of the present application.

[0069] In some embodiments, if the computing power of the target vNPU does not meet the preset conditions, step S301 is repeated: obtaining the computing power of the target vNPU. Until the computing power of the target vNPU meets the preset conditions, the target vNPU can obtain control of the computing unit through steps S302 and S303.

[0070] In some embodiments, if the computing power of the target vNPU meets the preset conditions, after the authorization instruction is executed successfully and the target vNPU obtains control over at least one computing unit, the computing unit begins to execute the work of the target vNPU: first, the computing unit reads the computing instruction, starts the computing function, and then performs the corresponding work according to the computing instruction.

[0071] In some embodiments, the calculation instruction is used to instruct the operation unit to perform the corresponding work. The operation unit is mainly divided into two categories, namely the data processing unit and the data transmission unit. Among them, the data processing unit is used to implement data calculation, including convolution operation, addition operation, multiplication operation, division operation, etc. The data transmission unit is used to implement data transmission work, for example, transferring the data required to implement the target vNPU from DDR to OCM.

[0072] S300: In response to a work completion instruction, obtaining a working state of at least one computing unit.

[0073] In some embodiments, when all computing units required by the target vNPU to perform the current work have been activated through computing instructions, the target vNPU will execute the instruction to complete the work and instruct the activated computing units to feedback the work status, which includes the completion of the work.

[0074] Specifically, the instruction to be completed can be a binary instruction with a length of 64 bits, including: bits 0 to 7 are instruction type bits, used to indicate that the type of the binary instruction is the instruction to be completed; bits 8 to 12 are check bits, used to check whether the reading of the instruction to be completed is correct; bits 13 to 17 are reserved spaces for function expansion; bit 18 is a new work judgment bit, used to judge whether the current work is a new job; bit 19 is a dot judgment bit, used to judge whether it is necessary to print the time point when the instruction to be completed needs to be printed; bits 20 to 27 are operation unit working status check bits, used to mark whether all at least one operation unit needs to check the working status; bits 28 to 63 are reserved spaces for function expansion.

[0075] Exemplarily, Table 2 is an instruction format of an instruction to be completed.

[0076] Table 2 .

[0077] In the pending work completion instruction shown in Table 2, bits 0 to 7 indicate that the instruction type is a pending work completion instruction, which is represented by an 8-bit hexadecimal number in the table and converted to binary as 10100011; bits 8 to 12 are the CRC check value of the instruction, which is used to check whether there is an error in the instruction reading process; bits 13 to 17 indicate reserved spaces, which can be used to add other functions to the instruction later; bit 18 indicates whether the current work is a new job, for example, 0 indicates that it is not a new job, and 1 indicates that it is a new job; bit 19 indicates whether it is necessary to print the time point when the instruction ends, for example, 0 indicates that printing is not required, and 1 indicates that printing is required; bits 20 to 27 indicate the computing unit that needs to wait for the completion of the work for this pending work completion instruction, for example, the NPU includes a total of 8 computing units, and each bit from bits 20 to 27 indicates whether to wait for an computing unit to finish work, 0 indicates not waiting for the computing unit to finish work, and 1 indicates waiting for the computing unit to finish work.

[0078] It should be noted that, in some embodiments, since the computing units are divided into two categories, namely, data processing units and data transmission units, and during the operation of the NPU, the working states of the computing units of the same category can be kept synchronized. Therefore, in order to simplify instructions and improve the working efficiency of the NPU, each bit from bit 20 to bit 27 can indicate whether to wait for the same category of computing units to finish working, for example, bit 20 indicates whether to wait for the data processing unit to finish working, and bit 21 indicates whether to wait for the data transmission unit to finish working.

[0079] Specifically, assuming that the NPU includes 8 computing units, including 6 data processing units and 2 data transmission units, the 20th bit indicates whether to wait for the 6 data processing units to finish their work, and the 21st bit indicates whether to wait for the 2 data transmission units to finish their work. If the NPU also includes a computing unit with a third function, the 22nd bit can indicate whether to wait for its work to finish.

[0080] Based on this, if the number of operation unit categories in the NPU is less than 8, only the values ​​of the bits equal to the number of operation unit categories are set, and the other bit positions are empty. For example, if the NPU includes 2 types of operation units, only the values ​​of bits 20 and 21 are set, and bits 22 to 27 are empty. The values ​​of bits 20 to 27 can be 00000011, indicating that it is necessary to wait for the data transmission unit and the data processing unit to finish their work.

[0081] It should be noted that when running the target vNPU, multiple computing units need to cooperate to implement it. For example, data transmission needs to be implemented through a data transmission unit, and then the computing function is implemented through a data processing unit. After the data transmission unit completes the data transmission required for the first job in the target vNPU, the data processing unit can process the data required for the first job. At this time, the data transmission unit can continue to transmit the data required for the second job in the target vNPU. Therefore, in order to improve the working efficiency of the NPU, the work completion instruction does not need to wait for the data processing unit and the data transmission unit to finish their work before it can be successfully executed. Instead, it can be successfully executed after one type of computing unit finishes its work, so as to start executing the next job. For example, the 20th and 27th bits of the wait-for-work-to-be-completed instruction can be 00000010, indicating that it is only necessary to wait for the data transmission unit to finish working, and there is no need to wait for the data processing unit to finish working. If the instruction is executed successfully, the data transmission unit can execute the next job and transmit the data required for the next job; or, the 20th and 27th bits of the wait-for-work-to-be-completed instruction can be 00000001, indicating that it is only necessary to wait for the data processing unit to finish working, and there is no need to wait for the data transmission unit to finish working. If the instruction is executed successfully, the data processing unit can execute the next job and perform data calculation for the next job.

[0082] In some embodiments, bits 28 to 63 of the wait-for-work-to-end instruction are reserved spaces, which are used to set whether to wait for the work of the exceeding operation unit categories to end when the number of operation unit categories in the NPU exceeds 8. For example, if the NPU includes 9 types of operation units, the 28th bit is used to indicate whether to wait for the work of another operation unit category exceeding the 8 types to end. 0 indicates that there is no need to wait for the work to end, and 1 indicates that there is a need to wait for the work to end.

[0083] It should be noted that the work completion instruction can also be in other instruction formats as long as it can indicate the working status of the operation unit, and no specific limitation is made in the embodiments of the present application.

[0084] In some embodiments, when the work-to-be-completed instruction is executed, it is necessary to check the work status continuously sent by the computing unit so that the vNPU can release the control right over the computing unit in time.

[0085] S400: If the working status of at least one computing unit is complete, the target vNPU releases the control right of the at least one computing unit after the completion instruction is successfully executed.

[0086] In some embodiments, the successful execution of the instruction to wait for the work to be completed requires the following: the instruction type of the instruction to wait for the work to be completed is correct; the instruction to wait for the work to be completed is verified; and the target vNPU successfully obtains the working status of at least one type of computing unit. For example, if the instruction type of the instruction to wait for the work to be completed is 0XA3 and it is verified, the bit information representing the working status of at least one type of computing unit in the NPU in the instruction to wait for the work to be completed is correct, and the work of the computing units to be waited for is completed, then the instruction to wait for the work to be completed is executed successfully.

[0087] It should be noted that the conditions for successful execution of the work completion instruction are determined by its instruction format and are not specifically limited in the embodiments of the present application.

[0088] In some embodiments, since the waiting work completion instruction can be set separately for different types of computing units, in order to ensure that all computing units in the NPU are unlocked at the same time for subsequent vNPU use, it is necessary to wait for the work of all types of computing units to be completed, that is, the waiting work completion instruction indicating the completion of the work of all types of computing units must be executed successfully before the target vNPU can release the control of all computing units and unlock the computing units.

[0089] Specifically, it is assumed that the target vNPU includes two instructions to be completed. The 20th to 27th bits of the first instruction are 00000001, indicating that the data processing unit has completed its work; the 20th to 27th bits of the second instruction are 00000010, indicating that the data transmission unit has completed its work. When the above two instructions to be completed are successfully executed, the target vNPU can release the control right of the operation unit, that is, the target vNPU no longer controls the operation unit to perform work. At this time, all vNPUs running in the NPU can obtain the control right of the operation unit according to the instruction to be authorized, thereby unlocking the operation unit.

[0090] Some embodiments of the present application also provide a computing unit lock control device, which is applied to an NPU. The NPU includes at least one computing unit and a distributor, and the distributor runs at least one vNPU. Figure 4 Schematic diagram of a computing unit lock control device provided in an embodiment of the present application. Figure 4 As shown, the operation unit lock control device includes: The computing power acquisition module 401 is configured to: obtain the computing power of the target vNPU on which the allocator is running in response to the instruction to be authorized, and the instruction to be authorized is used to instruct the target vNPU to obtain authorization; the locking module 402 is configured to: if the computing power of the target vNPU meets the preset conditions, the instruction to be authorized is executed successfully, so that the target vNPU obtains control over at least one computing unit; the working status acquisition module 403 is configured to: obtain the working status of at least one computing unit in response to the instruction to be completed, and the instruction to be completed is used to instruct at least one computing unit to feedback the working status, and the working status includes the end of work; the unlocking module 404 is configured to: if the working status of all at least one computing unit is the end of work, the instruction to be completed is executed successfully, and the control of the target vNPU over at least one computing unit is released.

[0091] Some embodiments of the present application also provide an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the operation unit lock control method provided in some embodiments of the present application.

[0092] For example, Figure 5 A schematic diagram of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the electronic device includes a processor 501, at least one communication bus 502, a user interface 503, at least one external communication interface 504 and a memory 505. The communication bus 502 is configured to realize the connection and communication between these components. The user interface 503 may include a display screen, and the external communication interface 504 may include a standard wired interface and a wireless interface. The memory 505 stores a computer program. The processor 501 is used to execute the computer program stored in the memory 505.

[0093] It can be seen from the above technical scheme that the present application provides a method, device and electronic device for controlling a computing unit lock, which are applied to an NPU. The NPU includes at least one computing unit and a distributor, and the distributor runs at least one vNPU. The computing unit lock control method includes: in response to an instruction to be authorized, obtaining the computing power of the target vNPU being run by the distributor, and the instruction to be authorized is used to instruct the target vNPU to obtain authorization; if the computing power of the target vNPU meets the preset conditions, the instruction to be authorized is executed successfully, so that the target vNPU obtains control over at least one computing unit; in response to an instruction to be completed, obtaining the working status of at least one computing unit, and the instruction to be completed is used to instruct at least one computing unit to feedback the working status, and the working status includes the end of work; if the working status of all at least one computing unit is the end of work, the instruction to be completed is executed successfully, and the control of the target vNPU over at least one computing unit is released.

[0094] The computing unit lock control method provided in the present application can determine whether the target vNPU can obtain the control right of the computing unit by executing the instruction to be authorized. When the target vNPU obtains the control right of the computing unit, other vNPUs can no longer control the computing unit, thereby locking the computing unit; then, by executing the instruction to be completed, it is determined whether the target vNPU releases the control right of the computing unit. After the target vNPU releases the control right, other vNPUs can obtain the control right of the computing unit according to the instruction to be authorized, thereby unlocking the computing unit. Avoid idle time of the computing unit, improve the utilization rate of the computing unit, and thus improve the work efficiency of the NPU.

[0095] Similar parts between the embodiments provided in this application can be referenced to each other. The specific implementation methods provided above are only a few examples under the general concept of this application and do not constitute a limitation on the protection scope of this application. For those skilled in the art, any other implementation methods expanded based on the scheme of this application without creative work belong to the protection scope of this application.

Claims

1. A method for controlling a computing unit lock, characterized in that: Applied to a neural network processor, the neural network processor includes at least one computing unit and a distributor, the distributor runs at least one virtual neural network processor, the method includes: In response to the pending authorization instruction, obtaining the computing power of the target virtual neural network processor being run by the allocator, wherein the pending authorization instruction is used to instruct the target virtual neural network processor to obtain authorization; If the computing power of the target virtual neural network processor meets the preset conditions, the to-be-authorized instruction is executed successfully, so that the target virtual neural network processor obtains control over the at least one computing unit; In response to a waiting work completion instruction, acquiring a working state of the at least one operation unit, the waiting work completion instruction is used to instruct the at least one operation unit to feedback a working state, the working state including the completion of work; If the working status of all of the at least one computing unit is completion of work, the waiting work completion instruction is executed successfully, and the control of the target virtual neural network processor over the at least one computing unit is released.

2. The operation unit lock control method according to claim 1, characterized in that: The distributor includes a current limiting module; The step of obtaining the computing power of the target virtual neural network processor being run by the allocator in response to the instruction to be authorized includes: Determine the target virtual neural network processor according to the instruction to be authorized; The computing power information related to the target virtual neural network processor is read through the current limiting module, and the computing power information includes at least one of the computing power currently being used by the target virtual neural network processor, the remaining computing power, and the computing power usage history data.

3. The operation unit lock control method according to claim 2, characterized in that: The preset conditions include: Within a preset time, the remaining computing power of the target virtual neural network processor is greater than or equal to the computing power required to perform the current task; wherein the computing power required to perform the current task is estimated based on the complexity of the current task and the required computing resources.

4. The operation unit lock control method according to claim 1, characterized in that: The authorization-to-be-authorized instruction is successfully executed, including: The instruction type of the instruction to be authorized is correct; The instruction to be authorized is verified and passed; The target virtual neural network processor successfully obtains usage information of all of the at least one computing units.

5. The operation unit lock control method according to claim 1, characterized in that: After the step of if the computing power of the target virtual neural network processor meets the preset conditions and the to-be-authorized instruction is executed successfully, so that the target virtual neural network processor acquires the control right over the at least one computing unit, the method further includes: Acquire a calculation instruction, where the calculation instruction is used to instruct the at least one calculation unit to start a calculation function; The at least one computing unit is controlled according to the computing instruction to execute the work of the target virtual neural network processor.

6. The operation unit lock control method according to claim 1, characterized in that: The step of obtaining the working state of the at least one computing unit in response to the instruction to complete the work includes: Sending a working status query request to the at least one computing unit, wherein the working status query request is used to request the at least one computing unit to continuously send the working status; Receive the working status sent by the at least one computing unit.

7. The operation unit lock control method according to claim 1, characterized in that: The execution of the waiting work completion instruction is successful, including: The instruction type of the pending work completion instruction is correct; The waiting work completion instruction is verified to be passed; The working status of the at least one computing unit is obtained successfully.

8. The operation unit lock control method according to claim 1, characterized in that: The instruction to be authorized is a 64-bit binary instruction, including: Bits 0 to 7 are instruction type bits, used to indicate that the type of the binary instruction is the instruction to be authorized; Bits 8 to 12 are check bits, used to check whether the reading of the instruction to be authorized is correct; Bits 13 to 17 are reserved for function expansion; Bit 18 is the new job judgment bit, which is used to judge whether the current job is a new job; Bit 19 is a dot determination bit, which is used to determine whether it is necessary to print the time point at which the instruction to be authorized ends; Bits 20 to 35 are operation unit enable flag bits, used to mark whether all of the at least one operation unit is enabled; Bits 36 to 63 are reserved for function expansion.

9. The operation unit lock control method according to claim 1, characterized in that: The waiting work completion instruction is a 64-bit binary instruction, including: Bits 0 to 7 are instruction type bits, used to indicate that the type of the binary instruction is the to-be-finished instruction; Bits 8 to 12 are check bits, used to check whether the reading of the pending work completion instruction is correct; Bits 13 to 17 are reserved for function expansion; Bit 18 is the new job judgment bit, which is used to judge whether the current job is a new job; Bit 19 is a dot determination bit, which is used to determine whether it is necessary to print the time point at which the instruction to wait for the work to be completed ends; Bits 20 to 27 are bits for checking the working status of the computing unit, used to mark whether the working status of all of the at least one computing unit needs to be checked; Bits 28 to 63 are reserved for function expansion.

10. A computing unit lock control device, characterized in that: Applied to a neural network processor, the neural network processor includes at least one computing unit and a distributor, the distributor runs at least one virtual neural network processor, the device includes: A computing power acquisition module is configured to: acquire the computing power of the target virtual neural network processor being run by the allocator in response to an instruction to be authorized, wherein the instruction to be authorized is used to instruct the target virtual neural network processor to obtain authorization; The locking module is configured to: if the computing power of the target virtual neural network processor meets the preset conditions, the execution of the instruction to be authorized is successful, so that the target virtual neural network processor obtains the control right over the at least one computing unit; A working status acquisition module is configured to: acquire the working status of the at least one operation unit in response to a waiting work completion instruction, wherein the waiting work completion instruction is used to instruct the at least one operation unit to feedback the working status, wherein the working status includes the completion of the work; The unlocking module is configured to: if the working status of all of the at least one computing unit is completed, the waiting work completion instruction is executed successfully, and the control of the target virtual neural network processor over the at least one computing unit is released.

11. An electronic device, characterized in that: include: one or more processors; a memory configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the operation unit lock control method according to any one of claims 1 to 9.

Images