Processing method, device and equipment for feature map of neural network by neural network processor

Abstract

To provide a processing method, a device and equipment for a feature map of a neural network by a neural network processor.SOLUTION: A method includes the steps of: determining an arithmetic instruction for calculating a feature map to be processed; determining idle state information for executing the arithmetic instruction by an arithmetic unit of a neural network processor; adjusting a first idle parameter for executing the arithmetic instruction by the arithmetic unit based upon the idle state information, and then determining a second idle parameter after the adjustment; and controlling the arithmetic unit based upon the second idle parameter to execute the arithmetic instruction by the arithmetic unit, and acquiring a processing result corresponding to the feature map to be processed.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to artificial intelligence technology, and in particular to a method, apparatus and device for processing feature maps of a neural network by a neural network processor. [Background technology]

[0002] In the field of artificial intelligence technology, the operations of different neural network models may have significantly different bandwidth demands, for example, the convolution operations of different neural network models may have significantly different bandwidth demands. When the neural network feature map is calculated by the calculation circuit (also called the calculation unit or tensor calculation circuit) of the neural network processor, it is easy to encounter a situation where the calculation is passively paused due to the input and output bandwidth restrictions, or a situation where the calculation circuit and other calculation circuits have data dependencies and need to wait for the results of the calculations of other calculation circuits. However, when a calculation circuit pauses its calculation for a long time, the uniformity of the power consumption distribution is low, which is likely to cause problems with power supply integrity. Summary of the Invention [Problem to be solved by the invention]

[0003] In order to solve the above-mentioned problems such as low uniformity of power consumption distribution, one embodiment of the present disclosure provides a method, device, and equipment for processing a neural network feature map by a neural network processor, thereby improving the uniformity of power consumption distribution. [Means for solving the problem]

[0004] A method for processing a feature map of a neural network by a neural network processor according to a first aspect of the present disclosure includes the steps of: determining an operation instruction for calculating a feature map to be processed; determining pause state information for an operation unit in the neural network processor to execute the operation instruction; adjusting a first pause parameter for the operation unit to execute the operation instruction based on the pause state information, and determining an adjusted second pause parameter; and controlling the operation unit to execute the operation instruction based on the second pause parameter, thereby obtaining a processing result corresponding to the feature map to be processed.

[0005] A processing device for a neural network feature map by a neural network processor according to a second aspect of the present disclosure includes a determination module for determining an operation instruction for calculating a feature map to be processed, a first processing module for determining pause state information for an operation unit in the neural network processor to execute the operation instruction, a second processing module for adjusting a first pause parameter for the operation unit to execute the operation instruction based on the pause state information and determining an adjusted second pause parameter, and a third processing module for controlling the operation unit to execute the operation instruction based on the second pause parameter, thereby obtaining a processing result corresponding to the feature map to be processed.

[0006] A computer-readable storage medium according to a third aspect of the present disclosure has stored thereon a computer program for executing the method for processing feature maps of a neural network by a neural network processor according to any of the above embodiments of the present disclosure.

[0007] An electronic device according to a fourth aspect of the present disclosure comprises a processor and a memory for storing instructions executable by the processor, wherein the processor reads the executable instructions from the memory and executes the instructions to perform a method for processing a neural network feature map by a neural network processor described in any of the above embodiments of the present disclosure, or comprises an apparatus for processing a neural network feature map by a neural network processor described in any of the above embodiments of the present disclosure.

[0008] A computer program product according to a fifth aspect of the present disclosure performs the method of processing feature maps of a neural network by a neural network processor provided in any of the above embodiments of the present disclosure when instructions in the computer program product are executed by a processor. Effect of the Invention

[0009] According to the method, device and equipment for processing a feature map of a neural network by a neural network processor according to the above embodiment of the present disclosure, when processing a feature map of a neural network by a neural network processor, a calculation command for calculating a feature map to be processed is determined, pause state information for each arithmetic unit in the neural network processor to execute the calculation command is determined, a first pause parameter in the pause state information of each arithmetic unit is adjusted to determine a second pause parameter after adjustment, and the arithmetic unit is controlled to execute each arithmetic command based on the second pause parameter to obtain a processing result corresponding to the feature map to be processed. The second pause parameter is obtained by adjusting the first pause parameter, and the second pause parameter can make the time distribution and / or spatial distribution of the arithmetic units pausing the operation more uniform, thereby reducing the width and duration of the sudden change in power consumption each time, thereby making the pause time distribution of the entire arithmetic units of the neural network processor more uniform, thereby improving the uniformity of the power consumption distribution of the neural network processor, and helping to alleviate or solve the power supply integrity problem. [Brief description of the drawings]

[0010] [Figure 1] 1 is an exemplary application scenario of a method for processing feature maps of a neural network by a neural network processor according to the present disclosure; [Diagram 2] 1 is a flowchart of a method for processing feature maps of a neural network by a neural network processor according to an exemplary embodiment of the present disclosure. [Diagram 3] 11 is a flowchart of a method for processing feature maps of a neural network by a neural network processor according to another exemplary embodiment of the present disclosure. [Figure 4] FIG. 13 is a schematic diagram of power consumption distribution of time dimension adjustment according to an exemplary embodiment of the present disclosure. [Diagram 5]11 is a flowchart of a method for processing feature maps of a neural network by a neural network processor according to yet another exemplary embodiment of the present disclosure. [Figure 6] 11 is a flowchart of a method for processing feature maps of a neural network by a neural network processor according to yet another exemplary embodiment of the present disclosure. [Figure 7] 11 is a flowchart of a method for processing neural network feature maps by a neural network processor according to yet another exemplary embodiment of the present disclosure. [Figure 8] FIG. 2 is a schematic diagram of the principle of spatial dimension adjustment according to an exemplary embodiment of the present disclosure. [Figure 9] FIG. 13 is a schematic diagram of power consumption distribution of spatial dimension adjustment according to an exemplary embodiment of the present disclosure. [Figure 10] 11 is a flowchart of a method for processing feature maps of a neural network by a neural network processor according to yet another exemplary embodiment of the present disclosure. [Figure 11] 11 is a flowchart of a method for processing feature maps of a neural network by a neural network processor according to yet another exemplary embodiment of the present disclosure. [Figure 12] FIG. 2 is a schematic diagram of the control principle of a computing unit according to an exemplary embodiment of the present disclosure. [Figure 13] FIG. 2 is a schematic diagram of the structure of a processing device for neural network feature maps by a neural network processor according to an exemplary embodiment of the present disclosure. [Figure 14] FIG. 13 is a schematic diagram of the structure of a processing device for neural network feature maps by a neural network processor according to another exemplary embodiment of the present disclosure. [Figure 15] FIG. 13 is a schematic diagram of the structure of a processing device for neural network feature maps by a neural network processor according to yet another exemplary embodiment of the present disclosure. [Figure 16] 1 is a structural diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] In the following, exemplary embodiments of the present disclosure will be described in detail with reference to the drawings in order to interpret the present disclosure. Furthermore, the described embodiments are only a part of the embodiments of the present disclosure, and are not all of the embodiments, and the present disclosure is not limited to the exemplary embodiments.

[0012] Unless otherwise specifically stated, the relative arrangement of components and steps, formulas and numerical values ​​described in these embodiments do not limit the scope of the present disclosure.

[0013] [Summary of this disclosure] In the process of realizing the present disclosure, the inventors have found that in the field of artificial intelligence technology, the operations of different neural network models may have significantly different bandwidth demands, for example, the convolution operations of different neural network models may have significantly different bandwidth demands. When the neural network processor's arithmetic circuit (also called an arithmetic unit or tensor calculation circuit) calculates the feature map of the neural network, it is easy to encounter a situation where the operation is passively paused due to the input and output bandwidth restrictions, or a situation where the arithmetic circuit and other arithmetic circuits have data dependencies and are waiting for the results of the other arithmetic circuits. However, when an arithmetic circuit pauses operation for a long time, the uniformity of the power consumption distribution is low, which is easy to cause power supply integrity problems.

[0014] [Illustrative Overview] FIG. 1 is an exemplary application scenario of the method for processing a feature map of a neural network by a neural network processor according to the present disclosure. As shown in FIG. 1, the neural network processor may include a controller 210 and at least one arithmetic unit 220. The controller 210 is for controlling the operation of the arithmetic unit 220, for example, the controller 210 activates the arithmetic unit 220 so that the arithmetic unit 220 is in an operating state, and controls the arithmetic unit 220 so that the arithmetic unit 220 executes an arithmetic instruction to complete a corresponding operation. The arithmetic unit 220 may include a multiply-accumulate array (MAC_ARRAY) arithmetic unit, a multiply-accumulate post-processing (MAC_PP) arithmetic unit, etc. The multiply-accumulate array arithmetic unit may be for completing multiplication and accumulation, such as convolution operation, matrix multiplication, etc. The multiply-accumulate post-processing arithmetic unit may be for completing post-processing operation after multiplication and accumulation, such as adding a convolution result with a feature of a skip connection in a neural network model. The neural network processor may further include a memory and other related components (not shown in FIG. 1). For example, the memory can store a feature map to be processed, an operation instruction, weight data, etc. According to the method for processing a feature map of a neural network by a neural network processor of the present disclosure (which can be executed by the controller 210), an operation instruction for calculating a feature map to be processed is determined, pause state information for the operation unit 220 in the neural network processor to execute the operation instruction is determined, a first pause parameter for the operation unit 220 to execute the operation instruction is adjusted based on the pause state information to determine an adjusted second pause parameter, and based on the second pause parameter, the operation unit 220 is controlled so that the operation unit 220 executes the operation instruction, thereby obtaining a processing result corresponding to the feature map to be processed.The method disclosed herein obtains a second pause parameter by adjusting a first pause parameter for the computing unit 220 to execute an operation instruction, and the second pause parameter can make the time distribution and / or spatial distribution of the computing unit 220 pausing operations more uniform, thereby reducing the width and duration of each sudden change in power consumption, making the pause time distribution across the computing units of the neural network processor more uniform, thereby improving the uniformity of the power consumption distribution of the neural network processor, and helping to mitigate or solve the power supply integrity problem.

[0015] [Exemplary Method] 2 is a flowchart of a method for processing a feature map of a neural network by a neural network processor according to an exemplary embodiment of the present disclosure. This embodiment is specifically applicable to electronic devices such as in-vehicle computing platforms or other terminal devices with image processing needs, and includes the following steps 301 to 304, as shown in FIG.

[0016] In step 301, a computation command for computing a feature map to be processed is determined.

[0017] In some alternative embodiments, the feature map to be processed may be a feature map to be operated by a neural network processor at any stage in a neural network model. The neural network model may include, for example, a target detection model, an image classification model, a semantic segmentation model, etc. The feature map to be processed may be, for example, any feature map to be operated by a convolution operation in a neural network model, a feature map to be multiplied by a matrix, etc. The operation instruction is an instruction to perform an operation on the feature map to be processed, which is executed by the neural network processor. The operation instruction is obtained by compiling the neural network model and can be arranged in a terminal device to realize accelerated estimation of the model by the neural network processor. For example, the operation instruction may include an operation instruction corresponding to a convolution operation, an operation instruction corresponding to a matrix multiplication, etc. The operation instruction may include one or more instructions to provide the operation unit with data required for the operation. For example, an operation instruction corresponding to a convolution operation may include relevant information such as the address and size of an input tensor (e.g., a feature map to be processed or a part of a tensor in the feature map to be processed), relevant information such as the address and size of a convolution weight, and relevant information of the operation such as an operation type (e.g., multiplication, addition, etc.), an operation cycle, etc. The operation instruction further includes relevant information such as the storage address and size of an output tensor, so as to store the tensor output from the operation and provide an operation result corresponding to subsequent processing. Thereby, the operation unit of the neural network processor can execute the corresponding operation instruction to complete the operation on the feature map to be processed. The specific operation instruction is determined according to the specific operation of the actual scene, and is not limited to the embodiments of the present disclosure.

[0018] In some optional embodiments of the present disclosure, the calculation instructions for the feature map to be processed can be read from a memory space that stores the calculation instructions, and the calculation instructions for the feature map to be processed can be determined.

[0019] In step 302, the arithmetic units in the neural network processor determine pause state information for executing arithmetic instructions.

[0020] In some alternative embodiments, as shown in FIG. 1, the arithmetic unit in the neural network processor may include one or more arithmetic units. In general, it includes multiple arithmetic units. Each arithmetic unit cooperates with others to complete an operation for a feature map to be processed. Each arithmetic unit may include at least one calculation array of a predetermined size, for example, the arithmetic unit may include at least one multiplication and accumulation array (MAC), and each multiplication and accumulation array may complete a multiplication of one input feature value and a weight at each time (i.e., at each calculation cycle) to obtain an operation result. For example, the input feature value 7 may be expressed as 00000111 in binary and multiplied by a weight 00001011, and the multiplication and accumulation array may multiply the input feature value by each bit of the weight for each bit to obtain a calculation result (also called a partial product) corresponding to each bit of the input feature value, and the calculation result of each bit may be compressed to obtain a multiplication result of the input feature value and the weight.

[0021] In some optional embodiments, the hibernation information for the arithmetic unit to execute the arithmetic instruction means that after the arithmetic unit enters an operating state, due to factors such as data dependencies between the arithmetic units or input / output bandwidth limitations, the arithmetic unit needs to perform calculations for a certain period of time to obtain its own calculation result, and then enter a hibernation state, for example, waiting for the calculation results of other arithmetic units, waiting for new input data, waiting for the output of its own calculation result, etc.

[0022] In some alternative embodiments, the pause state information for the computing unit to execute the computing instruction may be determined according to a specific computing instruction. For example, the execution time of each instruction may be determined according to the computing instruction, and the computing activation period and the pause period of the computing unit to execute the computing instruction may be determined according to the execution time of each instruction. The pause state information may include the computing activation period and the pause period for the computing unit to execute the computing instruction. For example, the time when each computing unit enters the working state from the idle state (when it can perform the computation but may not yet perform the computation) is taken as an initial time point (for example, time 0), the computing unit A executes the computing instruction in a period from 0 to t1 to enter the computing activation state, t1 being one time point, the computing unit B executes the computing instruction in a period from 0 to t2 to enter the computing activation state, t2 being another time point, for example, t2 is greater than t1, and if the computing result of the computing unit A needs to be synchronized with the computing result of the computing unit B, the computing unit A needs to pause the computation after completing the current computation and wait for the computing result of the computing unit B. That is, the arithmetic unit A is in a pause state during the period from t1 to t2. For example, if the arithmetic unit A includes 10 sum-of-products arrays and needs to calculate 100 multiplications, the 10 sum-of-products arrays need to complete the calculation of the 100 multiplications in 10 calculation cycles (each sum-of-products array completes one multiplication in each calculation cycle), but if another operation that needs to be synchronized requires 20 calculation cycles, the 10 sum-of-products arrays need to enter a pause state after completing the operation of 10 calculation cycles (these 10 calculation cycles are also called the first operation activation period) and wait for the completion of the other operation. This allows each arithmetic unit to obtain pause state information for executing an operation instruction.

[0023] In step 303, a first pause parameter for the computing unit to execute the computing instruction is adjusted based on the pause state information, and a second pause parameter after the adjustment is determined.

[0024] Here, the first pause parameter is the original pause parameter of the computing unit determined according to the computing instruction. For example, the first pause parameter can include a first computing activation period and a first pause period for each computing unit to execute the computing instruction. For any computing unit, the first computing activation period and the first pause period of the computing unit are two periods that are consecutive in time, and the total time of the two periods can represent the time from the start of the current computing task by the computing unit to the start of the next computing task. For example, multiple computing units cooperate to complete the calculation of the feature map to be processed, and the current computing task of the computing unit A among the multiple computing units is to complete 100 multiplications. In order to synchronize with the data of other computing units or due to the influence of the input-output bandwidth, after the computing unit A starts the operation, it completes the 100 multiplications within the first computing activation period and then enters the first pause period, and the computing unit A pauses within the first pause period and waits for the next computing task. The next computing task can continue to be processed according to the processing flow of the current computing task.

[0025] In some alternative embodiments, the first computation activation period may include a first computation activation start time and a first end time, and the first sleep period may include a second sleep start time and a second end time. Here, the time may be a relative time, for example, the initial time when the computing unit enters the computation activation state is time 0, the first computation activation period start time is this time 0, and the first computation activation period end time is a time delayed by the first computation activation period time from the first start time. The second computation activation period start time may be a first end time of the first computation activation period, and the computing unit exits the computation activation state and enters the sleep state at the first end time. The second end time of the first sleep period is a time delayed by the first sleep period time from the second start time.

[0026] In some alternative embodiments, the first computation activation period and the first pause period can be displayed in computation cycles, for example, the first computation activation period includes n computation cycles from 1 to n, and the first pause period includes computation cycles from n+1 to N. For example, the first computation activation period includes 10 computation cycles from 1 to 10, and the first pause period includes 10 computation cycles from 11 to 20. The specific display manner of the first computation activation period and the first pause period is not limited.

[0027] In some alternative embodiments, for the plurality of computing units, a first pause parameter may correspond to each computing unit. Based on the pause state information of each computing unit, the first pause parameter of some of the computing units may be adjusted to obtain the adjusted second pause parameter. For example, based on the dependency between each computing unit, a target computing unit to be adjusted is determined from each computing unit, and the first pause parameter for the target computing unit to execute a computing instruction is adjusted to obtain the adjusted second pause parameter of the target computing unit. Alternatively, based on the pause state information, the first pause parameter for the computing unit to execute a computing instruction may be adjusted, where the adjusted pause parameter for the target computing unit to be adjusted is set as the adjusted second pause parameter of the target computing unit, and the first pause parameter of the computing unit for the computing unit that does not need to be adjusted is set as the adjusted second pause parameter. In some alternative embodiments, the purpose of adjusting the first pause parameter is to distribute the distribution of the pause periods of at least one computing unit among the computing units evenly in either one of time and space, thereby improving the evenness of the power consumption distribution of the entire computing units. For example, the first sleep parameter of the computing unit is adjusted from one continuous sleep period to at least two non-consecutive sleep periods, for example, the first sleep parameter of the computing unit is to first activate the computing for a predetermined period (first computing activation period) and then sleep for a predetermined period (first sleep period). The first sleep parameter is adjusted to first activate the computing for a predetermined period and then sleep for a predetermined period, and then activate the computing for a predetermined period and then sleep for a predetermined period, to obtain an adjusted second sleep parameter, which includes two non-consecutive second computing activation times and two non-consecutive second sleep times. That is, the first sleep period of the computing unit is adjusted to two non-consecutive second sleep periods, which effectively improves the uniformity of the distribution of power consumption of the entire computing unit in the time dimension.Also, for example, the first sleep parameter of the computing unit is adjusted so that some subunits of the computing unit (e.g., called the first subunit) activate the operation first for a predetermined period, during which some other subunits of the computing unit (e.g., called the second subunit) enter a sleep state, and then the first subunit enters a sleep state and the second subunit enters a computationally active state. In this way, the first subunit and the second subunit alternately sleep, realizing a uniform distribution of the power consumption of the computing unit in space. Furthermore, the first sleep parameter can be adjusted by integrating the time dimension and the space dimension to realize a uniform distribution of the power consumption of the computing unit.

[0028] In some alternative embodiments, a first pause parameter of the computing unit can be adjusted according to a predetermined adjustment principle based on the pause state information to obtain an adjusted second pause parameter.

[0029] In some alternative embodiments, the adjustment principle can include a principle of adjusting according to a time dimension and a principle of adjusting according to a space dimension, and the principle of adjusting according to the space dimension can include two kinds of principles: a principle of adjusting only according to the space dimension and a principle of adjusting according to a mixture of time and space. The principle of adjusting according to a time dimension can include that the total time of the operation activation period and the pause period in the second pause parameter after the adjustment is consistent with the total time of the first pause parameter, the total pause time of the second pause parameter after the adjustment is consistent with the total pause time of the first pause parameter, and the total operation activation time of the second pause parameter after the adjustment is consistent with the total operation activation time of the first pause parameter, thereby ensuring that the operation unit can complete the current operation task within the total time of the second pause parameter. For example, the first sleep parameter includes a period of 0 to t, the total time is t, the first computation activation period is 0 to t1, the first sleep period is t1 to t, and the second sleep parameter includes at least two non-consecutive second sleep periods, and taking two second sleep periods as an example, the second sleep period 1 is 0 to t2, the second computation activation period is t2 to t3, the second sleep period 2 is t3 to t, and t3-t2=t1. The principle of adjusting according to the space dimension includes ensuring that the total time of the adjusted second sleep parameter is consistent with the total time of the first sleep parameter (i.e., the total time of computation activation and sleep of each computing subunit divided according to space is consistent with the total time of the first computation activation period and the first sleep period), and the computing unit can complete the current computing task within the total time of the second sleep parameter when the power consumption is uniform. For example, the first computation activation period of computing unit A is from 0 to t1, and the first idle period is from t1 to t, computing unit A is divided into two computing subunits, and the second computation activation period of computing subunit 1 is from 0 to t1, and the second idle period of computing subunit 1 is from t1 to t. The second idle period of computing subunit 2 is from 0 to t1, and the second computation activation period of computing subunit 2 is from t1 to t.

[0030] In some alternative embodiments, the adjustment principle may include that the total time of the computation activation and the pause in the second pause parameter is consistent with the first pause parameter, and the total computation amount of the computation unit in the second pause parameter is consistent with the total computation amount of the first pause parameter. For example, in terms of unit computation power (e.g., a multiply-add array) in the computation unit, the sum of the computation activation time (or the number of computation cycles of the computation activation) of each multiply-add array in the second pause parameter is equal to the sum of the computation activation time (or the number of computation cycles of the computation activation) of each multiply-add array in the first pause parameter. For example, the calculation unit A includes 10 sum-of-products arrays, and in the first pause parameters, the first calculation activation period of the 10 sum-of-products arrays includes 10 calculation cycles, and the first pause period includes 10 calculation cycles, i.e., the number of calculation cycles of calculation activation of each sum-of-products array is 10, and the sum of the numbers of calculation cycles of calculation activation of each sum-of-products array is 10*10=100, which corresponds to the number of calculation cycles that one sum-of-products array needs to activate calculation for completing a current calculation task being 100. Taking the spatial dimension adjustment as an example, in the second pause parameter, the second operation activation period of five multiply-add arrays includes 10 operation cycles from 1 to 10, and the second pause period includes 10 operation cycles from 11 to 20, and the second pause period of another five multiply-add arrays includes 10 operation cycles from 1 to 10, and the second operation activation period includes 10 operation cycles from 11 to 20, satisfying 5*10+5*10=100, the current operation task can be completed within the total time of the first pause parameter, and the distribution of the pause time of the operation units in the spatial dimension is more even.

[0031] In actual use, the specific adjustment principles are not limited to the above adjustment principles and can be set according to actual needs.

[0032] In step 304, the computing unit is controlled to execute a computing instruction based on the second pause parameter to obtain a processing result corresponding to the feature map to be processed.

[0033] Here, after obtaining the second pause parameter of the arithmetic unit, the arithmetic unit is controlled so as to execute a calculation instruction based on the second pause parameter, and a processing result corresponding to the feature map to be processed is obtained.

[0034] In some alternative embodiments, for the second sleep parameter obtained by equalizing in the time dimension, the second sleep parameter of at least one of the computing units includes at least two non-consecutive sleep periods, thereby enabling the computational activation and sleep of each computing unit to be controlled based on the second sleep parameter, thereby realizing equalization of the power consumption distribution in the time dimension and improving the evenness of the power consumption distribution across the computing units.

[0035] In some selectable embodiments, for the second sleep parameter obtained by equalizing in the spatial dimension, at least one of the computing units is divided into multiple computing subunits in space, and the second sleep parameter of the computing unit includes a second sleep parameter corresponding to each computing subunit, thereby controlling the multiple computing subunits to be alternately paused based on the second sleep parameter corresponding to each computing subunit of the computing unit, thereby achieving equalization of the power consumption distribution in the spatial dimension and improving the uniformity of the power consumption distribution across the computing units.

[0036] In some alternative embodiments, a second sleep parameter is obtained by integrating and equalizing the time dimension and the space dimension, and the second sleep parameter can be used to control the activation and sleep of the computing unit and multiple computing sub-units of at least one computing unit in the time dimension and the space dimension, thereby achieving equalization of the power consumption distribution in the time dimension and the space dimension and improving the uniformity of the power consumption distribution across the computing units.

[0037] In the method for processing a feature map of a neural network by a neural network processor according to the present embodiment, when the feature map of a neural network is processed by the neural network processor, a calculation command for calculating the feature map to be processed is determined, and pause state information for each calculation unit in the neural network processor to execute the calculation command is determined, and then a first pause parameter in the pause state information of each calculation unit is adjusted to determine a second pause parameter after the adjustment, and the calculation unit is controlled to execute each calculation command based on the second pause parameter to obtain a processing result corresponding to the feature map to be processed. The second pause parameter is obtained by adjusting the first pause parameter, and the second pause parameter can make the time distribution and / or spatial distribution of the calculation units pausing the calculation more uniform, thereby reducing the width and duration of the sudden change in power consumption each time, and making the pause time distribution of the entire calculation units of the neural network processor more uniform, thereby improving the uniformity of the power consumption distribution of the neural network processor, and helping to alleviate or solve the power supply integrity problem.

[0038] FIG. 3 is a flowchart of a method for processing feature maps of a neural network by a neural network processor according to another exemplary embodiment of the present disclosure.

[0039] In some optional embodiments, as shown in FIG. 3, step 303 of adjusting a first pause parameter for the computing unit to execute the computing instruction based on the pause state information and determining an adjusted second pause parameter includes steps 3031 and 3032.

[0040] In step 3031, determine a first computation activation period for the computing unit to execute a computation instruction and a first pause period based on the pause state information.

[0041] Here, the pause state information may include a first operation activation period and a first pause period for the computing unit to execute an operation instruction, and therefore, based on the pause state information, the first operation activation period and the first pause period for the computing unit to execute an operation instruction may be determined.

[0042] In step 3032, the first computation activation period and the first pause period are adjusted according to at least one of a time dimension and a space dimension to obtain an adjusted second computation activation period and a second pause period as second pause parameters.

[0043] Here, the time dimension refers to keeping the computing unit constant and adjusting the distribution of the idle periods of the computing units. The space dimension refers to dividing the computing unit into multiple computing subunits and adjusting the idle periods of the computing subunits. For details, please refer to the above embodiment, and redundant explanations will be omitted here.

[0044] In some alternative embodiments, for the plurality of computing units, the first computation activation period and the first idle period of the adjustable target computing unit among the plurality of computing units can be adjusted according to at least one of the time dimension and the space dimension to obtain the adjusted second computation activation period and the second idle period of the target computing unit, which is set as the second idle parameter of the target computing unit. The number of the target computing units can be one or more. For the computing units among the plurality of computing units that do not need to be adjusted, the first computation activation period and the first idle period of the computing unit can be maintained, or the first computation activation period and the first idle period of the computing unit can be set as the second computation activation period and the second idle period of the computing unit.

[0045] The present embodiment adjusts the first computational activation period and the first idle period according to at least one of the time dimension and the space dimension, so that the adjusted second computational activation period and the second idle period are evenly distributed among the idle periods in at least one of the time dimension and the space dimension, which helps to improve the evenness of the power consumption distribution across the computing units, thereby mitigating power supply integrity issues.

[0046] In some alternative embodiments, step 3032 of adjusting the first computation activation period and the first idle period according to at least one of the time dimension and the space dimension to obtain an adjusted second computation activation period and second idle period includes adjusting the first computation activation period and the first idle period into at least two idle sub-periods and at least one computation activation sub-period sorted in a pre-set manner in the time dimension, each sorted computation activation sub-period being a second computation activation period, and each sorted idle sub-period being a second idle period.

[0047] Here, the preset manner may be any feasible manner, for example, the preset manner may require that at least two sleep subperiods are non-consecutively sorted, and at least one computation activation period is included between any two sleep subperiods. Exemplarily, the two sleep subperiods and the two computation activation subperiods are sorted in the manner of: sleep subperiod→computation activation subperiod→sleep subperiod→computation activation subperiod, or in the manner of: computation activation subperiod→sleep subperiod→computation activation subperiod→sleep subperiod, and the adjusted second sleep parameter includes two second computation activation subperiods and two second sleep periods sorted in the preset manner. If the second sleep parameter is for controlling the operation of a computing unit, the computing unit is made to alternate between computation activation and sleep in the preset manner, thereby improving the uniformity of the power consumption distribution of the computing unit.

[0048] For example, Fig. 4 is a schematic diagram of power consumption distribution of time dimension adjustment according to an exemplary embodiment of the present disclosure. As shown in Fig. 4, before adjustment, the idle periods of the computing units are concentrated, the range of power consumption is relatively large, and the duration is relatively long. When adjusted in the time dimension (after adjustment), the idle periods of the computing units are dispersed, and the idle periods are evenly distributed in the time dimension, without reducing the performance of the entire computing unit, the change range and duration of the power consumption each time are reduced, and the evenness of the power consumption distribution is effectively improved, which is helpful to alleviate the problem of power supply integrity.

[0049] This embodiment adjusts the first computing activation period and the first idle period into at least two idle sub-periods and at least one computing activation sub-period sorted in a preset manner in the time dimension, thereby obtaining an adjusted second idle parameter of the computing unit, thereby realizing an even distribution of the idle times of the computing units in the time dimension, and helping to improve the even distribution of power consumption across the entire computing units.

[0050] FIG. 5 is a flowchart of a method for processing feature maps of a neural network by a neural network processor according to yet another exemplary embodiment of the present disclosure.

[0051] In some optional embodiments, as shown in FIG. 5, step 3032 of adjusting a first computation activation period and a first idle period according to at least one of a time dimension and a space dimension to obtain an adjusted second computation activation period and a second idle period includes steps 30321 to 30323.

[0052] In step 30321, a first grouping manner of the computing units is determined based on the first computing activation period and the first idle period.

[0053] Here, the first grouping manner can include a grouping number of the computing units and a manner of dividing the computing subunits. The grouping number of the computing units is a first number. Different grouping manners can correspond to different first numbers. Specifically, the first grouping manner of the computing units can be determined based on the time relationship between the first computing activation period and the first idle period, or the first grouping manner of the computing units can be determined according to the time ratio between the first computing activation period and the first idle period of the computing unit to be adjusted.

[0054] In some alternative embodiments, the specific determining manner of determining the first grouping manner of the computing units according to the first computing active period and the first idle period can be any feasible manner. For example, the total time of the first computing active period and the first idle period can be divided into at least two periods, in each period, some computing subunits of the first number of computing subunits are in a computing active state and other some computing subunits are in an idle state, and second computing active periods and second idle periods respectively corresponding to each computing subunit are obtained, thereby achieving equalization of the power consumption distribution in the space dimension when satisfying the computing requirements.

[0055] In some selectable embodiments, for the plurality of computing units, a target computing unit to be adjusted can be determined from the plurality of computing units according to a first computing activation time and a first pause time, and a first grouping manner of the target computing unit can be determined. Other computing units among the plurality of computing units other than the target computing unit are not grouped.

[0056] In step 30322, divide the computing unit into a first number of computing sub-units according to a first grouping scheme.

[0057] Here, the first grouping manner includes the grouping number of the arithmetic unit and the manner of dividing the arithmetic subunit, so that the first number is determined according to the grouping number, and the arithmetic unit is divided into the first number of arithmetic subunits according to the manner of dividing the arithmetic subunit. The manner of dividing the arithmetic subunit may include the number of calculation arrays included in each arithmetic subunit. For example, the arithmetic unit includes 10 sum-of-products arrays and is divided into two arithmetic subunits, each of which includes 5 sum-of-products arrays, or one arithmetic subunit includes a relatively small number of sum-of-products arrays (e.g., includes 4 sum-of-products arrays), and the other arithmetic subunit includes a relatively large number of sum-of-products arrays (e.g., includes 6 sum-of-products arrays). The specific manner of dividing the arithmetic subunits may be determined according to the demand for actual computing performance and the demand for power consumption uniformity.

[0058] In some alternative embodiments, the target computing units to be adjusted are divided into a first number of computing subunits according to a first grouping manner of the target computing units to be adjusted, and other computing units may not be grouped.

[0059] In step 30323, a second computational activation period and a second sleep period corresponding to each computing subunit are determined based on the first computational activation period and the first sleep period.

[0060] In some alternative embodiments, the grouping manner of the computing units can be determined based on the time relationship between the first computing activation period and the first idle period. For example, if the time of the first computing activation period and the first idle period are the same, the first number can be any integer greater than 1, and the second computing activation period and the second idle period corresponding to each determined computing subunit can be parameters for alternatingly idling each computing subunit, or can be parameters for grouping and alternatingly idling each computing subunit. For example, if the first number is 2, the two computing subunits alternately idling, that is, in the first computing activation period, one computing subunit (e.g., called the first computing subunit) of the two computing subunits activates the operation, and the other computing subunit (e.g., called the second computing subunit) idling, and in the first idle period, the first computing subunit idling, and the second computing subunit activates the operation. For the first computing subunit, the second sleep parameter includes a second computing activation period and a second sleep period arranged in sequence. For the second computing subunit, the second sleep parameter includes a second sleep period and a second computing activation period arranged in sequence. For example, if the first number is three, the computing unit can be divided into a first computing subunit, a second computing subunit, and a third computing subunit, and the three computing subunits can be alternately paused. That is, the total time of the first computing activation period and the first sleep period can be divided into three periods, for example, period 1, period 2, and period 3, and the three computing subunits can be paused for three periods each. That is, one computing subunit is paused in each period. For example, in period 1, the first computing subunit is paused and the second computing subunit and the third computing subunit activate operations; in period 2, the second computing subunit is paused and the first computing subunit and the third computing subunit activate operations; and in period 3, the third computing subunit is paused and the first computing subunit and the second computing subunit activate operations.The specific order of the operation subunits is not limited. For example, if the first number is 5, the total time of the first operation activation period and the first pause period can be divided into at least two periods, and the five operation subunits can be paused in turn. For example, in the first period, three operation subunits activate the operation, and the remaining two operation subunits pause, and in the second period, the remaining two operation subunits activate the operation, and the three operation subunits pause. If the time of the first operation activation period is longer than the time of the first pause period, the total time of the first operation activation period and the first pause period can be divided into at least two periods, and in each period, a relatively large number of operation subunits in the operation unit can be in an operation state, and a relatively small number of operation subunits can be in a pause state. If the duration of the first computing activation period is shorter than the duration of the first idle period, the total duration of the first computing activation period and the first idle period can be divided into at least two periods, so that in each period, a relatively small number of computing subunits in the computing unit are in a computing activation state, and a relatively large number of computing subunits are in a idle state. The specific manner of determining the first number and the second idle parameter of the computing subunits is not limited to the above manner, and can be any other feasible manner as long as it can ensure that the computing unit completes the computation within the total duration of the first computing activation period and the first idle period.

[0061] For example, a first grouping manner of the computing units can be determined according to the time ratio of the first computation active period and the first pause period of the computing unit to be adjusted, so that the pause period of the computing unit can be evenly distributed when the computing unit meets the computation requirements. For example, if the computing unit A is the computing unit to be adjusted, the time of the first computation active period of the computing unit A is 60 computation cycles, the time of the first pause period is 40 computation cycles, the purpose of the pause is that the computing unit A needs to synchronize with the computing unit B, and the time of the first computation active period of the computing unit B is 100 computation cycles, then the computing unit A only needs to complete the computation within 100 computation cycles. If the time ratio between the first operation activation period and the first idle period is 3:2, the operation unit A can be divided into five operation subunits (or the operation unit A can be divided into two uneven operation subunits), and the total time of the first operation activation period and the first idle period can be divided into at least two periods, for example, two periods, or the first operation activation period and the first idle period are each two periods, and in each period, at least three operation subunits activate operations and the remaining operation subunits are idle.

[0062] In this embodiment, by grouping the computing units into a first number of computing subunits, a second computing activation period and a second idle period corresponding to each computing subunit are determined based on the first computing activation period and the first idle period, so that in each calculation cycle, some computing subunits are in a computing activation state and some other computing subunits are in an idle state, thereby realizing equalization of the power consumption distribution in the spatial dimension and helping to improve the equalization of the power consumption distribution of the entire computing units.

[0063] FIG. 6 is a flowchart of a method for processing feature maps of a neural network by a neural network processor according to yet another exemplary embodiment of the present disclosure.

[0064] In some optional embodiments, as shown in FIG. 6, step 30323 of determining a second computation activation period and a second idle period corresponding to each computation subunit based on the first computation activation period and the first idle period includes steps 303231 and 303232.

[0065] In step 303231, the first computation activation period is alternately made into the second computation activation period of each computing subunit.

[0066] In some alternative embodiments, if the computing demand can be met, the first computing activation period can be alternately used as the second computing activation period for each computing subunit. For example, the first computing activation period includes 20 computing cycles, the first idle period includes 40 computing cycles, and three computing subunits alternately use the first computing activation period as the second computing activation period for each computing subunit, i.e., each computing subunit alternately activates the computing for the duration of the first computing activation period. That is, the total time of the first calculation activation period and the first sleep period is divided into three periods, and in the first period, calculation subunit 1 activates calculation, and calculation subunit 2 and calculation subunit 3 are sleepy, in the second period, calculation subunit 2 activates calculation, and calculation subunit 1 and calculation subunit 3 are sleepy, and in the third period, calculation subunit 3 activates calculation, and calculation subunit 1 and calculation subunit 2 are sleepy, thereby realizing alternating calculation activation of each calculation subunit.

[0067] In some alternative embodiments, the first number can be determined according to a ratio relationship between the time of the first computation activation period and the time of the first pause period, so that when the first computation activation period is alternately taken as the second computation activation period of each computing subunit, the computation request can be met and the time required for the computing unit to complete the computation does not exceed the total time of the first computation activation period and the first pause period. For example, if the ratio between the time of the first computation activation period and the time of the first pause period is 1:2, the first number can be set to 3, so that the total time of the first computation activation period and the first pause period includes three first computation activation periods, so that the first computation activation period is alternately taken as the second computation activation period of each computing subunit.

[0068] In some optional embodiments, the second operation activation period in which each operation subunit activates operations in turn can be further determined based on the total time of the first operation activation period and the first idle period, thereby better matching the first number and improving the distribution evenness of the idle time under the premise of ensuring the operation performance.

[0069] In step 303232, a second idle period of each computing subunit is determined based on the second computing activation period of each computing subunit.

[0070] Here, for any computing subunit, after the second computing activation period of the computing subunit is determined, the period other than the second computing activation period in the total time is the second idle period. For example, if the total time is 0 to t, and 0 to t is divided into three periods 0 to t1, t1 to t2, and t2 to t3, and t1 to t2 is the second computing activation period of the computing subunit, the computing subunit is idle in the periods 0 to t1 and t1 to t2, that is, the periods 0 to t1 and t1 to t2 are the second idle periods of the computing subunit.

[0071] This embodiment realizes alternating computation activation of each computing subunit by alternating the first computation activation period with the second computation activation period of each computing subunit, thereby making the distribution of downtime of each computing subunit more even and helping to improve the evenness of the power consumption distribution of the entire computing unit.

[0072] FIG. 7 is a flowchart of a method for processing feature maps of a neural network by a neural network processor according to yet another exemplary embodiment of the present disclosure.

[0073] In some optional embodiments, as shown in FIG. 7, step 30323 of determining a second computation activation period and a second idle period corresponding to each computation subunit based on the first computation activation period and the first idle period includes steps 30323a and 30323b.

[0074] In step 30323a, the first sleep period is alternately made into a second sleep period for each computing sub-unit.

[0075] In some alternative embodiments, when the computing demand can be met, the first idle period can be alternately set as the second idle period of each computing subunit, and each computing subunit can be alternately idled. For example, the first computing activation period includes 40 computing cycles, the first idle period includes 20 computing cycles, and three computing subunits alternately set the first idle period as the second computing activation period of each computing subunit, and each computing subunit can be alternately idled for 20 computing cycles, thereby realizing an even distribution of idle times.

[0076] In some optional embodiments, the second idle period in which each computing subunit alternately sleeps can be further determined based on the total time of the first computing activation period and the first idle period, thereby better matching the first number and improving the evenness of distribution of the idle time under the premise of ensuring computing performance.

[0077] In step 30323b, a second computing activation period of each computing subunit is determined based on the second sleep period of each computing subunit.

[0078] Here, for any computing subunit, after the second idle period of the computing subunit is determined, the period other than the second idle period in the total time is the second computing activation period.

[0079] In this embodiment, the first pause period is alternately used as the second pause period for each computing subunit, thereby realizing alternating pauses for each computing subunit, thereby making the pause time distribution for each computing subunit more even and helping to improve the evenness of the power consumption distribution across the computing units.

[0080] In some alternative embodiments, FIG. 8 is a schematic diagram of the principle of spatial dimension adjustment according to an exemplary embodiment of the present disclosure. As shown in FIG. 8, before the adjustment, the computing units all activate operations in the first computing activation period and all pause in the first pause period, the spatial utilization distribution of the computing units is concentrated, and the uniformity of the overall power consumption distribution is low. After the adjustment, some computing subunits of the computing units activate operations in the first period, and some other computing subunits pause. In the second period, some computing subunits go from a computing activation state to a pause state, and some other computing subunits go from a pause state to a computing activation state, which reduces the spatial utilization of the computing units in the same period and helps to improve the uniformity of the power consumption distribution of the entire computing units. FIG. 9 is a schematic diagram of the power consumption distribution of the spatial dimension adjustment according to an exemplary embodiment of the present disclosure. As shown in FIG. 9, the schematic diagram of the power consumption distribution corresponding to FIG. 8 is a schematic diagram of the power consumption distribution corresponding to FIG. 8, and the spatial dimension adjustment causes some computing resources to pause in the same period, which helps to reduce the average power consumption of the computing units and improve the uniformity of the power consumption distribution.

[0081] FIG. 10 is a flowchart of a method for processing feature maps of a neural network by a neural network processor according to yet another exemplary embodiment of the present disclosure.

[0082] In some optional embodiments, as shown in FIG. 10 , step 3032 of adjusting a first computation activation period and a first idle period according to at least one of a time dimension and a space dimension to obtain an adjusted second computation activation period and a second idle period includes steps 3032A to 3032C.

[0083] In step 3032A, a second grouping manner of the computing units and an adjustment manner in a time dimension are determined based on the first computing activation period and the first idle period.

[0084] Here, the first computing activation period and the first idle period of the computing unit to be adjusted can be adjusted by integrating the time dimension and the space dimension, thereby more effectively improving the uniformity of the power consumption distribution of the computing units. The second grouping manner corresponds to the space dimension, and the adjustment manner in the time dimension corresponds to the time dimension.

[0085] In some alternative embodiments, the second grouping scheme may include a grouping number of the arithmetic units and a scheme for dividing the arithmetic subunits. The grouping number of the arithmetic units is also called the second number. The second grouping scheme may include two grouping schemes, namely uniform grouping and non-uniform grouping. The uniform grouping is to uniformly divide the computation arrays in the arithmetic units into a second number of groups to obtain computation subunits having the same computation capability of the second number. The non-uniform grouping is to obtain computation subunits having a different computation capability of the second number, in which the number of computation arrays included in each divided computation subunit may not be the same. For example, the arithmetic unit includes 10 sum-of-products arrays, and the arithmetic unit is uniformly divided into two computation subunits, each of which includes five sum-of-products arrays. Or, the arithmetic unit is unevenly divided into two computation subunits, one of which includes six sum-of-products arrays and the other of which includes four sum-of-products arrays.

[0086] The determination principle of the second grouping manner and the adjustment manner in the time dimension is to make the idle time of the computing unit more equal in the time dimension and the space dimension under the premise of ensuring the computing performance of the computing unit. For example, the adjustment manner in the time dimension can include dividing the idle time of each computing subunit into at least two idle subperiods, and for each computing subunit, the at least two idle subperiods of the computing subunit are non-contiguous in the total time between the first computing activation period and the first idle period. The idle subperiods of different computing subunits are different periods in the total time, thereby achieving double equalization in the time dimension and the space dimension.

[0087] Step 3032B divides the computing unit into a second number of computing sub-units based on a second grouping scheme.

[0088] Here, after the second grouping scheme is determined, the computing unit can be divided into a second number of computing subunits according to the second grouping scheme, and different second grouping schemes can correspond to different second numbers.

[0089] In step 3032C, a second computing activation period and a second idle period corresponding to each computing subunit respectively are determined based on the adjustment manner in the time dimension.

[0090] In some alternative embodiments, after determining the adjustment manner in the time dimension, the computation activation period and the idle period of each computing subunit can be adjusted according to the adjustment manner in the time dimension, and the original idle parameters of each computing subunit are consistent with the computing unit to which the computing subunit belongs, i.e., both are the first computation activation period and the first idle period.

[0091] In some alternative examples, the first computation activation period includes 20 computation cycles, the first idle period includes 40 computation cycles, the computation unit can be divided into at least two computation subunits, and the second computation activation period is determined according to computation demands, which is evenly distributed in the 60 computation cycles of each computation subunit, and the second idle period in the 60 computation cycles of each computation subunit is determined based on the second computation activation period of each computation subunit. For example, the computation unit can be divided into two computation subunits, each computation subunit needs to activate the computation for 20 computation cycles and needs to idle for 40 computation cycles, and the computation cycles in which the two computation subunits need to activate the computation can be evenly distributed in the 60 computation cycles to obtain the second computation activation period corresponding to each computation subunit, or the computation cycles in which the two computation subunits are idle can be evenly distributed in the 60 computation cycles to obtain the second idle period corresponding to each computation subunit. For example, calculation subunit 1 activates calculations in calculation cycles 1-5, 16-20, 31-35, and 46-50, and pauses during other periods, while calculation subunit 2 activates calculations in calculation cycles 6-10, 21-25, 36-40, and 51-55, and pauses during other periods. These are merely exemplary distribution methods, and the specific distribution methods are not limited to the above forms.

[0092] In this embodiment, by determining a second grouping method of the computing units and an adjustment method in the time dimension, the first computing activation period and the first idle period of the computing units are adjusted in an integrated manner in the time dimension and the space dimension to obtain doubly equalized second computing activation period and second idle period, which helps to further improve the evenness of the power consumption distribution of the entire computing units.

[0093] FIG. 11 is a flowchart of a method for processing feature maps of a neural network by a neural network processor according to yet another exemplary embodiment of the present disclosure.

[0094] In some optional embodiments, as shown in FIG. 11 , step 304 of controlling the computing unit to execute a computing instruction based on a second pause parameter to obtain a processing result corresponding to the feature map to be processed includes steps 3041 and 3042.

[0095] In step 3041, generate a corresponding control signal based on the second pause parameter for controlling the arithmetic unit to execute the arithmetic instruction.

[0096] Here, the control signal is a signal for controlling the arithmetic unit to enter an operating state and perform arithmetic activation or pause. For example, the control signal can include an enable signal. The enable signal can include two signals: an enable signal (e.g., a high level signal or 1) representing that the arithmetic unit enters an operating state and performs arithmetic activation, and an enable signal (e.g., a low level signal or 0) representing that the arithmetic unit does not enter an operating state. Or, the control signal can include an enable signal and a pause signal. In this case, the enable signal is for controlling whether the arithmetic unit enters an operating state, and the pause signal is for controlling the arithmetic unit to perform arithmetic activation (e.g., the pause signal is 0 or low level) or pause (e.g., the pause signal is 1 or high level). The specific implementation manner of the control signal is not limited.

[0097] In some alternative embodiments, a specific operation state (e.g., operation active or idle) of the computing unit at a current time (or a current calculation cycle) can be determined based on the second operation activation period and the second idle period of the computing unit included in the second idle parameter, and a control signal corresponding to the computing unit can be generated based on the specific operation state of the computing unit. For example, if the computing unit at the current calculation cycle is in an operation activation state, the generated control signal is a signal that causes the computing unit to enter an operation activation state, for example, the enable signal is 1, or the enable signal is 1 and the idle signal is 0.

[0098] In step 3042, a control signal is sent to the arithmetic unit to obtain the arithmetic result of the arithmetic unit, and the processing result corresponding to the feature map to be processed is obtained.

[0099] Here, the arithmetic unit activates or pauses the arithmetic operation according to a control signal. For example, when the arithmetic unit activates the arithmetic operation, the calculation array in the arithmetic unit calculates the input feature value according to the calculation command, and for example, the product-sum array multiplies the input feature value by a weight, and the calculation result of each calculation array constitutes the arithmetic operation result of the arithmetic unit. The controller acquires the calculation result of each arithmetic unit, and can obtain a processing result corresponding to the feature map to be processed based on the calculation result of each arithmetic unit. For example, a processing result of performing a convolution operation on the feature map to be processed is obtained.

[0100] In some alternative embodiments, after executing an arithmetic instruction and obtaining an arithmetic result, the arithmetic unit can return the arithmetic result to the controller, or can store the arithmetic result, for example, in a register or a designated storage space in a memory, and the controller can obtain the arithmetic result of each arithmetic unit from the register or the designated storage space.

[0101] This embodiment generates a control signal for the computing unit based on the adjusted second sleep parameter, and controls the computing unit to activate and sleep based on the adjusted second sleep parameter, thereby realizing an even distribution of the sleep time, improving the evenness of the power consumption distribution in the computing process, and thereby helping to alleviate the power supply integrity problem.

[0102] In some selectable embodiments, step 3041 of generating a corresponding control signal for controlling the arithmetic unit to execute the arithmetic instruction based on the second pause parameter includes, for any arithmetic cycle of the arithmetic instruction, determining an operating state of the arithmetic unit in that arithmetic cycle based on the second pause parameter, and generating a control signal corresponding to the arithmetic unit based on the operating state of the arithmetic unit in that arithmetic cycle.

[0103] Here, the calculation period is the above calculation period, the second pause parameters include the second calculation activation period and the second pause period of the calculation unit, and each second calculation activation period can include one or more calculation periods. Each second pause period can include one or more calculation periods. For any calculation period, the working state of the calculation unit in the calculation period can be determined by determining whether the calculation period belongs to the second calculation activation period or the second pause period. Each calculation unit performs calculation in a calculation period, and the specific calculation types performed by the calculation unit in different periods are the same, such as multiplication, addition, etc. However, the input feature data may be different, for example, the calculation unit needs to complete 10 multiplications, and the multiplier and multiplicand of each multiplication can be different. The second pause parameter of the computing unit includes a second computation activation period and a second pause period of the computing unit, and a specific working state of the computing unit in any computing cycle (called a current computing cycle) can be determined according to the second computation activation period and the second pause period, for example, when the current computing cycle is within a second computation activation period of the computing unit, the working state of the computing unit in the computing cycle is determined as a computation activation state. The generated control signal of the computing unit is a control signal that causes the computing unit to perform computation activation.

[0104] In this embodiment, in each calculation cycle, the operating state of the calculation unit in that calculation cycle is determined based on the second pause parameter, which facilitates accurate generation of the control signal for the calculation unit, realizes accurate and effective control of the calculation unit, and helps achieve the purpose of equalizing the power consumption distribution.

[0105] In some alternative embodiments, the step of generating a control signal corresponding to the arithmetic unit based on the operation state of the arithmetic unit in the arithmetic cycle includes: The method includes the steps of determining an enable state and a pause state of the arithmetic unit based on the operating state of the arithmetic unit in the calculation cycle; generating an enable signal indicating enable in response to the enable state of the arithmetic unit being enabled, where the enable signal indicating enable enables the arithmetic unit to perform calculations when the pause state is non-pause; generating an enable signal indicating disenable in response to the enable state of the arithmetic unit being disabled; generating a pause signal indicating pause in response to the pause state of the arithmetic unit in the calculation cycle being paused, where the pause signal causes the arithmetic unit to pause calculation; and generating a pause signal indicating non-pause in response to the pause state of the arithmetic unit in the calculation cycle being non-pause.

[0106] Here, the enable state represents whether the arithmetic unit enters an operating state, and the enable state can include two kinds of states: enable and disable. The pause state represents whether the arithmetic unit pauses an operation. The pause state can include two kinds of states: pause and not pause. If the enable state of the arithmetic unit is enable, it represents that the arithmetic unit is in an operating state and can be activated for operation, and if the enable state of the arithmetic unit is disable, it represents that the state of the arithmetic unit is in an idle state and cannot be activated for operation. If the pause state of the arithmetic unit is pause, it represents that the state of the arithmetic unit is an operation pause state, and if the pause state of the arithmetic unit is not pause, it represents that the state of the arithmetic unit is an operation activation state. However, it is necessary that the arithmetic unit can enter the operation activation state only when the enable state of the arithmetic unit is enable and the pause state is not pause, thereby helping to realize active pause of the arithmetic unit by the pause state. The enable signal representing the enable can be realized in any form, for example, the enable signal representing the enable may be a high level signal, or the signal state may be 1. Conversely, the enable signal representing disabling may be a low level signal or may have a signal state of 0. The pause signal representing pausing may be a high level signal or may have a signal state of 1. The pause signal representing not pausing may be a low level signal or may have a signal state of 0.

[0107] In some alternative embodiments, FIG. 12 is a schematic diagram of the control principle of the arithmetic unit according to an exemplary embodiment of the present disclosure. As shown in FIG. 12, the neural network processor may further include a memory 230, an input data channel 240, and a weight data channel 250. en represents an enable signal, and stall represents a pause signal. The drawing shows only a schematic diagram of the connection relationship of one arithmetic unit 220, and the other arithmetic units 220 correspond to the connection relationship of the illustrated arithmetic units 220. That is, the controller 210 can generate an enable signal and a pause signal for each arithmetic unit 220 to control the arithmetic units to enter an operating state, activate operation, pause, exit from an operating state, etc. The input data channel 240 reads input feature data (FIN_DATA, for example, a feature map to be processed) from the memory 230 according to the enable signal of the controller 210, and provides the input feature data to the arithmetic unit 220. The weight data channel 250 reads weight data (WT_DATA) from the memory 230 according to the enable signal of the controller 210, and provides the weight data to the calculation unit 220, so that the calculation unit 220 can calculate the input feature data and the weight data to obtain the calculation result according to the enable signal and the pause signal of the controller 210. Here, the calculation unit 220 enters the calculation activation state when en is valid (1 or high level) and stall is invalid (stall is 0 or low level). The calculation unit 220 enters the pause state when en is valid and stall is valid.

[0108] This embodiment realizes control over the state of the computing unit through an enable signal and a pause signal, which helps to realize active pausing of the computing unit through the pause signal on top of the passive pausing of the related technology, thereby supporting adjustment of the first pause parameter of the passive pausing, and actively controlling the pausing of the computing unit through the pause signal based on the adjusted second pause parameter, thereby achieving the purpose of improving the evenness of power consumption distribution.

[0109] In some selectable embodiments, step 3041 of generating a corresponding control signal for controlling the arithmetic unit to execute the arithmetic instruction based on the second pause parameter includes, for any arithmetic cycle of the arithmetic instruction, determining an operating state of each arithmetic subunit in the arithmetic unit in that arithmetic cycle based on the second pause parameter, and generating a control signal corresponding to each arithmetic subunit according to the operating state of each arithmetic subunit.

[0110] Step 3042 of sending control signals to the arithmetic units to obtain arithmetic results of the arithmetic units includes sending control signals respectively corresponding to each arithmetic subunit to each arithmetic subunit to obtain arithmetic results respectively corresponding to each arithmetic subunit.

[0111] Here, for the adjustment method in which spatial dimension adjustment is present, a control signal corresponding to each calculation subunit is generated in each calculation period for the calculation unit that divides the calculation subunits, so that each calculation subunit can be controlled to perform calculation activation and pausing based on the adjusted second pause parameter, and the specific control principle is the same as that of the control of the above-mentioned calculation unit, and redundant explanations will be omitted here.

[0112] In this embodiment, the computing unit that adjusts the spatial dimension for each computing cycle determines the operating state of each computing subunit in the computing cycle based on the second pause parameter of each computing subunit, and generates a control signal corresponding to each computing subunit based on the operating state, thereby realizing respective control of each computing subunit, and making it possible to distribute the pause time of each computing subunit more evenly, thereby improving the evenness of the power consumption distribution of the computing units.

[0113] The method for processing feature maps of a neural network by a neural network processor according to an embodiment of the present disclosure dynamically adjusts the time utilization and space utilization of the computing units, thereby controlling the evenness of the power consumption distribution of the computing units and mitigating the power supply integrity problem.

[0114] Each of the above-mentioned embodiments of the present disclosure may be implemented alone or in any combination manner if not contradictory, and specifically may be set according to actual needs, and is not limited in the present disclosure.

[0115] Any of the methods for processing a feature map of a neural network by a neural network processor according to the embodiments of the present disclosure may be executed by any suitable device having data processing capabilities, including, but not limited to, a terminal device, a server, etc. Alternatively, any of the methods for processing a feature map of a neural network by a neural network processor according to the embodiments of the present disclosure may be executed by a processor, for example, the processor executes any of the methods for processing a feature map of a neural network by a neural network processor mentioned in the embodiments of the present disclosure by invoking corresponding instructions stored in a memory. Hereinafter, duplicated descriptions will be omitted.

[0116] [Example Device] 13 is a schematic diagram of the structure of a neural network feature map processing device by a neural network processor according to an exemplary embodiment of the present disclosure. The neural network feature map processing device of the embodiment is for realizing an embodiment of a neural network feature map processing method by a neural network processor corresponding to the present disclosure, and the device shown in FIG. 13 includes a determination module 51, a first processing module 52, a second processing module 53 and a third processing module 54.

[0117] The determination module 51 determines the calculation instructions for calculating the feature map to be processed.

[0118] The first processing module 52 determines pause state information for the arithmetic units in the neural network processor to execute arithmetic instructions.

[0119] The second processing module 53 adjusts first pause parameters for the arithmetic units to execute the arithmetic instructions based on the pause state information, and determines second pause parameters after the adjustment.

[0120] The third processing module 54 controls the arithmetic unit to execute an arithmetic instruction based on the second pause parameter, and obtains a processing result corresponding to the feature map to be processed.

[0121] Each module of the apparatus of the embodiment of the present disclosure can be realized by any of a method such as software, hardware, or a combination of software and hardware.

[0122] FIG. 14 is a schematic diagram of the structure of a processing device for a feature map of a neural network by a neural network processor according to another exemplary embodiment of the present disclosure.

[0123] In some alternative embodiments, as shown in FIG. 14, the second processing module 53 includes a first processing unit 531 and a second processing unit 532.

[0124] The first processing unit 531 determines, based on the pause state information, a first operation activation period for the operation unit to execute the operation instruction and a first pause period.

[0125] The second processing unit 532 adjusts the first computation activation period and the first pause period according to at least one of the time dimension and the space dimension to obtain adjusted second computation activation period and second pause period as second pause parameters.

[0126] In some alternative embodiments, specifically, the second processing unit 532 adjusts the first computation activation period and the first idle period into at least two idle sub-periods and at least one computation activation sub-period sorted in a pre-set manner in the time dimension, and each of the sorted computation activation sub-periods is a second computation activation period, and each of the sorted idle sub-periods is a second idle period.

[0127] In some selectable embodiments, specifically, the second processing unit 532 determines a first grouping scheme of the computing units based on the first computing activation period and the first idle period, divides the computing units into a first number of computing subunits according to the first grouping scheme, and determines second computing activation periods and second idle periods corresponding to each computing subunit, respectively, based on the first computing activation period and the first idle period. In some alternative embodiments, specifically, the second processing unit 532 alternates the first computation activation period as the second computation activation period of each computation subunit, and determines the second idle period of each computation subunit based on the second computation activation period of each computation subunit.

[0128] In some alternative embodiments, specifically, the second processing unit 532 alternates between the first idle period and the second idle period of each computing subunit, and determines the second computing activation period of each computing subunit based on the second idle period of each computing subunit.

[0129] In some selectable embodiments, specifically, the second processing unit 532 determines a second grouping scheme of the computing units and an adjustment scheme in the time dimension based on the first computing activation period and the first idle period, divides the computing units into a second number of computing subunits based on the second grouping scheme, and determines second computing activation periods and second idle periods respectively corresponding to each computing subunit based on the adjustment scheme in the time dimension.

[0130] FIG. 15 is a schematic diagram of the structure of a processing device for a feature map of a neural network by a neural network processor according to yet another exemplary embodiment of the present disclosure.

[0131] In some alternative embodiments, as shown in FIG. 15, the third processing module 54 includes a third processing unit 541 and a control unit 542.

[0132] The third processing unit 541 generates, based on the second pause parameter, a corresponding control signal for controlling the arithmetic unit to execute the arithmetic instruction.

[0133] The control unit 542 transmits a control signal to the arithmetic unit to obtain the arithmetic result of the arithmetic unit, and obtains the processing result corresponding to the feature map to be processed.

[0134] In some selectable embodiments, specifically, for any calculation cycle of the calculation instruction, the third processing unit 541 determines the operating state of the calculation unit in the calculation cycle based on the second pause parameter, and generates a control signal corresponding to the calculation unit based on the operating state of the calculation unit in the calculation cycle.

[0135] In some selectable embodiments, specifically, the third processing unit 541 determines the enable state and the pause state of the arithmetic unit based on the operating state of the arithmetic unit in the calculation cycle, generates an enable signal representing enable when the enable state of the arithmetic unit is enabled, generates an enable signal representing disenable when the enable state of the arithmetic unit is disabled, generates a pause signal when the pause state of the arithmetic unit in the calculation cycle is paused, and generates a pause signal representing non-pause when the pause state of the arithmetic unit in the calculation cycle is non-pause, where the enable signal representing enable enables the arithmetic unit to perform calculation when the pause state is non-pause, and the pause signal causes the arithmetic unit to pause calculation.

[0136] In some selectable embodiments, specifically, the third processing unit 541 determines, for any calculation cycle of the calculation instruction, the operating state of each calculation subunit in the calculation unit in that calculation cycle based on the second pause parameter, and generates a control signal corresponding to each calculation subunit according to the operating state of each calculation subunit.

[0137] Specifically, the control unit 542 transmits control signals corresponding to each calculation subunit to each calculation subunit, obtains calculation results corresponding to each calculation subunit, and obtains processing results corresponding to the feature map to be processed.

[0138] The beneficial technical effects corresponding to the exemplary embodiments of the present apparatus may refer to the beneficial technical effects corresponding to the exemplary methods described above, and redundant descriptions will be omitted here.

[0139] [Examples of electronic devices] FIG. 16 is a structural diagram of an electronic device including at least one processor 11 and a memory 12 according to an embodiment of the present disclosure.

[0140] The processor 11 may be a central processing unit (CPU) or other type of processing unit having data processing and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0141] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as, for example, volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache). The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. The computer-readable storage media may store one or more computer program instructions, and the processor 11 may execute one or more computer program instructions to realize the methods and / or other desired functions according to the embodiments of the present disclosure described above.

[0142] As an example, electronic device 10 may further include input device(s) 13 and output device(s) 14 connected to each other via a bus system and / or other form of connection mechanism (not shown).

[0143] The input device 13 may further include, for example, a keyboard and a mouse.

[0144] The output device 14 can output various types of information to the outside, and can include, for example, a display, a speaker, a printer, a communication network, and a remote output device connected thereto.

[0145] For simplicity, Fig. 16 shows only some of the components related to the present disclosure in the electronic device 10, and omits components such as a bus, an input / output interface, etc. In addition, the electronic device 10 may further include any other appropriate components according to a specific application situation.

[0146] Furthermore, an embodiment of the present disclosure further provides an electronic device including a processing device for a neural network feature map by the neural network processor according to any of the above embodiments, or including a neural network processor according to any of the above embodiments.

[0147] [Exemplary Computer Program Products and Computer-Readable Storage Media] In addition to the methods and apparatus described above, embodiments of the present disclosure may provide a computer program product including computer program instructions that, when executed by a processor, cause the processor to perform the steps of the methods of the various embodiments of the present disclosure described in the "Exemplary Methods" section above.

[0148] The computer program product may be produced with program code for carrying out operations of the disclosed embodiments in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and traditional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partly on the user computing device and partly on a remote computing device, or entirely on the remote computing device or a server.

[0149] Moreover, embodiments of the present disclosure further provide a computer-readable storage medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform the steps of the methods of the various embodiments of the present disclosure described in the "Exemplary Method" section above.

[0150] The computer-readable storage medium may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0151] Although the basic principles of the present disclosure have been described above with reference to specific embodiments, the advantages, merits, effects, etc. mentioned in the present disclosure are merely illustrative and not limiting, and these advantages, merits, effects, etc. are not necessarily possessed by each embodiment of the present disclosure. In addition, the specific details of the above disclosure are merely illustrative and easy-to-understand actions and are not limiting, and the above details do not necessarily limit the present disclosure to be realized by the above specific details.

[0152] Those skilled in the art can make various modifications and variations to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. 1. A method of processing a feature map of a neural network by a neural network processor, comprising: determining a computation command for computing a feature map to be processed; determining pause state information for an arithmetic unit in the neural network processor to execute the arithmetic instruction; adjusting a first pause parameter for the computing unit to execute the computing instruction based on the pause state information, and determining an adjusted second pause parameter; and controlling the arithmetic unit to execute the arithmetic instruction based on the second pause parameter to obtain a processing result corresponding to the feature map to be processed.

13. A method for processing a feature map of a neural network by a neural network processor, comprising:

2. The step of adjusting a first pause parameter for the computing unit to execute the computing instruction based on the pause state information and determining a second pause parameter after the adjustment, determining a first operational activation period and a first idle period for the operational unit to execute the operational instruction based on the idle state information; adjusting the first computational activation period and the first pause period according to at least one of a time dimension and a space dimension to obtain an adjusted second computational activation period and a second pause period as the second pause parameter.

2. The method of claim 1.

3. Adjusting the first computational activation period and the first idle period according to at least one of a time dimension and a space dimension to obtain adjusted second computational activation period and second idle period includes: adjusting the first computation activation period and the first idle period into at least two idle sub-periods and at least one computation activation sub-period sorted in a time dimension in a preset manner, each of the sorted computation activation sub-periods being the second computation activation period, and each of the sorted idle sub-periods being the second idle period; 3. The method according to claim 2 .

4. Adjusting the first computational activation period and the first idle period according to at least one of a time dimension and a space dimension to obtain adjusted second computational activation period and second idle period, the step of: determining a first grouping manner of the computing units according to the first computing activation period and the first sleep period; dividing the computing units into a first number of computing sub-units according to the first grouping scheme; determining a second computational activation period and a second sleep period corresponding to each of the computing subunits based on the first computational activation period and the first sleep period, respectively; 3. The method according to claim 2 .

5. determining a second computational activation period and a second sleep period, each corresponding to each of the computing subunits, based on the first computational activation period and the first sleep period, rotating the first computational activation period into a second computational activation period of each of the computing subunits; determining a second idle period for each of the computing subunits based on the second computing activation period for each of the computing subunits; 5. The method according to claim 4.

6. determining a second computational activation period and a second sleep period, each corresponding to each of the computing subunits, based on the first computational activation period and the first sleep period, rotating the first idle period into a second idle period for each of the computing subunits; determining a second computational activation period of each of the computing subunits based on the second dormant period of each of the computing subunits; 5. The method according to claim 4.

7. Adjusting the first computational activation period and the first idle period according to at least one of a time dimension and a space dimension to obtain adjusted second computational activation period and second idle period, the step of: determining a second grouping manner and a time dimension adjustment manner of the computing units according to the first computing activation period and the first idle period; dividing the computing units into a second number of computing sub-units based on the second grouping scheme; determining second computing activation periods and second idle periods corresponding to each of the computing subunits according to the time dimension adjustment manner; 3. The method according to claim 2 .

8. The step of controlling the arithmetic unit to execute the arithmetic instruction based on the second pause parameter to obtain a processing result corresponding to the feature map to be processed includes: generating a corresponding control signal based on the second pause parameter for controlling the computing unit to execute the computing instruction; transmitting the control signal to the arithmetic unit, acquiring an arithmetic result of the arithmetic unit, and acquiring a processing result corresponding to the feature map to be processed; 3. The method according to claim 1, wherein the first and second components are fused together.

9. generating a corresponding control signal for controlling the arithmetic unit to execute the arithmetic instruction based on the second pause parameter, determining an operation state of the arithmetic unit in any one of the operation cycles of the operation instruction based on the second pause parameter; generating a control signal corresponding to the arithmetic unit based on an operation state of the arithmetic unit in the calculation cycle; 9. The method of claim 8.

10. The step of generating a control signal corresponding to the arithmetic unit based on an operation state of the arithmetic unit in the calculation cycle includes: determining an enabled state or a paused state of the arithmetic unit based on an operation state of the arithmetic unit in the calculation cycle; generating an enable signal representing an enable in response to the enable state of the computing unit being enabled, the enable signal representing an enable allowing the computing unit to perform a computation when the pause state is non-pause; generating an enable signal representing disabling in response to the enable state of the arithmetic unit being disabled; generating a pause signal representing a pause in response to the pause state of the arithmetic unit in the corresponding calculation cycle being a pause, the pause signal representing a pause causing the arithmetic unit to pause calculation; generating a pause signal representing a non-pause state in response to the pause state of the computing unit in the computing cycle being a non-pause state; 10. The method of claim 9.

11. generating a corresponding control signal for controlling the arithmetic unit to execute the arithmetic instruction based on the second pause parameter, determining, for any one of the operation cycles of the operation instruction, an operation state of each of the operation subunits in the operation unit during the operation cycle based on the second pause parameter; generating a control signal corresponding to each of the computing subunits based on the operating state of each of the computing subunits; The step of transmitting the control signal to the arithmetic unit and obtaining the arithmetic result of the arithmetic unit includes: sending control signals corresponding to each of the operation subunits to each of the operation subunits to obtain operation results corresponding to each of the operation subunits; 9. The method of claim 8.

12. 1. An apparatus for processing a feature map of a neural network by a neural network processor, comprising: a determination module for determining a computation command for computing a feature map to be processed; a first processing module for determining pause state information for an arithmetic unit in the neural network processor to execute the arithmetic instruction; a second processing module for adjusting a first pause parameter for the computing unit to execute the computing instruction based on the pause state information, and determining an adjusted second pause parameter; and a third processing module for controlling the arithmetic unit to execute the arithmetic instruction based on the second pause parameter to obtain a processing result corresponding to the feature map to be processed. A device for processing a feature map of a neural network by a neural network processor.

13. 1. A computer-readable storage medium, comprising: The storage medium stores a computer program for executing the method for processing feature maps of a neural network by a neural network processor according to any one of claims 1. A computer-readable storage medium comprising:

14. A processor; a memory for storing instructions executable by the processor; The processor reads and executes the executable instructions from the memory to perform the method of processing neural network feature maps with a neural network processor according to any one of claims 1 to 5. Or, A device for processing feature maps of a neural network according to claim 12, 1. An electronic device comprising:

Images