Method, apparatus and device for processing neural network feature maps by a neural network processor

Abstract

To provide a processing method, a device and equipment for a feature map of a neural network by a neural network processor.SOLUTION: A method includes the steps of: determining an arithmetic instruction for calculating a feature map to be processed; determining idle state information for executing the arithmetic instruction by an arithmetic unit of a neural network processor; adjusting a first idle parameter for executing the arithmetic instruction by the arithmetic unit based upon the idle state information, and then determining a second idle parameter after the adjustment; and controlling the arithmetic unit based upon the second idle parameter to execute the arithmetic instruction by the arithmetic unit, and acquiring a processing result corresponding to the feature map to be processed.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to artificial intelligence technology, and more particularly to a method, apparatus and device for processing feature maps of a neural network by a neural network processor. [Background technology]

[0002] In the field of artificial intelligence, the bandwidth requirements for different neural network model operations can vary greatly. For example, the bandwidth requirements for convolution operations of different neural network models can vary greatly. When a neural network processor's arithmetic circuit (also known as an arithmetic unit or tensor calculation circuit) calculates a neural network feature map, it is likely that the circuit will passively pause operations due to input and output bandwidth limitations. Furthermore, data dependencies exist between the arithmetic circuit and other arithmetic circuits, making it necessary to wait for the results of other arithmetic circuits' operations. However, when an arithmetic circuit pauses operations for a long period of time, the power consumption distribution becomes uneven, potentially causing power integrity issues. Summary of the Invention [Problem to be solved by the invention]

[0003] In order to solve the above-mentioned problems such as low uniformity of power consumption distribution, one embodiment of the present disclosure provides a method, device, and equipment for processing neural network feature maps using a neural network processor, thereby improving the uniformity of power consumption distribution. [Means for solving the problem]

[0004] A method for processing a feature map of a neural network by a neural network processor according to a first aspect of the present disclosure includes the steps of: determining an operation instruction for calculating a feature map to be processed; determining pause state information for an operation unit in the neural network processor to execute the operation instruction; adjusting, based on the pause state information, a first pause parameter for the operation unit to execute the operation instruction, and determining an adjusted second pause parameter; and controlling, based on the second pause parameter, the operation unit to execute the operation instruction, thereby obtaining a processing result corresponding to the feature map to be processed.

[0005] A processing device for a neural network feature map using a neural network processor according to a second aspect of the present disclosure includes: a determination module for determining an operation instruction for calculating a feature map to be processed; a first processing module for determining pause state information for an operation unit in the neural network processor to execute the operation instruction; a second processing module for adjusting a first pause parameter for the operation unit to execute the operation instruction based on the pause state information and determining an adjusted second pause parameter; and a third processing module for controlling the operation unit to execute the operation instruction based on the second pause parameter, thereby obtaining a processing result corresponding to the feature map to be processed.

[0006] A computer-readable storage medium according to a third aspect of the present disclosure stores a computer program for executing the method for processing feature maps of a neural network by a neural network processor according to any of the above embodiments of the present disclosure.

[0007] An electronic device according to a fourth aspect of the present disclosure comprises a processor and a memory for storing instructions executable by the processor, wherein the processor reads the executable instructions from the memory and executes the instructions to perform a method for processing a neural network feature map by a neural network processor as described in any of the above embodiments of the present disclosure, or comprises an apparatus for processing a neural network feature map by a neural network processor as described in any of the above embodiments of the present disclosure.

[0008] A computer program product according to a fifth aspect of the present disclosure performs the method for processing feature maps of a neural network by a neural network processor provided in any of the above embodiments of the present disclosure when instructions in the computer program product are executed by a processor. [Effects of the Invention]

[0009] According to the method, apparatus, and device for processing a neural network feature map using a neural network processor according to the above embodiments of the present disclosure, when processing a neural network feature map using a neural network processor, the following steps are performed: determining operation instructions for calculating the feature map to be processed; determining pause state information for each processing unit in the neural network processor to execute the operation instructions; adjusting first pause parameters in the pause state information for each processing unit to determine adjusted second pause parameters; and controlling the processing units to execute their respective operation instructions based on the second pause parameters to obtain a processing result corresponding to the feature map to be processed. The second pause parameters are obtained by adjusting the first pause parameters, and the second pause parameters can more uniformly distribute the time and / or space during which the processing units pause, thereby reducing the width and duration of sudden changes in power consumption each time, and more uniformly distributing the pause times across all processing units of the neural network processor, thereby improving the uniformity of the power consumption distribution of the neural network processor and helping to mitigate or resolve power supply integrity issues. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is an exemplary application scenario of a method for processing feature maps of a neural network by a neural network processor according to the present disclosure; [Figure 2] 1 is a flowchart of a method for processing feature maps of a neural network by a neural network processor according to an exemplary embodiment of the present disclosure. [Figure 3] 10 is a flowchart of a method for processing feature maps of a neural network by a neural network processor according to another exemplary embodiment of the present disclosure. [Figure 4] FIG. 10 is a schematic diagram of a power consumption distribution of time dimension adjustment according to an exemplary embodiment of the present disclosure. [Figure 5]10 is a flowchart of a method for processing feature maps of a neural network by a neural network processor according to yet another exemplary embodiment of the present disclosure. [Figure 6] 10 is a flowchart of a method for processing feature maps of a neural network by a neural network processor according to yet another exemplary embodiment of the present disclosure. [Figure 7] 10 is a flowchart of a method for processing neural network feature maps by a neural network processor according to yet another exemplary embodiment of the present disclosure. [Figure 8] FIG. 1 is a schematic diagram of the principle of spatial dimension adjustment according to an exemplary embodiment of the present disclosure. [Figure 9] FIG. 10 is a schematic diagram of power consumption distribution of spatial dimension adjustment according to an exemplary embodiment of the present disclosure. [Figure 10] 10 is a flowchart of a method for processing feature maps of a neural network by a neural network processor according to yet another exemplary embodiment of the present disclosure. [Figure 11] 10 is a flowchart of a method for processing feature maps of a neural network by a neural network processor according to yet another exemplary embodiment of the present disclosure. [Figure 12] FIG. 2 is a schematic diagram of the control principle of a computing unit according to an exemplary embodiment of the present disclosure. [Figure 13] FIG. 1 is a schematic diagram of the structure of a processing device for neural network feature maps by a neural network processor according to an exemplary embodiment of the present disclosure. [Figure 14] FIG. 10 is a schematic diagram of the structure of a processing device for neural network feature maps by a neural network processor according to another exemplary embodiment of the present disclosure. [Figure 15] FIG. 10 is a schematic diagram of the structure of a processing device for neural network feature maps by a neural network processor according to yet another exemplary embodiment of the present disclosure. [Figure 16] 1 is a structural diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0011] In order to understand the present disclosure, exemplary embodiments of the present disclosure will be described in detail below with reference to the drawings. Furthermore, the described embodiments are only a part of the embodiments of the present disclosure, not all of the embodiments, and the present disclosure is not limited to the exemplary embodiments.

[0012] Unless otherwise specifically stated, the relative arrangements of components and steps, formulas and numerical values ​​described in these embodiments do not limit the scope of the present disclosure.

[0013] [Summary of this disclosure] In the process of realizing the present disclosure, the inventors discovered that in the field of artificial intelligence technology, the operations of different neural network models may have significantly different bandwidth demands, for example, the convolution operations of different neural network models may have significantly different bandwidth demands. When a neural network processor's arithmetic circuit (also known as an arithmetic unit or tensor calculation circuit) calculates a neural network feature map, it is likely that the operation will be passively paused due to input and output bandwidth limitations, or that data dependencies exist between the arithmetic circuit and other arithmetic circuits, resulting in situations where the arithmetic circuit must wait for the results of other arithmetic circuits. However, when an arithmetic circuit pauses its operation for a long period of time, the power consumption distribution becomes less uniform, which can easily cause power supply integrity issues.

[0014] [Illustrative Overview] FIG. 1 illustrates an exemplary application scenario of the method for processing a neural network feature map by a neural network processor according to the present disclosure. As shown in FIG. 1, the neural network processor may include a controller 210 and at least one arithmetic unit 220. The controller 210 controls the operation of the arithmetic unit 220. For example, the controller 210 activates the arithmetic unit 220 to activate it and controls the arithmetic unit 220 to execute an arithmetic instruction to complete the corresponding operation. The arithmetic unit 220 may include a multiply-accumulate array (MAC_ARRAY) arithmetic unit, a multiply-accumulate post-processing (MAC_PP) arithmetic unit, etc. The multiply-accumulate array arithmetic unit may be used to complete multiply-accumulate operations such as convolution and matrix multiplication. The multiply-accumulate post-processing arithmetic unit may be used to complete post-processing operations after multiply-accumulate operations, such as adding the convolution result with a skip-connection feature in a neural network model. The neural network processor may further include memory and other related components (not shown in FIG. 1). For example, the memory can store a feature map to be processed, an operation instruction, weight data, etc. The method for processing a feature map of a neural network by a neural network processor of the present disclosure (which can be executed by the controller 210) determines an operation instruction to calculate the feature map to be processed, determines pause state information for the operation unit 220 in the neural network processor to execute the operation instruction, adjusts a first pause parameter for the operation unit 220 to execute the operation instruction based on the pause state information and determines a second pause parameter after adjustment, and controls the operation unit 220 to execute the operation instruction based on the second pause parameter, thereby obtaining a processing result corresponding to the feature map to be processed.The method of the present disclosure obtains a second pause parameter by adjusting a first pause parameter for the computing unit 220 to execute an operation instruction, and the second pause parameter can make the temporal and / or spatial distribution of the times at which the computing unit 220 pauses operations more uniform, thereby reducing the width and duration of each sudden change in power consumption, making the pause time distribution across the computing units of the neural network processor more uniform, thereby improving the uniformity of the power consumption distribution of the neural network processor and helping to mitigate or solve power supply integrity issues.

[0015] [Exemplary Method] 2 is a flowchart of a method for processing a feature map of a neural network by a neural network processor according to an exemplary embodiment of the present disclosure. This embodiment is particularly applicable to electronic devices such as in-vehicle computing platforms or other terminal devices with image processing needs, and includes the following steps 301 to 304, as shown in FIG.

[0016] In step 301, a calculation command for calculating the feature map to be processed is determined.

[0017] In some alternative embodiments, the feature map to be processed may be a feature map to be computed by a neural network processor at any stage in a neural network model. The neural network model may include, for example, a target detection model, an image classification model, a semantic segmentation model, etc. The feature map to be processed may be, for example, any feature map to be subject to a convolution operation in the neural network model, a feature map to be subject to matrix multiplication, etc. The computation instruction is an instruction executed by the neural network processor to perform a computation on the feature map to be processed. The computation instruction can be obtained by compiling the neural network model and placed in a terminal device to realize accelerated estimation of the model by the neural network processor. For example, the computation instruction may include a computation instruction corresponding to a convolution operation, a computation instruction corresponding to a matrix multiplication, etc. The operation instruction may include one or more instructions to provide the operation unit with data required for the operation. For example, an operation instruction corresponding to a convolution operation may include relevant information such as the address and size of an input tensor (e.g., a feature map to be processed or a portion of a tensor in the feature map to be processed), relevant information such as the address and size of a convolution weight, relevant information such as an operation type (e.g., multiplication, addition, etc.), an operation cycle, etc. The operation instruction may further include relevant information such as the storage address and size of an output tensor to store the tensor output from the operation and provide an operation result corresponding to subsequent processing. The operation unit of the neural network processor may then execute the corresponding operation instruction to complete the operation on the feature map to be processed. The specific operation instruction may be determined according to the specific operation in an actual scenario and is not limited to the embodiments of the present disclosure.

[0018] In some optional embodiments of the present disclosure, the calculation instructions for the feature map to be processed can be determined by reading the calculation instructions for the feature map to be processed from a memory space that stores the calculation instructions.

[0019] In step 302, the arithmetic units in the neural network processor determine pause state information for executing arithmetic instructions.

[0020] In some alternative embodiments, as shown in FIG. 1 , the arithmetic unit in the neural network processor may include one or more arithmetic units. Generally, it includes multiple arithmetic units. Each arithmetic unit cooperates with others to complete an operation on the feature map to be processed. Each arithmetic unit may include at least one calculation array of a predetermined size. For example, the arithmetic unit may include at least one multiplication and accumulation array (MAC), where each multiplication and accumulation array completes the multiplication of one input feature value and a weight at each time (i.e., at each calculation cycle) to obtain an operation result. For example, the input feature value 7 may be represented in binary as 00000111 and multiplied by the weight 00001011. The multiplication and accumulation array multiplies the input feature value by each bit of the weight as a multiplier, thereby obtaining a calculation result (also called a partial product) corresponding to each bit of the input feature value. The calculation results for each bit are then compressed to obtain the multiplication result of the input feature value and the weight.

[0021] In some alternative embodiments, the pause state information for an arithmetic unit to execute an arithmetic instruction means that after an arithmetic unit enters an operating state, due to factors such as data dependencies between arithmetic units or input / output bandwidth limitations, the arithmetic unit must perform calculations for a certain period of time to obtain its own calculation results and then enter a pause state, for example, waiting for the calculation results of other arithmetic units, waiting for new input data, waiting for the output of its own calculation results, etc.

[0022] In some alternative embodiments, the pause state information for the computing units to execute the computation instructions may be determined according to the specific computation instructions. For example, the execution time of each instruction may be determined according to the computation instructions, and the computation active period and the pause period for the computing unit to execute the computation instructions may be determined based on the execution time of each instruction. The pause state information may include the computation active period and the pause period for the computing unit to execute the computation instructions. For example, the time when each computing unit enters an active state from an idle state (when it is capable of performing computation but may not yet be performing computation) is taken as an initial point in time (e.g., time 0). Computing unit A executes a computation instruction in a period from 0 to t1 to enter the computation active state, where t1 is one point in time. Computing unit B executes a computation instruction in a period from 0 to t2 to enter the computation active state, where t2 is another point in time, e.g., t2 is greater than t1. If the computation result of computing unit A needs to be synchronized with the computation result of computing unit B, computing unit A needs to pause its current computation after completing the current computation and wait for the computation result of computing unit B. That is, arithmetic unit A enters a sleep state during the period from t1 to t2. For example, if arithmetic unit A includes 10 sum-of-products arrays and needs to calculate 100 multiplications, the 10 sum-of-products arrays need to complete the calculation of the 100 multiplications in 10 calculation cycles (each sum-of-products array completes one multiplication in each calculation cycle), but if another operation that needs to be synchronized requires 20 calculation cycles, the 10 sum-of-products arrays enter a sleep state after completing the operation of 10 calculation cycles (these 10 calculation cycles are also called a first operation activation period) and need to wait for the completion of the other operation. This allows each arithmetic unit to obtain sleep state information for executing an operation instruction.

[0023] In step 303, the first pause parameters for the computing unit to execute the computing instruction are adjusted based on the pause state information, and the adjusted second pause parameters are determined.

[0024] Here, the first pause parameter is the original pause parameter of the computing unit determined according to the computing instruction. For example, the first pause parameter may include a first computation activation period and a first pause period for each computing unit to execute the computing instruction. For each computing unit, the first computation activation period and the first pause period are two consecutive periods, and the total time of the two periods may represent the time from the start of the current computing task by the computing unit to the start of the next computing task. For example, multiple computing units cooperate to complete the calculation of a feature map to be processed, and the current computing task of computing unit A among the multiple computing units is to complete 100 multiplications. After computing unit A starts its calculation, it completes the 100 multiplications within the first computation activation period and then enters a first pause period, during which computing unit A pauses and waits for the next computing task. The next computing task can continue to be processed according to the processing flow of the current computing task.

[0025] In some alternative embodiments, the first computational activation period can include a first computational activation start time and a first computational activation end time, and the first sleep period can include a second computational activation start time and a second sleep end time. Here, the time points can be relative. For example, if the initial time at which the computing unit enters the computational activation state is time point 0, the first computational activation period start time is time point 0, and the first computational activation period end time is a time delayed by the first computational activation period time from the first start time. The second computational activation period start time can be the first computational activation period end time, at which the computing unit exits the computational activation state and enters the sleep state. The second sleep period end time is a time delayed by the first sleep period time from the second start time.

[0026] In some alternative embodiments, the first calculation activation period and the first sleep period can be displayed in calculation cycles, for example, the first calculation activation period includes n calculation cycles from 1 to n, and the first sleep period includes calculation cycles from n+1 to N. For example, the first calculation activation period includes 10 calculation cycles from 1 to 10, and the first sleep period includes 10 calculation cycles from 11 to 20. The specific display method of the first calculation activation period and the first sleep period is not limited.

[0027] In some alternative embodiments, a first sleep parameter may correspond to each of the multiple computing units. Based on the sleep state information of each computing unit, the first sleep parameters of some of the computing units may be adjusted to obtain adjusted second sleep parameters. For example, a target computing unit to be adjusted is determined from each computing unit based on dependencies between the computing units, and the first sleep parameter for the target computing unit to execute a computing instruction is adjusted to obtain adjusted second sleep parameters for the target computing unit. Alternatively, based on the sleep state information, the first sleep parameters for the computing units to execute a computing instruction may be adjusted, where the adjusted sleep parameter for the target computing unit to be adjusted is considered to be the adjusted second sleep parameter for the target computing unit, and the first sleep parameter for the computing unit that does not need to be adjusted is considered to be the adjusted second sleep parameter for the target computing unit. In some alternative embodiments, the purpose of adjusting the first sleep parameter is to evenly distribute the sleep periods of at least one computing unit among the computing units in either time or space, thereby improving the evenness of power consumption distribution across the computing units. For example, the first sleep parameter of the computing unit may be adjusted from one continuous sleep period to at least two non-contiguous sleep periods, e.g., the first sleep parameter of the computing unit may be adjusted to first activate the computing unit for a predetermined period (first computing activation period) and then sleep for a predetermined period (first sleep period). The first sleep parameter may be adjusted to first activate the computing unit for a predetermined period and then sleep for a predetermined period, and then activate the computing unit for a predetermined period and then sleep for a further predetermined period, thereby obtaining an adjusted second sleep parameter, which includes two non-contiguous second computing activation periods and two non-contiguous second sleep periods. That is, adjusting the first sleep period of the computing unit to two non-contiguous second sleep periods effectively improves the uniformity of the distribution of power consumption across the computing unit in the time dimension.For example, the first sleep parameter of the computing unit may be adjusted so that some subunits (e.g., referred to as the first subunits) of the computing unit first activate their operations for a predetermined period, during which some other subunits (e.g., referred to as the second subunits) of the computing unit enter a sleep state, after which the first subunit enters a sleep state and the second subunit enters an active operation state. In this way, the first subunits and the second subunits alternately sleep, achieving a spatially equalized distribution of power consumption of the computing units. Furthermore, the first sleep parameter may be adjusted by integrating the time dimension and the spatial dimension to achieve a spatially equalized distribution of power consumption of the computing units.

[0028] In some alternative embodiments, a first pause parameter of the computing unit can be adjusted according to a predetermined adjustment principle based on the pause state information to obtain an adjusted second pause parameter.

[0029] In some alternative embodiments, the adjustment principle may include a principle of adjusting according to a time dimension and a principle of adjusting according to a space dimension, and the principle of adjusting according to the space dimension may include two kinds of principles: a principle of adjusting only according to the space dimension and a principle of adjusting according to a mixture of time and space. The principle of adjusting according to the time dimension may include: a total time of the operation activation period and the pause period in the adjusted second pause parameter is consistent with a total time of the first pause parameter; a total pause time of the adjusted second pause parameter is consistent with a total pause time of the first pause parameter; and a total operation activation time of the adjusted second pause parameter is consistent with a total operation activation time of the first pause parameter, thereby ensuring that the operation unit can complete the current operation task within the total time of the second pause parameter. For example, the first sleep parameter includes a period from 0 to t, the total time is t, the first computation activation period is from 0 to t1, and the first sleep period is from t1 to t2. The second sleep parameter includes at least two non-consecutive second sleep periods. For example, the second sleep period 1 is from 0 to t2, the second computation activation period is from t2 to t3, and the second sleep period 2 is from t3 to t3, where t3-t2=t1. The principle of adjusting according to the spatial dimension includes ensuring that the total time of the adjusted second sleep parameter is consistent with the total time of the first sleep parameter (i.e., the total time of the computation activation and sleep of each computing subunit divided according to space is consistent with the total time of the first computation activation period and the first sleep period), and that the computing unit can complete the current computing task within the total time of the second sleep parameter when the power consumption is uniform. For example, the first computation activation period of computing unit A is from 0 to t1, and the first sleep period is from t1 to t, and computing unit A is divided into two computing subunits, and the second computation activation period of computing subunit 1 is from 0 to t1, and the second sleep period of computing subunit 1 is from t1 to t. The second sleep period of computing subunit 2 is from 0 to t1, and the second computation activation period of computing subunit 2 is from t1 to t.

[0030] In some alternative embodiments, the adjustment principle may include that the total time between computation activation and pause under the second pause parameter is consistent with the first pause parameter, and the total computation amount of the computing unit under the second pause parameter is consistent with the total computation amount of the first pause parameter. For example, in terms of unit computation power (e.g., sum-of-products array) in the computing unit, the sum of the computation activation times (or the number of computation cycles of computation activation) of each sum-of-products array under the second pause parameter is equal to the sum of the computation activation times (or the number of computation cycles of computation activation) of each sum-of-products array under the first pause parameter. For example, if the calculation unit A includes 10 sum-of-products arrays, and the first sleep parameter is set such that the first calculation activation period of the 10 sum-of-products arrays includes 10 calculation cycles, and the first sleep period includes 10 calculation cycles, that is, the number of calculation cycles for calculation activation of each sum-of-products array is 10, and the sum of the number of calculation cycles for calculation activation of each sum-of-products array is 10*10=100, which corresponds to the number of calculation cycles for which one sum-of-products array needs to activate its calculation to complete the current calculation task being 100. Taking the spatial dimension adjustment as an example, under the second sleep parameter, the second operation activation period of five multiply-accumulate arrays includes 10 operation cycles from 1 to 10, and the second sleep period includes 10 operation cycles from 11 to 20; the second operation activation period of another five multiply-accumulate arrays includes 10 operation cycles from 1 to 10, and the second operation activation period includes 10 operation cycles from 11 to 20, satisfying 5*10+5*10=100, which can complete the current operation task within the total time of the first sleep parameter, and the sleep time of the operation units is distributed more evenly in the spatial dimension.

[0031] In actual use, the specific adjustment principles are not limited to the above adjustment principles, and can be set according to actual needs.

[0032] In step 304, the computing unit is controlled to execute a computing instruction based on the second pause parameter, and a processing result corresponding to the feature map to be processed is obtained.

[0033] Here, after obtaining the second sleep parameter of the arithmetic unit, the arithmetic unit is controlled so that the arithmetic unit executes a calculation instruction based on the second sleep parameter, and a processing result corresponding to the feature map to be processed is obtained.

[0034] In some alternative embodiments, for the second sleep parameter obtained by equalizing in the time dimension, the second sleep parameter of at least one of the computing units includes at least two non-consecutive sleep periods, thereby enabling the computation activation and sleep of each computing unit to be controlled based on the second sleep parameter, thereby realizing equalization of the power consumption distribution in the time dimension and improving the equality of the power consumption distribution across the computing units.

[0035] In some alternative embodiments, for the second sleep parameters obtained by equalizing in the spatial dimension, at least one of the computing units is divided into multiple computing subunits in space, and the second sleep parameters of the computing unit include second sleep parameters corresponding to each computing subunit, thereby controlling the multiple computing subunits to sleep alternately based on the second sleep parameters corresponding to each computing subunit of the computing unit, thereby achieving equalization of the power consumption distribution in the spatial dimension and improving the uniformity of the power consumption distribution across the computing units.

[0036] In some alternative embodiments, a second sleep parameter is obtained by integrating and equalizing the time dimension and the space dimension, and the activation and sleep of the computing unit and multiple computing subunits of at least one computing unit can be controlled in the time dimension and the space dimension based on the second sleep parameter, thereby realizing equalization of the power consumption distribution in the time dimension and the space dimension and improving the equality of the power consumption distribution across the computing units.

[0037] In this embodiment, a method for processing a neural network feature map using a neural network processor includes determining operation instructions for calculating the target feature map when processing the neural network feature map using the neural network processor, determining sleep state information for each operation unit in the neural network processor to execute the operation instructions, adjusting first sleep parameters in the sleep state information for each operation unit to determine adjusted second sleep parameters, and controlling the operation units to execute their respective operation instructions based on the second sleep parameters to obtain a processing result corresponding to the target feature map. The second sleep parameters are obtained by adjusting the first sleep parameters, and the second sleep parameters can more uniformly distribute the time and / or space during which the operation units pause, thereby reducing the width and duration of sudden changes in power consumption each time and more uniformly distributing the sleep times across all operation units of the neural network processor, thereby improving the uniformity of the power consumption distribution of the neural network processor and helping to mitigate or resolve power supply integrity issues.

[0038] FIG. 3 is a flowchart of a method for processing feature maps of a neural network by a neural network processor according to another exemplary embodiment of the present disclosure.

[0039] In some alternative embodiments, as shown in FIG. 3, step 303 of adjusting first pause parameters for the computing unit to execute the computing instruction based on pause state information and determining adjusted second pause parameters includes steps 3031 and 3032.

[0040] In step 3031, determine a first computation activation period and a first sleep period for the computing unit to execute the computation instruction based on the sleep state information.

[0041] Here, the pause state information may include a first operation activation period and a first pause period for the operation unit to execute an operation instruction, and therefore, based on the pause state information, the first operation activation period and the first pause period for the operation unit to execute an operation instruction may be determined.

[0042] In step 3032, the first computation activation period and the first rest period are adjusted according to at least one of the time dimension and the space dimension to obtain an adjusted second computation activation period and a second rest period, which are set as second rest parameters.

[0043] Here, the time dimension refers to keeping the computing unit constant and adjusting the distribution of the idle periods of the computing unit. The space dimension refers to dividing the computing unit into multiple computing subunits and adjusting the idle periods of the computing subunits. For details, please refer to the above embodiment, and redundant explanations will be omitted here.

[0044] In some alternative embodiments, for a plurality of computing units, the first computation activation period and the first sleep period of an adjustable target computing unit among the plurality of computing units can be adjusted according to at least one of a time dimension and a space dimension to obtain an adjusted second computation activation period and a second sleep period of the target computing unit, which can be set as the second sleep parameter of the target computing unit. The number of target computing units can be one or more. For a computing unit among the plurality of computing units that does not need to be adjusted, the first computation activation period and the first sleep period of the computing unit can be maintained, or the first computation activation period and the first sleep period of the computing unit can be set as the second computation activation period and the second sleep period of the computing unit.

[0045] This embodiment adjusts the first computation activation period and the first idle period according to at least one of the time dimension and the space dimension, so that the adjusted second computation activation period and the second idle period are evenly distributed among the idle periods in at least one of the time dimension and the space dimension, which helps to improve the evenness of the power consumption distribution across the computing units, thereby mitigating power supply integrity issues.

[0046] In some alternative embodiments, step 3032 of adjusting the first computation activation period and the first idle period according to at least one of the time dimension and the space dimension to obtain adjusted second computation activation period and second idle period includes adjusting the first computation activation period and the first idle period into at least two idle sub-periods and at least one computation activation sub-period sorted in a predetermined manner in the time dimension, and setting each sorted computation activation sub-period as a second computation activation period, and setting each sorted idle sub-period as a second idle period.

[0047] Here, the preset manner may be any feasible manner. For example, the preset manner may require that at least two sleep subperiods are sorted non-consecutively and that at least one computation active period be included between any two sleep subperiods. Exemplarily, the two sleep subperiods and the two computation active subperiods are sorted in the manner of sleep subperiod → computation active subperiod → sleep subperiod → computation active subperiod, or in the manner of computation active subperiod → sleep subperiod → computation active subperiod → sleep subperiod, and the adjusted second sleep parameter includes two second computation active subperiods and two second sleep periods sorted in the preset manner. When the second sleep parameter is for controlling the operation of a computing unit, the computing unit alternates between computation active and sleep in the preset manner, thereby improving the uniformity of the power consumption distribution of the computing unit.

[0048] 4 is a schematic diagram of power consumption distribution after time dimension adjustment according to an exemplary embodiment of the present disclosure. As shown in FIG. 4, before adjustment, the idle periods of the computing units are concentrated, the range of power consumption is relatively large, and the duration is relatively long. After adjustment in the time dimension (after adjustment), the idle periods of the computing units are dispersed and the idle periods are evenly distributed in the time dimension, which does not degrade the performance of the entire computing units, reduces the range and duration of change in power consumption each time, effectively improves the evenness of power consumption distribution, and thereby helps alleviate power supply integrity issues.

[0049] This embodiment adjusts the first computing activation period and the first sleep period into at least two sleep sub-periods and at least one computing activation sub-period sorted in a predetermined manner in the time dimension, thereby obtaining adjusted second sleep parameters for the computing unit, thereby realizing a more even distribution of the sleep times of the computing units in the time dimension, and helping to improve the equalization of the power consumption distribution across the computing units.

[0050] FIG. 5 is a flowchart of a method for processing feature maps of a neural network by a neural network processor according to yet another exemplary embodiment of the present disclosure.

[0051] In some alternative embodiments, as shown in FIG. 5, step 3032 of adjusting a first computation activation period and a first rest period according to at least one of a time dimension and a space dimension to obtain an adjusted second computation activation period and a second rest period includes steps 30321 to 30323.

[0052] In step 30321, a first grouping manner of the computing units is determined based on the first computing activation period and the first sleep period.

[0053] Here, the first grouping scheme may include a grouping number of the computing units and a division scheme of the computing subunits. The grouping number of the computing units is a first number. Different grouping schemes may correspond to different first numbers. Specifically, the first grouping scheme of the computing units may be determined based on the time relationship between the first computing activation period and the first sleep period, or may be determined according to the time ratio between the first computing activation period and the first sleep period of the computing unit to be adjusted.

[0054] In some alternative embodiments, the specific manner of determining the first grouping manner of the computing units based on the first computing active period and the first idle period can be any feasible manner, for example, the total time of the first computing active period and the first idle period can be divided into at least two periods, in each period, some computing subunits of the first number of computing subunits are in a computing active state and some other computing subunits are in an idle state, and second computing active periods and second idle periods corresponding to each computing subunit are obtained, thereby achieving equalization of power consumption distribution in the space dimension when satisfying computing requirements.

[0055] In some alternative embodiments, for the plurality of computing units, a target computing unit to be adjusted can be determined from the plurality of computing units based on a first computing activation time and a first pause time, and a first grouping method for the target computing unit can be determined, and computing units other than the target computing unit among the plurality of computing units are not grouped.

[0056] In step 30322, the computing unit is divided into a first number of computing sub-units according to a first grouping scheme.

[0057] Here, the first grouping scheme includes the number of groupings of the arithmetic units and a scheme for dividing the arithmetic subunits. Therefore, the first number is determined based on the number of groupings, and the arithmetic units are divided into the first number of arithmetic subunits based on the scheme for dividing the arithmetic subunits. The scheme for dividing the arithmetic subunits may include the number of calculation arrays included in each arithmetic subunit. For example, a arithmetic unit includes 10 sum-of-products arrays and is divided into two arithmetic subunits, each of which includes five sum-of-products arrays. Alternatively, one arithmetic subunit includes a relatively small number of sum-of-products arrays (e.g., four sum-of-products arrays) and the other arithmetic subunit includes a relatively large number of sum-of-products arrays (e.g., six sum-of-products arrays). The specific scheme for dividing the arithmetic subunits can be determined based on actual computing performance demands and power consumption uniformity requirements.

[0058] In some alternative embodiments, the target computing units to be adjusted are divided into a first number of computing subunits based on a first grouping scheme of the target computing units, and other computing units may not be grouped.

[0059] In step 30323, a second computation activation period and a second sleep period corresponding to each computing subunit are determined based on the first computation activation period and the first sleep period.

[0060] In some alternative embodiments, the grouping of the computing units can be determined based on the time relationship between the first computing activation period and the first sleep period. For example, if the first computing activation period and the first sleep period are the same in duration, the first number can be any integer greater than 1, and the determined second computing activation period and second sleep period corresponding to each computing subunit can be parameters for alternatingly sleeping each computing subunit, or parameters for grouping the computing subunits and alternatingly sleeping each computing subunit. For example, if the first number is 2, the two computing subunits sleep alternately. That is, during the first computing activation period, one of the two computing subunits (e.g., referred to as the first computing subunit) activates a computation, and the other computing subunit (e.g., referred to as the second computing subunit) sleeps, and during the first sleep period, the first computing subunit sleeps, and the second computing subunit activates a computation. For the first computing subunit, the second sleep parameter includes a second computation activation period and a second sleep period arranged in sequence. For the second computing subunit, the second sleep parameter includes a second sleep period and a second computation activation period arranged in sequence. For example, if the first number is three, the computing unit can be divided into a first computing subunit, a second computing subunit, and a third computing subunit, and these three computing subunits can sleep in turn. That is, the total time of the first computation activation period and the first sleep period can be divided into three periods, for example, period 1, period 2, and period 3, and the three computing subunits can each sleep for three periods. That is, one computing subunit sleeps in each period. For example, in period 1, the first computing subunit is asleep, and the second computing subunit and the third computing subunit activate operations; in period 2, the second computing subunit is asleep, and the first computing subunit and the third computing subunit activate operations; and in period 3, the third computing subunit is asleep, and the first computing subunit and the second computing subunit activate operations.The specific order of the computing subunits is not limited. For example, if the first number is five, the total time of the first computing activation period and the first sleep period can be divided into at least two periods, and the five computing subunits can be alternately sleep. For example, in the first period, three computing subunits activate their operations and the remaining two computing subunits sleep. In the second period, the remaining two computing subunits activate their operations and the three computing subunits sleep. If the duration of the first computing activation period is longer than the duration of the first sleep period, the total time of the first computing activation period and the first sleep period can be divided into at least two periods, and in each period, a relatively large number of computing subunits in the computing unit can be in a sleep state and a relatively small number of computing subunits can be in a sleep state. If the duration of the first computing activation period is shorter than the duration of the first sleep period, the total duration of the first computing activation period and the first sleep period can be divided into at least two periods, and in each period, a relatively small number of computing subunits in the computing unit are in a computing activation state, and a relatively large number of computing subunits are in a sleep state. The specific manner of determining the first number and the second sleep parameter of the computing subunits is not limited to the above manner, and any other feasible manner may be used as long as it can ensure that the computing unit completes the computation within the total duration of the first computing activation period and the first sleep period.

[0061] For example, a first grouping method for the computing units can be determined according to the time ratio between the first active period and the first idle period of the computing unit to be adjusted, so that the idle periods of the computing units can be evenly distributed when the computing units meet the computing requirements. For example, if computing unit A is the computing unit to be adjusted, the duration of the first active period of computing unit A is 60 calculation cycles, the duration of the first idle period is 40 calculation cycles, and the purpose of the idle period is that computing unit A needs to synchronize with computing unit B, and the duration of the first active period of computing unit B is 100 calculation cycles, then computing unit A only needs to complete the calculation within 100 calculation cycles. If the time ratio between the first calculation activation period and the first sleep period is 3:2, the calculation unit A can be divided into five calculation subunits (or the calculation unit A can be divided into two uneven calculation subunits), and the total time of the first calculation activation period and the first sleep period can be divided into at least two periods, for example, two periods, or the first calculation activation period and the first sleep period can be two periods each, and in each period, at least three calculation subunits activate calculations and the remaining calculation subunits sleep.

[0062] In this embodiment, by grouping the computing units into a first number of computing subunits, a second computing activation period and a second sleep period corresponding to each computing subunit are determined based on the first computing activation period and the first sleep period, so that in each calculation cycle, some computing subunits are in a computing activation state and some other computing subunits are in a sleep state, thereby realizing equalization of the power consumption distribution in the spatial dimension and helping to improve the equalization of the power consumption distribution across the computing units.

[0063] FIG. 6 is a flowchart of a method for processing feature maps of a neural network by a neural network processor according to yet another exemplary embodiment of the present disclosure.

[0064] In some alternative embodiments, as shown in FIG. 6, step 30323 of determining a second operation activation period and a second idle period corresponding to each operation subunit based on the first operation activation period and the first idle period includes steps 303231 and 303232.

[0065] In step 303231, the first computation activation period is alternately made the second computation activation period of each computing subunit.

[0066] In some alternative embodiments, if the computational demand can be met, the first computation activation period can alternately become the second computation activation period of each computing subunit. For example, the first computation activation period includes 20 computation cycles, the first sleep period includes 40 computation cycles, and three computing subunits alternately use the first computation activation period as the second computation activation period of each computing subunit, i.e., each computing subunit alternately activates a computation for the duration of the first computation activation period. That is, the total time of the first operation activation period and the first sleep period is divided into three periods, and in the first period, operation subunit 1 activates operation, and operation subunit 2 and operation subunit 3 sleep, in the second period, operation subunit 2 activates operation, and operation subunit 1 and operation subunit 3 sleep, and in the third period, operation subunit 3 activates operation, and operation subunit 1 and operation subunit 2 sleep, thereby realizing alternating operation activation of each operation subunit.

[0067] In some alternative embodiments, the first number can be determined according to a proportional relationship between the duration of the first computation activation period and the duration of the first sleep period, so that when the first computation activation period alternates as the second computation activation period of each computing subunit, the computation request can be met and the time required for the computing unit to complete the computation does not exceed the total duration of the first computation activation period and the first sleep period. For example, if the ratio between the duration of the first computation activation period and the duration of the first sleep period is 1:2, the first number can be set to 3, so that the total duration of the first computation activation period and the first sleep period includes three first computation activation periods, so that the first computation activation period alternates as the second computation activation period of each computing subunit.

[0068] In some alternative embodiments, the second calculation activation period in which each calculation subunit activates calculations in turn can be further determined based on the total time of the first calculation activation period and the first idle period, thereby better matching the first number and improving the distribution uniformity of the idle time under the premise of ensuring calculation performance.

[0069] In step 303232, a second sleep period of each computing subunit is determined based on the second computing activation period of each computing subunit.

[0070] Here, for any computing subunit, after the second computing activation period of the computing subunit is determined, the period in the total time other than the second computing activation period is the second sleep period. For example, if the total time is 0 to t, and 0 to t is divided into three periods, 0 to t1, t1 to t2, and t2 to t3, and t1 to t2 is the second computing activation period of the computing subunit, the computing subunit sleeps in the periods 0 to t1 and t1 to t2, i.e., the periods 0 to t1 and t1 to t2 are the second sleep periods of the computing subunit.

[0071] This embodiment realizes alternating computation activation of each computation subunit by alternating the first computation activation period with the second computation activation period of each computation subunit, thereby making the idle time distribution of each computation subunit more even and helping to improve the evenness of the power consumption distribution of the entire computation unit.

[0072] FIG. 7 is a flowchart of a method for processing feature maps of a neural network by a neural network processor according to yet another exemplary embodiment of the present disclosure.

[0073] In some alternative embodiments, as shown in FIG. 7, step 30323 of determining a second computation activation period and a second sleep period corresponding to each computation subunit based on the first computation activation period and the first sleep period includes steps 30323a and 30323b.

[0074] In step 30323a, the first sleep period is alternately made into the second sleep period for each computing sub-unit.

[0075] In some alternative embodiments, if the computational demand can be met, the first sleep period can be alternately used as the second sleep period for each computing subunit, allowing each computing subunit to take turns pausing. For example, the first computation activation period includes 40 computation cycles, the first sleep period includes 20 computation cycles, and three computing subunits alternately use the first sleep period as the second computation activation period for each computing subunit, allowing each computing subunit to take turns pausing for 20 computation cycles, thereby achieving an even distribution of sleep times.

[0076] In some alternative embodiments, the second sleep period in which each computing subunit alternately sleeps can be further determined based on the total time of the first computing activation period and the first sleep period, thereby better matching the first number and improving the distribution uniformity of the sleep time under the premise of ensuring computing performance.

[0077] In step 30323b, a second computing activation period of each computing subunit is determined based on the second sleep period of each computing subunit.

[0078] Here, for any computing subunit, after the second sleep period of the computing subunit is determined, the period other than the second sleep period in the total time is the second computing activation period.

[0079] This embodiment realizes alternating pauses for each calculation subunit by alternately using the first pause period as the second pause period for each calculation subunit, thereby making the pause time distribution for each calculation subunit more even and helping to improve the evenness of the power consumption distribution across the calculation units.

[0080] In some alternative embodiments, FIG. 8 is a schematic diagram illustrating the principle of spatial dimension adjustment according to an exemplary embodiment of the present disclosure. As shown in FIG. 8, before adjustment, all computing units activate operations during a first computing activation period and all suspend operations during a first suspend period, resulting in a concentrated spatial utilization distribution of the computing units and low uniformity in the overall power consumption distribution. After adjustment, some computing subunits of the computing units activate operations during a first period, while other computing subunits suspend operations. During a second period, some computing subunits transition from an active state to a suspend state, while other computing subunits transition from a suspend state to an active state, thereby reducing the spatial utilization of the computing units during the same period and helping to improve the uniformity of the overall power consumption distribution of the computing units. FIG. 9 is a schematic diagram illustrating the power consumption distribution according to an exemplary embodiment of the present disclosure. As shown in FIG. 9, the spatial dimension adjustment is a schematic diagram illustrating the power consumption distribution corresponding to FIG. 8. By suspending some computing resources during the same period through spatial dimension adjustment, the average power consumption of the computing units during operation is reduced, helping to improve the uniformity of the power consumption distribution.

[0081] FIG. 10 is a flowchart of a method for processing feature maps of a neural network by a neural network processor according to yet another exemplary embodiment of the present disclosure.

[0082] In some alternative embodiments, as shown in FIG. 10 , step 3032 of adjusting a first computation activation period and a first rest period according to at least one of a time dimension and a space dimension to obtain an adjusted second computation activation period and a second rest period includes steps 3032A to 3032C.

[0083] In step 3032A, a second grouping manner of the computing units and an adjustment manner in the time dimension are determined based on the first computing activation period and the first sleep period.

[0084] Here, the first operation activation period and the first sleep period of the operation unit to be adjusted can be adjusted by integrating the time dimension and the space dimension, thereby more effectively improving the uniformity of the power consumption distribution of the operation units. The second grouping method corresponds to the space dimension, and the adjustment method in the time dimension corresponds to the time dimension.

[0085] In some alternative embodiments, the second grouping scheme may include a grouping number of the arithmetic units and a division scheme of the arithmetic subunits. The grouping number of the arithmetic units is also referred to as the second number. The second grouping scheme may include two types of grouping schemes: uniform grouping and non-uniform grouping. Uniform grouping refers to uniformly dividing the computation arrays in the arithmetic unit into a second number of groups to obtain computation subunits with the same second number of computation capabilities. Non-uniform grouping refers to obtaining computation subunits with different second number of computation capabilities, where the number of computation arrays included in each divided computation subunit may not be the same. For example, a computation unit may include 10 sum-of-products arrays, and the computation unit may be uniformly divided into two computation subunits, each of which includes five sum-of-products arrays. Alternatively, the computation unit may be unevenly divided into two computation subunits, one of which includes six sum-of-products arrays and the other of which includes four sum-of-products arrays.

[0086] The determining principle of the second grouping method and the time dimension adjustment method is to make the sleep times of the computing units more equal in the time dimension and the space dimension under the premise of ensuring the computing performance of the computing units. For example, the time dimension adjustment method can include dividing the sleep time of each computing subunit into at least two sleep subperiods, and for each computing subunit, the at least two sleep subperiods of the computing subunit are non-contiguous in the total time between the first computing activation period and the first sleep period. The sleep subperiods of different computing subunits are different in the total time, thereby achieving double equalization in the time dimension and the space dimension.

[0087] In step 3032B, the computing unit is divided into a second number of computing sub-units based on a second grouping scheme.

[0088] Here, after determining the second grouping scheme, the computing unit can be divided into a second number of computing subunits according to the second grouping scheme, and different second grouping schemes can correspond to different second numbers.

[0089] In step 3032C, a second computing activation period and a second sleep period corresponding to each computing subunit are determined based on the adjustment manner in the time dimension.

[0090] In some alternative embodiments, after determining the adjustment manner in the time dimension, the computation activation period and the sleep period of each computing subunit can be adjusted according to the adjustment manner in the time dimension, and the original sleep parameters of each computing subunit are consistent with the computing unit to which the computing subunit belongs, i.e., both are the first computation activation period and the first sleep period.

[0091] In some alternative examples, the first computation activation period includes 20 computation cycles, the first sleep period includes 40 computation cycles, the computation unit can be divided into at least two computation subunits, and second computation activation periods are determined according to computation demands, with the second sleep period in the 60 computation cycles of each computation subunit being evenly distributed. Furthermore, the second sleep period in the 60 computation cycles of each computation subunit is determined based on the second computation activation period of each computation subunit. For example, the computation unit can be divided into two computation subunits, each of which needs to activate its computation for 20 computation cycles and sleep for 40 computation cycles. The computation periods in which the two computation subunits need to activate their computations can be evenly distributed in the 60 computation cycles to obtain the second computation activation period corresponding to each computation subunit. Alternatively, the computation periods in which the two computation subunits sleep can be evenly distributed in the 60 computation cycles to obtain the second sleep period corresponding to each computation subunit. For example, calculation subunit 1 activates calculations in calculation cycles 1 to 5, 16 to 20, 31 to 35, and 46 to 50, and is inactive during other periods, while calculation subunit 2 activates calculations in calculation cycles 6 to 10, 21 to 25, 36 to 40, and 51 to 55, and is inactive during other periods. These are merely exemplary distribution methods, and the specific distribution methods are not limited to the above forms.

[0092] By determining a second grouping method of the computing units and an adjustment method in the time dimension, this embodiment adjusts the first computing activation period and the first sleep period of the computing units in the time dimension and the space dimension in an integrated manner, thereby obtaining doubly equalized second computing activation period and second sleep period, which helps to further improve the equality of the power consumption distribution of the entire computing units.

[0093] FIG. 11 is a flowchart of a method for processing feature maps of a neural network by a neural network processor according to yet another exemplary embodiment of the present disclosure.

[0094] In some alternative embodiments, as shown in FIG. 11 , step 304 of controlling the arithmetic unit to execute an arithmetic instruction based on a second pause parameter to obtain a processing result corresponding to the feature map to be processed includes steps 3041 and 3042.

[0095] In step 3041, generate a corresponding control signal based on the second pause parameter to control the arithmetic unit to execute the arithmetic instruction.

[0096] Here, the control signal is a signal that controls the arithmetic unit to enter an operating state and perform operation activation or pause. For example, the control signal can include an enable signal. The enable signal can include two signals: an enable signal (e.g., a high-level signal or 1) that indicates that the arithmetic unit enters an operating state and performs operation activation, and an enable signal (e.g., a low-level signal or 0) that indicates that the arithmetic unit does not enter an operating state. Alternatively, the control signal can include an enable signal and a pause signal. In this case, the enable signal is for controlling whether the arithmetic unit enters an operating state, and the pause signal is for controlling the arithmetic unit to perform operation activation (e.g., the pause signal is 0 or low level) or pause (e.g., the pause signal is 1 or high level). The specific implementation of the control signal is not limited.

[0097] In some alternative embodiments, a specific operating state (whether it is operating or not) of the operating unit at the current time (or the current calculation cycle) can be determined based on the second calculation activation period and the second sleep period of the operating unit included in the second sleep parameter, and a control signal corresponding to the operating unit can be generated based on the specific operating state of the operating unit. For example, if the operating unit is in the calculation activation state at the current calculation cycle, the generated control signal is a signal that causes the operating unit to enter the calculation activation state, for example, the enable signal is 1, or the enable signal is 1 and the sleep signal is 0.

[0098] In step 3042, a control signal is sent to the arithmetic unit to obtain the arithmetic result of the arithmetic unit, and the processing result corresponding to the feature map to be processed is obtained.

[0099] Here, the arithmetic unit activates or pauses its operation in response to a control signal. For example, when the arithmetic unit activates its operation, the calculation array in the arithmetic unit calculates an input feature value in response to an operation instruction. For example, a multiply-and-accumulate array multiplies the input feature value by a weight, and the calculation results of each calculation array constitute the arithmetic unit's calculation result. The controller acquires the calculation results of each arithmetic unit and, based on the calculation results of each arithmetic unit, obtains a processing result corresponding to the feature map to be processed. For example, a processing result of performing a convolution operation on the feature map to be processed is obtained.

[0100] In some alternative embodiments, after executing an arithmetic instruction and obtaining an arithmetic result, the arithmetic unit can return the arithmetic result to the controller, or can store the arithmetic result, for example, in a register or a designated storage space in memory, and the controller can obtain the arithmetic result of each arithmetic unit from the register or the designated storage space.

[0101] This embodiment generates a control signal for the computing unit based on the adjusted second sleep parameter, and controls the computing unit to activate and sleep based on the adjusted second sleep parameter, thereby realizing an even distribution of sleep time and improving the evenness of power consumption distribution during the computing process, thereby helping to alleviate power supply integrity issues.

[0102] In some alternative embodiments, step 3041 of generating a corresponding control signal for controlling the arithmetic unit to execute the arithmetic instruction based on the second pause parameter includes, for any arithmetic cycle of the arithmetic instruction, determining the operating state of the arithmetic unit in that arithmetic cycle based on the second pause parameter, and generating a control signal corresponding to the arithmetic unit based on the operating state of the arithmetic unit in that arithmetic cycle.

[0103] Here, the calculation period is the calculation period, and the second pause parameters include a second calculation activation period and a second pause period of the calculation unit, where each second calculation activation period can include one or more calculation periods. Each second pause period can include one or more calculation periods. For any calculation period, determining whether the calculation period belongs to the second calculation activation period or the second pause period can determine the operating state of the calculation unit in that calculation period. Each calculation unit performs calculations in a calculation period, and the specific calculation types performed by the calculation units in different periods are the same, such as multiplication and addition. However, the input feature data may be different. For example, a calculation unit needs to complete 10 multiplications, and the multiplier and multiplicand of each multiplication may be different. The second sleep parameters of the computing unit include a second computation activation period and a second sleep period of the computing unit, and the specific operating state of the computing unit in any computing period (called a current computing period) can be determined based on the second computation activation period and the second sleep period. For example, if the current computing period is within a second computation activation period of the computing unit, the operating state of the computing unit in the computing period is determined as a computation activation state. The generated control signal for the computing unit is a control signal that causes the computing unit to perform computation activation.

[0104] In this embodiment, in each calculation cycle, the operating state of the calculation unit in that calculation cycle is determined based on the second pause parameter, which facilitates accurate generation of the control signal for the calculation unit, realizes accurate and effective control of the calculation unit, and helps achieve the goal of equalizing power consumption distribution.

[0105] In some alternative embodiments, the step of generating a control signal corresponding to the arithmetic unit based on the operating state of the arithmetic unit in the arithmetic cycle includes: The method includes the steps of determining the enabled state and the paused state of the arithmetic unit based on the operating state of the arithmetic unit in the calculation cycle; generating an enable signal indicating enable in accordance with the enable state of the arithmetic unit being enabled, where the enable signal indicating enable enables the arithmetic unit to perform calculations when the paused state is non-pause; generating an enable signal indicating disenable in accordance with the enable state of the arithmetic unit being disabled; generating a pause signal indicating pause in accordance with the paused state of the arithmetic unit in the calculation cycle being paused, where the pause signal causes the arithmetic unit to pause calculations; and generating a pause signal indicating non-pause in accordance with the paused state of the arithmetic unit in the calculation cycle being non-pause.

[0106] Here, the enable state indicates whether the computing unit enters an operating state, and can include two states: enable and disable. The pause state indicates whether the computing unit pauses its operation. The pause state can include two states: pause and not pause. When the enable state of a computing unit is enable, it indicates that the computing unit is in an operating state and computation activation is possible. When the enable state of a computing unit is disable, it indicates that the computing unit is in an idle state and computation activation is not possible. When the pause state of a computing unit is pause, it indicates that the computing unit is in a computation pause state. When the pause state of a computing unit is not paused, it indicates that the computing unit is in a computation activation state. However, it is necessary that the computing unit can enter the computation activation state only when the enable state of a computing unit is enable and the pause state is not paused, thereby helping to realize active pausing of the computing unit through the pause state. The enable signal indicating enable can be implemented in any form. For example, the enable signal indicating enable may be a high-level signal or may have a signal state of 1. Conversely, an enable signal representing a disable state may be a low level signal, or the signal state may be 0. A pause signal representing a pause state may be a high level signal, or the signal state may be 1. A pause signal representing a non-pause state may be a low level signal, or the signal state may be 0.

[0107] In some alternative embodiments, FIG. 12 is a schematic diagram of the control principle of a computing unit according to an exemplary embodiment of the present disclosure. As shown in FIG. 12, the neural network processor may further include a memory 230, an input data channel 240, and a weight data channel 250. en represents an enable signal, and stall represents a stall signal. The diagram shows only the schematic diagram of the connection relationship of one computing unit 220; the other computing units 220 correspond to the connection relationship of the illustrated computing unit 220. That is, the controller 210 generates an enable signal and a stall signal for each computing unit 220 to control the computing unit's entry into an operating state, activation of operation, stalling, exiting from an operating state, etc. The input data channel 240 reads input feature data (FIN_DATA, e.g., a feature map to be processed) from the memory 230 in response to the enable signal from the controller 210 and provides the input feature data to the computing unit 220. The weight data channel 250 reads weight data (WT_DATA) from the memory 230 according to the enable signal of the controller 210 and provides the weight data to the calculation unit 220, so that the calculation unit 220 can calculate the input feature data and the weight data to obtain a calculation result according to the enable signal and the pause signal of the controller 210. Here, the calculation unit 220 enters a calculation activation state when en is valid (1 or high level) and stall is invalid (stall is 0 or low level). The calculation unit 220 enters a pause state when en is valid and stall is valid.

[0108] This embodiment realizes control over the state of the computing unit through the enable signal and the pause signal, and helps realize active pausing of the computing unit through the pause signal, on top of the passive pausing of the related art, thereby supporting the adjustment of the first pause parameter of the passive pausing, and actively controlling the pausing of the computing unit through the pause signal based on the adjusted second pause parameter, thereby achieving the purpose of improving the uniformity of power consumption distribution.

[0109] In some alternative embodiments, step 3041 of generating a corresponding control signal for controlling the arithmetic unit to execute the arithmetic instruction based on the second pause parameter includes, for any arithmetic cycle of the arithmetic instruction, determining the operating state of each arithmetic subunit in the arithmetic unit in that arithmetic cycle based on the second pause parameter, and generating a control signal corresponding to each arithmetic subunit according to the operating state of each arithmetic subunit.

[0110] Step 3042 of sending a control signal to the arithmetic unit and obtaining the arithmetic result of the arithmetic unit includes a step of sending a control signal corresponding to each arithmetic subunit to each arithmetic subunit and obtaining the arithmetic result corresponding to each arithmetic subunit.

[0111] Here, for the adjustment method in which spatial dimension adjustment is present, by generating a control signal corresponding to each calculation subunit in each calculation cycle for the calculation unit that divides the calculation subunit, each calculation subunit can be controlled to perform calculation activation and pause based on the adjusted second pause parameter, and the specific control principle is the same as the control of the above calculation unit, so duplicated explanations will be omitted here.

[0112] In this embodiment, the computing unit that adjusts the spatial dimension for each computing cycle determines the operating state of each computing subunit in that computing cycle based on the second sleep parameter of each computing subunit, and generates a control signal corresponding to each computing subunit based on the operating state, thereby realizing individual control of each computing subunit and distributing the sleep time of each computing subunit more evenly, thereby improving the evenness of the power consumption distribution of the computing units.

[0113] A method for processing feature maps of a neural network by a neural network processor according to an embodiment of the present disclosure dynamically adjusts the time utilization rate and space utilization rate of the processing units, thereby controlling the uniformity of the power consumption distribution of the processing units and mitigating the power supply integrity problem.

[0114] Each of the above-described embodiments of the present disclosure may be implemented alone or in any combination as long as there is no contradiction between them. Specific examples can be set according to actual needs, and are not limited by the present disclosure.

[0115] Any of the methods for processing a neural network feature map by a neural network processor according to embodiments of the present disclosure may be executed by any suitable device having data processing capabilities, including, but not limited to, a terminal device, a server, etc. Alternatively, any of the methods for processing a neural network feature map by a neural network processor according to embodiments of the present disclosure may be executed by a processor, for example, the processor executes any of the methods for processing a neural network feature map by a neural network processor mentioned in the embodiments of the present disclosure by invoking corresponding instructions stored in a memory. Hereinafter, redundant description will be omitted.

[0116] [Example Device] 13 is a schematic diagram of the structure of an apparatus for processing a feature map of a neural network by a neural network processor according to an exemplary embodiment of the present disclosure. The apparatus for processing a feature map of a neural network by a neural network processor of this embodiment is for realizing an embodiment of a method for processing a feature map of a neural network by a neural network processor according to the present disclosure. The apparatus shown in FIG. 13 includes a determination module 51, a first processing module 52, a second processing module 53, and a third processing module 54.

[0117] The determination module 51 determines the operation instructions for calculating the feature map to be processed.

[0118] The first processing module 52 determines the pause state information for the arithmetic units in the neural network processor to execute arithmetic instructions.

[0119] The second processing module 53 adjusts the first pause parameters for the arithmetic units to execute the arithmetic instructions based on the pause state information, and determines the adjusted second pause parameters.

[0120] The third processing module 54 controls the arithmetic unit to execute the arithmetic instruction based on the second pause parameter, and obtains a processing result corresponding to the feature map to be processed.

[0121] Each module of the apparatus according to the embodiment of the present disclosure can be realized by any of software, hardware, a combination of software and hardware, and the like.

[0122] FIG. 14 is a schematic diagram of the structure of a processing device for a feature map of a neural network by a neural network processor according to another exemplary embodiment of the present disclosure.

[0123] In some alternative embodiments, as shown in FIG. 14, the second processing module 53 includes a first processing unit 531 and a second processing unit 532.

[0124] The first processing unit 531 determines, based on the pause state information, a first operation activation period for the operation unit to execute an operation instruction and a first pause period.

[0125] The second processing unit 532 adjusts the first calculation activation period and the first rest period according to at least one of the time dimension and the space dimension to obtain adjusted second calculation activation period and second rest period as second rest parameters.

[0126] In some alternative embodiments, specifically, the second processing unit 532 adjusts the first computation activation period and the first idle period into at least two idle sub-periods and at least one computation activation sub-period sorted in a predetermined manner in the time dimension, and each sorted computation activation sub-period is a second computation activation period, and each sorted idle sub-period is a second idle period.

[0127] In some alternative embodiments, specifically, the second processing unit 532 determines a first grouping scheme of the computing units based on the first computing activation period and the first idle period, divides the computing units into a first number of computing subunits according to the first grouping scheme, and determines second computing activation periods and second idle periods corresponding to each computing subunit, respectively, based on the first computing activation period and the first idle period. In some alternative embodiments, specifically, the second processing unit 532 alternately sets the first computation activation period as the second computation activation period of each computation subunit, and determines the second sleep period of each computation subunit based on the second computation activation period of each computation subunit.

[0128] In some alternative embodiments, specifically, the second processing unit 532 alternately sets the first idle period as the second idle period for each computing subunit, and determines the second computing activation period for each computing subunit based on the second idle period for each computing subunit.

[0129] In some alternative embodiments, specifically, the second processing unit 532 determines a second grouping scheme of the computing units and an adjustment scheme in the time dimension based on the first computing activation period and the first sleep period, divides the computing units into a second number of computing subunits based on the second grouping scheme, and determines second computing activation periods and second sleep periods corresponding to each computing subunit respectively based on the adjustment scheme in the time dimension.

[0130] FIG. 15 is a schematic diagram of the structure of a processing device for a feature map of a neural network by a neural network processor according to yet another exemplary embodiment of the present disclosure.

[0131] In some alternative embodiments, as shown in FIG. 15, the third processing module 54 includes a third processing unit 541 and a control unit 542.

[0132] The third processing unit 541 generates, based on the second pause parameter, a corresponding control signal for controlling the arithmetic unit to execute the arithmetic instruction.

[0133] The control unit 542 transmits a control signal to the arithmetic unit to obtain the arithmetic result of the arithmetic unit, and obtains the processing result corresponding to the feature map to be processed.

[0134] In some alternative embodiments, specifically, for any operation cycle of the operation instruction, the third processing unit 541 determines the operating state of the operation unit in the operation cycle based on the second pause parameter, and generates a control signal corresponding to the operation unit based on the operating state of the operation unit in the operation cycle.

[0135] In some alternative embodiments, specifically, the third processing unit 541 determines the enable state and the pause state of the computing unit based on the operating state of the computing unit in the computing cycle, generates an enable signal representing enable when the enable state of the computing unit is enabled, generates an enable signal representing disable when the enable state of the computing unit is disabled, generates a pause signal when the pause state of the computing unit in the computing cycle is paused, and generates a pause signal representing non-pause when the pause state of the computing unit in the computing cycle is non-pause, the enable signal representing enable allows the computing unit to perform computation when the pause state is non-pause, and the pause signal causes the computing unit to pause computation.

[0136] In some alternative embodiments, specifically, for any operation cycle of an operation instruction, the third processing unit 541 determines the operating state of each operation subunit in the operation unit during that operation cycle based on the second pause parameter, and generates a control signal corresponding to each operation subunit according to the operating state of each operation subunit.

[0137] Specifically, the control unit 542 transmits a control signal corresponding to each calculation subunit to each calculation subunit, obtains a calculation result corresponding to each calculation subunit, and obtains a processing result corresponding to the feature map to be processed.

[0138] The beneficial technical effects corresponding to the exemplary embodiments of the present device may refer to the beneficial technical effects corresponding to the exemplary methods described above, and redundant descriptions will be omitted here.

[0139] [Example electronic devices] FIG. 16 is a structural diagram of an electronic device including at least one processor 11 and a memory 12 according to an embodiment of the present disclosure.

[0140] The processor 11 may be a central processing unit (CPU) or other type of processing unit having data processing and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0141] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, and flash memory. The computer-readable storage media may store one or more computer program instructions, and the processor 11 may execute the one or more computer program instructions to implement the methods and / or other desired functions described above in accordance with the embodiments of the present disclosure.

[0142] By way of example, electronic device 10 may further include input devices 13 and output devices 14 connected to one another via a bus system and / or other form of connection (not shown).

[0143] The input device 13 may further include, for example, a keyboard and a mouse.

[0144] The output device 14 can output various types of information to the outside, and can include, for example, a display, a speaker, a printer, a communication network, and a remote output device connected thereto.

[0145] 16 shows only some of the components related to the present disclosure in the electronic device 10, and omits components such as buses, input / output interfaces, etc. The electronic device 10 may further include any other appropriate components depending on the specific application.

[0146] Furthermore, an embodiment of the present disclosure further provides an electronic device including a processing device for a neural network feature map by the neural network processor according to any of the above embodiments, or including a neural network processor according to any of the above embodiments.

[0147] Exemplary Computer Program Products and Computer-Readable Storage Media In addition to the methods and apparatus described above, embodiments of the present disclosure may provide a computer program product including computer program instructions that, when executed by a processor, cause the processor to perform the steps of the methods of various embodiments of the present disclosure described in the "Exemplary Methods" section above.

[0148] The computer program product may be written with program code for carrying out operations of embodiments of the present disclosure in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and traditional procedural programming languages ​​such as "C" or similar programming languages. The program code may execute entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on the remote computing device or a server.

[0149] Additionally, embodiments of the present disclosure further provide a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform the steps of the methods of various embodiments of the present disclosure described in the "Exemplary Method" section above.

[0150] The computer-readable storage medium may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0151] Although the basic principles of the present disclosure have been described above with reference to specific embodiments, the benefits, advantages, effects, etc. mentioned in the present disclosure are merely illustrative and not limiting, and these benefits, advantages, effects, etc. are not necessarily possessed by each embodiment of the present disclosure. Furthermore, the specific details of the above disclosure are merely illustrative and easy-to-understand functions and are not limiting, and the above details do not necessarily limit the present disclosure to those realized by the above specific details.

[0152] Those skilled in the art can make various modifications and variations to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include these modifications and variations.

Claims

1. 1. A method of processing feature maps of a neural network by a neural network processor, comprising: determining a computation command for computing a feature map to be processed; determining pause state information for an arithmetic unit in the neural network processor to execute the arithmetic instruction; adjusting a first pause parameter for the execution of the operation instruction by the operation unit based on the pause state information, and determining an adjusted second pause parameter; controlling the arithmetic unit to execute the arithmetic instruction based on the second pause parameter, and obtaining a processing result corresponding to the feature map to be processed.

10. A method for processing a feature map of a neural network by a neural network processor, comprising:

2. The step of adjusting a first pause parameter for the execution of the operation instruction by the operation unit based on the pause state information and determining a second pause parameter after the adjustment, determining a first operational activation period and a first idle period for the operational unit to execute the operational instruction based on the idle state information; adjusting the first computational activation period and the first pause period according to at least one of a time dimension and a space dimension to obtain adjusted second computational activation period and second pause period, which are the second pause parameters; 2. The method of claim 1.

3. adjusting the first computational activation period and the first idle period according to at least one of a time dimension and a space dimension to obtain adjusted second computational activation period and second idle period; adjusting the first computation activation period and the first idle period into at least two idle sub-periods and at least one computation activation sub-period sorted in a predetermined manner in a time dimension, and determining each of the sorted computation activation sub-periods as the second computation activation period and each of the sorted idle sub-periods as the second idle period; 3. The method of claim 2.

4. adjusting the first computational activation period and the first idle period according to at least one of a time dimension and a space dimension to obtain adjusted second computational activation period and second idle period; determining a first grouping manner of the computing units according to the first computing activation period and the first sleep period; dividing the arithmetic unit into a first number of arithmetic subunits according to the first grouping scheme; determining a second computational activation period and a second sleep period corresponding to each of the computing subunits based on the first computational activation period and the first sleep period, respectively; 3. The method of claim 2.

5. determining a second computational activation period and a second sleep period corresponding to each of the computing subunits based on the first computational activation period and the first sleep period, respectively; rotating the first computational activation period into a second computational activation period for each of the computing subunits; determining a second sleep period for each of the computing subunits based on the second computational activation period for each of the computing subunits; 5. The method according to claim 4.

6. determining a second computational activation period and a second sleep period corresponding to each of the computing subunits based on the first computational activation period and the first sleep period, respectively; rotating the first idle period as a second idle period for each of the computing subunits; determining a second computational activation period for each of the computing subunits based on the second sleep period for each of the computing subunits; 5. The method according to claim 4.

7. adjusting the first computational activation period and the first idle period according to at least one of a time dimension and a space dimension to obtain adjusted second computational activation period and second idle period; determining a second grouping manner and a time dimension adjustment manner of the computing units according to the first computing activation period and the first sleep period; dividing the computing units into a second number of computing sub-units based on the second grouping scheme; determining second operational activation periods and second idle periods corresponding to each of the operational subunits according to the time dimension adjustment method; 3. The method of claim 2.

8. the step of controlling the arithmetic unit to execute the arithmetic instruction based on the second pause parameter, and obtaining a processing result corresponding to the feature map to be processed, generating a corresponding control signal based on the second pause parameter for controlling the arithmetic unit to execute the arithmetic instruction; transmitting the control signal to the arithmetic unit, acquiring an arithmetic result from the arithmetic unit, and acquiring a processing result corresponding to the feature map to be processed; 3. The processing method according to claim 1, wherein the processing method is a method for processing a substrate.

9. generating a corresponding control signal for controlling the arithmetic unit to execute the arithmetic instruction based on the second pause parameter, determining an operation state of the arithmetic unit in any one of the operation cycles of the operation instruction based on the second pause parameter; generating a control signal corresponding to the arithmetic unit based on the operating state of the arithmetic unit in the arithmetic cycle; 9. The method of claim 8.

10. The step of generating a control signal corresponding to the arithmetic unit based on the operation state of the arithmetic unit in the arithmetic cycle includes: determining an enabled state or a paused state of the arithmetic unit based on the operation state of the arithmetic unit in the arithmetic cycle; generating an enable signal representing enable in response to the enable state of the arithmetic unit being enabled, the enable signal representing enable allowing the arithmetic unit to perform an operation when the pause state is non-pause; generating an enable signal representing disabling in response to the enable state of the arithmetic unit being disabled; generating a pause signal indicating a pause in response to the pause state of the arithmetic unit in the relevant calculation cycle being paused, the pause signal indicating a pause causing the arithmetic unit to pause calculation; generating a pause signal indicating non-pause in response to the pause state of the arithmetic unit being non-pause in the calculation cycle; 10. The method of claim 9.

11. generating a corresponding control signal for controlling the arithmetic unit to execute the arithmetic instruction based on the second pause parameter, determining, for any one of the operation cycles of the operation instruction, the operation state of each operation subunit in the operation unit in the operation cycle based on the second pause parameter; generating a control signal corresponding to each of the computing subunits based on the operating state of each of the computing subunits; The step of transmitting the control signal to the arithmetic unit and obtaining the arithmetic result of the arithmetic unit includes: transmitting control signals corresponding to the respective operation subunits to the respective operation subunits to obtain operation results corresponding to the respective operation subunits; 9. The method of claim 8.

12. 1. An apparatus for processing a feature map of a neural network by a neural network processor, comprising: a determination module for determining an operation instruction for calculating a feature map to be processed; a first processing module for determining pause state information for an arithmetic unit in the neural network processor to execute the arithmetic instruction; a second processing module for adjusting a first pause parameter for the execution of the operation instruction by the operation unit based on the pause state information, and determining a second pause parameter after the adjustment; a third processing module for controlling the arithmetic unit to execute the arithmetic instruction based on the second pause parameter, and for obtaining a processing result corresponding to the feature map to be processed. A neural network processor for processing a feature map of a neural network, comprising:

13. 1. A computer-readable storage medium, comprising: The storage medium stores a computer program for executing the method for processing a feature map of a neural network by a neural network processor according to any one of claims 1. A computer-readable storage medium comprising:

14. a processor; a memory for storing instructions executable by the processor; The processor reads and executes the executable instructions from the memory to perform the method for processing neural network feature maps by a neural network processor according to claim 1. Or, 13. A neural network processor according to claim 12, comprising a device for processing neural network feature maps. An electronic device characterized by:

Images