Voltage frequency adjustment method and apparatus, neural network accelerator, and storage medium
By dynamically adjusting the voltage frequency of the neural network processor, the mismatch between the speed at which the computing engine accesses and computes data is achieved, thus improving performance and reducing power consumption.
Patent Information
- Application Number
- CN202210133672.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-07
- Publication Date
- 2026-01-09
- Estimated Expiration
- 2042-02-07
AI Technical Summary
In neural network accelerators, the application of a fixed voltage frequency causes a mismatch between the speed at which the computing engine accesses and computes data, resulting in poor performance and high power consumption.
By dynamically adjusting the voltage frequency of the neural network processor for each computation layer, high or low voltage frequencies are used to match the needs of computation and data transmission based on the differences in computation time and memory access time.
It improves the performance of neural network processors and reduces power consumption.
Smart Images
Figure CN116611484B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] Embodiments of the present application relate to the technical field of communication, in particular to a voltage frequency adjustment method and device, a neural network accelerator and a storage medium. BACKGROUND
[0002] In a neural network accelerator, a neural network processor generally includes a tensor processing engine, a vector processing engine, a data transfer unit, etc., and its storage system is cascaded by multiple memories according to memory size from small to large, and the access speed from fast to slow.
[0003] The neural network algorithm implemented by the neural network accelerator has a main calculation layer of a convolution layer, and different calculation layers need to process different sizes of feature image data in the calculation process, and the convolution kernel parameters of each calculation layer are also different. The data of each calculation layer can be stored in one or more memories, and the data access speeds of different calculation layers are different. However, for the neural network accelerator, a fixed voltage frequency is usually applied, which will cause the speed of the calculation engine accessing data and calculating data to be mismatched, the performance of the neural network processor to be poor, and the power consumption to be large. SUMMARY
[0004] Embodiments of the present application provide a voltage frequency adjustment method and device, a neural network accelerator and a storage medium, which improve the performance of the neural network processor and reduce the power consumption of the neural network processor.
[0005] The technical solution of the embodiments of the present application is as follows:
[0006] The embodiments of the present application provide a voltage frequency adjustment method applied to a neural network accelerator including a neural network processor and multiple memories, the multiple memories are cascaded according to memory size, and the memory with the smallest memory size is connected to a calculation engine in the neural network processor; the method includes:
[0007] For each calculation layer in a preset neural network, the size of the corresponding feature image and the size of the corresponding convolution kernel are used to determine the corresponding total calculation amount, and the time length for the calculation engine to complete the corresponding total calculation amount is estimated to determine the corresponding calculation time length;
[0008] For each calculation layer in the preset neural network, the storage amount of the corresponding feature image in each memory of the multiple memories and the transmission bandwidth of each memory of the multiple memories are used to estimate the time length for the corresponding feature image to be transmitted to the calculation engine to determine the corresponding memory access time length;
[0009] During the neural network processor sequentially executing each calculation layer in the preset neural network by using the calculation engine, a voltage frequency of the neural network processor is dynamically adjusted based on a calculation time length and a memory access time length of each calculation layer.
[0010] In the method, for each calculation layer in the preset neural network, a total calculation amount of the corresponding calculation layer is determined by using a size of a corresponding feature image and a size of a contained convolution kernel.
[0011] For each calculation layer in the preset neural network, a product of twice of the size of the corresponding feature image and the size of the contained convolution kernel is calculated to determine the total calculation amount.
[0012] In the method, for each calculation layer in the preset neural network, the size of the corresponding feature image is a product of an image width, an image height, and an image channel number, and the size of the contained convolution kernel is a product of a convolution kernel number and a convolution kernel plane size.
[0013] In the method, for each calculation layer in the preset neural network, a time length for the calculation engine to complete the total calculation amount is estimated to determine the calculation time length.
[0014] A working frequency of the calculation engine is obtained.
[0015] For each calculation layer in the preset neural network, a number of calculation units participating in calculation in the calculation engine is obtained to determine a corresponding unit number.
[0016] For each calculation layer in the preset neural network, a product of the unit number and the working frequency is calculated to determine a unit calculation amount, and a quotient of the total calculation amount and the unit calculation amount is calculated to obtain the calculation time length.
[0017] In the method, for each calculation layer in the preset neural network, the number of calculation units participating in calculation in the calculation engine is obtained to determine the corresponding unit number.
[0018] A total number of calculation units contained in the calculation engine is obtained.
[0019] For each calculation layer in the preset neural network, a utilization rate of a corresponding calculation unit in the calculation engine is obtained.
[0020] For each calculation layer in the preset neural network, a product of the total number of calculation units and the utilization rate of the corresponding calculation unit is calculated to determine the unit number.
[0021] In the method, for each calculation layer in the preset neural network, the storage amount of the corresponding feature image in each of the plurality of memories and the transmission bandwidth of each of the plurality of memories are used to estimate the time length for the corresponding feature image to be transmitted to the calculation engine, and the estimated time length is determined as the corresponding memory access time length.
[0022] For each calculation layer in the preset neural network, the storage amount of the corresponding feature image in each of the plurality of memories and the transmission bandwidth of each of the plurality of memories are used to estimate the time length for the image data stored in each memory in the corresponding feature image to be transmitted to the calculation engine, and the estimated time length is determined as the corresponding plurality of transmission time lengths.
[0023] For each calculation layer in the preset neural network, the maximum time length in the corresponding plurality of transmission time lengths is determined as the corresponding memory access time length.
[0024] In the method, the voltage frequency of the neural network processor is dynamically adjusted based on the corresponding calculation time length and memory access time length of each calculation layer.
[0025] In a case where the calculation time length of the executed calculation layer is greater than the memory access time length, the neural network processor is controlled to be at a high voltage frequency.
[0026] In a case where the calculation time length of the executed calculation layer is less than the memory access time length, the neural network processor is controlled to be at a low voltage frequency.
[0027] Embodiments of the present application provide a voltage frequency adjustment device applied to a neural network accelerator including a neural network processor and a plurality of memories, the plurality of memories are cascaded according to storage sizes, and the smallest memory is connected to a calculation engine in the neural network processor; the device includes:
[0028] A determination module is configured to:
[0029] For each calculation layer in the preset neural network, the size of the corresponding feature image and the size of the included convolution kernel are used to determine the corresponding total calculation amount, and the time length for the calculation engine to complete the corresponding total calculation amount is estimated to determine the corresponding calculation time length.
[0030] For each calculation layer in the preset neural network, the storage amount of the corresponding feature image in each of the plurality of memories and the transmission bandwidth of each of the plurality of memories are used to estimate the time length for the corresponding feature image to be transmitted to the calculation engine, and the estimated time length is determined as the corresponding memory access time length.
[0031] The adjusting module is configured to: during the neural network processor sequentially executing each calculation layer in the preset neural network by using the calculation engine, dynamically adjusting a voltage frequency of the neural network processor based on a calculation time length and a memory access time length corresponding to each calculation layer.
[0032] In the apparatus, the determining module is specifically configured to:
[0033] For each calculation layer in the preset neural network, a product of twice a size of a corresponding feature image and a size of a contained convolution kernel is calculated to determine a corresponding total calculation amount.
[0034] In the apparatus, for each calculation layer in the preset neural network, the size of the corresponding feature image is a product of an image width, an image height, and an image channel number, and the size of the contained convolution kernel is a product of a convolution kernel number and a convolution kernel plane size.
[0035] In the apparatus, the determining module is specifically configured to:
[0036] An operating frequency of the calculation engine is obtained.
[0037] For each calculation layer in the preset neural network, a number of calculation units participating in calculation in the calculation engine is obtained to determine a corresponding unit number.
[0038] For each calculation layer in the preset neural network, a product of the unit number and the operating frequency is calculated to determine a corresponding unit calculation amount, and a quotient of the total calculation amount and the unit calculation amount is calculated to obtain a corresponding calculation time length.
[0039] In the apparatus, the determining module is specifically configured to:
[0040] A total number of calculation units contained in the calculation engine is obtained.
[0041] For each calculation layer in the preset neural network, a corresponding calculation unit utilization rate in the calculation engine is obtained.
[0042] For each calculation layer in the preset neural network, a product of the total number of calculation units and the corresponding calculation unit utilization rate is calculated to determine the unit number.
[0043] In the apparatus, the determining module is specifically configured to:
[0044] For each calculation layer in the preset neural network, using a storage amount of a corresponding feature image in different memories in the plurality of memories and a transmission bandwidth of each memory in the plurality of memories, a time length for image data stored in each memory in the corresponding feature image to be transmitted to the calculation engine is estimated to obtain a plurality of corresponding transmission time lengths.
[0045] For each calculation layer in the preset neural network, the maximum time length in the corresponding multiple transmission time lengths is determined as the corresponding memory access time length.
[0046] In the device, the adjusting module is specifically configured to:
[0047] In the case where the calculation time length corresponding to the executed calculation layer is greater than the memory access time length, the neural network processor is controlled to be at a high voltage frequency;
[0048] In the case where the calculation time length corresponding to the executed calculation layer is less than the memory access time length, the neural network processor is controlled to be at a low voltage frequency.
[0049] Embodiments of the present application provide a voltage frequency adjusting device, comprising a processor, a memory and a communication bus;
[0050] The communication bus is configured to realize the communication connection between the processor and the memory.
[0051] The processor is configured to execute one or more programs stored in the memory to realize the above-mentioned voltage frequency adjusting method.
[0052] Embodiments of the present application provide a neural network accelerator, comprising a neural network processor and multiple memories, the multiple memories are cascaded according to storage size, and the smallest memory is connected with the calculation engine in the neural network processor, and further comprising the above-mentioned voltage frequency adjusting device.
[0053] Embodiments of the present application provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the above-mentioned voltage frequency adjusting method.
[0054] This application provides a voltage frequency adjustment method applied to a neural network accelerator including a neural network processor and multiple memories. The multiple memories are cascaded according to their storage size, with the smallest memory connected to the computing engine in the neural network processor. The method includes: for each computational layer in a preset neural network, determining the corresponding total computational load using the size of the corresponding feature image and the size of the convolutional kernel, and estimating the time required for the computing engine to complete the corresponding total computational load, which is then determined as the corresponding computational time; for each computational layer in the preset neural network, estimating the time required to transmit the corresponding feature image to the computing engine using the storage size of the corresponding feature image in each of the multiple memories and the transmission bandwidth of each memory, which is then determined as the corresponding memory access time; and dynamically adjusting the voltage frequency of the neural network processor based on the computational time and memory access time corresponding to each computational layer during the sequential execution of each computational layer in the preset neural network by the computing engine. The voltage frequency adjustment method provided in this application improves the performance of the neural network processor and reduces its power consumption by flexibly and dynamically adjusting the voltage frequency of the neural network processor. Attached Figure Description
[0055] Figure 1 This is a schematic diagram of the structure of a neural network accelerator provided in an embodiment of this application;
[0056] Figure 2 A schematic flowchart illustrating a voltage frequency adjustment method provided in an embodiment of this application;
[0057] Figure 3 This is a schematic diagram illustrating an exemplary feature image storage relationship corresponding to a computational layer, provided as an embodiment of this application.
[0058] Figure 4 An exemplary voltage frequency variation diagram provided for an embodiment of this application;
[0059] Figure 5 A schematic diagram of the structure of a voltage and frequency regulation device provided in this application embodiment. Figure 1 ;
[0060] Figure 6 A schematic diagram of the structure of a voltage and frequency regulation device provided in this application embodiment. Figure 2 . Detailed Implementation
[0061] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0062] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be specifically explained below by way of examples in combination with the drawings. The following examples can be combined with each other, and the same or similar concepts or processes can not be described again in some examples.
[0063] In addition, the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.
[0064] The embodiment of the present application provides a voltage frequency adjustment method, applied to a neural network accelerator comprising a neural network processor and a plurality of memories, wherein the plurality of memories are cascaded according to storage size, and the smallest memory in storage size is connected to a computing engine in the neural network processor.
[0065] It should be noted that in the embodiments of the present application, the neural network accelerator is used to implement a preset neural network, wherein the computing engine in the neural network processor is used to sequentially execute each computing layer in the preset neural network, and the plurality of memories are used to store the feature images corresponding to each computing layer. When the computing engine executes a computing layer, the corresponding feature images need to be transmitted from the plurality of memories to the computing engine as input, so as to perform the operation of the computing layer. The feature images corresponding to each computing layer in the preset neural network can be stored in one or more of the plurality of memories. If they are stored in multiple memories, i.e. the feature images are divided into multiple parts, each part is stored in a specific memory. The specific preset neural network and the plurality of memories can be set according to actual needs and application scenarios, which are not limited in the embodiments of the present application.
[0066] It should be noted that in the embodiments of the present application, the computing engine included in the neural network processor in the neural network accelerator is generally divided into a vector processing engine and a tensor processing engine. The vector processing engine is mainly responsible for processing one-dimensional array operations, and the tensor processing engine is mainly responsible for operations between multi-dimensional arrays and is responsible for large-scale convolution parallel operations. Therefore, the computing engine in the present application can specifically refer to the tensor processing engine. Of course, if the data required by some computing layer is a one-dimensional array and needs to be executed by the vector processing engine, the computing engine can also specifically refer to the vector processing engine.
[0067] Figure 1 A structural schematic diagram of a neural network accelerator provided by the embodiment of the present application is shown in FIG. 1. Figure 1 As shown in FIG. 1, the neural network accelerator comprises a digital signal processor, a neural network processor, a memory 1, a memory 2 and a memory 3. The storage sizes of the memory 1, the memory 2 and the memory 3 increase in turn, and the memories are located in the neural network processor and are connected to the tensor processing engine in the neural network processor. The neural network processor further comprises a vector processing engine and a data transfer unit.
[0068] The voltage frequency adjustment method provided by the embodiments of the present application is implemented by the voltage frequency adjustment device. Figure 2 A flowchart of a voltage frequency adjustment method provided by the embodiments of the present application is shown in FIG. 1. As shown in the figure, the voltage frequency adjustment method mainly includes the following steps: Figure 2
[0069] S101, for each calculation layer in the preset neural network, the size of the corresponding feature image and the size of the contained convolution kernel are used to determine the corresponding total calculation amount, and the time length for the calculation engine to complete the corresponding total calculation amount is estimated to determine the corresponding calculation time length.
[0070] In the embodiments of the present application, the voltage frequency adjustment device can first determine the corresponding total calculation amount for each calculation layer in the preset neural network, so as to estimate the time length for the calculation engine to complete the corresponding total calculation amount, and determine the corresponding calculation time length.
[0071] Specifically, in the embodiments of the present application, the voltage frequency adjustment device determines the corresponding total calculation amount for each calculation layer in the preset neural network by using the size of the corresponding feature image and the size of the contained convolution kernel, which includes: for each calculation layer in the preset neural network, calculating twice the product of the size of the corresponding feature image and the size of the contained convolution kernel to determine the corresponding total calculation amount.
[0072] It should be noted that in the embodiments of the present application, the calculation layer in the preset neural network actually implements convolution operation, that is, it contains two algorithms, multiplication and addition, and the calculation amount of each of the multiplication and addition of each calculation layer is equal, which is the product of the size of the corresponding feature image and the size of the contained convolution kernel, so the total calculation amount of each calculation layer, that is, twice the product of the size of the corresponding feature image and the size of the contained convolution kernel.
[0073] It should be noted that in the embodiments of the present application, the size of the corresponding feature image of each calculation layer in the preset neural network is the product of the image width, the image height, and the image channel number, and the size of the contained convolution kernel is the product of the convolution kernel number and the convolution kernel plane size.
[0074] In the embodiments of the present application, the total calculation amount of each calculation layer in the preset neural network can be calculated according to the following formula (1) respectively:
[0075] OPs = IW x IH x IC x K x OC x 2 (1) 2
[0076] Wherein, OP is the total calculation amount corresponding to the calculation layer, IW is the image width of the feature image corresponding to the calculation layer, IH is the image height of the feature image corresponding to the calculation layer, IC is the image channel number of the feature image corresponding to the calculation layer, K 2 is the kernel plane size of the convolution kernel contained in the calculation layer, and OC is the number of convolution kernels of the convolution kernel contained in the calculation layer.
[0077] It should be noted that in the embodiments of the present application, the voltage frequency adjusting device, for each calculation layer in the preset neural network, determines the corresponding total calculation amount, and further estimates the time length for the calculation engine to complete the corresponding total calculation amount.
[0078] Specifically, in the embodiments of the present application, the voltage frequency adjusting device, for each calculation layer in the preset neural network, estimates the time length for the calculation engine to complete the corresponding total calculation amount, and determines the corresponding calculation time length, including: obtaining the working frequency of the calculation engine; for each calculation layer in the preset neural network, obtaining the number of calculation units in the calculation engine participating in the calculation, and determining the corresponding unit number; for each calculation layer in the preset neural network, calculating the product of the corresponding unit number and the working frequency, determining the corresponding unit calculation amount, and calculating the quotient of the corresponding total calculation amount and the unit calculation amount, to obtain the corresponding calculation time length.
[0079] Specifically, in the embodiments of the present application, the voltage frequency adjusting device, for each calculation layer in the preset neural network, obtains the number of calculation units in the calculation engine participating in the calculation, and determines the corresponding unit number, including: obtaining the total number of calculation units contained in the calculation engine; obtaining the utilization rate of the corresponding calculation unit in the calculation engine for each calculation layer in the preset neural network; for each calculation layer in the preset neural network, calculating the product of the total number of calculation units and the corresponding calculation unit utilization rate, to determine the corresponding unit number.
[0080] It should be noted that in the embodiments of the present application, the calculation engine can contain a certain number of multipliers and adders, each multiplier and each adder is a calculation unit. The calculation units in the calculation engine are pre-configured, and the voltage frequency adjusting device can obtain the number of calculation units contained in the calculation engine. In addition, since the number of addition and multiplication operations involved in each calculation layer in the preset neural network is limited, the calculation units applied when each calculation layer is executed by the calculation engine are limited, and each calculation layer has its corresponding calculation unit utilization rate, which can be obtained by the voltage frequency adjusting device.
[0081] In the embodiments of the present application, the corresponding calculation time length of each calculation layer in the preset neural network can be calculated according to the following formula (2) respectively:
[0082]
[0083] wherein, time mac is the calculation duration corresponding to the calculation layer, MAC is the number of adders or multipliers contained in the calculation engine, since the number of adders and multipliers in the calculation engine is the same, MAC x 2 is the number of calculation units contained in the calculation engine, Freq is the working frequency of the calculation engine, and MAC_Urate is the calculation unit utilization rate corresponding to the calculation layer.
[0084] In S102, for each calculation layer in the preset neural network, the storage amount of the corresponding feature image in each memory of the plurality of memories and the transmission bandwidth of each memory of the plurality of memories are used to estimate the duration of transmission of the corresponding feature image to the calculation engine, which is determined as the corresponding memory access duration.
[0085] In the embodiments of the present application, the voltage frequency adjusting device can not only determine the corresponding calculation duration for each calculation layer in the preset neural network, but also estimate the corresponding memory access duration for each calculation layer.
[0086] Specifically, in the embodiments of the present application, the voltage frequency adjusting device estimates the duration of transmission of the corresponding feature image to the calculation engine for each calculation layer in the preset neural network by using the storage amount of the corresponding feature image in each memory of the plurality of memories and the transmission bandwidth of each memory of the plurality of memories, and determines the corresponding memory access duration, which includes: for each calculation layer in the preset neural network, the storage amount of the corresponding feature image in different memories of the plurality of memories and the transmission bandwidth of each memory of the plurality of memories are used to estimate the duration of transmission of the image data of the corresponding feature image stored in each memory to the calculation engine, to obtain a plurality of transmission durations; for each calculation layer in the preset neural network, the maximum duration in the corresponding plurality of transmission durations is determined as the corresponding memory access duration.
[0087] It can be understood that in the embodiments of the present application, the corresponding feature image of each calculation layer in the preset neural network can be completely stored in one memory, or can be divided into multiple parts and stored in different memories. For example, Figure 3 As shown in the figure, the corresponding feature image of each calculation layer in the five calculation layers can be stored in different memories.
[0088] It should be noted that in the embodiments of the present application, when the corresponding feature image of a calculation layer is transmitted to the calculation engine, the memories storing the image data of each part of the feature image will be transmitted in parallel. For a memory with a large memory size, it is far away from the calculation engine and needs to be transmitted to the upper level memory of the cascade first, and then transmitted in turn until it is transmitted to the calculation engine.
[0089] It should be noted that in the embodiments of the present application, the transmission bandwidths of different memories are different, which represents the amount of data transmitted by the memories in a unit of time. The voltage frequency adjusting apparatus can calculate the time length of the image data in the feature image stored in each memory being transmitted to the computing engine, in combination with the storage condition of the corresponding feature image and the transmission bandwidth of each memory, for each calculation layer in the preset data neural network. Among them, if one or more of the plurality of memories do not store any image data in the feature image corresponding to a calculation layer, the transmission time length corresponding to each of the one or more memories in the plurality of transmission time lengths corresponding to the calculation layer is actually zero.
[0090] It can be understood that in the embodiments of the present application, since different memories transmit data at the same time, the memory access time length corresponding to the calculation layer is limited by the maximum time length in the plurality of transmission time lengths corresponding to the calculation layer, that is, the maximum time length in the plurality of transmission time lengths corresponding to the calculation layer is the memory access time length corresponding to the calculation layer, denoted as time data .
[0091] S103, during the period in which the neural network processor executes each calculation layer in the preset neural network in turn by using the computing engine, dynamically adjusting the voltage frequency of the neural network processor based on the calculation time length and the memory access time length corresponding to each calculation layer.
[0092] In the embodiments of the present application, the voltage frequency adjusting apparatus can dynamically adjust the voltage frequency of the neural network processor based on the calculation time length and the memory access time length corresponding to each calculation layer during the period in which the neural network processor executes each calculation layer in the preset neural network in turn, after estimating the calculation time length and the memory access time length corresponding to each calculation layer in the preset neural network.
[0093] Specifically, in the embodiments of the present application, the voltage frequency adjusting apparatus dynamically adjusts the voltage frequency of the neural network processor based on the calculation time length and the memory access time length corresponding to each calculation layer, including: in the case where the calculation time length corresponding to the calculation layer being executed is greater than the memory access time length, controlling the neural network processor to be at a high voltage frequency; in the case where the calculation time length corresponding to the calculation layer being executed is less than the memory access time length, controlling the neural network processor to be at a low voltage frequency.
[0094] It can be understood that in the embodiments of the present application, for a calculation layer, if the corresponding calculation time length is greater than the memory access time length, it represents that the execution of the calculation layer is subject to the computing engine, and by controlling the neural network processor to be at a high voltage frequency, the performance can be improved, and if the corresponding calculation time length is less than the memory access time length, it represents that the execution of the calculation layer is subject to data transmission, and by controlling the neural network processor to be at a low voltage frequency, the power consumption can be reduced. The specific high voltage frequency and low voltage frequency are not limited in the embodiments of the present application.
[0095] It should be noted that in the embodiments of the present application, for the case where the memory access duration is equal to the calculation duration, the voltage frequency adjusting device can control the neural network processor to be in a standard voltage frequency. The standard voltage frequency is between the high voltage frequency and the low voltage frequency.
[0096] Figure 4 An exemplary voltage frequency change diagram is provided for the embodiments of the present application. As shown in the figure, Figure 4 When the calculation engine executes to different calculation layers, the voltage frequency adjusting device can dynamically adjust the voltage frequency of the neural network processor according to the size relationship between the calculation duration and the memory access duration corresponding to the calculation layer. One period represents the duration of executing all calculation layer operations in a preset neural network once.
[0097] It should be noted that in the embodiments of the present application, for the calculation layers with adjacent execution sequences, if the calculation duration and the memory access duration are similar, the voltage frequency adjusting device can combine the adjacent calculation layers to perform voltage frequency adjustment once.
[0098] The voltage frequency adjusting method provided by the embodiments of the present application is applied to a neural network accelerator including a neural network processor and a plurality of memories. The plurality of memories are cascaded according to storage size, and the smallest memory is connected with a calculation engine in the neural network processor. The method comprises: for each calculation layer in a preset neural network, determining the corresponding total calculation amount by using the size of the corresponding feature image and the size of the contained convolution kernel, and estimating the duration of the calculation engine completing the corresponding total calculation amount as the corresponding calculation duration; for each calculation layer in the preset neural network, estimating the duration of the corresponding feature image being transmitted to the calculation engine by using the storage amount of the corresponding feature image in each memory of the plurality of memories and the transmission bandwidth of each memory of the plurality of memories, and determining the corresponding memory access duration; during the execution of each calculation layer in the preset neural network by the calculation engine in the neural network processor, dynamically adjusting the voltage frequency of the neural network processor based on the corresponding calculation duration and the memory access duration of each calculation layer. The voltage frequency adjusting method provided by the embodiments of the present application improves the performance of the neural network processor and reduces the power consumption of the neural network processor by dynamically adjusting the voltage frequency of the neural network processor.
[0099] The embodiments of the present application provide a voltage frequency adjusting device applied to a neural network accelerator including a neural network processor and a plurality of memories. The plurality of memories are cascaded according to storage size, and the smallest memory is connected with the calculation engine in the neural network processor. Figure 5 The structure of the voltage frequency adjusting device provided by the embodiments of the present application is shown in Figure 1 the figure.Figure 5 The voltage frequency adjusting device comprises:
[0100] The determining module 501 is configured to:
[0101] For each calculation layer in the preset neural network, the size of the corresponding feature image and the size of the contained convolution kernel are used to determine the corresponding total calculation amount, and the time length for the calculation engine to complete the corresponding total calculation amount is estimated, which is determined as the corresponding calculation time length.
[0102] For each calculation layer in the preset neural network, the storage amount of the corresponding feature image in each memory of the plurality of memories and the transmission bandwidth of each memory of the plurality of memories are used to estimate the time length for the corresponding feature image to be transmitted to the calculation engine, which is determined as the corresponding memory access time length.
[0103] The adjusting module 502 is configured to: during the process that the neural network processor executes each calculation layer in the preset neural network in sequence by using the calculation engine, dynamically adjusting the voltage frequency of the neural network processor based on the corresponding calculation time length and memory access time length of each calculation layer.
[0104] In an embodiment of the present application, the determining module 501 is specifically configured to:
[0105] For each calculation layer in the preset neural network, the product of twice the size of the corresponding feature image and the size of the contained convolution kernel is calculated, which is determined as the corresponding total calculation amount.
[0106] In an embodiment of the present application, the size of the corresponding feature image of each calculation layer in the preset neural network is the product of the image width, the image height, and the image channel number, and the size of the contained convolution kernel is the product of the convolution kernel number and the convolution kernel plane size.
[0107] In an embodiment of the present application, the determining module 501 is specifically configured to:
[0108] The working frequency of the calculation engine is obtained.
[0109] For each calculation layer in the preset neural network, the number of calculation units participating in calculation in the calculation engine is obtained, which is determined as the corresponding unit number.
[0110] For each calculation layer in the preset neural network, the product of the corresponding unit number and the working frequency is calculated, which is determined as the corresponding unit calculation amount, and the quotient of the corresponding total calculation amount and the unit calculation amount is calculated to obtain the corresponding calculation time length.
[0111] In an embodiment of the present application, the determining module 501 is specifically configured to:
[0112] Obtain the total number of computing units contained in the computing engine;
[0113] Obtain the utilization rate of the corresponding computing unit in the computing engine for each computing layer in the preset neural network;
[0114] For each computational layer in the preset neural network, the product of the total number of units and the corresponding computational unit utilization rate is calculated to determine the corresponding number of units.
[0115] In one embodiment of this application, the determining module 501 is specifically used for:
[0116] For each computational layer in the preset neural network, the storage amount of the corresponding feature image in different memories in the multiple memories, and the transmission bandwidth of each memory in the multiple memories are used to estimate the time for the image data stored in each memory in the corresponding feature image to be transmitted to the computational engine, thus obtaining multiple corresponding transmission times.
[0117] For each computational layer in the preset neural network, the longest of the multiple transmission durations is determined as the corresponding memory access duration.
[0118] In one embodiment of this application, the adjustment module 502 is specifically used for:
[0119] If the computation time corresponding to the computation layer being executed is greater than the memory access time, the neural network processor is controlled to operate at a high voltage frequency.
[0120] When the computation time corresponding to the computation layer being executed is less than the memory access time, the neural network processor is controlled to operate at a low voltage frequency.
[0121] Figure 6 A schematic diagram of the structure of a voltage and frequency regulation device provided in this application embodiment. Figure 2 .like Figure 6 As shown, the voltage frequency adjustment device includes: a processor 601, a memory 602, and a communication bus 603;
[0122] The communication bus 603 is used to realize the communication connection between the processor 601 and the memory 602;
[0123] The processor 601 is used to execute one or more programs stored in the memory 602 to implement the voltage frequency adjustment method described above.
[0124] The embodiment of the present application provides a neural network accelerator, comprising a neural network processor and a plurality of memories, the plurality of memories are cascaded according to storage sizes, and the memory with the smallest storage size is connected with a computing engine in the neural network processor, and the voltage frequency adjusting device is further included. The specific structure of the neural network accelerator can be referred to Figure 1 , of course, Figure 1 is only an exemplary structure, the number of memories included in the neural network processor and other hardware such as the computing engine can be set according to actual needs, and the embodiment of the present application is not limited. The embodiment of the present application provides a computer readable storage medium, and a computer program is stored on the computer readable storage medium. The computer program is executed by the processor to realize the voltage frequency adjusting method. The computer readable storage medium can be a volatile memory (volatile memory, RAM) or a non-volatile memory (non-volatile memory, ROM), a flash memory, a hard disk (Hard Disk Drive, HDD) or a solid state disk (Solid-State Drive, SSD). It can also be a respective device including one or any combination of the above memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant
[0125] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can be in the form of a hardware embodiment, a software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to magnetic disk storage and optical storage) containing computer usable program code.
[0126] The present application is described with reference to the implementation flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram and the combination of the flow and / or block in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device realize the functions specified in the flowchart Figure 1 flow or multiple flows and / or blocks Figure 1 one or more blocks or multiple blocks.
[0127] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flowchart or flowsheet and / or block Figure 1 of the flowchart or flowsheet and / or block
[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions that are executed on the computer or other programmable apparatus provide steps for implementing the Figure 1 function specified in the flowchart or flowsheet and / or block of the flowchart or flowsheet and / or block
[0129] The above description is only specific implementation of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.
Claims
1. A voltage frequency regulation method applied to a neural network accelerator comprising a neural network processor and a plurality of memories, the plurality of memories cascaded in terms of storage size, and a memory with the smallest storage size connected to a compute engine in the neural network processor; characterized in that, The method comprises the following steps: For each calculation layer in the preset neural network, the total calculation amount of the corresponding feature image and the size of the included convolution kernel are determined, and the time length of the calculation engine completing the corresponding total calculation amount is estimated to determine the corresponding calculation time length; For each calculation layer in the preset neural network, the storage amount of the corresponding feature image in each memory of the plurality of memories and the transmission bandwidth of each memory of the plurality of memories are used to estimate the time length of the corresponding feature image transmission to the calculation engine to determine the corresponding memory access time length; During the execution of each calculation layer of the preset neural network by the calculation engine of the neural network processor, the voltage frequency of the neural network processor is dynamically adjusted based on the corresponding calculation time length and memory access time length of each calculation layer.
2. The method of claim 1, wherein, The method comprises the following steps: For each calculation layer in the preset neural network, the product of the size of the corresponding feature image and the size of the included convolution kernel is calculated to determine the corresponding total calculation amount.
3. The method according to claim 1 or 2, characterized in that, The size of the corresponding feature image of each calculation layer in the preset neural network is the product of the image width, the image height, and the image channel number, and the size of the included convolution kernel is the product of the convolution kernel number and the convolution kernel plane size.
4. The method of claim 1, wherein, The method comprises the following steps: Obtain the working frequency of the calculation engine; For each calculation layer in the preset neural network, the number of calculation units participating in the calculation in the calculation engine is obtained to determine the corresponding unit number; For each calculation layer in the preset neural network, the product of the corresponding unit number and the working frequency is calculated to determine the corresponding unit calculation amount, and the quotient of the corresponding total calculation amount and the unit calculation amount is calculated to obtain the corresponding calculation time length.
5. The method of claim 4, wherein, The method comprises the following steps: Obtain the total number of calculation units included in the calculation engine; Obtain the utilization rate of the corresponding calculation unit in the calculation engine for each calculation layer in the preset neural network; For each calculation layer in the preset neural network, the product of the total number of units and the utilization rate of the corresponding calculation unit is calculated to determine the corresponding unit number.
6. The method of claim 1, wherein, The method comprises the following steps: For each calculation layer in the preset neural network, the storage amount of the corresponding feature image in each of the plurality of memories and the transmission bandwidth of each of the plurality of memories are used to estimate the time length for which the image data stored in each memory in the corresponding feature image is transmitted to the calculation engine, to obtain a plurality of transmission time lengths corresponding thereto. For each calculation layer in the preset neural network, the maximum time length in the plurality of transmission time lengths corresponding thereto is determined as the memory access time length corresponding thereto.
7. The method of claim 1, wherein, The voltage frequency of the neural network processor is dynamically adjusted based on the calculation time length and the memory access time length corresponding to each calculation layer, including: In the case where the calculation time length corresponding to the calculation layer being executed is greater than the memory access time length, the neural network processor is controlled to be at a high voltage frequency. In the case where the calculation time length corresponding to the calculation layer being executed is less than the memory access time length, the neural network processor is controlled to be at a low voltage frequency. 8.A voltage frequency regulation apparatus applied to a neural network accelerator comprising a neural network processor and a plurality of memories, the plurality of memories are cascaded according to storage size, and the memory with the smallest storage size is connected to a computing engine in the neural network processor; characterized in that, Including: The determination module is configured to: For each calculation layer in the preset neural network, the size of the corresponding feature image and the size of the convolution kernel contained therein are used to determine the total calculation amount corresponding thereto, and the time length for the calculation engine to complete the total calculation amount corresponding thereto is estimated as the calculation time length corresponding thereto. For each calculation layer in the preset neural network, the storage amount of the corresponding feature image in each of the plurality of memories and the transmission bandwidth of each of the plurality of memories are used to estimate the time length for which the image data stored in each memory in the corresponding feature image is transmitted to the calculation engine, to obtain a plurality of transmission time lengths corresponding thereto. The adjustment module is configured to: during the execution of each calculation layer in the preset neural network by the calculation engine using the neural network processor, dynamically adjust the voltage frequency of the neural network processor based on the calculation time length and the memory access time length corresponding to each calculation layer.
9. A voltage frequency regulating device, characterized by Including: A processor, a memory and a communication bus; The communication bus is used to realize the communication connection between the processor and the memory; The processor is used to execute one or more programs stored in the memory to realize the voltage frequency adjustment method of any one of claims 1-7.
10. A neural network accelerator comprising a neural network processor and a plurality of memories, the plurality of memories cascaded by memory size, and a memory with the smallest memory size connected to a compute engine in the neural network processor, characterized in that, Further comprising the voltage frequency adjustment device of claim 8 or 9.
11. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the voltage frequency adjustment method of any one of claims 1-7. The computer program is executed by the processor to realize the voltage frequency adjustment method of any one of claims 1-7.
Citation Information
Patent Citations
Data processing apparatus and method
CN109478144A
Ultra low-power negative margin timing monitoring method for neural network circuit
US20210174184A1