Method and device for calculating dsp resource usage, equipment and storage medium

Abstract

A method, device and computer readable storage medium for calculating DSP resource usage, comprising: determining an operator type of a neural network to be processed, wherein the operator type comprises a convolution layer and a fully connected layer; determining full pipeline clock dilution and DSP multiplexing rate of the convolution layer according to parameters of the convolution layer; determining full pipeline clock dilution and DSP multiplexing rate of the fully connected layer according to parameters of the fully connected layer; and calculating the DSP resource usage of the neural network to be processed based on the full pipeline clock dilution, the DSP multiplexing rate and static parameters of the convolution layer, and the full pipeline clock dilution, the DSP multiplexing rate and static parameters of the fully connected layer. The method solves the technical problem of over-reliance on engineer experience and inaccurate evaluation of hardware resources for deploying a full pipeline design architecture neural network in related technologies, and achieves independence from engineer experience and improved accuracy and efficiency.

Description

Technical Field

[0001] This application relates to the field of resource management, specifically to a method, apparatus, device, and computer-readable storage medium for calculating DSP resource usage. Background Technology

[0002] In the deployment of deep neural networks, the fully pipelined design architecture has the advantage of extremely fast inference speed due to its pipelined operation paradigm, which can meet the real-time requirements of projects. Therefore, this type of algorithm has a promising future in practical application scenarios.

[0003] In practical applications, deployment methods based on a fully pipelined architecture require first unfolding each layer of the neural network according to its actual deployment. Hardware resources are then allocated customarily based on the computational needs of each layer to ensure that during network inference, the allocated hardware resources are sufficient to complete the feature map computation for that layer within a unit of time. This allows for pipelined computation across layers, processing inputs fed in at different times at different layers, and achieving real-time inference. However, each layer requires significant hardware resources during inference, placing high demands on the resources of the hardware deployment platform. Pre-calculating hardware resource usage before deployment can solve the challenge of deploying a fully pipelined architecture.

[0004] Currently, in the design of full pipelined architecture deployment, the size of network computation and parameters is used to roughly assess the hardware resource requirements of the network by referring to the resource usage of networks of the same magnitude after previous deployments. However, this approach relies too heavily on engineers' experience and the assessment is inaccurate. Summary of the Invention

[0005] This application provides a method, apparatus, device, and computer-readable storage medium for calculating DSP resource usage, which can solve the technical problems in the prior art where hardware resources for deploying fully pipelined neural network architectures rely excessively on engineer experience and are inaccurately evaluated.

[0006] In a first aspect, embodiments of this application provide a method for calculating DSP resource usage, the method comprising:

[0007] Determine the operator type of the neural network to be processed, wherein the operator type includes convolutional layers and fully connected layers;

[0008] Based on the parameters of the convolutional layer, the full pipeline clock dilution of the convolutional layer and the reuse rate of the DSP are determined, wherein the parameters of the convolutional layer include static parameters and dynamic parameters;

[0009] Based on the parameters of the fully connected layer, the full pipeline clock dilution of the fully connected layer and the reuse rate of the DSP are determined, wherein the parameters of the convolutional layer include static parameters and dynamic parameters;

[0010] Based on the full pipeline clock dilution of the convolutional layer, the DSP reuse rate and static parameters, as well as the full pipeline clock dilution of the fully connected layer, the DSP resource usage of the neural network to be processed is calculated.

[0011] In conjunction with the first aspect, in one implementation, determining the full pipeline clock dilution of the convolutional layer and the DSP reuse rate based on the parameters of the convolutional layer includes:

[0012] The full pipelined clock dilution of the convolutional layer is determined based on the static parameters of the convolutional layer.

[0013] Based on the dynamic parameters of the convolutional layer, the DSP reuse rate of the convolutional layer is determined. In conjunction with the first aspect, in one embodiment, determining the full pipeline clock dilution and DSP reuse rate of the fully connected layer based on the parameters of the fully connected layer includes:

[0014] The full pipeline clock dilution of the fully connected layer is determined based on the static parameters of the fully connected layer.

[0015] Based on the dynamic parameters of the fully connected layer, the DSP reuse rate of the fully connected layer is determined. In conjunction with the first aspect, in one embodiment, the calculation of the DSP resource usage of the neural network to be processed based on the full pipeline clock dilution of the convolutional layer, the DSP reuse rate, and static parameters, as well as the full pipeline clock dilution of the fully connected layer, the DSP reuse rate, and static parameters, includes:

[0016] The DSP resource usage of the convolutional layer is calculated based on the full pipeline clock dilution, DSP reuse rate, and static parameters. The static parameters of the convolutional layer include input channel parameters, output channel parameters, and convolution kernel parameters.

[0017] Based on the full pipeline clock dilution of the fully connected layer and the multiplexing rate and static parameters of the DSP, the DSP resource usage of the fully connected layer is calculated, wherein the static parameters of the fully connected layer include input channel parameters and output channel parameters.

[0018] The sum of the DSP resource usage of the convolutional layer and the fully connected layer is obtained to calculate the DSP resource usage of the neural network to be processed.

[0019] In conjunction with the first aspect, in one embodiment, the convolutional layer includes a regular convolutional layer, a DW convolutional layer, and / or a PW convolutional layer, and the number of the regular convolutional layer, DW convolutional layer, and / or PW convolutional layer is at least one layer.

[0020] In conjunction with the first aspect, in one implementation, calculating the DSP resource usage of the convolutional layer based on the full pipeline clock dilution of the convolutional layer, the DSP reuse rate, and static parameters includes:

[0021] If the convolutional layer is a DW convolutional layer, then the static parameters include the kernel parameters and output channel parameters of the DW convolutional layer;

[0022] The DSP resource usage of the DW convolutional layer is calculated based on the full pipeline clock dilution, DSP reuse rate, convolution kernel parameters, and output channel parameters.

[0023] In conjunction with the first aspect, in one implementation, calculating the DSP resource usage of the convolutional layer based on the full pipeline clock dilution of the convolutional layer, the DSP reuse rate, and static parameters includes:

[0024] If the convolutional layer is a PW convolutional layer, then the static parameters include the kernel parameters and input channel parameters of the PW convolutional layer;

[0025] The DSP resource usage of the PW convolutional layer is calculated based on the full pipeline clock dilution, DSP reuse rate, convolution kernel parameters, and input channel parameters.

[0026] Secondly, embodiments of this application provide a device for calculating DSP resource usage, the device comprising:

[0027] The first determining module is used to determine the operator type of the neural network to be processed, wherein the operator type includes convolutional layers and fully connected layers;

[0028] The second determining module is used to determine the full pipeline clock dilution of the convolutional layer and the reuse rate of the DSP based on the parameters of the convolutional layer, wherein the parameters of the convolutional layer include static parameters and dynamic parameters;

[0029] The third determining module is used to determine the full pipeline clock dilution of the full connection layer and the multiplexing rate of the DSP based on the parameters of the full connection layer, wherein the parameters of the full connection layer include static parameters and dynamic parameters;

[0030] The calculation module is used to calculate the DSP resource usage of the neural network to be processed based on the full pipeline clock dilution and DSP reuse rate of the convolutional layer, and the full pipeline clock dilution and DSP reuse rate of the fully connected layer.

[0031] Thirdly, embodiments of this application provide a DSP resource usage calculation device, which includes a processor, a memory, and a DSP resource usage calculation program stored in the memory and executable by the processor. When the DSP resource usage calculation program is executed by the processor, it implements the steps of the DSP resource usage calculation method described above.

[0032] Fourthly, embodiments of this application provide a computer-readable storage medium storing a DSP resource usage calculation program, wherein when the DSP resource usage calculation program is executed by a processor, it implements the steps of the DSP resource usage calculation method described above.

[0033] The beneficial effects of the technical solution provided in this application embodiment include: determining the operator type of the neural network to be processed, wherein the operator type includes convolutional layers and fully connected layers; determining the full pipelined clock dilution and DSP reuse rate of the convolutional layer based on the parameters of the convolutional layer, wherein the parameters of the convolutional layer include static parameters and dynamic parameters; determining the full pipelined clock dilution and DSP reuse rate of the fully connected layer based on the parameters of the fully connected layer, wherein the parameters of the convolutional layer include static parameters and dynamic parameters; calculating the DSP resource usage of the neural network to be processed based on the full pipelined clock dilution, DSP reuse rate and static parameters of the convolutional layer, and the full pipelined clock dilution, DSP reuse rate and static parameters of the fully connected layer, thereby solving the technical problems in related technologies where the hardware resources for deploying a fully pipelined neural network architecture rely excessively on engineer experience and are inaccurately evaluated. This allows for accurate calculation of the DSP resource usage of the neural network to be processed before actual deployment, achieving independence from engineer experience and improving accuracy and efficiency. Attached Figure Description

[0034] Figure 1 This is a flowchart illustrating an embodiment of the method for calculating DSP resource usage in this application;

[0035] Figure 2 This is a schematic diagram of the DSP multiplexing rate in this application;

[0036] Figure 3 This is a functional block diagram of an embodiment of the DSP resource usage calculation device of this application;

[0037] Figure 4 This is a schematic diagram of the hardware structure of the computing device for calculating DSP resource usage in the embodiments of this application. Detailed Implementation

[0038] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0039] First, some of the technical terms used in this application will be explained to help those skilled in the art understand this application.

[0040] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0041] In a first aspect, embodiments of this application provide a method for calculating DSP resource usage.

[0042] In one embodiment, reference is made to Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the method for calculating DSP resource usage according to this application. Figure 1 As shown, the calculation method for DSP resource usage includes:

[0043] S10: Determine the operator type of the neural network to be processed, wherein the operator type includes convolutional layers and fully connected layers;

[0044] As an example, the operator type of the neural network to be processed is determined, which includes convolutional layers and fully connected layers. The convolutional layers include ordinary convolutional layers, DW convolutional layers, and / or PW convolutional layers, and the number of ordinary convolutional layers, DW convolutional layers, PW convolutional layers, and / or fully connected layers in the neural network to be processed is at least one. For example, the neural network to be processed includes one ordinary convolutional layer, one DW convolutional layer, one PW convolutional layer, and / or one fully connected layer; or, it includes two ordinary convolutional layers, two DW convolutional layers, two PW convolutional layers, and / or one fully connected layer.

[0045] S20: Determine the full pipeline clock dilution of the convolutional layer and the reuse rate of the DSP based on the parameters of the convolutional layer, wherein the parameters of the convolutional layer include static parameters and dynamic parameters;

[0046] As an example, the parameters of a convolutional layer are obtained, including static and dynamic parameters. Based on the obtained static and dynamic parameters of the convolutional layer, the corresponding full pipeline clock dilution and the corresponding DSP reuse rate are determined, respectively.

[0047] S30: Determine the full pipeline clock dilution of the fully connected layer and the reuse rate of the DSP based on the parameters of the fully connected layer, wherein the parameters of the convolutional layer include static parameters and dynamic parameters;

[0048] As an example, the parameters of the fully connected layer are obtained, including static and dynamic parameters. Based on the obtained static and dynamic parameters of the fully connected layer, the corresponding full pipeline clock dilution and the corresponding DSP multiplexing rate are determined, respectively.

[0049] S40: Based on the full pipeline clock dilution of the convolutional layer, the reuse rate of the DSP, and the static parameters, as well as the full pipeline clock dilution of the fully connected layer, the reuse rate of the DSP, and the static parameters, calculate the DSP resource usage of the neural network to be processed.

[0050] As an example, the DSP resource usage of the convolutional layer is calculated using the full pipelined clock dilution, DSP reuse rate, and static parameters. Similarly, the DSP resource usage of the fully connected layer is calculated using the same parameters. The DSP resource usage of the convolutional and fully connected layers is then summed to obtain the total DSP resource usage, which is used as the DSP resource usage of the neural network to be processed.

[0051] In this embodiment, the DSP resource usage of the convolutional layer is calculated using the parameters of the convolutional layer, and the DSP resource usage of the fully connected layer is calculated using the parameters of the fully connected layer. This allows for the calculation of the DSP resource usage of the neural network to be processed, thus solving the technical problems in related technologies where the hardware resources for deploying a fully pipelined neural network architecture rely excessively on engineer experience and are inaccurately evaluated. As a result, the DSP resource usage of the neural network to be processed can be accurately calculated before actual deployment, achieving independence from engineer experience and improving accuracy and efficiency.

[0052] Further, in one embodiment, determining the full pipeline clock dilution of the convolutional layer and the DSP reuse rate based on the parameters of the convolutional layer includes: determining the full pipeline clock dilution of the convolutional layer based on the static parameters of the convolutional layer; and determining the DSP reuse rate of the convolutional layer based on the dynamic parameters of the convolutional layer.

[0053] As an example, the static parameters of a convolutional layer include the convolution stride, which determines the full pipelined clock dilution. For instance, when the convolutional layer is 1, the corresponding convolution stride s1 is obtained. Based on the convolution stride s1, the full pipelined clock dilution of this convolutional layer is determined. For example, the full pipelined clock dilution formula is obtained. Where, d i This represents the full pipelined clock dilution of layer i, where the convolutional layer resides. As the convolution stride accumulates, the pipelined time for one operation becomes longer than that of the upper layers, thus enabling the use of fewer DSP resources. This parameter for diluting resources in the full pipelined architecture is called the clock dilution d. i Alternatively, when the convolutional layer includes a regular convolutional layer, a DW convolutional layer, and a PW convolutional layer, obtain the convolution stride of the regular convolutional layer, the DW convolutional layer, and the PW convolutional layer respectively, and obtain the clock dilution of each regular convolutional layer, the DW convolutional layer, and the PW convolutional layer.

[0054] The dynamic parameters of a convolutional layer include the quantization bit width, which includes the quantization bit width of the activation layer and the quantization bit width of the weight layer. The reuse rate of the corresponding DSP can be obtained by using the quantization bit widths of the activation and weight layers of the convolutional layer. For example, the quantization bit widths of the activation and weight layers of the convolutional layer can be used to query... Figure 2 The DSP reuse rate table shown provides the corresponding DSP reuse rate.

[0055] In this embodiment, the full pipeline clock dilution of the convolutional layer is determined by the static parameters of the convolutional layer; the DSP reuse rate is determined by the dynamic parameters of the convolutional layer. This solves the problem of relying on engineers' experience to estimate the full pipeline clock dilution of the convolutional layer and the DSP reuse rate in related technologies, and improves the accuracy.

[0056] Further, in one embodiment, determining the full pipeline clock dilution of the fully connected layer and the DSP reuse rate based on the parameters of the fully connected layer includes: determining the full pipeline clock dilution of the fully connected layer based on the static parameters of the fully connected layer; and determining the DSP reuse rate of the fully connected layer based on the dynamic parameters of the fully connected layer.

[0057] As an example, the static parameters of the fully connected layer include a preset stride, which determines the full pipeline clock dilution. For instance, when there are 8 fully connected layers, the corresponding convolution stride z8 is obtained. Based on the preset stride z8, the full pipeline clock dilution of this fully connected layer is determined. For example, the full pipeline clock dilution formula is obtained. Where, d j This indicates the full pipeline clock dilution of the fully connected layer in layer j.

[0058] The dynamic parameters of a fully connected layer include the quantization bit width, which includes the quantization bit width of the activation layer and the weight layer. The reuse rate of the DSP corresponding to the convolutional layer can be obtained by using the quantization bit widths of the activation and weight layers of the fully connected layer. For example, the quantization bit widths of the activation and weight layers of the fully connected layer can be used to query parameters such as... Figure 2 The DSP reuse rate table shown provides the corresponding DSP reuse rate.

[0059] In this embodiment, the full pipeline clock dilution of the full connection layer is determined by the static parameters of the full connection layer; the DSP reuse rate is determined by the dynamic parameters of the full connection layer. This solves the problem of relying on engineers' experience to estimate the full pipeline clock dilution of the full connection layer and the DSP reuse rate in related technologies, and improves the accuracy.

[0060] Further, in one embodiment, calculating the DSP resource usage of the neural network to be processed based on the full pipelined clock dilution of the convolutional layer, the DSP reuse rate and static parameters, and the full pipelined clock dilution of the fully connected layer, the DSP reuse rate and static parameters, includes: calculating the DSP resource usage of the convolutional layer based on the full pipelined clock dilution of the convolutional layer, the DSP reuse rate and static parameters, wherein the static parameters of the convolutional layer include input channel parameters, output channel parameters and convolution kernel parameters; calculating the DSP resource usage of the fully connected layer based on the full pipelined clock dilution of the fully connected layer, the DSP reuse rate and static parameters, wherein the static parameters of the fully connected layer include input channel parameters and output channel parameters; obtaining the sum of the DSP resource usage of the convolutional layer and the DSP resource usage of the fully connected layer, and calculating the DSP resource usage of the neural network to be processed.

[0061] As an example, when determining the full pipeline clock dilution and DSP reuse rate of the convolutional layer, the static parameters of the convolutional layer are obtained. If the convolutional layer is a regular convolutional layer, its static parameters include input channel parameters, output channel parameters, and kernel parameters. The DSP resource usage of the regular convolutional layer is calculated using the full pipeline clock dilution, DSP reuse rate, input channel parameters, output channel parameters, and kernel parameters. For example, the formula for obtaining the DSP resource usage of a regular convolutional layer is provided. Where, n i For the DSP resource usage of ordinary convolutional layers, i i For the input channel parameters of the i-th layer of a normal convolutional layer, k i For the kernel parameters of the i-th layer of a normal convolutional layer, o i For the output channel parameters of the i-th layer of a normal convolutional layer, η i The DSP reuse rate of the i-th layer of a normal convolutional layer, di is the full pipelined clock dilution of the i-th layer of a normal convolutional layer.

[0062] If the convolutional layer is a DW convolutional layer, its static parameters include output channel parameters and kernel parameters. The DSP resource usage of the DW convolutional layer can be calculated using the full pipelined clock dilution, DSP multiplexing rate, output channel parameters, and kernel parameters. For example, the formula for obtaining the DSP resource usage of the DW convolutional layer can be used. Where, n i For the DSP resource usage of the DW convolutional layer, k i For the convolution kernel parameters of the i-th layer of the DW convolutional layer, o i For the output channel parameters of the i-th layer of the DW convolutional layer, η i The DSP reuse rate of the i-th layer of the DW convolutional layer, d i is the full pipelined clock dilution of the i-th layer of the DW convolutional layer.

[0063] If the convolutional layer is a Pipeline-Warping (PW) convolutional layer, its static parameters include the input channel parameters and the kernel parameters. The DSP resource usage of the PW convolutional layer is calculated using the full pipeline clock dilution, DSP reuse rate, input channel parameters, and kernel parameters. For example, the formula for obtaining the DSP resource usage of the PW convolutional layer can be used. Where, n i For the DSP resource usage of PW convolutional layers, i i For the input channel parameters of the i-th layer of the PW convolutional layer, o i Let η be the output channel parameters of the i-th layer of the PW convolutional layer. i The DSP reuse rate of the i-th layer of the PW convolutional layer, d i is the full pipelined clock dilution of the i-th layer of the PW convolutional layer.

[0064] Obtain the static parameters of the fully connected layer, including input channel parameters and output channel parameters. Calculate the DSP resource usage of the fully connected layer using the full pipeline clock dilution, DSP multiplexing rate, input channel parameters, and output channel parameters. For example, the formula for obtaining the DSP resource usage of the fully connected layer is as follows: Where, n j For the DSP resource usage of the fully connected layer, i j For the input channel parameters of the j-th layer of the fully connected layer, o j For the output channel parameters of the j-th layer of the fully connected layer, η j For the DSP reuse rate of the j-th layer of the fully connected layer, d j is the full pipeline clock dilution of the j-th fully connected layer.

[0065] When the DSP resource usage of the ordinary convolutional layer, the DW convolutional layer and / or the PW convolutional layer, and the fully connected layer are obtained, the DSP resource usage of the ordinary convolutional layer, the DW convolutional layer and / or the PW convolutional layer, and the fully connected layer are summed to obtain the total DSP resource usage, which is the DSP resource usage of the neural network to be processed.

[0066] In this embodiment, the DSP resource usage of the neural network to be processed is calculated by calculating the DSP resource usage of the convolutional layer and the fully connected layer. This solves the technical problems in related technologies, such as excessive reliance on engineer experience and inaccurate evaluation of hardware resources for deploying fully pipelined neural network architectures. It achieves a solution that does not rely on engineer experience and improves accuracy and efficiency.

[0067] Secondly, embodiments of this application also provide a device for calculating DSP resource usage.

[0068] In one embodiment, reference is made to Figure 3 , Figure 3 This is a functional block diagram of an embodiment of the DSP resource usage calculation device of this application. Figure 3 As shown, the computing device for calculating DSP resource usage includes:

[0069] The first determining module S51 is used to determine the operator type of the neural network to be processed, wherein the operator type includes convolutional layers and fully connected layers;

[0070] The second determining module S52 is used to determine the full pipeline clock dilution of the convolutional layer and the reuse rate of the DSP based on the parameters of the convolutional layer, wherein the parameters of the convolutional layer include static parameters and dynamic parameters.

[0071] The third determining module S53 is used to determine the full pipeline clock dilution of the full connection layer and the multiplexing rate of the DSP based on the parameters of the full connection layer, wherein the parameters of the full connection layer include static parameters and dynamic parameters;

[0072] The calculation module S54 is used to calculate the DSP resource usage of the neural network to be processed based on the full pipeline clock dilution and DSP reuse rate of the convolutional layer, and the full pipeline clock dilution and DSP reuse rate of the fully connected layer.

[0073] Furthermore, in one embodiment, the second determining module S52 is used to:

[0074] The full pipelined clock dilution of the convolutional layer is determined based on the static parameters of the convolutional layer.

[0075] The reuse rate of the DSP in the convolutional layer is determined based on the dynamic parameters of the convolutional layer.

[0076] Furthermore, in one embodiment, the third determining module S53 is used to:

[0077] The full pipeline clock dilution of the fully connected layer is determined based on the static parameters of the fully connected layer.

[0078] The reuse rate of the DSP in the fully connected layer is determined based on the dynamic parameters of the fully connected layer.

[0079] Furthermore, in one embodiment, the calculation module S54 is used for:

[0080] The DSP resource usage of the convolutional layer is calculated based on the full pipeline clock dilution, DSP reuse rate, and static parameters. The static parameters of the convolutional layer include input channel parameters, output channel parameters, and convolution kernel parameters.

[0081] Based on the full pipeline clock dilution of the fully connected layer, the DSP reuse rate, and the static parameters, the DSP resource usage of the fully connected layer is calculated. The static parameters of the fully connected layer include input channel parameters and output channel parameters.

[0082] The sum of the DSP resource usage of the convolutional layer and the fully connected layer is obtained to calculate the DSP resource usage of the neural network to be processed.

[0083] Furthermore, in one embodiment, the second determining module S52 is used to:

[0084] If the convolutional layer is a DW convolutional layer, then the static parameters include the kernel parameters and output channel parameters of the DW convolutional layer;

[0085] The DSP resource usage of the DW convolutional layer is calculated based on the full pipeline clock dilution, DSP reuse rate, convolution kernel parameters, and output channel parameters.

[0086] Furthermore, in one embodiment, the second determining module S52 is used to:

[0087] If the convolutional layer is a PW convolutional layer, then the static parameters include the kernel parameters and input channel parameters of the PW convolutional layer;

[0088] The DSP resource usage of the PW convolutional layer is calculated based on the full pipeline clock dilution, DSP reuse rate, convolution kernel parameters, and input channel parameters.

[0089] The functions of each module in the aforementioned DSP resource usage calculation device correspond to the steps in the aforementioned DSP resource usage calculation method embodiment, and their functions and implementation processes will not be described in detail here.

[0090] Thirdly, embodiments of this application provide a device for calculating DSP resource usage. This device can be a personal computer (PC), a laptop, a server, or other device with data processing capabilities.

[0091] Reference Figure 4 , Figure 4 This is a schematic diagram of the hardware structure of the computing device for calculating DSP resource usage in the embodiments of this application. In the embodiments of this application, the computing device for calculating DSP resource usage may include a processor, memory, communication interface, and communication bus.

[0092] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.

[0093] Communication interfaces include input / output (I / O) interfaces, physical interfaces, and logical interfaces used for interconnecting internal components of the computing device to utilize DSP resources, as well as interfaces used for interconnecting the computing device with other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.

[0094] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0095] The processor can be a general-purpose processor, which can call the DSP resource usage calculation program stored in memory and execute the DSP resource usage calculation method provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the DSP resource usage calculation program is called can be referred to the various embodiments of the DSP resource usage calculation method of this application, and will not be repeated here.

[0096] Those skilled in the art will understand that Figure 4 The hardware structure shown does not constitute a limitation of this application and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0097] Fourthly, embodiments of this application also provide a computer-readable storage medium.

[0098] The present application has a computer-readable storage medium storing a program for calculating DSP resource usage, wherein when the program for calculating DSP resource usage is executed by a processor, it implements the steps of the DSP resource usage calculation method described above.

[0099] The method implemented when the DSP resource usage calculation program is executed can be referred to in various embodiments of the DSP resource usage calculation method of this application, and will not be repeated here.

[0100] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0101] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.

[0102] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.

[0103] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0104] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0105] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0106] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method of calculating a DSP resource usage amount, characterized by, The method for calculating the DSP resource usage amount comprises: determining an operator type of a neural network to be processed, wherein the operator type comprises a convolution layer and a fully connected layer; According to the parameters of the convolution layer, determine the full pipeline clock dilution of the convolution layer and the multiplexing rate of the DSP, wherein the parameters of the convolution layer include static parameters and dynamic parameters, the static parameters of the convolution layer include a convolution step, the dynamic parameters of the convolution layer include a quantization bit width, the quantization bit width of the convolution layer includes an activation layer quantization bit width and a weight layer quantization bit width, the full pipeline clock dilution of the convolution layer is determined by the convolution step s and the full pipeline clock dilution formula determined, represents the full pipeline clock dilution of the i-th layer where the convolution layer is located, and the multiplexing rate of the DSP of the convolution layer is determined by the activation layer quantization bit width and the weight layer quantization bit width of the convolution layer. determine full pipeline clock skews and multiplexing rates of DSPs of the full connection layer according to parameters of the full connection layer, wherein the parameters of the full connection layer include static parameters and dynamic parameters, the static parameters of the full connection layer include preset steps, the full pipeline clock skews of the full connection layer are determined by the preset steps and a full pipeline clock skew formula determined, denote the full pipeline clock skews of the jth layer in which the full connection layer is located, the dynamic parameters of the full connection layer include quantization bit widths, the quantization bit widths of the full connection layer include activation layer quantization bit widths and weight layer quantization bit widths, and the multiplexing rates of the DSPs of the full connection layer are determined by the activation layer quantization bit widths and the weight layer quantization bit widths of the full connection layer; The DSP resource usage of the to-be-processed neural network is obtained by adding and summing the DSP resource usage of the ordinary convolution layer and the DSP resource usage of the full connection layer, the DSP resource usage of the ordinary convolution layer is obtained by the full pipeline clock dilution of the convolution layer, the multiplexing rate of the DSP, the static parameter of the ordinary convolution layer, and an ordinary convolution layer DSP resource usage formula The static parameter of the ordinary convolution layer includes an input channel parameter, an output channel parameter, and a convolution kernel parameter, The DSP resource usage of the ordinary convolution layer is The input channel parameter of the i-th layer of the ordinary convolution layer is The convolution kernel parameter of the i-th layer of the ordinary convolution layer is The output channel parameter of the i-th layer of the ordinary convolution layer is The DSP multiplexing rate of the i-th layer of the ordinary convolution layer is The full pipeline clock dilution of the i-th layer of the ordinary convolution layer is, and the DSP resource usage of the full connection layer is obtained by the full pipeline clock dilution of the full connection layer, the multiplexing rate of the DSP, the static parameter, and a full connection layer DSP resource usage formula The static parameter of the full connection layer includes an input channel parameter and an output channel parameter, The DSP resource usage of the full connection layer is The input channel parameter of the j-th layer of the full connection layer is The output channel parameter of the j-th layer of the full connection layer is The DSP multiplexing rate of the j-th layer of the full connection layer is The full pipeline clock dilution of the j-th layer of the full connection layer is.

2. The method of claim 1, wherein, the method for determining the full pipeline clock dilution and the DSP multiplexing rate of the convolution layer according to the parameters of the convolution layer comprises: determining the full pipeline clock dilution of the convolution layer according to the static parameters of the convolution layer; determining the DSP multiplexing rate of the convolution layer according to the dynamic parameters of the convolution layer.

3. The method of claim 1, wherein the DSP resource usage is calculated by: the method for determining the full pipeline clock dilution and the DSP multiplexing rate of the fully connected layer according to the parameters of the fully connected layer comprises: determining the full pipeline clock dilution of the fully connected layer according to the static parameters of the fully connected layer; determining the DSP multiplexing rate of the fully connected layer according to the dynamic parameters of the fully connected layer.

4. The method of claim 1, wherein the DSP resource usage is calculated by: the method for calculating the DSP resource usage amount of the neural network to be processed based on the full pipeline clock dilution, the DSP multiplexing rate and the static parameters of the convolution layer, and the full pipeline clock dilution, the DSP multiplexing rate and the static parameters of the fully connected layer comprises: calculating the DSP resource usage amount of the convolution layer according to the full pipeline clock dilution, the DSP multiplexing rate and the static parameters of the convolution layer, wherein the static parameters of the convolution layer comprise an input channel parameter, an output channel parameter and a convolution kernel parameter; calculating the DSP resource usage amount of the fully connected layer according to the full pipeline clock dilution, the DSP multiplexing rate and the static parameters of the fully connected layer, wherein the static parameters of the fully connected layer comprise an input channel parameter and an output channel parameter; calculating the DSP resource usage amount of the neural network to be processed by adding the DSP resource usage amount of the convolution layer and the DSP resource usage amount of the fully connected layer.

5. The method of claim 1, wherein the DSP resource usage is calculated by: The convolution layer comprises an ordinary convolution layer, a DW convolution layer and / or a PW convolution layer, and the number of the ordinary convolution layer, the DW convolution layer and / or the PW convolution layer is at least one layer.

6. The method of claim 4, wherein the DSP resource usage is calculated by: the method for calculating the DSP resource usage amount of the convolution layer according to the full pipeline clock dilution, the DSP multiplexing rate and the static parameters of the convolution layer comprises: if the convolution layer is a DW convolution layer, the static parameters comprise a convolution kernel parameter and an output channel parameter of the DW convolution layer; calculating the DSP resource usage amount of the DW convolution layer according to the full pipeline clock dilution, the DSP multiplexing rate, the convolution kernel parameter and the output channel parameter of the DW convolution layer.

7. The method of claim 4, wherein the DSP resource usage is calculated by: the method for calculating the DSP resource usage amount of the convolution layer according to the full pipeline clock dilution, the DSP multiplexing rate and the static parameters of the convolution layer comprises: if the convolution layer is a PW convolution layer, the static parameters comprise a convolution kernel parameter and an input channel parameter of the PW convolution layer; calculating the DSP resource usage amount of the PW convolution layer according to the full pipeline clock dilution, the DSP multiplexing rate, the convolution kernel parameter and the input channel parameter of the PW convolution layer.

8. A device for calculating a usage amount of a DSP resource, characterized by comprising: the device for calculating the DSP resource usage amount comprises: a first determining module configured to determine an operator type of a neural network to be processed, wherein the operator type comprises a convolution layer and a fully connected layer; The second determining module is used for determining the full pipelined clock dilution of the convolution layer and the multiplexing rate of the DSP according to the parameters of the convolution layer, wherein the parameters of the convolution layer include static parameters and dynamic parameters, the static parameters of the convolution layer include a convolution step, the dynamic parameters of the convolution layer include a quantization bit width, the quantization bit width of the convolution layer includes an activation layer quantization bit width and a weight layer quantization bit width, the full pipelined clock dilution of the convolution layer is determined by a convolution step s and a full pipelined clock dilution formula determined, represents the full pipelined clock dilution of the i-th layer where the convolution layer is located, and the multiplexing rate of the DSP of the convolution layer is determined by the activation layer quantization bit width and the weight layer quantization bit width of the convolution layer. a third determining module configured to determine full-pipelined clock skid of the full connection layer and multiplexing rate of the DSP according to parameters of the full connection layer, wherein the parameters of the full connection layer include static parameters and dynamic parameters, the static parameters of the full connection layer include a preset step, the full-pipelined clock skid of the full connection layer is determined by the preset step and a full-pipelined clock skid formula determined, wherein the full-pipelined clock skid of the jth layer in which the full connection layer is located is determined, the dynamic parameters of the full connection layer include quantization bit width, the quantization bit width of the full connection layer includes activation layer quantization bit width and weight layer quantization bit width, and the multiplexing rate of the DSP of the full connection layer is determined by the activation layer quantization bit width and the weight layer quantization bit width of the full connection layer. The calculation module is used to calculate the DSP resource usage of the neural network to be processed based on the full pipeline clock dilution and DSP reuse rate of the convolutional layer, and the full pipeline clock dilution and DSP reuse rate of the fully connected layer. The DSP resource usage of the neural network to be processed is obtained by summing the DSP resource usage of the ordinary convolutional layer and the fully connected layer. The DSP resource usage of the ordinary convolutional layer is calculated using the full pipeline clock dilution of the convolutional layer, the DSP reuse rate, the static parameters of the ordinary convolutional layer, and the formula for calculating the DSP resource usage of the ordinary convolutional layer. The calculated static parameters of the ordinary convolutional layer include input channel parameters, output channel parameters, and convolution kernel parameters. For the DSP resource usage of ordinary convolutional layers, For the input channel parameters of the i-th layer of a normal convolutional layer, For the convolution kernel parameters of the i-th layer of a normal convolutional layer, For the output channel parameters of the i-th layer of a normal convolutional layer, For the DSP reuse rate of the i-th layer of a normal convolutional layer, The full pipeline clock dilution of the i-th layer of a normal convolutional layer is given. The DSP resource usage of the fully connected layer is determined by the full pipeline clock dilution of the fully connected layer, the DSP reuse rate, static parameters, and the DSP resource usage formula of the fully connected layer. The calculated static parameters of the fully connected layer include input channel parameters and output channel parameters. For the DSP resource usage of the fully connected layer, For the input channel parameters of the j-th layer of the fully connected layer, For the output channel parameters of the j-th layer of the fully connected layer, For the DSP reuse rate of the j-th layer of the fully connected layer, is the full pipeline clock dilution of the j-th fully connected layer.

9. A DSP resource usage amount calculating apparatus characterized by comprising: The DSP resource usage amount calculation device comprises a processor, a memory, and a DSP resource usage amount calculation program stored in the memory and executable by the processor, wherein the DSP resource usage amount calculation program, when executed by the processor, implements the steps of the DSP resource usage amount calculation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a DSP resource usage amount calculation program, wherein the DSP resource usage amount calculation program, when executed by a processor, implements the steps of the DSP resource usage amount calculation method according to any one of claims 1 to 7.

Images