Neural network hardware acceleration method based on Frobenius norm

Abstract

The invention relates to a neural network hardware acceleration method based on a Frobenius norm. The method comprises the following steps: S1, determining a to-be-pruned layer in a to-be-pruned neural network; wherein the to-be-pruned neural network is used for remote sensing target detection; s2, calculating an importance standard of each convolution kernel in the to-be-pruned layer; wherein the importance standard is obtained according to the Frobenius norm of the corresponding convolution kernel; s3, determining a to-be-pruned convolution kernel according to the convolution kernel importance standard of each convolution kernel; s4, performing structured pruning on the convolution kernel to be pruned to obtain a pruned neural network; s5, judging whether the speedup ratio of the pruned neural network reaches a preset speedup ratio or not; if not, returning to S2; and if yes, executing S6, and training the pruned neural network to obtain a target neural network. Through the technical scheme, the detection speed of remote sensing target detection is improved, the equipment burden is reduced, and the performance of the remote sensing target detection neural network is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing target detection, and in particular to a neural network hardware acceleration method based on Frobenius norm. Background Art

[0002] Modern deep neural networks, especially recent large language neural networks, require a lot of computing and storage resources. For example, the widely used ResNet-50 requires more than 95MB of memory for storage, contains more than 23 million trainable parameters, and requires 4GFLOPs (giga floating point operations) of computation. The size of VGG-16 trained on ImageNet is over 500MB. The GPT-3 neural network based on the Transformer architecture consists of up to 175 billion parameters, and the parameters of the GPT-4 neural network are even more. The current trend of scaling up the size of neural networks is expected to continue. The computational complexity and memory footprint of deep neural networks make them impractical to deploy on edge devices. However, these neural networks have a large number of redundant parameters from convolutional layers to fully connected layers, and a large number of neuron activation values ​​are close to 0. After removing these neurons, the neural network can show the same expressive power. This situation is called over-parameterization. To alleviate this problem, researchers have proposed various neural network compression techniques to design lightweight neural networks, including neural network pruning, low-rank decomposition of weight matrices, quantization, knowledge distillation, neural architecture search, and other compression techniques. Among them, neural network pruning has been proven to be an ideal and effective method to save memory space and computing time during inference while maintaining performance comparable to or even better than that of neural networks.

[0003] With the rapid development of neural networks, remote sensing target detection often uses neural networks for target detection. However, most neural networks currently have many parameters, which results in a burden on hardware devices or reduced detection speed when using neural networks for remote sensing target detection. Taking drone platforms as an example, this memory usage and computing consumption is unbearable for end-side drone devices that need to deploy remote sensing target detection. Summary of the invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides a neural network hardware acceleration method, device, electronic device and storage medium based on Frobenius norm.

[0005] According to a first aspect of an embodiment of the present invention, a neural network hardware acceleration method based on Frobenius norm is provided, the method comprising:

[0006] S1, determining a to-be-pruned layer in a to-be-pruned neural network; wherein the to-be-pruned neural network is used for remote sensing target detection;

[0007] S2, calculating the importance criterion of each convolution kernel in the to-be-pruned layer; wherein the importance criterion is obtained according to the Frobenius norm of the corresponding convolution kernel;

[0008] S3, determining the convolution kernel to be pruned according to the convolution kernel importance standard of each convolution kernel;

[0009] S4, performing structured pruning on the convolution kernel to be pruned to obtain a pruned neural network;

[0010] S5, determining whether the acceleration ratio of the pruned neural network reaches a preset acceleration ratio;

[0011] If not, return to S2; if so, execute S6 to train the pruned neural network to obtain the target neural network.

[0012] Optionally, the neural network to be pruned is a trained neural network.

[0013] Optionally, the importance standard of each convolution kernel is represented as follows:

[0014]

[0015] Among them, weightness(p;W i ) represents the weight matrix W of the i-th layer to be pruned i The importance criterion of the p-th convolution kernel in , ‖·‖ F Indicates the Frobenius norm, W p represents the weight of the pth convolution kernel, W q represents the weight of the qth convolution kernel, and ∏ represents the cumulative multiplication.

[0016] Optionally, S3 includes:

[0017] Get the number L′ of convolution kernels with the maximum importance standard in all layers to be pruned;

[0018] After sorting each convolution kernel in all the layers to be pruned according to the corresponding importance criteria, L′ convolution kernels with the smallest importance criteria are determined as the convolution kernels to be pruned.

[0019] Optionally, in each to-be-pruned layer, there is at least one convolution kernel whose importance criterion is a maximum value and is not pruned.

[0020] According to a second aspect of an embodiment of the present invention, there is provided a neural network hardware acceleration device based on Frobenius norm, the device comprising:

[0021] A to-be-pruned layer determination module is used to determine the to-be-pruned layer in the to-be-pruned neural network; wherein the to-be-pruned neural network is used for remote sensing target detection;

[0022] A parameter calculation module, used to calculate the importance criterion of each convolution kernel in the to-be-pruned layer; wherein the importance criterion is obtained according to the Frobenius norm of the corresponding convolution kernel;

[0023] A to-be-pruned convolution kernel determination module, used to determine the to-be-pruned convolution kernel according to the convolution kernel importance standard of each convolution kernel;

[0024] A pruning module, used for performing structured pruning on the convolution kernel to be pruned to obtain a pruned neural network;

[0025] A judgment module, used to judge whether the acceleration ratio of the pruned neural network reaches a preset acceleration ratio;

[0026] If not, return to the parameter calculation module; if so, execute the training module to train the pruned neural network to obtain the target neural network.

[0027] According to a third aspect of an embodiment of the present disclosure, there is provided a neural network hardware acceleration device based on Frobenius norm, comprising: a processor; a memory for storing instructions executable by the processor;

[0028] The processor is configured to: execute the executable instructions to implement the steps of the neural network hardware acceleration method based on the Frobenius norm described in any implementation of the first aspect above.

[0029] According to a fourth aspect of an embodiment of the present invention, there is provided a computer-readable storage medium having computer program instructions stored thereon, wherein when the program instructions are executed by a processor, the steps of the neural network hardware acceleration method based on the Frobenius norm described in the first aspect of the present invention are implemented.

[0030] The technical solution provided by the present invention may include the following beneficial effects:

[0031] The above technical solution is used to prune the neural network during remote sensing target detection, which reduces the number of neural network parameters and improves the reasoning speed of the neural network, thereby improving the detection speed of remote sensing target detection and reducing the burden on equipment. After training the pruned neural network, the accuracy of remote sensing target detection can also be improved. This shows that this method not only makes the neural network lighter after eliminating redundant parameters, but also improves the performance of the remote sensing target detection neural network.

[0032] Other features and advantages of the present invention will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present invention but do not constitute a limitation of the present invention. In the accompanying drawings:

[0034] Figure 1 The present invention is a flowchart of a neural network hardware acceleration method based on Frobenius norm according to an exemplary embodiment.

[0035] Figure 2 It is a schematic diagram of a neural network hardware acceleration process based on the Frobenius norm according to an exemplary embodiment.

[0036] Figure 3 It is a schematic diagram showing a method of pruning a convolution kernel according to an exemplary embodiment.

[0037] Figure 4 is a schematic diagram of another method of pruning a convolution kernel according to an exemplary embodiment.

[0038] Figure 5 It is a block diagram of a neural network hardware acceleration device based on Frobenius norm according to an exemplary embodiment.

[0039] Figure 6 It is a block diagram of an electronic device for a neural network hardware acceleration method based on Frobenius norm according to an exemplary embodiment.

[0040] Figure 7 It is a block diagram of another electronic device for a neural network hardware acceleration method based on Frobenius norm according to an exemplary embodiment. DETAILED DESCRIPTION

[0041] In order to facilitate the understanding of the scheme of the present invention, the relevant state of the prior art and the inventive concept of the present invention are first briefly described.

[0042] First, for a convolutional neural network with a depth of d, the output is calculated for a given input x. The output can be defined as follows:

[0043]

[0044] Where σ(·) represents the ReLU activation function, represents the convolution operation, W iRepresents the weight matrix of the i-th convolutional layer, whose dimensions can be expressed as: (ks,ks,in c ,out c ), ks is the convolution kernel size, in_c is the number of channels of the input feature map of this layer, and out_c is the number of channels of the output feature map of this layer. (1:d) Represents the set of neural network weight matrices, and the dimensions can be expressed as (ks, ks, in c ,out c ,d).

[0045] It is understandable that it is more complicated to evaluate the changes in the final output of the network according to the above formula. We can start from each layer of the neural network. For one layer, the weight matrix is ​​pruned to W i ⊙M i , where M i It is W i The mask matrix has the same dimension as W i The dimensions are the same, M i It is a binary matrix with only 0 and 1 elements: 0 means the corresponding convolution kernel is pruned, 1 means the corresponding convolution kernel is retained, and the operator ⊙ means the weight matrix is ​​multiplied by the corresponding element of the mask matrix. In order to minimize the performance degradation of the neural network after pruning, consider how to minimize the Frobenius norm distortion caused by pruning, that is:

[0046]

[0047] The above formula is equivalent to:

[0048] min||W i -W i ⊙M i || F ;

[0049] Among them, F stands for Frobenius norm, which is the square root of the sum of the squares of all elements of the matrix. For a matrix A, its Frobenius norm is expressed as follows:

[0050]

[0051] Among them, a i′′j′′ Represents the element in the i-th row and j-th column of matrix A, m represents the number of rows of the matrix, and n represents the number of columns of the matrix.

[0052] For ease of explanation, we can take the convolution kernel size ks to be 1, so min||W i -W i ⊙M i || F is equivalent to:

[0053]

[0054] Therefore, the optimization objective function of the entire network can be referred to as follows:

[0055] ||f(x;W (1:d) )-f(x;W (1:d) ⊙M (1:d) )|| F ;

[0056] Assume that only one convolution kernel with the smallest score is pruned each time, and assume that the order of pruning is from the last layer of the network to the first layer of the network in sequence. Under this condition, according to the properties of the ReLU function and the Frobenius norm, the optimization objective function of the entire network is derived from the Cauchy-Schwarz inequality as follows:

[0057]

[0058] According to this rule, when pruning to the i-th layer:

[0059]

[0060] in,

[0061]

[0062] According to the properties of the ReLU function, we can get:

[0063] ||σ(f(x;W (1:i-1) ))|| F ≤||f(x; W (1:i-1) )|| F ;

[0064] Therefore, there is an upper limit to the optimization objective function of the entire network, which can be expressed as follows:

[0065] (∏ j＞i ||W (j) || F )*||W i -W i ⊙M i || F *||f(x;W (1:i-1) )|| F ;

[0066] Among them, *||f(x;W (1:i-1) )|| F is the Frobenius norm of the unpruned part of the network, which can be regarded as a constant.

[0067] So the objective optimization function of the entire neural network becomes:

[0068]

[0069] In order to minimize the above equation, it is necessary to remove the layers with larger fractional terms. However, it is impossible to completely remove the weight matrix W of an entire layer. i , pruning can only be performed at the convolution kernel level. Therefore, the present invention proposes a neural network hardware acceleration method based on the Frobenius norm.

[0070] Figure 1 is a flowchart of a neural network hardware acceleration method based on Frobenius norm according to an exemplary embodiment. Figure 2 is a schematic diagram of a neural network hardware acceleration process based on Frobenius norm according to an exemplary embodiment, such as Figure 1 As shown, the method includes the following steps.

[0071] S1, determining a to-be-pruned layer in a to-be-pruned neural network; wherein the to-be-pruned neural network is used for remote sensing target detection;

[0072] It is understandable that if Figure 2 As shown in the figure, neural networks that perform specific tasks will have different structures. For example, the final output of the network that performs classification tasks is the probability of each category; the final output of the network that performs target detection tasks includes the location, type, and confidence of the object. The convolution kernels in the format of these connected outputs cannot be pruned, otherwise the network will not work properly.

[0073] Furthermore, the neural network to be pruned is a trained neural network. The neural network is first trained on the original target task training set.

[0074] S2, calculates the importance criterion of each convolution kernel in the layer to be pruned; wherein the importance criterion is obtained according to the Frobenius norm of the corresponding convolution kernel.

[0075] Optionally, the importance criterion of each convolution kernel is expressed as follows:

[0076]

[0077] Among them, weightness(p;W i’ ) represents the weight matrix W of the i′th layer to be pruned i’ The importance criterion of the p-th convolution kernel in , ‖·‖ F Indicates the Frobenius norm, W p represents the weight of the pth convolution kernel, W qrepresents the weight of the qth convolution kernel, and ∏ represents the cumulative multiplication.

[0078] S3, determining the convolution kernel to be pruned according to the convolution kernel importance standard of each convolution kernel.

[0079] Optionally, S3 may include:

[0080] Get the number L′ of convolution kernels with the maximum importance standard in all layers to be pruned;

[0081] After sorting each convolution kernel in all the layers to be pruned according to the corresponding importance criteria, the L′ convolution kernels with the smallest importance criteria are determined as the convolution kernels to be pruned.

[0082] Optionally, in each to-be-pruned layer, there is at least one convolution kernel whose importance criterion is a maximum value and is not pruned.

[0083] It is understandable that when performing pruning, the importance of all convolution kernels in the network to be pruned is first globally evaluated. Secondly, for each layer, at least one convolution kernel will be retained, because the score of the first weight (p is 1) is equal to 1, and there will be no larger score, which is impossible with traditional global pruning.

[0084] Specifically, each layer of the neural network is taken as a unit and sorted according to the importance standard of the convolution kernel. Each layer has one and only one maximum importance standard, which is 1. After the sorting of each layer is completed, the importance standards of all layers are sorted globally, so that the importance standards of the first L′ convolution kernels are the same and all have the maximum value of 1. According to the result of this global sorting, the L′ convolution kernels with the smallest importance standards are determined as the convolution kernels to be pruned. This importance index changes dynamically, and due to its inherent limitations, there will be at least one convolution kernel in each layer that will not be pruned.

[0085] S4, performs structured pruning on the convolution kernel to be pruned to obtain the pruned neural network.

[0086] Understandably, Figure 3 is a schematic diagram of pruning a convolution kernel according to an exemplary embodiment, such as Figure 3 As shown in the figure, the input feature map dimension is (hi, wi, c_in). When the weightness of the convolution kernel Kij is the smallest after global sorting, Kij is removed. Due to the pruning of Kij, the blue feature channel corresponding to the output feature map will be removed. At the same time, all the lower convolution kernels will remove one channel, that is, Figure 2 A blue square in the horizontal direction.

[0087] Figure 4is a schematic diagram of another method of pruning a convolution kernel according to an exemplary embodiment, such as Figure 4 As shown in, the pruning method for a neural network with a residual structure is as follows Figure 3 The pruning method of the first convolution A is the same as that of the ordinary convolution, and when a convolution kernel is pruned, it does not affect the number of output feature maps of the second convolution B. Which convolution kernel of the convolution kernel matrix B needs to be pruned is determined by the cross-layer convolution C. Because the output feature map of convolution B and the output feature map of the residual block need to be added, the dimensions must be the same. When a convolution kernel of the convolution kernel matrix C is pruned, the convolution kernel at the same position of the corresponding convolution kernel matrix B also needs to be pruned.

[0088] S5, determining whether the acceleration ratio of the pruned neural network reaches a preset acceleration ratio.

[0089] It can be understood that the speedup ratio is equal to the amount of computation of the neural network before pruning divided by the amount of computation of the neural network after pruning. The speedup ratio cannot be infinite. If the speedup ratio exceeds a certain threshold, that is, the number of model parameters is reduced to a certain extent, then the model performance will irreversibly deteriorate.

[0090] If not, return to S2; if so, execute S6 to train the pruned neural network to obtain the target neural network.

[0091] It is understandable that the performance of the pruned neural network is generally much lower than that of the original neural network. This is because most of the unimportant convolution kernel connections no longer exist, and the retained convolution kernel weights are still the size corresponding to the unpruned neural network, which is obviously not suitable for the pruned neural network. Therefore, it is necessary to train again to reach the level of the original neural network. The training process can refer to the training process of the original neural network.

[0092] In one embodiment, the effect of the present invention can be further illustrated by the following experimental data. A comparative experiment is conducted on the Yolov5 network using the public remote sensing dataset NWPU VHR-10 and using the method to set different acceleration ratios (1.5× and 2.0×) and L2 and L1 pruning methods.

[0093] Table 1

[0094]

[0095]

[0096] It can be seen from Table 1 that:

[0097] The use of the present invention to prune the neural network not only reduces the number of model parameters and improves the model's reasoning speed, but also slightly improves the detection accuracy after retraining, indicating that the method not only makes the model more lightweight after eliminating redundant parameters, but also improves the model's performance.

[0098] In the field of remote sensing target detection, due to the complexity and particularity of its tasks, for example, aerial sensing images can be taken from a height of several hundred meters to nearly ten thousand meters, and ground targets are of different sizes even if they are of the same type. In order to achieve a certain detection accuracy, the number of parameters and the amount of calculation of the model algorithm also increase, resulting in great difficulty in the edge-side deployment of its algorithm. For example, the RetinaNet model has 36.5M parameters and 150GFLOPs of calculations; for another example, YOLOv8l has 44M parameters and 165.2GFLOPS of calculations. Taking the drone platform as an example, this kind of memory usage and computing consumption is unbearable for the end-side drone equipment that needs to deploy remote sensing target detection.

[0099] Through the above technical solution, the present invention can greatly reduce the memory and computing consumption of the neural network to be pruned. While achieving the lightweight of the model, it will also improve the network operation speed and reduce energy consumption, so that the remote sensing image target detection model can cope with the complex detection tasks deployed on the edge platform. When the algorithm is applied to RetinaNet, its parameter amount and calculation amount are reduced to 41.1% and 39.0% of the original respectively; when the algorithm is applied to YOLOv8l, its parameter amount and calculation amount are reduced to 36.7% and 40.2% of the original respectively. At the same time, on the general remote sensing data set Dota, the average accuracy of its detection has not decreased significantly.

[0100] Figure 5 is a block diagram of a neural network hardware acceleration device based on Frobenius norm according to an exemplary embodiment, referring to Figure 5 , the apparatus 500 may include:

[0101] The to-be-pruned layer determination module 501 is used to determine the to-be-pruned layer in the to-be-pruned neural network; wherein the to-be-pruned neural network is used for remote sensing target detection;

[0102] The parameter calculation module 502 calculates the importance criterion of each convolution kernel in the to-be-pruned layer; wherein the importance criterion is obtained according to the Frobenius norm of the corresponding convolution kernel;

[0103] The to-be-pruned convolution kernel determining module 503 is used to determine the to-be-pruned convolution kernel according to the convolution kernel importance standard of each convolution kernel;

[0104] A pruning module 504 is used to perform structured pruning on the convolution kernel to be pruned to obtain a pruned neural network;

[0105] A judgment module 505 is used to judge whether the speedup ratio of the pruned neural network reaches a preset speedup ratio;

[0106] If not, return to the parameter calculation module 502; if so, execute the training module 506 to train the pruned neural network to obtain the target neural network.

[0107] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0108] Figure 6 is a block diagram of an electronic device for a neural network hardware acceleration method based on a Frobenius norm according to an exemplary embodiment. Figure 6 As shown, the electronic device 600 may include: a processor 601 and a memory 602 .

[0109] The processor 601 is used to control the overall operation of the electronic device 600 to complete all or part of the steps in the above-mentioned Frobenius norm-based neural network hardware acceleration method. The memory 602 is used to store various types of data to support the operation of the electronic device 600, and these data may include, for example, instructions for any application or method used to operate on the electronic device 600, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 602 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0110] In an exemplary embodiment, the electronic device 600 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned Frobenius norm-based neural network hardware acceleration method.

[0111] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the neural network hardware acceleration method based on the Frobenius norm are implemented. For example, the computer-readable storage medium can be the memory 602 including the program instructions, and the program instructions can be executed by the processor 601 of the electronic device 600 to complete the neural network hardware acceleration method based on the Frobenius norm.

[0112] Figure 7 7 is a block diagram of another electronic device for a neural network hardware acceleration method based on Frobenius norm according to an exemplary embodiment. For example, the electronic device 700 can be provided as a server. Figure 7 The electronic device 700 includes a processor 722, which may be one or more, and a memory 732 for storing a computer program executable by the processor 722. The computer program stored in the memory 732 may include one or more modules each corresponding to a set of instructions. In addition, the processor 722 may be configured to execute the computer program to perform the above-mentioned Frobenius norm-based neural network hardware acceleration method.

[0113] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the neural network hardware acceleration method based on the Frobenius norm are implemented. For example, the non-transitory computer-readable storage medium can be the memory 732 including the program instructions, and the program instructions can be executed by the processor 722 of the electronic device 700 to complete the neural network hardware acceleration method based on the Frobenius norm.

[0114] In another exemplary embodiment, a computer program product is also provided. The computer program product includes a computer program that can be executed by a programmable device. The computer program has a code portion for executing the above-mentioned Frobenius norm-based neural network hardware acceleration method when executed by the programmable device.

[0115] The preferred embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, a variety of simple modifications can be made to the technical solution of the present invention, and these simple modifications all belong to the protection scope of the present invention.

[0116] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations.

[0117] In addition, various embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the present invention, they should also be regarded as the contents disclosed by the present invention.

Claims

1. A neural network hardware acceleration method based on Frobenius norm, characterized in that: The method comprises: S1, determining a to-be-pruned layer in a to-be-pruned neural network; wherein the to-be-pruned neural network is used for remote sensing target detection; S2, calculating the importance criterion of each convolution kernel in the to-be-pruned layer; wherein the importance criterion is obtained according to the Frobenius norm of the corresponding convolution kernel; S3, determining the convolution kernel to be pruned according to the convolution kernel importance standard of each convolution kernel; S4, performing structured pruning on the convolution kernel to be pruned to obtain a pruned neural network; S5, determining whether the acceleration ratio of the pruned neural network reaches a preset acceleration ratio; If not, return to S2; if so, execute S6 to train the pruned neural network to obtain the target neural network.

2. The neural network hardware acceleration method based on Frobenius norm according to claim 1, characterized in that: The neural network to be pruned is a trained neural network.

3. The neural network hardware acceleration method based on Frobenius norm according to claim 1, characterized in that: The importance standard of each convolution kernel is expressed as follows: Among them, weightness(p;W i’ ) represents the weight matrix W of the i′th layer to be pruned i’ The importance criterion of the p-th convolution kernel in , ‖·‖ F Indicates the Frobenius norm, W p represents the weight of the pth convolution kernel, W q represents the weight of the qth convolution kernel, and ∏ represents the cumulative multiplication.

4. The neural network hardware acceleration method based on Frobenius norm according to claim 1, characterized in that S3 include: Get the number L′ of convolution kernels with the maximum importance standard in all layers to be pruned; After sorting each convolution kernel in all the layers to be pruned according to the corresponding importance criteria, the L′ convolution kernels with the smallest importance criteria are determined as the convolution kernels to be pruned.

5. The neural network hardware acceleration method based on Frobenius norm according to claim 4, characterized in that: In each layer to be pruned, there is at least one convolution kernel with the maximum importance criterion that has not been pruned.

6. A neural network hardware acceleration device based on Frobenius norm, characterized in that: The device comprises: A to-be-pruned layer determination module is used to determine the to-be-pruned layer in the to-be-pruned neural network; wherein the to-be-pruned neural network is used for remote sensing target detection; A parameter calculation module, used to calculate the importance criterion of each convolution kernel in the to-be-pruned layer; wherein the importance criterion is obtained according to the Frobenius norm of the corresponding convolution kernel; A to-be-pruned convolution kernel determination module, used to determine the to-be-pruned convolution kernel according to the convolution kernel importance standard of each convolution kernel; A pruning module, used for performing structured pruning on the convolution kernel to be pruned to obtain a pruned neural network; A judgment module, used to judge whether the acceleration ratio of the pruned neural network reaches a preset acceleration ratio; If not, return to the parameter calculation module; if so, execute the training module to train the pruned neural network to obtain the target neural network.

7. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to: execute the executable instructions to implement the steps of the neural network hardware acceleration method based on the Frobenius norm in any one of claims 1 to 5.

8. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the neural network hardware acceleration method based on the Frobenius norm described in any one of claims 1 to 5 are implemented.

Images