An image classification acceleration method and device based on a frequency domain decomposition neural network

Abstract

The application discloses an image classification acceleration method and device based on a frequency domain decomposition neural network. The method comprises the following steps: converting an original image into an RGB image, and scaling the image into an image with the same image pixel size that can be accepted by an input feature map of a convolutional neural network; inputting the image into a pre-trained lightweight neural network model to perform image classification and recognition, and obtaining an image classification and recognition result, wherein the lightweight neural network model is obtained by frequency domain decomposition and lightweight processing of a neural network. The lightweight neural network model converts the convolution operation in the spatial domain into the matrix multiplication operation in the frequency domain and performs compression, and can obtain an output that is approximately the same as the output of the original neural network, while having fewer parameter quantities and computation quantities measured by FLOPs. The application significantly reduces the computation cost and storage cost, and can effectively accelerate the image classification task.

Description

Technical Field

[0001] This invention relates to image classification and target recognition, and more specifically to an image classification acceleration method and device based on a frequency domain decomposition neural network. Background Technology

[0002] Computer vision applications such as image classification and object recognition are ubiquitous in daily life. However, their high computational and resource consumption results in slow inference speeds and high resource overhead, necessitating compression and acceleration methods to improve the performance of applications like image classification. Neural network pruning has been extensively studied in recent years, with most pruning methods performed in the spatial domain. Fine-grained pruning, such as unstructured pruning, reduces the number of non-zero parameters by setting unimportant parameters in the weight matrix to zero, thus sparsifying the weight matrix. The advantages of unstructured pruning are its ability to significantly reduce model memory usage and parameter count, and the ability to further utilize the pruned results for weight quantization and hash encoding. However, a drawback of unstructured pruning is the need to design hardware / function libraries adapted to specific algorithms to achieve the desired acceleration. Coarse-grained pruning, such as structured pruning, achieves practical acceleration based on existing hardware and function libraries. However, compared with unstructured pruning, structured pruning suffers a greater loss of precision at the same pruning rate, making it difficult to achieve good results with a high pruning rate. Furthermore, the interpretability and effectiveness of structured pruning have been questioned by some scholars. Summary of the Invention

[0003] The invention proposes a novel iterative-based method for frequency domain compression and acceleration of deep models, aiming to improve inference speed and reduce resource overhead in applications such as image classification and recognition.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] An image classification acceleration method based on frequency domain decomposition neural network includes the following steps:

[0006] The original image is converted into an RGB image with 3 channels, and the RGB image is scaled to the same pixel size that the input feature map of the convolutional neural network can accommodate.

[0007] An image is input into a pre-trained lightweight neural network model for image classification and recognition, resulting in an image classification and recognition result. The lightweight neural network model is obtained by performing frequency domain decomposition and lightweighting on the neural network.

[0008] According to an embodiment of the present invention, the lightweight neural network model is obtained through the following method:

[0009] S1. Pre-train the neural network model based on the image dataset to obtain a complete neural network model;

[0010] S2. Use discrete cosine transform to decompose the pre-trained neural network model in the frequency domain, use a series of frequency domain weight matrices W to represent the convolutional layer, and perform sparsification retraining on the neural network model in the frequency domain to obtain the sparsified weight matrix W.

[0011] S3. For each weight matrix W, select a sample subset Ω of the image dataset and apply a sample-based low-rank approximation algorithm to decompose matrix W into low-rank matrices U and V with an error tolerance ε.

[0012] S4. After fine-tuning the low-rank matrices U and V, recombine them to obtain a new weight matrix W' = UV;

[0013] S5. Let W = W' and repeat steps S3-S4 until the iteration stopping condition is met.

[0014] The present invention also provides a computer device, comprising:

[0015] One or more processors;

[0016] Memory; and

[0017] One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the steps of the image classification acceleration method based on frequency domain decomposition neural networks as described above.

[0018] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the image classification acceleration method based on frequency domain decomposition neural network as described above.

[0019] This invention offers the following advantages: Addressing the problems of complex algorithms, significant accuracy loss, and insufficient effectiveness in spatial domain pruning for image classification, this invention proposes a frequency domain pruning technique. According to the method of this invention, for an image, it is transformed into an input feature map of a convolutional neural network (CNN), and frequency domain features are extracted from this input feature map. Then, a lightweight neural network model obtained through frequency domain decomposition and lightweighting is used for image recognition. This lightweight neural network model converts spatial domain convolution operations into frequency domain matrix multiplication operations, and then uses low-rank approximation and fine-tuning techniques to iteratively sparsify and decompose the frequency domain matrix, resulting in a more compact model representation. The output of the compressed CNN is roughly the same as the original neural network, but the compressed neural network has fewer parameters and a computational cost measured in FLOPs. This invention demonstrates that model inference can be replaced by frequency domain matrix multiplication after decomposition, which can effectively and significantly reduce computational and storage costs. Extensive experiments on various CNN models for image classification and recognition were conducted based on multiple datasets, and the results show that the effect of reducing FLOPs and the number of parameters is significant without affecting test accuracy. Attached Figure Description

[0020] Figure 1 This is a flowchart of the image classification acceleration method according to an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of the transformation of a convolutional layer into a frequency domain matrix according to an embodiment of the present invention;

[0022] Figure 3 This is a comparison of the rank and non-zero coefficients of the frequency domain matrix and spatial domain matrix at each layer in an embodiment of the present invention. Detailed Implementation

[0023] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to embodiments and accompanying drawings. The content mentioned in the embodiments is not intended to limit the present invention.

[0024] Convolutional neural networks (CNNs) have been widely used in complex tasks such as image classification, object detection, and face recognition due to their powerful representation learning capabilities. However, as the performance of CNNs has improved, the number of layers and the number of parameters in the model have also increased. From the perspective of computer systems and hardware, the performance improvement of CNNs comes at the cost of consuming a large amount of memory and computing resources, making it very difficult to deploy large neural networks on devices with limited resources (such as computing resources, memory, and energy consumption). Currently, in many scenarios, neural network computations are performed on GPU servers with high computing power. However, most application clients (such as face recognition) are deployed on mobile devices with limited computing resources. Data collected by the client needs to be sent to the server over the network, and the server returns the results to the client. Therefore, to deploy neural networks on mobile devices, a model compression and acceleration technique is needed. This technique reduces the size of the CNN and decreases the resource consumption in the image classification process, such as reducing FLOPs, memory usage, and disk space consumption, thereby speeding up tasks such as image classification and object recognition.

[0025] The input to a convolutional neural network (CNN) is an image, typically composed of red, green, and blue (RGB) pixels, forming the image's three channels. The storage of an image in a computer is also accomplished using these three matrices. An image is usually 64×64 pixels or 256×256 pixels, which is commonly used as input data for CNNs. This input data is called input feature maps in the field of CNNs, representing an image with a spatial length × width and a certain depth across the image channels. Such images are fed into the convolutional layer, and the output is an output feature map.

[0026] Image processing involves applying convolutional kernels to each image patch. The kernels are shifted according to their size, number of channels, padding, and stride, influencing each patch and becoming part of the output feature map. This output feature map is then scaled and pooled to obtain the output matrix of the convolutional layer. However, this method typically requires a large amount of computation.

[0027] Some research in computer vision has shown that the human visual system has different sensitivities to different frequency components in natural images, with a higher sensitivity to low-frequency components than to high-frequency components. Frequency domain transformation of images, combined with techniques such as coding and quantization, can be applied to scenarios like JPEG compression and video compression. Features can be extracted from the frequency domain, enabling neural networks to directly classify JPEG-compressed images. Because natural images have spatially smooth characteristics, the parameters in the convolutional kernels used to process images in convolutional neural networks also exhibit smooth characteristics; that is, both the input image and the parameters in the convolutional kernels are dominated by low-frequency components in the frequency domain.

[0028] Image features are characterized by low-frequency components dominating energy, while high-frequency energy is often secondary and occupies a small amount of energy. Utilizing this characteristic, convolutional neural networks (CNNs) essentially perform target location and range recognition in shallow layers and category determination in deeper layers. Because of this characteristic of natural images, the input feature maps of CNNs also possess this characteristic. The input feature map is the input to each convolutional layer of a CNN, typically containing multiple channels, with the images in these channels having the same spatial size. A convolutional layer uses multiple convolutional kernels to process each image patch of the same size as the kernel in an input feature map, resulting in an output feature map with the same number of channels as the number of kernels. Since the input feature map has low-frequency components dominating energy, this also leads to low-frequency components dominating the parameters in the convolutional kernels, allowing for frequency domain transformation and low-rank decomposition of the kernel parameters.

[0029] The weight matrix in the frequency domain is not only much sparser but also has a lower rank, a characteristic that is highly advantageous for developing efficient compression algorithms for convolutional neural networks. To leverage these characteristics, this invention proposes the FreqPrune (frequency domain pruning) compression method, which reduces model parameters and accelerates inference without significant performance loss. This invention uses Discrete Cosine Transform (DCT) to transform the convolutional layers and their input feature maps to the frequency domain, demonstrating the feasibility of replacing spatial domain convolution operations with frequency domain matrix multiplication. Then, the frequency domain matrix of the convolutional neural network model is sparsified using regularization methods, and a finite number of retraining iterations are performed. Based on this, an iterative model compression method is proposed. Using a small sample dataset, a low-rank approximation algorithm is used to decompose the frequency domain matrix to obtain a compact model representation, and the model is iteratively fine-tuned until a preset compression ratio is achieved.

[0030] Reference Figure 1 and Figure 2 The method of the present invention specifically includes the following steps:

[0031] Step S1: Given a natural image, this image is converted into an RGB image with 3 channels and then scaled down to the same pixel size as the input feature map of the convolutional neural network. Specifically, the natural image is processed as three superimposed RGB matrices, each of the same size, arranged in RGB order, and normalized so that each value of all matrices is between 0 and 255. For images larger than the input feature map of the convolutional neural network and with blank or meaningless edges, they need to be cropped, converted into RGB images, and scaled down to the same pixel size as the input feature map of the convolutional neural network.

[0032] Step S2: Given an image dataset and a neural network model, specify the error tolerance ε and the computational requirement FLOPs (Floating Point Operations Per Second) or parameter requirement. The image dataset is typically derived from a publicly available image classification dataset or a user-provided dataset for training the neural network model. The error tolerance ε is usually chosen as a small value to control the speed of each iteration in the subsequent decomposition and compression processes. The computational requirement FLOPs or parameter requirement controls the algorithm's termination time to perform compression and acceleration according to the user-defined target computational requirements.

[0033] The neural network model is pre-trained on the aforementioned image dataset to obtain a complete neural network model. Optimization methods such as learning rate decay and stochastic gradient descent are typically used for pre-training to obtain a high-accuracy and high-performance model. This model boasts the strongest classification performance but consumes the most computational resources.

[0034] Step S3: Given a deep convolutional neural network model, use Discrete Cosine Transform (DCT) to represent the convolutional layer with a series of frequency domain weight matrices.

[0035] This invention is based on the principle of DCT, converting the CNN model into the frequency domain. Let D represent DCT. Given an input matrix... Its DCT coefficient Defined as In terms of elements, let and Where n is the number of rows and columns of the matrix. Its DCT can be written as:

[0036]

[0037] Where u and v represent the u-th row and v-th column of the matrix, αu α v The control factor representing the position of the corresponding matrix element. When u = 0 otherwise Once the transformation pair simultaneously satisfies separability and symmetry, i.e., the matrix... If the matrix is ​​square, then the matrix form of DCT can be written as follows: Where C = [C ij [n×n] is a transformation matrix, where the elements are...

[0038] Suppose a convolutional layer consists of c out It consists of _ ... in Each convolutional kernel. The convolutional layer is represented as a 4D tensor. The input is a three-dimensional tensor Where h in ×w in This refers to the size of the input feature map. The output of the convolutional layer is represented as... Where h out ×w out This refers to the dimension of the output feature map. The convolution operation (in the spatial domain) can be represented as... Where mat(·) is the matrix form of the tensor.

[0039] Let W, I, and O be respectively and The corresponding frequency domain representations are generated as follows: For convolutional layers, a DCT is applied to flatten each d×d convolutional kernel into a vector, and all vectors in the layer are grouped to form a frequency weight matrix. Similarly, DCT is applied to the input tensor to generate its frequency domain representation.

[0040] Next, refer to Figure 2 This explains how convolution operations in the spatial domain can be converted into matrix multiplication in the frequency domain. After DCT transformation, Where D is an orthogonal matrix. Similarly, After DCT transformation, Therefore:

[0041]

[0042] Because of this property, the original CNN model can be represented by a series of matrices in the frequency domain.

[0043] After frequency transformation, the model is iteratively retrained to sparse its weight matrix. The retraining process is performed in the frequency domain, as shown below. In forward propagation, the output of each layer is calculated solely by the above equation. In backpropagation, the gradient used to update W can be calculated using the following formula:

[0044]

[0045] in It is the objective function (also called the loss function), (·) ij J represents the matrix element at position (i, j). mn It is a single-element matrix where the element at position (m, n) is 1, and the elements at other positions are 0. W, I, and O are respectively... and The corresponding frequency domain representation.

[0046] To obtain a sparse representation in the frequency domain, a regularization term is added to force more weights to be as close to zero as possible. Therefore, the model is retrained using the following objective function:

[0047]

[0048] in is the loss function of the original deep model, D(model) is the weight of the model in the frequency domain, ||·||1 is the l1 regularization that forces the weights close to zero, and λ is the hyperparameter that balances sparsity.

[0049] Figure 3 It shows a comparison of the rank and nonzero coefficients of the frequency domain matrix and the spatial domain matrix at various levels. Figure 3 (a) represents the rank of the weight matrix of different convolutional layers in the spatial and frequency domains. Figure 3 (b) represents the percentage of non-zero coefficients in the spatial and frequency domains for different convolutional layers.

[0050] Step S4: For each weight matrix W, select a sample subset Ω of the dataset and apply a sample-based low-rank approximation algorithm to decompose matrix W into low-rank matrices U and V with an error tolerance ε. If W has no feasible decomposition, skip that layer and instead attempt to decompose the frequency domain weight matrix of the next convolutional layer.

[0051] To reduce the number of parameters and computational overhead of CNN models in the frequency domain, this invention introduces a low-rank approximation method based on small samples. This method decomposes the weight matrix into the multiplication of smaller matrices, while simultaneously producing similar outputs. Given a weight matrix W, the low-rank approximation seeks two matrices U and V to decompose W, i.e., W ≈ UV, where... And r ≤ min{m, n}. A natural approach is to minimize ||W-UV||2 , where ||·|| F It is the Frobenius norm. However, the optimal rank of matrix factorization is NP-hard, and performing such matrix factorization on each layer ignores the information flow through the CNN layers, which may lead to error accumulation. To address these issues, this invention proposes a low-rank approximation method based on small samples.

[0052] Consider a set of samples Ω = {ω1, ω2, ...}, which is a subset of the input dataset. For a sample ω... i It undergoes several convolutional operations, where the output of the j-th layer... It is used as the input to the (j+1)th layer. The superscript can be omitted in the description without causing confusion; the input to the convolutional layer is usually referred to as ω. i Based on the sample set, the low-rank approximation problem can be expressed as:

[0053]

[0054] In the formula, ∈ (0, 1) is a predefined error tolerance, also known as the error tolerance. The proposed low-rank approximation can be applied to any form of matrix W (full rank or non-full rank) to minimize the above formula.

[0055] The optimization problem described above seeks the UV decomposition of the weight matrix W, which has two constraints: (1) the difference between the decomposed matrix and the original matrix is ​​within a predefined threshold, and (2) the rank of the decomposed matrix is ​​lower than the rank of the original matrix. By fixing the rank of UV to rank(W)-1, a feasible solution to the optimization problem can be derived as follows. By introducing the Lagrange multiplier γ, the Lagrange functions of U, V, and γ can be written as:

[0056]

[0057] Using loss function have:

[0058]

[0059]

[0060] make and Using ADMM (Alternating Direction Multiplier Method) to update U and V separately, the update at step k is as follows:

[0061]

[0062]

[0063] in Let U denote the MP inverse of the matrix, and E denote the identity matrix. Iteratively update U. k and V k Until convergence, if the constraints of equation (1) hold, matrices U and V are used as the decomposition outputs of W. Based on the above low-rank approximation algorithm, the CNN model can be sequentially decomposed layer by layer in the frequency domain using the sample set Ω. After optimizing the current layer, its output is used as the input sample for the next layer, and the next layer is decomposed in the same way, so that error accumulation between layers is avoided.

[0064] Step S5: To restore model accuracy, fine-tune the low-rank matrices U and V after decomposition. After low-rank decomposition, the CNN model can be represented by a series of low-rank matrices U and V. Similar to traditional convolutional kernel pruning methods, the compressed model is fine-tuned using the original dataset to restore its accuracy.

[0065] Unlike existing work, this invention fine-tunes the decomposed low-rank matrices U and V in the frequency domain. To facilitate model training and inference using the decomposed low-rank matrices, the derivations of the forward and backward propagation processes are presented here.

[0066] For each layer, forward propagation is calculated using only the following formula:

[0067] I·U·V=O.

[0068] To illustrate the backpropagation process, the following derivation is performed given a loss function. In the case of gradients of different components U and V.

[0069]

[0070]

[0071] Where gradient and The gradients are obtained through backpropagation in subsequent layers. Using the gradients described above, stochastic gradient descent can be used to update U and V during backpropagation. In this invention, the learning rate is set to 0.1, and is reduced to 0.1 every 10 epochs. Backpropagation is then performed using the gradient update formula described above.

[0072] Step S6: After fine-tuning, W′=UV is recombined, where the rank of W′ is necessarily lower than that of W, rank(W′)≤rank(W). Therefore, model inference based on the low-rank matrix W′ results in lower computational cost (measured in FLOPs).

[0073] Step S7, let W = W′. Repeat steps S4-S6 until the desired compression ratio (measured in FLOPs) is met or the maximum number of iterations is reached. Users can also specify the number of parameters as the final computational requirement, thus stopping iteration when the given number of parameters is reached. Iterative execution is for further compression and matrix factorization; the rate of decomposition during iteration is controlled by adjusting the error tolerance ε.

[0074] Step S8: Deploy the output neural network model in an image classification application to classify and recognize the image dataset. The output neural network model retains only the decomposed matrices U and V, which features minimal accuracy loss, fast inference speed, and low memory and disk usage.

[0075] S9. Using the decomposed convolutional neural network matrix obtained above as a replacement for the original convolutional neural network. Using the image processing from S1, the image is used as the input feature map of the decomposed convolutional neural network, enabling the network to classify the image. The output after classification is a set of one-shot vectors. The maximum value in this vector represents the category with the highest probability of being output by the neural network at that element's corresponding position. Therefore, this category with the highest probability is used as the output classification of the convolutional neural network.

[0076] This invention transforms the image processing of convolutional neural networks into a matrix multiplication method based on low-rank decomposition. Each kernel of the convolutional network is subjected to discrete cosine transform to obtain a frequency domain matrix. Then, all matrices are rearranged according to the original parameter matrix order to obtain a frequency domain matrix representation. This matrix representation has the characteristics of concentrated low-frequency components occupying the majority of energy, and concentrated high-frequency components occupying a small amount of energy. Based on this characteristic, high-frequency components are compressed.

[0077] The matrix above is compressed using a low-rank decomposition method to obtain the decomposed matrix. The decomposed matrix has a lower rank and fewer parameters, thus yielding the decomposed convolutional neural network of the target.

[0078] This method can ultimately be demonstrated as follows: shallow convolutional layers extract features to obtain the target's range, location, and edge information; deeper layers further extract the image content to obtain the image's color and category. The final output is a one-shot matrix, which is used to determine the category.

[0079] This invention demonstrates that model retraining and inference can be performed based on the decomposed frequency domain matrix, significantly reducing the number of model parameters with negligible accuracy loss and substantially lowering inference costs. Deploying this method in image classification applications can improve task processing speed and reduce the memory and CPU runtime required for program execution.

[0080] Based on the same inventive concept, this application also provides a computer device, including: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the programs, when executed by the processors, implement the steps of the image classification acceleration method based on frequency domain decomposition neural networks as described above.

[0081] Based on the same inventive concept, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the image classification acceleration method based on frequency domain decomposition neural network as described above.

[0082] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware, and the program can be stored in a computer-readable storage medium. In the context of this invention, the computer-readable medium can be considered tangible and non-transitory. Non-limiting examples of non-transitory tangible computer-readable media include non-volatile memory circuits (e.g., flash memory circuits, erasable programmable read-only memory circuits, or masked read-only memory circuits), volatile memory circuits (e.g., static random access memory circuits or dynamic random access memory circuits), magnetic storage media (e.g., analog or digital magnetic tape or hard disk drives), and optical storage media (e.g., CDs, DVDs, or Blu-ray discs), etc.

[0083] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. It may include one or more intelligent hardware devices, such as a central processing unit (CPU), a microcontroller, an application-specific integrated circuit (ASIC), etc. The processor may also include multiple processors, such as general-purpose / application processors, DSPs, modem processors, video processors and / or sensor processors, graphics processing units (GPUs), etc. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0084] Furthermore, although the operations are described in a specific order, this should be understood as requiring that such operations be performed in the specific order shown or in sequential order, or requiring that all illustrated operations be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the invention. Certain features described in the context of individual embodiments may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.

[0085] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all fall within the protection scope of the present invention.

Claims

1. An image classification acceleration method based on frequency domain decomposition neural network, characterized in that, Includes the following steps: The original image is converted into an RGB image with 3 channels, and the RGB image is scaled to the same pixel size that the input feature map of the convolutional neural network can accommodate. An image is input into a pre-trained lightweight neural network model for image classification and recognition, yielding the image classification and recognition result. The lightweight neural network model is obtained by frequency domain decomposition and lightweighting of the neural network. Specifically, the lightweight neural network model is obtained through the following methods: S1. Pre-train the neural network model based on the image dataset to obtain a complete neural network model; S2. Use discrete cosine transform to decompose the pre-trained neural network model in the frequency domain, use a series of frequency domain weight matrices W to represent the convolutional layer, and perform sparsification retraining on the neural network model in the frequency domain to obtain the sparsified weight matrix W. S3. For each weight matrix W, select a sample subset Ω of the image dataset and apply a sample-based low-rank approximation algorithm to decompose the weight matrix W into low-rank matrices U and V under an error tolerance ε. S4. After fine-tuning the low-rank matrices U and V, recombine them to obtain a new weight matrix W' = UV; S5. Let W = W' and repeat steps S3-S4 until the iteration stopping condition is met.

2. The image classification acceleration method based on frequency domain decomposition neural network according to claim 1, characterized in that, In step S1, the neural network model is pre-trained using the learning rate decay and stochastic gradient descent optimization methods.

3. The image classification acceleration method based on frequency domain decomposition neural network according to claim 1, characterized in that, In step S2, the pre-trained neural network model is decomposed in the frequency domain using discrete cosine transform, and the convolutional layer is represented by a series of frequency domain weight matrices: A convolutional layer includes c out There are n convolutional kernels, each with a c of size d × d. in Each convolutional kernel represents the convolutional layer as a 4D tensor. The input is a three-dimensional tensor Where h in ×w in It is the size of the input feature map, and the output of the convolutional layer is represented as... Where h out ×w out It is the dimension of the output feature map. The convolution operation is represented in the spatial domain as... Where mat(·) is the matrix form of the tensor; W, I, and O are respectively and The corresponding frequency domain representation, for convolutional layers, applies DCT to flatten each d×d convolutional kernel into a vector, and groups all vectors in the layer to form a frequency weight matrix. Similarly, DCT is applied to the input tensor to generate its frequency domain representation. After DCT transformation, Where D is an orthogonal matrix. After DCT transformation, 4. The image classification acceleration method based on frequency domain decomposition neural network according to claim 3, characterized in that, In step S2, sparse retraining of the neural network model in the frequency domain includes: During forward propagation, the output of each layer is calculated by the following formula: In backpropagation, the gradient used to update W is calculated using the following formula: in It is the objective function, (·) ij J represents the matrix element at position (i,j). mn It is a single-element matrix where the element at position (m,n) is 1, and the elements at other positions are 0; The objective function is: in is the loss function of the original deep model, D(model) is the weight of the model in the frequency domain, ‖·‖1 is the l1 regularization that forces the weights close to zero, and λ is the hyperparameter that balances sparsity.

5. The image classification acceleration method based on frequency domain decomposition neural network according to claim 1, characterized in that, Step S3 includes: Obtain a sample subset Ω = {ω1, ω2, ...} of the image dataset. Based on this sample subset, establish a low-rank approximation problem for finding the UV decomposition of the weight matrix W, as follows: In the formula, ∈ (0,1) is a predefined error tolerance, ω i is the input to the convolutional layer; st represents the constraints. The first constraint states that the difference between the decomposed matrix and the original matrix is ​​within a predefined threshold, and the second constraint states that the rank of the decomposed matrix is ​​lower than the rank of the original matrix. The feasible solution to the optimization problem is derived by fixing the rank of UV to rank(W)-1. By introducing the Lagrange multiplier γ, the Lagrange functions in terms of U, V, and γ are: Using loss function have: make and Using the alternating direction multiplier method to update U and V separately, the update at step k is as follows: in Let U denote the MP inverse of the matrix, E denote the identity matrix, and U be iteratively updated. k and V k Until convergence, if the constraints of equation (1) hold, matrices U and V are the decomposition outputs of W.

6. The image classification acceleration method based on frequency domain decomposition neural network according to claim 4, characterized in that, In step S4, forward propagation is calculated using the following formula: I·U·V=0; during backward propagation, U and V are updated using stochastic gradient descent, where the gradient expression is as follows: Where gradient and It is obtained by backpropagation from subsequent layers.

7. The image classification acceleration method based on frequency domain decomposition neural network according to claim 1, characterized in that, In step S5, the iteration stopping condition is reaching the required compression ratio or reaching the maximum number of iterations.

8. A computer device, characterized in that, include: One or more processors; Memory; as well as One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the steps of the image classification acceleration method based on frequency domain decomposition neural networks as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the image classification acceleration method based on frequency domain decomposition neural network as described in any one of claims 1-7.

Images