A BTD compressed deep convolutional network construction method, device, and storage medium
By constructing a deep convolutional network using the BTD compression method, and by utilizing constrained iterative optimization and BTD decomposition, the problems of large number of parameters and high computational complexity of deep convolutional network models are solved, enabling efficient deployment and performance maintenance on edge devices.
Patent Information
- Application Number
- CN202410377286.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-29
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-03-29
AI Technical Summary
Existing deep convolutional network models have a large number of parameters and high computational complexity, making them difficult to deploy on resource-constrained edge devices.
We employ a BTD-compressed deep convolutional network construction method, which combines constrained iterative optimization algorithm and optimal rank selection method with tensor decomposition in the form of BTD decomposition to compress redundant information in the model, thereby reducing the number of parameters and computational complexity.
While ensuring model performance, efficient model compression was achieved, laying the foundation for deployment on edge devices and improving the algorithm's decomposition and generalization capabilities.
Smart Images

Figure CN118194953B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of battery capacity division technology, and specifically to a BTD compressed deep convolutional network construction method, device, and storage medium. Background Art
[0002] In recent years, with the rapid development of software algorithms and hardware computing power, people have gradually realized the importance of neural networks, and network models represented by deep convolutional networks (CNN) have shown exponential growth. Especially with the help of many of the latest technologies, it is easy to train models with thousands of layers on powerful GPUs or TPU clusters, and they have been successfully applied to many artificial intelligence fields.
[0003] Although great breakthroughs have been made in the application of CNN, its shortcomings are still very obvious: as network performance continues to improve, the number of layers is also increasing, and the CNN architecture is becoming more and more complex, resulting in an increase in the number of model parameters and the use of more operating memory. It is highly dependent on high-computing devices, which seriously limits the deployment of high-performance CNN models on resource-constrained edge devices. Specifically,
[0004] Many redundant parameters: Related studies have shown that the increase in the number of network layers allows a lot of information to be reused. Although this improves performance, it greatly increases the space occupancy rate, and the weights of more than half of the parameters are extremely low, resulting in a large number of redundant parameters.
[0005] Large degree of redundant calculation: Due to information reuse, the calculation complexity increases sharply, but because the weights of many parameters are low, the calculation results will not be fully utilized, and the computing resources are seriously wasted.
[0006] Limited model deployment: Lightweight mobile devices do not support large amounts of running memory space and excellent computing device resources, which means that high-performance models cannot be quickly applied to mobile devices to empower various industries.
[0007] Since network compression can effectively compress redundant parameters in the network to reduce the number of parameters and computational complexity, it has become a new research hotspot for many scholars at home and abroad. Summary of the invention
[0008] The purpose of the present invention is to provide a BTD compressed deep convolutional network construction method, device, and storage medium. Through constrained iterative compression, the purpose of maximizing the reduction of parameters and computational complexity while ensuring the performance of the network model is achieved, laying the foundation for deploying the model to mobile devices.
[0009] To achieve the above object, the present invention provides the following technical solutions:
[0010] A BTD compressed deep convolutional network construction method, the method comprises the following steps:
[0011] Taking the untrained deep convolutional neural network as input, it enters the constrained framework for compression;
[0012] The randomly initialized model is trained with a constrained iterative optimization algorithm, and the low-rank characteristic with the optimal rank is used as a constraint condition. The minimum loss of the constrained compressed model is obtained as the objective function, and a constrained iterative loop is entered to obtain a low-rank characteristic model.
[0013] The low-rank characteristic model is fine-tuned until the low-rank characteristic model meets the output condition, and an optimal compression model is output to perform compression through the optimal compression model.
[0014] In the above technical solution, the randomly initialized model is trained with a constrained iterative optimization algorithm, including:
[0015] Train the network on a randomly initialized model;
[0016] The optimal rank set is calculated according to the optimal rank selection algorithm;
[0017] Combining the optimal rank set, compressing the training model to calculate the weight of the compressed model;
[0018] The training model and the compression model are fine-tuned, the low-rank property with the optimal rank is used as a constraint condition, the minimum loss of the constrained compression model is obtained as the objective function, and a constrained iteration loop is entered.
[0019] In the above technical solution, the network training of the randomly initialized model includes:
[0020] Determine the type of training model, and determine the training data set, learning criteria, and optimization algorithm based on the type;
[0021] The training model is trained by learning criteria and an optimization algorithm to obtain the weight of the training model.
[0022] In the above technical solution, the optimal rank set is calculated according to the optimal rank selection algorithm, including:
[0023] Determine observation points, probability distribution and confidence through Gaussian process, and calculate the potential value of the objective function;
[0024] The next sampling point is calculated through the EI sampling function, the optimal value of the objective function is determined, and the optimal rank set is formed.
[0025] In the above technical solution, the training model is compressed in combination with the optimal rank set to calculate the weight of the compressed model, including:
[0026] Introduce regularization and gradually update the training low-rank feature model;
[0027] The low-rank characteristic model is associated with the mapping relationship, and an optimized low-rank characteristic model is obtained by compression using a tensor decomposition method in the form of BTD decomposition.
[0028] In the above technical solution, the training model and the compression model are fine-tuned, the low-rank property with the optimal rank is used as a constraint condition, the minimum loss of the constrained compression model is obtained as the objective function, and a constraint iteration loop is entered, including:
[0029] Fine-tune the compression model, calculate the fine-tuned network compression model, and update the dual multiplier factor;
[0030] The low-rank property with the optimal rank is used as a constraint condition, and the minimum loss of the constrained compression model is obtained as the objective function. The constraint iteration loop is entered to calculate the weight of the training model, the weight of the compression model, and the dual multiplier factor;
[0031] The loop ends when the training model and the compression model meet the output conditions;
[0032] Fine-tuning the low-rank characteristic model until the low-rank characteristic model meets the output condition, and outputting the optimal compression model, including:
[0033] The compression model is fine-tuned until the error requirement is met, and an optimal compression model is output.
[0034] In the above technical solution, the fine-tuned compression model is calculated by the following formula:
[0035]
[0036] Where n represents the number of sample pairs, h(·) represents the functional relationship between the output features and the input features, represents the training sample pair, L(·,·) represents the network loss function, represents the compressed model after fine-tuning.
[0037] In the above technical solution, until the training model and the compression model meet the output condition and the optimal compression model is output, it also includes:
[0038] The training model and the compression model meet the output conditions, and the loop ends;
[0039] Fine-tune the compression model until the error requirement is met, and output an optimal compression model;
[0040] The output condition is that the weight of the training model, the weight of the compression model and the dual multiplier factor are within a preset threshold, or the number of iterations is completed;
[0041] The error requirement is that the approximate error between the compression model and the training model is within a preset range.
[0042] The present application also provides a device, including multiple processors, a memory, and a computer program stored in the memory and executable on the processor. When the multiple processors execute the computer program, a BTD compressed deep convolutional network construction method as described above is implemented.
[0043] The present application also provides a storage medium having a computer program stored thereon, and when the computer program is executed, the method for constructing a BTD compressed deep convolutional network as described above is implemented.
[0044] Compared with the prior art, the present invention has the following beneficial effects:
[0045] The present application provides a method, device, and storage medium for constructing a BTD compressed deep convolutional network. To address the problems of large number of existing network parameters, high computational complexity, and difficulty in deployment in edge devices, an advanced constrained iterative optimization algorithm is adopted to make the training gradually tend to low-rank characteristics. The optimal rank selection method is further adopted in the constrained iterative compression framework to automatically select two ranks along the input and output channels in the multi-rank decomposition to form a rank set, thereby achieving more efficient compression of the redundant information of the model. The tensor decomposition method in the form of BTD decomposition is guided by the selected optimal rank set, which increases the decomposition and generalization capabilities of the algorithm while ensuring the accuracy of the model. Therefore, the present invention has excellent compression capabilities and lays a good foundation for the next step of model packaging and deployment. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 It is a schematic diagram of a flow chart of an embodiment of the present invention;
[0047] Figure 2 A schematic diagram of a low-rank characteristic model training process according to an embodiment of the present invention;
[0048] Figure 3 A schematic diagram of a network training process according to an embodiment of the present invention;
[0049] Figure 4 A schematic diagram of a process for the most preferred rank of an embodiment of the present invention;
[0050] Figure 5 It is a schematic diagram comparing the BTD decomposition method with the Tucker and CP decomposition methods according to an embodiment of the present invention. DETAILED DESCRIPTION
[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0052] This application provides a BTD compressed deep convolutional network construction method, which, through constrained iterative compression, maximizes the reduction of parameters and computational complexity while ensuring the performance of the network model, laying the foundation for deploying the model to mobile devices.
[0053] Specifically, Figure 1 As shown, the method comprises the following steps:
[0054] Step S1: An untrained deep convolutional neural network is used as input and enters a constrained framework for compression.
[0055] Generally, tensor decomposition compression neural networks require a pre-trained model as input to ensure the performance of the compressed model. In this application, in order to reduce the compression time, the input end is a non-pre-trained deep convolutional network model, and the first network learning process is skipped after entering the constraint framework.
[0056] Step S2, perform training including a constrained iterative optimization algorithm on the randomly initialized model, use the low-rank characteristic with the optimal rank as a constraint condition, obtain the minimum loss of the constrained compression model as the objective function, enter the constrained iterative loop, and obtain the low-rank characteristic model.
[0057] like Figure 2 As shown, training is performed in a constrained framework, and through iterative cyclic optimization, the compressed optimal low-rank model is finally obtained. This patent introduces regularization to gradually update the randomly initialized model to train low-rank characteristics. The low-rank characteristic model is related to the decomposition mapping. The specific mapping model will be included in the fine-tuning BTD low-rank characteristic model for calculation and iterative fine-tuning. Finally, the optimized low-rank characteristic model is obtained through the set threshold or number of iterations.
[0058] In the field of compression model research, the constrained iteration mode is applied to the BTD decomposition model method. The framework of the method is: assuming that when compressing an M-layer tensor network, the weight of the network model to be compressed is a fourth-order tensor In order to obtain the optimal compression model, the low-rank property with the optimal rank is first added as a constraint to the training process, and the minimum loss of the constrained model is obtained as the objective function. The formula is shown in (2):
[0059]
[0060] where r * represents the desired rank of the layer, that is, along the input and output channel dimensions, two analytical solutions are selected by the BayesOpt method under the Bayesian rule As the expected rank, rank(W)≤r * It indicates that the rank of the training model weight W under the constraint should be less than or equal to the desired optimal rank of each core tensor of BTD.
[0061] Formula (2) has been expressed as a problem with constraints. To facilitate the solution, rank(W)≤r * The corresponding training model weight W is written in the form of a set e = {W|rank(W)≤r *}, and introduce variables and e, the corresponding function f(·) can be expressed as follows:
[0062]
[0063] Combined with formula (3), formula (2) can be rewritten as:
[0064]
[0065] The network model weight W is used to represent the training model weight, Represents the compression model weight. In order to optimize and solve the compression model weight and the training model weight simultaneously, the above non-convex optimization problem is transformed into an unconstrained convex optimization problem. Similarly, the proportional dual form of the above problem is introduced into the augmented Lagrangian to obtain Formula 5:
[0066]
[0067] Where δ is the dual multiplier factor and υ is the quadratic penalty parameter. It belongs to the low-rank characteristic model and is associated with the expected value. The weight W of the training model and the weight of the compression model can be iteratively converted Rewritten in terms of the iteration steps:
[0068]
[0069]
[0070]
[0071] Where i is the number of iteration steps in the setting. Formula (6-8) shows that the step-by-step solution of Formula 4 is transformed into two models W and During the solution process, the model of the previous iteration is selected to obtain another model, and the dual multiplier factor δ of the previous iteration is used in each iteration to add the difference between the two models in the current iteration step to optimize the dual multiplier factor δ.
[0072] The corresponding training model weight W and compression model weight are transformed into two problems to be solved, so the network training, optimal rank, network compression, and single-step fine-tuning process are generated, and these processes are iterated one by one. Specifically, the following steps are included:
[0073] Network training: Perform network training on randomly initialized models;
[0074] Best rank: The best rank set is calculated according to the best rank selection algorithm;
[0075] Network compression: combining the optimal rank set, compressing the training model to calculate the weight of the compressed model;
[0076] Single-step fine-tuning: fine-tune the training model and the compression model, take the low-rank property with the optimal rank as a constraint condition, obtain the minimum loss of the constrained compression model as the objective function, and enter the constrained iteration loop.
[0077] Among them, Figure 4 As shown, network training is also the process of solving the problem of training the weight W of the model. It includes:
[0078] Determine the type of training model, and determine the training data set, learning criteria, and optimization algorithm based on the type;
[0079] The training model is trained by learning criteria and an optimization algorithm to obtain the weight of the training model.
[0080] Furthermore, the training process can select suitable learning criteria and optimization algorithms for deep convolutional networks with different functions. For example, if the model is an image classification network, the code can select a more matching cross entropy loss and momentum cycle learning rate with hot restart as the learning criterion and optimization algorithm. If the model is a super-resolution network, the code can select the mean square loss and the Adam algorithm as the learning criterion and optimization algorithm to obtain higher training model performance.
[0081] When training the model, it is necessary to find the optimal solution for W, so it is necessary to decompose formula (5) into a problem with W as an independent variable, such as formula (9):
[0082]
[0083] Where W is the weight of the training model. Since both terms on the right side of the formula are differentiable, the code can be used to automatically select and optimize the algorithm to solve W. i+1During the process, the code automatically selects an adaptive learning criterion to make the convergence faster and easier to escape from sharp local optimal points or saddle points.
[0084] like Figure 3 As shown, the optimal rank selection method uses the Bayesian rule rank selection method with the theoretical optimal solution. Compared with the traversal rank selection and empirical method, the selection of rank makes the network structure information less lost and the time is shorter. It improves the compression efficiency and can guide the network compression to obtain a better compression model. It includes the following steps:
[0085] Determine observation points, probability distribution and confidence through Gaussian process, and calculate the potential value of the objective function;
[0086] Given N observation points, use f(x 1:N )=[f(x1),...,f(x N )] as the objective function, by solving the prior distribution P(f(x 1:N )), and obtain the posterior P(f(x * )|f(x 1:N )). GP regression uses a smooth Màtern 5 / 2 covariance function, and the corresponding prior distribution is shown in formula (10).
[0087]
[0088] Where D represents the dimension, θ 0:D [θ0,...,θ D ] is a hyperparameter, the mean μ0 and covariance Σ0 can be expressed as formula (11),
[0089]
[0090] The posterior distribution is calculated through the prior distribution, as shown in formula (12):
[0091]
[0092] The next sampling point is calculated through the EI sampling function, the optimal value of the objective function is determined, and the optimal rank set is formed.
[0093] As shown in formulas (13) and (14):
[0094] EI k (x * )=E k [max(f(x best )-f(x * ),0)] (13)
[0095] x n+1 =argmaxEI k (x *) (14)
[0096] The EI function obtains the expected value of the max function under the posterior distribution, pre-sets the number of times to search for the optimal rank, and sets the last expected value x n+1 As the optimal solution x of f(x) best The EI function can make the current point x best is the point where the loss of f(x) is the smallest, and the EI function is a closed loop under GP regression. Therefore, the sampling function EI can be used to determine the optimal point of the objective function f(x). After all the cycles, the global analytical solution is obtained as the final expectation r * Form a ranked set.
[0097] After determining the optimal rank set, network compression is performed. Specifically, regularization is introduced to gradually update the training low-rank feature model;
[0098] The low-rank characteristic model is associated with the mapping relationship, and an optimized low-rank characteristic model is obtained by compression using a tensor decomposition method in the form of BTD decomposition.
[0099] The network compression process is to solve the optimal problem of compression model weights To solve the compression model weight T, fix the calculated W i+1 , set only Formula (15) is in the form:
[0100]
[0101] in For a model with low-rank characteristics, in solving During the process, the optimal rank set will be used to guide compression to ensure that important information with a relatively large weight ratio will be retained after the model is compressed.
[0102] The solution process is related to the rank, so the process is not differentiable. The relationship is established through the mapping relationship Π(·):
[0103]
[0104] Π e (·) represents the projection of the compression method in the set e, where the compression method is selected to have a tensor decomposition BTD form, and the number of core tensors is pre-set to K, such as Figure 5 As shown in Figure 3, BTD can achieve better decomposition ability than the classic tensor compression Tucker and CP decomposition methods.
[0105] In order to reduce the approximation error and further restore the performance of the compressed network, this method needs to fine-tune the low-rank feature model. The low-rank feature with the optimal rank is used as a constraint condition, and the minimum loss of the constrained compressed model is obtained as the objective function, and a constraint iteration loop is entered, including:
[0106] Fine-tune the compression model, calculate the fine-tuned network compression model, and update the dual multiplier factor;
[0107] The low-rank property with the optimal rank is used as a constraint condition, and the minimum loss of the constrained compression model is obtained as the objective function. The constraint iteration loop is entered to calculate the weight of the training model, the weight of the compression model, and the dual multiplier factor;
[0108] The loop ends when the training model and the compression model meet the output conditions.
[0109] By constructing the network structure after BTD decomposition, the corresponding input-output operation relationship is obtained:
[0110]
[0111] Where n represents the number of sample pairs, h(·) represents the functional relationship between the output features and the input features, represents the training sample pair, L(·,·) represents the network loss function, represents the compressed model after fine-tuning.
[0112] After a fixed number of iterations, the weight W of the training model and the weight of the compression model are obtained. Iterative solution of the problem. When solving the weight W of each training model and the weight of the compressed model After the problem is solved, the dual multiplier factor is updated by formula (8), which is the weight W of the training model and the weight of the compression model. The dual multiplier factor obtains the optimal training model and compression model after the set threshold or number of iterations, thereby jumping out of the loop.
[0113] Step S3: fine-tune the low-rank characteristic model until the low-rank characteristic model meets the output condition, and output the optimal compression model to perform compression through the optimal compression model.
[0114] The approximation degree between the compression model and the training model is verified, and the compression model is fine-tuned a few times, and the network performance is compared with the training model to meet the error requirements. If the approximation error is within a reasonable range, the optimal compression model is output. The error requirement is that the approximation error between the compression model and the training model is within a preset range.
[0115] This application can be used for efficient compression of different network models. As shown in the following embodiment, it is a use case on an image classification network; in order to better run and test the performance and compression rate of the algorithm on the relevant network, the relevant configuration and some parameters of the computer are required.
[0116] For the image classification network, the relevant hardware environment used in the present invention is configured as a Gen Intel (R) Core (TM) I7-11700 processor and an NVIDIA GeForce RTX3080Ti (12G) graphics card. The software environment configuration used includes a deep learning framework of PyTorch v1.7.1+cu110, corresponding to Python version v3.6.5, a tensor runtime library tensorly version v0.7.0, and a tensor decomposition library tntorch version v1.0.1. The relevant parameter configuration is shown in Table 1, wherein an untrained initialization model is used to save time and steps, and the epoch is set to 40, the number of iterative optimizations is set to 20, the learning criterion is cross entropy loss, and the optimization algorithm uses a momentum cyclic learning rate algorithm with hot restart, the momentum is 0.9, and the initial learning rates are 0.005 and 0.006 respectively. The introduction of the cyclic learning rate makes the epoch 0 of the learning step in each iterative optimization the initial learning rate, and it decays exponentially with the increase of epoch. The penalty coefficient introduced by the LC framework is set to 0.5e-3, and the penalty coefficient is increased by The storage compression ratio ρ of the original network in the comparative experiment Storage And floating point calculation compression ratio ρ Flops Assumed to be 1, Flops in this article is the compression rate of each method for one addition and multiplication, that is:
[0117] ρ Storage =Compressed model parameter amount / original model parameter amount,
[0118] ρ Flops =Compressed model Flops / Original model Flops.
[0119]
[0120]
[0121] The CIFAR10 dataset is used for the provided cases. This dataset is a color image library that is closer to universal animals or objects. It is a small dataset for identifying universal objects. It is currently mainly used for image classification tasks. The dataset is mixed with a lot of noise, and the proportions, features, distances, and angles of real animals and objects in reality are all different, which brings great difficulties to identification and can better test the performance of the network.
[0122] According to this example, by compressing the classification network ResNet and VGG network, this method can obtain an efficient analytical solution, thereby achieving a better compression rate for network redundant parameters and computational complexity. This method ultimately achieves optimal compression while ensuring model performance.
[0123] This application solves the problems of large number of parameters and large amount of calculation in existing deep convolutional networks. Aiming at the problems of large number of parameters, high computational complexity and difficulty in deploying in edge devices, the present invention adopts advanced constrained iterative optimization algorithm to make the training gradually tend to low-rank characteristics, and further introduces BayesOpt rank selection method in the constrained iterative compression framework, automatically selects two ranks along the input and output channels in the multi-rank decomposition, and realizes more efficient compression of the redundant information of the model, and adopts tensor decomposition method in the form of BTD decomposition, which increases the decomposition and generalization ability of the algorithm while ensuring the accuracy of the model, and has excellent compression ability, laying a good foundation for the next step of model packaging and deployment.
[0124] Based on the same inventive concept, the present invention also provides a storage medium, which stores a computer program, and when the computer program is executed by a processor, it implements the steps of the aforementioned BTD compressed deep convolutional network construction method.
[0125] Among them, the computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form, etc. Computer readable media may include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in computer readable media can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer readable media do not include electric carrier signals and telecommunication signals.
[0126] The present invention can be used in many general or special computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.
[0127] The present application also provides an electronic device, including a memory, multiple processors, and a program stored in the memory, wherein the program is configured to be executed by the processor, and when the multiple processors execute the program, the steps of the above-mentioned BTD compressed deep convolutional network construction method are implemented.
[0128] In addition, the present invention also provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the aforementioned method for controlling the function duration of an intelligent clothes drying machine are implemented. The present invention can be used in many general or special computing system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.
[0129] The device in this embodiment and the method in the aforementioned embodiment are two aspects based on the same inventive concept. The method implementation process has been described in detail above, so those skilled in the art can clearly understand the structure and implementation process of the system in this embodiment based on the aforementioned description. For the sake of brevity of the specification, they will not be repeated here.
[0130] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A method for constructing a BTD compressed deep convolutional network, characterized in that: The method comprises the following steps: An untrained deep convolutional neural network is used as an input end and enters a constrained framework for compression; the untrained deep convolutional neural network is applied to an image classification network; The randomly initialized model is trained with a constrained iterative optimization algorithm, and the low-rank characteristic with the optimal rank is used as a constraint condition. The minimum loss of the constrained compressed model is obtained as the objective function, and a constrained iterative loop is entered to obtain a low-rank characteristic model. Fine-tune the low-rank characteristic model until the low-rank characteristic model meets the output condition, and output an optimal compression model to perform compression through the optimal compression model; Training of a randomly initialized model using a constrained iterative optimization algorithm includes: Perform network training on the randomly initialized model to obtain a training model; The optimal rank set is calculated according to the optimal rank selection algorithm; Combining the optimal rank set, compressing the training model to calculate the weight of the compressed model; Fine-tune the training model and the compression model, use the low-rank property with the optimal rank as the constraint condition, find the minimum loss of the constrained compression model as the objective function, and enter the constraint iteration loop; The fine-tuned compressed model is calculated using the following formula: Where n represents the number of sample pairs, h(·) represents the functional relationship between the output features and the input features, represents the training sample pair, L(·,·) represents the network loss function, T * represents the compressed model after fine-tuning.
2. A BTD compressed deep convolutional network construction method according to claim 1, characterized in that: Train the network on a randomly initialized model, including: Determine the type of training model, and determine the training data set, learning criteria, and optimization algorithm based on the type; The training model is trained by learning criteria and an optimization algorithm to obtain the weight of the training model.
3. The method for constructing a BTD compressed deep convolutional network according to claim 1, characterized in that: The optimal rank set is calculated according to the optimal rank selection algorithm, including: Determine observation points, probability distribution and confidence through Gaussian process, and calculate the potential value of the objective function; The next sampling point is calculated through the EI sampling function, the optimal value of the objective function is determined, and the optimal rank set is formed.
4. A BTD compressed deep convolutional network construction method according to claim 2, characterized in that: Combining the optimal rank set, compressing the training model to calculate the weight of the compressed model includes: Introduce regularization and gradually update the training low-rank feature model; The low-rank characteristic model is associated with the mapping relationship, and an optimized low-rank characteristic model is obtained by compression using a tensor decomposition method in the form of BTD decomposition.
5. The method for constructing a BTD compressed deep convolutional network according to claim 1, characterized in that: The training model and the compression model are fine-tuned, the low-rank property with the optimal rank is used as a constraint condition, the minimum loss of the constrained compression model is obtained as the objective function, and a constraint iteration loop is entered, including: Fine-tune the compression model, calculate the fine-tuned network compression model, and update the dual multiplier factor; The low-rank property with the optimal rank is used as a constraint condition, and the minimum loss of the constrained compression model is obtained as the objective function. The constraint iteration loop is entered to calculate the weight of the training model, the weight of the compression model, and the dual multiplier factor; The loop ends when the training model and the compression model meet the output conditions; Fine-tuning the low-rank characteristic model until the low-rank characteristic model meets the output condition, and outputting the optimal compression model, including: The compression model is fine-tuned until the error requirement is met, and an optimal compression model is output.
6. A method for constructing a BTD compressed deep convolutional network according to claim 5, characterized in that: Until the training model and the compression model meet the output conditions, the optimal compression model is output, which also includes: The training model and the compression model meet the output conditions, and the loop ends; Fine-tune the compression model until the error requirement is met, and output the optimal compression model; the output condition is that the weight of the training model, the weight of the compression model, and the dual multiplier factor are within a preset threshold, or the number of iterations is completed; The error requirement is that the approximate error between the compression model and the training model is within a preset range.
7. A device, characterized in that: The invention comprises a plurality of processors, a memory and a computer program stored in the memory and executable on the processor. When the plurality of processors execute the computer program, a method for constructing a BTD compressed deep convolutional network as described in any one of claims 1 to 6 is implemented.
8. A storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed, a method for constructing a BTD compressed deep convolutional network as described in any one of claims 1-6 is implemented.
