A lightweight change detection method combining quantization, pruning and deep learning

Abstract

The application discloses a light-weight change detection method combining quantization, pruning and deep learning, which comprises the following steps: quantizing network parameters of a pre-trained VGG network to obtain a pure-integer VGG network; freezing part of weight parameters of the pure-integer VGG network, training weight parameters of unfrozen network convolution layers to obtain a VGG network model; inputting two types of image samples with maximum / minimum change probability selected by a block mean-based affinity matrix method into the VGG network model for pure-integer training, and pruning the network in the filter dimension to obtain a trained VGG network model; inputting two types of images for testing into the trained VGG network model to obtain a difference image, and classifying the difference image into a final change image by using the maximum inter-class variance method. The application applies the quantization and pruning method to the VGG network, and thus precise detection of image change can be realized.

Description

Technical Field

[0001] This invention relates to the field of multi-source remote sensing image change detection technology, and in particular to a lightweight change detection method that combines quantization, pruning and deep learning. Background Technology

[0002] Change detection is of great significance in both military and civilian applications, enabling immediate identification of affected areas and guidance for rescue efforts after disasters such as earthquakes and floods. In recent years, many scholars have proposed various methods for change detection in heterogeneous remote sensing images, such as homogeneous transformation methods based on K-nearest neighbors and change detection methods based on K-nearest neighbors blocks. However, these methods rely on semi-supervised region selection and expert-based design, and are relatively slow and memory-intensive, further limiting their practical application. With the development of artificial intelligence and deep learning, methods based on deep neural networks are increasingly used in this task. Deep learning methods can produce good detection accuracy, but in many cases, they are time-consuming, memory-intensive, and not lightweight. Therefore, many researchers focus on lightweight change detection, aiming to propose methods with faster detection speeds and lower memory consumption. One method based on Siamese VGG neural networks is currently a very lightweight approach. However, it is well known that VGG still has significant room for further lightweighting. Existing methods suffer from long detection times and high memory consumption.

[0003] Existing homogeneous transformation methods based on deep learning require floating-point calculations when training neural networks. However, the parameters of the original neural network typically contain a large amount of redundancy, which means that floating-point calculations and a large number of parameters will eventually lead to overfitting of the model and reduce detection accuracy. At the same time, a huge number of parameters will also slow down the running time and consume a lot of storage space. Summary of the Invention

[0004] The present invention aims to at least partially solve one of the technical problems in the related art.

[0005] To address this, this invention proposes a lightweight change detection method that combines quantization, pruning, and deep learning. This method is suitable for lightweight change detection tasks in heterogeneous remote sensing images. Furthermore, the introduction of a training-while-pruning approach further reduces network storage during training.

[0006] Another objective of this invention is to propose a lightweight change detection device that combines quantization, pruning, and deep learning.

[0007] To achieve the above objectives, this invention proposes a lightweight change detection method that combines quantization, pruning, and deep learning, comprising:

[0008] The network parameters of the pre-trained VGG network are quantized to obtain a VGG network with pure integers;

[0009] Freeze some weight parameters of the VGG network with pure integers, and train the weight parameters of the unfrozen convolutional layers of the network to obtain the VGG network model;

[0010] The two types of image samples with the maximum / minimum change probability selected by the affinity matrix method based on block mean are input into the VGG network model for pure integer training, and the network is pruned in the filter dimension to obtain the trained VGG network model.

[0011] The two types of images used for testing are input into the trained VGG network model to obtain the difference images. The difference images are then binary classified into the final change images using the maximum inter-class variance method.

[0012] The lightweight change detection method combining quantization, pruning, and deep learning in this invention embodiment may also have the following additional technical features:

[0013] In one embodiment of the present invention, the weight parameters of the unfrozen network convolutional layers are trained, including the weights Conv 5-4 of the 8th convolutional layer Conv 3-4, the 12th convolutional layer Conv 4-4, and the 16th convolutional layer of the VGG network.

[0014] In one embodiment of the present invention, the two classes of image samples with the maximum / minimum change probability selected by the affinity matrix method based on block mean include:

[0015] Divide the two heterogeneous remote sensing image samples into small square blocks and calculate the mean of each block.

[0016] The distance information between each small block and other small blocks is calculated based on the mean of each small block.

[0017] Based on the distance information, the probability of change of each small block is measured using the affinity matrix formula, and the probability of change of each small block is obtained by solving the problem.

[0018] The probabilities of change are sorted, and two types of image samples are selected based on the probability sorting results; wherein, the two types of image samples include the image patch with the highest probability of change and the image patch with the lowest probability of change.

[0019] In one embodiment of the present invention, the loss function used for training the VGG network model is:

[0020]

[0021] in, This represents the Conv m-4 (m=3,4,5) output of subnetwork p (p=1,2) when the input is a cat-class sample; i, j, and k represent the coordinates of the output; where W m H m and C m This represents the width, length, and number of channels of the output matrix; α m It is the weight set.

[0022] In one embodiment of the present invention, pruning the network at the filter dimension includes:

[0023] Calculate the L1 norm of each filter in the two subnetworks of the VGG network model:

[0024]

[0025] in, It represents t p (p=1,2) The c-th filter of the l-th layer of the network. It is the number of filters in the l-th layer;

[0026] In adaptive pruning, the maximum value L of the loss function from the start of training to the time of pruning. max The loss function value L during the last pruning last The loss function value L during the backtracking check corresponding to this pruning. k Set the backtracking condition as follows:

[0027] L max -L k <θ(L max -L last )

[0028] Where θ is the tolerance for the loss function to rise.

[0029] To achieve the above objectives, another aspect of the present invention proposes a lightweight change detection device that combines quantization, pruning, and deep learning, comprising:

[0030] The network quantization module is used to quantize the network parameters of a pre-trained VGG network to obtain a VGG network with pure integers.

[0031] The parameter freezing module is used to freeze some weight parameters of the pure integer VGG network and train the weight parameters of the unfrozen network convolutional layers to obtain the VGG network model.

[0032] The network pruning training module is used to input two types of image samples with the maximum / minimum change probability selected by the affinity matrix method based on block mean into the VGG network model for pure integer training, and to prune the network in the filter dimension to obtain the trained VGG network model.

[0033] The network model output module is used to input the two types of images used for testing into the trained VGG network model to obtain the difference image, and to use the maximum inter-class variance method to classify the difference image into the final change image.

[0034] The lightweight change detection method and apparatus combining quantization, pruning, and deep learning in this invention first divides the image into blocks and calculates the average value of each block. Then, using an affinity matrix-based method, small blocks with the highest and lowest change probabilities are obtained as training samples. Subsequently, a quantization training method is used, quantizing the weights into int8 type variables at the initial stage, and using pure integer calculations to update network parameters throughout the entire network training process. Furthermore, filter pruning is performed during network training to further lightweight the network.

[0035] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0036] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0037] Figure 1 This is a flowchart of a lightweight change detection method combining quantization, pruning, and deep learning according to an embodiment of the present invention;

[0038] Figure 2 This is a basic framework diagram of lightweight change detection that combines quantization, pruning and deep learning according to an embodiment of the present invention;

[0039] Figure 3 This is a flowchart of another lightweight change detection method combining quantization, pruning, and deep learning according to an embodiment of the present invention;

[0040] Figure 4 This is a diagram illustrating the effect of change detection according to an embodiment of the present invention;

[0041] Figure 5 This is a schematic diagram of a lightweight change detection device that combines quantization, pruning, and deep learning according to an embodiment of the present invention. Detailed Implementation

[0042] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0043] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0044] A lightweight change detection method and apparatus combining quantization, pruning, and deep learning, according to an embodiment of the present invention, is described below with reference to the accompanying drawings.

[0045] Figure 1 This is a flowchart of a lightweight change detection method combining quantization, pruning, and deep learning according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes:

[0046] S1, quantize the network parameters of the pre-trained VGG network to obtain a VGG network with pure integers;

[0047] S2, freeze some weight parameters of the pure integer VGG network, and train the weight parameters of the unfrozen network convolutional layers to obtain the VGG network model;

[0048] S3. The two types of image samples with the maximum / minimum change probability selected by the affinity matrix method based on block mean are input into the VGG network model for pure integer training, and the filter dimension of the network is pruned to obtain the trained VGG network model.

[0049] S4. Input the two types of images used for testing into the output of the trained VGG network model to obtain the difference image, and use the maximum inter-class variance method to classify the difference image into the final change image.

[0050] Specifically, the overall structure of the present invention is as follows: Figure 2 As shown, this involves subnetwork t1, subnetwork t2, and the transformation solution operation f. d In this invention, the images obtained before and after the change are input into subnetworks t1 and t2 of the VGG network model for feature extraction, respectively. Then, the features extracted from the two subnetworks are input into the change solving operation f. d The changed image is obtained by solving the algorithm, where each pixel value ranges from [0,1], and the closer the value is to 1, the greater the probability of change. The overall flowchart of this invention is shown below. Figure 3The main structure of this invention is based on a method using a Siamese VGG neural network, an unsupervised training sample selection method based on block mean affinity matrix comparison, and a method for quantizing training and pruning during filter dimension training.

[0051] Specifically, the method proposed in this invention includes a training process and an inference process, both of which are based on transfer learning for training and inference. The specific flowchart is as follows: Figure 3 As shown.

[0052] Specifically, remote sensing images of the ground changes before and after the disaster were acquired, with a size of W×H, where W represents the number of rows and H represents the number of columns. A VGG neural network pre-trained on the Cifar10 natural image dataset was used.

[0053] Specifically, all parameters of the VGG network pre-trained on the Cifar10 natural image dataset are quantized from float32 to int8 to obtain a pure integer network. Most of the network weight parameters are frozen, and only the weights of the 8th convolutional layer (Conv 3-4), the 12th convolutional layer (Conv 4-4), and the 16th convolutional layer (Conv 5-4) are trained.

[0054] Furthermore, two classes of image samples with the highest / lowest change probabilities, selected using a novel affinity matrix method based on block mean, are input into the neural network for training. Network training requires selecting the two types of samples using an affinity matrix comparison method, the process of which is as follows:

[0055] 1) Divide the two heterogeneous remote sensing images into uniform square blocks. Calculate the mean of each block. Here, pre- and post- represent images before and after the incident. i represents the i-th small block.

[0056] 2) Calculate the distance between each small block and other small blocks: Specifically:

[0057]

[0058] 'm' represents the image before or after the event. The greater the distance, the weaker the correlation between the two patches. The greater the difference in this correlation between the two images, the more likely the patch is to have changed.

[0059] 3) The probability of change in each small patch is quantitatively measured based on distance. Here, the affinity matrix formula is used for measurement, and the formula is:

[0060]

[0061] Then use The probability of change for each small patch is obtained by normalization. After sorting the probabilities, the 20 image patches with the highest probability of change and the 30 with the lowest probability of change are selected and input into the network for training.

[0062] Furthermore, the loss function used for training is:

[0063]

[0064] in This represents the Conv m-4 (m=3,4,5) output of subnetwork p (p=1,2) when the input is cat-class samples (hid is the sample with the minimum change probability, and sat is the sample with the maximum change probability). i, j, and k represent the coordinates of the output. Where W... m H m and C m This represents the width, length, and number of channels of the output matrix. α m The weights are manually set to balance the importance of the outputs of the three convolutional layers.

[0065] Furthermore: Using the existing CUTLASS library to perform pure integer convolution computations on the GPU to... Figure 2 The neural network is trained using pure integers. During each int8 convolution calculation, the GPU outputs an int32 data type to prevent data overflow. Subsequent shifting and rounding operations are then performed on the int32. First, the number of effective bits occupied by the data with the largest absolute value in the entire int32 matrix is ​​evaluated. in This is the int32 output of the l-th layer. Then, this invention performs a bit right shift on each matrix. The decimal part is rounded after shifting.

[0066] Furthermore, to achieve a lightweight effect and easier practical application after pruning, this invention prunes the network at the filter dimension after training for a certain number of generations. During pruning, this invention first calculates the L1 norm of each filter in the two sub-networks:

[0067]

[0068] in It represents t p (p=1,2) The c-th filter of the l-th layer of the network. This represents the number of filters in the l-th layer. Then, the norms of the filters at each symmetrical position in the two sub-networks are summed to represent the importance of the filters at that position. Finally, the importance is ranked and N filters are subtracted. This invention uses a pruning algorithm with some adaptive capability, which includes a backtracking operation to reduce the accuracy loss caused by excessive pruning. During adaptive pruning, this invention considers three variables: the maximum value L of the loss function from the start of training to the pruning point. max The loss function value L during the last pruning last The loss function value L during the backtracking check corresponding to this pruning. k (k represents the number of iterations corresponding to this backtracking check). Excessive pruning usually leads to a significant increase in the loss function, so this invention sets the backtracking condition as follows:

[0069] L max -L k <θ(L max -L last )

[0070] Here, θ represents the tolerance for an increase in the loss function. If the condition is not met, it means that the pruned filters have little impact on detection performance, and training continues. If the condition is met, it means that important filters were pruned, leading to a significant increase in the loss function. In this case, the invention needs to perform a backtracking operation, restoring the network to its state before the last pruning and pruning an even smaller number of filters, N / 2. This integer training and pruning operation is repeated until the total number of iterations reaches 1000. Ultimately, this invention yields two quantized change detection networks adapted to remote sensing images, each containing fewer filters.

[0071] Furthermore, the two images are input into two sub-networks for inference to obtain a difference image. At this point, each pixel in the difference image represents the probability of that pixel actually changing.

[0072] Furthermore, the difference images are then binary classified into the final change images using the Otsu's method. "1" indicates change, and "0" indicates no change.

[0073] Furthermore, Table 1 shows the comparison results of change detection indicators for heterogeneous remote sensing images, as shown in Table 1:

[0074] Table 1

[0075] method Kappa Memory usage time <![CDATA[Affinity matrix + S 3 N]]> 0.4668 2045.5MiB 551.1s The method proposed in this invention 0.5112 617.1MiB 60.1s

[0076] Furthermore, the affinity matrix of the present invention +S 3 The detection effects of the N method and the detection effects of the method proposed in this invention are as follows: Figure 4 As shown.

[0077] In summary, this invention optimizes network computation (primarily convolution computation) to int8 convolution calculations, while employing a pseudo-random shifting and rounding method. Furthermore, upon reaching a specific algebraic level, it calculates the L1 norm of the filters at each position within each convolution kernel of the two sub-networks and performs filter pruning based on the L1 norm. On one hand, by using pure integers to train the neural network, the required storage for each layer's activation, weights, and weight gradients is reduced from 32 bits to at least 8 bits, saving four times the storage space. The introduction of the on-the-fly pruning method further reduces network storage during training. On the other hand, this invention utilizes the existing CUTLASS library to optimize the speed of integer convolution operations on large matrices, significantly improving the network's training speed on GPUs. Additionally, because the number of filters requiring computation gradually decreases with increasing training algebraic levels, this further accelerates the training process.

[0078] According to embodiments of the present invention, a lightweight change detection method combining quantization, pruning, and deep learning first divides the image into blocks and calculates the average value of each block. Then, a method based on the affinity matrix is ​​used to obtain small blocks with the highest and lowest change probabilities as training samples. Subsequently, a quantization training method is used, quantizing the weights into int8 type variables at the initial stage, and using pure integer calculations to update network parameters throughout the entire network training process. Furthermore, filter pruning is performed during network training to further lightweight the network. This can further accelerate the response speed of the change detection task and further reduce storage requirements.

[0079] To achieve the above embodiments, such as Figure 5 As shown, this embodiment also provides a lightweight change detection device 10 that combines quantization, pruning, and deep learning. The device 10 includes:

[0080] The network quantization module 100 is used to quantize the network parameters of the pre-trained VGG network to obtain a VGG network with pure integers.

[0081] The parameter freezing module 200 is used to freeze some weight parameters of the pure integer VGG network and train the weight parameters of the unfrozen network convolutional layers to obtain the VGG network model.

[0082] The network pruning training module 300 is used to input two types of image samples with the maximum / minimum change probability selected by the affinity matrix method based on block mean into the VGG network model for pure integer training, and to prune the network in the filter dimension to obtain the trained VGG network model.

[0083] The network model output module 400 is used to input the two types of images used for testing into the trained VGG network model to obtain the difference image, and to use the maximum inter-class variance method to classify the difference image into the final change image.

[0084] Furthermore, the weight parameters of the unfrozen network convolutional layers are trained, including the weights Conv5-4 of the 8th convolutional layer Conv3-4, the 12th convolutional layer Conv4-4, and the 16th convolutional layer of the VGG network.

[0085] Furthermore, the network pruning training module 300 is also used for:

[0086] Divide the two heterogeneous remote sensing image samples into small square blocks and calculate the mean of each block.

[0087] The distance information between each small block and other small blocks is calculated based on the mean of each small block.

[0088] Based on the distance information, the probability of change of each small block is measured using the affinity matrix formula, and the probability of change of each small block is obtained by solving the problem.

[0089] The probabilities of change are sorted, and two types of image samples are selected based on the probability sorting results; wherein, the two types of image samples include the image patch with the highest probability of change and the image patch with the lowest probability of change.

[0090] Furthermore, the loss function used for training the VGG network model is:

[0091]

[0092] in, This represents the Conv m-4 (m=3,4,5) output of subnetwork p (p=1,2) when the input is a cat-class sample; i, j, and k represent the coordinates of the output; where W m H m and C m This represents the width, length, and number of channels of the output matrix; α m It is the weight set.

[0093] Furthermore, the network pruning training module 300 prunes the network at the filter dimension, including:

[0094] Calculate the L1 norm of each filter in the two subnetworks of the VGG network model:

[0095]

[0096] in, It represents t p(p=1,2) The c-th filter of the l-th layer of the network. It is the number of filters in the l-th layer;

[0097] In adaptive pruning, the maximum value L of the loss function from the start of training to the time of pruning. max The loss function value L during the last pruning last The loss function value L during the backtracking check corresponding to this pruning. k Set the backtracking condition as follows:

[0098] L max -L k <θ(L max -L last )

[0099] Where θ is the tolerance for the loss function to rise.

[0100] The lightweight change detection device combining quantization, pruning, and deep learning according to embodiments of the present invention first divides the image into blocks and calculates the average value of each block. Then, it uses an affinity matrix-based method to obtain small blocks with the highest and lowest change probabilities as training samples. Subsequently, a quantization training method is used, quantizing the weights into int8 type variables in the initial stage, and using pure integer calculations to update network parameters throughout the entire network training process. Furthermore, filter pruning is performed during network training to further lightweight the network. This allows for faster response times and reduced storage requirements in change detection tasks.

[0101] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0102] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A lightweight change detection method combining quantization, pruning, and deep learning, characterized in that, include: The network parameters of the pre-trained VGG network are quantized to obtain a VGG network with pure integers; Freeze some weight parameters of the VGG network with pure integers, and train the weight parameters of the unfrozen convolutional layers of the network to obtain the VGG network model; The two types of image samples with the highest and lowest change probabilities, selected by the affinity matrix method based on block mean, are input into the VGG network model for pure integer training, and the network is pruned in the filter dimension to obtain the trained VGG network model. The two types of images used for testing are input into the trained VGG network model to obtain the difference image, and the difference image is binary classified into the final change image using the maximum inter-class variance method. The two classes of image samples with the highest and lowest change probabilities selected by the affinity matrix method based on block mean include: Divide the two heterogeneous remote sensing image samples into small square blocks and calculate the mean of each block. The distance information between each small block and other small blocks is calculated based on the mean of each small block. Based on the distance information, the probability of change of each small block is measured using the affinity matrix formula, and the probability of change of each small block is obtained by solving the problem. The probabilities of change are sorted, and two types of image samples are selected based on the probability sorting results; wherein, the two types of image samples include the image patch with the highest probability of change and the image patch with the lowest probability of change; The affinity matrix formula measures each value in the matrix. The calculation is as follows: Where exp is an exponential function, .

2. The method according to claim 1, characterized in that, The weight parameters of the unfrozen network convolutional layers are trained, including the weights Conv 3-4 of the 8th convolutional layer, Conv 4-4 of the 12th convolutional layer, and Conv 5-4 of the 16th convolutional layer of the VGG network.

3. The method according to claim 1, characterized in that, The loss function used for training the VGG network model is: in, This represents the input as Subnetwork of sample classes of The output of m-4, where p=1,2, m=3,4,5; x, y, and z represent the output coordinates; where, , and This represents the width, length, and number of channels of the output matrix; It is the weight set.

4. The method according to claim 3, characterized in that, Pruning the network at the filter dimension includes: Calculate the L1 norm of each filter in the two subnetworks of the VGG network model: in, It represents The network's first l The first layer c There are 3 filters, where p = 1, 2. It is the first l The number of layer filters; In adaptive pruning, the maximum value of the loss function from the start of training to the time of pruning. The loss function value at the last pruning The loss function value during the backtracking check corresponding to this pruning. Set the backtracking condition as follows: in, It is the tolerance for the loss function to rise.

5. A lightweight change detection device combining quantization, pruning, and deep learning, characterized in that, include: The network quantization module is used to quantize the network parameters of a pre-trained VGG network to obtain a VGG network with pure integers. The parameter freezing module is used to freeze some weight parameters of the pure integer VGG network and train the weight parameters of the unfrozen network convolutional layers to obtain the VGG network model. The network pruning training module is used to input two types of image samples with the maximum and minimum change probabilities selected by the affinity matrix method based on block mean into the VGG network model for pure integer training, and to prune the network in the filter dimension to obtain the trained VGG network model. The network model output module is used to input the two types of images used for testing into the trained VGG network model to obtain the difference image, and to use the maximum inter-class variance method to classify the difference image into the final change image. The two classes of image samples with the highest and lowest change probabilities selected by the affinity matrix method based on block mean include: Divide the two heterogeneous remote sensing image samples into small square blocks and calculate the mean of each block. The distance information between each small block and other small blocks is calculated based on the mean of each small block. Based on the distance information, the probability of change of each small block is measured using the affinity matrix formula, and the probability of change of each small block is obtained by solving the problem. The probabilities of change are sorted, and two types of image samples are selected based on the probability sorting results; wherein, the two types of image samples include the image patch with the highest probability of change and the image patch with the lowest probability of change; The affinity matrix formula measures each value in the matrix. The calculation is as follows: Where exp is an exponential function, .

6. The apparatus according to claim 5, characterized in that, The weight parameters of the unfrozen network convolutional layers are trained, including the weights Conv 3-4 of the 8th convolutional layer, Conv 4-4 of the 12th convolutional layer, and Conv 5-4 of the 16th convolutional layer of the VGG network.

7. The apparatus according to claim 5, characterized in that, The loss function used for training the VGG network model is: in, This represents the input as Subnetwork of sample classes of The output of m-4, where p=1,2, m=3,4,5; x, y, and z represent the output coordinates; where, , and This represents the width, length, and number of channels of the output matrix; It is the weight set.

8. The apparatus according to claim 7, characterized in that, The network pruning training module prunes the network at the filter dimension, including: Calculate the L1 norm of each filter in the two subnetworks of the VGG network model: in, It represents The network's first l The first layer c There are several filters, p=1,2. It is the first l The number of layer filters; In adaptive pruning, the maximum value of the loss function from the start of training to the time of pruning. The loss function value at the last pruning The loss function value during the backtracking check corresponding to this pruning. Set the backtracking condition as follows: in, It is the tolerance for the loss function to rise.

Images