Convolutional neural network channel pruning method based on continuous-time dynamic modeling

Abstract

This invention discloses a channel pruning method for convolutional neural networks based on continuous-time dynamic modeling, relating to the field of computer vision. The method includes: determining the target convolutional layer in the convolutional neural network to be pruned, and preparing a training dataset corresponding to the computer vision task; obtaining the output feature map of the target convolutional layer and generating initial values ​​for channel importance states; evolving and updating the importance states of each channel in the convolutional neural network based on continuous-time dynamic modeling to obtain channel importance scores; generating structured control signals for channels based on the channel importance scores and applying them as channel masks to the output feature map of the target convolutional layer; performing structured channel pruning operations on the channels based on the channel importance scores or channel mask results; and fine-tuning the pruned network. This invention achieves a unified process of channel importance assessment and channel pruning by continuously and dynamically modeling the channel importance in the convolutional neural network.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and more specifically to a channel pruning method for convolutional neural networks based on continuous-time dynamic modeling. Background Technology

[0002] Convolutional neural networks (CNNs), a commonly used model architecture in deep learning, are widely applied to computer vision tasks such as image classification, object detection, and semantic segmentation. As application scenarios demand increasingly higher recognition accuracy, CNNs typically enhance their expressive power by increasing network depth or the number of channels, leading to a significant increase in model parameter size and computational cost. While these models can run in high-computing environments such as servers, their computational overhead and storage requirements often become major limiting factors in practical deployments on mobile terminals, embedded devices, and edge computing scenarios.

[0003] To reduce the computational complexity of convolutional neural networks, various model compression methods have been proposed in the prior art, among which channel pruning is a commonly used structured compression technique. This method evaluates the importance of each channel in the convolutional layer and removes channels that contribute less to the model output, thereby reducing the number of parameters and computational load while maintaining the overall regularity of the network structure, which facilitates the deployment and acceleration of the model in practical systems.

[0004] Existing channel pruning methods typically rely on static channel importance assessment strategies, such as scoring channels based on convolutional kernel weights, batch normalization parameters, or feature activation statistics, and performing pruning operations based on these scores during or after training. However, these methods generally assume that channel importance remains constant across different input images, making it difficult to reflect changes in channel contribution under different input semantic conditions. This can easily lead to inaccurate pruning results, affecting the stability and generalization performance of the pruned model.

[0005] In addition, in some existing technologies, the channel importance assessment process and the channel pruning decision process are independent of each other. There is a lack of a unified modeling mechanism between the scoring results and the final pruning operation. Pruning usually needs to be completed by manually setting thresholds or additional rules, which makes the pruning process dependent on experience, lacks consistency, and is difficult to reuse stably in different models or application scenarios.

[0006] Therefore, existing channel pruning techniques still have shortcomings in terms of dynamic modeling capabilities for channel importance and the consistency between channel importance assessment and pruning processes. It is necessary to propose a new channel pruning method to more accurately characterize the changing process of channel importance and improve the reliability and engineering applicability of channel pruning. Summary of the Invention

[0007] To address the aforementioned shortcomings in existing technologies, this invention provides a channel pruning method for convolutional neural networks based on continuous-time dynamic modeling. By continuously and dynamically modeling the channel importance in the convolutional neural network, the method unifies the channel importance assessment and channel pruning process, thereby improving the accuracy, stability, and engineering applicability of the channel pruning results in computer vision tasks.

[0008] To achieve the aforementioned objectives, the present invention employs the following technical solution: a convolutional neural network channel pruning method based on continuous-time dynamic modeling, comprising the following steps: S1: Determine the target convolutional layer in the convolutional neural network that needs to perform channel pruning, and prepare the training dataset corresponding to the computer vision task; S2: Based on the training dataset, obtain the output feature map of the target convolutional layer and generate initial values ​​for channel importance states; S3: Based on continuous-time dynamic modeling, the importance state of each channel in the convolutional neural network is evolved and updated to obtain the channel importance score; S4: Generate channel structured control signals based on channel importance scores, and apply them as channel masks to the output feature map of the target convolutional layer; S5: Perform structured channel pruning on the channels in the convolutional neural network based on the channel importance score or channel mask result; S6: Fine-tune the pruned convolutional neural network and use it in a fixed structure for actual deployment and inference.

[0009] The beneficial effects of the above solution are as follows: First, this invention provides a channel pruning method for computer vision convolutional neural networks based on continuous-time dynamic modeling. By continuously and dynamically modeling the channel importance in the convolutional neural network, it achieves the unification of channel importance assessment and channel pruning process, thereby improving the accuracy, stability, and engineering applicability of channel pruning results in computer vision tasks. Second, this invention directly uses the channel importance assessment results for channel pruning decisions, enabling the channel scoring process and the pruning process to be completed within the same framework, reducing manual threshold setting and post-processing steps, and improving the consistency and reliability of the pruning process. Finally, by using a structured channel pruning method, the parameter scale and computational load of the convolutional neural network are reduced, allowing the pruned model to maintain a regular network structure, which facilitates deployment and acceleration in different computing platforms and computer vision application scenarios.

[0010] Furthermore, the continuous-time dynamic modeling is achieved through numerical solution, specifically including: S31: Based on the initial value of the channel importance state, the channel importance is represented as a state that evolves over time; S32: The evolution process is solved using numerical methods to obtain the state at the termination time; S33: The channel importance score is obtained by normalizing the state at the termination time.

[0011] The beneficial effects of the above-mentioned further solutions are: by introducing a continuous-time dynamic modeling mechanism, the present invention models the change process of the importance of each channel in the convolutional neural network, so that the importance of the channel can dynamically evolve with the input visual features or the training process, avoiding the errors caused by the static evaluation method.

[0012] Furthermore, the initial value of the channel importance state is:

[0013] in, This is the initial value for the channel importance state. For global average pooling, Output feature maps for the target convolutional layer; Channel importance is represented as a state that evolves over time, and this evolution is described by a learnable derivative function:

[0014] in, For time The state of evolution, For a learnable derivative function, This is the termination time.

[0015] The beneficial effect of the above-mentioned further scheme is that the dynamic channel scoring module treats channel importance as a state variable that changes over time and updates the channel importance state based on a continuous-time dynamic modeling method. Through this continuous-time dynamic modeling process, the channel importance state can continuously evolve according to input features and the training process, thereby characterizing the dynamic changes in channel contribution.

[0016] Furthermore, the numerical solution method includes any one of single-step numerical integration, second-order numerical integration, or multi-step numerical integration.

[0017] The beneficial effect of the above-mentioned further solutions is that the numerical solution method is only used as an implementation method for continuous-time dynamic modeling, and can be selected according to training stability and computational cost.

[0018] Furthermore, the termination time state is as follows:

[0019] in, The state at the termination time. Operators for solving ordinary differential equations For learnable parameters The derivative function, This represents the initial value of the channel's importance state.

[0020] The beneficial effect of the above-mentioned further scheme is that the state at the termination time is obtained by numerically solving the above initial value problem, which is then used to calculate the channel importance score.

[0021] Furthermore, the channel importance score is as follows:

[0022] in, Score the importance of the channel. For Sigmoid normalization, This represents the number of channels.

[0023] The beneficial effect of the above-mentioned further scheme is that after the evolution of the channel importance state is completed within the predetermined time interval, the dynamic channel scoring module outputs the channel importance score corresponding to each channel, which is used for subsequent channel pruning control.

[0024] Furthermore, the step of obtaining the output feature map of the target convolutional layer and generating initial values ​​for channel importance states specifically includes: obtaining the output feature map of the target convolutional layer, performing spatial dimension feature aggregation processing on the output feature map, and generating channel-level feature representations that correspond one-to-one with the number of channels, as initial values ​​for channel importance states.

[0025] The beneficial effect of the above-mentioned further scheme is that the channel-level feature representation is used to reflect the overall semantic information of each channel and serves as the initial state for channel importance modeling.

[0026] The proposed solution also employs the following technical solution: a convolutional neural network channel pruning system based on continuous-time dynamic modeling, comprising: The core module of the convolutional neural network is used to determine the target convolutional layer in the convolutional neural network that needs to perform channel pruning, and to prepare the training dataset corresponding to the computer vision task; based on the training dataset, it obtains the output feature map of the target convolutional layer and generates the initial values ​​of the channel importance state. Dynamic Channel Scoring Module: Used to evolve and update the importance state of each channel in the convolutional neural network based on continuous-time dynamic modeling, and obtain the channel importance score; Structured Gating Mapping Module: Used to generate channel structured control signals based on channel importance scores, and apply them as channel masks to the output feature map of the target convolutional layer; Channel pruning execution module: used to perform structured channel pruning operations on the channels in the convolutional neural network based on channel importance scores or channel mask results; Post-pruning fine-tuning module: Used to fine-tune the training of the pruned convolutional neural network and to use it in actual deployment and inference with a fixed structure.

[0027] The beneficial effects of the above scheme are: This scheme provides a convolutional neural network channel pruning system based on continuous-time dynamic modeling. It introduces a continuous-time dynamic modeling mechanism in the training stage of computer vision convolutional neural networks, realizes the unification of channel importance assessment and channel pruning control, and makes the pruning process more stable and controllable.

[0028] Furthermore, after pruning, the corresponding dynamic channel scoring module and structured gating mapping module are removed, and the pruned network model is used for inference deployment in a fixed structure.

[0029] The beneficial effect of the above-mentioned further solutions is that the model retains a regular structure after pruning, which facilitates deployment and acceleration. Attached Figure Description

[0030] Figure 1 This is a flowchart of a channel pruning method for convolutional neural networks based on continuous-time dynamic modeling.

[0031] Figure 2 This is a block diagram of a channel pruning method for convolutional neural networks based on continuous-time dynamic modeling.

[0032] Figure 3 Flowchart for continuous-time dynamic modeling and solution.

[0033] Figure 4 This is a structural diagram of a convolutional neural network channel pruning system based on continuous-time dynamic modeling. Detailed Implementation

[0034] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0035] To address the problems existing in current computer vision convolutional neural network channel pruning techniques, this invention mainly solves the following technical problems, listed in order of importance: 1. The problem of static channel importance assessment Existing channel pruning methods typically evaluate channels once based on convolution kernel weight parameters or feature activation statistics. This makes it difficult to reflect the changes in the contribution of channels under different input images or different training stages, resulting in inaccurate channel importance evaluation results and affecting the pruning effect.

[0036] 2. The difficulty in characterizing the process of changes in the importance of channels. Existing technologies mostly rely on single forward propagation or instantaneous feature information to determine channels, lacking modeling methods for the continuous evolution of channel importance over time or during training, making it difficult to describe the dynamic process of channel contribution changes under different visual semantic conditions.

[0037] 3. The problem of inconsistent assessment of channel importance and pruning decisions. In some existing technologies, the channel importance assessment process is independent of the actual channel pruning operation. There is a lack of a unified modeling mechanism between the scoring results and the pruning decision. Pruning operations are usually completed by manually setting thresholds or adding rules, which leads to insufficient consistency and poor stability in the pruning process.

[0038] To achieve the above objectives, this plan will be implemented as follows: Example 1, as Figure 1 and Figure 2 As shown, a channel pruning method for convolutional neural networks based on continuous-time dynamic modeling includes the following steps: S1: Determine the target convolutional layer in the convolutional neural network that needs to perform channel pruning, and prepare the training dataset corresponding to the computer vision task; Convolutional neural networks are used to perform feature extraction and prediction output for computer vision classification tasks. They can be any neural network model containing convolutional layers, including but not limited to residual networks, lightweight convolutional networks, or other convolutional neural network structures.

[0039] During training, the convolutional neural network propagates forward through the input image according to a predetermined network structure and outputs feature maps at a pre-defined target convolutional layer. The target convolutional layer can be some or all of the convolutional layers in the network, and the specific choice can be determined based on the model size and pruning requirements.

[0040] S2: Based on the training dataset, obtain the output feature map of the target convolutional layer and generate initial values ​​for channel importance states; The step of obtaining the output feature map of the target convolutional layer and generating initial values ​​for channel importance states specifically includes: obtaining the output feature map of the target convolutional layer, performing spatial dimension feature aggregation processing on the output feature map, generating channel-level feature representations that correspond one-to-one with the number of channels, and using them as initial values ​​for channel importance states.

[0041] S3: Based on continuous-time dynamic modeling, the importance state of each channel in the convolutional neural network is evolved and updated to obtain the channel importance score; like Figure 3 As shown, the continuous-time dynamic modeling is achieved through numerical solution, specifically including: S31: Based on the initial value of the channel importance state, the channel importance is represented as a state that evolves over time; Let the target convolutional layer output feature map be:

[0042] in, For the number of channels, , For spatial dimensions; Global average pooling is performed on the feature map to obtain the initial values ​​of the channel importance states:

[0043] in, This is the initial value for the channel importance state. For global average pooling, Output feature maps for the target convolutional layer; Channel importance is represented as a state that evolves over time, and this evolution is described by a learnable derivative function:

[0044] in, For time The state of evolution, For a learnable derivative function, This is the termination time.

[0045] S32: The evolution process is solved using numerical methods to obtain the state at the termination time; The numerical solution method includes any one of single-step numerical integration, second-order numerical integration, or multi-step numerical integration.

[0046] The state is updated using single-step numerical integration:

[0047] in, for The state at time 1 for The state at any given moment, For time intervals.

[0048] Using second-order numerical integration to improve local consistency:

[0049] in, In the current At time t, the derivative function Based on the current state Input features The first slope value obtained from the calculation, In the state along After the direction is updated once, the derivative function is used. The calculated second slope value.

[0050] Multi-step numerical integration is used to incorporate historical gradient information to improve stability:

[0051] In the numerical solution process, let's denote... For virtual time steps The corresponding channel importance status, and These represent the state values ​​of the previous / next time step, used for calculating intermediate quantities and updating the current state during second-order or multi-step numerical integration; when using single-step numerical integration, only these values ​​are needed. Iterative updates will be performed.

[0052] The termination time status is:

[0053] in, The state at the termination time. Operators for solving ordinary differential equations, used in the interval [0, ... The derivative function The defined evolution equations are solved by numerical integration; For learnable parameters The derivative function, This represents the initial value of the channel's importance state.

[0054] S33: The channel importance score is obtained by normalizing the state at the termination time.

[0055] The channel importance score is as follows:

[0056] in, Score the importance of the channel. For Sigmoid normalization, This represents the number of channels.

[0057] S4: Generate channel structured control signals based on channel importance scores, and apply them as channel masks to the output feature map of the target convolutional layer; The structured control signal acts as a channel mask on the output feature map of the target convolutional layer, causing each channel to be preserved or suppressed according to its importance during forward propagation. Channels with lower importance scores are gradually weakened during training, and their impact on the network output decreases continuously.

[0058] The structured gating mapping module and the dynamic channel scoring module work together under the same training framework, enabling the channel importance evaluation results to directly participate in channel pruning control. This gradually forms a stable channel sparse structure during the training phase, avoiding inconsistencies caused by the independent operation of the channel scoring process and the channel pruning operation.

[0059] S5: Perform structured channel pruning on the channels in the convolutional neural network based on the channel importance score or channel mask result; The channel pruning execution module is used to perform structured pruning operations on the channels in the convolutional neural network after training, based on the channel importance score or channel mask result.

[0060] In this module, the set of channels to be deleted is determined according to the preset pruning strategy, and the corresponding convolutional kernel parameters and related connections are deleted simultaneously. This ensures that the pruned convolutional neural network retains a reduced number of channels while maintaining the network topology, thereby effectively compressing the model parameter size and computational load.

[0061] S6: Fine-tune the pruned convolutional neural network and use it in a fixed structure for actual deployment and inference.

[0062] By fine-tuning the training, the pruned network parameters are refitted to the new network structure to recover the accuracy changes caused by channel pruning. After fine-tuning, the dynamic channel scoring module and the structured gating mapping module are removed, and the pruned convolutional neural network is used for actual deployment and inference in a fixed structure.

[0063] Example 2, as Figure 4 As shown, a convolutional neural network channel pruning system based on continuous-time dynamic modeling includes: The core module of the convolutional neural network is used to determine the target convolutional layer in the convolutional neural network that needs to perform channel pruning, and to prepare the training dataset corresponding to the computer vision task; based on the training dataset, it obtains the output feature map of the target convolutional layer and generates the initial values ​​of the channel importance state. Dynamic Channel Scoring Module: Used to evolve and update the importance state of each channel in the convolutional neural network based on continuous-time dynamic modeling, and obtain the channel importance score; Structured Gating Mapping Module: Used to generate channel structured control signals based on channel importance scores, and apply them as channel masks to the output feature map of the target convolutional layer; Channel pruning execution module: used to perform structured channel pruning operations on the channels in the convolutional neural network based on channel importance scores or channel mask results; Post-pruning fine-tuning module: Used to fine-tune the training of the pruned convolutional neural network and to use it in actual deployment and inference with a fixed structure.

[0064] After pruning, the corresponding dynamic channel scoring module and structured gating mapping module are removed, and the pruned network model is used for inference deployment in a fixed structure.

[0065] The above continuous-time dynamic modeling process is only executed during the model training phase. After channel pruning is completed, the dynamic channel scoring module can be removed without affecting the computational complexity and running efficiency of the pruned model during the inference phase.

[0066] Compared with existing computer vision convolutional neural network channel pruning techniques, the convolutional neural network channel pruning method and system proposed in this invention, based on continuous-time dynamic modeling, continuously and dynamically models channel importance and completes channel importance assessment and channel pruning control within a unified framework, thus having at least the following beneficial effects: (1) Improve the accuracy of channel importance assessment by continuous-time dynamic modeling. This invention treats the importance of each channel in a convolutional neural network as a state variable that changes over time, and updates the channel importance through continuous-time dynamic modeling, rather than using the one-time static evaluation method based on weight parameters or feature activation statistics in existing technologies.

[0067] Since channel importance can continuously evolve with the input image features and network state during model training, this invention can more accurately characterize the changes in channel contribution under different visual semantic conditions, thereby reducing misjudgments caused by static evaluation methods and improving the accuracy of channel importance evaluation results.

[0068] (2) Achieve consistency between channel importance assessment and channel pruning decision-making, and improve pruning consistency. In this invention, the channel importance assessment process and the channel pruning control process are completed under the same modeling framework. The channel importance score is directly used to generate channel-level structured control signals and participates in the joint optimization of network parameters during the training process.

[0069] Compared to existing technologies where channel scoring and pruning operations are independent and pruning relies on manual thresholds or additional rules, this invention effectively avoids the problem of inconsistency between scoring results and pruning decisions, making the channel pruning process more consistent and controllable, thereby improving the reliability of pruning results.

[0070] (3) Introduce dynamic modeling during the training phase and maintain static structure during the deployment phase to balance performance and engineering applicability. In this invention, the relevant modules used for dynamic modeling of channel importance and pruning control only participate in the calculation during the model training phase and are removed after channel pruning is completed, so that the pruned convolutional neural network maintains a fixed structure.

[0071] Therefore, compared with some existing technologies that still require the introduction of additional dynamic control or computational overhead during the inference stage, the present invention does not introduce additional computational burden during the actual deployment and inference stages of the model, thus ensuring the flexibility of the channel pruning process while taking into account the inference efficiency of the pruned model and the needs of engineering applications.

[0072] (4) The structured channel pruning method is adopted, which facilitates model acceleration and cross-platform deployment. This invention reduces the number of channels in a convolutional neural network by using channel-level structured pruning and simultaneously deletes the corresponding convolutional kernel parameters, thus maintaining the regularity of the pruned network structure.

[0073] Compared with unstructured pruning methods, the pruning model obtained by this invention is easier to accelerate on different hardware platforms and computing frameworks, and is suitable for deployment and use in practical application systems related to computer vision.

[0074] (5) The technical solution has good versatility and scalability. The channel pruning method described in this invention is independent of specific convolutional neural network structures and is applicable to different types of convolutional neural network models and various computer vision application scenarios. By adjusting the pruning layers and modeling parameters, this invention can be flexibly applied to network models of different sizes and complexities, exhibiting good versatility and engineering adaptability.

[0075] (6) It helps to improve the stability and repeatability of the channel pruning process. Because this invention evolves channel importance through continuous-time dynamic modeling and gradually forms a sparse channel structure during training, compared to channel pruning methods that rely on single evaluations or empirical rules, this invention is less sensitive to input changes and training conditions, which helps improve the stability and repeatability of pruning results.

[0076] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of the invention.

Claims

1. A channel pruning method for convolutional neural networks based on continuous-time dynamic modeling, characterized in that, Includes the following steps: S1: Determine the target convolutional layer in the convolutional neural network that needs to perform channel pruning, and prepare the training dataset corresponding to the computer vision task; S2: Based on the training dataset, obtain the output feature map of the target convolutional layer and generate initial values ​​for channel importance states; S3: Based on continuous-time dynamic modeling, the importance state of each channel in the convolutional neural network is evolved and updated to obtain the channel importance score; S4: Generate channel structured control signals based on channel importance scores, and apply them as channel masks to the output feature map of the target convolutional layer; S5: Perform structured channel pruning on the channels in the convolutional neural network based on the channel importance score or channel mask result; S6: Fine-tune the pruned convolutional neural network and use it in a fixed structure for actual deployment and inference.

2. The convolutional neural network channel pruning method based on continuous-time dynamic modeling according to claim 1, characterized in that, The continuous-time dynamic modeling is achieved through numerical solution, specifically including: S31: Based on the initial value of the channel importance state, the channel importance is represented as a state that evolves over time; S32: The evolution process is solved using numerical methods to obtain the state at the termination time; S33: The channel importance score is obtained by normalizing the state at the termination time.

3. The convolutional neural network channel pruning method based on continuous-time dynamic modeling according to claim 2, characterized in that, The initial value of the channel importance state is: in, This is the initial value for the channel importance state. For global average pooling, Output feature maps for the target convolutional layer; Channel importance is represented as a state that evolves over time, and this evolution is described by a learnable derivative function: in, For time The state of evolution, For a learnable derivative function, This is the termination time.

4. The convolutional neural network channel pruning method based on continuous-time dynamic modeling according to claim 3, characterized in that, The numerical solution method includes any one of single-step numerical integration, second-order numerical integration, or multi-step numerical integration.

5. The convolutional neural network channel pruning method based on continuous-time dynamic modeling according to claim 4, characterized in that, The termination time status is: in, The state at the termination time. Operators for solving ordinary differential equations For learnable parameters The derivative function, This represents the initial value of the channel's importance state.

6. The convolutional neural network channel pruning method based on continuous-time dynamic modeling according to claim 5, characterized in that, The channel importance score is as follows: in, Score the importance of the channel. For Sigmoid normalization, This represents the number of channels.

7. The convolutional neural network channel pruning method based on continuous-time dynamic modeling according to claim 1, characterized in that, The step of obtaining the output feature map of the target convolutional layer and generating initial values ​​for channel importance states specifically includes: obtaining the output feature map of the target convolutional layer, performing spatial dimension feature aggregation processing on the output feature map, generating channel-level feature representations that correspond one-to-one with the number of channels, and using them as initial values ​​for channel importance states.

8. A system for a convolutional neural network channel pruning method based on continuous-time dynamic modeling as described in any one of claims 1-7, characterized in that, include: Convolutional Neural Network Backbone Module: Used to determine the target convolutional layer in the convolutional neural network that needs to perform channel pruning, and to prepare the training dataset corresponding to the computer vision task; Based on the training dataset, obtain the output feature map of the target convolutional layer and generate initial values ​​for channel importance states; Dynamic Channel Scoring Module: Used to evolve and update the importance state of each channel in the convolutional neural network based on continuous-time dynamic modeling, and obtain the channel importance score; Structured Gating Mapping Module: Used to generate channel structured control signals based on channel importance scores, and apply them as channel masks to the output feature map of the target convolutional layer; Channel pruning execution module: used to perform structured channel pruning operations on the channels in the convolutional neural network based on channel importance scores or channel mask results; Post-pruning fine-tuning module: Used to fine-tune the training of the pruned convolutional neural network and to use it in actual deployment and inference with a fixed structure.

9. The convolutional neural network channel pruning system based on continuous-time dynamic modeling according to claim 8, characterized in that, After pruning, the corresponding dynamic channel scoring module and structured gating mapping module are removed, and the pruned network model is used for inference deployment in a fixed structure.

Images