A model pruning method, an image processing method and related devices
By combining single-stage pruning and iterative pruning methods, optimizing the pruning ratio and the number of fine-tuning steps, the problems of long pruning time and limited pruning upper limit are solved, achieving efficient training and resource conservation of model pruning.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- JILUO TECH (SHANGHAI) CO LTD
- Filing Date
- 2022-12-22
- Publication Date
- 2026-07-31
AI Technical Summary
Iterative pruning takes too long to train in neural network models, and the upper limit of pruning in a single pruning operation is limited, making it difficult to maintain model compression while accelerating training.
By combining single-stage pruning and iterative pruning methods, and optimizing the pruning ratio information and the number of fine-tuning steps, the training model is gradually pruned and fine-tuned to determine the target number of pruning channels and the number of iterations, thus forming the pruned model.
It accelerates the model pruning training process, increases the pruning limit, reduces resource consumption, and improves response time, making it suitable for terminals with limited computing resources.
Smart Images
Figure CN116011550B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of model compression technology, and in particular to a model pruning method, an image processing method, and related apparatus. Background Technology
[0002] With the development of artificial intelligence technology, the application of neural network models is becoming increasingly widespread. Considering the large number of parameters and computational demands of these models, compression is necessary to improve their processing speed. Model compression aims to reduce model size, decrease resource consumption, and improve response time.
[0003] Model pruning is a common model compression method. This method compresses the neural network model by removing parameters with smaller weights after training. Current model pruning methods are divided into iterative pruning and one-shot pruning. Iterative pruning requires fine-tuning after each pruning step, resulting in longer training times. Therefore, accelerating the training process of iterative pruning is a crucial technical problem that needs to be solved. Summary of the Invention
[0004] This invention provides a model pruning method, an image processing method, and related apparatus to solve the above-mentioned problems.
[0005] This invention provides a model pruning method, comprising:
[0006] Obtain the model to be pruned and its corresponding pruning target; wherein, the model to be pruned is trained based on image data, and the pruning target includes pruning ratio information, which is used to characterize the proportion of single pruning and iterative pruning in the model pruning process;
[0007] The model to be pruned is pruned once according to the pruning target to obtain the first pruned model;
[0008] The first pruned model is iteratively pruned according to the pruning objective to obtain the second pruned model, and the second pruned model is used as the target model.
[0009] According to the model pruning method provided by the present invention, the pruning objective includes pruning the total number of training iterations;
[0010] Accordingly, after pruning the model to be pruned once according to the pruning target to obtain the first pruned model, the method further includes:
[0011] The first pruned model is fine-tuned and trained according to the first fine-tuning number to obtain the first fine-tuned model; wherein, the first fine-tuning number is determined based on the proportion of one pruning in the model pruning process in the pruning ratio information and the total number of pruning training iterations.
[0012] The step of iteratively pruning the first pruned model according to the pruning objective to obtain the second pruned model includes:
[0013] The first fine-tuned model is iteratively pruned according to the pruning objective to obtain the second pruned model.
[0014] According to the model pruning method provided by the present invention, the pruning target further includes a target number of pruning channels;
[0015] Accordingly, the step of iteratively pruning the first fine-tuned model according to the pruning objective to obtain the second pruned model includes:
[0016] S1, Prune one channel of the first fine-tuned model to obtain a single-channel pruned model;
[0017] S2, fine-tune the single-channel pruned model according to the second fine-tuning number to obtain the second fine-tuned model; wherein, the second fine-tuning number is determined according to the proportion of iterative pruning in the model pruning process in the pruning ratio information, the target number of pruning channels, and the total number of pruning training iterations.
[0018] S3, S1-S2 are repeated based on the second fine-tuned model until the preset number of iterations is reached to obtain the latest second fine-tuned model, and the latest second fine-tuned model is used as the second pruned model;
[0019] The preset number of iterations is determined based on the number of iterative pruning channels, which is calculated based on the target number of pruning channels and the proportion of iterative pruning in the model pruning process in the pruning ratio information.
[0020] According to a model pruning method provided by the present invention, the second fine-tuning number is determined based on the proportion of iterative pruning in the model pruning process, the target number of pruning channels, and the total number of pruning training iterations in the pruning ratio information, including:
[0021] The total number of iterations for fine-tuning training is calculated based on the proportion of iterative pruning in the model pruning process and the total number of pruning training iterations.
[0022] The second fine-tuning count is calculated based on the total number of iterations of pruning and fine-tuning training and the number of iteration pruning channels.
[0023] According to the model pruning method provided by the present invention, the pruning target further includes a target number of pruning channels;
[0024] Accordingly, the step of pruning the model to be pruned according to the pruning target to obtain the first pruned model includes:
[0025] Obtain the first norm of each channel in the model to be pruned, and sort the channels in the model to be pruned in ascending order of the value of the first norm to obtain the channel sequence;
[0026] The first p channels in the channel sequence are taken as the first channels to be pruned, where p is calculated based on the target number of pruning channels and the proportion of one pruning in the model pruning process in the pruning ratio information.
[0027] The model to be pruned is pruned according to the first pruning channel to obtain the first pruned model.
[0028] According to a model pruning method provided by the present invention, step S1 involves pruning one channel of the first fine-tuned model to obtain a single-channel pruned model, including:
[0029] Obtain the second norm of each channel in the first fine-tuned model;
[0030] The channel with the smallest second norm value is selected as the second channel to be pruned.
[0031] The first fine-tuned model is pruned according to the second pruning channel to obtain a single-channel pruned model.
[0032] The present invention also provides an image processing method, comprising:
[0033] Obtain the image to be processed;
[0034] The image to be processed is input into a trained image processing model, and the trained image processing model processes the image to obtain the processing result.
[0035] The trained image processing model is obtained through any of the model pruning methods described above.
[0036] The present invention also provides a model pruning device, comprising:
[0037] The model and pruning target acquisition module is used to acquire the model to be pruned and its corresponding pruning target; wherein, the model to be pruned is trained based on image data, and the pruning target includes pruning ratio information, which is used to characterize the proportion of single pruning and iterative pruning in the model pruning process;
[0038] A primary pruning module is used to perform a primary pruning on the model to be pruned according to the pruning target, to obtain the first pruned model.
[0039] The iterative pruning module is used to iteratively prune the first pruned model according to the pruning target to obtain a second pruned model, and use the second pruned model as the target model.
[0040] The present invention also provides an image processing apparatus, comprising:
[0041] The image acquisition module is used to acquire the image to be processed.
[0042] An image processing module is used to input the image to be processed into a trained image processing model, and to process the image to be processed by the trained image processing model to obtain a processing result; wherein, the trained image processing model is obtained by the model pruning device described above.
[0043] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the model pruning methods or image processing methods described above.
[0044] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any of the model pruning methods or image processing methods described above.
[0045] The model pruning method, image processing method, and related apparatus provided by this invention, wherein the model pruning method can accelerate the training process through two methods, single pruning and iterative pruning, and improves the pruning upper limit compared to using the single pruning method alone. Attached Figure Description
[0046] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0047] Figure 1 This is one of the flowcharts illustrating the model pruning method provided in this embodiment of the invention;
[0048] Figure 2 This is the second flowchart illustrating the model pruning method provided in this embodiment of the invention;
[0049] Figure 3This is a flowchart illustrating the image processing method provided in an embodiment of the present invention;
[0050] Figure 4 This is a schematic diagram of the model pruning device provided in an embodiment of the present invention;
[0051] Figure 5 This is a schematic diagram of the image processing apparatus provided in an embodiment of the present invention;
[0052] Figure 6 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0054] Figure 1 This is one of the flowcharts illustrating the model pruning method provided in this embodiment of the invention; Figure 2 This is a second schematic flowchart of the model pruning method provided in this embodiment of the invention; as shown below. Figure 1 as well as Figure 2 As shown, the pruning method of this model includes:
[0055] S101, Obtain the model to be pruned and its corresponding pruning target.
[0056] The model to be pruned is trained based on image data, and the pruning target includes pruning ratio information, which is used to characterize the proportion of single pruning and iterative pruning during the model pruning process.
[0057] In this step, the model to be pruned is a pre-trained model, which can be a convolutional neural network, trained based on training images and their corresponding image labels, and used for image processing.
[0058] The pruning target can be the desired size of the pre-trained model, the pruning rate of the entire pre-trained model, or specifically the number of pruning channels. It can also include the total number of pruning training iterations. In this invention, the pruning target can further include pruning ratio information, which characterizes the proportion of single-stage pruning and iterative pruning in the model pruning process. This information is used to configure the respective proportions of single-stage pruning and iterative pruning in the entire pruning training process.
[0059] S102, perform a pruning operation on the model to be pruned according to the pruning target to obtain the first pruned model.
[0060] In this step, the model to be pruned is pruned using a one-shot pruning method according to the pruning target (such as pruning ratio information), and the pruned model is used as the first pruned model.
[0061] S103, the first pruned model is iteratively pruned according to the pruning target to obtain the second pruned model, and the second pruned model is used as the target model.
[0062] In this step, based on the first pruned model described above, and according to the pruning target, pruning is continued. An iterative pruning method is used to prune the first pruned model, and the pruned model is used as the second pruned model. This second pruned model is the final output target model, which is used for image processing.
[0063] In existing technologies, fine-tuning training is performed after each iterative pruning. Assuming iterative pruning of the model involves 10 iterations, and each iteration requires 1000 fine-tuning iterations, the total number of fine-tuning iterations is 10000. However, if the model is pruned only once, and the number of fine-tuning iterations is set to 2000, the total number of fine-tuning iterations is also 2000. As this example shows, the total number of fine-tuning iterations in iterative pruning is significantly higher than in a single pruning method, resulting in an excessively long pruning training process. Conversely, using a single pruning method limits the upper limit of the pruned model's performance. Therefore, this invention integrates two methods: single-stage pruning and iterative pruning. For example, in the first stage of pruning, 1000 fine-tuning training iterations are performed. In the iterative pruning stage, 5 iterations are performed, and 1000 fine-tuning training iterations are performed. Therefore, the total number of fine-tuning training iterations in the entire pruning process is 1000 + 5 * 1000 = 6000. Comparing 6000 with 10000 shows that integrating these two methods significantly reduces the number of fine-tuning training iterations, thereby shortening the training time. Furthermore, compared to using only the single-stage pruning method, it also increases the pruning ceiling.
[0064] The model pruning method provided in this invention can accelerate the training process through both single-stage pruning and iterative pruning, and improves the pruning upper limit compared to using the single-stage pruning method alone.
[0065] In some embodiments of the present invention, the pruning target includes pruning ratio information and the total number of pruning training iterations iters_all. The pruning ratio information is used to characterize the proportion of single pruning and iterative pruning in the model pruning process. More specifically, the pruning ratio information is the pruning ratio value. Assuming that the proportion of single pruning is α (α∈[0,1]), the proportion of iterative pruning is 1-α.
[0066] Accordingly, after pruning the model to be pruned once according to the pruning target to obtain the first pruned model, the method further includes:
[0067] The first pruned model is fine-tuned and trained according to the first fine-tuning number to obtain the first fine-tuned model.
[0068] The first fine-tuning number is determined based on the proportion of a single pruning step in the model pruning process from the pruning ratio information and the total number of pruning training iterations, iters_all. Specifically, if the proportion of a single pruning step is α, then the first fine-tuning number iter1 = 0.8 * α * iters_all, where 0.8 is a set constant.
[0069] The step of iteratively pruning the first pruned model according to the pruning objective to obtain the second pruned model includes:
[0070] The first fine-tuned model is iteratively pruned according to the pruning objective to obtain the second pruned model. That is, iterative pruning is performed based on a one-shot pruning.
[0071] In addition, to further accelerate the pruning training process, the proportion of single pruning α is set to 0.8 based on experience, and the corresponding proportion of iterative pruning is 0.2, thereby maximizing the speed of the pruning training process while ensuring the pruning effect.
[0072] The model pruning method provided in this embodiment of the invention determines the number of fine-tuning training iterations of the model after the first pruning based on the proportion of each pruning iteration in the model pruning process, thereby accelerating the pruning training process.
[0073] In some embodiments of the present invention, the pruning target further includes the target number of pruning channels c_target.
[0074] Accordingly, the step of iteratively pruning the first fine-tuned model according to the pruning objective to obtain the second pruned model includes:
[0075] S1, prune one channel of the first fine-tuned model to obtain a single-channel pruned model.
[0076] S2, fine-tune the single-channel pruned model according to the second fine-tuning number to obtain the second fine-tuned model.
[0077] The second fine-tuning number is determined based on the proportion of iterative pruning in the model pruning process, the target number of pruning channels, and the total number of pruning training iterations in the pruning ratio information.
[0078] Specifically, the second fine-tuning number is determined based on the proportion of iterative pruning in the model pruning process, the target number of pruning channels, and the total number of pruning training iterations in the pruning ratio information, including:
[0079] Based on the proportion of iterative pruning in the model pruning process (the proportion of iterative pruning is 1-α) and the total number of pruning training iterations iters_all, the total number of iterative pruning fine-tuning training iterations is calculated as (1-0.8*α)*iters_all.
[0080] The number of fine-tuning iterations (i.e., the second fine-tuning iteration, iter2) required for each iteration is calculated based on the total number of iterations for fine-tuning and training, and the number of iteration pruning channels, c2. Specifically,
[0081] S3, based on the second fine-tuned model, repeat S1-S2 until the preset number of iterations is reached to obtain the latest second fine-tuned model, and use the latest second fine-tuned model as the second pruned model. That is, prune one channel of the second fine-tuned model to obtain a new single-channel pruned model. Fine-tune and train the new single-channel pruned model according to the second fine-tuning number to obtain a new second fine-tuned model. Repeat this process until the preset number of iterations is reached (i.e., pruning is completed according to the target number of pruned channels).
[0082] The preset number of iterations iter2 is determined based on the number of iteration pruning channels c2. The number of iteration pruning channels c2 is calculated based on the target number of pruning channels c_target and the proportion of iteration pruning in the model pruning process in the pruning ratio information (1-α), specifically c2 = c_target * (1-α).
[0083] The model pruning method provided in this embodiment of the invention performs iterative pruning based on a single pruning step, which not only accelerates the pruning training process but also increases the pruning ceiling.
[0084] In some embodiments of the present invention, the pruning target further includes the target number of pruning channels c_target.
[0085] Accordingly, the step of pruning the model to be pruned according to the pruning target to obtain the first pruned model includes:
[0086] Obtain the first norm of each channel in the model to be pruned, and sort the channels in the model to be pruned in ascending order of the value of the first norm to obtain the channel sequence.
[0087] Specifically, the sum of the norm values of the weights in each channel of each network layer in the model to be pruned is calculated, and the channels are sorted in ascending order according to the sum of the norm values of the weights corresponding to each channel (i.e., the first norm). The total number of channels is denoted as T.
[0088] The first p channels in the channel sequence are designated as the first channels to be pruned.
[0089] Wherein, p is calculated based on the target number of pruning channels c_target and the proportion α of a single pruning operation in the model pruning process from the pruning ratio information. Specifically, p = c_target * α.
[0090] The model to be pruned is pruned according to the first channel to be pruned to obtain the first pruned model. That is, p channels are pruned from the model to be pruned according to the identification information of the first channel to be pruned to obtain the first pruned model.
[0091] In this embodiment, the norm value of the weight is obtained by calculating the L1 norm of the weight. In other embodiments of the present invention, it can also be calculated by other methods such as L0 norm and L2 norm.
[0092] The model pruning method provided in this embodiment of the invention determines the channel with lower pruning sensitivity as the first channel to be pruned based on the norm value of each channel, and performs pruning once to obtain the first pruned model.
[0093] In some embodiments of the present invention, step S1, pruning one channel of the first fine-tuned model to obtain a single-channel pruned model, includes:
[0094] Obtain the second norm of each channel in the first fine-tuned model. That is, calculate the sum of the norm values of the channel weights in all network layers of the first fine-tuned model.
[0095] The channel with the smallest second norm value is selected as the second channel to be pruned.
[0096] The first fine-tuned model is pruned according to the second pruning channel to obtain a single-channel pruned model. That is, in the iterative pruning process, only one channel is pruned in each iteration.
[0097] In this embodiment, the norm value of the weight is obtained by calculating the L1 norm of the weight. In other embodiments of the present invention, it can also be calculated by other methods such as L0 norm and L2 norm.
[0098] Figure 3 This is a flowchart illustrating the image processing method provided in an embodiment of the present invention; as shown below. Figure 3 As shown, the image processing method includes:
[0099] S301, Obtain the image to be processed.
[0100] S302, the image to be processed is input into the trained image processing model, and the trained image processing model is used to process the image to obtain the processing result.
[0101] The trained image processing model is obtained through the model pruning method described above.
[0102] In this embodiment, a trained image processing model is obtained through the aforementioned model pruning method. This trained model is then ported to a terminal with limited computing power. After acquiring the image to be processed, the terminal inputs the image into the trained image processing model to directly obtain the prediction result, reducing resource consumption and improving response time. For example, the image processing model is a sign classification model. After being trained and pruned in the cloud or on other devices with abundant computing resources, it is ported to the autonomous vehicle. After acquiring images of the area around the vehicle, the vehicle uses this sign classification model to perform classification prediction and obtain the classification result. Alternatively, the image processing model may be a facial attribute recognition model used to identify the gender, age, ethnicity, etc., of a person in an input facial image. In this case, after training and pruning, the facial attribute recognition model is ported to a terminal with limited computing resources to recognize the facial images acquired by the terminal and obtain the facial recognition result.
[0103] The image processing method provided in this embodiment of the invention obtains an image processing model based on the aforementioned pruning method, and uses this image processing model to implement image processing on a terminal with limited computing resources, thereby achieving the goal of reducing resource consumption and improving response time.
[0104] The model pruning device and image processing device provided by the present invention are described below. The model pruning device described below and the model pruning method described above can be referred to in correspondence with each other, and the image processing device and the image processing method can be referred to in correspondence with each other.
[0105] Figure 4 This is a schematic diagram of the model pruning device provided in an embodiment of the present invention, as shown below. Figure 4As shown, the model pruning device includes a model and pruning target acquisition module 401, a primary pruning module 402, and an iterative pruning module 403.
[0106] The model and pruning target acquisition module 401 is used to acquire the model to be pruned and its corresponding pruning target.
[0107] The model to be pruned is trained based on image data, and the pruning target includes pruning ratio information, which is used to characterize the proportion of single pruning and iterative pruning during the model pruning process.
[0108] In this module, the model to be pruned is a pre-trained model, which can be a convolutional neural network. It is trained based on training images and their corresponding image labels and is used for image processing.
[0109] The pruning target can be the desired size of the pre-trained model, the pruning rate of the entire pre-trained model, or specifically the number of pruning channels. It can also include the total number of pruning training iterations. In this invention, the pruning target can further include pruning ratio information, which characterizes the proportion of single-stage pruning and iterative pruning in the model pruning process. This information is used to configure the respective proportions of single-stage pruning and iterative pruning in the entire pruning training process.
[0110] The first pruning module 402 is used to perform a first pruning on the model to be pruned according to the pruning target to obtain the first pruned model.
[0111] In this module, the model to be pruned is pruned using a one-shot pruning method according to the pruning target (such as pruning ratio information), and the pruned model is used as the first pruned model.
[0112] The iterative pruning module 403 is used to iteratively prune the first pruned model according to the pruning target to obtain a second pruned model, and use the second pruned model as the target model.
[0113] In this module, based on the first pruned model described above, and according to the pruning target, pruning is continued. An iterative pruning method is used to prune the first pruned model, and the pruned model is used as the second pruned model. This second pruned model is the final output target model, which is used for image processing.
[0114] In existing technologies, fine-tuning training is performed after each iterative pruning. Assuming iterative pruning of the model involves 10 iterations, and each iteration requires 1000 fine-tuning iterations, the total number of fine-tuning iterations is 10000. However, if the model is pruned only once, and the number of fine-tuning iterations is set to 2000, the total number of fine-tuning iterations is also 2000. As this example shows, the total number of fine-tuning iterations in iterative pruning is significantly higher than in a single pruning method, resulting in an excessively long pruning training process. Conversely, using a single pruning method limits the upper limit of the pruned model's performance. Therefore, this invention integrates two methods: single-stage pruning and iterative pruning. For example, in the first stage of pruning, 1000 fine-tuning training iterations are performed. In the iterative pruning stage, 5 iterations are performed, and 1000 fine-tuning training iterations are performed. Therefore, the total number of fine-tuning training iterations in the entire pruning process is 1000 + 5 * 1000 = 6000. Comparing 6000 with 10000 shows that integrating these two methods significantly reduces the number of fine-tuning training iterations, thereby shortening the training time. Furthermore, compared to using only the single-stage pruning method, it also increases the pruning ceiling.
[0115] The model pruning device provided in this embodiment of the invention can accelerate the training process through two methods: single pruning and iterative pruning. Compared with using the single pruning method alone, it increases the pruning limit.
[0116] Figure 5 This is a schematic diagram of the image processing device provided in an embodiment of the present invention; as shown below. Figure 5 As shown, the image processing device includes an image acquisition module 501 and an image processing module 502.
[0117] Image acquisition module 501 is used to acquire the image to be processed.
[0118] The image processing module 502 is used to input the image to be processed into a trained image processing model, and to process the image to be processed through the trained image processing model to obtain the processing result.
[0119] The trained image processing model is obtained through the model pruning device described above.
[0120] In this embodiment, a trained image processing model is obtained through the aforementioned model pruning method. This trained model is then ported to a terminal with limited computing power. After acquiring the image to be processed, the terminal inputs the image into the trained image processing model to directly obtain the prediction result, reducing resource consumption and improving response time. For example, the image processing model is a sign classification model. After being trained and pruned in the cloud or on other devices with abundant computing resources, it is ported to the autonomous vehicle. After acquiring images of the area around the vehicle, the vehicle uses this sign classification model to perform classification prediction and obtain the classification result. Alternatively, the image processing model may be a facial attribute recognition model used to identify the gender, age, ethnicity, etc., of a person in an input facial image. In this case, after training and pruning, the facial attribute recognition model is ported to a terminal with limited computing resources to recognize the facial images acquired by the terminal and obtain the facial recognition result.
[0121] The image processing apparatus provided in this embodiment of the invention obtains an image processing model based on the aforementioned pruning method, and uses this image processing model to perform image processing in a terminal with limited computing resources, thereby achieving the goal of reducing resource consumption and improving response time.
[0122] Figure 6 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a model pruning method. The model pruning method includes: obtaining a model to be pruned and its corresponding pruning target, wherein the model to be pruned is trained based on image data, and the pruning target includes pruning ratio information, which is used to characterize the proportion of single pruning and iterative pruning in the model pruning process; performing a single pruning on the model to be pruned according to the pruning target to obtain a first pruned model; performing iterative pruning on the first pruned model according to the pruning target to obtain a second pruned model, and using the second pruned model as the target model.
[0123] Alternatively, an image processing method may be performed, the image processing method comprising: acquiring an image to be processed; inputting the image to be processed into a trained image processing model, processing the image to be processed through the trained image processing model, and obtaining a processing result; wherein the trained image processing model is obtained through the model pruning method described above.
[0124] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0125] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a model pruning method. The model pruning method includes: acquiring a model to be pruned and its corresponding pruning target, wherein the model to be pruned is trained based on image data, and the pruning target includes pruning ratio information, which characterizes the proportion of single pruning and iterative pruning during the model pruning process; performing a single pruning operation on the model to be pruned according to the pruning target to obtain a first pruned model; performing iterative pruning on the first pruned model according to the pruning target to obtain a second pruned model, and using the second pruned model as the target model.
[0126] Alternatively, an image processing method may be performed, the image processing method comprising: acquiring an image to be processed; inputting the image to be processed into a trained image processing model, processing the image to be processed through the trained image processing model, and obtaining a processing result; wherein the trained image processing model is obtained through the model pruning method described above.
[0127] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0128] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.
[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A model pruning method, characterized in that, include: Obtain the model to be pruned and its corresponding pruning target; wherein, the model to be pruned is trained based on image data, and the pruning target includes pruning ratio information and the total number of pruning training iterations, and the pruning ratio information is used to characterize the proportion of one pruning and iterative pruning in the model pruning process; The model to be pruned is pruned once according to the pruning target to obtain the first pruned model; The first pruned model is fine-tuned and trained according to the first fine-tuning number to obtain the first fine-tuned model; wherein, the first fine-tuning number is determined based on the proportion of one pruning in the model pruning process in the pruning ratio information and the total number of pruning training iterations. The first fine-tuned model is iteratively pruned according to the pruning objective to obtain the second pruned model, and the second pruned model is used as the target model.
2. The model pruning method according to claim 1, characterized in that, The pruning target also includes the target number of pruning channels; Accordingly, the step of iteratively pruning the first fine-tuned model according to the pruning objective to obtain the second pruned model includes: S1, Prune one channel of the first fine-tuned model to obtain a single-channel pruned model; S2, fine-tune the single-channel pruned model according to the second fine-tuning number to obtain the second fine-tuned model; wherein, the second fine-tuning number is determined according to the proportion of iterative pruning in the model pruning process in the pruning ratio information, the target number of pruning channels, and the total number of pruning training iterations. S3, S1-S2 are repeated based on the second fine-tuned model until the preset number of iterations is reached to obtain the latest second fine-tuned model, and the latest second fine-tuned model is used as the second pruned model; The preset number of iterations is determined based on the number of iterative pruning channels, which is calculated based on the target number of pruning channels and the proportion of iterative pruning in the model pruning process in the pruning ratio information.
3. The model pruning method according to claim 2, characterized in that, The second fine-tuning number is determined based on the proportion of iterative pruning in the model pruning process, the target number of pruning channels, and the total number of pruning training iterations in the pruning ratio information, including: The total number of iterations for fine-tuning training is calculated based on the proportion of iterative pruning in the model pruning process and the total number of pruning training iterations. The second fine-tuning count is calculated based on the total number of iterations of pruning and fine-tuning training and the number of iteration pruning channels.
4. The model pruning method according to claim 1, characterized in that, The pruning target also includes the target number of pruning channels; Accordingly, the step of pruning the model to be pruned according to the pruning target to obtain the first pruned model includes: Obtain the first norm of each channel in the model to be pruned, and sort the channels in the model to be pruned in ascending order of the value of the first norm to obtain the channel sequence; The first p channels in the channel sequence are taken as the first channels to be pruned, where p is calculated based on the target number of pruning channels and the proportion of one pruning in the model pruning process in the pruning ratio information. The model to be pruned is pruned according to the first pruning channel to obtain the first pruned model.
5. The model pruning method according to claim 2, characterized in that, S1 involves pruning one channel of the first fine-tuned model to obtain a single-channel pruned model, including: Obtain the second norm of each channel in the first fine-tuned model; The channel with the smallest second norm value is selected as the second channel to be pruned. The first fine-tuned model is pruned according to the second pruning channel to obtain a single-channel pruned model.
6. An image processing method, characterized in that, include: Obtain the image to be processed; The image to be processed is input into a trained image processing model, and the trained image processing model processes the image to obtain the processing result. The trained image processing model is obtained by the model pruning method described in any one of claims 1-5.
7. A model pruning device, characterized in that, include: The model and pruning target acquisition module is used to acquire the model to be pruned and its corresponding pruning target; wherein, the model to be pruned is trained based on image data, and the pruning target includes pruning ratio information and the total number of pruning training iterations, and the pruning ratio information is used to characterize the proportion of one pruning and iterative pruning in the model pruning process. A primary pruning module is used to perform primary pruning on the model to be pruned according to the pruning target to obtain a first pruned model; and to perform fine-tuning training on the first pruned model according to a first number of fine-tuning iterations to obtain a first fine-tuned model; wherein the first number of fine-tuning iterations is determined based on the proportion of primary pruning in the model pruning process in the pruning ratio information and the total number of pruning training iterations; an iterative pruning module is used to perform iterative pruning on the first fine-tuned model according to the pruning target to obtain a second pruned model, and to use the second pruned model as the target model.
8. An image processing apparatus, characterized in that, include: The image acquisition module is used to acquire the image to be processed. An image processing module is used to input the image to be processed into a trained image processing model, and to process the image to be processed by the trained image processing model to obtain a processing result; wherein, the trained image processing model is obtained by the model pruning device as described in claim 7.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the model pruning method as described in any one of claims 1 to 5 or the image processing method as described in claim 6.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the model pruning method as described in any one of claims 1 to 5 or the image processing method as described in claim 6.