Deep learning model optimization method and system based on semi-structured pruning

Abstract

The invention discloses a deep learning model optimization method and system based on semi-structured pruning, and relates to the technical field of model optimization, and the method comprises the following steps: obtaining a yolov5s neural network model trained based on a training set, and marking the model as an initial training model; generating a father-child layer grouping list based on the dependency relationship between the father-child layers of the initial training model; pruning the initial training model based on all pruning convolution kernel methods to obtain an initial pruning mode, and screening the initial pruning mode based on a father-child layer grouping list and a calculated L2 norm to obtain a final pruning model; performing training adjustment on a non-zero weight part of the final pruning model based on the training set; the method is used for solving the problem that in an existing semi-structured pruning deep learning model optimization technology, layer-by-layer analysis and configuration are needed, and consequently calculation complexity is high.

Description

Technical Field

[0001] The present invention relates to the technical field of model optimization, and specifically to a deep learning model optimization method and system based on semi-structured pruning. Background Art

[0002] Existing deep learning models have high computational complexity and large storage space, so pruning technology is proposed. Pruning technology can not only improve the operating efficiency of the model by optimizing the deep learning model, but also reduce the complexity and resource consumption of the model while maintaining or improving the model performance.

[0003] The existing technology has the problems of high computational complexity of the pruning process, great difficulty in controlling the performance degradation of the model after pruning, and poor balance between the accuracy and computational efficiency of the model after pruning. For example, the patent application with application publication number CN114881136A discloses a classification method and related equipment based on pruned convolutional neural networks. The pruning method of this scheme involves layer-by-layer sensitivity analysis and pruning rate configuration of the convolution kernel in each convolution layer. This layer-by-layer analysis and configuration requires a lot of calculations, especially in deep neural networks. The computational complexity will increase rapidly, that is, the layer-by-layer analysis and configuration in the existing semi-structured pruning deep learning model optimization technology leads to high computational complexity. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems in the prior art to a certain extent, by generating a parent-child layer grouping list based on the dependency relationship between the parent-child layers of the initial training model, pruning the initial training model based on all pruning convolution kernel methods to obtain an initial pruning mode, screening the initial pruning mode based on the parent-child layer grouping list and calculating the L2 norm to obtain the final pruning model, and training and adjusting the non-zero weight part of the final pruning model based on the training set, so as to solve the problem of high computational complexity in the existing semi-structured pruning deep learning model optimization technology that requires layer-by-layer analysis and configuration.

[0005] To achieve the above objectives, in a first aspect, the present application provides a deep learning model optimization method based on semi-structured pruning, characterized in that it includes the following steps:

[0006] Get the yolov5s neural network model trained based on the training set and mark it as the initial training model;

[0007] Generate a parent-child layer grouping list based on the dependency relationship between the parent-child layers of the initial training model;

[0008] Prune the initial training model based on all pruning convolution kernel methods to obtain the initial pruning mode, and filter the initial pruning mode based on the parent-child layer grouping list and the calculated L2 norm to obtain the final pruning model;

[0009] The non-zero weights of the final pruned model are trained and adjusted based on the training set.

[0010] Furthermore, generating a parent-child layer grouping list based on the dependency relationship between the parent-child layers of the initial training model includes the following sub-steps:

[0011] Based on the layers and dependencies in the pre-trained model, a computational graph is generated with nodes representing each layer of the model and edges representing the data flow or dependency between layers.

[0012] Initialize an empty list and use the depth-first search algorithm to traverse each layer of the computational graph. If the layer has a parent layer, add the layer as a child layer to the column where its parent layer is located. If the layer has no parent layer, use the layer as the new parent layer and fill in a new column until all layers are assigned to the corresponding parent layer columns to complete the parent-child layer grouping list.

[0013] Furthermore, pruning the initial training model based on all pruning convolution kernel methods to obtain an initial pruning mode includes the following sub-steps:

[0014] Determine whether it is 2EP mode or 3EP mode. If it is in 3EP mode, each convolution kernel only retains 3 non-zero weights, and the rest of the weights are set to 0;

[0015] In 2EP mode, each convolution kernel only retains 2 non-zero weights, and the rest of the weights are set to 0;

[0016] Then, all the pruning convolution kernel methods are obtained through the combination method, and the convolution kernel is pruned using all the pruning convolution kernel methods, which is marked as the initial pruning mode.

[0017] Furthermore, pruning the initial training model based on all pruning convolution kernel methods to obtain the initial pruning mode also includes the following sub-steps:

[0018] When pruning the convolution kernel of the initial training model, all parent layers are pruned, and then the child layers are pruned according to the pruning results of the parent layer corresponding to the parent-child layer grouping list.

[0019] Furthermore, selecting the initial pruning mode based on the parent-child layer grouping list and calculating the L2 norm to obtain the final pruning model includes the following sub-steps:

[0020] Establish a plane rectangular coordinate system, mark the convolution kernel in the initial pruning mode as the unfiltered convolution kernel, put the unfiltered convolution kernel in the first quadrant, and there are two adjacent sides of the unfiltered convolution kernel parallel to the X-axis and the Y-axis respectively.

[0021] Furthermore, selecting the initial pruning mode based on the parent-child layer grouping list and calculating the L2 norm to obtain the final pruning model includes the following sub-steps:

[0022] Get the non-zero weight of any unscreened convolution kernel, mark it as the initial non-zero weight, establish a 3x3 convolution kernel field with the initial non-zero weight as the center, mark it as the screening field, and mark the convolution kernel in the screening field except the initial non-zero weight as the eight-neighborhood convolution kernel;

[0023] Determine whether the eight-neighborhood convolution kernel of the screening area contains non-zero weights. If not, delete this initial pruning mode. If yes, continue to establish one or more new screening areas with all the non-zero weights in the eight-neighborhood convolution kernel as the center, find the newly appeared non-zero weights in the new screening area, mark them as newly added non-zero weights, and establish a new screening area again with the newly added non-zero weights as the center of the new screening area, and continue to establish new screening areas until no new non-zero weights appear in the new reference area;

[0024] Record the number of non-zero weights in all screening areas and mark them as the number of non-zero weights. If the number of non-zero weights is less than 2 under 2EP, delete the initial pruning mode. If the number of non-zero weights is less than 3 under 3EP, delete the initial pruning mode and mark the remaining initial pruning mode as the screening pruning mode.

[0025] Furthermore, selecting the initial pruning mode based on the parent-child layer grouping list and calculating the L2 norm to obtain the final pruning model also includes the following sub-steps:

[0026] The non-zero weight values ​​of the convolution kernels in the filtering and pruning mode are obtained and marked as Wi;

[0027] Calculate the L2 norm of the convolution kernel in the filtering and pruning mode. The calculation formula is:

[0028] Where L2n is the L2 norm of the convolution kernel in the screening pruning mode;

[0029] All L2n are calculated and the corresponding screening pruning mode with the minimum value of L2n is marked as the final pruning mode.

[0030] Furthermore, training and adjusting the non-zero weight part of the final pruned model based on the training set includes the following sub-steps:

[0031] The learning rate is set to half of the learning rate of the initial training model, and then the SGD training algorithm is used for fine-tuning. During the back propagation process during fine-tuning, only the non-zero weights will calculate the gradient, and the weights that are set to zero remain unchanged. For each additional first number of training sets, the SGD training algorithm is used again to train and adjust the non-zero weight part of the final pruned model.

[0032] Second, a deep learning model optimization system based on semi-structured pruning, including: a model acquisition module, a parent-child relationship acquisition module, a pruning method screening module and a model update module;

[0033] The model acquisition module is used to obtain the yolov5s neural network model trained based on the training set, marked as the initial training model;

[0034] The parent-child relationship acquisition module is used to generate a parent-child layer grouping list based on the dependency relationship between the parent-child layers of the initial training model;

[0035] The screening pruning method module is used to prune the initial training model based on all pruning convolution kernel methods to obtain an initial pruning mode, and to screen the initial pruning mode based on the parent-child layer grouping list and the calculated L2 norm to obtain the final pruning model;

[0036] The model updating module is used to perform training adjustment on the non-zero weight part of the final pruning model based on the training set.

[0037] Beneficial effects of the present invention: The present invention generates a parent-child layer grouping list based on the dependency relationship between the parent-child layers of the initial training model, prunes the initial training model based on all pruning convolution kernel methods to obtain an initial pruning mode, and screens the initial pruning mode based on the parent-child layer grouping list and the calculated L2 norm to obtain a final pruning model. The advantage is that the generated parent-child layer grouping list is divided into a parent layer and a child layer, so as to determine which layers' pruning operations can share calculation results, so that when the parent layer is pruned, its pruning results can be applied to its child layer, thereby reducing unnecessary repeated pruning calculations and significantly reducing the time and cost of pruning calculations;

[0038] The present invention adjusts the non-zero weight part of the final pruned model by training based on the training set. The advantage is that by retraining the pruned model on the training set, the retained non-zero weights are optimized and the accuracy before pruning is restored or improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flow chart of the steps of the method of the present invention;

[0040] Figure 2 It is a functional block diagram of the system of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] Example 1, please refer to Figure 1As shown, in the first aspect, the present application provides a deep learning model optimization system based on semi-structured pruning, including: a model acquisition module, a parent-child relationship acquisition module, a pruning method screening module and a model update module;

[0043] The Get Model module is used to obtain the yolov5s neural network model trained based on the training set, marked as the initial training model; the layers and parameters are trained on the training set, including convolutional layers, fully connected layers, etc.

[0044] The parent-child relationship acquisition module is used to generate a parent-child layer grouping list based on the dependency relationship between the parent-child layers of the initial training model;

[0045] The module for obtaining parent-child relationship is configured with a strategy for generating a parent-child layer grouping list. The strategy for generating a parent-child layer grouping list includes: based on the layers in the pre-trained model and their dependencies, a computational graph is generated with nodes representing each layer of the model and edges representing data flow or dependencies between layers; the computational graph describes the dependencies between the layers, that is, how the parent layer affects the child layer;

[0046] Initialize an empty list and use the depth-first search algorithm to traverse each layer of the computational graph. If the layer has a parent layer, add the layer as a child layer to the column where its parent layer is located. If the layer has no parent layer, use the layer as the new parent layer and fill in a new column until all layers are assigned to the corresponding parent layer columns to complete the parent-child layer grouping list.

[0047] The pruning method screening module is used to prune the initial training model based on all pruning convolution kernel methods to obtain the initial pruning mode, and to screen the initial pruning mode based on the parent-child layer grouping list and the calculated L2 norm to obtain the final pruning model;

[0048] The screening pruning method module is configured with an initial pruning mode strategy, which includes: determining whether the 2EP mode is 2EP mode or 3EP mode. If it is 3EP mode, each convolution kernel only retains 3 non-zero weights, and the remaining weights are set to 0;

[0049] In 2EP mode, each convolution kernel only retains 2 non-zero weights, and the rest of the weights are set to 0;

[0050] Then, all the pruning convolution kernel methods are obtained through the combination method, and the convolution kernel is pruned using all the pruning convolution kernel methods, which is marked as the initial pruning mode.

[0051] In practical applications, a 3×3 convolution kernel is:

[0052] [0.8,0.7,0.2]

[0053] [0.5,0.9,0.3]

[0054] [0.4,0.6,0.1];

[0055] Each weight is non-zero. In 3EP mode, each convolution kernel only retains 3 non-zero weights, and the remaining weights are set to 0, that is, nine out of three. The calculation combination method contains 84 pruning methods. For example, a pruned convolution kernel is:

[0056] [0,0,0.2]

[0057] [0,0,0.3]

[0058] [0,0,0.1];

[0059] The screening pruning method module is configured with a pruning sub-layer strategy, which includes: when pruning the convolution kernel of the initial training model, all parent layers are pruned, and then the sub-layer is pruned according to the pruning results of the parent layer corresponding to the parent-child layer grouping list; when pruning, the pruning results of the parent layer can be directly applied to the sub-layer to avoid calculating the sub-layer separately again.

[0060] The screening and pruning method module is configured with a strategy for obtaining a screening and pruning mode. The strategy for obtaining a screening and pruning mode includes:

[0061] Establish a plane rectangular coordinate system, mark the convolution kernel in the initial pruning mode as the unfiltered convolution kernel, put the unfiltered convolution kernel in the first quadrant, and there are two adjacent sides of the unfiltered convolution kernel parallel to the X-axis and the Y-axis respectively.

[0062] Get the non-zero weight of any unfiltered convolution kernel, mark it as the initial non-zero weight, establish a 3x3 convolution kernel field with the initial non-zero weight as the center, mark it as the screening field, and mark the convolution kernel excluding the initial non-zero weight in the screening field as the eight-neighborhood convolution kernel; establish the screening field to determine whether the eight weights around all the non-zero weights in the unfiltered convolution kernel are non-zero weights;

[0063] Determine whether the eight-neighborhood convolution kernel of the screening area contains non-zero weights. If not, delete this initial pruning mode. If yes, continue to establish one or more new screening areas with all the non-zero weights in the eight-neighborhood convolution kernel as the center, find the newly appeared non-zero weights in the new screening area, mark them as newly added non-zero weights, and establish a new screening area again with the newly added non-zero weights as the center of the new screening area, and continue to establish new screening areas until no new non-zero weights appear in the new reference area;

[0064] Record the number of non-zero weights in all screening areas and mark them as the number of non-zero weights. If the number of non-zero weights is less than 2 under 2EP, delete the initial pruning mode. If the number of non-zero weights is less than 3 under 3EP, delete the initial pruning mode and mark the remaining initial pruning mode as the screening pruning mode.

[0065] The principle is to search around with any non-zero weight. At the end of the search, it is determined whether the search number reaches the number of all non-zero weights to determine whether all non-zero weights have adjacent non-zero weights. After pruning, the convolution kernel still retains certain spatial structural characteristics. Retaining adjacent non-zero weights helps to maintain the local spatial continuity of the convolution operation, which is crucial for the convolution network to capture local features in the image. If the retained non-zero weights are scattered, the convolution kernel will lose spatial coherence, which may lead to a decline in model performance.

[0066] The screening pruning method module is configured with a strategy for obtaining a final pruning mode, and the strategy for obtaining a final pruning mode includes: obtaining non-zero weight values ​​of the convolution kernels in the screening pruning mode and marking them as Wi;

[0067] Calculate the L2 norm of the convolution kernel in the filtering and pruning mode. The calculation formula is:

[0068] Where L2n is the L2 norm of the convolution kernel in the screening pruning mode;

[0069] Calculate and obtain all L2n, and mark the corresponding screening pruning mode with the minimum value of L2n as the final pruning mode; the smaller the L2 norm, the smaller the impact of these pruned weights on the model performance; therefore, selecting the mode with the smallest L2 norm can achieve maximum pruning with minimum loss;

[0070] In practical applications, the L2 norm of the convolution kernel in the screening pruning mode is calculated. For example, if the non-zero weight values ​​Wi are 0.2, 0.3, and 0.1 respectively, that is, Then L2n is 0.374. The result is rounded to three decimal places. All L2n are calculated and the minimum L2n is 0.374. The final pruning mode is:

[0071] [0,0,0.2]

[0072] [0,0,0.3]

[0073] [0,0,0.1].

[0074] The model update module is used to train and adjust the non-zero weight part of the final pruned model based on the training set;

[0075] The model update module is configured with an update mode strategy, which includes: setting the learning rate to half the learning rate of the initial training model, and then using the SGD training algorithm for fine-tuning. During fine-tuning, during the back propagation process, only non-zero weights will calculate gradients, and the weights that are set to zero remain unchanged. For each additional first number of training sets, the SGD training algorithm is used again to train and adjust the non-zero weight part of the final pruned model. The pruned model may have a decrease in accuracy due to the pruning of some important weights. The update mode is to restore the model performance by retraining the pruned model on the original or target data set and optimizing the retained non-zero weights to restore or improve the accuracy before pruning. The update mode adapts to the new sparse architecture. After pruning, the architecture of the model changes. Fine-tuning can help the model better adapt to task requirements under the new structure. The weights that are set to zero remain unchanged, so that the pruned structure remains unchanged.

[0076] In practical applications, the learning rate is half of the learning rate of the initial training model. It can also be set to other values ​​but cannot be higher than the learning rate of the initial training model. The learning rate will be appropriately reduced to avoid excessive updates to the retained weights, which will lead to model instability. Each time the first number of training sets is added, the SGD training algorithm is used again to train and adjust the non-zero weight part of the final pruned model to avoid overfitting and improve model accuracy.

[0077] Example 2, please refer to Figure 2 As shown, in the second aspect, the present application provides a deep learning model optimization method based on semi-structured pruning, comprising the following steps: Step S1, obtaining a yolov5s neural network model trained based on a training set, and marking it as an initial training model.

[0078] Step S2, generating a parent-child layer grouping list based on the dependency relationship between the parent-child layers of the initial training model, step S2 includes the following sub-steps:

[0079] Step S201, based on the layers in the pre-trained model and their dependencies, a computational graph is generated with nodes representing each layer of the model and edges representing data flow or dependencies between layers;

[0080] Step S202, initialize an empty list, use the depth-first search algorithm to traverse each layer of the computational graph, if the layer has a parent layer, add the layer as a child layer to the column where its parent layer is located, if the layer has no parent layer, use the layer as a new parent layer and fill in a new column until all layers are assigned to the corresponding parent layer columns to complete the parent-child layer grouping list.

[0081] Step S3, pruning the initial training model based on all pruning convolution kernel methods to obtain an initial pruning mode, and filtering the initial pruning mode based on the parent-child layer grouping list and calculating the L2 norm to obtain the final pruning model. Step S3 includes the following sub-steps:

[0082] Step S301, determine whether it is 2EP mode or 3EP mode. If it is 3EP mode, each convolution kernel only retains 3 non-zero weights, and the remaining weights are set to 0;

[0083] Step S302, if in 2EP mode, each convolution kernel only retains 2 non-zero weights, and the remaining weights are set to 0;

[0084] Step S303, then all pruning convolution kernel methods are obtained by a combination method, and the convolution kernel is pruned using all pruning convolution kernel methods, which is marked as an initial pruning mode; when pruning the convolution kernel of the initial training model, all parent layers are pruned, and then the sub-layers are pruned according to the pruning results of the parent layer corresponding to the parent-child layer grouping list;

[0085] Step S304, establish a plane rectangular coordinate system, mark the convolution kernel in the initial pruning mode as an unscreened convolution kernel, put the unscreened convolution kernel in the first quadrant, and there are two adjacent sides of the unscreened convolution kernel parallel to the X axis and the Y axis respectively; obtain the non-zero weight of any unscreened convolution kernel, mark it as the initial non-zero weight, establish a 3x3 convolution kernel field with the initial non-zero weight as the center, mark it as the screening field, and mark the convolution kernel in the screening field except the initial non-zero weight as the eight-neighborhood convolution kernel;

[0086] Step S305, determine whether the eight-neighborhood convolution kernel of the screening area contains non-zero weights. If not, delete the initial pruning mode. If yes, continue to establish one or more new screening areas with all the non-zero weights in the eight-neighborhood convolution kernel as the center, find the newly appeared non-zero weights in the new screening area, mark them as newly added non-zero weights, and establish a new screening area again with the newly added non-zero weights as the center of the new screening area, and continue to establish new screening areas until no new non-zero weights appear in the new reference area;

[0087] Step S306, record the number of non-zero weights in all screening areas and mark them as the number of non-zero weights. If the number of non-zero weights is less than 2 under 2EP, delete the initial pruning mode. If the number of non-zero weights is less than 3 under 3EP, delete the initial pruning mode and mark the remaining initial pruning mode as the screening pruning mode.

[0088] Step S307, respectively obtain the non-zero weight values ​​of the convolution kernels in the screening and pruning mode and mark them as Wi;

[0089] Step S308, calculate the L2 norm of the convolution kernel in the screening pruning mode, and the calculation formula is:

[0090] Where L2n is the L2 norm of the convolution kernel in the screening pruning mode;

[0091] Step S309: Calculate and obtain all L2n, and mark the screening pruning mode corresponding to the minimum value of L2n as the final pruning mode.

[0092] Step S4, training and adjusting the non-zero weight part of the final pruning model based on the training set, step S4 includes the following sub-steps:

[0093] Step S401, set the learning rate to half of the learning rate of the initial training model, and then use the SGD training algorithm for fine-tuning. During the back propagation process during fine-tuning, only the non-zero weights will calculate the gradient, and the weights that are set to zero remain unchanged. Each time the first number of training sets are added, the SGD training algorithm is used again to train and adjust the non-zero weight part of the final pruned model.

[0094] It should be understood by those skilled in the art that embodiments of the present invention may provide methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program codes. Among them, the storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, disk or optical disk. These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0095] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

Claims

1. A deep learning model optimization method based on semi-structured pruning, characterized in that: The steps include: Get the yolov5s neural network model trained based on the training set and mark it as the initial training model; Generate a parent-child layer grouping list based on the dependency relationship between the parent-child layers of the initial training model; Prune the initial training model based on all pruning convolution kernel methods to obtain the initial pruning mode, and filter the initial pruning mode based on the parent-child layer grouping list and the calculated L2 norm to obtain the final pruning model; The non-zero weights of the final pruned model are trained and adjusted based on the training set.

2. The deep learning model optimization method based on semi-structured pruning according to claim 1, characterized in that: Generating a parent-child layer grouping list based on the dependency relationship between the parent-child layers of the initial training model includes the following sub-steps: Based on the layers and dependencies in the pre-trained model, a computational graph is generated with nodes representing each layer of the model and edges representing the data flow or dependency between layers. Initialize an empty list and use the depth-first search algorithm to traverse each layer of the computational graph. If the layer has a parent layer, add the layer as a child layer to the column where its parent layer is located. If the layer has no parent layer, use the layer as the new parent layer and fill in a new column until all layers are assigned to the corresponding parent layer columns to complete the parent-child layer grouping list.

3. The deep learning model optimization method based on semi-structured pruning according to claim 2, characterized in that: Pruning the initial training model based on all pruning convolution kernel methods to obtain the initial pruning mode includes the following sub-steps: Determine whether it is 2EP mode or 3EP mode. If it is in 3EP mode, each convolution kernel only retains 3 non-zero weights, and the rest of the weights are set to 0; In 2EP mode, each convolution kernel only retains 2 non-zero weights, and the rest of the weights are set to 0; Then, all the pruning convolution kernel methods are obtained through the combination method, and the convolution kernel is pruned using all the pruning convolution kernel methods, which is marked as the initial pruning mode.

4. The deep learning model optimization method based on semi-structured pruning according to claim 3 is characterized in that: Pruning the initial training model based on all pruning convolution kernel methods to obtain the initial pruning mode also includes the following sub-steps: When pruning the convolution kernel of the initial training model, all parent layers are pruned, and then the child layers are pruned according to the pruning results of the parent layer corresponding to the parent-child layer grouping list.

5. The deep learning model optimization method based on semi-structured pruning according to claim 4 is characterized in that: The final pruning model is obtained by selecting the initial pruning mode based on the parent-child layer grouping list and calculating the L2 norm, which includes the following sub-steps: Establish a plane rectangular coordinate system, mark the convolution kernel in the initial pruning mode as the unfiltered convolution kernel, put the unfiltered convolution kernel in the first quadrant, and there are two adjacent sides of the unfiltered convolution kernel parallel to the X-axis and the Y-axis respectively.

6. The deep learning model optimization method based on semi-structured pruning according to claim 5, characterized in that: The final pruning model is obtained by selecting the initial pruning mode based on the parent-child layer grouping list and calculating the L2 norm, which includes the following sub-steps: Get the non-zero weight of any unscreened convolution kernel, mark it as the initial non-zero weight, establish a 3x3 convolution kernel field with the initial non-zero weight as the center, mark it as the screening field, and mark the convolution kernel in the screening field except the initial non-zero weight as the eight-neighborhood convolution kernel; Determine whether the eight-neighborhood convolution kernel of the screening area contains non-zero weights. If not, delete this initial pruning mode. If yes, continue to establish one or more new screening areas with all the non-zero weights in the eight-neighborhood convolution kernel as the center, find the newly appeared non-zero weights in the new screening area, mark them as newly added non-zero weights, and establish a new screening area again with the newly added non-zero weights as the center of the new screening area, and continue to establish new screening areas until no new non-zero weights appear in the new reference area; Record the number of non-zero weights in all screening areas and mark them as the number of non-zero weights. If the number of non-zero weights is less than 2 under 2EP, delete the initial pruning mode. If the number of non-zero weights is less than 3 under 3EP, delete the initial pruning mode and mark the remaining initial pruning mode as the screening pruning mode.

7. The deep learning model optimization method based on semi-structured pruning according to claim 6, characterized in that: The final pruning model is obtained by selecting the initial pruning mode based on the parent-child layer grouping list and calculating the L2 norm, which also includes the following sub-steps: The non-zero weight values ​​of the convolution kernels in the filtering and pruning mode are obtained and marked as Wi; Calculate the L2 norm of the convolution kernel in the filtering and pruning mode. The calculation formula is: Where L2n is the L2 norm of the convolution kernel in the screening pruning mode; All L2n are calculated and the corresponding screening pruning mode with the minimum value of L2n is marked as the final pruning mode.

8. The deep learning model optimization method based on semi-structured pruning according to claim 7, characterized in that: The training and adjustment of the non-zero weight part of the final pruned model based on the training set includes the following sub-steps: The learning rate is set to half of the learning rate of the initial training model, and then the SGD training algorithm is used for fine-tuning. During the back propagation process during fine-tuning, only the non-zero weights will calculate the gradient, and the weights that are set to zero remain unchanged. For each additional first number of training sets, the SGD training algorithm is used again to train and adjust the non-zero weight part of the final pruned model.

9. A deep learning model optimization system based on semi-structured pruning, used to implement the deep learning model optimization method based on semi-structured pruning according to any one of claims 1 to 8, characterized in that: include: Obtain model module, obtain parent-child relationship module, filter pruning method module and model update module; The model acquisition module is used to obtain the yolov5s neural network model trained based on the training set, marked as the initial training model; The parent-child relationship acquisition module is used to generate a parent-child layer grouping list based on the dependency relationship between the parent-child layers of the initial training model; The screening pruning method module is used to prune the initial training model based on all pruning convolution kernel methods to obtain an initial pruning mode, and to screen the initial pruning mode based on the parent-child layer grouping list and the calculated L2 norm to obtain the final pruning model; The model updating module is used to perform training adjustment on the non-zero weight part of the final pruning model based on the training set.

Images