Detection model pruning method, device, electronic device and storage medium

Abstract

The present invention provides a detection model pruning method, device, electronic device and storage medium, which belongs to the field of target detection technology. The method decomposes the network layer of the detection model to be pruned to obtain a decomposed network layer; constructs a dependency graph of the decomposed network layer according to the connection relationship and pruning strategy between the decomposed network layers; groups the decomposed network layer according to the dependency graph to obtain network layer groups; performs sparsity training on the network layer groups to determine the importance of the network layer groups and the relative importance of the parameters in the network layer groups; removes the parameters in the network layer groups according to the importance of the network layer groups, the relative importance of the parameters and a preset pruning rate to complete the pruning. The present invention uses a dependency graph to describe the dependency relationship of the decomposed network layer. When pruning the mutually dependent parameters in the network layer groups, they will be removed together to avoid damaging the network structure. The sparsity training makes the coupled parameters in the network layer groups sparse, avoiding the removal of important parameters.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a detection model pruning method, device, electronic device and storage medium. Background Art

[0002] Autonomous driving is a key application in smart transportation, enabling functions such as emergency braking, pedestrian detection, collision warning, lane change assistance, automatic parking, and adaptive cruise control. As smart cars become increasingly intelligent, the requirements for environmental perception in autonomous driving are also increasing. Object detection is a key task in environmental perception, enabling real-time, high-precision detection of surrounding objects and obtaining precise 3D position information. This is a prerequisite for autonomous driving path planning and control, and the foundation for achieving precise driving, directly improving the safety and environmental adaptability of autonomous driving.

[0003] Existing target detection models can detect surrounding objects in real time, but they have many parameters and a high computational load, making them difficult to meet the needs of real-time detection. Given limited computing resources, pruning and compressing the model is essential. Pruning is categorized into two types: non-structural pruning and structural pruning. The former randomly resets some network weights to zero without changing the network's structure, while the latter can remove not only network weights but also channels, layers, modules, and more. In contrast, structural pruning does not rely on specific hardware support (such as support for sparse matrix operations) and can significantly compress the model, speeding up inference. This makes the application of structural pruning more widespread.

[0004] The network structure of the target detection model includes a large number of complex connections such as residual connections and cascade connections. A common problem with existing structural pruning methods is that they fail to handle the complex coupling of network parameters well and do not consider the importance differences between interdependent parameter groups, which can easily lead to a decline in model performance after pruning. Summary of the Invention

[0005] The present invention provides a detection model pruning method, device, electronic device and storage medium to address the defects in the prior art that the complex coupling of network parameters is not well handled, the importance differences between interdependent parameter groups are not taken into account, and pruning easily causes a decline in model performance.

[0006] In a first aspect, the present invention provides a detection model pruning method, comprising:

[0007] Decompose the network layer of the to-be-pruned detection model to obtain a decomposed network layer;

[0008] Constructing a dependency graph of the decomposed network layers according to the connection relationships and pruning strategies between the decomposed network layers, wherein the dependency graph is used to characterize the dependency relationships of the decomposed network layers;

[0009] Grouping the decomposed network layers according to the dependency graph to obtain network layer groups;

[0010] Performing sparsity training on the network layer groups to determine the importance of the network layer groups to be pruned and the relative importance of parameters in the network layer groups;

[0011] According to the importance of the network layer grouping, the relative importance of the parameters and a preset pruning rate, the parameters in the network layer grouping are removed to complete pruning.

[0012] According to a detection model pruning method provided by the present invention, the sparsity training of the network layer grouping is performed to determine the importance of the network layer grouping to be pruned and the relative importance of parameters in the network layer grouping, including:

[0013] By introducing a regularization term into the loss function of the detection model to be pruned, the network layer groups are trained for sparsity;

[0014] determining the importance of the network layer packet according to a loss offset generated by pruning the network layer packet;

[0015] The relative importance of the parameters in the network layer group is determined according to the weight norm of the parameters in the network layer group.

[0016] According to a detection model pruning method provided by the present invention, determining the importance of the network layer group according to the loss offset generated by pruning the network layer group includes:

[0017] Constructing a parameter loss offset model corresponding to the parameter based on the partial derivatives of the parameters in the pruned network layer group, the weight norm of the parameter, and the Hessian matrix;

[0018] The parameter loss offset model is solved based on the Fisher information matrix, and the sum of the parameter loss offsets corresponding to all the parameters is used as the importance of the network layer grouping.

[0019] According to a detection model pruning method provided by the present invention, the regularization term is determined based on the weight norm of each parameter in the network layer group and the scaling factor corresponding to the parameter.

[0020] According to a detection model pruning method provided by the present invention, decomposing a network layer of a detection model to be pruned to obtain a decomposed network layer includes:

[0021] The input part and the output part of each network layer are decomposed, and the decomposed input layer and output layer are used as the decomposed network layer.

[0022] According to a detection model pruning method provided by the present invention, constructing a dependency graph of the decomposed network layers according to the connection relationship between the decomposed network layers and the pruning strategy includes:

[0023] Determining inter-layer dependencies between the decomposed network layers according to the connection relationships between the decomposed network layers;

[0024] Determining, according to a pruning strategy for the decomposed network layer, an intra-layer dependency relationship of the decomposed network layer;

[0025] A dependency graph of the decomposed network layer is constructed according to the inter-layer dependency relationship and the intra-layer dependency relationship.

[0026] According to a detection model pruning method provided by the present invention, the parameters in the network layer group are removed to complete pruning according to the importance of the network layer group, the relative importance of the parameters, and a preset pruning rate, including:

[0027] Determining a target network layer group to be pruned according to the importance of the network layer group;

[0028] Selecting target parameters to be pruned from the target network layer group according to the relative importance of the parameters and the pruning rate;

[0029] The target parameter is removed from the target network layer group to complete pruning.

[0030] In a second aspect, the present invention further provides a detection model pruning device, comprising:

[0031] A decomposition module is used to decompose the network layer of the pruning detection model to obtain a decomposed network layer;

[0032] A construction module, configured to construct a dependency graph of the decomposed network layers according to the connection relationships and pruning strategies between the decomposed network layers, wherein the dependency graph is used to characterize the dependency relationships of the decomposed network layers;

[0033] a grouping module, configured to group the decomposed network layers according to the dependency graph to obtain network layer groups;

[0034] a determination module, configured to perform sparsity training on the network layer grouping, determine the importance of the network layer grouping to be pruned and the relative importance of parameters in the network layer grouping;

[0035] The pruning module is used to remove the parameters in the network layer grouping to complete pruning according to the importance of the network layer grouping, the relative importance of the parameters and a preset pruning rate.

[0036] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the detection model pruning method described above are implemented.

[0037] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described detection model pruning methods.

[0038] The detection model pruning method, device, electronic device and storage medium provided by the present invention use a dependency graph to describe the dependency relationship of the decomposed network layer and establish network layer groupings. When pruning the mutually dependent parameters in the network layer groupings, they are removed together to avoid destroying the network structure. At the same time, the network layer groupings are trained for sparsity so that the coupled parameters within the network layer groupings are sparse, avoiding the removal of important parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 This is one of the flow charts of the detection model pruning method provided by the present invention;

[0041] Figure 2 This is the second flow chart of the detection model pruning method provided by the present invention;

[0042] Figure 3a is a schematic structural diagram of the linear layer provided by the present invention, Figure 3b It is a schematic diagram of the structure of the convolutional layer provided by the present invention;

[0043] Figure 4 It is a schematic diagram of the process of constructing a dependency graph provided by the present invention;

[0044] Figure 5 It is a schematic diagram of the structure of the dependency graph provided by the present invention;

[0045] Figure 6 is a schematic diagram of the process of importance assessment provided by the present invention;

[0046] Figure 7a This is a schematic diagram of the parameter weight arrangement before the consistent sparsity training provided by the present invention. Figure 7b Schematic diagram of parameter weight arrangement after consistent sparsity training provided by the present invention;

[0047] Figure 8 It is a schematic diagram of the process of parameter pruning provided by the present invention;

[0048] Figure 9 This is a comparison chart of the detection accuracy of different pruning methods provided by the present invention after pruning at different pruning rates;

[0049] Figure 10 Schematic diagram of the reduction in the number of parameters and the amount of multiplication and addition calculations at different pruning rates provided by the present invention;

[0050] Figure 11a This is one of the schematic diagrams of road detection after the pruned detection model provided by the present invention is deployed on a vehicle. Figure 11b This is the second schematic diagram of road detection after the pruned detection model provided by the present invention is deployed on a vehicle;

[0051] Figure 12 It is a structural schematic diagram of the detection model pruning device provided by the present invention;

[0052] Figure 13 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0053] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0054] It should be noted that, in the description of the present invention, the terms "comprise," "include," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. Without further limitation, the phrase "comprises a..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising the elements. Terms such as "upper" and "lower" indicate positions or relationships based on those shown in the accompanying drawings and are intended solely to facilitate the description of the present invention and simplify the description. They are not intended to indicate or imply that the devices or elements referred to must have a specific orientation, be constructed, or operate in a specific orientation, and are therefore not to be construed as limitations on the present invention. Unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be broadly construed, for example, to mean fixed, removable, or integral; mechanical or electrical; direct or indirect through an intermediary; or internal communication between two elements. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0055] The terms "first," "second," and so forth, used herein are used to distinguish similar objects, not to describe a specific order or precedence. It should be understood that such terms are interchangeable where appropriate, allowing embodiments of the present invention to be implemented in an order other than that illustrated or described herein. Furthermore, the terms "first," "second," and so forth generally distinguish objects of a single type, and do not limit the number of objects. For example, the first object may be one or more. Furthermore, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates an "or" relationship between the connected objects.

[0056] DepGraph-based pruning methods can better analyze the coupling relationship between network layers. DepGraph-based pruning methods describe the dependency relationship between network layers by constructing a dependency graph and grouping the dependent layers, so that pruning a certain network layer will trigger the pruning of the network layers that depend on it. In addition, DepGraph uses a consistent sparsity strategy to ensure that the parameters removed together within a group are sparse. DepGraph can more accurately analyze and process the complex coupling relationship of parameters in the network, especially for models with complex network connections. However, after grouping the network layers, DepGraph does not consider the importance of each group, but prunes all groups. In fact, if important groups that contribute more to model performance are pruned, it is easy to cause a decline in model performance.

[0057] To this end, this paper proposes a structural pruning method for detection models. This method utilizes a dependency graph to address the complex coupling of network parameters and introduces a Group Importance Evaluation (GIE) strategy to ensure that removed parameters are unimportant, thus effectively addressing the problem of lossless structural pruning of 3D detection models. Compared to existing technologies, the proposed method can better analyze the coupling dependencies between network layers, effectively avoiding the problem of pruning damaging the network structure. The grouping derived from the DepGraph and the GIE strategy complement each other, better ensuring that pruning does not negatively impact the network performance of the detection model. This significantly reduces the number of parameters and improves inference speed while ensuring negligible loss in detection accuracy.

[0058] The following combination Figures 1-13 The present invention describes the detection model pruning method, device, electronic device and storage medium provided by the present invention.

[0059] Figure 1 This is one of the flow charts of the detection model pruning method provided by the present invention. Figure 2 This is the second flow chart of the detection model pruning method provided by the present invention, such as Figure 1 and Figure 2 As shown, including but not limited to the following steps:

[0060] Step 101: Decompose the network layer of the to-be-pruned detection model to obtain a decomposed network layer;

[0061] The detection model to be pruned is determined based on the model's application scenario, such as a LiDAR 3D detection model, an ultrasonic 3D detection model, or an image 3D detection model used to capture 3D spatial information around the vehicle in an autonomous driving scenario. This disclosure uses a LiDAR 3D detection model as an example.

[0062] Decomposing the network layer of the pruning detection model and dividing it into multiple basic units can better analyze the coupling and dependency relationships between the network layer and its parameters.

[0063] Step 102: constructing a dependency graph of the decomposed network layers according to the connection relationships and pruning strategies between the decomposed network layers, wherein the dependency graph is used to represent the dependency relationships of the decomposed network layers;

[0064] Specifically, the inter-layer relationship between the decomposed network layers can be determined based on the connection relationship between the decomposed network layers. The intra-layer relationship of the decomposed network layers can be determined based on whether the pruning strategies of the decomposed network layers are the same. The dependency graph can be constructed based on the inter-layer relationship and the intra-layer relationship.

[0065] Step 103: Group the decomposed network layers according to the dependency graph to obtain network layer groups; group the decomposed network layers according to the dependency graph, that is, group the mutually dependently coupled decomposed network layers into the same group. The purpose of grouping is that when pruning a certain parameter of a certain group, the parameters of all layers in the same group that are coupled with the parameter can be pruned together.

[0066] Step 104: Perform sparsity training on the network layer grouping to determine the importance of the network layer grouping to be pruned and the relative importance of the parameters in the network layer grouping; the purpose of sparsity training is to make the mutually dependent coupled parameters important or sparse, that is, to distinguish between parameters with larger weights and parameters with smaller weights, so that a group of dependent coupled parameters to be pruned are all unimportant.

[0067] Step 105: According to the importance of the network layer group, the relative importance of the parameters and a preset pruning rate, the parameters in the network layer group are removed to complete pruning.

[0068] Select less important network layers for grouping and pruning, then determine the number of parameters to be pruned according to the pruning rate, arrange the parameters from low to high according to their relative importance, remove the corresponding number of parameters, and complete the pruning.

[0069] It can be understood that the present invention uses a dependency graph to describe the dependency relationship of the decomposed network layer and establishes network layer groups. When pruning the mutually dependent parameters in the network layer groups, they will be removed together to avoid destroying the network structure. At the same time, the network layer groups are trained for sparsity so that the coupled parameters within the network layer groups are sparse, avoiding the removal of important parameters.

[0070] Based on the above embodiment, as an optional embodiment, decomposing the network layer of the to-be-pruned detection model to obtain the decomposed network layer includes:

[0071] The input part and the output part of each network layer are decomposed, and the decomposed input layer and output layer are used as the decomposed network layer.

[0072] Figure 3a Schematic diagram of the structure of the linear layer provided by the present invention, such as Figure 3a As shown, the linear layer input part is , represents a two-dimensional weight matrix, the output part is , if you want to prune the first k channels, the two-dimensional weight matrix should be removed for the input part. , and for the output part, you should remove , that is, the former removes an entire row of a two-dimensional matrix, and the latter removes an entire column of a two-dimensional matrix. Their pruning strategies are different and should be handled separately.

[0073] Figure 3b It is a structural diagram of the convolutional layer provided by the present invention, such as Figure 3b As shown, for the convolutional layer, if the input part needs to be removed , the output part needs to be removed Therefore, we first need to divide each network layer into two parts, input and output, in a more refined way.

[0074] For a detection model ,in Indicates the i network layers, the input is , the output is , all network layers of the detection model are decomposed into When modeling the dependency graph later, the input layer and output layer are used as units, rather than the entire network layer.

[0075] It is understandable that the present invention divides the network layer into two parts, the input layer and the output layer, in order to better solve the problem of mismatching pruning strategies between the input part and the output part of the same network layer when subsequently establishing the dependency graph.

[0076] Figure 4 This is a flow chart of the process of constructing a dependency graph provided by the present invention, such as Figure 4 As shown, based on the above embodiment, as an optional embodiment, constructing the dependency graph of the decomposed network layer according to the connection relationship and pruning strategy between the decomposed network layers includes:

[0077] Step 401 : determining the inter-layer dependency relationship between the decomposed network layers according to the connection relationship between the decomposed network layers; if the decomposed network layers have a connection relationship, the decomposed network layers are considered to have an inter-layer dependency relationship.

[0078] Step 402: Determine the intra-layer dependency of the decomposed network layer according to the pruning strategy of the decomposed network layer; if the decomposed network layer adopts the same pruning strategy, it is determined that the decomposed network layer has an intra-layer dependency.

[0079] Step 403: construct a dependency graph of the decomposed network layer according to the inter-layer dependency relationship and the intra-layer dependency relationship.

[0080] The decomposed network layer includes the input layer and the output layer of the network layer. The dependency relationship of the decomposed network layer is modeled as follows:

[0081] ;

[0082] Among them, Inter-layer Dep refers to inter-layer dependency, and Intra-layer Dep refers to intra-layer dependency. Indicates two adjacent network layers that are connected to each other, and the ones in brackets are decomposed network layers.

[0083] Inter-layer dependency: Two decomposed network layers directly connected, such as , recorded as and There are dependencies between layers, and the connection relationship of the decomposed network layers can be directly obtained from the topological structure of the detection model (the path of gradient return during training).

[0084] Intra-layer dependencies: Network layer Decomposed into input layer and output layer After that, the pruning strategies of the two are the same, that is, , recorded as and There are intra-layer dependencies. After input-output decomposition, some network layers, such as the Batch Normalization (BN) layer, meet the same pruning strategy requirements. Their input and output layers are intra-layer dependent. However, some network layers, such as convolutional and linear layers, do not meet the same pruning strategy requirements. Their input and output layers are not intra-layer dependent.

[0085] The modeling formula of the dependency graph is as follows:

[0086] ;

[0087] in, , is the pruning strategy for the input layer, is the pruning strategy for the output layer.

[0088] Figure 5 It is a schematic diagram of the structure of the dependency graph provided by the present invention, such as Figure 5 As shown, the residual block of the lidar 3D detection model PointPillars includes - There are seven network layers. After decomposing the input and output parts, there are 14 decomposed network layers. The layers that are dependent on each other are connected by short black lines. and are directly connected, so there is inter-layer dependency. and is a residual connection, so there is also inter-layer dependency. 、 、 、 、 , the pruning strategies of the input and output layers are the same, so there is intra-layer dependency, and the convolutional layer and There are no inter-layer dependencies.

[0089] The decomposed network layers are grouped according to the dependency graph, except All layers are in one group, namely:

[0090] ;

[0091] in, Group the network layers.

[0092] Figure 5 middle and The pruning strategy is different, all other layers in g are different from The channels coupled to the second channel are also pruned, i.e. The second column and The second row needs to be pruned as well. and The pruning strategy is the same as that of The second row also needs pruning From this we can see that pruning a layer in a group will trigger pruning of all layers in the group.

[0093] It can be understood that the present invention uses a dependency graph to describe the coupling relationship between parameters of network layers and model the dependencies between network layers, so that mutually dependent parameters are removed together during pruning, thus avoiding damage to the network structure.

[0094] Figure 6 is a flow chart of the importance assessment provided by the present invention, such as Figure 6 As shown, based on the above embodiment, as an optional embodiment, the performing sparsity training on the network layer grouping and determining the importance of the network layer grouping to be pruned and the relative importance of the parameters in the network layer grouping include:

[0095] Step 601: Perform sparsity training on the network layer group by introducing a regularization term into the loss function of the detection model to be pruned; the regularization term is determined based on the weight norm of each parameter in the network layer group and the scaling factor corresponding to the parameter.

[0096] When pruning, the parameters to be pruned are usually determined by a preset pruning criterion, usually based on the norm of the weights. Sure. The smaller the value, the smaller the contribution of the parameter weight to the output, and the smaller the impact on the detection model performance after pruning. For a network layer group, since the coupled parameters are pruned together, it is not possible to consider only one parameter. , but consider all the parameters coupled with it Therefore, given a network layer group g, the pruning criterion is , that is, summing the weights of all coupled parameters and selecting the set of parameters with the smallest sum for pruning. However, for the set of parameters with the smallest sum, there may still be more important (larger weighted) parameters in a certain layer.

[0097] Figure 7a This is a schematic diagram of the parameter weight arrangement before the consistent sparsity training provided by the present invention, such as Figure 7a As shown in the figure, the larger the number, the more important the parameter. In order to make the set of coupling parameters to be cut unimportant, it is necessary to make this set of coupling parameters unimportant. Figure 7b This is a schematic diagram of the parameter weight arrangement after the consistent sparsity training provided by the present invention, such as Figure 7b As shown in the figure, after consistent sparsity training, the coupled parameters can be made important or sparse. To this end, a regularization term is introduced into the loss function:

[0098] ;

[0099] in, is the regularization term of the network layer group g, K is the number of parameters in the group, represents the scaling factor, is the square of the L2 norm of the jth parameter weight. After adding the regularization term, consistent sparsity training can be performed to learn the detection model.

[0100] Step 602: determining the importance of the network layer packet based on the loss offset generated by pruning the network layer packet;

[0101] Optionally, determining the importance of the network layer packet according to a loss offset generated by pruning the network layer packet includes:

[0102] Step 6021: construct a parameter loss offset model corresponding to the parameter based on the partial derivatives of the parameters in the pruned network layer group, the weight norm of the parameter, and the Hessian matrix;

[0103] For the detection model trained with consistent sparsity, several network layer groups can be obtained, such as 28 groups in PointPillars. During pruning, a certain number of groups to be pruned need to be selected based on the pruning rate. The specific selection of groups requires a group-level importance evaluation strategy. During pruning, the negative impact of pruning on the performance of the detection model needs to be reduced. Therefore, groups that do not contribute much to the performance of the detection model will be selected for pruning as much as possible. The loss offset after pruning the group can be calculated to determine whether pruning the group will have a significant impact on the performance of the detection model. The formula for the parameter loss offset model is as follows:

[0104] ,

[0105] in, Represents a network layer within the group. Indicates the network layer k parameters, H is the Hessian matrix. is the loss function value of the detection model when it is not pruned, To cut off a set of parameters , which is the loss function value of the detection model after setting the parameters to zero, Indicates that the loss function takes partial derivatives of the pruning parameters. Represents a higher-order infinitesimal.

[0106] Step 6022: Solve the parameter loss offset model based on the Fisher information matrix, and use the sum of the parameter loss offsets corresponding to all the parameters as the importance of the network layer grouping.

[0107] The parameter loss offset model is approximated by the Fisher information matrix:

[0108]

[0109] in, Represents a network layer within the group. Indicates the network layer k parameters, is the loss function value of the detection model when it is not pruned, To cut off a set of parameters , which is the loss function value of the detection model after setting the parameters to zero, Indicates that the loss function takes partial derivatives of the pruning parameters. Represents a higher-order infinitesimal.

[0110] The calculation formula for the importance of the final network layer packet is as follows:

[0111] ;

[0112] Where M is the number of network layers in the network layer group. The importance of all groups can be calculated by this formula, and then these groups are sorted according to their importance, with the groups with lower importance being pruned first.

[0113] Step 603: Determine the relative importance of the parameters in the network layer group according to the weight norm of the parameters in the network layer group.

[0114] Determine the group to be pruned based on the pruning rate Then, the relative importance of each parameter is calculated for each group:

[0115] ;

[0116] in, Indicates the relative importance of parameters in the group of network layers to be pruned, is the weight norm of the parameters in the group of network layers to be pruned, n The number of network layer groups to be pruned.

[0117] As you can understand, the present invention proposes a GIE strategy based on group-level importance estimation. This strategy estimates the contribution of a group to model performance and assesses its importance by calculating the loss offset caused by pruning a group of coupling parameters. This allows pruning to prioritize less important groups, avoiding significant negative impacts on network performance, thereby maintaining detection accuracy while ensuring a compressed model.

[0118] Figure 8 This is a flow chart of parameter pruning provided by the present invention, such as Figure 8 As shown, based on the above embodiment, as an optional embodiment, the removing of the parameters in the network layer group to complete pruning according to the importance of the network layer group, the relative importance of the parameters, and the preset pruning rate includes:

[0119] Step 801, based on the importance of the network layer groups, determine the target network layer groups to be pruned; sort the network layer groups from small to large according to their importance, select the network layer groups with the highest order, i.e., the less important ones, as the target network layer groups to be pruned, and give priority to pruning the target network layer groups.

[0120] Step 802: Select target parameters to be pruned from the target network layer group based on the relative importance of the parameters and the pruning rate; determine the number of parameters to be pruned based on the pruning rate, sort the parameters in the target network layer group from small to large based on their relative importance, select the parameters with the highest ranking, i.e., the less important ones, as the target parameters to be pruned, and give priority to pruning the target parameters.

[0121] Step 803: Remove the target parameter from the target network layer group to complete pruning.

[0122] It is understandable that the present invention allows for giving priority to pruning less important groups during pruning, thereby avoiding a significant negative impact on network performance after pruning, thereby maintaining detection accuracy while guaranteeing the compression model.

[0123] The technical effects of the present invention are described in detail below in conjunction with experimental data.

[0124] Figure 9 This is a comparison chart of the detection accuracy of different pruning methods provided by the present invention after pruning at different pruning rates. Figure 9 As shown in the figure, the present invention compares the detection accuracy of pruning methods based on dependency graphs (DepGraph), network slimming (NS), ThiNet, sparse structure selection (SSS), and the present invention (DepGraph-GIE) at different pruning rates. ThiNet is based on filter pruning, formalizing the filter pruning operation as an optimization problem, and using the statistical information of the next layer to guide the pruning of the current layer.

[0125] from Figure 9 As can be seen, at a 60% pruning rate, the proposed pruning method (DepGraph-GIE) shows negligible loss in detection accuracy after pruning, maintaining mAP at 93.83%. However, other pruning methods experience a decrease in detection accuracy when the pruning rate exceeds 50%. mAP is an object detection evaluation metric, representing the average of the AP values ​​across categories. AP is the area under the precision and recall curves.

[0126] At a pruning rate of 60%, the detection accuracy of different pruning methods for detecting various types of targets is shown in Table 1.

[0127]

[0128] Table 1

[0129] In Table 1, Bus, Car, Cycle, Ped, and Truck are all detection targets, Baseline is the basic data, and data such as 90.14 and 98.47 are all AP values.

[0130] Figure 10 The figure shows the reduction in the number of parameters and the number of multiplication and addition calculations under different pruning rates provided by the present invention. The pruning rate that can be achieved without loss of accuracy is 60%. Under this pruning rate, the reduction in the number of parameters and the number of multiplication and addition calculations (MACs) are as follows: Figure 10 As shown in the figure, the number of parameters can be reduced to 33.2% of the original number of parameters, and the number of MACs can be reduced to 52.6% of the original number.

[0131] The detection model PointPillars was converted to the ONNX format, accelerated using the NVIDIA TensorRT SDK, and deployed on the test vehicle. Table 2 shows the detection accuracy and inference speed at different quantization accuracies.

[0132]

[0133] Table 2

[0134] In Table 2, single-precision Float32, single-precision Float32 (after pruning), Int8, and Int8 (after pruning) are all pruning precisions, and Runtime is the inference speed. As can be seen from Table 2, the inference speed increased by 26% after pruning in Float32, and by 23% in Int8 quantization. The actual scene road detection diagram is shown in the figure. Figure 11a and Figure 11b shown.

[0135] In summary, the present invention provides a pruning method based on dependency graph and group-level importance evaluation, which solves the problem that the coupling of lidar 3D detection network parameters is complex and difficult to analyze, avoids the damage to the network structure caused by pruning, maintains the detection performance of the network, overcomes the disadvantage that the importance of grouping is not taken into account, and solves the problem that pruning easily causes a decrease in model detection accuracy. The loss of detection accuracy can be ignored while significantly compressing the model.

[0136] It should be noted that the execution subject of the task construction method provided by the present invention can be a server or computer device, such as a mobile phone, tablet computer, laptop computer, PDA, vehicle-mounted electronic device, wearable device, ultra-mobile personal computer (UMPC), netbook or personal digital assistant (PDA), etc.

[0137] Figure 12 It is a structural diagram of the detection model pruning device provided by the present invention, such as Figure 12 As shown, the present invention also provides a detection model pruning device, comprising:

[0138] A decomposition module 1210 is configured to decompose the network layer of the to-be-pruned detection model to obtain a decomposed network layer;

[0139] A construction module 1220 is configured to construct a dependency graph of the decomposed network layers according to the connection relationships and pruning strategies between the decomposed network layers, wherein the dependency graph is used to represent the dependency relationships of the decomposed network layers;

[0140] A grouping module 1230 is configured to group the decomposed network layers according to the dependency graph to obtain network layer groups;

[0141] a determination module 1240, configured to perform sparsity training on the network layer grouping to determine the importance of the network layer grouping to be pruned and the relative importance of parameters in the network layer grouping;

[0142] The pruning module 1250 is configured to remove parameters in the network layer grouping to complete pruning according to the importance of the network layer grouping, the relative importance of the parameters, and a preset pruning rate.

[0143] It should be noted that the detection model pruning device provided by the present invention can execute the detection model pruning method described in any of the above embodiments during specific operation, which will not be described in detail in this embodiment.

[0144] Figure 13 Schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 13 As shown, the electronic device may include: a processor 1310, a communications interface 1320, a memory 1330, and a communication bus 1340, wherein the processor 1310, the communications interface 1320, and the memory 1330 communicate with each other via the communication bus 1340. The processor 1310 may call logic instructions in the memory 1330 to execute a detection model pruning method, which includes: decomposing a network layer of a detection model to be pruned to obtain decomposed network layers; constructing a dependency graph of the decomposed network layers based on the connection relationships and pruning strategies between the decomposed network layers, the dependency graph being used to characterize the dependency relationships of the decomposed network layers; grouping the decomposed network layers according to the dependency graph to obtain network layer groups; performing sparsity training on the network layer groups to determine the importance of the network layer groups to be pruned and the relative importance of parameters in the network layer groups; and removing parameters in the network layer groups based on the importance of the network layer groups, the relative importance of the parameters, and a preset pruning rate to complete pruning.

[0145] Furthermore, the logic instructions in the aforementioned memory 1330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0146] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the detection model pruning method provided by the above-mentioned embodiments, the method including: decomposing the network layer of the detection model to be pruned to obtain a decomposed network layer; constructing a dependency graph of the decomposed network layer according to the connection relationship and pruning strategy between the decomposed network layers, and the dependency graph is used to characterize the dependency relationship of the decomposed network layer; grouping the decomposed network layer according to the dependency graph to obtain network layer groups; performing sparsity training on the network layer groups to determine the importance of the network layer groups to be pruned and the relative importance of parameters in the network layer groups; removing the parameters in the network layer groups according to the importance of the network layer groups, the relative importance of the parameters and a preset pruning rate to complete pruning.

[0147] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the detection model pruning method provided in the above-mentioned embodiments, the method comprising: decomposing the network layer of the detection model to be pruned to obtain a decomposed network layer; constructing a dependency graph of the decomposed network layer according to the connection relationship and pruning strategy between the decomposed network layers, the dependency graph being used to characterize the dependency relationship of the decomposed network layer; grouping the decomposed network layer according to the dependency graph to obtain network layer groups; performing sparsity training on the network layer groups to determine the importance of the network layer groups to be pruned and the relative importance of the parameters in the network layer groups; removing the parameters in the network layer groups according to the importance of the network layer groups, the relative importance of the parameters and a preset pruning rate to complete pruning.

[0148] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0149] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A detection model pruning method, characterized in that: include: Decompose the network layer of the to-be-pruned detection model to obtain a decomposed network layer; The detection models to be pruned include a LiDAR 3D detection model used to capture 3D spatial information around the vehicle in autonomous driving scenarios; Constructing a dependency graph of the decomposed network layers according to the connection relationships and pruning strategies between the decomposed network layers, wherein the dependency graph is used to characterize the dependency relationships of the decomposed network layers; Grouping the decomposed network layers according to the dependency graph to obtain network layer groups; Performing sparsity training on the network layer groups to determine the importance of the network layer groups to be pruned and the relative importance of parameters in the network layer groups; Removing the parameters in the network layer group to complete pruning according to the importance of the network layer group, the relative importance of the parameters, and a preset pruning rate; The performing sparsity training on the network layer grouping to determine the importance of the network layer grouping to be pruned and the relative importance of parameters in the network layer grouping includes: By introducing a regularization term into the loss function of the detection model to be pruned, the network layer groups are trained for sparsity; determining the importance of the network layer packet according to a loss offset generated by pruning the network layer packet; determining relative importance of the parameters in the network layer group according to weight norms of the parameters in the network layer group; Determine the group to be pruned based on the pruning rate Then, the relative importance of each parameter is calculated for each group: ； in, Indicates the relative importance of parameters in the group of network layers to be pruned, is the weight norm of the parameters in the group of network layers to be pruned, n The number of network layer groups to be pruned; Determining the importance of the network layer packet according to the loss offset generated by pruning the network layer packet includes: Constructing a parameter loss offset model corresponding to the parameter based on the partial derivatives of the parameters in the pruned network layer group, the weight norm of the parameter, and the Hessian matrix; The parameter loss offset model is solved based on the Fisher information matrix, and the sum of the parameter loss offsets corresponding to all the parameters is used as the importance of the network layer grouping; by calculating the loss offset caused by pruning a certain group of coupling parameters, the contribution of the group to the model performance is estimated, and the importance of the group is evaluated. When pruning, groups with lower importance are pruned first.

2. The detection model pruning method according to claim 1, characterized in that: The regularization term is determined based on a weight norm of each parameter in the network layer group and a scaling factor corresponding to the parameter.

3. The detection model pruning method according to claim 1, characterized in that: The network layer of the to-be-pruned detection model is decomposed to obtain a decomposed network layer, including: The input part and the output part of each network layer are decomposed, and the decomposed input layer and output layer are used as the decomposed network layer.

4. The detection model pruning method according to claim 1, characterized in that: The step of constructing a dependency graph of the decomposed network layers according to the connection relationship and pruning strategy between the decomposed network layers includes: Determining inter-layer dependencies between the decomposed network layers according to the connection relationships between the decomposed network layers; Determining, according to a pruning strategy for the decomposed network layer, an intra-layer dependency relationship of the decomposed network layer; A dependency graph of the decomposed network layer is constructed according to the inter-layer dependency relationship and the intra-layer dependency relationship.

5. The detection model pruning method according to claim 1, characterized in that: The removing of the parameters in the network layer group to complete pruning according to the importance of the network layer group, the relative importance of the parameters, and a preset pruning rate includes: Determining a target network layer group to be pruned according to the importance of the network layer group; Selecting target parameters to be pruned from the target network layer group according to the relative importance of the parameters and the pruning rate; The target parameter is removed from the target network layer group to complete pruning.

6. A detection model pruning device, characterized in that: include: A decomposition module is used to decompose the network layer of the pruning detection model to obtain a decomposed network layer; The detection models to be pruned include a LiDAR 3D detection model used to capture 3D spatial information around the vehicle in autonomous driving scenarios; A construction module, configured to construct a dependency graph of the decomposed network layers according to the connection relationships and pruning strategies between the decomposed network layers, wherein the dependency graph is used to characterize the dependency relationships of the decomposed network layers; a grouping module, configured to group the decomposed network layers according to the dependency graph to obtain network layer groups; a determination module, configured to perform sparsity training on the network layer grouping, determine the importance of the network layer grouping to be pruned and the relative importance of parameters in the network layer grouping; A pruning module, configured to remove parameters from the network layer group to complete pruning based on the importance of the network layer group, the relative importance of the parameters, and a preset pruning rate; The performing sparsity training on the network layer grouping to determine the importance of the network layer grouping to be pruned and the relative importance of parameters in the network layer grouping includes: By introducing a regularization term into the loss function of the detection model to be pruned, the network layer groups are trained for sparsity; determining the importance of the network layer packet according to a loss offset generated by pruning the network layer packet; determining relative importance of the parameters in the network layer group according to weight norms of the parameters in the network layer group; Determine the group to be pruned based on the pruning rate Then, the relative importance of each parameter is calculated for each group: ； in, Indicates the relative importance of parameters in the group of network layers to be pruned, is the weight norm of the parameters in the group of network layers to be pruned, n The number of network layer groups to be pruned; Determining the importance of the network layer packet according to the loss offset generated by pruning the network layer packet includes: Constructing a parameter loss offset model corresponding to the parameter based on the partial derivatives of the parameters in the pruned network layer group, the weight norm of the parameter, and the Hessian matrix; The parameter loss offset model is solved based on the Fisher information matrix, and the sum of the parameter loss offsets corresponding to all the parameters is used as the importance of the network layer grouping; by calculating the loss offset caused by pruning a certain group of coupling parameters, the contribution of the group to the model performance is estimated, and the importance of the group is evaluated. When pruning, groups with lower importance are pruned first.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the detection model pruning method according to any one of claims 1 to 5 are implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the detection model pruning method according to any one of claims 1 to 5 are implemented.

Images