Target detection model pruning method and target detection method
Through the object detection model pruning method, network parameters pruning are performed according to the rank of the output feature map, which solves the problem of low detection efficiency caused by large computing and storage resources of deep learning models, and realizes efficient object detection.
Patent Information
- Application Number
- CN202111376597.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-19
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-11-19
AI Technical Summary
The deep learning object detection model is large in computing and storage resources, resulting in low real-time detection efficiency and inability to achieve efficient object detection.
Through the pruning method of the object detection model, the model to be pruned is obtained, the rank of the output feature map is calculated, and the classification and pruning rate is determined according to the rank, and the parameters of the pruning network are obtained to obtain the trained model, reduce the calculation and storage resources, and improve the detection efficiency .
The object detection model is simplified, the computing and storage resource requirements are reduced, the detection efficiency is improved, and efficient object detection is achieved.
Smart Images

Figure CN114241230B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a target detection model pruning method and a target detection method. Background Art
[0002] With the development of computer technology, the use of computer image processing technology for target detection is becoming more and more popular. Target detection technology has a wide range of application value in intelligent transportation systems, intelligent monitoring systems, etc. For example, target detection technology can be applied to the identification of defects in power transmission equipment, that is, in the operation and maintenance of smart grids, target detection technology can be used to improve the recognition of components and defects.
[0003] In traditional technologies, deep learning target detection models are often used to achieve target detection. Deep learning methods can autonomously learn deep-level abstract features of data and have strong detection performance and generalization capabilities.
[0004] However, due to the large computing and storage resources required by the deep learning target detection model, real-time detection cannot be achieved when using the deep learning target detection model for target detection, resulting in low detection efficiency. Summary of the invention
[0005] Based on this, it is necessary to provide a target detection model pruning method, device, computer equipment, computer-readable storage medium and computer program product that can improve detection efficiency, as well as a target detection method, device, computer equipment, computer-readable storage medium and computer program product that can achieve efficient detection in response to the above-mentioned technical problems.
[0006] In a first aspect, the present application provides a method for pruning a target detection model. The method comprises:
[0007] Obtaining a target detection model to be pruned, where the target detection model to be pruned is trained based on sample images in a sample image set;
[0008] Input the sample image into the target detection model to be pruned, and obtain the output feature map of each layer of the neural network corresponding to the sample image;
[0009] Determine the rank of the output feature map, and classify each layer of the neural network in the target detection model to be pruned according to the rank of the output feature map to obtain a classification result;
[0010] According to the classification results, determine the pruning rate corresponding to each layer of the neural network;
[0011] According to the pruning rate, the network parameters of each layer of the neural network are pruned to obtain a trained target detection model.
[0012] In one embodiment, obtaining the target detection model to be pruned includes:
[0013] Obtain an initial target detection model and a set of sample images with annotation boxes;
[0014] Input the sample images in the sample image set into the initial target detection model to obtain the prediction box corresponding to the sample image;
[0015] According to the annotation box and the prediction box, the model loss function corresponding to the initial target detection model is obtained, and the network parameter matrix of the initial target detection model is obtained;
[0016] According to the model loss function and the network parameter matrix, the initial target detection model is sparsely trained to obtain the target detection model to be pruned.
[0017] In one embodiment, the initial target detection model is sparsely trained according to the model loss function and the network parameter matrix to obtain the target detection model to be pruned, including:
[0018] According to the model loss function and the network parameter matrix, the update gradient of the network parameters in the initial target detection model is calculated;
[0019] According to the updated gradient, the network parameters in the initial target detection model are adjusted;
[0020] Return to the step of inputting the sample image in the sample image set into the initial target detection model to obtain the prediction box corresponding to the sample image, until the latest model loss function meets the preset stop iteration condition, and obtains the target detection model to be pruned.
[0021] In one embodiment, each layer of the neural network in the target detection model to be pruned is classified according to the rank of the output feature map, and the classification results include:
[0022] According to the rank of the output feature map, determine the rank corresponding to each layer of the neural network;
[0023] The ranks corresponding to each layer of the neural network are sorted, and each layer of the neural network in the target detection model to be pruned is classified according to the preset number of categories and the sorting results to obtain the classification results.
[0024] In one embodiment, the network parameters of each layer of the neural network are updated according to the pruning rate, and the trained target detection model is obtained, including:
[0025] Sort the network parameters of each layer of the neural network, and determine the network parameters to be reset for each layer of the neural network based on the sorting results and the pruning rate;
[0026] Reset the network parameters to be reset to obtain the trained object detection model.
[0027] In one embodiment, resetting the network parameters to be reset to obtain the trained target detection model includes:
[0028] Reset the network parameters to be reset to obtain a first target detection model;
[0029] Using the sample images in the sample image set, training the first object detection model to obtain a second object detection model;
[0030] Quantifying the accuracy of network parameters of each layer of the neural network of the second target detection model to obtain a third target detection model;
[0031] Call the TensorRT tool to optimize the third object detection model to obtain a trained object detection model.
[0032] In a second aspect, the present application also provides a target detection model pruning device. The device comprises:
[0033] A model acquisition module is used to acquire a target detection model to be pruned, where the target detection model to be pruned is trained based on sample images in a sample image set;
[0034] The first processing module is used to input the sample image into the target detection model to be pruned, and obtain the output feature map of each layer of the neural network corresponding to the sample image;
[0035] A classification module is used to determine the rank of the output feature map, and classify each layer of the neural network in the target detection model to be pruned according to the rank of the output feature map to obtain a classification result;
[0036] The second processing module is used to determine the pruning rate corresponding to each layer of the neural network according to the classification result;
[0037] The updating module is used to update the network parameters of each layer of the neural network according to the pruning rate to obtain a trained target detection model.
[0038] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:
[0039] Obtaining a target detection model to be pruned, where the target detection model to be pruned is trained based on sample images in a sample image set;
[0040] Input the sample image into the target detection model to be pruned, and obtain the output feature map of each layer of the neural network corresponding to the sample image;
[0041] Determine the rank of the output feature map, and classify each layer of the neural network in the target detection model to be pruned according to the rank of the output feature map to obtain a classification result;
[0042] According to the classification results, determine the pruning rate corresponding to each layer of the neural network;
[0043] According to the pruning rate, the network parameters of each layer of the neural network are pruned to obtain a trained target detection model.
[0044] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:
[0045] Obtaining a target detection model to be pruned, where the target detection model to be pruned is trained based on sample images in a sample image set;
[0046] Input the sample image into the target detection model to be pruned, and obtain the output feature map of each layer of the neural network corresponding to the sample image;
[0047] Determine the rank of the output feature map, and classify each layer of the neural network in the target detection model to be pruned according to the rank of the output feature map to obtain a classification result;
[0048] According to the classification results, determine the pruning rate corresponding to each layer of the neural network;
[0049] According to the pruning rate, the network parameters of each layer of the neural network are pruned to obtain a trained target detection model.
[0050] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:
[0051] Obtaining a target detection model to be pruned, where the target detection model to be pruned is trained based on sample images in a sample image set;
[0052] Input the sample image into the target detection model to be pruned, and obtain the output feature map of each layer of the neural network corresponding to the sample image;
[0053] Determine the rank of the output feature map, and classify each layer of the neural network in the target detection model to be pruned according to the rank of the output feature map to obtain a classification result;
[0054] According to the classification results, determine the pruning rate corresponding to each layer of the neural network;
[0055] According to the pruning rate, the network parameters of each layer of the neural network are pruned to obtain a trained target detection model.
[0056] In a sixth aspect, the present application provides a target detection method. The method comprises:
[0057] Obtain an image to be detected and a trained target detection model, where the trained target detection model is obtained by the target detection model pruning method described above;
[0058] Input the image to be detected into the trained target detection model to obtain the target detection result.
[0059] In a seventh aspect, the present application also provides a target detection device. The device comprises:
[0060] A data acquisition module is used to acquire an image to be detected and a trained target detection model, where the trained target detection model is obtained by the target detection model pruning method described above;
[0061] The detection module is used to input the image to be detected into the trained target detection model to obtain the target detection result.
[0062] In an eighth aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:
[0063] Obtain an image to be detected and a trained target detection model, where the trained target detection model is obtained by the target detection model pruning method described above;
[0064] Input the image to be detected into the trained target detection model to obtain the target detection result.
[0065] In a ninth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:
[0066] Obtain an image to be detected and a trained target detection model, where the trained target detection model is obtained by the target detection model pruning method described above;
[0067] Input the image to be detected into the trained target detection model to obtain the target detection result.
[0068] In a tenth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:
[0069] Obtain an image to be detected and a trained target detection model, where the trained target detection model is obtained by the target detection model pruning method described above;
[0070] Input the image to be detected into the trained target detection model to obtain the target detection result.
[0071] The above-mentioned target detection model pruning method, device, computer equipment, computer-readable storage medium and computer program product obtain the target detection model to be pruned by training the sample images in the sample image set, input the sample image into the target detection model to be pruned, obtain the output feature map of each layer of the neural network corresponding to the sample image, determine the rank of the output feature map, and can classify each layer of the neural network in the target detection model to be pruned according to the rank of the output feature map to obtain the classification result, and then use the classification result to determine the pruning rate corresponding to each layer of the neural network, and then according to the pruning rate, the network parameters of each layer of the neural network can be pruned to obtain the trained target detection model. The whole process can simplify the target detection model through pruning, thereby reducing the computing and storage resources required for the target detection model and improving the detection efficiency.
[0072] The above-mentioned target detection method, apparatus, computer device, computer-readable storage medium and computer program product, by acquiring an image to be detected and a trained target detection model, inputting the image to be detected into the trained target detection model, can utilize the pruned trained target detection model to achieve efficient detection and quickly obtain target detection results. BRIEF DESCRIPTION OF THE DRAWINGS
[0073] Figure 1 Schematic diagram of a process of a target detection model pruning method in one embodiment;
[0074] Figure 2 Schematic diagram of a target detection method in one embodiment;
[0075] Figure 3 is a flow chart of a target detection method in another embodiment;
[0076] Figure 4 is a structural block diagram of a target detection model pruning device in one embodiment;
[0077] Figure 5 is a structural block diagram of a target detection device in one embodiment;
[0078] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0079] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0080] In one embodiment, Figure 1As shown, a target detection model pruning method is provided. This embodiment uses the method applied to a server as an example for illustration. It is understandable that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. Among them, the terminal can be but is not limited to various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server can be implemented as an independent server or a server cluster consisting of multiple servers. In this embodiment, the method includes the following steps:
[0081] Step 102, obtaining a to-be-pruned target detection model, where the to-be-pruned target detection model is trained based on sample images in a sample image set.
[0082] Among them, the target detection model to be pruned refers to the model obtained by training the initial target detection model according to the sample images in the sample image set. The sample image in the sample image set refers to the image with the annotation box, and the annotation box is used to mark the target object in the sample image. For example, in the identification of power transmission equipment defects, the sample image may specifically refer to the sample image marked with power transmission equipment defects. The initial target detection model refers to an untrained target detection model. For example, the initial target detection model may specifically refer to a model based on the yolo-v3 (You Only Look Once-version3) algorithm, etc. This embodiment does not limit the initial target detection model here, as long as it is a model that can achieve target detection.
[0083] Specifically, the server obtains an initial target detection model and a set of sample images with annotation boxes, and trains the initial target detection model using sample images in the set of sample images with annotation boxes to obtain a target detection model to be pruned.
[0084] Step 104: input the sample image into the target detection model to be pruned to obtain an output feature map of each layer of the neural network corresponding to the sample image.
[0085] The target detection model to be pruned is composed of multiple layers of neural networks, and each layer of the neural network in the target detection model to be pruned processes the sample image. For example, each layer of the neural network can specifically refer to convolution layers of different scales, which can convolve the sample image. The output feature map refers to the image features obtained after the sample image is processed by the neural network. For example, the output feature map can be specifically represented by a feature vector.
[0086] Specifically, after obtaining the target detection model to be pruned, the server will again input the sample images into the target detection model to be pruned in sequence to obtain an output feature map of each layer of the neural network corresponding to the sample image.
[0087] Step 106, determining the rank of the output feature map, and classifying each layer of the neural network in the target detection model to be pruned according to the rank of the output feature map to obtain a classification result.
[0088] Among them, rank is a term in linear algebra. In linear algebra, the rank of a matrix is the highest order of its non-zero minor expressions, and the rank of a vector group is the number of vectors contained in its largest independent group. Since the output feature map can specifically refer to the eigenvector, the rank of the output feature map refers to the rank of the eigenvector.
[0089] Specifically, the server will use the calculation method of the rank of the matrix in linear algebra to calculate the rank of the output feature map, and calculate the rank corresponding to each layer of the neural network based on the rank of the output feature map. According to the rank corresponding to each layer of the neural network, each layer of the neural network in the target detection model to be pruned is classified to obtain the classification result. Among them, calculating the rank corresponding to each layer of the neural network based on the rank of the output feature map means calculating the average rank of the output feature map of the same layer of the neural network for the sample images in the sample image set, and taking the average rank of the output feature map as the rank corresponding to each layer of the neural network.
[0090] Among them, classifying each layer of the neural network in the target detection model to be pruned according to the rank corresponding to each layer of the neural network means sorting the rank corresponding to each layer of the neural network, and classifying according to the preset number of categories and the sorting result. The preset number of categories can be set as needed. For example, when the preset number of categories is 3, the neural network layer can be divided into three categories according to the preset number of categories. For another example, when the preset number of categories is 4, the neural network layer can be divided into four categories according to the preset number of categories. Furthermore, when classifying, the neural network layer is mainly classified according to the size of the rank, that is, the neural network layer with a larger rank will be classified into one category, and the neural network layer with a smaller rank will be classified into one category.
[0091] Step 108, determining the pruning rate corresponding to each layer of the neural network according to the classification result.
[0092] Among them, pruning refers to avoiding some unnecessary traversal processes through certain judgments. Figuratively speaking, it is to prune some "branches" in the search tree. In this embodiment, it mainly refers to pruning some network parameters of each layer of the neural network in the target detection model to be pruned. The pruning rate refers to the proportion of network parameters that need to be adjusted. For example, when the pruning rate is 85%, it means that 85% of the network parameters in this layer of the neural network need to be adjusted.
[0093] Specifically, after the neural network layers are classified according to the size of the rank and the classification results are obtained, the server will determine the pruning rate corresponding to each layer of the neural network according to the classification results and the pre-set category and pruning rate correspondence. Among them, the pre-set category and pruning rate correspondence is pre-set with pruning rates corresponding to different categories. For example, the pre-set category and pruning rate correspondence can be specifically in the form of a pruning rate of a class of neural network layers with the largest rank is X, a pruning rate of a class of neural network layers with the second largest rank is Y, and a pruning rate of a class of neural network layers with the smallest rank is Z. The specific pruning rate of each category can be set as needed. The main setting idea is to set a smaller pruning rate for a larger rank and a larger pruning rate for a smaller rank.
[0094] Step 110, pruning the network parameters of each layer of the neural network according to the pruning rate to obtain a trained target detection model.
[0095] Specifically, after obtaining the pruning rate, the server will update the network parameters of each layer of the neural network according to the pruning rate, that is, perform pruning, to obtain a first target detection model, and then use the sample images in the sample image set to train the first target detection model to adjust the network parameters in the first target detection model to obtain a second target detection model, and then optimize the second target detection model through precision quantization and model inference optimization to obtain a trained target detection model.
[0096] The above-mentioned target detection model pruning method obtains the target detection model to be pruned obtained by training according to the sample images in the sample image set, inputs the sample image into the target detection model to be pruned, obtains the output feature map of each layer of the neural network corresponding to the sample image, and determines the rank of the output feature map. According to the rank of the output feature map, each layer of the neural network in the target detection model to be pruned can be classified to obtain the classification result, and then the pruning rate corresponding to each layer of the neural network can be determined by using the classification result. Then, according to the pruning rate, the network parameters of each layer of the neural network can be pruned to obtain the trained target detection model. The whole process can simplify the target detection model through pruning, thereby reducing the computing and storage resources required by the target detection model and improving the detection efficiency.
[0097] In one embodiment, obtaining the target detection model to be pruned includes:
[0098] Obtain an initial target detection model and a set of sample images with annotation boxes;
[0099] Input the sample images in the sample image set into the initial target detection model to obtain the prediction box corresponding to the sample image;
[0100] According to the annotation box and the prediction box, the model loss function corresponding to the initial target detection model is obtained, and the network parameter matrix of the initial target detection model is obtained;
[0101] According to the model loss function and the network parameter matrix, the initial target detection model is sparsely trained to obtain the target detection model to be pruned.
[0102] Among them, the sample image in the sample image set refers to an image with a labeling box, and the labeling box is used to mark the target object in the sample image. For example, the labeling in this embodiment may specifically refer to manual labeling, and the labeling may specifically be a VOC (Visual Object Classes) format labeling. For example, in the identification of defects in power transmission equipment, the sample image may specifically refer to a sample image marked with defects in power transmission equipment. It should be noted that in this embodiment, image enhancement can also be performed on the sample image using methods such as image rotation, random brightness adjustment, and random contrast adjustment to obtain more diverse sample images. The initial target detection model refers to an untrained target detection model. For example, the initial target detection model may specifically refer to a model based on the yolo-v3 algorithm, etc. This embodiment does not limit the initial target detection model here, as long as it is a model that can achieve target detection.
[0103] The prediction box is used to mark the target object in the sample image predicted by the initial target detection model. For example, in the identification of power transmission equipment defects, the prediction box is used to mark the power transmission equipment defects in the sample image predicted by the initial target detection model. The network parameter matrix refers to the matrix composed of the network parameters of each layer of the neural network in the initial target detection model.
[0104] Specifically, the server will obtain the initial target detection model and a set of sample images with annotated boxes, and input the sample images in the sample image set into the initial target detection model. The initial target detection model will perform target detection on the sample images to output a prediction box corresponding to the sample images. After obtaining the prediction box, the server will calculate the model loss function corresponding to the initial target detection model based on the annotated box and the prediction box, and obtain the network parameter matrix of the initial target detection model. According to the model loss function and the network parameter matrix, the server will adjust the network parameters in the initial target detection model to obtain a new initial target detection model, and then return to the step of inputting the sample images in the sample image set into the initial target detection model to obtain the prediction box corresponding to the sample image, until the latest model loss function meets the preset stop iteration condition, and the target detection model to be pruned is obtained.
[0105] Among them, the preset stop iteration condition can be set as needed. For example, the preset stop iteration condition can specifically refer to when the number of iterations reaches a preset iteration number threshold. For another example, the preset stop iteration condition can specifically refer to the convergence of the model loss function. In this embodiment, the preset stop iteration condition is not limited here.
[0106] In this embodiment, by collecting sample images and inputting them into the initial target detection model, a prediction box corresponding to the sample image is obtained. The model loss function corresponding to the initial target detection model can be obtained based on the labeled box and the prediction box. Therefore, after obtaining the network parameter matrix of the initial target detection model, the model loss function and the network parameter matrix can be used to perform sparse training on the initial target detection model to achieve the acquisition of the target detection model to be pruned.
[0107] In one embodiment, the initial target detection model is sparsely trained according to the model loss function and the network parameter matrix to obtain the target detection model to be pruned, including:
[0108] According to the model loss function and the network parameter matrix, the update gradient of the network parameters in the initial target detection model is calculated;
[0109] According to the updated gradient, the network parameters in the initial target detection model are adjusted;
[0110] Return to the step of inputting the sample image in the sample image set into the initial target detection model to obtain the prediction box corresponding to the sample image, until the latest model loss function meets the preset stop iteration condition, and obtains the target detection model to be pruned.
[0111] Specifically, after obtaining the model loss function and the network parameter matrix, the server will derive each network parameter according to the model loss function and the network parameter matrix to calculate the updated gradient of each network parameter in the initial target detection model, so that backpropagation updates can be performed according to the updated gradients, and the network parameters in the initial target detection model are adjusted. The step of inputting the sample image in the sample image set into the initial target detection model to obtain the prediction box corresponding to the sample image is returned, and the updated initial target detection model is used to perform target detection prediction on the sample image in the sample image set again to obtain a new prediction box, and then continue to obtain the model loss function corresponding to the initial target detection model according to the labeled box and the new prediction box, and obtain the network parameter matrix of the initial target detection model, and use the model loss function and the network parameter matrix to recalculate the new updated gradient, and adjust the network parameters in the initial target detection model again until the latest model loss function meets the preset stop iteration condition to obtain the target pruning model to be pruned.
[0112] Among them, according to the model loss function and the network parameter matrix, each network parameter is differentiated to calculate the update gradient of each network parameter in the initial target detection model, which means that a network parameter regularization term is added to the model loss function. The form of the network parameter regularization term can be the 1 norm of the network parameter. The superimposed value is used as the new model loss function, and each network parameter is differentiated to calculate the update gradient of each network parameter in the initial target detection model. For example, the formula of the superimposed value can be:
[0113] loss(W;x;y)=loss raw (W; x; y) + λ||W||1;
[0114] Among them, loss raw (W; x; y) refers to the model loss function, W refers to the network parameter matrix, x is the network input (i.e., the sample image in the sample image set), y is the network output (i.e., the label box of the sample image), λ is the regularization parameter (which can be set as needed, specifically can be set to 0.1), the model loss function is calculated by network forward propagation, and the network parameter matrix is updated by back propagation. After a certain number of iterations, the training is completed and the target pruning model to be pruned is obtained.
[0115] Next, any network parameter ω in the initial target detection model i Adjustments are given to illustrate with examples.
[0116]
[0117]
[0118]
[0119] Among them, L refers to the new model loss function, L(W) refers to the model loss function calculated by the annotation box and the prediction box, refers to the network parameter regularization term, sign(ω i ) is the sign function, ω i =0,sign(ω i )=0;ω i >0,sign(ω i )=1;ω i <0, sign(ω i )=-1, η refers to the step, which can be set as needed. is the updated gradient obtained after derivation.
[0120] In this embodiment, the update gradient of the network parameters in the initial target detection model is calculated according to the model loss function and the network parameter matrix, and the network parameters in the initial target detection model are adjusted according to the updated gradient, and then the training is returned to retrain until the latest model loss function meets the preset stop iteration condition, thereby obtaining the target detection model to be pruned, which can realize sparse training of the target detection model to be pruned.
[0121] In one embodiment, each layer of the neural network in the target detection model to be pruned is classified according to the rank of the output feature graph, and the classification results include:
[0122] According to the rank of the output feature map, determine the rank corresponding to each layer of the neural network;
[0123] The ranks corresponding to each layer of the neural network are sorted, and each layer of the neural network in the target detection model to be pruned is classified according to the preset number of categories and the sorting results to obtain the classification results.
[0124] Specifically, the server will calculate the average rank of the output feature map of the same layer of the neural network based on the rank of the output feature map, and use the average rank of the output feature map as the rank corresponding to each layer of the neural network. Then, the rank corresponding to each layer of the neural network is sorted, and the neural network layer is evenly divided according to the preset number of categories and the size of the rank, and the neural network layer with a larger rank is classified into one category, and the neural network layer with a smaller rank is classified into another category to obtain the classification result. The preset number of categories can be set as needed. For example, when the preset number of categories is 3, the neural network layer can be divided into three categories according to the preset number of categories. For another example, when the preset number of categories is 4, the neural network layer can be divided into four categories according to the preset number of categories.
[0125] In this embodiment, the rank of the output feature map can be used to classify the neural network layer.
[0126] In one embodiment, the network parameters of each layer of the neural network are updated according to the pruning rate to obtain a trained target detection model including:
[0127] Sort the network parameters of each layer of the neural network, and determine the network parameters to be reset for each layer of the neural network based on the sorting results and the pruning rate;
[0128] Reset the network parameters to be reset to obtain the trained object detection model.
[0129] The network parameters to be reset refer to the network parameters that need to be reset in each layer of the neural network.
[0130] Specifically, after obtaining the pruning rate, the server will sort the network parameters of each layer of the neural network, and select the network parameters with smaller parameter values corresponding to the pruning rate as the network parameters to be reset according to the sorting results and the pruning rate, and reset the parameter values of the network parameters to be reset to 0 to obtain the first target detection model, and then use the sample images in the sample image set to train the first target detection model to adjust the network parameters in the first target detection model to obtain the second target detection model, and then optimize the second target detection model through precision quantization and model inference optimization to obtain the trained target detection model. For example, when the pruning rate is 85%, the server will select 85% of the network parameters with smaller parameter values in the neural network layer as the network parameters to be reset, and reset the parameter values of the network parameters to be reset to 0.
[0131] In this embodiment, the pruning rate can be used to reset the network parameters of each layer of the neural network, so as to achieve pruning and obtain a trained target detection model.
[0132] In one embodiment, resetting the network parameters to be reset to obtain the trained target detection model includes:
[0133] Reset the network parameters to be reset to obtain a first target detection model;
[0134] Using the sample images in the sample image set, training the first object detection model to obtain a second object detection model;
[0135] Quantifying the accuracy of network parameters of each layer of the neural network of the second target detection model to obtain a third target detection model;
[0136] Call the TensorRT tool to optimize the third object detection model to obtain a trained object detection model.
[0137] Specifically, after resetting the network parameters to be reset, the server will obtain the first target detection model. At this time, the pruned first target detection model needs to be trained again. The server will use the sample images in the sample image set to train the first target detection model to obtain the second target detection model. After obtaining the second target detection model, in order to further compress the model size, the server will perform precision quantization on the network parameters of each layer of the neural network of the second target detection model to obtain the third target detection model. Finally, in order to accelerate the reasoning speed of the model, the server will call the TensorRT tool to optimize the third target detection model, and merge the convolutional layer, nonlinear activation layer, and bn (BatchNormalization) layer in the third target detection model to obtain the trained target detection model. Among them, precision quantization refers to adjusting the accuracy of network parameters. For example, precision quantization can specifically refer to converting the original FP32 single-precision floating-point value into a half-precision FP16 floating-point value.
[0138] Among them, when the first target detection model is trained using sample images in the sample image set to obtain the second target detection model, the server will directly compare the prediction box and the annotation box output by the first target detection model to obtain the loss function corresponding to the first target detection model, and use the loss function to adjust the network parameters of the first target detection model until the loss function reaches the pre-set iteration stop condition corresponding to the first target detection model, thereby obtaining the second target detection model. Among them, the iteration stop condition corresponding to the first target detection model can be set as needed. For example, the iteration stop condition corresponding to the first target detection model can specifically refer to when the number of iterations reaches a pre-set iteration number threshold. For another example, the iteration stop condition corresponding to the first target detection model can specifically refer to the convergence of the loss function. In this embodiment, the iteration stop condition is not limited here.
[0139] In one embodiment, Figure 2 As shown, a target detection method is provided. This embodiment uses the method applied to a server as an example for illustration. It is understandable that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. Among them, the terminal can be but is not limited to various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server can be implemented as an independent server or a server cluster consisting of multiple servers. In this embodiment, the method includes the following steps:
[0140] Step 202, obtaining an image to be detected and a trained target detection model, where the trained target detection model is obtained by the target detection model pruning method described above.
[0141] Step 204: input the image to be detected into the trained target detection model to obtain the target detection result.
[0142] Specifically, when target detection is required, the server obtains the image to be detected and the trained target detection model, inputs the image to be detected into the trained target detection model, and obtains the target detection result.
[0143] For example, the target detection method involved in this embodiment can be applied to the identification of defects in power transmission equipment. Specifically, when applying it, the trained target detection model needs to be recompiled and optimized on the computing platform on the drone side, so that the trained target detection model can be used to perform real-time identification of images collected by the drone to detect defects in the power transmission equipment.
[0144] The above target detection method, by acquiring an image to be detected and a trained target detection model, inputs the image to be detected into the trained target detection model, and can use the pruned trained target detection model to achieve efficient detection and quickly obtain target detection results.
[0145] In one embodiment, Figure 3 As shown, a flow chart is used to illustrate the target detection method of the present application, and the target detection method specifically includes the following steps:
[0146] Step 302, obtaining an initial target detection model and a set of sample images with annotation boxes;
[0147] Step 304, inputting sample images in the sample image set into the initial object detection model to obtain a prediction box corresponding to the sample image;
[0148] Step 306, obtaining a model loss function corresponding to the initial target detection model according to the marked box and the predicted box, and obtaining a network parameter matrix of the initial target detection model;
[0149] Step 308, calculating the update gradient of the network parameters in the initial target detection model according to the model loss function and the network parameter matrix;
[0150] Step 310, adjusting the network parameters in the initial target detection model according to the updated gradient;
[0151] Step 312, return to step 304, until the latest model loss function meets the preset stop iteration condition, and obtain the target detection model to be pruned;
[0152] Step 314, inputting the sample image into the target detection model to be pruned, and obtaining an output feature map of each layer of the neural network corresponding to the sample image;
[0153] Step 316, determining the rank of the output feature map, and determining the rank corresponding to each layer of the neural network according to the rank of the output feature map;
[0154] Step 318, sorting the ranks corresponding to each layer of the neural network, and classifying each layer of the neural network in the target detection model to be pruned according to the preset number of categories and the sorting results to obtain a classification result;
[0155] Step 320, determining the pruning rate corresponding to each layer of the neural network according to the classification result;
[0156] Step 322, sorting the network parameters of each layer of the neural network, and determining the network parameters to be reset for each layer of the neural network according to the sorting result and the pruning rate;
[0157] Step 324, resetting the network parameters to be reset to obtain a first target detection model;
[0158] Step 326, training the first object detection model using the sample images in the sample image set to obtain a second object detection model;
[0159] Step 328, quantizing the accuracy of the network parameters of each layer of the neural network of the second target detection model to obtain a third target detection model;
[0160] Step 330, calling the TensorRT tool to optimize the third object detection model to obtain a trained object detection model;
[0161] Step 332, obtaining an image to be detected;
[0162] Step 334: input the image to be detected into the trained target detection model to obtain the target detection result.
[0163] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0164] Based on the same inventive concept, the embodiment of the present application also provides a target detection model pruning device for implementing the target detection model pruning method involved above and a target detection device for implementing the target detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more target detection model pruning devices and target detection device embodiments provided below can refer to the limitations of the target detection model pruning method and the target detection method above, and will not be repeated here.
[0165] In one embodiment, Figure 4 As shown, a target detection model pruning device is provided, including: a model acquisition module 402, a first processing module 404, a classification module 406, a second processing module 408 and an updating module 410, wherein:
[0166] A model acquisition module 402 is used to acquire a detection model of an object to be pruned, where the detection model of an object to be pruned is trained based on sample images in a sample image set;
[0167] The first processing module 404 is used to input the sample image into the target detection model to be pruned, and obtain an output feature map of each layer of the neural network corresponding to the sample image;
[0168] A classification module 406 is used to determine the rank of the output feature map, and classify each layer of the neural network in the target detection model to be pruned according to the rank of the output feature map to obtain a classification result;
[0169] The second processing module 408 is used to determine the pruning rate corresponding to each layer of the neural network according to the classification result;
[0170] The updating module 410 is used to update the network parameters of each layer of the neural network according to the pruning rate to obtain a trained target detection model.
[0171] The above-mentioned target detection model pruning device obtains the target detection model to be pruned obtained by training according to the sample images in the sample image set, inputs the sample image into the target detection model to be pruned, obtains the output feature map of each layer of the neural network corresponding to the sample image, and determines the rank of the output feature map. According to the rank of the output feature map, each layer of the neural network in the target detection model to be pruned can be classified to obtain the classification result, and then the pruning rate corresponding to each layer of the neural network can be determined by using the classification result, and then the network parameters of each layer of the neural network can be pruned according to the pruning rate to obtain the trained target detection model. The whole process can simplify the target detection model through pruning, thereby reducing the computing and storage resources required by the target detection model and improving the detection efficiency.
[0172] In one embodiment, the model acquisition module is also used to obtain an initial target detection model and a set of sample images with annotated boxes, input the sample images in the sample image set into the initial target detection model, obtain the prediction box corresponding to the sample image, obtain the model loss function corresponding to the initial target detection model based on the annotated box and the prediction box, and obtain the network parameter matrix of the initial target detection model, perform sparse training on the initial target detection model based on the model loss function and the network parameter matrix, and obtain the target detection model to be pruned.
[0173] In one embodiment, the model acquisition module is also used to calculate the update gradient of the network parameters in the initial target detection model according to the model loss function and the network parameter matrix, adjust the network parameters in the initial target detection model according to the updated gradient, return to the step of inputting the sample image in the sample image set into the initial target detection model, and obtain the prediction box corresponding to the sample image, until the latest model loss function meets the preset stop iteration condition, and obtains the target detection model to be pruned.
[0174] In one embodiment, the classification module is also used to determine the rank corresponding to each layer of the neural network according to the rank of the output feature map, sort the ranks corresponding to each layer of the neural network, and classify each layer of the neural network in the pruned target detection model according to the preset number of categories and the sorting results to obtain the classification results.
[0175] In one embodiment, the update module is also used to sort the network parameters of each layer of the neural network, determine the network parameters to be reset for each layer of the neural network according to the sorting results and the pruning rate, reset the network parameters to be reset, and obtain a trained target detection model.
[0176] In one embodiment, the update module is also used to reset the network parameters to be reset, obtain a first target detection model, use sample images in a sample image set to train the first target detection model, obtain a second target detection model, quantize the network parameters of each layer of the neural network of the second target detection model, obtain a third target detection model, call the TensorRT tool, optimize the third target detection model, and obtain a trained target detection model.
[0177] In one embodiment, Figure 5 As shown, a target detection device is provided, including: a data acquisition module 502 and a detection module 504, wherein:
[0178] A data acquisition module 502 is used to acquire an image to be detected and a trained target detection model, where the trained target detection model is obtained by the target detection model pruning method described above;
[0179] The detection module 504 is used to input the image to be detected into the trained target detection model to obtain the target detection result.
[0180] The above-mentioned target detection device, by acquiring the image to be detected and the trained target detection model, inputs the image to be detected into the trained target detection model, and can use the pruned trained target detection model to achieve efficient detection and quickly obtain the target detection result.
[0181] The above-mentioned target detection model pruning device and each module in the target detection device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0182] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as sample image sets. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a target detection model pruning method and a target detection method are implemented.
[0183] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0184] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:
[0185] Obtaining a target detection model to be pruned, where the target detection model to be pruned is trained based on sample images in a sample image set;
[0186] Input the sample image into the target detection model to be pruned, and obtain the output feature map of each layer of the neural network corresponding to the sample image;
[0187] Determine the rank of the output feature map, and classify each layer of the neural network in the target detection model to be pruned according to the rank of the output feature map to obtain a classification result;
[0188] According to the classification results, determine the pruning rate corresponding to each layer of the neural network;
[0189] According to the pruning rate, the network parameters of each layer of the neural network are pruned to obtain a trained target detection model.
[0190] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining an initial target detection model and a set of sample images with labeled boxes; inputting sample images in the sample image set into the initial target detection model to obtain a prediction box corresponding to the sample image; obtaining a model loss function corresponding to the initial target detection model based on the labeled box and the prediction box, and obtaining a network parameter matrix of the initial target detection model; performing sparse training on the initial target detection model based on the model loss function and the network parameter matrix to obtain a target detection model to be pruned.
[0191] In one embodiment, when the processor executes the computer program, the following steps are also implemented: according to the model loss function and the network parameter matrix, the update gradient of the network parameters in the initial target detection model is calculated; according to the update gradient, the network parameters in the initial target detection model are adjusted; and the step of returning to the sample image in the sample image set, inputting the sample image into the initial target detection model, and obtaining the prediction box corresponding to the sample image, until the latest model loss function meets the preset stop iteration condition, and the target detection model to be pruned is obtained.
[0192] In one embodiment, when the processor executes the computer program, the following steps are also implemented: according to the rank of the output feature map, the rank corresponding to each layer of the neural network is determined; the rank corresponding to each layer of the neural network is sorted, and according to the preset number of categories and the sorting result, each layer of the neural network in the target detection model to be pruned is classified to obtain a classification result.
[0193] In one embodiment, when the processor executes the computer program, the following steps are also implemented: sorting the network parameters of each layer of the neural network, and determining the network parameters to be reset for each layer of the neural network based on the sorting results and the pruning rate; resetting the network parameters to be reset to obtain a trained target detection model.
[0194] In one embodiment, when the processor executes the computer program, the following steps are also implemented: resetting the network parameters to be reset to obtain a first target detection model; using sample images in the sample image set to train the first target detection model to obtain a second target detection model; quantizing the accuracy of the network parameters of each layer of the neural network of the second target detection model to obtain a third target detection model; calling the TensorRT tool to optimize the third target detection model to obtain a trained target detection model.
[0195] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:
[0196] Obtain an image to be detected and a trained target detection model, where the trained target detection model is obtained by the target detection model pruning method described above;
[0197] Input the image to be detected into the trained target detection model to obtain the target detection result.
[0198] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:
[0199] Obtaining a target detection model to be pruned, where the target detection model to be pruned is trained based on sample images in a sample image set;
[0200] Input the sample image into the target detection model to be pruned, and obtain the output feature map of each layer of the neural network corresponding to the sample image;
[0201] Determine the rank of the output feature map, and classify each layer of the neural network in the target detection model to be pruned according to the rank of the output feature map to obtain a classification result;
[0202] According to the classification results, determine the pruning rate corresponding to each layer of the neural network;
[0203] According to the pruning rate, the network parameters of each layer of the neural network are pruned to obtain a trained target detection model.
[0204] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: obtaining an initial target detection model and a set of sample images with labeled boxes; inputting sample images in the sample image set into the initial target detection model to obtain a prediction box corresponding to the sample image; obtaining a model loss function corresponding to the initial target detection model based on the labeled box and the prediction box, and obtaining a network parameter matrix of the initial target detection model; performing sparse training on the initial target detection model based on the model loss function and the network parameter matrix to obtain a target detection model to be pruned.
[0205] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: based on the model loss function and the network parameter matrix, the update gradient of the network parameters in the initial target detection model is calculated; based on the update gradient, the network parameters in the initial target detection model are adjusted; and the step of returning to the sample image in the sample image set, inputting the sample image into the initial target detection model, and obtaining the prediction box corresponding to the sample image, until the latest model loss function meets the preset stop iteration condition, and the target detection model to be pruned is obtained.
[0206] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: according to the rank of the output feature map, the rank corresponding to each layer of the neural network is determined; the rank corresponding to each layer of the neural network is sorted, and according to the preset number of categories and the sorting result, each layer of the neural network in the target detection model to be pruned is classified to obtain a classification result.
[0207] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: sorting the network parameters of each layer of the neural network, and determining the network parameters to be reset for each layer of the neural network based on the sorting results and the pruning rate; resetting the network parameters to be reset to obtain a trained target detection model.
[0208] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: resetting the network parameters to be reset to obtain a first target detection model; using sample images in a sample image set to train the first target detection model to obtain a second target detection model; quantizing the accuracy of the network parameters of each layer of the neural network of the second target detection model to obtain a third target detection model; calling the TensorRT tool to optimize the third target detection model to obtain a trained target detection model.
[0209] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:
[0210] Obtain an image to be detected and a trained target detection model, where the trained target detection model is obtained by the target detection model pruning method described above;
[0211] Input the image to be detected into the trained target detection model to obtain the target detection result.
[0212] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:
[0213] Obtaining a target detection model to be pruned, where the target detection model to be pruned is trained based on sample images in a sample image set;
[0214] Input the sample image into the target detection model to be pruned, and obtain the output feature map of each layer of the neural network corresponding to the sample image;
[0215] Determine the rank of the output feature map, and classify each layer of the neural network in the target detection model to be pruned according to the rank of the output feature map to obtain a classification result;
[0216] According to the classification results, determine the pruning rate corresponding to each layer of the neural network;
[0217] According to the pruning rate, the network parameters of each layer of the neural network are pruned to obtain a trained target detection model.
[0218] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: obtaining an initial target detection model and a set of sample images with labeled boxes; inputting sample images in the sample image set into the initial target detection model to obtain a prediction box corresponding to the sample image; obtaining a model loss function corresponding to the initial target detection model based on the labeled box and the prediction box, and obtaining a network parameter matrix of the initial target detection model; performing sparse training on the initial target detection model based on the model loss function and the network parameter matrix to obtain a target detection model to be pruned.
[0219] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: based on the model loss function and the network parameter matrix, the update gradient of the network parameters in the initial target detection model is calculated; based on the update gradient, the network parameters in the initial target detection model are adjusted; and the step of returning to the sample image in the sample image set, inputting the sample image into the initial target detection model, and obtaining the prediction box corresponding to the sample image, until the latest model loss function meets the preset stop iteration condition, and the target detection model to be pruned is obtained.
[0220] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: according to the rank of the output feature map, the rank corresponding to each layer of the neural network is determined; the rank corresponding to each layer of the neural network is sorted, and according to the preset number of categories and the sorting result, each layer of the neural network in the target detection model to be pruned is classified to obtain a classification result.
[0221] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: sorting the network parameters of each layer of the neural network, and determining the network parameters to be reset for each layer of the neural network based on the sorting results and the pruning rate; resetting the network parameters to be reset to obtain a trained target detection model.
[0222] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: resetting the network parameters to be reset to obtain a first target detection model; using sample images in a sample image set to train the first target detection model to obtain a second target detection model; quantizing the accuracy of the network parameters of each layer of the neural network of the second target detection model to obtain a third target detection model; calling the TensorRT tool to optimize the third target detection model to obtain a trained target detection model.
[0223] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:
[0224] Obtain an image to be detected and a trained target detection model, where the trained target detection model is obtained by the target detection model pruning method described above;
[0225] Input the image to be detected into the trained target detection model to obtain the target detection result.
[0226] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.
[0227] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0228] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A target detection model pruning method, characterized in that: The method comprises: Obtaining a to-be-pruned target detection model, wherein the to-be-pruned target detection model is trained based on sample images in a sample image set; Inputting the sample image into the to-be-pruned object detection model to obtain an output feature map of each layer of the neural network corresponding to the sample image; Determine the rank of the output feature map of each layer of the neural network corresponding to the sample image, calculate the average rank of the output feature map of the sample image in the same layer of the neural network for the sample image in the sample image set, and for each layer of the neural network, use the average rank of the output feature map of the sample image in the targeted layer of the neural network as the rank corresponding to the targeted layer of the neural network; Sorting the ranks corresponding to each layer of the neural network, and classifying each layer of the neural network in the target detection model to be pruned according to the preset number of categories and the sorting result to obtain a classification result; Determine the pruning rate corresponding to each layer of the neural network according to the classification result; According to the pruning rate, the network parameters of each layer of the neural network are updated to obtain a trained target detection model.
2. The method according to claim 1, characterized in that: The obtaining of the target detection model to be pruned comprises: Obtain an initial target detection model and a set of sample images with annotation boxes; Inputting a sample image from the sample image set into the initial target detection model to obtain a prediction box corresponding to the sample image; According to the marked box and the predicted box, a model loss function corresponding to the initial target detection model is obtained, and a network parameter matrix of the initial target detection model is obtained; According to the model loss function and the network parameter matrix, the initial target detection model is sparsely trained to obtain a target detection model to be pruned.
3. The method according to claim 2, characterized in that The performing sparse training on the initial target detection model according to the model loss function and the network parameter matrix to obtain the target detection model to be pruned comprises: Calculating the update gradient of the network parameters in the initial target detection model according to the model loss function and the network parameter matrix; According to the updated gradient, adjusting the network parameters in the initial target detection model; Return to the step of inputting the sample images in the sample image set into the initial target detection model to obtain a prediction box corresponding to the sample image, until the latest model loss function meets the preset stop iteration condition, and obtain the target detection model to be pruned.
4. The method according to claim 1, characterized in that: The updating of the network parameters of each layer of the neural network according to the pruning rate to obtain the trained target detection model comprises: Sorting the network parameters of each layer of the neural network, and determining the network parameters to be reset for each layer of the neural network according to the sorting result and the pruning rate; Reset the network parameters to be reset to obtain a trained target detection model.
5. A target detection method, characterized in that: The method comprises: Acquire an image to be detected and a trained target detection model, wherein the trained target detection model is obtained by the method described in any one of claims 1 to 4 above; The image to be detected is input into the trained target detection model to obtain a target detection result.
6. A target detection model pruning device, characterized in that: The device comprises: A model acquisition module, used to acquire a target detection model to be pruned, wherein the target detection model to be pruned is trained based on sample images in a sample image set; A first processing module, used for inputting the sample image into the to-be-pruned object detection model to obtain an output feature map of each layer of the neural network corresponding to the sample image; A classification module is used to determine the rank of the output feature map of each layer of the neural network corresponding to the sample image, and for the sample images in the sample image set, calculate the average rank of the output feature map of the sample image in the same layer of the neural network, and for each layer of the neural network, use the average rank of the output feature map of the sample image in the targeted layer of the neural network as the rank corresponding to the targeted layer of the neural network, sort the ranks corresponding to each layer of the neural network, and classify each layer of the neural network in the target detection model to be pruned according to the preset number of categories and the sorting result to obtain a classification result; A second processing module is used to determine the pruning rate corresponding to each layer of the neural network according to the classification result; The updating module is used to update the network parameters of each layer of the neural network according to the pruning rate to obtain a trained target detection model.
7. The device according to claim 6, characterized in that The model acquisition module is also used to acquire an initial target detection model and a set of sample images with annotated boxes, input a sample image in the sample image set into the initial target detection model, obtain a prediction box corresponding to the sample image, obtain a model loss function corresponding to the initial target detection model based on the annotated box and the prediction box, and obtain a network parameter matrix of the initial target detection model, perform sparse training on the initial target detection model based on the model loss function and the network parameter matrix, and obtain a target detection model to be pruned.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.
